Methods for Estimating and Improving Robustness of Language Models.

Varování

Publikace nespadá pod Pedagogickou fakultu, ale pod Fakultu informatiky. Oficiální stránka publikace je na webu muni.cz.
Autoři

ŠTEFÁNIK Michal

Rok publikování 2022
Druh Článek ve sborníku
Konference Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop
Fakulta / Pracoviště MU

Fakulta informatiky

Citace
www https://aclanthology.org/2022.naacl-srw.6/
Doi http://dx.doi.org/10.18653/v1/2022.naacl-srw.6
Klíčová slova natural language processing; transformers; robustness; generalization
Popis Despite their outstanding performance, large language models (LLMs) suffer notorious flaws related to their preference for shallow textual relations over full semantic complexity of the problem. This proposal investigates a common denominator of this problem in their weak ability to generalise outside of the training domain. We survey diverse research directions providing estimations of model generalisation ability and find that incorporating some of these measures in the training objectives leads to enhanced distributional robustness of neural models. Based on these findings, we present future research directions enhancing the robustness of LLMs.
Související projekty:

Používáte starou verzi internetového prohlížeče. Doporučujeme aktualizovat Váš prohlížeč na nejnovější verzi.