Measure and Improve Robustness in NLP Models: A Survey (2022)

Abstract: As NLP models achieved state-of-the-art performances over benchmarks and gained wide applications, it has been increasingly important to ensure the safe deployment of these models in the real world, e.g., making sure the models are robust against unseen or challenging scenarios. Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research. In this paper, we aim to provide a unifying survey of how to define, measure and improve robustness in NLP. We first connect multiple definitions of robustness, then unify various lines of work on identifying robustness failures and evaluating models’ robustness. Correspondingly, we present mitigation strategies that are data-driven, model-driven, and inductive-prior-based, with a more systematic view of how to effectively improve robustness in NLP models. Finally, we conclude by outlining open challenges and future directions to motivate further research in this area.

## introduction nlp models, especially with the recent advances of large pre-trained language models have achieved great progress and gained wide applications in the real world. despite the performance gains, nlp models are still fragile and brittle to out-of-domain data @xcite , adversarial attacks @xcite @xcite , or small perturbation to the input @xcite . those failures could hinder the safe deployment of these models in the real world, and impact nlp models' trustworthiness to users. as a result, an increasing line of work has been conducted to understand robustness issues in the language technologies communities. still, diverse sets of research across multiple dimensions and numerous levels of depth exist and are scattered across various communities; for instance, using a variety of definitions on a wide range of very different nlp tasks. in this work, we provide a unifying overview of what is robustness in nlp, how to identify robustness failures and evaluate model's robustness, and systematic ways to improve robustness, as well as a conceptual schema categorizing ongoing research directions. we identify gaps between the to-date robustness work, the technical opportunities, and discuss possible paths forward. the above definition works for a range of nlp tasks like text classification and sequence labeling where y is defined over a fixed set of discrete labels. for tasks like text generation, robustness is less well defined and can manifest as positional bias @xcite , or hallucination @xcite @xcite . one major challenge here is a lack of robust metrics in evaluating the quality of the generated text @xcite , i.e., we need a reliable metric to determine the relationship between f (x ) and y when both are open-ended texts. ## robustness in vision vs. in nlp despite the widely study of robustness in vision, the study of robustness in nlp cannot always directly borrow the ideas. we categorize the main differences with the three following points: ## identify robustness failures as robustness gained increasing attention in nlp literature, various lines of work have proposed ways to identify robustness failures in nlp models. existing works can be roughly categorized by how the failures are identified, among which a large portion of work relies on human priors and error analyses over existing nlp models (section 4.1), and other lines of work adopt model-based approaches (section 4.2). the identified robustness failure patterns are usually organized into challenging/adversarial benchmark datasets to more accurately measure an nlp model's robustness. in ## improve model robustness correspondingly, there are multiple lines of directions that try to improve robustness in nlp models. depending on where and how the intervention is applied, those approaches can be categorized into the following categories: data-driven (section 5.1), model-based and training-scheme-based (section 5.2), inductive-prior-based (section 5.3) and finally causal intervention (section 5.4). ## open questions in addition to the challenges mentioned above, we list below a few open questions that call for additional research going forward. identifying unknown robustness failures existing identification around robustness failures rely heavily on human priors and error analyses, which usually pre-define a small or limited set of patterns that the model could be vulnerable to. this requires extensive amount of expertise and efforts, and might still suffer from human or subjective biases in the end. how to proactively discover and identify models' unrobust regions automatically and comprehensively remains challenging. interpreting and mitigating spurious correlations interpretability matters for large nlp models, especially key to the robustness and spurious patterns. how can we develop ways to attribute or interpret these vulnerable portions of nlp models and communicate these robustness failures with designers, practitioners, and users? in addition, recent work @xcite @xcite show interpretability methods can be utilized to better understand how a model makes its decision, which in turn can be used to uncover models' bias, diagnose errors, and discover spurious correlations. furthermore, the mitigation of spurious correlations often suffers from the trade-off between removing shortcuts and sacrificing model performance @xcite . additionally, most existing mitigation strategies work in a pipeline fashion where defining and detecting spurious correlations are prerequisites, which might lead to error cascades in this process. how to design end-to-end frameworks for automatic mitigation deserves much attention. ## conclusion in this paper, we provided a unifying overview over robustness definitions, evaluations and mitigation strategies in the nlp domain. we also highlighted open challenges in this area to motivate future research, encouraging people to think deeply about more comprehensive benchmarks, transferability and validity of adversarial examples, unified framework to evaluate and improve robustness, user-centered measures and mitigation, and finally how to potentially achieve human-like linguistic generalization more meaningfully.