Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets (2024)

Abstract: Language models, characterized by their black-box nature, often hallucinate and display sensitivity to input perturbations, causing concerns about trust. To enhance trust, it is imperative to gain a comprehensive understanding of the model’s failure modes and develop effective strategies to improve their performance. In this study, we introduce a methodology designed to examine how input perturbations affect language models across various scales, including pre-trained models and large language models (LLMs). Utilizing fine-tuning, we enhance the model’s robustness to input perturbations. Additionally, we investigate whether exposure to one perturbation enhances or diminishes the model’s performance with respect to other perturbations. To address robustness against multiple perturbations, we present three distinct fine-tuning strategies. Furthermore, we broaden the scope of our methodology to encompass large language models (LLMs) by leveraging a chain of thought (CoT) prompting approach augmented with exemplars. We employ the Tabular-NLI task to showcase how our proposed strategies adeptly train a robust model, enabling it to address diverse perturbations while maintaining accuracy on the original dataset.

## introduction language models (lms), which have become increasingly integrated into various aspects of daily lives, hold immense potential to revolutionize how we interact with technology. their ubiquity underscores the importance of thoroughly examining their robustness and generalizability, which will be instrumental in fostering trust among users. one notable challenge is their sensitivity to even slight changes in input. for instance, while a human can easily interpret and understand a statement regardless of minor alterations, lms struggle (wang figure 1 : an example of tabular premise and hypotheses from infotabs @xcite . original hypotheses (h 1 ,h 2 ,h 3 ,h 4 ,h 5 ) and perturbed hypothesis @xcite . this inconsistency becomes notably apparent when minor perturbations to the input, which do not inherently modify the underlying meaning, result in a marked decline in the performance of the model @xcite . examples of such perturbations for the task of tabular inference @xcite , is illustrated in figure 1 . addressing these sensitivities to input perturbation is crucial for the advancement and reliability of lms in real-world applications. empirical evidence supports the effectiveness of finetuning models using perturbed input samples from challenge sets @xcite . for instance, @xcite ; @xcite showcased that a pre-trained language model (plm) utilizing masked language modeling (mlm) and trained for a specific nlp task becomes significantly robust to input perturbations when further fine-tuned using a small set of perturbed examples. however, the ability of these models to generalize across different types of perturbations is still a subject of investigation @xcite . the implications of fine-tuning a model on a particular challenge/perturbation set, especially concerning its impact on handling other perturbations, warrant further exploration (refer to figure 2 ). it remains unclear if a model's increased robustness to character perturbations postfine-tuning extends to addressing challenges from other perturbations, like paraphrasing. in this study, we address lms robustness to input perturbations, seeking to answer the following two questions: how does fine-tuning a model on one perturbation set affect performance on other types of perturbations? is it possible to guarantee consistent robustness across multiple distinct perturbation sets? in particular, we extend the singleset inoculation approach of @xcite , to a more generic multi-sets robustness, which we refer to as multi-set inoculation. to the best of our knowledge, we are the first to introduce and extensively study the robustness of lms to multiple perturbations. our proposed methodology is adept at handling both (a) transformer-based pre-trained language models (plms) such as bert @xcite and roberta @xcite , which are amenable to direct fine-tuning on end-user gpus, and (b) large generative language models such as gpt-3.5-turbo (gpt-3.5) (brown et al., 2020), gpt-4, and llama, llama-2 @xcite , flan-t5 @xcite , which are costly and have limited access to re-training (and model weights). for these generative models, we leverage the fewshot chain of thought @xcite as an alternative to traditional fine-tuning. this methodology circumvents the computational intricacies inherent in the fine-tuning of llms. it proficiently manages the tuning of a multitude of model parameters using a limited constrained set of training samples. additionally, we also study inoculation with llm, prior studies @xcite ; @xcite ; @xcite have been limited to traditional bert style models. within our framework, we investigate three distinct multi-set fine-tuning methods for plms and adapt them for llms via cot, each designed to assess and enhance model robustness across diverse perturbation sets. our study makes the following contributions: code and dataset for the experiments with multiset inoculation framework on different models are available at: https://msin-infotabs.github.io/ . ## proposed methodology in this section, we detail the methodology for multiset inoculation. we evaluate the robustness of the model by subjecting it to different input perturbations. subsequently, we introduce multiset fine-tuning techniques, which improve the model's performance on diverse perturbed datasets. figure 3 shows a high-level flowchart of our methodology. terminology. given a pre-trained language model (plm) denoted by m, fine-tuned on the original (unperturbed) training set o for a natural language processing (nlp) task t. perturbation π j is applied only to o s j , producing the perturbation/challenge set π s j j : this results in m perturbation sets {π s j j } m @xmath0. we use p j as shorthand for the final perturbation set π s j j . we evaluate the performance of model m on heldout perturbation set samples q j for @xmath1, . . . , m. each q j serves as the test set specifically tailored for perturbation π j . ## case study on tabular inference original dataset (o). we utilize the tabular-nli dataset, infotabs @xcite , along with its adversarial perturbations, as detailed in @xcite . the infotabs dataset features a wide range of table domains, categories, and keys, covering various entity types and forms. it includes three test splits: α 1 (original test set), α 2 (adversarial set), and α 3 (zero-shot or out-of-domain set). perturbed challenge datasets (p, q). our dataset incorporates perturbations from shankarampeta et al., 2022, enhanced using tools such as textattack @xcite and nlp checklist @xcite , alongside manual adjustments. each perturbation specifically targets the hypothesis of an input sample. for every perturbation type, we create challenge sets of up to 1,500 samples. only those samples that are pertinent post-perturbation are selected. when the number of such samples exceeds 1500, we narrow down to the most diverse 1500 samples using fixed-size determinantal point processes (k -dpps) @xcite . perturbations used for tabular-nli tasks are character-level perturbation (char, c), negation-type perturbation (neg, n), numeric perturbation (num, m), location perturbation (loc, l) and paraphrasing perturbation (stan, s) (refer figure 1 ). train/test. (a.) bert based models (plm) : for any perturbation type, we represent q j consisting of 1000 examples for testing and p j consisting of 500 examples for fine-tuning. we define the union of all challenge test sets as @xmath2. (b.) large language models (llm) : as llms inference is costly we limit our evaluations to 300 random samples from q j , where q j contains original premise and perturbed hypothesis using perturbation π j . q' j contains the original premise along with the corresponding unperturbed hypothesis as pairs. we evaluate performance on both q' j and q j to access if the llm model forgets the original input distribution after fine-tuning on perturbation sets. table representation. in line with neeraja et al., 2021, we employed alignment techniques @xcite to eliminate distracting rows (drr). we selected the top-8 rows for table representation as a premise (drr@8), enhancing accuracy through evidence-based grounding of relevant information for hypothesis labeling. evaluation metric. we use accuracy which is equivalent to the micro-f1 score for the nli task where the label for each example can be only one of entailment e, contradiction c, neutral n. the improvement over the multi-challenge sets is considered by taking the average of the improved performance over each challenge set q j and this is used as the score(µ) for multi-perturbation setting. implementation and hyperparameter details for all experiments are mentioned in appendix a.3. ## results and analysis our experiments answer the following questions:- ## related works model robustness issues. deep learning models in vision and language domains have exhibited sensitivity to adversarial examples and input distribution shifts, as highlighted in prior studies @xcite @xcite @xcite @xcite @xcite . the search for model robustness in language processing has led to work on contrast sets @xcite , @xcite , and attack algorithms @xcite . ensuring model robustness is crucial @xcite , as minor input changes can significantly impact performance due to model complexity and distribution overfitting @xcite @xcite . recently, @xcite introduce adversarial prompts to analyse model robustness to perturbation in prompts. our work focuses on analyzing model performance with clean prompts across several perturbations/attacks on input samples simultaneously. improving model robustness. utilizing adversarial examples during training provides a degree of mitigation to input sensitivity of a deep learning model @xcite @xcite @xcite , however, it falls short of a comprehensive solution for achieving widespread robustness, as it deals only with one facet, i.e., single-set inoculation. our proposed framework is adept at evaluating model robustness across multiple challenge sets. our research complements and extends the work on robustness explored in @xcite @xcite . @xcite integrates consistency loss and data augmentation during training, our framework applies to models already in use or deployed. similarly, lu, 2022 addresses dataset artifacts in natural language inference (nli) with a multi-scale data augmentation method. in contrast, our work focuses on limited fine-tuning of pre-trained models and expands to additional dimensions of robustness. meanwhile, zheng and saparov, 2023 examines llm robustness to perturbed inputs by increasing noisy exemplars. our study offers a broader framework for assessing the robustness of both plms and llms, using fine-tuning, improving instruction quality, and enhancing exemplars in both diversity and quantity. ## conclusion and future works we demonstrate that input perturbation poses difficulties for lms at all scales. while fine-tuned models on a single challenge set can produce robust models, their generalizability to unfamiliar perturbations remains questionable. this motivates the problem of multi-set inoculation, aiming to train a singular model resilient to a myriad of distinct perturbations. we introduce a comprehensive framework to systematically evaluate lm robustness against multiple input perturbations. additionally, we propose three strategies to fine-tune the model on multiple challenge/pertubations sets. our results underscore the superiority of mixed finetuning in training robust models. furthermore, we expand our framework to llms, leveraging a cot prompting enriched with exemplar demonstrations.