S ynth E val: Hybrid Behavioral Testing of NLP Models with Synthetic C heck L ists (2024)

Abstract: Traditional benchmarking in NLP typically involves using static, held-out test sets and calculating aggregated statistics based on diverse examples. However, this approach often results in an overestimation of performance and lacks the ability to offer comprehensive, interpretable, and dynamic assessments of NLP models. Recently, works like DynaBench and Checklist have addressed these limitations through behavioral testing of NLP models with test types generated by a multi-step human-annotated pipeline. Unfortunately, manually creating a variety of test types requires significant human labor, thus weakening efficiency. In this work, we propose SynthEval, a hybrid behavioral testing framework that leverages large language models (LLMs) to generate a wide range of test types for a comprehensive evaluation of NLP models. The SynthEval framework first generates sentences via LLMs using controlled generation, and then identifies challenging examples by comparing the predictions made by LLMs with task-specific NLP models. In the last stage, human experts investigate the challenging examples, manually design templates, and identify the types of failures the task-specific models consistently exhibit. We apply SynthEval to two classification tasks and show that our framework is effective in identifying weaknesses of strong models on these tasks.

## introduction the typical pipeline before deploying nlp models for practical use involves training, validating, and testing phases. a model that performs well on a held-out test set, as measured by a single aggregate statistic, e.g., accuracy, is expected to be capable traditional benchmarking roberta large sst-2 val. set large-scale generation with llms and behavioral testing fine-tuned on 15 sentiment analysis datasets 92.4% accuracy syntheval template gold label fine-tuned roberta large accuracy i thought this {booknoun} was {awfulneg adj}. { i was wrongrevision}. positive 😀 10.17% figure 1 : using a held-out val. set for evaluation overestimates the performance. roberta large , fine-tuned on 15 diverse sentiment analysis datasets @xcite , performs strongly on traditional benchmarks (92.4%). however, syntheval, which generates behavioral tests with the help of llms, demonstrates roberta large 's bad performance (10.17%) when tested on a sentence containing a simple revision: "i was wrong". to assess the true capabilities of a taskmodel, especially uncovering its vulnerabilities, many works go beyond simply evaluating against a single aggregate statistic. these approaches evaluate multiple aspects of a model such as robustness, consistency and error types @xcite @xcite . though providing ways to evaluate a model's competence in different facets, these methods fail to provide comprehensive guidance on how to evaluate the model. checklist, a method that breaks down capability failures into specific failures, is proposed to fill the gap @xcite . checklist leverages a strategy called behavioral testing or black-box testing, originating from software engineering, which tests the application by providing inputs and then examining the outputs without knowing what the software does internally to arrive at those outputs @xcite . although recent works like checklist @xcite and dynabench @xcite introduce a wide range of test types by proposing multi-step human-annotated evaluation and template-based analysis of nlp models, these types are manually extracted and summarized. this involves substantial human labor, which is not only tedious but can also miss model vulnerabilities. due to recent advancement of llms, they now have a human-comparable ability to generate many types of high-quality data @xcite @xcite @xcite . because of this capability, llms have also been used to help identify and even fix the possible weaknesses of smaller models, either through llms suggesting the test types (ribeiro and lundberg, 2022) or through llms generating instances for test types @xcite . these methods either heavily require human effort in the pipeline, or ask for the test types to be extracted before llms can generate instances for them. inspired by this line of work, we pose two research questions: (1) how can we directly generate a large number of test types using llms? and (2) how can we reduce the burden of annotators when identifying challenging test types? to this end, we propose syntheval, a novel hybrid behavioral testing methodology based on llms. syntheval leverages an llm to generate diverse examples for a given task (in our case, classification). the generated examples are fed to both a taskmodel (the model that we perform behavioral testing on) and the reference model (the same llm used to generate the examples) for predictions. then human experts need only extract and summarize the examples on which the prediction of taskmodel and reference model diverge. finally, we can generate a lot of test types automatically by applying the templates created by human experts. the procedure greatly reduces human labor and enables us to generate diverse examples. the contributions of this work are as follows: (i) we propose syntheval, a framework that partially automates the process of generating diverse and challenging test types for evaluating nlp models. (ii) we validate the reliability of synthe-val on two classification tasks: sentiment analysis and toxic language detection. (iii) we conduct a comprehensive, linguistically informed analysis to identify patterns in sentences where classification models face challenges. limitations in handling linguistic complexity while classification models, including those used for tasks like sentiment analysis, have demonstrated the ability to outperform humans on traditional datasets (dang et al., 2020), they face limitations with more complex linguistic structures. recent studies indicate that language models, despite their advancements, often struggle with complex syntax and nuanced expressions. @xcite reveal that neural language models like bert still struggle with complex syntactic sentences easily handled by humans. experiments conducted by @xcite demonstrate that pretrained language models often fail to correctly process negation in sentences. @xcite also indicate that language models like bert, gpt-2, and roberta rely on semantic cues for syntactic predictions, highlighting a limitation in their syntactic understanding. interpretable behavioral testing @xcite propose checklist, a suite of tests for evaluating model robustness across various linguistic phenomena. this framework helps pinpoint fundamental linguistic shortcomings in models that perform well on standard benchmarks. however, checklist is challenging in practice: it is costly and requires experienced annotators who are competent to conduct qualitative assessments of templates @xcite . @xcite propose testaug, which utilizes gpt-3 to generate more test cases based on checklist's existed templates, but no new patterns are detected. @xcite also attempt to use llm to try to reduce the human burden of testing the performance of machine translation systems. other methodologies like hatecheck @xcite , red teaming @xcite , and targeted data generation @xcite complement check-list by offering dynamic ways to test and improve model evaluation. these approaches underscore the need for ongoing, refined assessments to build more robust nlp systems. recent developments in these areas focus on scalability, enhancing template quality, and ensuring relevance across various languages and cultures. adaptive testing (ribeiro and lundberg, 2022) highlights the benefits of integrating human insights with llms in testing frame-works, pointing to a more collaborative approach in advancing model reliability and fairness. prior methods require substantial human intervention for creating templates and identifying confusing sentences without any references, which can be labor-intensive and may overlook issues. ## methodology: syntheval we propose syntheval, a hybrid and dynamic evaluation framework to reveal behavioral failures of task-specific nlp models with the help of large language models and human annotation. we refer to task-specific nlp models as taskmodels; taskmodels are pretrained language models finetuned to perform a specific task such as sentiment analysis classification. these models are widely used in industry, even in the era of large language models, since they achieve good performance on test sets and are cheaper to deploy. in figure 2 , we illustrate syntheval, which consists of three steps: (1) diverse synthetic test set generation (synthtest), (2) identification of a challenging subset of the test set (synthtest hard ), and (3) manual formalization and verification of behavioral patterns from synthtest hard . ## analysis we apply syntheval on two diverse tasks: sentiment analysis and text toxicity detection. we selected them since they are widely studied both in academia and industry. furthermore, current taskmodels for sentiment analysis can achieve very high performance on traditional benchmarks which presents an interesting challenge for syntheval to uncover potential weaknesses. of the two, toxicity detection is the more challenging task since it requires recognizing harmful content that is often implied and context-sensitive. toxicity detection failure of current models can marginalize minority groups @xcite . thus, more effective behavioral testing has great potential benefits for this task. in our experiments, we use llama2 7b @xcite as the main llm both for test set generation and for creating a challenging subset with few-shot in-context learning. ## conclusion we presented syntheval, a semi-automatic framework that utilizes the generative and generalization capabilities of llms to decrease human workload and provide novel insights into behav-ioral testing for identifying vulnerabilities in classification models. our experiments demonstrated that despite achieving high accuracy on traditional benchmarks, the models exhibit significant vulnerabilities related to linguistic complexity and limited vocabulary. syntheval has proven effective across two diverse classification tasks, offering a valuable tool for model evaluation with reduced human effort.