Targeted Data Generation: Finding and Fixing Model Weaknesses (2023)

Abstract: Even when aggregate accuracy is high, state-of-the-art NLP models often fail systematically on specific subgroups of data, resulting in unfair outcomes and eroding user trust. Additional data collection may not help in addressing these weaknesses, as such challenging subgroups may be unknown to users, and underrepresented in the existing and new data. We propose Targeted Data Generation (TDG), a framework that automatically identifies challenging subgroups, and generates new data for those subgroups using large language models (LLMs) with a human in the loop. TDG estimates the expected benefit and potential harm of data augmentation for each subgroup, and selects the ones most likely to improve within-group performance without hurting overall performance. In our experiments, TDG significantly improves the accuracy on challenging subgroups for state-of-the-art sentiment analysis and natural language inference models, while also improving overall test accuracy.

## introduction despite very high accuracy, state-of-the-art nlp models still exhibit systematic failures on specific subgroups of data. for example, @xcite found that a 95%-accurate sentiment analysis model did much worse on club reviews (90%) and movie theater reviews (85%), while stuart-ulin (2018) notes how a commercial chatbot avoids any engagement on topics that even mention islam or the middle east. the existence of these challenging subgroups can lead to unfair outcomes, erode user trust, and ultimately limit deployment of models, even when aggregate accuracy is very high. one possible solution is to collect or generate more data. however, the additional data may still under-sample from specific challenging subgroups, even if data collection is adversarial @xcite , especially when subgroups are not immediately obvious or salient to humans. therefore it helps little in addressing these weaknesses. tools for discovering challenging subgroups still require human creativity and effort @xcite . @xcite ; @xcite show that experts are able to improve existing subgroups via careful data augmentation with large language models (llms), but finding such challenging subgroups still requires human ingenuity. perhaps more importantly, they find that naively augmenting certain subgroups can drastically hurt other subgroups and overall performance @xcite . hence, the challenge is not only to find challenging subgroups, but also to determine which subgroups are amenable to data augmentation, and how to augment them effectively. in this work, we propose targeted data generation (tdg), a framework to automatically identify challenging subgroups that can benefit from more data, and then generate that data with llms (figure 1 ). given a target model, tdg clusters validation data into potential challenging subgroups. we then use held-out data to estimate how much each subgroup would benefit from more data, and how much additional data would hurt performance in other regions. finally, having identified challenging subgroups amenable to data augmentation, we use @xcite coupled with local subgroup models to generate new data, so as to improve subgroup performance while remaining faithful to the original data distribution. we evaluate tdg on three tasks: sentiment analysis (sst), paraphrase detection (qqp), and natural language inference (mnli). we evaluate various clustering techniques, and find that clustering based on the target model's own representation yields the clusters most amenable to data augmentation (with the exception of qqp, where our analysis in the automatic subgroup discovery stage, tdg identifies challenging clusters that can benefit from additional data while minimizing potential negative impacts on performance in other regions (i.e., high generalization (gc) and low interference (ic), as defined in section 2.1). in the subgroup augmentation with llm stage, tdg utilizes gpt-3 to generate additional examples for identified challenging clusters. ## targeted data generation let m be a target model trained on a training dataset d train , and let d test be a held-out test dataset. we assume access to a validation dataset d val , which we use to identify and evaluate challenging subgroups. we cluster d val into k disjoint clusters, @xmath0, c 2 , . . . , c k }, using some clustering technique (we explore various options in section 2.2, and drop the subscript when talking about a single cluster, for clarity). we divide d val randomly into two halves, so that each cluster is divided into c train and c test ( c val can be further divided from c train if necessary), to simulate the effect of data augmentation and its impact on the same subgroup. we say a cluster c is a challenging cluster if the target model m performs much worse on it than on the overall validation dataset, i.e., acc(m, c train ∪ c val ) << acc(m, d val ). given a challenging cluster c, our goal is to identify whether it is amenable to data augmentation, i.e., more data would generalize and improve performance on c test , without hurting performance on d test . ## related work recent research in machine learning has focused on enhancing the robust performance of models by identifying challenging subgroups and improving their performance. ## conclusion in this work, we presented a thorough analysis of error distribution among different groups and introduced targeted data generation (tdg), a framework that automatically identifies challenging groups that are amenable to improvement through data augmentation using large language models (llms) without negatively impacting overall accuracy. our experiments with state-of-the-art models demonstrate that tdg is able to improve ingroup performance by 2-13% while also increasing overall accuracy. furthermore, tdg was able to improve performance for every single selected cluster without interference, indicating its potential as a reliable approach for a new data collection framework. as llms continue to advance and are trained on more diverse and large corpora, tdg represents a promising approach for addressing the weaknesses of simpler models.