Achilles-Bench: A Challenging Benchmark for Low-Resource Evaluation (2024)

Abstract: With promising yet saturated results in high-resource settings, low-resource datasets have gradually become crucial benchmarks (e.g., BigBench Hard, superGLUE) for evaluating the learning ability of advanced neural networks. In this work, we find that there exists a set of “hard examples” in low-resource settings that challenge neural networks but are not well evaluated, which causes over-estimated performance. We first give a theoretical analysis on which factors bring the difficulty of low-resource learning. It then motivates us to propose a challenging benchmark Achilles-Bench to better evaluate the learning ability, which covers 11 datasets, including 8 natural language process (NLP) datasets and 3 computer vision (CV) datasets. Experiments on a wide range of models show that neural networks, even pre-trained language models, have sharp performance drops on our benchmark, demonstrating the effectiveness of evaluating the weaknesses of neural networks. On NLP tasks, we surprisingly find that despite better results on traditional low-resource benchmarks, pre-trained networks, does not show performance improvements on our benchmarks. there is still a large robustness gap between existing models and human-level performance, highlighting the need for robust low-resource learning models.

## introduction large-scale models have shown strong capabilities in learning from a handful of examples @xcite @xcite , resulting in an increased demand for low-resource benchmarks. numerous research studies have highlighted the rapid adaptability of such models to new tasks, utilizing techniques like in-context learning @xcite . consequently, the evaluation of large-scale pre-trained models has shifted towards assessing their ability to quickly learn new downstream tasks with limited available samples, including superglue @xcite and big-bench hard @xcite . however, many low-resource datasets usually use random or manual selection methods to sample data from the cleaned and balanced training data. they struggle to capture the data biases and increased difficulty commonly encountered in realworld scenarios. consequently, these benchmarks fall short in evaluating the true learning gap between existing models and human-level models. while some models can surpass human performance on these benchmarks (e.g., sst-2) @xcite @xcite , many studies have revealed that these robust models still face challenges such as spurious correlation @xcite or bias @xcite , which are relatively uncommon in human learning. as depicted in figure 1 , models on a randomly sampled lowresource set demonstrate performance comparable 1 2057 to human-level in sentiment analysis. however, their performance significantly deviates from human level when confronted with challenging examples. it motivates us to propose a challenging low-resource benchmark. in this work, we aim to find challenging examples given any tasks. this approach differs significantly from existing challenging benchmarks, which are either focused on complex tasks such as big-bench-hard @xcite or specific to extremely few-shot settings like fewglue. in contrast to these previous studies @xcite , our proposed benchmark aims to generate difficult examples for any given task foot_0 . in addition, realworld low-resource data samples often exhibit biases towards specific domains, such as blank backgrounds in image detection or short sentences in handwritten hate speech. therefore, our evaluation also includes a bias assessment. specifically, we consider two dimensions: misleading examples with smaller classification margins for performance evaluation, and biased examples for robust evaluation. we begin by conducting a comprehensive analysis of how these two dimensions impact lowresource learning. based on the insights derived from our analysis, we present an empirical solution to construct a challenging low-resource benchmark. the final benchmark encompasses 3 computer vision datasets and 8 natural language processing datasets. to prove the effectiveness of the constructed benchmark, we evaluate 13 models, including 8 pre-trained models, such as t5 @xcite , llama @xcite , etc. all these models struggle to handle our benchmarks, with a large performance gap compared with randomlysampled low-resource benchmarks. on nlp tasks, we surprisingly find that despite better results on traditional low-resource benchmarks, pre-trained networks, do not show performance improvements on our benchmarks. the contribution of this paper is summarized as: 1) we propose achilles-bench, a challenging benchmark designed to expose achilles' heel (weaknesses) of neural networks. this benchmark provides a reflective view of the current progress in the field of low-resource learning. 2) we conduct a comprehensive analysis to identify the factors that particularly exacerbate the difficulty of low-resource learning. 3) experimental results demonstrate that our proposed benchmark effectively challenges existing models, including robust pre-trained networks and large language models. ## related work low-resource evaluation learning on lowresource datasets has recently come into the spotlight with the introduction of more powerful models @xcite . recent low-resource benchmarks use a transfer learning setting @xcite as well as in-context learning @xcite , and they have also added up on dataset difficulty @xcite . among these, there are two major types of low-resource benchmark: natural low-resource datasets, and sampled low-resource datasets. the former requires additional dataset curation @xcite @xcite and currently, most lowresource benchmarks are uniformly sampled from larger datasets @xcite @xcite @xcite . challenging benchmark previous approaches in constructing challenging benchmark mainly curate from natural data @xcite @xcite . these methods require heavy annotation and faces misalignment between human-perceived difficulty and samples hard for models. our methods, however, create an annotation-free framework for building challenging training sets, which has the potential to quickly apply to any available task. other work involved benchmarking a more comprehensive and challenging list of tasks @xcite @xcite , which deviates from our focus in finding model weakness on common tasks. ## understanding the difficulty of low-resource learning to better understand the challenges of low resource learning, we first look at the teacher-student setting in learning perceptrons. consider a large cu- intuitively, three dimensions amount to the difficulty of learning perceptron j: (1) the number of training samples p (here we base the study of data scarity on the sample per parameter variable α low ); we elaborate on simulation settings in the appendix. difficult data especially challenges low resource learning. we first compare the setting that increases data difficulty to the random-sampled version of low-resource learning. we vary our dataset size from 1% to 500% trainable parameters. as shown in figure 3 , the dark blue line corresponds to the setting where data is uniformly selected, and lighter lines range in data difficulty from margin 0.1 to 1. the functions of ϵ g to α yield a crossover between the function for randomsampled training data and the one for increased difficulty training data, showing that increased data difficulty affects low resource settings more than sufficient data settings. also, the increase in generalization error is more distinct for slightly larger training sets. as when the low-resource training set only has a few samples, it requires model to have strong generalization ability to beat the rule of generalization ϵ ∝ α -1 and the task is challenging enough. low resource learning is more sensitive to spurious correlations. in the biased learning scenario as shown in figure 4 , we compare students trained on biased datasets (red lines) to students trained on random-sampled datasets (blue lines). when the bias probe is more distinct from the teacher (larger θ), the drop in performance is more distinct. this is in line with the phenomenon that when a model overfits on spurious features that contain information distant from semantics, the model tends to suffer on generalization. also, for smaller bias, low resource learning sees a larger drop in generalization while models with abundant data barely suffer. this show that low-resource learning is sensitive to even small biases. theoretical perspective here we use theoretical analysis in addition to simulations to study the scenario that results in failed generalization in low resource learning. again, we focus on the scenario where we have a large dataset d that represents the natural task distribution p . we sample a low resource dataset d low from d that form the distribution p low . we theoretically show that the generalization error for the model trained on the lowresource dataset is bounded by a function of data difficulty and the distribution bias of low-resource dataset. theorem 3.1. (low-resource generalization measured by distribution shift and data difficulty) let h be the hypothesis space @xmath0. f low is the empirical risk ϵ p low (f ) minimizer, and f is the hypothesis that minimizes expected risk ϵ q (f ), m is the smallest margin of d to decision boundary of f . mmd(p low , p ) describes the maximum mean discrepancy @xcite between the sampled distribution and the original distribution. then with probability over 1-δ, the value of the equation 1 right-hand side increases when m decreases and the term mmd(p low , p ) increases, corresponding to the increase in data difficulty and the presence of data bias. this theorem applies not only to our simu-lated scenario of perceptron learning but also to deeper models. in our biased learning setting, the distribution gap between low resource data distribution is larger for biased training set than randomsampled training set, i.e., mmd(p θ low , p ) > mmd(p random low , p ), since data samples forming p random low are sampled uniformly from p . based on our simulation experiments and theoretical results in the previous section, we find that low-resource learning is more likely to suffer from performance drop due to data difficulty and dataset bias. however, these scenarios are not covered in previous low-resource benchmarks. this motivates us to propose a challenging benchmark achilles-bench for better evaluation. ## achilles-bench challenge we propose a new challenging benchmark that elevates low-resource learning difficulty on some well-known datasets. unlike previous low-resource datasets that are randomly sampled from a training set, we curate the benchmark by selecting one of the most challenging low-resource training sets from glue, cifar10, cifar100, and imagenet. following our theoretical analysis, we introduce the simple yet effective approach to build hard-bench: first, we train a predictor for only one epoch on a large benchmark, obtaining a biased predictor; then, we score each sample on data difficulty for this stage of training. for each label, we pick the top k samples as our selected low-resource training set. we elaborate on the data difficulty metrics and the biased predictor respectively in section 4.1. ## conclusion this paper proposes a challenging benchmark for low-resource learning. we first analyze which factors affect the difficulty of low-resource learning. we prove that low-resource generalization results in worse performance with more difficult and biased datasets. hence we choose two metrics for measuring data difficulty, which result in two variants, achilles-bench (loss) and achilles-bench (gradnorm) . experiments show that both can better tell the learning gap between existing models than randomly-sampled low-resource datasets. here suppose test set q matches the distribution of data for this classification task, and p is constructed by sampling n i.i.d. samples from the distribution q. using lemma c.3 we have,