Venue: NAACL 2025

# In-Context Learning (and Unlearning) of Length Biases

Stephanie Schoch Yangfeng Ji

Department of Computer Science

University of Virginia

Charlottesville, VA 22903

{sns2gr,yangfeng}@virginia.edu

###### Abstract

Large language models have demonstrated strong capabilities to learn in-context, where exemplar input-output pairings are appended to the prompt for demonstration. However, existing work has demonstrated the ability of models to learn lexical and label biases in-context, which negatively impacts both performance and robustness of models. The impact of other statistical data biases remains under-explored, which this work aims to address. We specifically investigate the impact of length biases on in-context learning. We demonstrate that models do learn length biases in the context window for their predictions, and further empirically analyze the factors that modulate the level of bias exhibited by the model. In addition, we show that learning length information in-context can be used to counter the length bias that has been encoded in models (e.g., via fine-tuning). This reveals the power of in-context learning in debiasing model prediction behaviors without the need for costly parameter updates.

## 1 Introduction

In-context learning (ICL) has emerged as a new ability in large language models (LLMs), representative of a novel learning paradigm Wei et al. (2022). With in-context learning, an LLM learns to perform an unseen task by seeing a number of demonstrations in the context window Brown et al. (2020). Whereas previous methods such as fine-tuning update the model parameters to teach the model a desired task, ICL provides the model with input-output pairs as task exemplars directly at inference, with no parameter updates. While the goal of increased task accuracy is the same, the underlying mechanisms contributing to in-context learning are still being understood.

This has motivated a body of work aiming to understand how in-context learning works. Some works have demonstrated simlarities between fine-tuning and in-context learning. For example, instability due to the choice of examples occurs both in few-shot finetuning Schick and Schutze (2021); Gao et al. (2021) and in-context learning Rubin et al. (2022); Liu et al. (2022); Wu et al. (2023). However, other work has shown counterintuitive results when comparing the apparent learning mechanisms of in-context learning and finetuning Min et al. (2022).

A key area that is underexplored is whether incontext learning exhibits similar biases to finetuning with regard to statistical data biases. Statistical data biases can be defined as correlations between features and class labels. Under traditional learning paradigms such as fine-tuning, language models can learn exploitable statistical biases in the data. Such biases, or shallow features, can be exploited by a model as discriminatory features when they exhibit biased distributions across classes or are correlated with a specific class. This can lead to overestimates of a model's performance on the underlying task Poliak et al. (2018); Gururangan et al. (2018).

Prior work has identified length as an exploitable statistical bias in natural inference datasets. For example, in the MultiNLI and SNLI datasets, length has been shown to be a discriminatory feature Gururangan et al. (2018), and on the ROC story cloze

Figure 1: An illustration of our experiment setup and hypothesis. When sampling from the tails of the distribution (left of image), we introduce a data length bias. If the model can learn this shortcut feature in-context, we expect class performance on the data of similar length to be higher than data of the opposite length than what was seen in the context window (right of image).



task choosing the longer ending performs above random baseline (Cai et al., 2017). However, length biases have been largely ignored in prior work on ICL, and some existing studies on which factors impact ICL have treated length as a static variable, selecting examples with similar lengths to test inputs (Min et al., 2022). It is therefore unclear whether models can exploit length biases in the data under an in-context learning setting (similar to finetuning) or whether length is another factor with counterintuitive tendencies.

In this work, we perform a series of empirical studies to investigate the ability of LLMs to learn statistical data biases in the context window during ICL (Figure 1). This has been studied in the fine-tuning literature, yet is underexplored in the ICL literature. We demonstrate empirically the ability of LLMs to learn length biases in-context. In the sections to follow, we analyze which factors influence these results, and we show the efficacy of ICL in debiasing finetuned models. Our results show that ICL can introduce biases to LLMs that negatively influence task performance. Specifically, our contributions are as follows:

1. We empirically demonstrate the ability of a range of LLM families to learn length biases in-context.
2. We investigate the influence of number of examples, number of model parameters, and class-length difference on how models learn biases.
3. We show that ICL can debias a model that contains existing length biases.

## 2 Background

In-context learningIn-context learning is an emergent ability of LLMs that enables pre-trained models to learn an unseen task using a set of exemplars concatenated in the context window (see Figure 2). Formally, given a test example \(x\), in-context learning concatenates \(K\) demonstration examples to the task instruction \(I\), where \(S=\{x_{i},y_{i}\}^{K}_{i=1}\) denotes the example set. The performance of innContext learning, however, is highly dependent on both the selected examples (Rubin et al., 2022; Liu et al., 2022; Wu et al., 2023; Ye et al., 2023) and their orderings (Lu et al., 2022; Chen et al., 2023), resulting in performance variation from nearly random to comparable with finetuned models.

In-Context Learning & BiasWhile in-context learning has shown significant potential as a way to extract relevant information from an LLM and align the model with user expectations, it has also exhibited brittleness to an assortment of factors. These include selected examples (Rubin et al., 2022; Liu et al., 2022; Wu et al., 2023; Ye et al., 2023) and their orderings (Lu et al., 2022; Chen et al., 2023), which have recently been categorized under the umbrella of demonstration biases (Li et al., 2024).

Beyond demonstration bias, instability of ICL has been attributed to biases in the model toward predicting certain answers due to majority label bias, recency bias, and common token bias (Zhao et al., 2021). Correspondingly, several works have looked at identifying and mitigating label bias (Zhao et al., 2021; Fei et al., 2023) (Fei et al., 2023) with respect to lexical information, and Ali et al. (2024) have looked at mitigating "copy bias", where LLMs copy lexical information from demonstrations rather than learning underlying task information.

However, statistical data biases such as length information have been largely ignored in the in-context learning literature, yet received wide attention in the natural language inference literature with respect to traditional finetuned models (McCoy et al., 2019b; Poliak et al., 2018; Cai et al., 2017; Gururangan et al., 2018). Our work bridges this gap by looking at in-context learning with relation to a specific statistical bias: length bias.

## 3 Experiment Setup

Figure 2: An overview of in-context learning using \(K\) input-output demonstrations concatenated to the test input \(\{x_{test},y_{test}\}\).

 

In this section, we describe the experiment setup used in our analyses.

DatasetsWe use \(7\) binary classification datasets, representing natural language inference, sentiment analysis, and paraphrase detection tasks. As we sample from the tails of the length distributions, binary classification is ideal for our setting. For each dataset, we utilize the splits available from Huggingface. Dataset statistics are provided in Table 1, with detailed descriptions in subsection A.1. To count the length of each input, we use the nltk word-tokenize package Bird and Loper (2004) rather than the LLM-specific tokenizers, to maintain consistency across experiments. Prompts are adapted from Gao et al. (2023) and provided in subsection A.2.

ModelsExperiments in section 4 are run using five models from different LLM families, listed in Table 2. The selected models vary in size from 2.7B parameters to 8B parameters. Notably, the upper bound of the parameter range is due to our resource constraint, as each experiment is run using a single NVIDIA A100 GPU. For experiments in section 5 and section 6, we use a subset of these models, Llama3 and GPT-Neo. For experiments in subsection 5.1, we use the OPT model family.

Other DetailsFollowing Min et al. (2022), unless otherwise noted, all experiments use \(k=16\) demonstrations. For finetuning experiments, we use \(k=200\) finetuning examples. To minimize the impact of ordering effects, each result represents the mean of \(4\) trials, with standard deviation shown using error bars. Results are all run on the full validation split of each dataset.

In section 4, we investigate whether LLMs can learn length biases in-context. To further analyze these results, in section 5 we look at the impact of model parameter size, number of examples, and length distribution. Finally, section 6 demonstrates the utility of ICL to debias finetuned models that exhibit length biases.

## 4 Length Biases in Finetuning and ICL

In this section, we investigate the question _do models learn length biases in-context?_ We demonstrate empirically the ability of LLMs to learn length biases in-context.

### Method

Consider a dataset \(D=\{(x_{i},y_{i})\}_{i=1}^{n}\) that contains \(n\) training instances. In this work, we consider binary classification datasets, where \(Y=\{y^{1},y^{2}\}\). We aim to introduce a distributional bias in the input lengths with respect to class. To introduce a length bias in \(k\) demonstrations drawn from \(D\), we sample from the tails of each class's input length distribution. Specifically, we sample the top-\(\frac{k}{2}\) examples belonging to \(y^{1}\) and the bottom-\(\frac{k}{2}\) from \(y^{2}\) (and vice versa). This effectively produces a "worst-case scenario" in maximizing the distance between the classes under the given length distributions.

To provide a baseline for comparison, we compare against finetuning. Specifically, we finetune each model (using LoRA adapters Hu et al. (2022))

\begin{table}
\begin{tabular}{l l l c c} \hline \hline
**Category** & **Dataset** & **Task** & **\#Train** & **\#Val** \\ \hline  & QNLI Wang et al. (2018) & Natural Language Inference & 105k & 5.46k \\ Inference & RTE [1] & Natural Language Inference & 2.49k & 277 \\  & WNLI Levesque et al. (2011) & Natural Language Inference & 635 & 71 \\  & HANS McCoy et al. (2019) & Natural Language Inference & 30k & 30k \\ \hline Single Sentence & SST-2 Socher et al. (2013) & Sentiment Analysis & 67.3k & 872 \\ \hline Paraphrase & MRPC Dolan et al. (2004) & Paraphrase Detection & 3.67k & 408 \\ Detection & PAWS-X\({}_{\text{EN}}\)Yang et al. (2019) & Paraphrase Detection & 49.4k & 2k \\ \hline \hline \end{tabular}
\end{table}
Table 1: Datasets used in our experiments. We use the distributions available from Huggingface Hoest et al. (2021), and use the respective validation sets to measure performance. Dataset descriptions can be found in Table 3.

\begin{table}
\begin{tabular}{p{113.8pt} p{227.6pt}} \hline \hline
**Model** & **Parameters** \\ \hline LLMa3 Dubey et al. (2024) & 8B \\ LLMa2 Touvron et al. (2023) & 7B \\ Mistral Jiang et al. (2023) & 7B \\ OPT Zhang et al. (2022) & 6.7B \\ GPT-Neo Black et al. (2021) & 2.7B \\ \hline \hline \end{tabular}
\end{table}
Table 2: Models used in section 4.

 

on \(k=200\) training instances selected using the same procedure as above. As an additional baseline, for all experiments, we compare against randomly sampling the demonstrations and finetuning examples.

For results, we utilize a binning procedure. Specifically, we bin the validation set based on length, with \(b=6\) bins. In this setting, bin \(1\) represents the shortest \(16.67\%\) of validation instances and bin \(6\) represent the longest \(16.67\%\) validation instances across both classes. If a model has learned a length bias, for the validation class with the training set drawn from the shortest instances, we expect performance on bin \(1\) to be higher than bin \(6\), and vice versa for the validation class where the training set is drawn from the longest instances.

As a performance measurement, we report the sum of the individual accuracy from each class. As there may be a slight imbalance across classes in each bin, reporting individual class accuracy rather than the percentage of the entire bin ensures we account for class imbalances across bins.

### Results

We report results on HANS and PAWS-XEN under finetuning (Figure 4 and Figure 6) and ICL (Fig

Figure 4: Finetuning validation performance across different models on the Hans dataset. For each graph, \(y_{1}\) (Blue) was sampled from the short instances, and \(y_{2}\) (Orange) was sampled from the long instances.

Figure 5: In-context learning validation performance across different models on the PAWS-XEN dataset. For each graph, \(y_{1}\) (Blue) was sampled from the short instances, and \(y_{2}\) (Orange) was sampled from the long instances.

Figure 3: In-context learning validation performance across different models on the Hans dataset. For each graph, \(y_{1}\) (Blue) was sampled from the short instances, and \(y_{2}\) (Orange) was sampled from the long instances.

 

ure 3 and Figure 5), where \(y_{1}\) demonstrations were sampled from short instances and \(y_{2}\) demonstrations were sampled from long instances. \(y_{1}\) and \(y_{2}\) correspond to the Blue and Orange bars, respectively. Our results show decreased performance on validation examples that do not have a similar length as the demonstrations belonging to each respective class. This indicates that models can pick up length biases in-context. Additional results can be found in the Appendix.

## 5 Analysis of Influencing Factors

In this section, we investigate a further question of _what factors influence how LLMs learn length biases in-context?_ We find that increased numbers of examples can exacerbate learned biases, and models across a range of sizes can learn length biases. Further, we find that length bias can be learned from as little as a few tokens of difference in average length between classes.

### Number of Model Parameters

Existing work has suggested that the number of model parameters influences the ability of models to learn in-context, with larger models performing better Milios et al. (2023); Lu et al. (2022). In this section, we investigate whether the number of parameters also influences the ability of models to learn length biases in-context. For example, if larger models are better at learning in-context, are they more susceptible or more resilient to learning statistical biases in the data?

We use the OPT model family Zhang et al. (2022) across \(p=\{350\mathrm{M},1.3\mathrm{B},2.7\mathrm{B},6.7\mathrm{B}\}\) parameters with \(k=16\) in-context examples. Note that the parameter count is upper-bounded based on computational resources. We use the procedure described in section 4 to introduce a length bias in the in-context demonstrations.

ResultsWe report results on HANS and PAWS-XEN in Figure 7. Notably, both datasets are designed to be challenging (see subsection A.1 for descriptions). Remaining datasets and conditions are reported in the Appendix. While we do observe length bias across varying model parameter sizes, there is not a consistent pattern of increased or decreased bias with increased model parameter sizes. Accordingly, we observe a dataset-model dependence with regard to the degree of length bias a model may learn.

### Number of Examples

The performance of ICL when using various numbers of examples has been studied in prior work Wu et al. (2023); Min et al. (2022); Lu et al. (2022). As such, we investigate the sensitivity of LLM's ability to learn length bias across different numbers of in-context examples.

We use \(k=\{2,4,8,16,24,32\}\) in-context examples on the datasets in Table 1 using Llama3 8B and GPT Neo 2.7B. Following the procedure from section 4, we select the longest \(\frac{k}{2}\) examples from \(y^{1}\) and shortest \(\frac{k}{2}\) examples from the \(y^{2}\) (and vice versa), thereby introducing a bias in the length distribution of inputs across classes.

ResultsWe report on the PAWS-XEN dataset using Llama3 (8B) in Figure 8 and provide the average length for each class in subsection A.3. Our results show that models can generally begin learning biases around 8 in-context examples, with the effect typically strengthening with increased numbers of examples.

Longer context models are gaining traction,

Figure 7: Validation performance across different numbers of model parameters using the OPT model family, on the PAWS-XEN and Hans datasets. In this figure, in-context examples from \(y_{2}\) were sampled from long instances, and in-context examples from \(y_{1}\) were sampled from short instances. Each subfigure shows results on the validation instances in the respective class, with Bin 0 containing the shortest demonstrations and Bin 5 containing the longest demonstrations. Additional results can be found in the Appendix.

 

with a recent line of work focusing on scaling in-context learning to larger numbers of demonstrations. Longer contexts can increase performance and decrease sensitivity to ordering effects (Cai et al., 2023; Hao et al., 2022), and contexts (beginning around \(k=8\)) can decrease model calibration errors, where calibration is a measure of the faithfulness of a model's predictive uncertainty (Zhang et al., 2024). Our results demonstrate that longer contexts exhibit a greater potential for statistical data biases being learned in-context, and underscore the need for balanced selection methods with regard to potential data biases.

### Difference in Average Demonstration Length Between Classes

Given the results from the previous section, we investigate whether the difference of the average demonstration length between classes influences the ability of LLMs to identify a length bias. We keep the number of examples consistent at \(k=16\) and sample from \(p=\{0.25\%,0.5\%,0.75\%\}\) of the longest and shortest inputs for each class, respectively. For example, if \(y_{1}\) is the long class, we sample \(\frac{k}{2}\) instances from the longest \(p=\{0.25\%,0.5\%,0.75\%\}\) of the instances belonging to \(y_{1}\), and sample \(\frac{k}{2}\) instances from the shortest \(p=\{0.25\%,0.5\%,0.75\%\}\) of the instances belonging to \(y_{2}\).

ResultsWe report results using Llama 3 (8B) on the PAWS-XEN dataset in Figure 9, where \(0.03\) corresponds to an approximate sampling percentage from the previous experiment setup. We observe a length bias across different sampling percentages, despite the decrease in difference between average class lengths (see Table 7). Intuitively, as the difference increases, so does the spread between performance across bins of different lengths. This indicates that while models can learn length biases from a few tokens difference (approximately \(3\) tokens on HANS under \(0.75\) sampling), the biases are amplified in the model as they are amplified in the demonstrations.

Figure 8: Validation performance of Llama3 (8B) across different numbers of demonstrations on the PAWS-XEN dataset. In this figure, in-context examples from \(y_{2}\) were sampled from long instances, and in-context examples from \(y_{1}\) were sampled from short instances. Each subfigure shows results on the validation instances in the respective class. The conditions that introduce length bias in the context window (a and c subfigures) demonstrate a larger spread between performance on short and long validation instances, indicating greater potential to learn bias with longer contexts. Additional results can be found in the Appendix.

Figure 9: Validation performance across different data sampling percentages using Llama 3 (8B), on the PAWS-XEN and Hans datasets. In this figure, in-context examples from \(y_{2}\) were sampled from long instances, and in-context examples from \(y_{1}\) were sampled from short instances. Each subfigure shows results on the validation instances in the respective class, with Bin 0 containing the shortest demonstrations and Bin 5 containing the longest demonstrations. Additional results can be found in the Appendix.

 



## 6 ICL for Debiasing Finetuned Models

In-context learning eliminates the need for expensive model parameter updates incurred when fine-tuning. However, it is often the case that a model may have encoded biases picked up from the pretraining and/or finetuning. As our previous experiments show that in-context learning can learn

Figure 10: HANS validation set performance on a finetuned Llama 3 (8B) model exhibiting a length bias (see Figure 9(a) for finetuning performance prior to intervention). Figure 9(b) and Figure 9(c) (respectively) show results on two debiasing conditions: ICL demonstrations (\(k=16\)) sampled from the opposite lengths from what the model saw during finetuning (i.e. \(y_{1}\) long demonstrations, \(y_{2}\) short demonstrations), and random sampling.

Figure 11: PAWS-XEN validation set performance on a finetuned Llama 3 (8B) model exhibiting a length bias (see Figure 5(a) for finetuning performance prior to intervention). Figure 4(b) and Figure 7(c) (respectively) show results on two debiasing conditions: ICL demonstrations (\(k=16\)) sampled from the opposite lengths from what the model saw during finetuning (i.e. \(y_{1}\) long demonstrations, \(y_{2}\) short demonstrations), and random sampling.

Figure 12: WNLI validation set performance on a finetuned Llama 3 (8B) model exhibiting a length bias (see Figure 11(a) for finetuning performance prior to intervention). Figure 11(b) and Figure 11(c) (respectively) show results on two debiasing conditions: ICL demonstrations (\(k=16\)) sampled from the opposite lengths from what the model saw during finetuning (i.e. \(y_{1}\) long demonstrations, \(y_{2}\) short demonstrations), and random sampling.



length information, a natural extension is to question whether ICL can be used to "unlearn" or mitigate previously learned length biases. In this section, we answer the question _can ICL be used as an intervention to mitigate biases in finetuned models?_

We use previously finetuned models from section 4 and modify the length distribution to try to counteract the biases. Specifically we experiment with two conditions: 1) using in-context demonstrations drawn from the opposite tail of the length distribution from what was seen during finetuning, and 2) using randomly sampled in-context demonstrations. We again use Llama3 8B and GPT-Neo 2.7B for these experiments using the datasets in Table 1.

ResultsResults using Llama3 (8B) on HANS, PAWS-XEN, and WNLI are reported in Figure 10, Figure 11 and Figure 12, respectively. We find that random sampling was able to counteract the bias, essentially "unlearning" the finetuned bias. This suggests that balanced data sampling is critical to minimize the likelihood of learning biases in-context. Further, if a dataset is balanced, random sampling may be sufficient. However, if a dataset contains shortcut features, more sophisticated sampling methods to mitigate the bias may be necessary.

Moreover, our results suggest balanced sampling over showing the models demonstrations of opposite lengths with respect to the finetuned bias. Specifically, the models learned the bias in the length information in the context window, regardless of how it contradicted what was seen during finetuning. One possible explanation is that the task may be implicitly encoded during pretraining and ICL extracts the knowledge Xie et al. (2022); Min et al. (2022), however, further study is warranted on whether knowledge-extraction from ICL overrides knowledge-gain during finetuning.

## 7 Discussion

In this work, we investigate the impact of demonstration length bias on model performance when learning in-context. Under this setting, length is a statistical data bias, where the shallow feature (length) is correlated with class labels. It is important to make the distinction, however, between length as a linguistic feature containing information relevant to the underlying task, and length as an artifact of the data collection protocol. For example, length is an informative syntactic feature in classifying truthfulness vs. deceptiveness Yancheva and Rudzicz (2013) and authorship detection Yule (1939), however, length biases can also arise from artifacts reflective of heuristics used by human data annotators Gururangan et al. (2018). Our work pertains to the latter settings where length is an artifact rather than a task-informative natural language feature.

Which variables have the greatest impact on models learning length biases?In subsection A.3, we observe that when varying the number of in-context examples, the distance between classes is greater with fewer in-context examples. However, the amount of bias increases with increased numbers of examples. Further, while we observe that bias increases with increased length difference, we still observe learned bias when class length difference is reduced to as few as \(3\) tokens on the HANS dataset. This suggests that a key factor in learning bias is the number of examples the model sees. Additionally, our results suggest that any model can learn bias, and model parameter size is not necessarily correlated with increased ability to learn biases in-context.

## 8 Conclusion

In this work, we empirically investigated the ability of LLMs to learn length biases under an in-context learning paradigm. Our results show that LLMs can learn statistical biases in the data. We further show the impact of model parameter sizes, number of examples, and class length difference on length biases. Finally, we demonstrate the potential for ICL to be used as a tool to debias fine-tuned models with previously learned length biases.

## 9 Limitations

While we test models up to \(8\mathrm{B}\) parameters, we acknowledge a limitation of this work is the parameter threshold due to available computational resources. We believe our results scale to larger models.

### Acknowledgments

We thank the reviewers for their insightful feedback, which has helped us improve this paper significantly. We also thank Jason Stock for his helpful feedback and suggestions. This research was supported in part by NSF SaTC #2124538 and NSF III #2007492 



## References

* Ali et al. (2024) Ameen Ali, Lior Wolf, and Ivan Titov. 2024. Mitigating copy bias in in-context learning through neuron pruning. _arXiv preprint arXiv:2410.01288_.
* Haim et al. (2006) Roy Bar Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second PASCAL recognising textual entailment challenge.
* Bentivogli et al. (2009) Luisa Bentivogli, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo, and Bernardo Magnini. 2009. The fifth PASCAL recognizing textual entailment challenge.
* Bird and Loper (2004) Steven Bird and Edward Loper. 2004. NLTK: The natural language toolkit. In _Proceedings of the ACL Interactive Poster and Demonstration Sessions_, pages 214-217, Barcelona, Spain. Association for Computational Linguistics.
* Black et al. (2021) Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow.
* Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In _Advances in Neural Information Processing Systems_, volume 33, pages 1877-1901. Curran Associates, Inc.
* Cai et al. (2023) Tianle Cai, Kaixuan Huang, Jason D Lee, and Mengdi Wang. 2023. Scaling in-context demonstrations with structured attention. _arXiv preprint arXiv:2307.02690_.
* Cai et al. (2017) Zheng Cai, Lifu Tu, and Kevin Gimpel. 2017. Pay attention to the ending:strong neural baselines for the ROC story cloze task. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 616-622, Vancouver, Canada. Association for Computational Linguistics.
* Chen et al. (2023) Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, and He He. 2023. On the relation between sensitivity and accuracy in in-context learning. In _Findings of the Association for Computational Linguistics: EMNLP 2023_, pages 155-167, Singapore. Association for Computational Linguistics.
* Dagan et al. (2006) Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. In _Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment_, pages 177-190. Springer.
* Dolan et al. (2004) William Dolan, Chris Quirk, Chris Brockett, and Bill Dolan. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources.
* Dubey et al. (2006) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahale, Aiesha Lettman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Stavankumar, Artem Korenev, Arthur Hinnsvark, Avnu Rao, Aston Zhang, Aureien Rodriguez, Austen Gregerson, Ara Spataruw, Baptiste Rozierz, Betham Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Choe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaids, Damian Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esibou, Dhruv Chaudhury, Dhruva Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlbBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabriele Lee, Georgia Lewis Anderson, Graeme Nail, Gregoire Milalon, Guan Pang, Guillem Cucurrell, Hailey Nguyen, Hannah Korcevar, Hui Xu, Hugo Touvorno, Hiyan Zarov, Imanol Arrieta Ibarra, Isabel Klumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteg Jia, Kalyan Vasudeh Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arni, Krithika Iyer, Kashitiz Malik, Kuenley Chiui, Kunal Bhalla, Lauren Rentala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madanan, Lubo Malo, Lukas BECher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Olivier Duchenne, Onur Celebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prijwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiaco Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silviera Cabral, Robert Stoqinic, Roberta Raileanu, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas

Scialom, Tobias Speckbacher, Todor Mihalyov, Tong
* Xiao et al. (2017) Xiao, Ujjwal Karn, Vedanqui Goswami, Vibthor Gupta, Vignesh Ramanathan, Viktor Kerkerz, Vincent Gonuguet, Virginie Do, Vish Vogeti, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenvin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaoqing Ellen Tan, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakios, Aaditya Singh, Aaron Grattatiori, Abha Jain, Adam Kelsey, Adam Shajinfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alex Vaughan, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Anam Yunus, Andrei Lupu, Andres Avlaroado, Andrew Coulges, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Franco, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardt, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranajape, Bing Liu, Bo Wu, Boyu Ni, Braden Enstock, Bram Wasti, Brandon Spence, Broni Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Changshan Wang, Changkyyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyatt, David Akkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Firat Ozgenel, Francesco Caggioni, Francisco Guzzian, Frank Kanayet, Franik Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Govind Thattai, Grant Herman, Grigory Sizov, Guangyi, Zhang, Gunna Lakshminarayanan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Asgeprem, Hunter Goldman, Ibrahim Damali, Igor Molbybog, Igor Tufanov, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Jaaphet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carroll, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjije Wang, Kai Wu, Kam Hou U, Karan Saxena, Karthik Prasad, Kartikvhandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kun Huang, Kunal Chawla, Kwala Lakhotia, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeug Guo, Licheng Yu, Liron Moshkovitch, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Maria Timspoukelli, Martynas Markus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Kenelly, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatsikov, Mixaly Samvelyan, Mike Clark, Mike Macey, Mike Wang, Mujouetr Hermoso, Mo Metatan, Mohammad Rastegarit, Munisi Bahansal, Nandhnii Santhanam, Natachaa Parks, Natasha White, Navyata Baswa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikolay Pavlovi Laptev, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salplekar, Ozlem Kalinnil, Parkin Kent, Parth Parekh, Paul Saabo, Paw Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Ragbotham Murthy, Raghu Nagani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Rohan Mahesswarni, Russ Howes, Ruyt Rinot, Sau Jayesh Bondu, Samyak Datta, Sara Chapta, Sarah Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurbah Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Sheng Feng, Shengaho Lin, Shengxin Cindy Zha, Shiva Shankar, Shuqiang Zhang, Shinqing Wang, Sneha Agarwal, Soji Squjugibe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varnu Vontimitis, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vitor Albiero, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaofang Wang, Xiaojiang Wu, Xiaolan Wang, Xide Xia, Xilun Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhao, Yossi Adi, Younglin Nam, Yu, Wang, Yuchen Hao, Yundi Qian, Yuzi He, Zach Rait, Zachary DeVito, Zel Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. 2024. The llama 3 herd of models. _Preprint_, arXiv:2407.21783.
* Fei et al. (2023) Yu Fei, Yifan Hou, Zengm Chen, and Antoine Bosseltut, 2023. Mitigating label biases for in-context learning. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 14014-14031, Toronto, Canada. Association for Computational Linguistics.
* Gao et al. (2023) Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonnell, Niklas Mueninghoff, Chris Olecipa, Jason Phang, Laris Reynolds, Hailey Schoeklopf, Ariya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. 2023. A framework for few-shot language model evaluation.
* Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics 

and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 3816-3830, Online. Association for Computational Linguistics.
* Giampiccolo et al. (2007) Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recognizing textual entailment challenge. In _Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing_, pages 1-9. Association for Computational Linguistics.
* Gururangan et al. (2018) Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_, pages 107-112, New Orleans, Louisiana. Association for Computational Linguistics.
* Hao et al. (2022) Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, and Furu Wei. 2022. Structured prompting: Scaling in-context learning to 1,000 examples. _arXiv preprint arXiv:2212.06713_.
* Hu et al. (2022) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In _International Conference on Learning Representations_.
* Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lelio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scaco, Thibaut Lavrii, Thomas Wang, Timothee Lacroix, and William El Sayed. 2023. Mistral 7b. _Preprint_, arXiv:2310.06825.
* Levesque et al. (2011) Hector J Levesque, Ernest Davis, and Leora Morgenstern. 2011. The Winograd schema challenge. In _AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning_, volume 46, page 47.
* Lhoest et al. (2021) Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Phu, Lewis Tunstall, Joe Davison, Mario Sakko, Gunjan Chhablani, Bhavitya Malik, Simon Brandeis, Tevern Le Scaco, Victor Sanh, Cannon Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clement Delangue, Theo Matussiere, Lysandre Debut, Stas Bekman, Pierric Cisatec, Thibault Goehringer, Victor Mustar, Francois Lagunas, Alexander Rush, and Thomas Wolf. 2021. Datasets: A community library for natural language processing. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_, pages 175-184, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Li et al. (2022) Lvxue Li, Jiaqi Chen, Xinyu Lu, Yaojie Lu, Hongyu Lin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongji Liu, Xiameei Han, and Le Sun. 2022. Debising in-context learning by instructing LLMs how to follow demonstrations. In _Findings of the Association for Computational Linguistics ACL 2024_, pages 7203-7215, Bangok, Thailand and virtual meeting. Association for Computational Linguistics.
* Liu et al. (2022) Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for GPT-3? In _Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures_, pages 100-114, Dublin, Ireland and Online. Association for Computational Linguistics.
* Lu et al. (2022) Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 8086-8098, Dublin, Ireland. Association for Computational Linguistics.
* McCoy et al. (2019a) R. Thomas McCoy, Ellie Pavlick, and Tal Linzen. 2019a. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. _CoRR_, abs/1902.01007.
* McCoy et al. (2019b) Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019b. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 3428-3448, Florence, Italy. Association for Computational Linguistics.
* Milios et al. (2023) Aristides Milios, Siva Reddy, and Dzmitry Bahdanau. 2023. In-context learning for text classification with many labels. In _Proceedings of the 1st GenBench Workshop on (Benchmarking) Generalisation in NLP_, pages 173-184, Singapore. Association for Computational Linguistics.
* Min et al. (2022) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikela Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 11048-11064, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* Poliak et al. (2018) Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis only baselines in natural language inference. In _Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics_, pages 180-191, New Orleans, Louisiana. Association for Computational Linguistics.
* Rubin et al. (2022) Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. Learning to retrieve prompts for in-context 

learning. In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2655-2671, Seattle, United States. Association for Computational Linguistics.
* Schick and Schutze (2021) Timo Schick and Hinrich Schutze. 2021. It's not just size that matters: Small language models are also few-shot learners. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2339-2352, Online. Association for Computational Linguistics.
* Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In _Proceedings of the 2013 conference on empirical methods in natural language processing_, pages 1631-1642.
* Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. _arXiv preprint arXiv:2302.13971_.
* Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_, pages 353-355, Brussels, Belgium. Association for Computational Linguistics.
* Wei et al. (2022) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Demy Zhou, Donald Metzler, Ed H. Chi, Tatsumori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language models. _Transactions on Machine Learning Research_. Survey Certification.
* Wu et al. (2023) Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. 2023. Self-adaptive in-context learning: An information compression perspective for inconstext example selection and ordering. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1423-1436, Toronto, Canada. Association for Computational Linguistics.
* Xie et al. (2022) Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. An explanation of in-context learning as implicit bayesian inference. In _International Conference on Learning Representations_.
* Yancheva and Rudzicz (2013) Maria Yancheva and Frank Rudzicz. 2013. Automatic detection of deception in child-produced speech using syntactic complexity features. In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 944-953, Sofia, Bulgaria. Association for Computational Linguistics.
* Yang et al. (2019) Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. PAWS-X: A cross-lingual adversarial dataset for paraphrase identification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3687-3692, Hong Kong, China. Association for Computational Linguistics.
* Ye et al. (2023) Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2023. Compositional exemplars for in-context learning. In _Proceedings of the 40th International Conference on Machine Learning_, volume 202 of _Proceedings of Machine Learning Research_, pages 39818-39833. PMLR.
* Velte (1939) G Udny Yule. 1939. On sentence-length as a statistical characteristic of style in prose: With application to two cases of disputed authorship. _Biometrika_, 30(3/4):363-390.
* Zhang et al. (2024) Hanlin Zhang, YiFan Zhang, Yaodong Yu, Dhruv Madeka, Dean Foster, Eric Xing, Himabindu Lakkaraju, and Sham Kakade. 2024. A study on the calibration of in-context learning. In _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 6118-6136, Mexico City, Mexico. Association for Computational Linguistics.
* Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. _arXiv preprint arXiv:2205.01068_.
* Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In _Proceedings of the 38th International Conference on Machine Learning_, volume 139 of _Proceedings of Machine Learning Research_, pages 12697-12706. PMLR.

 

Appendix

### Datasets

#### Prompts
 



## 6 Summary

\begin{table}
\begin{tabular}{p{142.3pt} p{85.4pt}} \hline \multicolumn{2}{c}{**Binary NLI Datasets**} \\ \hline
**Dataset** & **Description** \\ \hline QNLI Wang et al. (2018) & The Stanford Question Answering Dataset. A corpus of question-sentence pairs, with context sentences extracted from Wikipedia and questions written by a human annotator. The model is tasked with determining whether the context contains the answer to (entails) the question. \\ \hline RTE [2] & The Recognizing Textual Entailment (RTE) datasets. A corpus constructed from annual textual entailment challenges based on news and Wikipedia text. \\ \hline WNLI Levesque et al. (2011) & The Winograd Schema Challenge. A corpus of reading comprehension sentence pairs, where ambiguous pronouns are replaced with each possible referent. The task is to predict if the substituted sentence is entailed by the original sentence. \\ \hline HANS McCoy et al. (2019a) & Heuristic Analysis for NLI Systems. A corpus of challenging premise and hypothesis pairs designed to target evaluation of lexical overlap, sub-sequence, and constituent heuristics. \\ \hline  & **Single-Sentence Datasets** \\ \hline
**Dataset** & **Description** \\ \hline SST-2 Socher et al. (2013) & The Starford Sentiment Treebank. A corpus of sentences extracted from movie reviews, with human judgments of positive or negative sentiment. \\ \hline  & **Similarity \& Paraphrase Detection Datasets** \\ \hline
* Cross-lingual. A corpus of challenging paraphrase and non-paraphrase pairs created using data from Wikipedia. Sentence pairs were generated using controlled word swapping and back translation to ensure high lexical overlap, and human raters judged semantic equivalence. Our experiments utilize the English data split. \\ \hline \end{tabular}
\end{table}
Table 3: Dataset descriptions.




\begin{table}
\begin{tabular}{l l} \hline \multicolumn{2}{c}{**Binary NLI Datasets**} \\ \hline
**Dataset** & **Prompt** \\ \hline QNLI Wang et al. (2018) & \begin{tabular}{c} \{sentence\} \\ Question: \{question\} True or False? \\ Answer: \\ \end{tabular} \\ \hline RTE 3 & \begin{tabular}{c} \{sentence1\} \\ Question: \{sentence2\} True or False? \\ Answer: \\ \end{tabular} \\ \hline WNLI Levesque et al. (2011) & \begin{tabular}{c} \{sentence1\} \\ Question: \{sentence2\} True or False? \\ Answer: \\ \end{tabular} \\ \hline HANS McCoy et al. (2019) & 
\begin{tabular}{c} \{premise\} \\ Question: \{hypothesis\} True or False? \\ Answer: \\ \end{tabular} \\ \hline  & \multicolumn{2}{c}{**Single-Sentence Datasets**} \\ \hline
**Dataset** & **Description** \\ \hline SST-2 Socher et al. (2013) & 
\begin{tabular}{c} \{sentence\} \\ Question: Is this sentence positive or negative? \\ Answer: \\ \end{tabular} \\ \hline \multicolumn{2}{c}{**Similarity \& Paraphrase Detection Datasets**} \\ \hline
**Dataset** & **Description** \\ \hline MRPC Dolan et al. (2004) & \begin{tabular}{c} Sentence 1: \{sentence1\} \\ Sentence 2: \{sentence2\} \\ Question: Do both sentences mean the same thing? \\ Answer: \\ \end{tabular} \\ \hline PAWS-XEN Yang et al. (2019) & 
\begin{tabular}{c} Sentence 1: \{sentence1\} \\ Sentence 2: \{sentence2\} \\ Question: Do both sentences mean the same thing? \\ Answer: \\ \end{tabular} \\ \hline \end{tabular}
\end{table}
Table 4: Prompts used in our experiments.





\begin{table}
\begin{tabular}{c c l c c c c c c} \hline \hline
**Dataset** & **Condition** & **Class** & \multicolumn{6}{c}{**\# Examples**} \\ \cline{3-10}  & & 2 & 4 & 8 & 16 & 24 & 32 \\ \hline \multirow{5}{*}{QNLI} & Random & True & 51.33 & 42.5.00 & 47.67 & 44.81 & 48.48 & 45.49 \\ \cline{2-10}  & False & 48.60 & 36.5 & 49.94 & 47.70 & 46.86 & 48.44 \\ \cline{2-10}  & False-L & True & 14.00 & 14.00 & 14.75 & 15.38 & 15.75 & 16.06 \\ \cline{2-10}  & False & 227.00 & 207.50 & 191.50 & 175.00 & 165.75 & 159.50 \\ \cline{2-10}  & True-L & True & 446.00 & 445.40 & 445.00 & 358.63 & 305.92 & 276.00 \\ \cline{2-10}  & False & 15.00 & 15.50 & 15.75 & 15.88 & 16.00 & 16.25 \\ \hline \multirow{5}{*}{RTE} & Random & True & 60.00 & 59.33 & 61.25 & 66.57 & 63.02 & 65.77 \\ \cline{2-10}  & False & 44.00 & 57.71 & 53.00 & 63.06 & 56.10 & 63.36 \\ \cline{2-10}  & False-L & True & 17.00 & 17.50 & 18.5 & 19.63 & 20.17 & 20.75 \\ \cline{2-10}  & False & 277.00 & 253.00 & 233.75 & 216.75 & 206.83 & 200.75 \\ \cline{2-10}  & True-L & True & 195.00 & 194.00 & 192.00 & 187.25 & 183.67 & 181.19 \\ \cline{2-10}  & False & 16.00 & 17.50 & 18.75 & 19.75 & 20.75 & 21.50 \\ \hline \multirow{5}{*}{WNLI} & Random & True & 37.75 & 45.50 & 41.07 & 41.33 & 38.20 & 39.62 \\ \cline{2-10}  & False & 41.75 & 38.17 & 39.50 & 35.18 & 36.65 & 36.72 \\ \cline{2-10}  & False-L & True & 19.00 & 19.50 & 19.75 & 20.38 & 20.83 & 21.25 \\ \cline{2-10}  & False & 92.00 & 91.50 & 87.75 & 82.38 & 79.67 & 77.13 \\ \cline{2-10}  & True-L & True & 93.00 & 91.50 & 89.50 & 84.88 & 82.42 & 80.63 \\ \cline{2-10}  & False & 19.00 & 19.00 & 19.50 & 20.13 & 20.58 & 21.00 \\ \hline \multirow{5}{*}{HANS} & Random & True & 18.80 & 19.25 & 21.50 & 20.21 & 20.17 & 20.91 \\ \cline{2-10}  & False & 20.67 & 21.38 & 21.22 & 21.31 & 21.41 & 21.05 \\ \cline{2-10}  & False-L & True & 13.00 & 13.00 & 13.00 & 13.00 & 13.00 & 13.00 \\ \cline{2-10}  & False & 27.00 & 27.00 & 27.00 & 27.00 & 27.00 & 27.00 \\ \cline{2-10}  & True-L & True & 27.00 & 27.00 & 27.00 & 27.00 & 27.00 & 27.00 \\ \cline{2-10}  & False & 15.00 & 15.00 & 15.00 & 15.00 & 15.00 & 15.00 \\ \hline \multirow{5}{*}{SST-2} & Random & True & 21.00 & 21.00 & 20.85 & 19.21 & 19.62 & 19.30 \\ \cline{2-10}  & False & 22.33 & 18.00 & 19.16 & 18.00 & 17.78 & 19.16 \\ \cline{2-10}  & False-L & True & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 \\ \cline{2-10}  & True-L & False & 61.00 & 61.00 & 60.50 & 60.00 & 59.58 & 59.19 \\ \cline{2-10}  & True-L & True & 62.00 & 61.50 & 61.00 & 60.00 & 59.08 & 58.44 \\ \cline{2-10}  & False & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 \\ \hline \multirow{5}{*}{MRPC} & Random & True & 59.00 & 57.00 & 59.33 & 58.67 & 59.13 & 60.22 \\ \cline{2-10}  & False & 57.29 & 59.91 & 58.61 & 63.06 & 62.06 & 60.46 \\ \cline{2-10}  & False-L & True & 34.00 & 34.00 & 34.50 & 35.13 & 35.67 & 36.19 \\ \cline{2-10}  & False & 97.00 & 91.50 & 88.25 & 86.00 & 85.00 & 84.25 \\ \cline{2-10}  & True-L & True & 82.00 & 81.50 & 81.25 & 80.25 & 79.50 & 79.00 \\ \cline{2-10}  & False & 32.00 & 32.00 & 33.00 & 33.50 & 34.00 & 34.69 \\ \hline \multirow{5}{*}{PAWS-XEN} & Random & True & 63.33 & 57.50 & 58.50 & 57.84 & 58.11 & 58.00 \\ \cline{2-10}  & False & 61.60 & 58.60 & 58.06 & 54.31 & 58.79 & 59.42 \\ \cline{2-10}  & False-L & True & 26.00 & 27.00 & 27.50 & 28.25 & 28.50 & 28.63 \\ \cline{2-10}  & False & 85.00 & 85.00 & 84.50 & 83.50 & 83.00 & 82.50 \\ \cline{2-10}  & True-L & 86.00 & 86.00 & 86.00 & 85.63 & 85.25 & 84.94 \\ \cline{2-10}  & False & 26.00 & 26.00 & 26.25 & 27.63 & 28.33 & 28.75 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Average input length (including prompt length) across different numbers of examples.





\begin{table}
\begin{tabular}{l l l} \hline \hline
**Dataset** & **Class** & **Average Length** \\ \hline \multirow{2}{*}{QNLI} & True & 50.03 \\ \cline{2-3}  & False & 47.50 \\ \hline \multirow{2}{*}{RTE} & True & 63.25 \\ \cline{2-3}  & False & 64.99 \\ \hline \multirow{2}{*}{WNLI} & True & 40.23 \\ \cline{2-3}  & False & 37.00 \\ \hline \multirow{2}{*}{HANS} & True & 20.53 \\ \cline{2-3}  & False & 20.99 \\ \hline \multirow{2}{*}{SST2} & True & 28.17 \\ \cline{2-3}  & False & 28.93 \\ \hline \multirow{2}{*}{MRPC} & True & 57.72 \\ \cline{2-3}  & False & 61.16 \\ \hline \multirow{2}{*}{PAWS-XEN} & True & 58.63 \\ \cline{2-3}  & False & 58.63 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Average input length for each validation set. Reported input lengths include the prompt length (consistent across all inputs); prompts can be found in subsection A.2.





\begin{table}
\begin{tabular}{c c c c c c} \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Condition**} & \multirow{2}{*}{**Class**} & \multicolumn{3}{c}{**\% Sampled**} \\ \cline{4-6}  & & & 0.25 & 0.50 & 0.75 \\ \hline \multirow{3}{*}{QNLI} & False-L & True & 31.35 & 36.27 & 40.91 \\ \cline{3-6}  & & False & 65.87 & 57.55 & 52.05 \\ \cline{3-6}  & True-L & True & 69.98 & 60.16 & 53.93 \\ \cline{3-6}  & & False & 31.56 & 36.33 & 40.61 \\ \hline \multirow{3}{*}{RTE} & False-L & True & 33.05 & 40.03 & 46.23 \\ \cline{3-6}  & & False & 122.98 & 90.66 & 76.35 \\ \cline{3-6}  & True-L & True & 124.51 & 91.02 & 77.78 \\ \cline{3-6}  & & False & 33.44 & 39.98 & 47.21 \\ \hline \multirow{3}{*}{WNLI} & False-L & True & 25.23 & 28.01 & 31.70 \\ \cline{3-6}  & & False & 56.01 & 47.21 & 40.60 \\ \cline{3-6}  & True-L & True & 60.08 & 48.73 & 43.12 \\ \cline{3-6}  & & False & 24.72 & 28.12 & 31.05 \\ \hline \multirow{3}{*}{HANS} & False-L & True & 16.92 & 18.72 & 19.65 \\ \cline{3-6}  & & False & 23.59 & 22.65 & 21.97 \\ \cline{3-6}  & True-L & True & 23.29 & 22.39 & 21.77 \\ \cline{3-6}  & & False & 18.17 & 19.40 & 20.16 \\ \hline \multirow{3}{*}{SST2} & False-L & True & 11.07 & 12.66 & 14.92 \\ \cline{3-6}  & & False & 29.57 & 23.90 & 20.36 \\ \cline{3-6}  & True-L & True & 30.55 & 25.14 & 21.37 \\ \cline{3-6}  & & False & 11.06 & 12.26 & 14.27 \\ \hline \multirow{3}{*}{MRPC} & False-L & True & 44.89 & 49.24 & 53.12 \\ \cline{3-6}  & & False & 74.31 & 69.98 & 65.67 \\ \cline{3-6}  & True-L & True & 70.53 & 65.61 & 61.50 \\ \cline{3-6}  & & False & 47.12 & 52.35 & 56.78 \\ \hline \multirow{3}{*}{PAWS-XEN} & False-L & True & 44.61 & 49.51 & 53.98 \\ \cline{3-6}  & & False & 72.44 & 67.80 & 63.31 \\ \cline{3-6}  & True-L & True & 72.58 & 67.93 & 63.46 \\ \cline{3-6}  & & False & 44.56 & 49.53 & 54.04 \\ \hline \end{tabular}
\end{table}
Table 7: Average input length across different sampling bins (by percentage of data sampled from). Reported input lengths include the prompt length (consistent across all inputs); prompts can be found in subsection A.2.





### Additional Length Bias Results

Figure 13: ICL performance of Llama3 8B, Llama2 7B, Mistral 7B, OPT 6.7B, and GPT Neo 2.7B (from left to right) where \(y_{1}\) (Blue) samples long demonstrations and \(y_{2}\) (Orange) samples short demonstrations.

 

Figure 14: Finetuning performance of Llama3 8B, Llama2 7B, Mistral 7B, OPT 6.7B, and GPT Neo 2.7B (from left to right) where \(y_{1}\) (Blue) samples long demonstrations and \(y_{2}\) (Orange) samples short demonstrations.

 

Figure 15: ICL performance of Llama3 8B, Llama2 7B, Mistral 7B, OPT 6.7B, and GPT Neo 2.7B (from left to right) where \(y_{1}\) (Blue) samples short demonstrations and \(y_{2}\) (Orange) samples long demonstrations.

 

Figure 16: Finetuning performance of Llama3 8B, Llama2 7B, Mistral 7B, OPT 6.7B, and GPT Neo 2.7B (from left to right) where \(y_{1}\) (Blue) samples short demonstrations and \(y_{2}\) (Orange) samples long demonstrations.

 

Figure 17: ICL performance of Llama3 8B, Llama2 7B, Mistral 7B, OPT 6.7B, and GPT Neo 2.7B (from left to right) where \(y_{1}\) (Blue) and \(y_{2}\) (Orange) are both randomly sampled.

 

Figure 18: Finetuning performance of Llama3 8B, Llama2 7B, Mistral 7B, OPT 6.7B, and GPT Neo 2.7B (from left to right) where \(y_{1}\) (Blue) and \(y_{2}\) (Orange) are both randomly sampled.

 



### Additional Model Parameter (OPT) Results

Each of the following figures shows validation performance when varying the number of model parameters using the OPT model family. Bin 0 contains the shortest demonstrations and Bin 5 contains the longest demonstrations. Each subfigure shows the validation accuracy on a single class when in-context instances belonging to the respective class were sampled from long instances, short instances, and randomly sampled (left to right).

Figure 21: RTE dataset (OPT) Figure 24: SST-2 dataset (OPT)



### Additional Number of Examples Results

Each of the following figures shows validation performance when varying the number of examples using Llama3 8B and GPT Neo 2.7B. Bin 0 contains the shortest demonstrations and Bin 5 contains the longest demonstrations. Each subfigure shows the validation accuracy on a single class when incontext instances belonging to the respective class were sampled from long instances, short instances, and randomly sampled (left to right).

Figure 29: QNLI dataset (Llama 3 8B)

Figure 36: QNLI dataset (GPT Neo 2.7B)

Figure 34: PAWS-XEN dataset (GPT Neo 2.7B)

Figure 35: RTE dataset (GPT Neo 2.7B)





### Additional Length Difference Results

Each of the following figures shows validation performance when varying the sampling percentage from each class using Llama3 8B and GPT Neo 2.7B. Bin 0 contains the shortest demonstrations and Bin 5 contains the longest demonstrations. Each subfigure shows the validation accuracy on a single class when in-context instances belonging to the respective class were sampled from long instances (left) and short instances (right).

Figure 41: PAWS-X\({}_{\text{EN}}\) dataset (Llama 3 8B)

Figure 42: RTE dataset (Llama 3 8B)

Figure 43: QNLI dataset (Llama 3 8B)

Figure 45: MPRC dataset (Llama 3 8B) Figure 46: SST-2 dataset (Llama 3 8B) Figure 47: Hans dataset (GPT Neo 2.7B)



Figure 51: WNLI dataset (GPT Neo 2.7B)

Figure 53: SST-2 dataset (GPT Neo 2.7B)

Figure 54: WNLI dataset (GPT Neo 2.7B)





### Additional Intervention Results

Each of the following figures shows validation set performance on a finetuned Llama 3 8B or GPT Neo 2.7B model exhibiting a length bias. For each figure, (a) shows finetuning performance prior to intervention. (b) and (c) show results on two debiasing conditions: ICL demonstrations (\(k=16\)) sampled from the opposite lengths from what the model saw during finetuning (e.g \(y_{1}\) long demonstrations, \(y_{2}\) short demonstrations), and random sampling, respectively.



Figure 54: RTE (Llama 3 8B)

Figure 56: MRPC (Llama 3 8B)

Figure 57: SST-2 (Llama 3 8B)



Figure 61: QNLI (Llama 3 8B)

Figure 59: PAWS-XEN (Llama 3 8B)



Figure 64: SST-2 (Llama 3 8B)

Figure 65: Hans (GPT Neo 2.7B)

Figure 63: MRPC (Llama 3 8B)

Figure 64: SST-2 (Llama 3 8B)



Figure 68: QNLI (GPT Neo 2.7B)

Figure 69: WNLI (GPT Neo 2.7B)



Figure 71: SST-2 (GPT Neo 2.7B)

Figure 73: PAWS-\({}_{\text{EN}}\) (GPT Neo 2.7B)



Figure 76: WNLI (GPT Neo 2.7B)

Figure 74: RTE (GPT Neo 2.7B)

Figure 75: QNLI (GPT Neo 2.7B)



Figure 78: SST-2 (GPT Neo 2.7B)