Mitigating Label Biases for In-context Learning
Year: 2023

Abstract:
Various design settings for in-context learning (ICL), such as the choice and order of the incontext examples, can bias the model’s predictions. While many studies discuss these design choices, there have been few systematic investigations into categorizing them and mitigating their impact. In this work, we define a typology for three types of label biases in ICL for text classification: vanilla-label bias, contextlabel bias, and domain-label bias (which we conceptualize and detect for the first time). Our analysis demonstrates that prior label bias calibration methods fall short of addressing all three types of biases. Specifically, domainlabel bias restricts LLMs to random-level performance on many tasks regardless of the choice of in-context examples. To mitigate the effect of these biases, we propose a simple bias calibration method that estimates a language model’s label bias using random indomain words from the task corpus. After controlling for this estimated bias when making predictions, our novel domain-context calibration significantly improves the ICL performance of GPT-J and GPT-3 on a wide range of tasks. The gain is substantial on tasks with large domain-label bias (up to 37% in MacroF1). Furthermore, our results generalize to models with different scales, pretraining methods, and manually-designed task instructions, showing the prevalence of label biases in ICL.

heading: 1 Introduction
text: Large language models (LLMs) can perform unseen tasks by conditioning on a context prompt that consists of a few training example-label pairs (Brown et al., 2020). However, such in-context learning ability is highly sensitive to various design settings, such as the choice (Liu et al., 2021) and order (Lu et al., 2021) of the in-context samples.
*Equal contribution. †Work done while interning at EPFL.
Recently, Zhao et al. (2021) showed that the instability of ICL largely arises from the fact that these design settings bias the model toward predicting certain answers (e.g., LLMs often predict the label of the last in-context example). As a result, the sensitivity of the results in ICL studies calls for a systematic discussion of biases in ICL and new methods to properly categorize, detect, and comprehensively mitigate various types of biases.
In this work, we conduct a thorough investigation of biases in ICL for text classification. We start by defining a typology of three types of label biases (the model’s undesirable preference toward certain label names): vanilla label bias, context-label bias, and domain-label bias. What we term vanilla label bias captures the model’s non-contextualized preference for the label names (e.g., the common token bias mentioned by Zhao et al. (2021) caused by different frequencies of label names in the pretraining corpus). Context-label bias summarizes the effects of the context prompt (e.g., LLMs tend to prefer the majority and last label of the in-context examples). Finally, domain-label bias captures the effects of the task corpus on the model’s predictions.
We show that domain-label biases significantly affect a model’s prediction in ICL. For example,
14014
on a hate detection task with two nearly balanced classes, simply seeing random words sampled from the dataset severely biases the model towards predicting the label hate (Fig. 1(a)), while seeing random English words does not show such an effect. More importantly, on many tasks with large domain-label bias, LLMs achieve no better than random performance, regardless of the choice of in-context examples (Fig. 2). Moreover, we find that existing bias mitigation methods, such as Contextual Calibration (CC; Zhao et al., 2021) do not combat this effect.
To this end, we propose Domain-context Calibration (DC) to mitigate label biases in ICL. DC first estimates the effects of different label biases holistically using random words sampled from the task corpus. Specifically, we compute the probabilities assigned by the model to each label using random in-domain words as the task input (with optional real in-context learning examples prepended). Using random words limits the semantic meaning of the input, allowing us to estimate the vanilla-label and context-label biases while using in-domain words accounts for the effect of the task corpus. Then, at inference time, we use this label bias estimate to calibrate the model’s output probabilities.
We evaluate the impact of DC on 24 classification datasets, showing that DC improves the average ICL performance of GPT-J (Wang and Komatsuzaki, 2021) and GPT-3 by 20% and 18%. We observe substantial gains on tasks with large domainlabel bias (up to 37% in Macro-F1). DC also bene-
fits models with different scales, instruction-tuning (e.g., Instruct-GPT, Ouyang et al., 2022), and provided with task instructions. Finally, we show that DC improves the zero-shot prompting performance of smaller models like RoBERTa (Liu et al., 2019), demonstrating that label bias can be mitigated in prompt-based frameworks beyond ICL.
Overall, our work proposes a new typology of label biases in prompt-based methods, and a simple method for mitigating them. When studying ICL on a diverse collection of datasets, the results on datasets with severe label bias can obfuscate the actual behaviors of the model. Thus, rigorous design for dataset selection (that accounts for confounders) and fine-grained analysis of individual datasets are essential for effectively studying ICL.

heading: 2 Categorizing Label Biases in ICL
text: In this paper, we focus on in-context learning (ICL; Fig. 3) for classification tasks. Formally, we consider a dataset of examples {xi, yi} where xi are text inputs and each yi can be mapped to a verbalization in a label name set L. We assume each class has one label name. For example, in a sentiment task, L could be composed of positive and negative as label names. Given a context prompt C consisting of a few labeled examples and an input text xi, the model M determines the label of xi by computing: argmaxy∈L PM (y|xi, C). Using this notation, we define our typology of label biases based on the mathematical formulation of ICL.

heading: 2.1 A Typology of Label Biases
text: To perform a classification task, a model needs to learn the underlying text-label mapping, i.e., P (y|x). In supervised learning, such mapping is learned by optimizing the model using the training data. In ICL, on the other hand, the model is fixed, and it determines the label of a text by computing the probabilities of predicting the label names
PM (y|x,C). Notice that there are three components involved in the inference: the label name y, the text x from a specific task corpus, and the context C. Accordingly, as shown in Fig. 4, we can define three types of label biases that lead to a discrepancy between PM (y|x,C) and P (y|x). Vanilla-label bias pertains to the uncontextual preference of the model towards predicting certain label names. One possible cause is the pre-training term frequencies of the label names. Zhao et al. (2021) reported a high correlation between the frequency of the DBPedia dataset label names and the rate at which GPT-3 predicts those labels.
Context-label bias summarizes the effect of the context prompt. With in-context learning, the model “learns” from a few examples, and the learning is particularly sensitive to seemingly arbitrary decisions such as the order of the in-context examples (Lu et al., 2021) and the task template used to map the example to text that the model can process (Mishra et al., 2021; Holtzman et al., 2021).
Domain-label bias captures the effect of the task corpus. Beyond the text-label association demonstrated in the in-context examples, the model also relies on its prior knowledge of the task when making predictions. We show that the association of words to the label names learned from pre-training is a potential pitfall and discuss domain-label bias in more detail in the next section.

heading: 3 Domain Label Bias
text: To illustrate how the domain of a task can induce label bias, consider a case where an LLM predicts whether a patient is sick or healthy based on some medical descriptions. Because medical descriptions are associated more often with people having health problems in the natural corpus, frequently used words in such documents are likely to have a stronger correlation with sick than healthy, leading to a systematic bias in the model’s predictions.
Supporting this intuition, we find that for many datasets, conditioning on random words from the dataset examples biases the model toward predicting certain label names. For example, in the hate speech detection task depicted in Fig. 1, we compute the model’s preference (prior) on both label names given random words as the input. A model such as GPT-J has no preference for either of the classes (neutral v.s. hate) given random English words, but given random in-domain words sampled
from the dataset, the label priors shift dramatically, becoming 0.95 (hate) v.s. 0.05 (neutral).
Motivated by this experiment, we quantify the domain-label bias of a model for a particular task using the distance of the model’s priors estimated using random English words and in-domain words. To make the measure more comparable on tasks with different numbers of classes, we define the following metric:
bias = 1
2
∑
y∈L
∣∣∣PM (y|xEng.)− PM (y|xi.d.) ∣∣∣, (1)
where xEng. and xi.d. correspond to L random English or random in-domain words and L is the average text length of the dataset.
We find that datasets1 exhibit different levels of domain-label bias (see Fig. 17 in App. A). More importantly, LLMs behave distinctively on datasets with small and large domain-label bias. As shown in Fig. 5, while GPT-J performs competitively on datasets with small domain-label bias, it rarely outperforms the random baselines on large-bias datasets, indicating that domain-label bias significantly affects ICL. Contextual calibration, which only considers vanilla-label bias and context-label bias, fails to handle domain-label bias.

heading: 4 Domain-context Calibration
text: In this section, we propose Domain-context Calibration (DC), which mitigates the effects of the multiple label biases of our typology (§ 2.1). Following contextual calibration (CC; Zhao et al., 2021), we estimate the overall label bias of a model
1We note that domain-label bias is model-dependent. However, we observe a high correlation of domain-label bias between LLMs in general (see App. B). Also, by definition, domain-label bias depends on the task formulation, particularly the choice of label names, which we discuss in § 7.
with respect to a task by estimating the label probabilities on a content-free example text. However, unlike CC, which uses a single, seemingly contentfree token (e.g., “N/A”) to approximate the label bias, we use random words sampled from the unlabeled evaluation dataset as the content-free text. Then, for all examples we classify for the task, we re-calibrate the model’s prediction probability using this estimated bias.
More formally, given a dataset, we first construct a bag-of-words B using the unlabeled texts {xi}. Assuming xi’s have average length L, we sample L words randomly from B to form a content-free random text, which captures the word distribution of the dataset domain. However, the random text still remains nearly content-free as it is not grammatically meaningful and potentially contains words from all classes, making it suitable for calibration. We repeat this process M times and average the estimated priors:
P̄ (y|C) = 1 M
M∑
j=1
P (y|[random text]j , C). (2)
The model then makes predictions according to the following estimate:
ŷi = argmax y∈L P (y|xi, C) P̄ (y|C) .
where P (y|xi, C) is the original probability assigned to label y for a particular example xi.

heading: 5 Experimental Setup
text: We conduct comprehensive experiments to analyze the effectiveness of our domain-context calibration in mitigating label biases in ICL.
Datasets We conduct experiments on 24 text classification datasets that cover a wide range of tasks. Most of these datasets are recently used for studying ICL (Zhao et al., 2021; Min et al., 2022; Lu et al., 2021). To control the evaluation budget, we
use a subset of the 24 datasets for GPT-3 experiments following Min et al. (2022). More details can be found in Appendix C.
Model and implementation details We use GPT-J (6B) and GPT-3 (175B) as models in our study. For all experiments, unless stated otherwise, we use k = 8 examples sampled randomly from the training set to construct the context prompt and evaluate 5 times using different random seeds. Following Min et al. (2022), we use simple and unified templates for all datasets and do not use any task instructions to keep human engineering at a minimal level. We discuss the effect of task instructions in § 6.1. The templates and label names we used for all datasets can be found in App. E. To further control the budget for evaluating with GPT-3, we follow Lu et al. (2021) and sample a subset of size 500 for all datasets whose test sets have sizes exceeding this number. For domain-context calibration, we use the unlabeled test set for sampling random in-domain words and aggregate using M = 20 random texts (Eq. 2). We discuss the sampling of random words in more detail in Appendix F. We use Open-AI’s API for GPT-3 experiments and Tesla V100 GPUs for GPT-J inference.
Evaluation Details For each model, we compare the performance of domain-context calibration to the following baselines: random performance, uncalibrated performance, and contextual calibration performance. Following prior work, we use the Macro-F1 score as the evaluation metric.

heading: 6 Experimental Results
text: We report the average Macro-F1 scores of GPT-J (6B) and GPT-3 (175B) across the entire evaluation suite in Figure 6. Furthermore, we stratify our results into three equal-sized subsets according to their levels of domain-label bias.
Our main finding is that domain-context calibration generally improves in-context learning, especially on tasks with large domain-label bias.
On all datasets, DC consistently boosts the performance for both models with an average improvement (Macro-F1) of 20% (GPT-J) and 18% (GPT3).2 As Fig. 6 shows, the performance gain of DC over baselines (original predictions and CC) largely increases when the degree of domain-label bias increases. On the tasks with the largest domain-label bias, DC is the only method that significantly outperforms the random baseline and achieves up to 37% (GPT-J) and 35% (GPT-3) performance improvement over other baselines, indicating that DC effectively mitigates domain-label bias.

heading: 6.1 Generalizability
text: Following the finding that DC improves ICL significantly on datasets with large domain-label bias, we analyze the robustness of DC under changes in model scale, number of in-context learning examples, and task instructions (all of which have been shown to improve ICL). We use three datasets that exhibit a high level of domain-label bias for GPT-J.
2See Tab. 6 in App. D for results on individual datasets.
Scaling up the model We evaluate GPT-3 models with sizes ranging from 350M to 175B. As Fig. 7 shows, larger models (both the original prediction and CC) do not exhibit better performance on tasks with large domain-label bias. However, DC consistently improves the performance of GPT3 models of all sizes while reducing the variance due to different choices of in-context examples.
Adding more in-context examples In Table 1, we study the effect of adding more in-context examples by evaluating GPT-J and GPT-3 using 0, 8, and 16 in-context examples. For both models, adding more examples does not seem to benefit the model’s original and CC performance on tasks with large domain-label bias. However, in all settings, DC gives the best results, and for GPT-3, DC further improves the performance when provided with more in-context examples.
Task instructions and instruction-tuning Instruction-tuning and task instructions have been shown to be beneficial to ICL. As shown in Table 2, for GPT-3, providing task instructions3 improves the performance with DC much larger than the
original and CC performance, showing that DC enables GPT-3 to make better use of the task instruction. For Instruct-GPT3 (text-davinci-002), adding task instructions largely improves the model’s original performance. Still, DC yields significant improvement, while CC actually hurts the model’s performance.

heading: 6.2 Analysis
text: To understand why DC outperforms CC, we conduct a systematic analysis using GPT-J of three differences between DC and CC: 1) the effect of a predefined content-free token such as “N/A” compared to using random words; 2) the length of the random word sequence; 3) the source of random words. Below, we summarize our results from Fig. 8.
Content-free token can also be biased First, we find that replacing the pre-defined content-free token from CC (i.e., “N/A”) with a single random English word improves GPT-J’s overall performance, indicating that specific content-free tokens may themselves can be biased toward particular labels. For example, as shown in Fig. 10, on sentiment tasks, calibration GPT-J using “N/A” leads to a systematic bias toward the positive class. Calibrating using random English words to estimate the label bias avoids this problem.
Calibrating with random texts of the average input length is beneficial As shown in Fig. 8, when
3The instructions we used can be found in App. E.
calibrating using random English words, increasing the number of words improves performance. Intuitively, using random texts of the average input length for calibration gives a more precise estimate of the effect of the context prompt. To test this, we select the longest and shortest 10% samples of all 24 datasets to construct a dataset with long and short inputs for a task. Then, we test the calibration performance using random English words of different lengths4. As shown in Fig. 11, longer (shorter) texts prefer longer (shorter) random texts as calibration sequences to estimate the label bias.
Calibrating using random in-domain words removes domain-label bias Finally, calibrating using random in-domain words yields a large improvement over calibrating using random English words. We plot the prediction distributions of GPTJ on Tweet hate after calibrating with both random English and in-domain words of various lengths in Fig. 9. We see that, when only calibrating using a few in-domain words, the word distribution of the dataset is not well-captured, and thus the domainlabel bias is not effectively removed. When cali-
4We use random English words rather than random indomain words to better study the effect of the length.

heading: DC (#rand. words=0.6L)
text: brating using more in-domain words, the prediction becomes more balanced, while after calibrating using more random English words, the model is still biased towards predicting label hate. Interestingly, we notice that the more DC mitigates the domainlabel bias, the more task performance increases.

heading: 6.3 Zero-shot Prompting
text: Smaller LLMs pre-trained using masked language modeling can also be efficiently adapted to unseen tasks by reformulating the task into a cloze problem using a natural language prompt (i.e., zero-shot prompting (Schick and Schütze, 2020)). To demonstrate that DC can be used even with smaller models, we evaluate the zero-shot prompting ability of RoBERTa-large (Liu et al., 2019) when it is calibrated using DC.5. We report our results in Tab. 7 (App.G) and find that across the same 24 datasets,
5Implementation details can be found in App. G
DC achieves a significant performance gain of 26%. Similar to ICL with GPT models, label biases affect RoBERTa’s zero-shot prompting priors. However, DC effectively mitigates these biases, leading to significant performance improvements.

heading: 7 Discussion
text: Label name selection as label bias mitigation Our results outline how LLMs can be biased to certain label names for different tasks. Intuitively, because the task is formulated as generating the label name given an example, the mechanism elicits the model’s prior knowledge about the task. To better understand whether domain-label bias could be mitigated through more careful label name selection, we test GPT-J with three different pairs of label names on Tweet hate: 1) neutral v.s. hate, which is the most task-relevant set of label names but introduces severe domain-label bias; 2) favor v.s. against, a pair of less task-relevant antonyms used by Min et al. (2022); 3) X v.s. Y, which are meaningless placeholders.
As shown in Fig. 12, calibrating using random English words or in-domain words makes little difference when choosing (X, Y) or (favor, against) as the label names, showing that they do not introduce domain-label bias. However, although GPT-J is able to achieve better original and CC performance on these label names, (neutral, hate) yields the best performance after removing domain-label bias using DC. Thus, with proper calibration, using the most task-indicative words as label names is likely to be the best option. Surprisingly, the manually picked label names (favor, against) under-perform the meaningless ones (X, Y) after applying DC, hinting that human-plausible label names are not necessarily good label names for LLMs.
Select datasets for ICL analysis The varying levels of domain-label bias in our studied datasets suggest a large variance in how ICL will perform on different datasets. Consequently, macro-averaging the performance on datasets with differing levels of label biases potentially obfuscates diverse results among different tasks. Our work encourages future studies to select datasets carefully to cover varying degrees of label bias among reported results, and to perform fine-grained analysis of individual datasets to when studying ICL performance.
Alternate causes of domain label bias When evaluating models for real-world applications such as hate speech detection, we usually use hard examples (e.g., non-hateful, but “hateful-looking” examples) to check the robustness of the model. However, LLMs trained on natural corpora are likely to be susceptible to adversarial word-level features (LLMs use word associations learned from pretraining to perform ICL). To some degree, adversarial examples could also be a source or large domain-label bias on many datasets.

heading: 8 Related Work
text: In-context learning (ICL) is the standard paradigm for adapting LLMs (Chowdhery et al., 2022; Wei et al., 2022; Zhang et al., 2022). Many recent works focus on understanding its mechanism to improve adaptation performance. For example, Lu et al. (2021) showed that ICL is sensitive to the order of in-context examples. Razeghi et al. (2022) demonstrated that the ICL performance on numerical reasoning tasks is strongly correlated with the pretraining term frequencies. Liu et al. (2021) found that using examples semantically close to the input texts is beneficial. Min et al. (2022) showed that for classification tasks, the input-label pairing format plays the most crucial role in ICL. Sorensen et al. (2022) found that structure of the prompt also significantly affected ICL performance, and that better prompts could be selected based on mutual information between the prompts and the model’s output. Complementary to these works, we comprehensively study the label bias problem in ICL. The existence of domain-label bias indicates that ICL is largely affected by the word-level associations LLMs learn during pre-training.
Other recent works discuss the bias problem in ICL. Zhao et al. (2021) proposed contextual calibration to mitigate three types of biases in ICL: the majority bias, recency bias, and common token bias. Holtzman et al. (2021) focused on the zeroshot setting and found that different surface forms of the answer can compete for probability mass given a question, leading to a bias when predicting with a single label name for each class. In contrast to these works, which consider a specific type of bias, we propose a typology of label biases and propose domain-context calibration that handles all biases in our typology.
The ability of the largest models, like PaLM (Chowdhery et al., 2022), to perform in-context learning under the flipped label or semanticallyunrelated label settings (Wei et al., 2023) is very relevant to this work. As the largest models tend to have emergent abilities, it would be interesting to test how vulnerable these models are to label biases (especially domain-label bias) and how domaincontext calibration would help. Unfortunately, we currently do not have access to them (e.g., PaLM and Flan-PaLM (Chung et al., 2022)). Nevertheless, the similar behavior of PaLM and Instruct-GPT (as shown in Wei et al. (2023)) and the fact that Instruct-GPT also suffers from domain-label bias
(Table 2) indicate that these more capable models may still be susceptible to label biases. Also, how scaling up or instruction-tuning would relieve label biases is an interesting direction to explore.

heading: 9 Conclusion
text: In this work, we define a typology of label biases that affect in-context learning (ICL). We categorize existing findings of label biases into two types: vanilla-label bias and context-label bias, and identify a new type of bias, domain-label bias, that significantly influences ICL performance. To mitigate these label biases, we propose domain-context calibration, which significantly improves ICL performance on a wide range of tasks, particularly on datasets with large domain-label bias. The various levels of domain-label bias in different datasets also suggest that when analyzing ICL, we need to select datasets with diverse types of label biases and report stratified that acknowledge this diversity beyond single aggregated scores.
Limitations
Data and Task Limitation In this work, we analyze domain-label bias and apply our domaincontext calibration to English. We leave analysis and mitigation methods for multilingual tasks to future works. In experiments, we discuss calibration on classification tasks. The effect of domain-label bias could exist differently for open-end tasks like text generation. Our analysis of domain-label bias also emphasizes more on the word-level bias. Other types of biases associated with a domain, such as topics and genders, may also impact model prediction. We leave the diverse analysis to future works. Due to budget limitations, we conduct experiments on a subset of the 24 reported datasets for GPT-3. One can evaluate all 24 datasets to get a complete picture with enough budget.
Model Limitation For large language models, we only focus on the GPT models and only select RoBERTa as the small-scale language model in experiments. Future work could consider expanding to other model types, such as PaLM for large models and DeBERTa for small models. Access to the OpenAI API for GPT-3 is also necessary for parts of our experiments. Future work can consider experimenting with open-source LLMs like the OPT-175B or BLOOM-176B model.
Ethics Statement
Our work focuses on analyzing the general label bias problem of the in-context learning ability of LLMs and improving their performance with a tuning-free method, which involves no large neural model pre-training, re-training, or fine-tuning. As we only use LLMs for inference, developing and applying our approach requires only minimal computational resources compared to methods that require dataset-specific model fine-tuning or engineering. We do not anticipate significant ethical issues introduced by our approach, as we use only off-the-shelf LLMs, and the datasets involved are all publicly available text classification datasets. The discussion of biases in our work is general and not specific to any real word context. Still, our analysis and typology of the label biases of ICL may motivate future work to analyze the bias problem of ICL and LLMs in areas with larger social impacts, such as healthcare or legal scenarios.

heading: A Domain-label Bias of All Datasets
text: We compute and illustrate the domain-label bias of all datasets we used with different LLMs in Fig. 17. Different datasets exhibit different levels of domain-label bias. Regarding task types, The detection tasks (red) show the largest domain-label bias, while the NLI tasks (orange) have the least. On sentiment and topic tasks, the domain-label bias is mostly small but can vary depending on the domain of the dataset. For example, for sentiment classification datasets, movie review datasets like SST-2 have relatively small domain-label bias. While financial statements and poem sentiment, whose texts are from rare domains, have much larger biases. We discuss the model dependency of domain-label bias in the next section.

heading: B Correlation of Domain-label Bias Estimated with Different LLMs
text: We compute the correlation of domain-label bias (defined by eq. (1)) computed with 5 different models on all 24 evaluation datasets. We use GPT-3 Ada (350M), GPT-3 Babbage (1.3B), GPT-3 curie (6.7B), GPT-3 DaVinci (175B), and GPT-J (6B). We show the correlation plot in Figure 13. Although domain-label bias is model-dependent by definition, the biases computed by different models are highly correlated.

heading: C Full Dataset Information
text: We use 24 datasets falling into three categories: sentiment and topic classification, NLI, and Detection. Most of the used datasets are from existing works. W(Min et al., 2022; Lu et al., 2021; Zhao et al., 2021)e added a few more detection datasets for better studying the domain-label bias as they tend to show the largest domain-label bias. We use the HuggingFace venison Lhoest et al. (2021) of all
datasets and use the test set, if available, for evaluation. Otherwise, we use the development set. We summarize the full dataset information in Table 3.

heading: D Full Few-shot Results
text: We report the full 8-shot results on individual datasets with the standard deviations (5 random seeds) in Table 6. We further show the few-shot performance gain (GPT-J) of DC over CC on all datasets in Figure 14.

heading: E Templates and Task Instructions
text: We show the templates and label names for all datasets in Table 4. The task instructions used in Table 2 are illustrated in Table 5. We always use exactly one word for every label name. To avoid label names being tokenized into subwords, we always use lower-cased label names except for tasks answering with True or False.

heading: F Sampling Analysis
text: In this section, we analyze two factors related to the random word sampling process involved in DC: 1) the number of random texts to use for estimating the prior (M in eq. (2)), and 2) the size of the unlabeled dataset to sample random in-domain words from. We conduct experiments on two smalldomain-label-bias datasets (SST-2 and AG News) and two large-domain-label-bias datasets (Tweet hate and Tweet irony).
How many random texts should we sample? First, we sample different numbers of random texts M for estimating the model’s prior as in eq. (2). As
shown in Figure 15, DC is able to achieve a good estimate with a relatively small number of sampling. We choose M = 20 as it achieves a good balance between computational efficiency and stability of prior estimation.
How large should the unlabeled task corpus be? In the main experiments, we use the whole unlabeled test set to construct a bag-of-words and sample random words from it. Here, we study the effect of the unlabeled dataset set size on the performance of DC. As shown in Figure 15, DC is able to achieve a good estimate with 50 unlabeled texts from the dataset.

heading: G Zero-shot Prompting Experiment
text: Templates for zero-shot prompting We adapt templates from Gao et al. (2020) for our zero-shot prompting experiments.
Sentiment and detection tasks:
⟨input⟩ It was ⟨mask⟩]
Subj:
⟨input⟩ This is ⟨mask⟩
Topic tasks:
⟨mask⟩ : ⟨input⟩
NLI tasks:
⟨sentence1⟩ ? ⟨mask⟩ , ⟨sentence2⟩
Full Results We show the full zero-shot prompting results with RoBERTa-large on individual datasets in Table 7.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
At the end of the paper, after the conclusion section.
3 A2. Did you discuss any potential risks of your work? At the end of the paper, after the limitation section.
3 A3. Do the abstract and introduction summarize the paper’s main claims? We summarize our main contributions in the abstract and introduction sections.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? Section 6.
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Section 5.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 5.
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 6 and Appendix D.
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 5 and Appendix C.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We leave analysis and mitigation methods for multilingual tasks to future works. In experiments, we discuss calibration on classification tasks. The effect of domain-label bias could exist differently for open-end tasks like text generation. Our analysis of domain-label bias also emphasizes more on the word-level bias. Other types of biases associated with a domain, such as topics and genders, may also impact model prediction. We leave the diverse analysis to future works. Due to budget limitations, we conduct experiments on a subset of the 24 reported datasets for GPT-3. One can evaluate all 24 datasets to get a complete picture with enough budget. Model Limitation For large language models, we only focus on the GPT models and only select RoBERTa as the small-scale language model in experiments. Future work could consider expanding to other model types, such as PaLM for large models and DeBERTa for small models. Access to the OpenAI API for GPT-3 is also necessary for parts of our experiments. Future work can consider experimenting with open-source LLMs like the OPT-175B or BLOOM-176B model