A Simple Yet Strong Domain-Agnostic De-bias Method for Zero-Shot Sentiment Classification
Year: 2023

Abstract:
Zero-shot prompt-based learning has made much progress in sentiment analysis, and considerable effort has been dedicated to designing high-performing prompt templates. However, two problems exist; First, large language models are often biased to their pre-training data, leading to poor performance in prompt templates that models have rarely seen. Second, in order to adapt to different domains, redesigning prompt templates is usually required, which is time-consuming and inefficient. To remedy both shortcomings, we propose a simple yet strong data construction method to debias a given prompt template, yielding a large performance improvement in sentiment analysis tasks across different domains, pre-trained language models, and prompt templates. Also, we demonstrate the advantage of using domainagnostic generic responses over the in-domain ground-truth data. We release the code here1.

heading: 1 Introduction
text: Over the past few years, zero-shot prompt-based learning has become a de facto standard in many Natural Language Processing (NLP) tasks where training data is unavailable. For sentiment analysis, much effort has also been dedicated to designing effective prompt templates to trigger the capability of Large Language Models (LLMs) such as RoBERTa (Liu et al., 2019) and GPT (Radford et al., 2018) to predict sentiment polarities, e.g., positive or negative. A prompt template typically consists of prompt text and a label token set corresponding to the sentiment class. Gao et al. (2021) demonstrates that It was {good,ok,bad}. is a highperforming prompt template for sentiment analysis task. As Figure 1 shows, the input to LLM is A must-watch movie. It was [MASK]., and the token ok is the most probable word over the label token set.
1https://github.com/repo4nlp/
IBM Confidential 3
0.02 0.003 … 0.01
good okay a of … bad
Pre-trained LM
I really love this movie. It was [MASK].
[MASK]
Probability in softmax layer over vocabulary
[1] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, NAACL 2019
good ok … … bad
Large Language Model (frozen)
A must-watch movie. It was [MASK].
[MASK]
probability distrution over vocab
good -> positive ok -> neutral bad -> negative
probability rescaling, W
Figure 1: Zero-shot prompt-based sentiment classification for a masked language model. The prompt is It was [MASK]. and the label token set is {good,ok,bad} which respectively stands for positive/neutral/negative sentiment polarities.
However, two problems remain. First, an LLM is often biased to its pre-training data, leading to poor performance in prompt templates that the LLM have rarely seen. Second, when it comes to different domains, such as financial and food, re-designing appropriate prompt templates to adapt to new domains is usually required, which is timeconsuming and inefficient. To mitigate the first problem, Zhao et al. (2021) proposed a probability rescaling method to calibrate the probability based on the assumption that LLMs should NOT express either positive or negative sentiment when the input is a meaningless sentence such as N/A.
Motivated by this, we relax their assumption and further hypothesize that a good LLM should be capable of accurately predicting the sentiment of an "absolute" positive, neutral, or negative sentence. For example, the phrase "thank you so much" mostly manifests an "absolutely" positive
3923
sentiment no matter what the context is2. In this spirit, we propose a simple method to construct "absolutely" sentimental instances and use them to learn a probability rescaling layer to de-bias LLM. Particularly, we are interested in the following two research questions; RQ1: how to construct "absolutely" positive/neutral/negative instances for training a probability rescaling layer to improve sentiment classification performance? RQ2: How does the quality of the constructed silver data compare to that of the ground-truth data?
To answer these questions, we employ generic responses from a dialogue corpus and apply a rulebased sentiment tagger to automatically generate sentiment-labeled instances with various sentiment polarities. Subsequently, the labeled generic responses are utilized to learn a rescaling parameter. Experimental result on seven mainstream sentiment analysis datasets shows that the proposed method outperforms baseline approaches by a large improvement across different domains, pre-trained LLMs, and prompt templates.
Our contributions are two-fold: (1) We propose a simple yet strong data construction method to generate sentiment-labeled instances for sentiment analysis task and human assessment validated the relatively high accuracy of these sentiment-labeled instances; (2) The proposed method obtained large performance improvement in zero-shot sentiment classification task across diverse domains, pretrained LLMs, and prompt templates. Also, we demonstrated the advantage of constructed sliver data over in-domain ground-truth data.

heading: 2 Methodology

heading: 2.1 Problem Formulation
text: Formally, we define an input sentence to be classified sentiment polarity as S, a prompt text as P (e.g., It was [MASK]. and a label token set as T (e.g., good,ok,bad), and a verbalizer Z to map label tokens T into a class label set C (e.g., "good" -> "positive"). LM is a pre-trained LLM that outputs the probability distribution over the vocabulary V . Thus, the sentiment prediction of sentence S is
Z(argmax t∈T
LM(S|P )), (1)
As shown in Figure 1, the LLM predicts the
2In the context of irony, "thank you so much" may not always convey a positive sentiment, However, such instances are very much infrequent in the corpus we are dealing with.
sentence as having a neutral sentiment. To adjust the output probability, a common practice is to rescale the probability distribution over vocabulary of LLMs in the softmax layer. Platt et al. (1999) applies an affine transformation to adjust the probability distribution p, p̂ = softmax(Wp+ b), where a weight matrix W and a bias vector b is learnable parameters, while Zhao et al. (2021) follows Guo et al. (2017) to restrict the matrix W to be diagonal and b to be zero to prevent the parameters from growing quadratically in the size of p̂; Then, they used meaningless strings such as "N/A" and an empty string to learn diagonal parameter W to shift the probability distribution, as shown in Figure 1. By following the same fashion, namely, our prediction is
Z(argmax t∈T
W · LM(S|P )), (2)
where W ∈ R|V |×|V | is a diagonal matrix where all elements not on the main diagonal are equal to zero, and |V| is the vocabulary size of an LLM. Different from (Zhao et al., 2021), where they only consider meaningless inputs to learn parameter W , we construct positive, neutral, and negative instances to learn the parameter W . It is worth noting that the LLM’s parameters are frozen during training, and only W is updated.

heading: 2.2 Proposed Method
text: We herein describe a simple procedure to construct domain-agnostic instances to learn diagonal parameter W . Central to this construction is to find instances that are not sensitive to the context change. To this end, we take inspiration from generic responses in the dialogue research and argue that generic responses serve as desirable instances for three reasons: (1) insensitive to context change; (2) relatively domain-agnostic; (3) easy to automatically annotate sentiment polarities. Thus, a two-step approach is proposed:
Step 1: Utterance Selection. We select a dialogue corpus D and extract each utterance to form a set of utterances U . We get frequency fi of each utterance ui in U and sort ui by its frequency fi in the descent order such that f1 ≥ f2 ≥ . . . ≥ fN . Then, since generic responses feature in high frequency in dialogue corpus, we determine a frequency threshold F and select utterances if its frequency fi is no less than
F to form a frequent utterance set S, as a set of generic responses to be sentiment-labeled.
Step 2: Annotation. We employ a pre-defined positive word list Lpos containing 2,006 words and a negative word list Lneg containing 4,783 words from (Hu and Liu, 2004) and automatically tag the sentiment polarity ti of each utterance ui in S in the following way:
1. if ui contains more than one positive word in list Lpos and no negative word in list Lneg: (a) tag ti "positive" if the number of negation words is an even number, or (b) tag ti "negative" if the number of negation words is an odd number.
2. if ui contains no positive word in list Lpos and more than one negative word in list Lneg: (a) tag ti "negative" if the number of negation words is an even number, or (b) tag ti "positive" if the number of negation words is an odd number.
3. if ui contains no positive word in Lpos and no negative word in Lneg , then tag ti "neutral".
We use labeled instances (ui, ti) of each class to train the rescaling parameter W .

heading: 3 Experiment

heading: 3.1 Experimental Details
text: We use Cornell Movie-Dialog Corpus3 (D) (Danescu-Niculescu-Mizil and Lee, 2011) which is under creative commons license and extract 304,713 dialogue utterance (U ). After deduplication and removal of interrogative sentences4, we set frequency threshold (F ) to 3 to obtain 2,211 sentences. After automatic annotation, we finally yielded 274 positive instances, 176 negative instances, and 1,761 neutral instances. We selected 100 instances for each class and in total, 300 sentiment-labeled instances to train the parameter W . Please refer to the Appendix A for examples.
3https://www.kaggle.com/datasets/rajathmc/cornellmoviedialog-corpus
4It is often difficult to determine the sentiment polarity of interrogative sentences such as "is it good?".
Quality Assessment of Automatic Annotation To assess the quality of automatic annotation, we assigned a human annotator5 to rate 300 instances (100 instances per class) by judging whether the instance is positive or not for 100 automatically annotated positive instances. The same process was followed for the negative and neutral classes. The results show that out of the 300 instances evaluated, 43 instances displayed inconsistencies with human judgment, resulting in an automatic annotation accuracy of 0.86.
For training, we split the data into a training set with 240 instances and dev set with 60 instances and select the best model based on the performance of the dev set. Then, we test the best model on all the datasets in Table 2. For the evaluation metric, we use accuracy for datasets whose label class is balanced (SST-2, IMDB, Yelp, and Amazon) and Macro-F1 for the datasets whose label class is unbalanced (Phrasebank, airline, and debate). We used the arithmetic average (Ave.) instead of the weighted average because we view each dataset and its representing domain equally important6. We use one A100 GPU to train the model by setting the batch size to 10, the learning rate to 1e-5, and the number of epochs to 100. It takes half an hour to finish the whole training. The parameters of LLMs are frozen during training.
5The annotator is based in Japan. 6We experimented with weighted average according to the size of each dataset in Table 2 and observed even better performance of the proposed method.

heading: 3.2 Dataset
text: Table 2 shows the dataset statistics. We employ publicly available sentiment analysis datasets in different domains; Kotzias et al. (2015) which contain three sentence-level sentiment datasets respectively from IMDB, yelp, and amazon datasets, SST-2 (Socher et al., 2013), Airline7, Debate8, and Phrasebank (Malo et al., 2014).

heading: 3.3 Prompt Template and Pre-trained LLMs
text: Although many prompt templates exist, we experimented with two high-performing prompt templates proposed by previous works for sentiment analysis tasks, as shown in Table 19. We experimented with three mainstream LLMs, BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), and GPT2-xl (Radford et al., 2019).
7https://www.kaggle.com/datasets/crowdflower/twitterairline-sentiment
8https://www.kaggle.com/datasets/crowdflower/first-gopdebate-twitter-sentiment
9we remove "polarity of <aspect>" in the original template, "The sentiment polarity of <aspect> is [MASK]", for adapting to our non-aspect sentiment classification setting.

heading: 4 Result and Analysis
text: Table 3 presents the result on two prompt templates. We have the following observations: (a) On average, our proposed method (#3, #6, #9, &3, and &6) outperforms other baselines by a large margin on both auto-encoder LLMs (BERT and RoBERTa) and auto-regressive language model (GPT2), validating the effectiveness of the proposed method; (b) Among the three LLMs, the vanilla RoBERTa (#4 and &4) and the RoBERTa enhanced by our method (#6 and &6) achieve the highest performance on both templates. This indicates the effectiveness of data scaling, considering that RoBERTa utilizes 160GB of text data for training, while BERT and GPT2-xl use 16GB and 40GB of text data, respectively.
Interestingly, domain-agnostic generic responses, such as "thank you very much," improve the sentiment classification performances, which touches upon our RQ2: How does the quality of our domain-agnostic silver data compared to the in-domain ground-truth data from each of seven sentiment analysis datasets?
Domain Analysis To answer this question, we conducted a leaveone-out experiment as shown in Table 4. For each dataset (domain), we selected 100 ground-truth instances for each label class. Given its superior performance in Table 3, we employ the RoBERTa model in combination with the prompt template, "<S> The sentiment is {positive,neutral,negative}." Here are our observations: (a) Models trained with in-domain data (e.g., IMDB, Debate, Airline, and Phrasebank) perform the best in their own domains; Nevertheless, on average, (b) our method achieves the best average performance, showing the advantage of using domain-agnostic data over domainspecific data to avoid the model to be twisted too much to any specific domains.

heading: 5 Related Works
text: Zero-shot sentiment classification has attracted a lot of attention as it does not require training data, offering advantages in many real-world use cases, particularly in low-resource scenarios. At the core of this task is the construction of an appropriate prompt, which transforms the sentiment classification task into a fill-in-the-blank format. Gao et al. (2021) explored various prompts for sentiment analysis and identified several highly effective prompt templates. Similarly, Liu et al. (2021) experimented with prompt templates in the context of aspect-based sentiment analysis. However, a challenge comes from the sensitivity of accuracy to prompt templates, where even a slight modification in prompt templates can result in a large performance change in zero-shot sentiment analysis tasks.
Zhao et al. (2021) argues that this "instability" arises from the bias of language models towards predicting certain answers that are common in the
pre-training data. To address this issue, they estimate the model’s bias towards each sentiment polarity class by requesting its prediction when presented with the prompt and a content-free input, such as "N/A." Subsequently, they determine calibration parameters to ensure a uniform probability distribution for this input across all sentiment polarity classes. Later on, Min et al. (2022) borrowed the idea from machine translation research and proposed a noisy channel to shift the probability distribution for few-shot text classification. Our work aligns with the approach proposed by Zhao et al. (2021) but with a notable distinction. Rather than considering the meaningless strings, we go a step further by incorporating positive, neutral and negative sentiment-labeled instances to address the bias in LLMs.

heading: 6 Conclusion and Future
text: In this work, we propose a simple yet effective domain-agnostic data construction method to enhance sentiment classification tasks. Our method was evaluated using three popular LLMs, namely BERT, RoBERTa, and GPT2-xl, along with two commonly employed prompt templates. The results show significant improvements in task performance. Also, we answered the research questions and demonstrated that our constructed domainagnostic data is superior to in-domain data in terms of overall performance. In the future, we plan to experiment with other dialogue corpora to assess the generalization capabilities of the proposed de-bias method.
Limitations
Although we sidestep the challenge of selecting a specific prompt template for experimentation by opting for widely-used templates from previously
published works, it is worth noting that numerous effective prompt templates are available, and the experimental results obtained using these templates would also provide valuable insights into testing our proposed method. Furthermore, while our method yields improvements, it is important to acknowledge that errors may exist in the rule-based automatic annotation of generic responses, which could potentially propagate to the learning of the diagonal parameter W .
Ethics Statement
While our work primarily centers on mitigating bias in Large Language Models (LLMs) using prompt templates, it also acknowledges the inherent risk shared by BERT, RoBERTa, and GPT models, as they may potentially generate biased language.

heading: Acknowledgements
text: We would like to thank the anonymous reviewers for their valuable comments. We also thank our colleagues, Issei Yoshida, Hiroshi Kanayama, and Akihiro Nakayama from their helpful suggestions and discussions.

heading: A Appendix
text: ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
The Limitation section is right after the Conclusion section.
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Our paper’s main claims (contribution) are in the last paragraph of the Introduction section.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? This work created a dataset (section 2.2 and section 3.1)
3 B1. Did you cite the creators of artifacts you used? section 3.1
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? section 3.1
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? section 3.1
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? please refer to the caption of the appendix A: Most sentences in our dataset do not contain personal information like people’s names. However, some sentences contain fake characters’ names in the movie.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? abstract section
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. section 3.1
C 3 Did you run computational experiments? section 3.1 and section 4.
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? section 3.1
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? section 3.1. we select the best model based on the performance on the dev set.
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? section 4. For the zero-shot evaluation, everything is deterministic so there are no random factors.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? section 3.1
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Section 3.1
7 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? no payment
3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? From the instruction in section 3.1, the human annotator understands how the data would be used to report the accuracy of machine-produced data.
3 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Determined exempt. The human annotation is just a binary label 1/0 and this labelled data is not used as training data for any models
3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? section 3.1

heading: Future Work
text: In the future, we plan to experiment with other dialogue corpora to assess the generalization capabilities of the proposed de-bias method. Limitations
Although we sidestep the challenge of selecting a specific prompt template for experimentation by opting for widely-used templates from previously
published works, it is worth noting that numerous effective prompt templates are available, and the experimental results obtained using these templates would also provide valuable insights into testing our proposed method. Furthermore, while our method yields improvements, it is important to acknowledge that errors may exist in the rule-based automatic annotation of generic responses, which could potentially propagate to the learning of the diagonal parameter W