Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models (2024)

Abstract: Text classification is a crucial task encountered frequently in practical scenarios, yet it is still under-explored in the era of large language models (LLMs). This study shows that LLMs are vulnerable to changes in the number and arrangement of options in text classification. Our extensive empirical analyses reveal that the key bottleneck arises from ambiguous decision boundaries and inherent biases towards specific tokens and positions. To mitigate these issues, we make the first attempt and propose a novel two-stage classification framework for LLMs. Our approach is grounded in the empirical observation that pairwise comparisons can effectively alleviate boundary ambiguity and inherent bias. Specifically, we begin with a self-reduction technique to efficiently narrow down numerous options, which contributes to reduced decision space and a faster comparison process. Subsequently, pairwise contrastive comparisons are employed in a chain-of-thought manner to draw out nuances and distinguish confusable options, thus refining the ambiguous decision boundary. Extensive experiments on four datasets (Banking77, HWU64, LIU54, and Clinic150) verify the effectiveness of our framework. Furthermore, benefitting from our framework, various LLMs can achieve consistent improvements. Our code and data are available in https://github.com/Chuge0335/PC-CoT.

heading: 1 Introduction
text: Text classification, as one of the most fundamental and common tasks in natural language processing (NLP), has a wide spectrum of applications (Qu et al., 2019; Lei et al., 2023b; Chen et al., 2022; Zaremba and Demir, 2023; Labonne and Moran,
∗ Equal contribution. † Corresponding authors: Wei Wei and Dangyang Chen.
2023). A common approach for text classification involves supervised fine-tuning on a specific dataset, which can be costly and may suffer from generalization issues (Sung et al., 2023; Bohra et al., 2023; Jiang et al., 2023). Recent Large Language Models (LLMs), such as ChatGPT (OpenAI, 2022), LLaMA2 (Touvron et al., 2023) and Qwen (Bai et al., 2023), have consistently exhibited remarkable language comprehension capabilities and performance emergence (Wei et al., 2022a). Such advancement has introduced a new paradigm of classification without the need for finetuning. LLMs can now make selections in a zero-shot manner, or employ techniques like few-shot demonstrations (Brown et al., 2020) and chain-of-thought (CoT) (Kojima et al., 2022).
Unfortunately, we observe that LLMs exhibit vulnerability to changes in the number and arrangement of options in text classification tasks. Our extensive empirical evaluation, involving 11 LLMs
7841
ranging from 1B to 70B parameters and classification tasks with over 50 options, demonstrates that boundary ambiguity and inherent bias are prevalent issues in various large language models. These issues cannot be effectively mitigated by longcontext capabilities, simple prompting strategies, or single increasing the model size. LLMs struggle with distinguishing among a growing number of options, a phenomenon called boundary ambiguity. For example, gpt-3.5-turbo achieves a zero-shot accuracy of only 32.51% when choosing the correct option from 60 categories, compared to 94.29% for two categories. LLMs also exhibit inherent bias, favoring specific positions or selectively retaining or discarding tokens in classification tasks. Placing the correct option consistently in the first position results in performance fluctuations of 35.29% for LLaMA2-70B-Chat and 38.52% for Qwen-72BChat. Furthermore, LLaMA2 shows an abnormal bias towards selecting “movies” and “convert” in the LIU54 benchmark.
To address these challenges for LLMs, our intuition is that LLMs often have the highest accuracy in pairwise classification, and focusing on two options at a time can significantly reduce the number of tokens and positions considered, potentially alleviating bias issues. This divide-andconquer approach also mirrors human decisionmaking with multiple options, where we first eliminate less likely options and then carefully differentiate between similar ones. The latter process often involves juxtaposing confusable objects pairwisely to assess their similarities and differences.
In view of this, we design a general two-stage LLM classification framework consisting of reduction and comparison stages. Within such a framework, we introduce two novel self-reduction techniques for the reduction process: one iteratively isolates the most probable option from candidates, and the other induces distinct option “windows” via clustering, operating within these windows to avoid similarity confusion, termed as ITR and CBWR. Additionally, we propose PC-CoT, an innovative contrastive chain-of-thought technique, to draw out nuances and distinguish features among confusable options, refining the ambiguous decision boundary. This type of detailed comparison encourages LLMs to conduct more in-depth analyses of actual content, rather than making shortcuts based solely on position or tokens. We conduct extensive experiments to demonstrate that our framework is con-
sistently effective for various LLMs to combat ambiguity and bias, even when the number of options increases to 150. Notably, our method achieves a relative improvement of 54.1% in average accuracy compared to the full-option zero-shot performance of gpt-3.5-turbo. It significantly reduces the token bias of LLaMA-70B-Chat, improving the relative performance by 36.88%.
To sum up, our contributions are as follows: (1) Our work provides extensive empirical analyses (covering 11 LLMs) to explore the vulnerability of LLMs in option number and arrangement in text classification. We identify that their ambiguous decision boundaries and inherent biases on tokens and positions remain a main challenge. (2) To address these issues, we propose utilizing pairwise comparison to combat such ambiguity and bias and develop an innovative two-stage classification framework for LLMs. Within the framework, novel self-reduction algorithms and a contrastive elimination strategy are introduced. (3) We carry out comprehensive experiments on four primary benchmarks along with diverse LLMs, demonstrating the effectiveness of our framework.

heading: 2 Preliminary Study
text: Setup We evaluate various LLMs to understand factors influencing classification accuracy. Our experiments include the gpt-3.5 series, LLaMA2Chat, and Qwen-Chat models. We use the challenging portions of the Banking77 and LIU54 datasets (details in Section 4) to reduce experimental costs. Due to space constraints, the main results are summarized in the main text, with detailed dataset samples and additional results provided in Appendix A.
Key Observations Our findings, presented in Figure 2 and Figure 3, reveal the following key observations and insights:
0 50
100 gpt-3.5-turbo (4k) gpt-3.5-turbo-1106 (16k) LLaMA-13B-Chat (4k) LongAlpaca-13B (32k)
0 20 40 60 0
50
100
0 20 40 60 0 20 40 60 0 20 40 60
0- sh ot 3- sh ot
Option Number
(a) LLMs classification accuracy in relation to the number of options consistently demonstrates a decline.
50
0
50
LongAlpaca-13B LLaMA-13B-Chat LLaMA-70B-Chat Qwen-72B-Chat
0 20 40 60
50
0
50
0 20 40 60 0 20 40 60 0 20 40 60
0- sh ot 3- sh ot
Golden Answer Position
(b) The relative performance change rate for fixed correct options in a position (range from 0 to 60) compared to random configuration performance. Simple option arrangement leads to dramatic performance fluctuations in both zero-shot and fewshot scenarios.
mu sic ord er qu eryrad io sto ck cof fee ev en ts
mo vie
s ne ga te qu irk y rec ipe rem ov e rep ea t
co nv
er t fac toi d tra ffic
cle an
ing like ne ss po dca sts set tin gs
Predicted Class
set game post greet music order query coffee movies quirky remove factoid traffic likeness settings wemo_off audiobook locations sendemail addcontact Tr ue C la ss
2 6 1 2 1 1 2 2
2 6 8 2
1 3 19 23 1 1
1 2 6 20 1 13 1 3 1 5 5 1
1 1 2 2 3 1 1 3 2 1 2 2 2 2 1 2 1
(c) The confusion matrix sheds light on the token bias of LLaMA-2 in the LIU54 dataset. Notably, the examination reveals a pronounced bias of LLaMA-2-70B-Chat towards the tokens “movies” and “convert”.
Figure 3: The influence of option number and arrangement on LLM performance. Full results are in the Appendix A.
(1) The option similarity consistently poses a challenge. We test randomly sampled or similarityretrieved options in Figure 2 and observe a consistent performance drop across various LLM sizes and types. This finding indicates that LLMs encounter difficulties when faced with the inherent fuzziness of semantics.
(2) The increased number of options consistently presents a challenge for LLMs. Across various LLMs, including LLaMA, Qwen, and gpt3.5-turbo, we consistently observe a decline in performance as the number of options increases, as shown in Figure 3a. In addition, this challenge persists when a longer context window is introduced, e.g., the gpt-3.5-turbo-1106 with 16k context length and LongAlpaca with 32k context length. This phenomenon suggests that the noise stemming from the proliferation of options remains a significant obstacle in the landscape of LLMs.
(3) Bias towards option positions and tokens. In Figure 3b, we discern a significant bias caused by the position of the correct choice, especially in open-source models such as LLaMA and Qwen, resulting in performance fluctuations exceeding 50%. In contrast, gpt-3.5-turbo demonstrates minimal effects, with fluctuations below 10%. This lack of robustness demonstrates inherent position bias in LLMs. In Figure 3c, we analyze the confusion matrix of LLaMA for the LIU54 benchmark. A preference for “movies” and “convert” tokens can be clearly observed regardless of the golden label,
which indicates the existence of token bias. (4) The above issues cannot be effectively mitigated by long-context capabilities, simple prompting strategies, or single increasing the model size. Even with a longer context, issues persist regarding the number and placement of options. For example, the gpt-3.5-turbo-1106 (16k context length) exhibits similar declining trends as its 4k-context counterpart with increasing option numbers. The LongAlpaca-32k has 50% performance fluctuation when the golden option is fixed at the 50th position. Furthermore, experiments with varying numbers of options show consistent trends in both few-shot and zero-shot scenarios. There are distinct bias distributions between 3-shot and zeroshot scenarios, indicating that in-context examples can alter the bias distributions of option positions but cannot effectively eliminate them. Regarding the model size, we observe that LLMs of different sizes all suffer from option similarity, as shown in Figure 2. Even 70B LLaMA and Qwen models suffer from an increase in option number and simple arrangement, as depicted in Figure 3.
From Full Option to Pairwise Comparison Inspired by Figure 3a, selecting the correct label from two choices is much easier for LLMs than directly selecting from all options. For instance, gpt-3.5turbo achieves a 94.29% zero-shot accuracy when choosing between two categories. To this end, we propose a pairwise comparison framework to max-
imize the capabilities of LLMs. Our approach focuses on comparing pairs of options, which reduces the range of positions involved in the classification process. This reduction has been shown to effectively mitigate position bias (Pezeshkpour and Hruschka, 2023). Furthermore, using pairwise comparisons allows for more nuanced comparative strategies, avoiding token shortcuts and ensuring that the most distinguish content guides decisionmaking.

heading: 3 Methodology

heading: 3.1 Formulation
text: Formally, given a fixed LLM parameterized by θ, we use x to denote the text classification input, Y = {y1, y2, · · · , yN} as N potential label options. Then, we utilize the prompt to instruct LLM to select the appropriate label y ∈ Y for input x:
y ∼ pθ(y|Prompt(x,Y)) (1)
For simplicity, we will subsequently refer to pθ(y|Prompt(x,Y)) as pPromptθ (y|x,Y)
The expressions for in-context learning (pICLθ ) and chain-of-thought methodology (pCoTθ ) for tackling text classification problems are formulated as:
pICLθ (y | Sicl, x,Y), pCoTθ (z, y | Scot, x,Y)
(2)
where the set S denotes the demonstration sets, with an empty size (S = ∅) in the zero-shot scenario and a non-empty size in the few-shot scenario. For the few-shot instances, Sicl includes pairs from {x′, y′ | y′ ∈ Y, x′ ̸= x}. For CoT methods, Scot additionally include texts z′, viewed as the step-bystep reasoning “thoughts”. These thoughts serve as instructional support for LLMs to induce the conditional distribution of y′ based on the given x′.

heading: 3.2 Proposed Framework
text: The fundamental concept of our approach is to compare options in a pairwise manner. In cases where the options set is substantial, directly contrasting original options in pairs becomes inefficient and costly. Therefore, we propose a two-stage framework: (i) Reduction, strategically eliminating extraneous options Y to a reduced set R ⊂ Y . (ii) Comparison, navigating through the interference options within R via pairwise comparison in a chain-of-thought fashion. In our two-stage framework, various reduction and comparison algorithms can be seamlessly integrated. This paper further introduces two new algorithms designed for this framework: self-reduction (in Section 3.3), which utilizes LLMs to filter out irrelevant options automatically, and the contrastive chain-of-thought technique (in Section 3.4), which mimics how humans compare confusable objects. The overall ar-
chitecture of our algorithm is illustrated in Figure 4.

heading: 3.3 Self Reduction
text: Given a sentence x and option sets Y , LLMs iteratively refine the options to R through a multiiteration strategy, i.e., pθ (R | x,Y). Specifically, we design two types of multi-iteration strategies to improve the elimination accuracy: (1) Iterative Top Reduction (ITR): where LLMs iteratively choose the most likely options from the given candidate set and subsequently isolate them in the following iteration. This cyclic process continues until a pre-set threshold is reached, facilitating the steady refinement of labels. (2) Cluster-Based Window Reduction (CBWR): Inspired from Figure 2, a high degree of option similarity may degrade selection precision. To mitigate this, CBWR refines the full set of options for an input sentence through iterative clustering and selection within a window. The remaining options are clustered into K clusters, with samples evenly sampled from each cluster and placed into a window. Top-N candidates are then selected from the window by LLM, progressively discarding less probable options. The detailed procedure can be found in Algorithm 2 in Appendix B.

heading: 3.4 Contrastive CoT Comparison
text: In this phase, the remaining candidates within the set R often exhibit significant ambiguity. Each option has a close relationship with input x from the perspective of LLMs. To accurately identify the candidate, we propose a novel strategy based on a “similar-difference-decide” reasoning chain during the pairwise comparison process, which we termed PC-CoT. This approach is inspired by the concept of contrastive learning.
As specified in Algorithm 1, our approach begins by initiating a candidate pool with the reduced label set, systematically selecting two labels in each iteration, denoted as y1, y2. Subsequently, the demonstration set expands to include the demonstrations of both labels, i.e., S = Sy1 ∪ Sy2 . The LLMs are then prompted to analyze the similarity (zs) and differences (zd) of demonstration sentences associated with these labels. This step aims for a better understanding of the relationship between the two candidates and the refinement of the final decision on which label to eliminate. This iterative removal of candidates continues until we arrive at a sole label, which is regarded as the final answer. The algorithm exhibits a time complexity of O(|R|).
Given that the size of R is typically small, our approach ensures a cost-efficient performance.
Algorithm 1 PC-CoT Require: Input x, reduced label options R = {y1, y2, · · · , yr}, language model pθ , training set D, similar analysis prompt Sim, difference analysis prompt Diff and decision prompt Dec.
1: while |R| > 1 do 2: Select two label options {y1, y2} fromR 3: R = R\{y1, y2} 4: Sample demonstrations pairs {x, y} ∈ D for {y1, y2} as Sy1 , Sy2 5: S ← Sy1 ∪ Sy2 6: Sample zs ∼ pSimθ (z | x, S, {y1, y2}) 7: Sample zd ∼ pDiffθ (z | zs, x, S, {y1, y2}) 8: Predict yp ∼ pDecθ (y | zs, zd, x, S, {y1, y2}) 9: R = R∪ {yp}
10: end while 11: return R

heading: 4 Experiments
text: To evaluate the performance of our framework, we choose four widely recognized classification datasets, including: Banking77 (Casanueva et al., 2020) with 77 labels in the banking services domain; HWU64 (Casanueva et al., 2020) of 64 intents from 21 different domains; LIU54 (Liu et al., 2021) of 54 classes gathered from Amazon Mechanical Turk; Clinc150 (Larson et al., 2019) covering 150 classes and 10 domains. Notice that we sample challenging segments from these datasets to reduce the experiment cost. The comprehensive sampling process and supervised performance are
detailed in Appendix A.2. Our experiments primarily utilized gpt-3.5-turbo alongside open-source models like LLaMA2-Chat and Qwen-Chat series. Here we use the instruction versions of models for better instruction following ability. We set the temperature to 0 for all models across different experiments.

heading: 4.1 Evaluation For the Reduction Stage
text: Setups. For evaluating the performance of the reduction stage and our self-reduction algorithm, we configure the size of the reduced set as |R| = 5 and report the HIT@5 metric on the benchmark. This metric measures the proportion of instances where the correct label is among the top 5 labels chosen by the label reducer. We compare the effectiveness of our two techniques, ITR and CBWR, against the following baselines: (1) standard prompt, manually prompts the LLM to select the top 5 labels in a single call. (2) self-consistency (Wang et al., 2023), which invokes the aforementioned prompt multiple times, followed by a voting process to determine the final results. Appendix B provides additional details about their prompts.
Results. Table 1 provides a detailed comparison of HIT@5 performance. It is obvious that both self-reduction strategies ITR and CBWR consistently outperform the standard prompt and the selfconsistency method by a relatively large margin. The self-consistency approach also exhibits an advantage over the standard prompt for gpt-3.5-turbo
and Qwen-72B. However, it may degrade the performance of the LLaMA2-70B model, which could result from the differences in training data. Another observation is significantly low reduction performance occurs on the LIU54 dataset. This could be attributed to a large proportion of short sentences in LIU54, making them challenging to interpret and reason effectively, more details are listed in Appendix E. Despite the above challenges, our strategies can consistently improve the reduction performance, minimizing error propagation to the next stage.

heading: 4.2 Evaluation For Overall Framework
text: Setups We primarily use accuracy as the performance metric for our framework, considering both the reduction and pairwise comparison stages, The selected baselines include common prompting techniques: zero-shot (ZS), zero-shot-CoT (ZS-CoT), few-shot (FS), and few-shot-CoT (FS-CoT). We evaluate each method in two scenarios: with and without the reduction stage, to study the impact of the overall framework and compare the pairwise comparison algorithm. In the former scenario, we prompt the LLM directly with the full set of options, while in the latter, we conduct comparison experiments based on the CBWR result for gpt-3.5turbo and Qwen, and the ITR result for LLaMA2. It is important to note that for the latter scenario, we compare four prompting methods and our proposed PC-CoT using the same reduction technique, to en-
sure a fair evaluation for the comparison stage. For the few-shot method, if the text exceeds the context length of 4096, we randomly omit demonstrations to reduce the context length. To avoid random errors, we conduct the experiments on each dataset five times and present the results in terms of mean and standard variance. Appendix C provides further details regarding the prompts.
Results As shown in Table 2, we have the following observations:
(1) Our framework significantly improves the stability and predictability of option selection in ZS, ZS-CoT, FS, and FS-CoT settings. Specifically, it boosts the zero-shot performance of gpt-3.5-turbo relatively by 11.88%, the few-shot performance of LLaMA2 from 26.45% to 54.01% (see Section 4.4 for detail analysis), and the FS-CoT performance of Qwen is relatively improved by 10.95%. These results demonstrate the effectiveness of our framework in mitigating ambiguity and bias.
(2) Our PC-CoT paired with our framework stands out with the best results for gpt-3.5-turbo and Qwen. Notably, it achieves a relative increase of 54.1% in average accuracy over a directly selected answer for gpt-3.5-turbo, from 40.24% to 62.01%. However, for LLaMA2-70B, our performance is slightly lower than the FS method on the Banking77 and Clinc150 benchmarks. We hypothesize this may have some relationship with LLaMA2 model training data, which makes it insensitive to the chain-of-thought guidance related to classification problems because we can observe that FS-CoT demonstrates a degradation in performance, dropping from 57.55% under FS to 39.18%.
(3) Compared to original CoT techniques, our PC-CoT demonstrates better generality across different types and sizes of LLMs. While the FSCoT method exhibits satisfactory performance with the Qwen model, it deteriorates the performance of LLaMA and even hinders the zero-shot performance of gpt-3.5-turbo on Clinc150 and LIU54. This result suggests that the prompting method in classification scenarios requires tailored adjustments to effectively prioritize comparison for ambiguous boundaries.
The detailed cases of all nine prompting methods in Table 2 can be found in Appendix F.

heading: 4.3 Ablation Study of PC-CoT algorithm
text: As outlined in Table 3, we demonstrate the impact of removing key components from our PC-
Dataset PC-CoT w/o S w/o D w/o SD
CoT. Without the similarity or difference analysis prompt, we observe an evident performance decline (approximately 15%). Furthermore, using only the difference analysis prompt leads to inferior results, supporting our belief that we need to discard the confusable points to better extract the distinctive features effectively. When both similarity and difference analyses are removed, our method reverts to plain FS-CoT, thus exhibiting a performance level similar to FS-CoT.

heading: 4.4 Debias Study
text: We find that our framework significantly improves the ZS-CoT and FS performance of LLaMA2 in Table 2. This enhancement is particularly evident in the reduced bias effect of certain tokens, such as “movies” and “convert” in the LIU54 dataset, and “iot/hue_lightchange” and “recommendation/movies” in the HWU64 dataset. By visualizing the confusion matrices for both datasets (Figure 5), we can observe a noticeable shift in preferences after applying our framework, demonstrating its effectiveness. This improvement can be attributed to our reduction technique and pairwise comparison approach, which reduce the number of tokens and positions considered, thereby mitigating noise interference and enhancing the capabilities of few-shot demonstration and CoT reasoning.

heading: 4.5 Efficiency Analysis
text: Our method shows promising results but may require more LLM calls due to multiple iterations for reduction and pairwise comparisons to enhance accuracy and reduce bias. To assess the applicability of our approach, we compare our inference speed and call frequency with other methods in Table 4. Note that all baseline methods require multiple calls to LLMs, except for partial methods using the full option. However, the full option
struggles with boundary ambiguity and inherent biases, leading to low accuracy and practical deployment difficulties. Our approach significantly outperforms other baselines by adding only a limited number of calls and achieves the highest performance gain (as shown in the last line of Table 4). In contrast, self-consistency-based methods require more calls than our approach but offer limited performance improvements. With the assistance of technologies such as flash-attn and vLLM, we have successfully reduced the computational overhead of our method when calling the 70B model. As a result, we can accurately predict 1,000 pieces of data in just half an hour, which is significantly
Method reminder agnews bbcnews restaurant
faster than the multiple-call methods used in SelfConsistency. Moreover, the future of technology is headed towards large-scale models. It is foreseeable that with technological advancements, the costs associated with API calls and model computations will decrease further. Our method, with its enhancements addressing bias and boundary issues, will become increasingly relevant and significant in solving real-world classification problems. Therefore, we believe that the scaling trend of language models will elevate the importance of our method in the future.

heading: 4.6 In Context Learning
text: We experiment with a similar in-context learning baseline (Milios et al., 2023) that also uses clustering algorithms in a few-shot setting. As shown in Table 5, our approach consistently outperforms the baseline. The main reason is that clustering similar demonstrations can blur the model’s decision boundary when options are highly confusable, reducing performance. In contrast, our method introduces diversity through ITR/CBWR and provides a more detailed comparison via PC-CoT.

heading: 4.7 Simple Text Classification Tasks
text: Our framework is appliable to simple text classification tasks and has comparable performance to the few shot chain-of-thought method, as shown in Table 6. In our primary experiments, we focused on classification tasks with over 50 categories to demonstrate our framework’s ability to handle unclear decision boundaries arising from numerous options and similar, easily confused elements.

heading: 4.8 Lightweight Models
text: As shown in Table 7, our method still applies to smaller models. While the 14B model outperforms other baselines, the Qwen 1.8B model slightly underperforms compared to the Few-Shot CoT methods. Because smaller models (<10B) may struggle
with tasks such as instruction following and generating high-quality few-shot examples, as highlighted in prior research (Fu et al., 2023).

heading: 5 Related Work
text: Text Classification Traditional text classification approaches typically involve feature extraction by RNN (Xie et al., 2020), GCN (Lin et al., 2021), or Transformers (Chai et al., 2020). These features are subsequently input into a classifier head and assigned the corresponding label. To deal with the common data scarcity issue, the metalearning paradigm emerges to identify novel categories with minimal labeled examples by leveraging prior knowledge acquired from known categories(Chen et al., 2022; Gao et al., 2021; Jiang et al., 2023; Liu et al., 2023; Sung et al., 2023). Nonetheless, these approaches require supervised fine-tuning and exhibit limited applicability. The powerful capabilities of modern LLMs enable zeroshot or few-shot classification without necessitating model update (Fei et al., 2023; Loukas et al., 2023). Despite these advancements, the exploration of LLM-based classification methodologies in addressing practical challenges, such as coping with ambiguous boundaries and inherent bias, remains underexplored.
Chain of Thought Wei et al. (2022b) introduces few-shot chain-of-thought, enriching LLM reasoning capabilities by incorporating intermediate reasoning steps in each demonstration. Kojima et al. (2022) highlights the efficacy of a straightforward prompt, “Let’s think step by step”, in enhancing zero-shot reasoning. While recent investigations have explored diversifying demonstrations (Zhang et al., 2022) or modeling the reasoning process using tree and graph structures (Yao et al., 2023; Lei et al., 2023a), the realm of classification problems continues to lack specialized CoT methods to com-
bat ambiguity and bias. THOR (Fei et al., 2023) exclusively addresses sentiment analysis, sequentially inducing aspects and opinions for sentiment identification. CARP (Sun et al., 2023) focuses on simple classification problems with only 2-4 options and utilizes supervised fine-tuning of the model, which is impractical in more complex classification scenarios. In contrast, we focus on handling a large number of label options and subtle semantic differences. We also address the migration of preferences for tokens and positions, enabling seamless integration into a wide range of text classification tasks.

heading: 6 Conclusions
text: In this paper, we investigate the vulnerability of LLMs in terms of option quantity and arrangement for text classification tasks. Our empirical analysis reveals that the main challenges lie in their ambiguous decision boundaries and inherent biases towards tokens and positions. To mitigate these challenges, we propose a novel two-stage classification framework for LLMs, involving reduction and comparison stages. This framework strategically employs pairwise comparisons to address ambiguity and bias. Additionally, we introduce innovative self-reduction algorithms, including ITR and CBWR strategies, and our PC-CoT, a contrastive chain-of-thought technique, to enhance the framework’s utility. Extensive experiments demonstrate our framework can bring improvement to the stability and debiasing for various prompting techniques. Our ITR and CBWR methods prove highly effective for reduction, while our PC-CoT exhibits superior performance in classification accuracy.
Limitations
Though our method significantly outperforms other baselines by adding only a limited number of calls
and achieves the highest performance gain, it does increase the number of LLM calls. For some smaller-scale LLMs, their limited reasoning ability can impede precise reduction and comparisons, potentially diminishing the effectiveness of our method.
Ethics Statement
In this study, the classification datasets and LLM responses used in our experiments only serve the specific purposes of evaluating our proposed approach. To address ethical considerations, we have incorporated a rule-based filter during our challenge set sampling process to exclude offensive and sensitive information. Our study investigates position bias and token bias during the classification process. Our algorithms help LLMs focus more on the actual content of options rather than making shortcut predictions based on preferences for specific tokens and positions, which is crucial for debiasing LLMs.

heading: Acknowledgements
text: This work was supported in part by the National Natural Science Foundation of China under Grant No. 62276110, No. 62172039 and in part by the fund of Joint Laboratory of HUST and Pingan Property Casualty Research (HPL). There are computational tasks are conducted using the HPC platform of Huazhong University of Science and Technology. The authors would also like to thank the anonymous reviewers for their comments on improving the quality of this paper.

heading: A More Detail of Preliminary Study And Dataset
text: A.1 Full result for Figure 3 In Figure 6a, we observe that across various LLM sizes and types, including LLaMA, Qwen, and gpt-3.5-turbo, we consistently observe a decline in performance as the number of options increases. Different LLMs may exhibit varying degrees of performance downgradations in response to the increased number of options, with the gpt-3.5-turbo series demonstrating heightened stability. The challenge persists even with longer contextual LLMs, though they may show a more gradual performance decline. This observation suggests that the noise stemming from the proliferation of options remains a significant obstacle in the landscape of LLMs.
In Figure 6b, we discern a significant bias caused by the position of the correct choice, especially evident in open-source models such as LLaMA and Qwen, resulting in performance fluctuations exceeding 50%. In contrast, gpt-3.5-turbo demonstrates minimal effects, with fluctuations below 10%, highlighting its superiority over open-source counterparts. This lack of robustness can be attributed to the inherent position bias inherent in LLMs, as elucidated by Pezeshkpour and Hruschka (2023). The bias in option positions becomes particularly apparent when the span of option choices is extensive, as the prolonged context amplifies the imbalance in position, presenting challenges for maintaining stable performance in classification problems.
In Figure 7, we show the full confusion matrix from Figure 3c. A preference for “movies” and “convert” tokens can be clearly observed regardless of the golden label. Similar token biases are also observed for “iot/hue_lightchange” and “recommendation/movies” for LLaMA on the HWU64 benchmark, or neglect the “music”, “query”, and “quirky“ for the Qwen model.
The long-context ability of LLM is insufficient in managing an increasing number of options or their respective positions. Despite the common intuition that extended context should aid generative models in handling larger label sets, we find that performance diminishes with a growing number of labels, coupled with a persistent bias towards specific option positions. This is exemplified by comparisons such as gpt-3.5-turbo-1106 (16k context length) versus gpt-3.5-turbo-0613 (4k context length) and LongAlpaca (32k context length) ver-
sus LLaMA-13B (4k context length). Although a longer context does impart stability and smoothness in the face of increased options, challenges persist concerning both the number and positioning of options. the LongAlpaca-32k also suffers from 50% performance fluctuation when changing the golden option to 50th position.
The In-context examples may reduce option bias for large models while altering the distribution. Examining in-context examples, we observe a reduction in bias for larger models like LLaMA-2-70B and Qwen-72B when moving from zero-shot to 3-shot demonstrations, while this effect is less pronounced for smaller models. Additionally, we note distinct bias distributions between 3-shot and zeroshot scenarios, suggesting that in-context examples may alter options position bias distributions. In contrast, for experiments involving varying numbers of options, the trends observed in few-shot and zero-shot scenarios remain essentially consistent.
A.2 Challenge Set Sampling
To align with real-world application scenarios, we employed a sampling strategy for a more challenging subset from these datasets (only their original test splits) by the precision margin (Li et al., 2023), M(f(x)) := max(0,maxyfy(x) − maxk ̸=yfk(x), which is an indicator of the confidence of the base classifier f (the supervised finetuned BERT model). i.e., the difference between the largest and second-largest label probability for every data item in the test dataset. A smaller prediction margin correlates with decreased confidence in the prediction, representing a more challenging setting. As for the training dataset, we randomly sample five items from their training splits as our few-shot demonstrations.
A.3 Supervised Finetuning Performance on the challenge set
To better illustrate how challenging our curated dataset is, we select a series of encoder-only models and decoder-only LLMs for supervised fine-tuning (SFT) in a few-shot setting. For the encoder-only series, we fine-tune them for 30 epochs. For the decoder-only series, due to the huge SFT cost, we only select TinyLLaMA-1.1B and Qwen-1.8B. We employ an alpaca format for fine-tuning (with the following prompt). The details of the results are listed in Table 8.
0 50
100 gpt-0613 (4k) gpt-1106 (16k) L-13B (4k) LA-13B (32k) L-70B Q-14B Q-72B
0 50 0
50
100
0 50 0 50 0 50 0 50 0 50 0 50
0- sh ot 3- sh ot
Option Number
(a) LLMs classification accuracy concerning the number of options consistently demonstrates a decline.
50 0
50
LA-13B L-13B L-70B Q-14B Q-72B gpt-0613 gpt-1106
0 50
50 0
50
0 50 0 50 0 50 0 50 0 50 0 50
0- sh ot 3- sh ot
Golden Answer Position
(b) The change rate of classification accuracy after moving the correct option to a fixed position compared to the original accuracy. This simple option arrangement can lead to dramatic performance fluctuations in both zero-shot and few-shot scenarios.
FineTuning Prompt Below is an instruction that describes a
task. Write a response that appropriately
completes the request.
### Instruction:
Given the sentence: "{text}", please select
the most possible topic from the following
OPTIONS: {options}.
### Response:
Table 8 indicates that the challenge datasets chosen pose significant difficulties for both BERT and RoBERTa, even when utilizing their larger versions
and fine-tuning them for 30 epochs. With only five demonstrations, BERT achieves a mere 12.02% accuracy, while RoBERTa fares slightly better at 23.70%. In contrast, decoder-only language models, particularly Qwen-1.8B, demonstrate better performance than BERT and RoBERTa. However, this superior performance is contingent upon resource-intensive fine-tuning processes and may suffer from limited generalizability, thereby constraining their practical applicability.

heading: B Detail of the Reduction Stage
text: Standard Reduction Prompt Consider the sentence: "{text}"
Please select {top_k}most possible topic
from following OPTIONS: {options}.
CHOICE:
The self-consistency, our ITR, and CBWR techniques employ the same reduction prompt as exemplified above. The specific procedural steps for CBWR are delineated in Algorithm 2.
Algorithm 2 Self-Reduction (CBWR) Require: Input sentence x, full options set Y = {y1, y2, · · · , yN}, LLM pθ , selection prompt Sel, selection number N , step limit T
1: Initialize the selection set S = Y 2: for t = 1 to T do 3: Cluster the option set S into K clusters 4: W → {} 5: for each cluster i from 1 to K do 6: Randomly select options from cluster i 7: add them into the window W 8: end for 9: Select candidate set C from W :
C ← Top-N ( {y | pSelθ (y | x,W )} )
10: Determine the discard option set D = W \ C 11: Update the remaining candidates S ← S \ D 12: end for 13: return the remaining selection set S

heading: C Detail of the Comparison Stage
text: C.1 Baseline Using Full Options
In this section, we list the prompt used in baselines without label reduction, i.e., the LLM is prompted to directly choose the correct label from the complete set of options.
Zero-Shot Prompt Given the sentence: "{text}"
Please select the most possible topic from
the following OPTIONS: {options}
CHOICE:
Zero-Shot-CoT Prompt Given the sentence: "{text}"
Please select the most possible topic from
the following OPTIONS: {options}
Let's think step by step and give your
explanation to verify your answer:
Few-Shot Prompt Below is a text classification problem,
Note that you can only select the label in
{options}
SENTENCE: {text1}
LABEL: {label1}
...
SENTENCE: {text}
LABEL:
Few-Shot-CoT Prompt Below is a text classification problem,
Note that you can only select the label in
{options}. Let's think step by step and give
your explanation to verify the answer.
SENTENCE: {text1}
EXPLANATION: {explain1}
LABEL: {label1}
...
SENTENCE: {text}
LABEL: {label}
EXPLANATION:
Notice that we obtain the explanation demonstrations for Few-Shot-CoT by prompting LLM to automatically generate the explanation according to the golden label without human annotation. The prompt is as follows:
Few-Shot-CoT Explanation Generation Below is a text classification problem.
Let's think step by step and give your
explanation to verify the SENTENCE label:
SENTENCE: Fears for T N pension after talks
Unions representing workers at Turner Newall
say they are 'disappointed' after talks with
stricken parent firm Federal Mogul.
LABEL: Business
EXPLANATION: The statement discusses talks
between unions and a parent firm, which
relates to business-related negotiations and
concerns regarding pensions.
SENTENCE: {text}
EXPLANATION:
C.2 Baselines with Pairwise Reduction
Here, we present the pairwise prompt used in baselines with label reduction. The LLMs are instructed to compare options in pairs within the refined set obtained from the first stage.
Zero-Shot Pairwise Prompt Which term is more likely to represent
the topic of "{text}" - "{label1}" or
"{label2}"?
Zero-Shot-CoT Pairwise Prompt Which term is more likely to represent
the topic of "{text}" - "{label1}" or
"{label2}"?
Let's think step by step and give your
explanation to verify your answer:
Few-Shot Pairwise Prompt Below is a text classification problem,
please complete the sentence by "{label1}"
or "{label2}":
SENTENCE: {text1}
LABEL: {label1}
...
SENTENCE: {text}
LABEL:
Few-Shot-CoT Pairwise Prompt Below is a text classification problem.
Let's think step by step and give your
explanation to verify which term is more
likely to represent the label of the
sentence - "{label1}" or "{label2}":
SENTENCE: {text1}
EXPLANATION: {explain1}
LABEL: {label1}
...
SENTENCE: {text}
EXPLANATION:
C.3 Our PC-CoT Prompt
Our proposed contrastive elimination methodology, PC-CoT, consists of three key stages: similarity analysis, difference analysis, and final decisionmaking. Specific system instructions and prompts for the three stages are provided below.
Instruction Below is a text classification problem:
SENTENCE: {text1}
LABEL: {label1}
SENTENCE:{text2}
LABEL: {label2}
Similarity Analysis The phrases can often be mistaken for
"{label1}" and "{label2}", due to certain
shared characteristics.
SHARED ASPECTS:
Difference Analysis Next, diligently contrast the deviations
between these two topics, putting aside the
mentioned shared characteristics. Concisely
explain, what is the key element that sets
them apart?
CONTRASTING POINTS:
DecidePrompt After scrutinizing the presented SHARED
ASPECTS and CONTRASTING POINTS, which term
- "{label1}" or "{label2}" - would be a
more accurate representation for the label
of {text}? Provide the final label in the
format "LABEL: a".

heading: D Detail of Decision Boundary Ambiguity
text: The silhouette score is a metric used to quantify the compactness and separation of clusters within a dataset. In the context of our study on text data classification, the silhouette score provides insight into the overlap between different class samples. It is calculated for each data point using the formula:
s(i) = b(i)− a(i)
max{a(i), b(i)} (3)
Here, a(i) is the average distance of a data point i to other points within the same cluster, and b(i) is the smallest average distance to points in a different cluster. The overall silhouette score for the dataset is the average of these individual scores, denoted as S:
S = 1
N
N∑
i=1
s(i) (4)
where N is the total number of data points. So A higher silhouette score indicates clearer clusters and distinct separation suggesting a more evident decision boundary and facilitating classification. Conversely, a lower score suggests increased overlap between class boundaries, indicating a more ambiguous decision boundary.
Figure 9 conducts an ablation study on the number of categories. From left to right, we add a new category (in dark blue), leading to increased
overlap with existing categories (particularly orange, green, and light blue), resulting in ambiguous boundaries. This overlap is reflected in the decreasing silhouette score, highlighted by the red line. Notably, Figure 8 demonstrates a clear positive correlation between silhouette score and accuracy, motivating our introduction of a self-reduction stage to decrease the number of categories.

heading: E The LIU54 Dataset
text: The challenge posed by the LIU54 dataset, as evidenced by the difficulty encountered by even advanced language models like gpt-3.5-turbo and Qwen-72B, underscores a significant obstacle in natural language understanding. Despite the formidable capabilities of these models, they struggle to achieve satisfactory performance, often achieving accuracy rates of only around 30% using conventional baselines. A closer examination of LIU54 reveals that its difficulty primarily stems from the brevity of its sentences, which hinders effective interpretation and reasoning. This observation underscores the importance of enhancing the capacity of large language models to capture subtle semantic nuances, particularly in the context
of classification tasks. Overall, this highlights a critical area for improvement in the development of language understanding technologies.
Semantic incompleteness sentence: search golden: factoid prediction: query
sentence: Investing golden: quirky prediction: stock
Label mismatch sentence: What’s a parsec? golden: factoid prediction: definition
sentence: how we calculate golden: sendemail prediction: maths
sentence: Add that song to my playlist. golden: likeness prediction: Music
Multi-label sentence: Set a notification for sports game golden: set prediction: game
sentence: Is the brightness of my screen running low? golden: quirky prediction: hue_lighton
sentence: What time does the downtown parade start? golden: query prediction: events
We analyzed failed predictions by various Language Model (LLM) architectures using a confusion matrix (see Fig. 10) and presented selected failed cases above. Our findings reveal striking similarities in the confusing classes identified by the three LLMs. We categorize the failed cases into three overarching types: (i) Semantic Incompleteness, (ii) Label Mismatch, and (iii) Text Contents of Multiple Classes.
Semantic incompleteness refers to grammatically incomplete sentences typically comprised of a few words, such as ’search’, ’Investing’, ’Bingo’, etc. Label mismatch occurs when the assigned label from the original dataset is inappropriate; for example, ’What’s a parsec?’ is more fittingly categorized as "definition" rather than "factoid". In cases of multi-label content, sentences express multiple aspects that satisfy multiple categories concurrently. For instance, ’What time does the downtown parade start?’ fulfills both the ’query’ and ’events’ class criteria simultaneously.
Based on our analyses, we posit that confusing labels within the test set of the LIU54 Dataset may introduce bias into results. However, it also
presents an opportunity for evaluating the compatibility of different LLMs, thus serving as a challenging benchmark task.

heading: F Case Study
text: We present the complete set of options and detailed cases for the pairwise comparison framework, ranging from Table 9 to Table 17, along with additional reduction cases outlined in Table 18.
Sentence So I just put my top-up into the card and it hasn’t changed. Label pending_top_up
Reduce CBWR ITR verify_top_up top_up_failed balance_not_updated_after_bank_transfer pending_top_up top_up_by_card_charge top_up_reverted top_up_by_card_charge top_up_failed balance_not_updated_after_bank_transfer topping_up_by_card
Sentence I made a transfer and am still waiting. Label balance_not_updated_after_bank_transfer
Reduce CBWR ITR failed_transfer transfer_into_account balance_not_updated_after_bank_transfer pending_transfer transfer_not_received_by_recipient pending_transfer transfer_not_received_by_recipient transfer_timing transfer_fee_charged failed_transfer
Sentence Where can I get my card at? Label order_physical_card

heading: Future Work
text: Moreover, the future of technology is headed towards large-scale models. It is foreseeable that with technological advancements, the costs associated with API calls and model computations will decrease further. Our method, with its enhancements addressing bias and boundary issues, will become increasingly relevant and significant in solving real-world classification problems. Therefore, we believe that the scaling trend of language models will elevate the importance of our method in the future