Venue: NAACL 2025

# Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification

via Efficient Multi-Token Prediction

Junlang Qian1  Zixiao Zhu1,2  Hanzhang Zhou1,2   Zijian Feng1,2   Zepeng Zhai3

Kezhi Mao1,2

Corresponding author

###### Abstract

Zero-shot text classification typically relies on prompt engineering, but the inherent prompt brittleness of large language models undermines its reliability. Minor changes in prompt can cause significant discrepancies in model performance. We attribute this prompt brittleness largely to the narrow focus on next-token probabilities in existing methods. To address this, we propose Placeholding Parallel Prediction (\({\mathbf{\mathcal{P}^{3}}}\)), a novel approach that predicts token probabilities across multiple positions and simulates comprehensive sampling of generation paths in a single run of a language model. Experiments show improved accuracy and up to 98% reduction in the standard deviation across prompts, boosting robustness. Even without a prompt, \({\cal P}^{3}\) maintains comparable performance, reducing the need for prompt engineering.

+
Footnote †: _(Volume 1: Long Papers)_, pages 7093–7115

+
Footnote †: _(Volume 1: Long Papers)_, pages 7093–7115

## 1 Introduction

Zero-shot text classification Radford et al. (2019); Hu et al. (2021); Wei et al. (2022); Wang et al. (2023b); Liu et al. (2023); Yang et al. (2023) is among the most challenging applications of pre-trained large language models (LLMs) Brown et al. (2020); Touvron et al. (2023); Anthropic (2024), as it aims to categorize text without additional data. In this context, prompt engineering has become a widely adopted approach to enhance accuracy. However, language models exhibit inherent **prompt brittleness**Sclar et al. (2023); Zamfirescu-Pereira et al. (2023); Zhou et al. (2024a): their outputs are highly sensitive to minor modifications in prompt wording or format (as shown in Figure 1), leading to inconsistent performance. This issue makes crafting effective and reliable prompts difficult, particularly in zero-shot scenarios where prior information is lacking.

While some efforts have attempted to mitigate this issue through training Tam et al. (2021), general methods for the inference stage are rare. Calibration Zhao et al. (2021) is one of the few, but its primary benefit lies in enhancing accuracy rather than robustness, leaving the severe prompt brittleness problem not sufficiently resolved.

In this paper, we present a new perspective. We observe that current methods predominantly rely on next-token prediction Bengio et al. (2000); Sutskever et al. (2014) to classify. We posit that this sole **reliance on the next token may contribute to prompt brittleness**, based on the following intuitions: (1) Since the next token directly follows the input, it would be inevitably impacted by the prompt, whereas later tokens are likely less sensitive1; (2) Language models may not immediately provide the answer as the next token. Instead, they often first generate non-discriminative words such as "so", "a", or "very", or engage in preliminary reasoning2, which could hurt next-token performance for certain prompts.

Figure 1: An example of prompt brittleness. The prompt “It is a –” yields a notably high score for “tree”, while “It is an –” overwhelmingly favors “insect”. The percentage scores are normalized for an arbitrary text unrelated to any class \({\cal T}=\text{``knows grammar.''}\)

Our experiments support our hypothesis. As illustrated in Figure 2, the accuracies across different prompts based on next-token prediction (\(\bullet\), green dots) are highly dispersed. In contrast, the accuracies for later tokens (\(\bullet\bullet\), blue and red dots) are more tightly clustered together and demonstrate better overall performance. This suggests that subsequent token predictions are more robust to prompt variations and have the potential to improve performance.

However, to unlock the potential of subsequent token predictions, we must address a key challenge: current auto-regressive **language models predict only the next token3**.

Footnote 3: An exception exists: one language model introduced by Gloekele et al. can predict up to four tokens, but is still insufficient for our needs (more than ten tokens). Therefore, this model is not included in our study.

A naive solution is a token-by-token generation strategy, but it explores only a single generation path. Consequently, this strategy introduces error propagation, controllability issues, dependency on decoding algorithms, and randomness in generation path selection, all of which can hurt robustness4. In zero-shot natural language generation (NLG) tasks, a popular method to enhance this strategy is to sample multiple alternative generation paths and ensemble the results Wang et al. (2023); Lin et al. (2024); Zhang et al. (2024). Although extensive sampling could theoretically outperform next-token predictions in zero-shot classification5, the computation of repeatedly generating numerous tokens many times is overly costly. Thus, the **inefficiency** stemming from token-by-token generation and path dependency limits our ability to leverage subsequent-token predictions to overcome prompt brittleness.

Footnote 4: In practice, generation-based strategies typically underpinron next-token prediction in classification tasks Puri and Catanzaro (2019); Wei et al. (2021).

To address this, we propose **Placeholding Parallel Prediction** (\({\cal P}^{S}\)), a novel and pluggable method. \({\cal P}^{S}\) appends placeholder tokens at the end of the input sequence to simulate comprehensive sampling of generation paths, thereby enabling multiple token predictions simultaneously within one model run. \({\cal P}^{S}\) introduces subsequent token predictions into zero-shot classification with high efficiency and without being affected by generation path dependency. This offers an effective solution to prompt brittleness. Through extensive experiments on seven public benchmarks, we demonstrate that \({\cal P}^{S}\) significantly alleviates prompt brittleness and surpasses SoTA accuracy. Notably, with \({\cal P}^{S}\), performance without any prompt instructions matches that with crafted prompts, significantly reducing the need for prompt engineering.

Figure 3: Next-Token Prediction versus Placeholding Parallel Prediction. Our proposed \({\cal P}^{S}\) obtains multiple token predictions in a single language model run.

Figure 2: Accuracies of plausible prompts using tokens at the first three positions. Each row corresponds to a position: the \(\bullet\) green dots represent the next token (the 0th position), and the \(\bullet\) blue and \(\bullet\) red dots represent the 1st and 2nd tokens. Each dot denotes a prompt, and its horizontal coordinate indicates its performance.

 

Our contributions are as follows:

* We propose a new perspective: the narrow focus on next-token prediction may be a key contributor to prompt brittleness in zero-shot classification.
* We introduce \({\cal P}^{3}\), the first method to obtain multiple subsequent token predictions for text classification within a single language model run. Our code is open-sourced.
* Extensive experiments demonstrate that \({\cal P}^{3}\) significantly reduces prompt brittleness and outperforms SoTA accuracy. Notably, we find that even without any prompt, the performance is comparable to that with crafted prompts, greatly reducing the reliance on prompt engineering.

## 2 Related Work

### Prompt Brittleness

Prompt brittleness has emerged as a significant challenge in zero-shot classification Krause et al. (2020); Schick and Schutze (2021). This brittleness refers to the sensitivity of model performance to the specific wording and structure of prompts, where minor changes can lead to significant variations in output quality Zhou et al. (2024). Training-based solutions to address prompt brittleness often rely on labeled or unlabeled data. For instance, Logan et al. (2022) introduced strategies for simplifying prompt engineering through finentuning, but their methods require datasets for parameter optimization. Similarly, Wang et al. (2023, 2023) leveraged contrastive self-training, which involves iterative pseudo-labeling using generated data, but this also depends on large amounts of unlabeled data to work. Such methods are therefore impractical for zero-shot learning.

Consequently, few attempts have focused on mitigating prompt brittleness directly during the inference phase, where the model operates without any further adjustments. Calibration methods Zhao et al. (2021) represent one of the few efforts in this area. Zhao et al. propose calibration strategies to normalize model outputs based on contextual priors, improving consistency across different prompts without requiring external data. Another line of research aims to bypass prompt brittleness by automatically generating task-specific prompts Przyant et al. (2023); Gao et al. (2021); Holtzman et al. (2021); Jiang et al. (2020). Rather than fundamentally addressing the brittleness problem, these approaches serve as workarounds, sacrificing interpretability and cross-task consistency.

### Zero shot Text Classification

Since class labels typically possess clear semantics, we can leverage the capabilities of language models for zero-shot classification Radford et al. (2021); Zhou et al. (2023).

Zero-shot text classification Puri and Catanzaro (2019); Brown et al. (2020); Liu et al. (2023) has evolved from earlier approaches like embedding-based semantic matching and natural language inference NLI. Embedding-based methods Cer et al. (2018); Reimers and Gurevych (2019) suffer from semantic ambiguity, as inputs with subtle differences can receive similar embeddings. NLI-based methods Bowman et al. (2015); Ma et al. (2021); Zhu et al. (2024) decompose a multi-class problem into binary tasks, but this often results in poor calibration across binary classifiers, with inconsistent performance between classes.

Recent methods focus on next-token prediction and generation-based techniques. In next-token prediction, the model assigns a score based on the generation probability of a class label token Zhao et al. (2021). Although efficient, this method considers only a single token, limiting its capacity Feng et al. (2024). Generation-based methods, on the other hand, prompt the model to generate text and analyze label presence Radford et al. (2019); Brown et al. (2020); Wang et al. (2023). While these approaches can capture richer semantics, they are constrained to one generation path at a time and face controllability issues Krause et al. (2020); Ouyang et al. (2022).

Although these recent approaches address some shortcomings of earlier methods, they remain incomplete, as one focuses narrowly on a single token and the other only explores one possible output path. This leaves room for improvements in achieving more comprehensive classification strategies.

## 3 Methodology

### Background

Text classification seeks to assign the correct label \(c^{\star}\) to a given text \(t\), with \({\cal C}=\{c_{1},c_{2},\ldots,c_{k}\}\) representing the predefined set of candidate categories. Current approaches to zero-shot text classification rely on the assumption that the correct class label 

\(c^{*}\) or its corresponding tokens will exhibit higher generation probabilities under the language model than incorrect ones:

\[P(c^{*}\mid\mathbf{t})>P(c^{\prime}\mid\mathbf{t}),\quad\forall c^{\prime}\in\mathcal{C },\,c^{\prime}\neq c^{*}.\]

A language model \(\mathcal{LM}\) predicts the generation probability of the next token given an input sequence \(\mathbf{x}\). Specifically,

\[\mathcal{LM}(\mathbf{x})_{x_{n}}=P(x_{n}\mid(x_{0},x_{1},\ldots,x_{n-1})),\]

where the input \(\mathbf{x}\) is constructed by integrating the given text \(\mathbf{t}\) into a prompt template \(p(\cdot)\):

\[\mathbf{x}=(x_{0},x_{1},\ldots,x_{n-1})=p(\mathbf{t}).\]

The most commonly used approach is **based on next token prediction**, which assumes that the class probability \(P(c\mid\mathbf{t})\) is approximated by the language model's next token probability:

\[P(c\mid\mathbf{t})\approx\mathcal{LM}(\mathbf{x})_{x_{n}=c}.\]

Therefore, the class label is predicted by:

\[\hat{c}\leftarrow\arg\max_{c\in\mathcal{C}}\mathcal{LM}(\mathbf{x})_{x_{n}=c}.\]

An alternative approach **based on generation** leverages the continuation capability of language models to generate an output sequence:

\[\mathbf{x}\xrightarrow{\mathcal{LM}}(x_{n},x_{n+1},\hat{\dots},x_{n+m-1}).\]

The class label is then directly determined by which one appears in the generated sequence:

\[\hat{c}\leftarrow\left\{\hat{x}_{n},\hat{x}_{n+1},\ldots,\hat{x}_{n+m-1}\right\} \cap\mathcal{C}.\]

However, both methods remain incomplete: next-token prediction focuses on a single token position, while generation-based methods consider only one instance among many possible generation paths, limiting their robustness.

### Placeholding Skipping Prediction (\(\mathcal{PSP}\))

Let the \(i\)-th output token be \(x_{n+i}\) (where the next token is the 0-th token). All possible generation prefixes can be expressed as:

\[\left\{(x_{n},x_{n+1},\ldots,x_{n+i-1})\mid x_{j}\in\mathcal{V},\,n\leq j<n+i \right\},\]

where \(\mathcal{V}\) denotes the vocabulary, and \(\mathbf{x}\) represents the known input.

The probability of each prefix is:

\[P\left(x_{n},x_{n+1},\ldots,x_{n+i-1}\mid\mathbf{x}\right).\]

Given this prefix, the probability of \(x_{n+i}=c\) is:

Thus, the probability of the entire generation path is:

\[P\left(x_{n},x_{n+1},\ldots,x_{n+i}=c\mid\mathbf{x}\right)\] \[\qquad=P\left(x_{n},x_{n+1},\ldots,x_{n+i-1}\mid\mathbf{x}\right)\] \[\qquad\times P\left(x_{n+i}=c\mid\mathbf{x},x_{n+1},\ldots,x_{n+i-1} \right).\]

Next, by enumerating all possible prefixes, the overall probability for \(x_{n+i}=c\) becomes:

\[P\left(x_{n+i}=c\mid\mathbf{x}\right)\] \[\qquad\times P\left(x_{n+i}=c\mid\mathbf{x},x_{n},x_{n+1},\ldots,x_{ n+i-1}\right).\] This is equivalent to: \[P\left(x_{n+i}=c\mid\mathbf{x}\right)\] \[\qquad=P\left(x_{n+i}=c\mid\mathbf{x},\underbrace{x_{n},x_{n+1}, \ldots,x_{n+i-1}}_{\text{unknown}}\right).\] We approximate this as: \[P\left(x_{n+i}=c\mid\mathbf{x}\right)\] \[\qquad\approx P\left(x_{n+i}=c\middle|\mathbf{x},\underbrace{<\!\mbox {ph$>$},<\!\mbox{ph$>$},\ldots,<\!\mbox{ph$>$}}_{\text{i times}}\right).\] This can be rewritten using the language model as: \[P\left(x_{n+i}=c\mid\mathbf{x}\right)\] \[\qquad\approx\mathcal{LM}\left(\mathbf{x},\underbrace{<\!\mbox{ ph$>$},<\!\mbox{ph$>$},\ldots,<\!\mbox{ph$>$}}_{\text{i times}}\right)_{x_{n+i}=c}.\] As shown in Figure 4(b), we append \(i\) placeholder tokens, <ph>6, to the input sequence 

and feed this extended sequence into the language model. This allows us to obtain the prediction for the \(i\)-th subsequent token, formally expressed as:

\[{\cal PSP}(\mathbf{x},i)={\cal LM}(\mathbf{x^{\prime}}),\]

where:

\[\mathbf{x^{\prime}}=(x_{0},x_{1},\ldots,x_{n-1},\underbrace{<\text{ph>},<\text{ ph>},\ldots,<\text{ph>}}_{i~{}\text{times}}).\]

We refer to this approximation of the \(i\)-th token prediction as Placeholding Skipping Prediction (\({\cal PSP}\)).

### Placeholding Parallel Prediction (\({\cal P}^{3}\))

In practical implementation, LLMs employ a transformer architecture with blocks stacked in both depth and sequence length, maintaining width-aligned input and output. Transformers with unidirectional attention produce an output for each prefix of the input sequence \(\mathbf{x}\) during inference:

\[\text{transformer}(\mathbf{x})=\begin{bmatrix}{\cal LM}(\varnothing),\\ {\cal LM}(x_{0}),\\ {\cal LM}(x_{0},x_{1}),\\ \dots,\\ {\cal LM}(x_{0},x_{1},\ldots,x_{n-1})\end{bmatrix}.\]

This property supports tasks such as computing text likelihood and parallel training. As depicted in Figure 4(a), current classification methods merely use the next-token prediction, \({\cal LM}(\mathbf{x})={\cal LM}(x_{0},x_{1},\ldots,x_{n-1})\), the last element of the transformer output.

Appending a series of <ph> tokens (subject to memory constraints) to the input sequence enables the transformer to compute all prefixes automatically:

\[\text{transformer}(\mathbf{x^{\prime}})\] \[=\begin{bmatrix}{\cal LM}(\varnothing),\\ {\cal LM}(x_{0}),\\ {\cal LM}(x_{0},x_{1}),\\ \cdots,\\ {\cal LM}(x_{0},x_{1},\ldots,x_{n-1},<\text{ph>}),\\ {\cal LM}(x_{0},x_{1},\ldots,x_{n-1},<\text{ph>},<\text{ph>}),\\ \dots&\end{bmatrix},\]

where:

\[\mathbf{x^{\prime}}=(x_{0},x_{1},\ldots,x_{n-1},<\text{ph>},<\text{ph>},\ldots).\]

As illustrated in Figure 4(c), extracting our desired elements, which correspond to the next-token and all subsequent tokens, we define:

\[{\cal P}^{3}(\mathbf{x}) =\begin{bmatrix}{\cal LM}(x_{0},x_{1},\ldots,x_{n-1}),\\ {\cal LM}(x_{0},x_{1},\ldots,x_{n-1},<\text{ph>}),\\ {\cal LM}(x_{0},x_{1},\ldots,x_{n-1},<\text{ph>},<\text{ph>}),\\ \dots&\end{bmatrix}\] \[=\begin{bmatrix}{\cal LM}(\mathbf{x}),\\ {\cal PSP}(\mathbf{x},1),\\ {\cal PSP}(\mathbf{x},2),\\ \dots&\end{bmatrix}=\begin{bmatrix}{\cal PSP}(\mathbf{x},0),\\ {\cal PSP}(\mathbf{x},1),\\ {\cal PSP}(\mathbf{x},2),\\ \dots&\end{bmatrix}.\]

By leveraging the transformer's inherent capability to handle multiple token predictions7 in a single run, this approach significantly enhances efficiency. We term this method \({\cal P}^{3}\).

Footnote 7: Note that our multi-token prediction does not generate a specific sequence of tokens; rather, it produces a probability distribution over the vocabulary for each token position.

## 4 Experiment

### Datasets, Models, and Prompts

We conducted extensive experiments on two large language models and seven publicly available

Figure 4: (a) Next-Token Prediction. (b) Placeholding Skipping Prediction (\({\cal PSP}\)). (c) Placeholding Parallel Prediction (\({\cal P}^{3}\)). The small green rectangles indicate the output tokens to be used, and the grey ones indicate those not to be used. ¡ph¿ represents a placeholder token.

 

datasets, using over 30 plausible prompts for each dataset.

We employed seven publicly available text classification datasets: Amazon Review Polarity Zhang et al. (2015), SST2 Socher et al. (2013), and IMDb Maas et al. (2011) for sentiment classification (binary labels: positive/negative) of product and movie reviews; AGnews Zhang et al. (2015) and DBpedia Bizer et al. (2009) for topic classification (four and fourteen categories, respectively) of news articles and Wikipedia content; Yahoo Answers Zhang et al. (2015) comprises questions

\begin{table}
\begin{tabular}{c|l l|r r|r r r r|r} \hline \hline  & **Dataset** & **Gen** & **3w-SC** & **NTP** & **Cali1** & **Cali2** & **Cali3** & **Ours** \\ \hline \multirow{4}{*}{IMdb} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & 68.11 & 79.36 & 77.92 & 75.08 & **82.35 +14.24** \\  & & std & & – & 0.1003 & 0.0899 & 0.0907 & 0.0962 & **0.0020** -98.05\% \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 47.92 & 53.13 & 61.60 & 71.23 & 71.82 & 68.99 & **75.86 +14.26** \\  & & std & 0.2053 & 0.3291 & 0.0729 & 0.0798 & 0.0749 & 0.0834 & **0.0721** -7.92\% \\ \hline \multirow{4}{*}{AGnews} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & \multirow{2}{*}{–} & 55.75 & 60.40 & 61.70 & 61.15 & **80.51 +24.76** \\  & & std & & & 0.1267 & 0.1361 & 0.1106 & 0.1053 & **0.0068** -94.64\% \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 58.59 & 50.88 & 48.24 & 49.37 & 50.38 & 49.62 & **80.14 +31.90** \\  & & std & 0.1180 & 0.3016 & 0.1544 & 0.1546 & 0.1398 & 0.1450 & **0.0329** -78.71\% \\ \hline \multirow{4}{*}{DBpedia} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & \multirow{2}{*}{–} & 64.39 & 56.52 & 61.91 & 60.04 & **64.47 +0.08** \\  & & std & & & 0.0965 & 0.2223 & 0.0922 & 0.0948 & **0.0052** -94.64\% \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 66.41 & 56.35 & 70.03 & 67.11 & 67.23 & 65.74 & **72.98 +2.95** \\  & & std & 0.0438 & 0.3288 & 0.0744 & 0.0694 & 0.0822 & 0.0865 & **0.0469** -36.92\% \\ \hline \multirow{4}{*}{Amazon} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & \multirow{2}{*}{–} & 65.35 & 77.37 & 78.03 & 78.62 & **81.62 +16.27** \\  & & std & & & 0.0986 & 0.1086 & 0.0886 & 0.0839 & **0.0065** -93.45\% \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 49.22 & 53.32 & 61.14 & 60.50 & 61.02 & 61.41 & **78.72 +17.58** \\  & & std & 0.1413 & 0.3315 & 0.0466 & 0.0766 & 0.0699 & 0.0772 & 0.0644 +38.39\% \\ \hline \multirow{4}{*}{ISEAR} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & \multirow{2}{*}{–} & 36.79 & 27.48 & 41.07 & 37.73 & **41.61 +4.82** \\  & & std & & & 0.0935 & 0.1771 & 0.0712 & 0.0960 & **0.0182** -80.51\% \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 24.09 & 25.88 & 36.90 & 40.59 & 33.30 & 37.17 & **43.22 +6.32** \\  & & std & 0.0208 & 0.1500 & 0.0935 & 0.0835 & 0.0716 & 0.0875 & **0.0653** -**30.18\%** \\ \hline \multirow{4}{*}{SST-2} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & \multirow{2}{*}{–} & 63.21 & 73.15 & 69.52 & 67.60 & **80.14 +16.92** \\  & & std & & & 0.0997 & 0.1145 & 0.0995 & 0.1131 & **0.0178** -**82.17\%** \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 43.49 & 47.85 & 57.30 & 66.64 & 66.98 & 60.57 & **71.07 +13.77** \\  & & std & 0.2419 & 0.3224 & 0.0612 & 0.1041 & 0.1007 & 0.0994 & 0.0843 +37.65\% \\ \hline \multirow{4}{*}{Yahoo} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & 41.90 & 34.94 & 45.12 & 45.48 & **50.50 +8.59** \\  & & std & & & 0.0967 & 0.1918 & 0.0797 & 0.0781 & **0.0073** -**92.43\%** \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 36.98 & 31.93 & 37.74 & 38.85 & 40.30 & 39.07 & **50.74 +13.00** \\  & & std & 0.0516 & 0.1931 & 0.1210 & 0.0980 & 0.1006 & 0.0844 & **0.0544** -**55.03\%** \\ \hline \hline \multirow{4}{*}{**Avg**} & \multirow{2}{*}{13B} & acc (\%) & \multirow{2}{*}{–} & \multirow{2}{*}{–} & 56.50 & 58.46 & 62.18 & 60.81 & **68.74 +12.24** \\  & & std & & & 0.1017 & 0.1486 & 0.0904 & 0.0953 & **0.0091** -**91.05\%** \\ \cline{2-10}  & \multirow{2}{*}{70B} & acc (\%) & 46.67 & 45.62 & 53.28 & 56.33 & 55.78 & 54.65 & **67.53 +14.25** \\ \cline{2-10}  & & std & 0.1175 & 0.2795 & 0.0899 & 0.0951 & 0.0914 & 0.0948 & **0.0600** -**33.26\%** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Results of accuracy and cross-prompt standard deviation (i.e., prompt brittless) for each dataset. Gen refers to a vanilla generative method, and 3w-SC denotes three-way self-consistency. NTP represents the vanilla next-token prediction method, which serves as our baseline. Cali1, Cali2, and Cali3 correspond to calibration methods using “N/A,” an empty string, and ”<unk>*5” as the calibration text, respectively. Avg represents the average across datasets. For further details on hyperparameter settings, see appendix A.3. As shown, our method significantly improves stability while achieving the highest accuracy.

 

and answers classified into ten categories. ISEAR Scherer and Wallbott (1994) consists of sentences labeled with seven emotions for emotion classification. To save time, we selected 10,000 random instances from the IMDb, Amazon, DBpedia, and Yahoo datasets for LLaMA2-70b, and from the Amazon dataset for LLaMA2-13b.

The two models are LLaMAA2-13b, with 13 billion parameters for efficient NLP tasks, and LLaMA2-70b, with 70 billion parameters for more complex tasks and broader context handling Touvron et al. (2023).

We employed a method similar to that of Gonen et al. Gonen et al. (2022) to generate prompts, using ChatGPT-4 OpenAI et al. (2024) to expand a set of seed prompts. A comprehensive list of all prompts used in our experiments is provided in the appendix A.9.

### Comparison Methods

Our baseline method employs the straightforward next-token prediction for classification. Additionally, we implemented a calibration method Zhao et al. (2021) to compare its performance with subsequent tokens. The calibration method adjusts the conditional generation probability (classification scores) of the next-token under different prompts by introducing meaningless text inputs and performing a single additional run of the model. This approach achieves state-of-the-art performance in zero-shot classification without requiring any manually collected external information, knowledge, or unlabeled data resources, thereby ensuring interpretability. The calibration method's enhancements to zero-shot classification focus on two critical aspects: average accuracy and stability, both of which are pivotal to our evaluation criteria. We used three different meaningless strings for calibration to ensure a comprehensive comparison. We also evaluated 256 samples per dataset using generative methods and three-way self-consistency to demonstrate our efficiency.

### Experimental Details

We conducted the experiments on two A6000 GPUs.

In our experiments, for each data sample and prompt combination, we appended 512 additional <ph> tokens8 to examine the effectiveness of our method across a broad range. To demonstrate that our approach is not reliant on this specific number, we introduced the hyperparameter \(\eta\) to control the range of tokens considered dynamically. Small \(\eta\) values focus on tokens close to the next position, while large \(\eta\) values consider tokens far down the sequence. Specifically, when \(\eta=0\), the method is equivalent to next-token prediction.

Footnote 8: The use of so many <ph> tokens was initially intended to ensure redundancy but turned out to be unnecessary.

The two models exhibit behavior differences9, prompting us to adapt our method accordingly. For LLaMA2-13B, classification is based on the probability distribution of the token located at a position proportional to the number of input tokens, determined by a slope of \(\tan(\eta)\). We set \(\eta\) as the angle and map it into the tangent space for smoother transitions and scaling. For LLaMA2-70b, we define \([0,\eta)\) as a fixed range to vote and apply calibration to enhance the results.

## 5 Results Analysis

### Overview

Aggregating results across seven datasets reveals that our \({\cal P}^{S}\) method substantially improves **efficiency**, **robustness**, and **accuracy**.

**Efficiency:** Unlike generative approaches that require sequential token-by-token predictions through multiple model runs, \({\cal P}^{S}\) obtains multiple token predictions simultaneously in a single run. As evidenced in Table 2, \({\cal P}^{S}\) achieves time complexity on par with direct next-token prediction, ensuring high efficii

\begin{table}
\begin{tabular}{l|c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{4}{c}{**Number of Runs**} \\ \cline{2-5}  & **Gen** & **3w-SC** & **NTP** & \({\cal P}^{S}\) \\ \hline Amazon & 30.75 & 92.26 & 1 & 1 \\ IMDb & 28.03 & 84.09 & 1 & 1 \\ AGnews & 46.98 & 140.94 & 1 & 1 \\ DBpedia & 36.09 & 108.27 & 1 & 1 \\ Yahoo & 34.52 & 103.55 & 1 & 1 \\ SST-2 & 25.80 & 77.39 & 1 & 1 \\ ISEAR & 26.86 & 80.58 & 1 & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The average number of output tokens used to reach an answer (number of model runs). For outputs exceeding 50 tokens without matching any of the options, the sequence is truncated at 50. Gen denotes the vanilla generative approach, 3w-SC denotes 3-way self-consistency, NTP denotes next-token prediction, and \({\cal P}^{S}\) is ours.

 

multiple token prediction, \({\cal P}^{3}\) consumes slightly more floating-point operations (FLOP) compared to next-token prediction, as shown in Table 3.

**Robustness and Accuracy: \({\cal P}^{3}\)** significantly reduces prompt brittleness and achieves better performance.

Figure 5 illustrates that applying a unified hyperparameter \(\eta\) to all datasets, \({\cal P}^{3}\)**consistently outperforms** next-token prediction (\(\eta=0\)) in overall accuracy and robustness within a broad range (\(0<\eta<60\)). Notably, the average cross-prompt standard deviation (i.e., prompt brittleness) reaches its minimum around \(\eta=5\) and stabilizes afterward, with accuracy peaking around the same point. This behavior implies that \(\eta\approx 5\) can serve as a nice practical default setting when without task-specific prior knowledge.

Our method adapts effectively to diverse specific scenarios, and the results for individual datasets are presented in Table 1. \({\cal P}^{3}\) achieves high accuracy and robustness on the evaluated datasets, outperforming baseline and state-of-the-art approaches. Especially on LLaMA2-13b, our method achieved over 80% reduction in standard deviation on every dataset, with an average reduction of 91%. For AGNews, the LLaMA2-13b and LLaMA2-70b models achieved 25% and 32% increases in accuracy, along with 94% and 79% reductions in standard deviation, respectively. On IMDb, the LLaMA2-13B model not only improved accuracy by 14% but also yielded a standard deviation of only 0.002 (with a decrease over 98%), almost eliminating prompt brittleness.

### Performance without Prompt

Given the significant reduction in standard deviation, the differences between various prompts

\begin{table}
\begin{tabular}{p{42.7pt}|c c|c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c|}{**13B**} & \multicolumn{2}{c}{**70B**} \\ \cline{2-5}  & **Crafted** & **NoP** & **Crafted** & **NoP** \\ \hline IMDb & 82.34 & 82.51 & 75.82 & 77.33 \\ AGNews & 80.52 & 80.26 & 80.15 & 79.85 \\ Yahoo & 50.47 & 51.06 & 50.84 & 48.04 \\ Amazon & 81.61 & 81.78 & 79.66 & 80.60 \\ SST-2 & 80.13 & 80.45 & 71.23 & 65.28 \\ DBpedia & 64.50 & 63.52 & 72.98 & 72.96 \\ ISEAR & 41.61 & 41.66 & 43.23 & 42.66 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The average accuracy comparison between null prompts (i.e., using no prompt) and crafted prompts. Crafted represents crafted prompts, and NoP indicates using no prompt.

Figure 5: Average accuracy and average cross-prompt standard deviation (i.e., prompt brittleness) across seven datasets of \({\cal P}^{3}\). The horizontal axis represents the hyperparameter \(\eta\), where \(\eta=0\) corresponds to the next-token prediction results, and larger \(\eta\) values indicate consideration of more distant token positions. \(\eta>0\) shows higher accuracy and lower standard deviation compared to next-token prediction.

 

have been largely minimized, making the choice of a specific prompt less critical. As shown in Table 4, using a null prompt (i.e., without any prompt) for zero-shot classification yields performance comparable to that of crafted prompts. This demonstrates that effectively mitigates prompt brittleness and substantially reduces the reliance on prompt engineering.

## 6 Disscusion

**What is the motivation of prompt engineering?** The need for prompt engineering arises from the prompt brittleness (or cross-prompt instability), as not all prompts yield the same accuracy. Prompt engineering would become unnecessary if all prompts achieve consistent accuracy. Addressing the issue of prompt brittleness would save significant costs associated with prompt engineering, making it a worthwhile research topic.

**Is prompt engineering necessary in zero-shot classification?** We leverage the explicit semantic information contained within the classification labels to perform zero-shot classification. We assign a sample to the label with which it has the highest semantic alignment. We use the token generation probability as classification scores in zero-shot classification. This approach assumes that if the context is closely related to the semantics of a certain class label, tokens associated with that class will exhibit a higher co-occurrence probability compared to tokens related to other classes. The purpose of adding a prompt is to guide the model to generate class-related tokens as the next token. These class-related tokens may not necessarily appear at the next position but could emerge later in the text. The misalignment between the classification task and NLG, leading to the out-of-distribution (OOD) problem, undermines zero-shot performance by causing LLMs to generate non-discriminative words before the relevant class labels. Some studies, like Gonen et al. (2022) and Zhou et al. (2023a), alleviate the OOD issue by calculating prompt perplexity, but we address the core principle directly. Once subsequent tokens are predicted, the prompt's influence reduces. Logically, the text and the categories should suffice for classification. However, next-token predictions are often unstable, with accuracy fluctuations exceeding 10%, leading to the introduction of prompt engineering. Subsequent token predictions are significantly more stable and do not require such adjustments. As analyzed in Section 5.2, our method allows for good classification scores without the need for a prompt.

**Why are zero-shot classification capabilities not commonly used as an evaluation metric for large language models?** Theoretically, zero-shot classification is an excellent, fair, and direct evaluation metric. However, the accuracy of next-token zero-shot classification is heavily influenced by the quality of the prompt. This reliance on prompt selection makes zero-shot classification a less reliable metric. Conversely, using the accuracy distribution of subsequent tokens obtained by our proposed \(\mathcal{P}^{3}\), which is much more stable and less affected by the prompt, provides a fairer and more reliable evaluation metric.

## 7 Conclusion

Prompt brittleness is a critical challenge in zero-shot classification, undermining the reliability and consistency of language model outputs. In this work, we proposed \(\mathcal{P}^{3}\), a novel method that introduces subsequent token predictions within a single model run to address this issue. Our extensive experiments across seven public benchmarks demonstrated that significantly mitigates prompt brittleness while achieving accuracy beyond the SoTA. Notably, performs equally well with or without prompts, greatly reducing the reliance on prompt engineering. These findings highlight \(\mathcal{P}^{3}\) 's potential as an effective solution for robust zero-shot classification, making classification more efficient and reliable.

## 8 Limitation

In the paper, we discussed using existing tokens as placeholders <ph>. Training soft tokens as placeholders may be another solution. For auto-regressive language models, this approach involves training a soft token embedding of the ideal <ph> with a corpus. The formal expression of this token's properties is:

where denotes the concatenation of sequences.

Future work will focus on the behavior of tokens generated by our \(\mathcal{P}^{3}\) method across different types of models and languages. Additionally, we plan to train a language model inherently equipped with <ph> tokens 



## References

* Almutiri and Nadeem (2022) Talal Almutiri and Farrukh Nadeem. 2022. Markov models applications in natural language processing: a survey. _Int. J. Inf. Technol. Comput. Sci_, 2:1-16.
* Anthropic (2024) AI Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. _Claude-3 Model Card_.
* Bengio et al. (2000) Yoshua Bengio, Rejean Ducharme, and Pascal Vincent. 2000. A neural probabilistic language model. In _Advances in Neural Information Processing Systems_, volume 13. MIT Press.
* a crystallization point for the web of data. _Journal of Web Semantics_, 7(3):154-165. The Web of Data.
* Bowman et al. (2015) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. _arXiv preprint arXiv:1508.05326_.
* Brown et al. (1992) Peter F Brown, Vincent J Della Pietra, Peter V Desouza, Jennifer C Lai, and Robert L Mercer. 1992. Class-based n-gram models of natural language. _Computational linguistics_, 18(4):467-480.
* Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. _Preprint_, arXiv:2005.14165.
* Cer et al. (2018) Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Bromni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder. _Preprint_, arXiv:1803.11175.
* Feng et al. (2024) Zijian Feng, Hanzhang Zhou, Sixiao Zhu, Junlang Qian, and Kezhi Mao. 2024. Unveiling and manipulating prompt influence in large language models. In _The Twelfth International Conference on Learning Representations_.
* Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. _arXiv preprint arXiv:2012.15723_.
* Gloeckle et al. (2024) Fabian Gloeckle, Badr Youbi Idrissi, Bapitste Roziere, David Lopez-Paz, and Gabriel Synnaeve. 2024. Better & faster large language models via multi-token prediction. _arXiv preprint arXiv:2404.19737_.
* Gonen et al. (2022) Hila Gonen, Srini Iyer, Terra Blevins, Noah A. Smith, and Luke Zettlemoyer. 2022. Demystifying prompts in language models via perplexity estimation. _Preprint_, arXiv:2212.04037.
* Holtzman et al. (2021) Ari Holtzman, Peter West, Verred Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. Surface form competition: Why the highest probability answer isn't always right. _arXiv preprint arXiv:2104.08315_.
* Hu et al. (2021) Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, and Maosong Sun. 2021. Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification. _arXiv preprint arXiv:2108.02035_.
* Jiang et al. (2020) Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. Can prompt-based learning be practical for low-resource tasks? _arXiv preprint arXiv:2001.07676_.
* Krause et al. (2020) Ben Krause, Akhiles Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2020. Gedi: Generative discriminator guided sequence generation. _Preprint_, arXiv:2009.06367.
* Lin et al. (2024) Lei Lin, Jiayi Fu, Pengli Liu, Qingyang Li, Yan Gong, Junchen Wan, Fuzheng Zhang, Zhongyuan Wang, Di Zhang, and Kun Gai. 2024. Just ask one more time! self-agreement improves reasoning of language models in (almost) all scenarios. In _Findings of the Association for Computational Linguistics ACL 2024_, pages 3829-3852.
* Liu et al. (2023) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhenghao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pretrain, prompt, and predict: A systematic survey of prompting methods in natural language processing. _ACM Computing Surveys_, 55(9):1-35.
* Logan IV et al. (2022) Robert L. Logan IV, Ivana Balazevic, Eric Wallace, Fabio Petroni, Sameer Singh, and Sebastian Riedel. 2022. Cutting down on prompts and parameters: Simple few-shot learning with language models. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 3461-3476.
* Ma et al. (2021) Tingting Ma, Jin-Ge Yao, Chin-Yew Lin, and Tiejun Zhao. 2021. Issues with entailment-based zero-shot text classification. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_, pages 786-796, Online. Association for Computational Linguistics.
* Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_, pages 142-150, Portland, Oregon, USA. Association for Computational Linguistics

OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aledman, Diogo Almeida, Janko Atlenschmidt, Sam Altaman, Shyamal Andradat, Red Avala, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Ballescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Braknan, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosic Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dolhan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simon Posada Fishman, Juston Forte, Isabella Fullford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidelberg, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angelia Jiang, Roger Jiang, Haozhn Jin, Denny Jin, Shimo Jomoto, Billie Jonn, Hewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Greten Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christiene McLeaveay, Paul McMillan, Jake McNNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Winnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Mura Murati, Oleg Murk, David Melby, Ashin Nair, Reitichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyennowo Noh, Long Ouyang, Cullen O'Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattarascandolo, Joel Parish, Empy Paraparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Poonde de Oliveira Pinto, Michael Pokarmy, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Bae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussee, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Hearth Schmidt, David Schnurr, John Schulman, Daniel Selsm, Kyla Sheppard, Toki Sherpakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Syzhrom Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeeline B. Thompson, Phil Thilt, Amin Tootoonchin, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Feipipe Cerno Uribe, Andrea Vallone, Arnu Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matit Weithoff, Dave Willmer, Clemens Winter, Samuel Wolich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. 2024. Gpt-4 technical report. _Preprint_, arXiv:2303.08774.
* Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diego Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In _Advances in Neural Information Processing Systems_, volume 35, pages 27730-27744. Curran Associates, Inc.
* Przyant et al. (2023) Reid Przyant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with" gradient descent" and beam search. _arXiv preprint arXiv:2305.03495_.
* Puri and Catanzaro (2019)Raul Puri and Bryan Catanzaro. 2019. Zero-shot text classification with generative language models. _arXiv preprint arXiv:1912.10165_.
* Radford et al. (2021)Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In _International conference on machine learning_, pages 8748-8763. PMLR.
* Radford et al. (2019)Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. _Preprint_, arXiv:1908.10084.
* Scherer and Wallbott (1994) Klaus R Scherer and Harald G Wallbott. 1994. Evidence for universality and cultural variation of differential emotion response patterning. _Journal of personality and social psychology_, 66(2):310 

* Schick and Schutze (2021) Timo Schick and Hinrich Schutze. 2021. Exploiting cloze questions for few shot text classification and natural language inference. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 255-269.
* Selar et al. (2023) Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. _arXiv preprint arXiv:2310.11324_.
* Shannon (1948) C. E. Shannon. 1948. A mathematical theory of communication. _The Bell System Technical Journal_, 27(3):379-423.
* Shannon (1951) Claude E Shannon. 1951. Prediction and entropy of printed english. _Bell system technical journal_, 30(1):50-64.
* Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In _Proceedings of the 2013 conference on empirical methods in natural language processing_, pages 1631-1642.
* Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In _Advances in Neural Information Processing Systems_, volume 27. Curran Associates, Inc.
* Tam et al. (2021) Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. 2021. Improving and simplifying pattern exploiting training. _arXiv preprint arXiv:2103.11955_.
* Touvron et al. (2021) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiboglu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Ian, Marcia Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloundann, Artem Korenev, Punit Singh Kouara, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Dianaiskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaloylov, Pushkar Mishra, Igor Molbyog, Yixin Nie, Andrew Poulton, Jeremy Reisenstein, Rashi Runeta, Kaylan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Pavxin Xu, Zheng Yan, Iiiyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialoom. 2023. Llama 2: Open foundation and fine-tuned chat models. _Preprint_, arXiv:2307.09288.
* Wang et al. (2023) Fan Wang, Lei Zhao, and Li Chen. 2023a. Generation-driven contrastive self-training for zero-shot text classification with instruction-tuned gpt. _arXiv preprint arXiv:2304.11872_.
* Wang et al. (2023b) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhrery, and Denny Zhou. 2023b. Self-consistency improves chain of thought reasoning in language models. _Preprint_, arXiv:2203.11171.
* Wang et al. (2023c) Yau-Shan Wang, Ta-Chung Chi, Ruohong Zhang, and Yiming Yang. 2023c. Pesco: prompt-enhanced self contrastive learning for zero-shot text classification. _arXiv preprint arXiv:2305.14963_.
* Wei et al. (2022) Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. _arXiv preprint arXiv:2109.01652_.
* Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In _Advances in Neural Information Processing Systems_, volume 35, pages 24824-24837. Curran Associates, Inc.
* Yang et al. (2023) Hui Yang, Sifu Yue, and Yunzhong He. 2023. Autogpt for online decision making: Benchmarks and additional opinions. _Preprint_, arXiv:2306.02224.
* Zamfirescu-Pereira et al. (2023) JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why johnry can't prompt: how non-ai experts try (and fail) to design llm prompts. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_, pages 1-21.
* Zhang et al. (2015a) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015a. Character-level convolutional networks for text classification. _Advances in neural information processing systems_, 28.
* Zhang et al. (2015b) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015b. Character-level convolutional networks for text classification. In _Advances in Neural Information Processing Systems_, volume 28. Curran Associates, Inc.
* Zhang et al. (2024) Ziqi Zhang, Cunxiang Wang, Xiong Xiao, Yue Zhang, and Donglin Wang. 2024. Nash cot: Multi-path inference with preference equilibrium. _arXiv preprint arXiv:2407.07099_.
* Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In _International conference on machine learning_, pages 12697-12706. PMLR.
* Zhou et al. (2024a) Han Zhou, Xingchen Wan, Lev Proleev, Diana Mincu, Jilin Chen, Katherine Heller, and Subhrajit Roy. 2024a. Batch calibration: Rethinking calibration for in-context learning and prompt engineering. _preprint_, arXiv:2309.17249 

Hanzhang Zhou, Zijian Feng, Zixiao Zhu, Junlang Qian, and Kezhi Mao. 2024b. Unibias: Unveiling and mitigating llm bias through internal attention and frn manipulation. _arXiv preprint arXiv:2405.20612_.
* Zhou et al. (2023a) Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviuitis, Harris Chan, and Jimmy Ba. 2023a. Large language models are human-level prompt engineers. _Preprint_, arXiv:2211.01910.
* Zhou et al. (2023b) Yunjiao Zhou, Jianfei Yang, Han Zou, and Lihua Xie. 2023b. Tent: Connect language models with iot sensors for zero-shot activity recognition. _arXiv preprint arXiv:2311.08245_.
* Zhu et al. (2024) Zixiao Zhu, Junlang Qian, Zijian Feng, Hanzhang Zhou, and Kezhi Mao. 2024. Edential: An entailment-based few-shot text classification with extensional definition. In _Findings of the Association for Computational Linguistics: NAACL 2024_, pages 1124-1137.

## Appendix A Appendix

### LM Output Examples

Some examples of the language model's greedy output are in Table 5. As shown, the generation of language models does not ensure that the next token will produce the discriminative words necessary for classification.

Zero-Shot Classification: Worth Studying Zero-shot text classification remains a crucial research area, even as few-shot learning gains attention. Few-shot learning cannot fully replace zero-shot approaches, as it still relies on labeled data, which is not always feasible in many practical scenarios.

**Broader Applicability and Fewer Constraints**: Zero-shot learning is more aligned with real-world needs, especially when users cannot provide labeled examples, or when constructing even a single example for complex inputs is impractical. In contrast, few-shot learning assumes a shared distribution between provided and test examples, which is not always guaranteed.

**Core Foundation for Research**: Zero-shot learning serves as a more fundamental base for research, offering cleaner results without complexities like sample selection or ordering seen in few-shot studies, which can obscure core contributions.

**Greater challenge and a benchmark for intrinsic model capabilities**: Zero-shot classification directly tests the inherent capabilities of language models, free from the influence of examples. Our \({\cal P}^{\it 3}\) method further enhances the reliability of zero-shot evaluations by mitigating prompt instability, making it a more stable and valuable metric.

### Selected \(\eta\)

The hyperparameter \(\eta\) selections are in Table 6.

### Dataset Size

The size of datasets we used in the paper is shown in Table 7.

### Behavior Differences of Language Models

Although, as discussed in Section 5.1, the use of the \({\cal P}^{\it 3}\) method consistently yields benefits, the classification performance of tokens at different positions varies. In this regard, the 13B and 7OB

\begin{table}
\begin{tabular}{l|c c} \hline \hline
**Dataset** & **13B** & **70B** \\ \hline IMDb & 37 & 42 \\ AGnews & 59 & 497 \\ Amazon & 4 & 17 \\ DBpedia & 62 & 4 \\ ISEAR & 8 & 5 \\ SST-2 & 6 & 7 \\ Yahoo & 65 & 4 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Hyperparameter settings for different datasets and models.

\begin{table}
\begin{tabular}{r|p{142.3pt}} \hline \hline Input: & My ACL paper is accepted. It is \_ \\ Output: & a very good paper. \\ \hline Input: & My legs are broken. I am \_ \\ Output: & in a lot of pain. \\ \hline Input: & I had my first job this year. It is \_ \\ Output: & a part-time job at a local restaurant. I am a waitress. I like my job very much. \\ \hline \hline \end{tabular}
\end{table}
Table 5: Greedy output examples from LLMa2-13B. Blue text indicates non-discriminative expressions and red text indicates discriminative words for target categories {positive, negative}.

 

models exhibit different behaviors. Specifically, the performance of the 13B model is not only position-dependent but also influenced by the number of input tokens, whereas the 70B model does not exhibit this phenomenon.

As shown in Figure 6, the distribution of well-performing (or poorly-performing) tokens in the 13B model follows a radial pattern, proportional to the number of input tokens. In contrast, the performance of tokens in the 70B model depends solely on their position and is independent of the input token length (exhibits vertical striping patterns).

In this paper, we treated it as an observed fact and leveraged it as a characteristic of language models. However, the exact underlying cause of this difference is challenging to determine due to the lack of transparency in LLaMA2's training details. We hypothesize that this distinction might arise from differences in the training process:

Stable positions in 70b: If the model underwent direct instruction-tuning or RLHF (Reinforcement Learning with Human Feedback) without distillation, the potential well-performing token positions would likely remain stable. This is because, for most classification tasks, answer lengths are consistent regardless of input variation.

Length-dependent positions in 13b: If the model underwent distillation from a larger model during instruction-tuning or RLHF, it might have inherited reasoning behaviors linked to input length, causing the observed correlation between input length and potential well-performing token positions. We would like to interpret this result as a possible consequence of discrepancies in training processes and parameter scales. While we have not confirmed this hypothesis, we believe deeper exploration could lead to a more rigorous explanation. Since this behavior difference does not impact our primary claims regarding the mitigation of prompt brittleness, we treated it as an existing fact in our experiments.

The two models exhibit behavior differences,

Figure 6: abcdefg**: Accuracy heatmaps for each dataset on 13b. In each subplot, the x-axis represents the text length \(l_{t}\) (number of tokens in the text), and the y-axis represents the position of the token used for classification (0 represents the next-token). **h** shows the average accuracy of the IMDb dataset at different slopes \(k\) (with intercept \(b=-10\)).

 

prompting us to adapt our method accordingly. For LLaMA2-13b, we observed a radial pattern in its plot, prompting us to set \(\eta\) as the angle and use a line with slope \(\tan(\eta)\) as our final result. For LLAMA2-70b, with a stable separability plot independent of input length, we define [0, \(\eta\)] as a fixed range for voting and applying calibration to enhance the results.

Furthermore, we found that using the votes of all 513 tokens obtained via \({\cal P}^{S}\) method as pseudo-labels also produces patterns similar to those of the true labels. However, we did not pursue further investigation into this matter.

### FLOPs Estimation for Transformer Models

To estimate the FLOPs for the forward pass of a Transformer model, we consider the main computational components: attention and feed-forward networks (FFN).

For the attention mechanism, the input matrices \(Q\), \(K\), and \(V\) (with dimensions \((B,s,h)\)) go through linear transformations to produce \(W_{q}\), \(W_{k}\), and \(W_{v}\):

QKV transformation FLOPs = 6Bsh^2 Next, the attention matrix is computed by multiplying the query matrix \(Q\) with the transposed key matrix \(K^{T}\), resulting in a matrix of size \((s,s)\):

Attention matrix computation FLOPs = 2Bs^2 h Then, the attention values are computed by multiplying the attention matrix with the value matrix \(V\):

Attention over values FLOPs = 2Bs^2 h Finally, a linear projection is applied to the resulting matrix:

Post-attention linear projection FLOPs = 2Bsh^2 For the FFN, where the input dimension \(h\) is transformed to 4\(h\) and then back to \(h\), the FLOPs for this part are:

FFN FLOPs = 16BSh^2 The language model head performs a linear transformation from the hidden size \(h\) to the vocabulary size \(V\), resulting in the following FLOPs:

LM head FLOPs = 2Bsh V The total FLOPs per layer, considering attention and FFN, is the sum of the components above:

FLOPs per layer = 24Bsh^2+4Bs^2 h Including the language model head, the total FLOPs per step for the forward pass in a model with \(l\) layers is:

Total FLOPs per step = l \[\left(24Bsh^2+4Bs^2h\right)\] + 2BshV Based on the formula, we also computed the case using the selected \(\eta\)s (Table 8), as well as the case where all 512 placeholder tokens were used in our experiments (Table 9).

As can be seen, the overall computational cost is proportional to the number of tokens, indicating that our method does not impose a significant overhead.

Figure 7: Consistency plot with pseudo-labels, where the pseudo-labels are determined by the voting results of 513 tokens.

 



### About Multi-Token Class Names

A common challenge in zero-shot classification, particularly when dealing with class names that span multiple tokens, is how to calculate the class score from the prediction of a single token. This issue arises due to the way current language models generate and calibrate token probabilities.

In typical zero-shot approaches based on token generation probabilities, directly multiplying probabilities for individual tokens within a sequence often fails to yield reliable classification scores. This problem is particularly pronounced when class descriptions consist of multiple tokens. Some studies address this challenge by integrating external resources, such as knowledge bases or validation datasets, which can help identify relevant tokens or synonyms associated with each class. For example, one common method is to sum the probabilities of all relevant tokens (e.g., "good" and "positive") to compute a cumulative score. However, such methods often require extensive preprocessing, which can limit their applicability in more dynamic or real-time scenarios.

In this work, we focus on exploring the robustness of subsequent token predictions. To ensure consistency with previous research and simplify the setup, we adopted the widely used practice of mapping each class to a single token (e.g., "good" for the positive class and "bad" for the negative class), as seen in standard datasets.

An interesting aspect of our \({\cal P}^{3}\) method, however, is its ability to naturally handle multi-token class names. By aggregating probabilities across subsequent tokens, \({\cal P}^{3}\) can provide a more coherent approach to multi-token class labels. Rather than relying on the prediction of a single token, \({\cal P}^{3}\) aggregates predictions across tokens, enabling it to better align with natural language. For instance, in cases where tokens like "ppi" or "ness" are unlikely to appear in the immediate context, \({\cal P}^{3}\) can still effectively aggregate across related tokens, such as "good" and "positive," to compute a more accurate class score.

This approach presents an important advantage over traditional methods, offering a more scalable and flexible framework for zero-shot classification tasks, particularly in scenarios involving complex or multi-token class names.

### Position Plot

#### Prompts used in the experiment.

To evaluate the effectiveness and robustness of our Placeholding Parallel Prediction method, we conduct extensive experiments on seven datasets with various kinds of prompts, including empty prompts. The specific prompt settings for different datasets are shown in Table 10-16.

Figure 8: Accuracy and cross-prompt standard deviation for each dataset using predicted tokens at fixed positions on 13b. The x-axis shows token position. Dots indicate average accuracy and shaded areas represent standard deviation.

\begin{table}
\begin{tabular}{l|c c|c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c|}{**13B**} & \multicolumn{2}{c}{**70B**} \\ \cline{2-5}  & **NTP** & \({\cal P}^{3}\) & **NTP** & \({\cal P}^{3}\) \\ \hline Amazon & 2.82 & 3.00 & 14.26 & 16.47 \\ IMDb & 8.31 & 14.73 & 41.91 & 47.41 \\ AGnews & 1.62 & 4.31 & 8.20 & 73.32 \\ DBpedia & 2.16 & 6.24 & 10.94 & 11.46 \\ Yahoo & 3.67 & 11.78 & 18.56 & 19.08 \\ SST-2 & 0.67 & 0.72 & 3.38 & 4.28 \\ ISEAR & 0.70 & 0.79 & 3.57 & 4.22 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Total Tflops calculation for the selected \(\eta\).





\begin{table}
\begin{tabular}{r|l} \hline ID & Template \\ \hline
1 & title\textbackslashturednMy feedback to it is \\
2 & title\textbackslashturednOverall, my feedback to it is \\
3 & title\textbackslashturednAfter considering all aspects, my feedback to it is \\
4 & title\textbackslashturednAfter considering all aspects, my viewpoint is \\
5 & title\textbackslashturednReflecting on the above, my viewpoint is \\
6 & title\textbackslashturednOverall, my perspective on it is \\
7 & title\textbackslashturednOverall, my takeaway is \\
8 & title\textbackslashturednIn summary, I would say \\
9 & title\textbackslashturednConsidering the details provided, my emotional reaction is \\
10 & title\textbackslashturednTaking into account the experience shared, my viewpoint is \\
11 & title\textbackslashturednReflecting on the content, my emotional stance is \\
12 & title\textbackslashturednGiven the information above, my perspective on it is \\
13 & title\textbackslashturednAnalyzing the feedback, my emotional assessment is \\
14 & title\textbackslashturednBased on the review, my overall sentiment impression is \\
15 & title\textbackslashturednWeighing up the insights, my sentiment conclusion is \\
16 & title\textbackslashturednAfter thoroughly considering the review, my sentiment perspective is \\
17 & Text: title textnSentiment: \\
18 & Text: title textnSentiment Analysis: The overall sentiment is \\
19 & title\textbackslashturednAll in all, it was \\
20 & title\textbackslashturednIn summary, it was \\
21 & title\textbackslashturednIn essence, it was \\
22 & title\textbackslashturednIn conclusion, it was \\
23 & title\textbackslashturednTo sum up, it’s \\
24 & title\textbackslashturednAll in all \\
25 & title\textbackslashturednText Just \\
26 & title\textbackslashturednIt was \\
27 & title\textbackslashturednIt is \\
28 & title\textbackslashturednThat is \\
29 & title\textbackslashturednThat's \\
30 & title\textbackslashturednBut it is \\
31 & title\textbackslashturedn

* [15] R. J. Baxter, M. C. Bader, and D. A. Kessler, "The \(\ell_{2}\)-norm of the \(\ell_{3}\) norm of a matrix," in _Proceedings of the 7th International Conference on Machine Learning_, pages 409-418. ACM, 2006.
* [16] S. L. Barnett, "A new method for computing the \(\ell_{2}\) norm of a matrix," _Journal of computational complexity_, vol. 10, no. 1, pp. 103-118, 2006.
* [17] T. H. Cover, "On the number of rows," _Computational Complexity_, vol. 10, no. 1, pp. 1-10, 1970.
* [18] G. E. Hinton, "Learning to compute the \(\ell_{2}\) norm of a matrix," _IEEE Transactions on Information Theory and Applications_, vol. 10, no. 1, pp. 1-10, 1971.
* [19] P. F. Hinton, "Deep learning with applications to machine learning," _arXiv preprint arXiv:1407.0581v1_, 2014.
* [20] P. F. Hinton, "An introduction to neural networks," _Advances in Neural Information Processing Systems_, vol. 1, no. 1, pp. 1-10, 1971.

[MISSING_PAGE_POST]

 

* [15] R. C. Myers, J. M. Sethammer, and D. A. Lowe, "The automated method for discovering the object's image in a large scale," _IEEE Transactions on Pattern Analysis and Machine Intelligence_**20**(4):367-389, 2002.
* [16] L. B. Krizhevsky, I. Sutskever, and E. Rabinovich, "A simple approach to detect objects from tiny images," _IEEE transactions on pattern analysis and machine intelligence_**20**(4):125-130, 2002.
* [17] P. G. LeCun, T. H. Lee, and N. Vedaldi, "An introduction to deep learning with deep convolutional neural networks," _arXiv preprint arXiv:1905.03018v1_ [cs], pp. 1-12.

[MISSING_PAGE_POST]

 



\begin{table}
\begin{tabular}{r|l} \hline ID & Template \\ \hline
1 & [title] [text] This topic is about \\
2 & [title] [text] The label that best describes this news article is \\
3 & [title] [text] This piece of news is regarding \\
4 & [title] [text] The news article is about \\
5 & [title] [text] Central themes of this news piece encompass \\
6 & [title] [text] The central theme of this article revolves around \\
7 & [title] [text] It can be labeled as \\
8 & [title] [text] Its category is \\
9 & [title] [text] In this article, it talks about \\
10 & [title] [text] The content is a kind of \\
11 & [title] [text] I think the news can be classified as \\
12 & [title] [text] I would classify it as \\
13 & [title] [text] Based on the description, its category is \\
14 & [title] [text] In this context, the content falls into the category of \\
15 & Text: [title] [text] Category: \\
16 & Text: [title] [text] Topic Classification: The overall topic is \\
17 & [title] [text] All in all, it was \\
18 & [title] [text] In summary, it was \\
19 & [title] [text] In essence, it was \\
20 & [title] [text] In conclusion, it was \\
21 & [title] [text] To sum up, it’s \\
22 & [title] [text] All in all \\
23 & [title] [text] Just \\
24 & [title] [text] It was \\
25 & [title] [text] It is \\
26 & [title] [text] That is \\
27 & [title] [text] That’s \\
28 & [title] [text] But it is \\
29 & [title] [text]



\begin{table}
\begin{tabular}{r|l} \hline \hline ID & Template \\ \hline
1 & [title] [text] The category of [title] is \\
2 & [title] [text] The label that best describes [title] is \\
3 & [title] [text] So, [title] is \\
4 & [title] [text] In this sentence, [title] is \\
5 & [title] [text] [title] is a kind of \\
6 & [title] [text] can be classified as \\
7 & [title] [text] [title] is an example of \\
8 & [title] [text] [title] belongs to \\
9 & [title] [text] I think [title] is \\
10 & [title] [text] I would classify [title] as \\
11 & [title] [text] Based on the description, its category is \\
12 & [title] [text] In this context, [title] falls into the category of \\
13 & Text: [title] [text] Category: \\
14 & Text: [title] [text] Topic Classification: The overall topic is \\
15 & [title] [text] All in all, it is \\
16 & [title] [text] In summary, it is \\
17 & [title] [text] In essence, it is \\
18 & [title] [text] In conclusion, it is \\
19 & [title] [text] To sum up, it’s \\
20 & [title] [text] All in all \\
21 & [title] [text] Just \\
22 & [title] [text] It was \\
23 & [title] [text] It is \\
24 & [title] [text] That is \\
25 & [title] [text] That’s \\
26 & [title] [text] But it is \\
27 & [title] [text] \\ \hline \end{tabular}
\end{table}
Table 14: Prompt list for DBpedia dataset.





\begin{table}
\begin{tabular}{r|c} \hline ID. & Template \\ \hline
1 & [title] [text] This topic is about \\
2 & [title] [text] The label that best describes this question is \\
3 & [title] [text] This issue is regarding \\
4 & [title] [text] This discussion is about \\
5 & [title] [text] This discussion is regarding \\
6 & [title] [text] This issue is about \\
7 & [title] [text] The label that best describes this issue is \\
8 & [title] [text] I would classify this question as \\
9 & [title] [text] It can be labeled as \\
10 & [title] [text] Overall, The most fitting category for this issue is \\
11 & [title] [text] The content is associated with \\
12 & [title] [text] I think it belongs to \\
13 & [title] [text] I would classify it as \\
14 & [title] [text] This issue falls into the category of \\
15 & Text: [title] [text] Category: \\
16 & Text: [title] [text] Topic Classification: The overall topic is \\
17 & [title] [text] All in all, it was \\
18 & [title] [text] In summary, it was \\
19 & [title] [text] In essence, it was \\
20 & [title] [text] In conclusion, it was \\
21 & [title] [text] To sum up, it’s \\
22 & [title] [text] All in all \\
23 & [title] [text] Just \\
24 & [title] [text] It was \\
25 & [title] [text] It is \\
26 & [title] [text] That is \\
27 & [title] [text] That’s \\
28 & [title] [text] But it is \\
29 & [title] [text]



\begin{table}
\begin{tabular}{r|l} \hline ID & Template \\ \hline
1 & [text] In summary, I would say \\
2 & [text] I think it is \\
3 & [text] Overall, I think it is \\
4 & [text] Considering everything, I think it is \\
5 & [text] Overall, I see it as \\
6 & [text] In summary, I would say \\
7 & [text] I feel \\
8 & [text] Overall, I feel \\
9 & [text] Overall, my feeling towards it is \\
10 & [text] This text expresses \\
11 & [text] It is a feeling of \\
12 & [text] The sentiment is \\
13 & [text] It is \\
14 & [text] This conveys a sense of \\
15 & [text] I am \\
16 & [text] The overall impression is \\
17 & [text] From my perspective, it is \\
18 & [text] In my view, the feeling is \\
19 & [text] This passage makes me feel \\
20 & [text] It seems to evoke a feeling of \\
21 & [text] This text primarily conveys \\
22 & [text] From this, I sense an emotion of \\
23 & [text] It can be interpreted as expressing \\
24 & [text] The underlying emotion seems to be \\
25 & [text] This narrative elicits \\
26 & [text] Feeling-wise, this comes across as \\
27 & [text] This evokes \\
28 & [text] The emotional tone here is \\
29 & [text] This story is imbued with \\
30 & [text] One could interpret this as \\
31 & [text] This text leaves the impression of \\
32 & Text: [text] Emotion: \\
33 & Text: [text] Emotion Recognition: The overall emotion is \\
34 & [text] All in all, it was \\
35 & [text] In summary, it was \\
36 & [text] In essence, it was \\
37 & [text] In conclusion, it was \\
38 & [text] To sum up, it was \\
39 & [text] All in all \\
40 & [text] Just \\
41 & [text] It was \\
42 & [text] It is \\
43 & [text] That is \\
44 & [text] That’s \\
45 & [text] But it is \\
46 & [text] \\ \hline \end{tabular}
\end{table}
Table 16: Prompt list for ISEAR dataset.