Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning (2024)

Abstract: Large language models (LLMs) have shown remarkable capabilities in various natural language understanding tasks with a few demonstration examples via in-context learning. Common strategies to boost such “in-context” learning ability are to ensemble multiple model decoded results and require the model to generate an explanation along with the prediction. However, these models often treat different class predictions equally and neglect the potential discrepancy between the explanations and predictions. To fully unleash the power of explanations, we propose EASE, an ExplanationAware Soft Ensemble framework to empower in-context learning with LLMs. We design two techniques, explanation-guided ensemble, and soft probability aggregation, to mitigate the effect of unreliable explanations and improve the consistency between explanations and final predictions. Experiments on 10 NLU tasks and 4 varying-size LLMs demonstrate the effectiveness of our framework.

heading: 1 Introduction
text: Recent advancements in Natural Language Processing (NLP) have witnessed the remarkable capabilities of Large Language Models (LLMs) (Brown et al., 2020; Anil et al., 2023; Touvron et al., 2023; OpenAI, 2023). These LLMs can rapidly adapt to new tasks by learning only on a few input-output pairs (a.k.a. demonstrations) in context (Wei et al., 2022a). Yet, beyond those demonstrations, a significant facet of human learning revolves around explanations. These explanations1, typically in the form of a few keywords or sentences, reveal the underlying principles connecting the input and output (Zaidan et al., 2007; Narang et al., 2020). Consequently, integrating free-text explanations into
∗ Work done during the internship at Google. E-mail: yueyu@gatech.edu
1In this paper, we use the term ‘explanations’ and ‘rationales’ interchangeably.
LLM prompting holds great potentials to further enhance in-context learning performance.
Recent studies have examined how to incorporate free-text explanations into LLM in-context learning scheme. For instance, the Predict-thenExplain pipeline (Lampinen et al., 2022) proposes to generate the explanation after making the prediction. Consequently, the predictions from LLM won’t directly benefit from their corresponding explanations. In contrast, the Explain-then-Predict pipeline (a.k.a “Chain-of-Thought”) (Wei et al., 2022b) generates explanations before making predictions via greedy sampling. When the LLMgenerated explanations are unreliable, predictions from this approach will be largely distracted and defective (Ye and Durrett, 2022; Turpin et al., 2023). To mitigate this issue, (Wang et al., 2023d) improves the “Chain-of-Thought” pipeline by generating multiple predictions with different explanations using temperature sampling and aggregating them via majority voting. However, this approach can be sub-optimal as (1) temperature sampling increases the inconsistency between generated explanations and their associated class predictions, and (2) majority voting treats predictions associated with explanations of varying qualities equally. How to robustly leverage free-text explanations for LLM in-context learning remains challenging.
In this work, we present an Explanation-aware Soft Ensemble framework, named EASE, to assist LLM in-context learning with explanations. Our technique integrates explanations into the ensemble procedure and employs soft probability to mitigate discrepancies between explanations and predictions. The key module of the EASE framework hinges upon the idea of weighted ensemble: As shown in Figure 1, instead of treating all predictions equally, we assign a score to each prediction based on the contextual relevance and inherent quality of its associated explanation, which will be used as a weight during the final ensemble stage.
14002
While several works have studied scoring the LLMgenerated text (Wang et al., 2023b; Khalifa et al., 2023; Jiang et al., 2023), these models typically require massive training labels on downstream tasks or intermediate reasoning annotations. To facilitate explanation scoring with under the few-shot setting studied in this work, we realize this explanationaware ensemble stage with an LLM — after generating explanations and predictions using temperature sampling for each test instance, we prompt the LLM to weight all class predictions based on their associated explanations in an in-context manner. While the LLM offers great promise for the weighting purpose, it is crucial to provide sufficient supervision signals as demonstrations to guide the LLM scoring, yet the primary constraint for this step lies in the absence of negative explanations from few-shot demonstrations. To construct negative examples, we first use LLM to generate explanations for few-shot demonstrations, then select explanations associated with incorrect predictions as the negative samples. In this way, the LLM scorer can be readily applied to perform explanation-aware ensembling without any additional annotation.
Beyond explanation-aware ensembling, EASE incorporates an additional technique named soft probability aggregation, which helps to mitigate the inconsistency between explanations and predictions, given the sampling process may inevitably infuse noises into the final prediction. Specifically, it employs probabilities across various classindicative verbalizers in place of the original onehot predictions. This design, although conceptually simple, can effectively reduce the discrepancies between explanations and predictions and further improve the final predictions accuracy.
Our contributions can be summarized as follows:
• We propose the EASE framework to better fa-
cilitate in-context learning for large language models with natural language explanations.
• We begin with an analysis of the limitations in current in-context learning methods, leading to the development of explanation-aware ensemble and soft probability aggregation techniques. These techniques aim to prioritize high-quality explanations and minimize inconsistencies between explanations and predictions, without requiring large amounts of additional annotation.
• We conduct experiments on seven natural language understanding (NLU) datasets spanning between natural language inference (NLI) and question answering (QA), and our method outperforms state-of-the-art approaches using 4 different LLMs as the backbone. Our analysis further justifies the advantage of EASE for dealing with unreliable explanations as well as mitigating inconsistent predictions.

heading: 2 Related Work
text: Two prevalent explanation types exist for interpreting NLP models: (1) extraction-based explanations that highlight important segments of the original input (DeYoung et al., 2020; Paranjape et al., 2020; Zhou et al., 2020; Yin and Neubig, 2022) and (2) free-form explanations that craft prediction rationales directly using natural language text (Rajani et al., 2019; Sun et al., 2022; Wiegreffe et al., 2021, 2022; Wang et al., 2023a; Ludan et al., 2023; Shen et al., 2023; Zhang et al., 2024). Beyond aiding in model interpretation, recent studies have demonstrated that these explanations can also enhance the few-shot reasoning capabilities of large language models. For example, (Wei et al., 2023b) use tasklevel explanations to assist the test-time adaptation
for LLMs. (Krishna et al., 2023) leverage the extracted keywords tokens as additional input to assist LLM in-context learning, but requires a white-box LLM. (Wei et al., 2022b; Zelikman et al., 2022) propose to prepend explanations before the answers while (Lampinen et al., 2022) suggest adding postanswer explanations. Given that these explanations are often derived during the LLM decoding stage and may contain noise, (Wang et al., 2023d, 2022b) advocate for generating multiple candidate explanations with predictions, followed by aggregating these predictions via majority voting. In our study, we focus on free-form explanations and explore how to better aggregate these predictions with explanations in a weighted ensemble. Using a bootstrapped LLM, we evaluate each explanation to enhance in-context learning.
Another line of research related to our study is automated explanation quality evaluation (Sun et al., 2022; Wang et al., 2022a; Joshi et al., 2023; Chen et al., 2023a,b). (Ye and Durrett, 2022) utilize lexical features to measure the faithfulness of explanations without considering their semantics. (Chen et al., 2021; Li et al., 2023b) leverage a NLI fine-tuned model to verify the reliability of explanations. (Fu et al., 2023; Liu et al., 2023; Qin et al., 2023; Jiang et al., 2023) also study how to use LLM to build a generic text quality scorers for NLP tasks. These studies rely on ground-truth labels and human annotations, making them less suitable when the labels for test instances are unknown. Besides, several works attempt to build up a scoring model (Wang et al., 2023b; Khalifa et al., 2023) and rely on a large training set for scoring model training, yet such labeled data can be infeasible to obtain under the true few-shot setting. In contrast, our research focuses more on effectively leveraging model-generated explanations to improve LLM in-context learning with a few demonstrations only.

heading: 3 Method
text: In this section, we first give a brief introduction to the problem definition. Then, we present our approach with two designs, namely explanationaware ensemble and soft probability aggregation, with the goal of leveraging the generated explanations to improve the final prediction performance.

heading: 3.1 Problem Definition
text: We are given an LLM M parameterized by θ, a set of few-shot demonstrations D = {(xi, ei, yi)}Ki=1 on a target classification task2, where K is the number of demonstrations, xi, yi are the input and label for the i-th example, and ei is the ground-truth explanation for xi. For each example x ∈ Dtest, we use M and D to predict its label. Our primary goal is to improve the prediction accuracy on Dtest.

heading: 3.2 Recap of Self-consistency Pipeline
text: Here we give an introduction to self-consistency (Wang et al., 2023d) for LLM in-context learning. For each test example x ∈ Dtest, it first forms the prompt for few-shot demonstrations as P = { T , shuffle(∥Ki=1(xi, ei, yi) ) }, where T is
the prompt template, and shuffle ( ∥Ki=1(xi, ei, yi) ) is a permutation of K demonstrations. Then, it generates N candidate explanations together with predictions (denoted as (ej , pj)) via sampling from the LLM with non-zero temperature as
(ej , pj) N j=1 ∼ pθ (e, p | P, x) . (1)
Finally, it aggregates these N candidates into the final prediction via majority voting as ỹ = argmaxy ∑N j=1 I(pj = y). Self-consistency enhances the standard explain-then-predict pipeline by utilizing multiple predictions derived from varied explanations. Despite its strong performance, through our examination, we’ve pinpointed two primary bottlenecks within the self-consistency pipeline, listed as follows:
• Explanation-agnostic Ensembling: Selfconsistency uniformly weights all predictions and aggregates them via majority voting. It overlooks the variance in explanation quality, which can be problematic when certain predictions stem from flawed reasoning paths evident in poor-quality explanations (Agarwal et al., 2024).
• Explanation-Prediction Inconsistency: During its prediction, self-consistency employs temperature sampling to draw samples from the LLM. This sampling step can introduce noise, leading to predictions that are inconsistent with their corresponding explanations (Ye and Durrett, 2022).
The identified limitations necessitate the need to better harvest intermediate explanations for obtaining the final prediction. Towards this goal, we
2Our main focus is on classification tasks, but our framework can also be extended to generative tasks (section 4.7).
propose EASE, a framework tailored to tackle the aforementioned challenges. EASE is comprised with two techniques, explanation-aware ensemble and soft probability aggregation, to optimize the LLM’s prediction accuracy when deriving final outcomes from multiple candidate explanations.

heading: 3.3 Explanation-guided Ensemble
text: LLMs typically produce multiple explanations along with their predictions through a sampling process. Due to the intrinsic randomness of this sampling, the quality of these predictions can fluctuate. To address the potential pitfalls where erroneous explanations results in inaccurate predictions, we introduce the explanation-aware ensemble technique. This method estimates the significance of each class prediction based on its corresponding explanation. Consequently, our explanation-aware ensemble technique ensures that predictions linked with better explanations carry greater weight during the final prediction aggregation phase. LLM as Few-shot Explanation Scorer To evaluate various explanations, past research either measured the lexical overlap between the explanation and the input (Ye and Durrett, 2022) or employed models fine-tuned on NLI tasks (Chen et al., 2021; Li et al., 2023b). In contrast to them, which either overlook the deep semantics of explanations or require extra human annotations, our explanation scorer is developed based on the LLM M, directly harnessing its inherent reasoning capabilities.
Given the original task input x and one explanation e, we use the verbalizer vpos(vneg) to represent the class of this explanation being “positive” (“negative”). A “positive” explanation means this explanation can help the model reach correct answer and a “negative” explanation means the other way around. Then, we craft a supplementary prompt Tscore = “Can this explanation be used to help the model answer the question?” for LLM prompting. With the verbalizers and prompts, we recast the problem of explanation scoring into determining the conditional probability of producing the verbalizer aligned with the positive label vpos as
ωe = pθ (y = vpos | Tscore, x, e) . (2)
In this way, the score ωe is normalized between 0 and 1 and a higher score indicates the explanation with better quality. Bootstrapped LLM Scorer Although the above approach can already produce scores for each prediction, the score generated with the LLM M can
still be biased and less precise (Wang et al., 2023c), especially under the zero-shot scenario where no demonstrations are provided. To mitigate the bias and generate reliable scores, we propose to provide additional examples to serve as “positive” and “negative” explanations to facilitate LLM scoring using the original few-shot demonstrations in D.
For each original demonstration instance, it is straightforward to obtain “positive” examples from the ground-truth explanation. Obtaining “negative” examples, on the other hand, can be more challenging as they are not explicitly provided. To tackle this issue, we exploit the assumption based on the utility of explanations: an ideal explanation should guide the model towards the accurate prediction of ground-truth labels (Wiegreffe et al., 2021). Consequently, it’s reasonable to classify explanations leading to erroneous predictions as "negative". In practice, for every instance (xi, yi) ∈ D, we randomly select k (8 in this work) exemplars from the training set and then use these as demonstrations and generate a set of candidate pairs Ci = {(eij , pij)}Nj=1 via sampling from the LLM. Then, if the explanation-prediction pair (eij , pij) from Ci satisfies yi ̸= pij , we select eij as the negative explanation set Ni for xi as
Ni = {(eij , pij) ∈ Ci | yi ̸= pij}. (3)
For every instance, a random “negative” explanation is chosen from the candidate set. This produces a balanced demonstration set for LLM explanation scoring without requiring extra human annotations.

heading: 3.4 Soft Probability Aggregation
text: In the preceding step, the primary objective is to assign a score to each prediction based on its associated explanation through the LLM M. This process, however, does not account for directly modeling the LLM’s output predictions. To bridge this gap, we propose soft probability aggregation,
a simple and intuitive approach to resolve the discrepancy between the explanations and predictions — rather than aggregating over the raw predictions, it directly computes the sum of the probabilities associated with each potential label, expressed as
ỹ = argmax y
N∑
j=1
pθ (y | P, x, ej) . (4)
The soft probability aggregation reduces the noise inherited in LLM sampling-based decoding algorithms, resulting in a more accurate final prediction.

heading: 3.5 Summary
text: By plugging these two techniques together, we obtain the final prediction ỹ for test instance x as
ỹ = argmax y
N∑
j=1
ωej × pθ (y | P, x, ej) , (5)
where ej is the explanation generated via Eq. 1, the ωej is the weight for ej using the bootstrapped LLM scorer in Eq. 2, and pθ (y | P, x, ej) is the soft probability generated using Eq. 4. Overall, calculating the score for explanations and soft probabilities both take an additional O(N) time complexity. Fortunately, these two steps do not require additional training and can be efficiently supported with distributed inference techniques in practice3. Other than these techniques, EASE keeps other components intact and can be plugged into most LLMs to empower their in-context learning ability.

heading: 4 Experiments

heading: 4.1 Experiment Setups
text: Tasks We evaluate our EASE framework on two types of tasks: natural language inference and question answering including E-SNLI (Camburu et al., 2018), ANLI (Nie et al., 2020), ECQA (Aggarwal et al., 2021), OpenbookQA (Mihaylov et al., 2018) and StrategyQA (Geva et al., 2021). Besides, we conduct additional experiments on XNLI (Conneau et al., 2018), GSM8k (Cobbe et al., 2021) and SVAMP (Patel et al., 2021) in Section 4.7.
Baselines We consider the following approaches: (1) Standard In-context Learning (ICL) (Brown et al., 2020), (2) Predict-then-Explain (PE) (Lampinen et al., 2022), (3) Explainthen-Predict (EP) (Wei et al., 2022b), (4) Self-consistency (Wang et al., 2022b, 2023d), (5) FLamE (Zhou et al., 2023), (6) OrderEnsem-
3More studies are in Appendix E.
ble (Lu et al., 2022) and (7) PromptBoost (Hou et al., 2023) as baselines. The details are in App. B.
Implementation Details In our main experiments, we use PaLM2-S and PaLM2-L (Anil et al., 2023) as the backbone model. Results on more (open source) backbone models are reported in Section 4.3. For each dataset, we set the size of fewshot examples to 48 following (Zhou et al., 2023; Marasovic et al., 2022), and fit as many instances as possible during inference until reached the maximum length. As the LLM is often sensitive to the selection of few-shot examples (Ye and Durrett, 2023; Liu et al., 2022), for each dataset, we create 5 splits from the original dataset, each containing 300 test examples, and report the average performance over 5 splits. During sampling, we set the default temperate to t = 0.7 and sample N = 9 candidate explanations for each instance.

heading: 4.2 Overall Results
text: Table 1 exhibits the performance of EASE and baselines on seven datasets using PaLM 2-S and PaLM 2-L as the backbone. From the results, we have the following findings: First, leveraging explanations often improves LLM in-context learning, especially when aggregating multiple predictions sampled from the LLM4. Conversely, the standard EP pipeline sometimes even hurts the performance of larger models. Existing boosting methods for incontext learning do not help much in improving performance. Second, despite its complex design, the latest baseline FLamE often falls short compared to other baselines, which suggests that fine-tuning an additional classifier is particularly important for FLamE and it might be less compatible with the LLM in-context learning framework. Third, EASE consistently outperforms all other methods across nearly all datasets and model sizes, providing a reliable way to improve in-context learning. Finally, When comparing EASE with its own variants (e.g. w/o BLS and SPA), it’s observed that the original EASE consistently holds an advantage, indicating the necessity of both PW and SA components in maximizing performance.

heading: 4.3 Results on Open-source Models
text: In order to demonstrate the generalizability of our EASE framework, as well as promote reproducibil-
4We have also tried to incorporate multiple explanations with temperature sampling for PE pipeline, but we find performance drops. This is because the prediction of the PE pipeline can not benefit from generated explanations.
ity, we extend our investigations to open-source LLMs including FLAN-UL2 (Tay et al., 2023)5 and Llama-2-7b (Touvron et al., 2023). Both models have publicly accessible weights6. Table 2 shows the open-source LLMs generally perform worse than PaLM 2 on the challenging NLU benchmarks, mainly due to having fewer parameters. Despite this, the experiment results still align with our prior findings, demonstrating that our proposed techniques can consistently yield performance enhancements across these open-source LLMs.
5https://github.com/google-research/ google-research/tree/master/ul2. We only test on StrategyQA since FLAN-UL2 is fine-tuned on labeled data from other datasets, violating the few-shot setting.
6https://huggingface.co/meta-llama/Llama-2-7b.

heading: 4.4 Study on Explanation-aware Ensemble
text: We perform additional experiments to further understand the benefit of the explanation-aware ensemble, and the result is shown in Table 3. Performance w/ Different Scoring Methods We first compare our LLM-based explanation scorer with a few alternatives including (1) lexical scoring, which estimates the reliability of explanations via the lexical gap (Ye and Durrett, 2022), and (2) NLI Scoring that uses an NLI model to verify the reliability of explanations. We use MT5XXL (Xue et al., 2021) fine-tuned on NLI datasets as the scorer. Overall, we observe that our model outperforms these models in most cases, indicating that LLM has a strong capacity for estimating the quality of the explanations with only a few demonstrations. In addition, we observe that pairwise scoring does not perform well for weighting the predictions. This is because it was originally proposed for text ranking, different from our scenarios in terms of input formats and relevance signals. Performance w/ Different Bootstrapping Strategies To justify the design of leveraging the Explainthen-Predict (EP) pipeline to generate negative
demonstrations, we also consider other ways including removing demonstrations as well as using the Predict-then-Explain (PE) pipeline. Overall, in many cases, using the EP pipeline leads to better results, as we observe that the PE pipeline sometimes causes the false negative issue: it will first generate incorrect predictions but followed with reasonable explanations. However, when the model performs reasonably well (e.g. PaLM 2-L on OpenbookQA), then it may make less erroneous prediction during the bootstrapping step, leading to insufficient training signals for EASE to perform well. In addition, no matter whether PE and EP are used, they both largely outperform the baseline where no demonstration is given, necessitating the role of demonstration for explanation-aware ensembling. EASE Assigns Higher Scores for Reliable Explanations To justify that better scores are assigned to explanations with correct answers, we calculate the average score for explanations associated with correct and incorrect predictions7. The results in Fig. 3 show that EASE leads to a higher average score for those explanations with correct answers. We further justify this point in Sec. 4.6 human study.

heading: 4.5 Study on Soft Probability Aggregation
text: The premise behind soft probability aggregation is the potential inaccuracy in the prediction token due to temperature sampling variability. To verify this, we calculate the proportion of cases where the prediction token pi is different than the prediction pi ̸= argmax p(·|P, x, ei). As exhibited in Table 4,
7To eliminate the effect of the sampling randomness, we calculate the prediction based on the soft probability in Eq. 4.
we observe that such inconsistency predictions appear in 10% to 15% of the cases, which is not rare in practice. By using the soft score, we observe that it consistently leads to performance boosts. The gain is more evident when the inconsistency issue is more severe — on E-SNLI dataset with PaLM 2-S as the backbone, around 15% examples have inconsistent predictions. Incorporating soft probability aggregation leads to a notable performance gain (from 68.68% to 73.84%). When compared to other methods for prediction correction, such as using hard predictions (i.e. argmax p(·|P, x, ei)) or generation probability conditioned on different verbalizers, EASE achieves better performance. More case studies on using soft probabilities are deferred to Appendix F.2.

heading: 4.6 Human Study on Explanations
text: We conduct additional human studies to further investigate whether the scores generated by LLM are aligned with human preferences. On two datasets, we randomly select 80 instances for evaluation.

heading: Faithfulness and Consistency Evaluation. To
text: provide in-depth analysis, we follow (Ye and Durrett, 2022) to consider two dimensions: (1) faithfulness: whether the explanation is grounded in the corresponding input context; (2) consistency: whether the explanation entails the prediction. We ask human raters to indicate the ‘faithfulness’ and ‘consistency’ of the explanations. The results are shown in Figure 4(a). For faithfulness, we observe a higher average score from the LLM scorer to those identified faithful explanations. For consistency, we observe that there is a significant drop in the inconsistency ratio when soft probability aggregation is used. These results further justify that two modules in EASE can collectively reduce the effect of unreliable explanations. Pairwise Evaluation. For each instance, we sample two explanations with different predictions as {(e1, p1), (e2, p2)}, with one being correct. We
compare our approach and two baselines (NLI model, lexical overlap) with human raters: for each pair of explanations, we first ask four humans to determine which explanation is better and use ci (i = 1, 2) to denote the number of raters that select ei as the better one. Then, we use different models to estimate the score for explanations separately, denoted as (se1 , se2). The final judge of “Win-Tie-Lose” is determined to be:
r = { win, if (c1 > c2 and se1 > se2 ) or (c1 < c2 and se1 < se2 ); tie, c1 = c2; lose, if (c1 < c2 and se1 > se2 ) or (c1 > c2 and se1 < se2 ).
The final results are shown in Figure 4(b). The cohen’s kappa among human raters are 0.75 (E-SNLI) and 0.64 (StrategyQA), which stands for “substantial agreement”. Overall, EASE aligns with human preferences the best, indicating its better ability to be the proxy for explanation quality estimation. We display more examples of generated explanations and the scores in Appendix F.1.

heading: 4.7 Results on More Challenging Tasks
text: We provide additional experiments to show that EASE can be applied to more challenging tasks where the initial explanation and predictions are not good enough. For all experiments, we use LLama2-7b as the backbone LLM. Cross-Lingual Transfer We extend our framework to multi-lingual NLI tasks, where we provide the demonstrations and explanations in English and inference on target languages including German (DE), French (FR), Spanish (ES), and Chinese (ZH) using the examples in XNLI dataset (Conneau et al., 2018). The performance (accuracy) of our method, as well as our direct baseline (self-consistency), are shown in Table 5. From the table, we observe that our method achieves a 7.4% performance gain when compared to the self-consistency baseline, demonstrating our method can be readily applied to challenging cross-lingual scenarios.
Arithmetic Reasoning For arithmetic reasoning, one difference is that this task requires the generation of a candidate answer in the form of a numerical value, rather than simply outputting a class or choice. While direct adoption of soft probability aggregation may not be feasible, we can still leverage the bootstrapped LLM scorer to assign weights to different outputs generated by LLMs. To evaluate our approach, we conducted experiments on GSM8k (Cobbe et al., 2021) and SVAMP (Patel et al., 2021). The results of our direct baseline (self-consistency) and our proposed method are summarized in Table 6. From these results, it is evident that incorporating the LLM as an explanation (reasoning) scorer can further enhance the reasoning ability of the LLM.

heading: 4.8 Additional Studies
text: As EASE relies on several key components such as prompts and sampling steps, we study their effect on the final prediction, using PaLM 2-S as the backbone.

heading: Effect of the Sampling Temperatures and
text: Prompt Templates We study the robustness EASE to different prompt templates by choosing three different prompt formats from (Bach et al., 2022) (the details are in Appendix C.3) on two datasets. Overall, from Figure 5(a) we observe that EASE is robust to them as all of the prompt formats lead to performance gains when compared to the strongest baseline self-consistency. Similarly, in Figure 5(b),
we observe that EASE also performs better than baseline under all temperature settings, further justify its robustness across different settings.
Effect of the Size of Demonstrations K Figure 6 illustrates the performance with different size of demonstrations. By increasing the number of K, the performance gradually increases, while EASE achieves performance gains under all value of K.
Effect of the Number of Generated Explanations N In Figure 7, we examine the influence of the number of explanations. On both datasets, increasing the explanations generally improves the performance, while EASE achieves better performance than the baselines using only 30% - 40% of the generated explanations, which can reduce the burden of sampling massive explanations while maintaining the performance.

heading: Effect of Verbalizers for Bootstrapped LLM
text: Scorer We investigate the role of verbalizers in representing the “positive” and “negative” explanations. We consider three set of verbalizers, namely V1:“Yes” and “No”, V2: “True” and “False”, and V3: “Foo” and “Jaa” using symbolic tuning (Wei et al., 2023a). Using PaLM 2-S as the backbone, the experimental results are shown in Table 7. From the results, we observe that the original “Yes” and “No” generally perform better. Symbolic tuning does not work as well as other verbalizers with concrete semantics, indicating it may not be strong enough for the explanation scoring task.

heading: 5 Conclusion
text: In this work, we empower LLM’s in-context learning ability with natural language explanations. Specifically, we design explanation-aware ensemble to weight multiple predictions using their associated explanations and realize this idea using a bootstrapped LLM scorer. In addition, we leverage a soft probability aggregation scheme to mitigate the issue of inconsistent predictions for ensembling. We conduct extensive experiments on seven datasets from a diverse task set and show our proposed framework can outperform previous stateof-the-art methods using four LLMs as backbones.

heading: Acknowledgements
text: We would like to thank reviewers and AE from the ACL Rolling Review for the helpful feedback. This work was supported in part by NSF IIS-2008334 and CAREER IIS-2144338.

heading: Limitations
text: In this work, our primary goal is to identify the existing issues to better leverage explanations to empower in-context learning. While our approach has shown promise, it also comes with increased computational demands, as both explanation-aware ensemble and soft probability aggregation steps require additional computation overhead8. Future work could explore designing more powerful prompts to let LLMs directly output the suffix tokens as quality score (Tian et al., 2023). Additionally, our methodology depends on the logits returned in both the explanation-aware ensemble and soft probability aggregation processes, making it less suitable to directly adapt to black-box LLMs (e.g. ChatGPT, OpenAI (2023)). To approximate the soft score, one strategy is to set the temperature to a non-zero value and conduct multiple sampling steps, then use the frequency of the corresponding verbalizers as the proxy of the score.
Besides, the key assumption of EASE is that different explanations are of diverse quality, while those explanation leads to correct predictions tend to be of higher quality. We mainly conduct empirical experiments to support this point, yet there often exists multiple facets to evaluate the quality of free-text explantions (Chen et al., 2023a,b; Sun et al., 2022). More in-depth metrics are needed to
8However, in Appendix E, we show that under the same inference time budget, our method still outperforms the best baseline (i.e. self-consistency).
faithfully evaluate the quality of free-text explanations and reveal the true inner workings of EASE.
We also assume that the LLM must possess a baseline of language modeling and reasoning capabilities for effective in-context learning. For tasks where in-context learning is not viable, alternatives such as continuous pretraining and task-specific fine-tuning might be more suitable to enhance the LLM’s task-specific knowledge. Besides, our experiments, which included a range of LLMs from Llama-2 to PaLM 2 and cover various NLP tasks, demonstrate that these models generally exhibit some level of reasoning capacity, performing better than random guesses.
Additionally, while EASE augments in-context learning by weighting predictions through explanations, it does not refine the explanation’s content. For future works, it is potential to leverage techniques such as self-refinement (Madaan et al., 2023; Ling et al., 2023) and debating (Du et al., 2023) to elevate explanation quality and strengthen the model’s reasoning abilities.

heading: Ethics Considerations
text: We acknowledge the risk that the explanations from large language model may inherit systematic biases from their pretraining data and contain incorrect information. Our approach aims to mitigate this by identifying the most reliable explanations. This process does not eliminate the possibility of bias, but offers a method to reduce its impact, emphasizing the need for future work on improving the way these models are trained and utilized.

heading: A Datasets Details

heading: A.1 Datasets Used in Main Experiments
text: The seven benchmarks in our experiments are all publicly available. The details for these datasets are included as follows:
• E-SNLI (Camburu et al., 2018) is an enriched version of the Stanford Natural Language Inference (SNLI) corpus (Bowman et al., 2015), augmented with human-annotated natural language explanations for entailment relations;
• ANLI-R1/R2/R3 (Nie et al., 2020) are a set of three collections of adversarially generated NLI examples curated through a human-in-the-loop process;
• ECQA (Aggarwal et al., 2021) is built upon CommonsenseQA benchmark (Talmor et al., 2019) and contains additional human-annotated question explanations;
• OpenbookQA (Mihaylov et al., 2018) is a QA dataset that requires comprehensive reasoning from open-book sources. As no ground-truth explanations are given, we use the provided facts as the proxy explanations.
• StrategyQA (Geva et al., 2021) focuses on reasoning over complex, multi-hop questions that often require strategic planning.

heading: A.2 Additional Datasets
text: • XNLI (Conneau et al., 2018) is a cross-lingual NLI dataset, designed to evaluate language understanding across different languages. It extends the Stanford NLI (SNLI) dataset by providing translations of the English examples into various languages.
• GSM8k (Cobbe et al., 2021) provides a diverse set of questions that require logical reasoning, which focuses on grade-school level mathematical problems.
• SVAMP (Patel et al., 2021) is centered around assessing models in solving arithmetic word problems, specifically designed to test the variation in problem phrasing.

heading: A.3 Download Links
text: Below are the links to downloadable versions of these datasets.
• E-SNLI: https://huggingface.co/ datasets/esnli;
• ANLI R1/R2/R3: https://github.com/ facebookresearch/anli;
• ECQA: https://github.com/allenai/feb;
• OpenbookQA: https://huggingface.co/ datasets/openbookqa;
• StrategyQA: for StrategyQA we use the question-only set from the link https:// github.com/google/BIG-bench/blob/main/ bigbench/benchmark_tasks/strategyqa;
• XNLI: https://huggingface.co/datasets/ xnli;
• GSM8k: https://huggingface.co/ datasets/gsm8k;
• SVAMP: https://github.com/arkilpatel/ SVAMP/tree/main/data

heading: A.4 Setups
text: Main Experiment Setups By default, we sample few-shot demonstrations from the train set and sample from the test split for all datasets. For OpenbookQA, as the original dataset only contains 500 test examples, in each split we use 100 examples. For ANLI, as some of the examples contain no explanations, while the explanations for some examples include task-irrelevant information such as ‘I think the computer was confused because so many of the words were similar to the description’. To reduce the effect of such examples, we remove those examples occurs with term ‘the system’, ‘the computer’, ‘the model’, ‘the AI’, and manually checked all the few-shot demonstrations to ensure that there is no such information in explanations. Setups for Multilingual NLI For the multi-lingual NLI task, we simulate a cross-lingual setting where we provide the demonstrations and explanations in English and inference on target languages including German (DE), French (FR), Spanish (ES), and Chinese (ZH) using the examples in XNLI dataset (Conneau et al., 2018). Note that this is one common setting studied in previous multilingual reasoning benchmarks (Shi et al., 2023). We use 16-shot examples from the E-SNLI dataset as demonstrations, and for each target language, we sample 300 instances.
Setups for Arithmetic Reasoning For each dataset, we employed 16 examples from the training set as few-shot demonstrations. We use the prompts provided in existing works (Wang et al., 2023d) to avoid additional prompt engineering.

heading: B Baselines
text: The detailed information for baselines are described as follows.
• Standard In-context Learning (ICL) (Brown et al., 2020): it solely uses the input-label pairs for few-shot learning without using explanations.
• Predict-then-Explain (PE) (Lampinen et al., 2022): it provides the explanation after the label for each instance when constructing demonstrations. During the inference stage, it generates the explanation after the prediction.
• Explain-then-Predict (EP) (Wei et al., 2022b): it is the standard chain-of-thought pipeline which provides an explanation before the label as demonstrations. During the inference stage, it first generates an explanation, then followed by the prediction. Note that for both PE and EP method, we use greedy sampling to obtain the explanation and prediction.
• Self-consistency (Wang et al., 2022b, 2023d): it improves over the standard EP pipeline by aggregating over multiple explanations from LLMs to enhance the result.
• FLamE (Zhou et al., 2023) is a recent LLM few-shot learning method that generates multiple label-conditioned explanations and determines the final prediction based on the label that achieves the highest logit after reviewing all explanations for the given instance9.
• Order-Ensemble (Lu et al., 2022) is an ensemble-based method, which generates multiple outputs via shuffling the order of the few-shot demonstrations, and use the majority voting for ensembling.
• PromptBoost (Hou et al., 2023) is also an ensemble-based method, which leverages AdaBoost algorithm to assign different weights for predictions from different prompts. 9In the original FLamE paper, the RoBERTa is used for
final classification. For a fair comparison, we adjusted FLamE to use the in-context LLM as the classifier.

heading: C Prompt Formats
text: In this section, we list the prompts used in our experiments.

heading: C.1 Prompt Format For In-context Learning
text: In this step, we list the prompt for generating the explanations and predictions. Many of the prompt formats are adapted from (Bach et al., 2022). Note that the blue text is instance-dependent, while the red text is the model’s expected output.
C.1.1 E-SNLI
Listing 1: Prompt Format for E-SNLI dataset, standard in-context learning.
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) Based on the premise: [premise], can we infer the hypothesis: [hypothesis] from the premise?
Choose among Yes , Maybe , and No. Answer: [Answer]
# test examples Based on the premise: [premise], can we infer the hypothesis: [hypothesis] from the premise?
Choose among Yes , Maybe , and No. Answer: [Answer]
Listing 2: Prompt Format for E-SNLI dataset, using predict-then-explain pipeline.
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) Based on the premise: [premise], can we infer the hypothesis: [hypothesis] from the premise?
Choose among Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
# test examples Based on the premise: [premise], can we infer the hypothesis: [hypothesis] from the premise?
Choose among Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
Listing 3: Prompt Format for E-SNLI dataset, using explain-then-predict pipeline.
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) Based on the premise: [premise], can we infer the hypothesis: [hypothesis] from the premise?
Choose among Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
# test examples Based on the premise: [premise], can we infer the hypothesis: [hypothesis] from the premise?
Choose among Yes , Maybe , and No. Explanation: [Explanation] Answer: [Answer]
C.1.2 ANLI
Listing 4: Prompt Format for ANLI dataset, standard in-context learning.
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) Based on the premise: [premise], can we infer the hypothesis: [premise] from the premise?
Choose among Yes , Maybe , and No. Answer: [Answer]
# test examples Based on the premise: [premise], can we infer the hypothesis: [premise] from the premise?
Choose among Yes , Maybe , and No. Answer: [Answer]
Listing 5: Prompt Format for ANLI dataset, using predict-then-explain pipeline.
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) [premise], Based on the previous passage , is it true that [hypothesis]? Choose among Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
# test examples [premise], Based on the previous passage , is it true that [hypothesis]? Choose among Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
Listing 6: Prompt Format for ANLI dataset, using explain-then-predict pipeline.
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) [premise], Based on the previous passage , is it true that [hypothesis]? Choose among Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
# test examples [premise], Based on the previous passage , is it true that [hypothesis]? Choose among Yes ,
Maybe , and No. Explanation: [Explanation] Answer: [Answer]

heading: C.1.3 ECQA & OpenbookQA
text: As both ECQA & OpenbookQA are multi-choice classification tasks, we use the same prompt formats for them.
Listing 7: Prompt format for multi-choice QA, standard in-context learning.
In this task , your job is to first read the question as well as the candidate choices. Then , choose one answer from the choices for the question.
# demonstrations (no more than 48) Given the following options , what do you think is the correct
answer to the question below? Question: [question] Choices: [choices] Answer: [Answer]
# test examples Given the following options , what do you think is the correct
answer to the question below? Question: [question] Choices: [choices] Answer: [Answer]
Listing 8: Prompt format for multi-choice QA, using predict-then-explain pipeline.
In this task , your job is to first read the question as well as the candidate choices. Then , choose one answer from the choices for the question.
# demonstrations (no more than 48) Given the following options , what do you think is the correct
answer to the question below? Question: [question] Choices: [choices] Answer: [Answer] Explanation: [Explanation]
# test examples Given the following options , what do you think is the correct
answer to the question below? Question: [question] Choices: [choices] Answer: [Answer] Explanation: [Explanation]
Listing 9: Prompt format for multi-choice QA, using explain-then-predict pipeline.
In this task , your job is to first read the question as well as the candidate choices. Then , choose one answer from the choices for the question.
# demonstrations (no more than 48) Given the following options , what do you think is the correct
answer to the question below? Question: [question] Choices: [choices] Explanation: [Explanation] Answer: [Answer]
# test examples Given the following options , what do you think is the correct
answer to the question below? Question: [question] Choices: [choices] Explanation: [Explanation] Answer: [Answer]

heading: C.1.4 StrategyQA
text: Listing 10: Prompt format for StrategyQA, standard in-context learning.
In this task , given a question , you need to answer True or False. # demonstrations (no more than 48) For the question: '[question]', do you think it is the True or False ? Answer: [Answer]
# test examples
For the question: '[question]', do you think it is the True or False ? Answer: [Answer]
Listing 11: Prompt format for StrategyQA, using predict-then-explain pipeline.
In this task , given a question , you need to answer True or False. # demonstrations (no more than 48) For the question: '[question]', do you think it is the True or False ? Answer: [Answer] Explanation: [Explanation]
# test examples For the question: '[question]', do you think it is the True or False ? Answer: [Answer] Explanation: [Explanation]
Listing 12: Prompt format for StrategyQA, using explain-then-predict pipeline.
In this task , given a question , you need to answer True or False.
# demonstrations (no more than 48) For the question: '[question]', do you think it is the True or False ? Explanation: [Explanation] Answer: [Answer]
# test examples For the question: '[question]', do you think it is the True or False ? Explanation: [Explanation] Answer: [Answer]

heading: C.2 Prompt Format For Explanation-aware Ensemble
text: Listing 13: Prompt format for LLM Scoring. Note that we use the probability of the ‘Answer’ token as the proxy for the quality score.
In this task , you will be given the input for the [task_name] task , your job is to determine whether the explanation provided is a good one for the given input . Please consider the explanation 's coherence , informativeness , and consistency with the prediction to evaluate its quality.
# demonstrations (no more than 48) For '[task input]', can you determine whether the explanation is a good one
for the given [task]? Explanation: [Explanation] Answer: [Answer] [Yes or No]
# test examples For '[task input]', can you determine whether the explanation is a good one
for the given [task]? Explanation: [Explanation] Answer: [Answer]

heading: C.3 Additional Prompt Format Used in Prompt Sensitivity Study
text: In section 4.8, we have studied the effect of different prompt templates. Here we list them in the following lists.
Listing 14: Prompt Format 2 for E-SNLI dataset
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) Based on [premise], does it follow that [hypothesis]? Choose among
Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
# test examples Based on [premise], does it follow that [hypothesis]? Choose among
Yes , Maybe , and No.
Explanation: [Explanation] Answer: [Answer]
Listing 15: Prompt Format 3 for E-SNLI dataset
In this task , given a premise and a hypothesis , your job is to
determine whether the hypothesis can be inferred from the premise.
# demonstrations (no more than 48) Based on the premise [premise], can we conclude the hypothesis that [hypothesis]? Choose among Yes , Maybe , and No. Answer: [Answer] Explanation: [Explanation]
# test examples Based on the premise [premise], can we conclude the hypothesis that [hypothesis]? Choose among Yes , Maybe , and No. Explanation: [Explanation] Answer: [Answer]
Listing 16: Prompt format 2 for StrategyQA, using explain-then-predict pipeline.
In this task , given a question , you need to answer True or False.
# demonstrations (no more than 48) Answer the question: '[question]', by True or False. Explanation: [Explanation] Answer: [Answer]
# test examples Answer the question: '[question]', by True or False. Explanation: [Explanation] Answer: [Answer]
Listing 17: Prompt format 3 for StrategyQA, using explain-then-predict pipeline.
In this task , given a question , you need to answer True or False.
# demonstrations (no more than 48)
EXAM: Answer by True of False. Question: '[question]' Explanation: [Explanation] Answer: [Answer]
# test examples EXAM: Answer by True of False. Question: '[question]' Explanation: [Explanation] Answer: [Answer]

heading: D Human Evaluation
text: Here we provide the guidelines for human evaluation
Listing 18: Human Evaluation Guideline for E-SNLI dataset.
For this explanation grading task , given the task input (e.g. the premise and hypothesis for the NLI task and the question for the QA task), ground -truth answer ,
as well as a pair of explanations from the LLM , you job is to
determine which explantion will reach the ground -truth answer for that input.
For the E-SNLI dataset , your task is to predict if the hypothesis
is entailed/neutral/contradicts the premise.
Listing 19: Human Evaluation Guideline for StrategyQA dataset.
For this explanation grading task , given the task input (e.g. the premise and hypothesis for the NLI task and the question for the QA task), ground -truth answer ,
as well as a pair of explanations from the LLM , you job is to
determine which explantion will reach the ground -truth answer for that input.
For the strategyQA dataset , your task is to answer the question with 'True ' or 'False '.

heading: E Additional Results on Complexity Comparison
text: In our method, the additional steps — "soft probabilistic aggregation" and "Explanation Scoring" — involve generating only a single token and its probability score. This process is less timeconsuming than the standard explain-then-predict step, which generates both explanations and predictions. Roughly speaking, suppose we generate N explanations for one example during the inference time, requiring t = N ∗D time for decoding (where D is the time for generating both explanations and predictions). Then for each explanation, suppose the decoding time for generating explanation scores and soft probability are d1 and d2, respectively. The additional time for explanation scoring and soft probability aggregation per explanation is t′ = N ∗(d1+d2). As has been discussed above, both d1 and d2 are much smaller than D as it only needs to decode for one token. As a result, the additional time t′ << 2t.
To evaluate the comparative effectiveness of the self-consistency approach when augmented with additional samples to match the overall computational costs of our method. For this purpose, we utilize the PaLM2-S and PaLM2-L models as the backbone Large Language Models. The results are summarized in Table 8, providing a direct comparison between the standard self-consistency method and our EASE approach under similar computational constraints. Overall, we observe that EASE consistently outperforms the self-consistency approach in most scenarios, even when the latter is allocated additional computational resources.

heading: F Additional Studies
text: F.1 Case study on explanation-aware ensemble
Tables 9 and 10 give an example of Explanationaware Ensemble process on E-SNLI dataset. Take the Table 9 as an example, where the original prediction using majority voting is “Entailment”. By leveraging LLM to score each explanation, the
LLM is able to reduce the effect of unreliable explanations: for example, explanation 3 copies a part of the premise without providing additional reasoning, thus it receives a low score (0.369). On the contrary, explanation 4 and explanation 6 correctly identifies the key reasoning step, thus receiving higher scores (0.677 and 0.612). After weighted majority voting, the total score of “Neutral” class is higher than the “Entailment” class, thus we are able to obtain the prediction “Neutral”, which is the ground-truth answer.
F.2 Case study on soft probability aggregation Tables 11 and 12 provide examples of Soft Probability Aggregation applied to the ESNLI and StrategyQA Datasets, respectively. For some examples (e.g. Explanation 4 in Table 11 and Explanation 1 in 12), a mismatch between predicted labels and their respective highest probability scores is observed, indicating the potential for inconsistency in LLM predictions. By aggregating the probability score in a soft manner, we are able to rectify the final predictions, justifying the effectiveness of soft probability aggregation.

heading: Future Work
text: This process does not eliminate the possibility of bias, but offers a method to reduce its impact, emphasizing the need for future work on improving the way these models are trained and utilized