Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting (2023)

Abstract: Recent work has shown how to prompt large language models with explanations to obtain strong performance on textual reasoning tasks, i.e., the chain-of-thought paradigm. However, subtly different explanations can yield widely varying downstream task accuracy. Explanations that have not been “tuned” for a task, such as off-the-shelf explanations written by non-experts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion. We first generate sets of candidate explanations for each example in the prompt using a leave-one-out scheme, then find an effective combination of these explanations with a two-stage framework. We first evaluate explanations for each in-context example in isolation according to two proxy metrics, log likelihood and accuracy on new examples. Then, we search over combinations of explanations to find one that yields high performance against a silver-labeled development set. Across four textual reasoning tasks spanning question answering, mathematical reasoning, and natural language inference, results show that our proxy metrics correlate with ground truth accuracy and our overall method can effectively improve prompts over crowdworker annotations and naive search strategies

## introduction large language models (llms) @xcite can be applied in various ways to do in-context learning (icl). one line of work shows including explanations can boost the prompting performance on a diverse of reasoning tasks @xcite @xcite . 2 despite the utility of such 1 code: https://github.com/xiye17/explselection . 2 our paper uses the general term explanation to denote both chain-of-thought demonstrations for multi-step reasoning tasks as well as rationales for tasks like commonsense question answering, which do not involve chains of intermediate steps in the same way. because we know that amy had 5 apples and alex had 7, the answer is 12. if we add the 5 apples that amy has with the 7 that alex has, then it's 12. ## method overview having candidate explanations for each question, we have reduced the search space from exponential in the vocabulary size to merely n k . we then search over possible combinations of explanations. we describe our method for scoring combinations and the constraints under which our search takes place. ## proxy metrics for finding promising combinations owning to the high cost, we only evaluate a small number (tens of combinations) of combinations against development set using o (eq ( 3) ). we first extract a set of promising combinations according to two proxy metrics, then evaluate those using our silver data. ## effectiveness of proxy metrics before showing the results of the complete system, we first present experiments for verifying the effectiveness of the two proxy metrics. we evaluate them on the basis of the best oracle accuracy on a small (gold) labeled test set that we can reach using the top-x candidates, referred to as max@x, ranked by s osacc or s osll . this gives an oracle upper bound for the performance that silver reranking via o can yield. setup we compare our metrics against a baseline which randomly scores combinations (naive). we mainly use code-davinci-002 for this experiment; please refer to appendix b for additional results on text-davinci-003. for s osacc , we silver-labeled 256 randomly drawn development with 48 samples of combinations. for each dataset, we experiment with four different exemplar sets t to control for randomness and report the average number. results ## related work we study prompting llms with chain-of-thought @xcite @xcite or textual explanations more generally @xcite . much of the past work focuses on exemplar selection in the presence of explanations @xcite or developing prompting methods for various reasoning tasks @xcite , which typically require manually engineered explanations. we focus instead on searching for betterperforming explanations. our approach leverages data without explanation annotations. similarly, prior work also explores the means of using few-show explanations together with data points without explanations annotations for improving downstream performance @xcite @xcite @xcite . many of these techniques need a large amount of fully labeled data to train models used for generating explanations @xcite or smaller models used as verifiers @xcite , whereas our work only uses a small unlabeled set. there is also work on automatically constructing cots @xcite starting zots @xcite , which also requires a fully labeled dataset. in particular, @xcite also use llms to silver labeled data points for finetuning the llms; our work instead treats llms as black-boxes and searches for better explanations instead of tuning the parameters. our work also closely relates to prompt optimization. while experts can potentially engineer better prompts @xcite , such a process requires heavy manual effort. this has attracted growing interest on automated prompt engineering. one line of work requires interacting with gradients @xcite or continuous embeddings @xcite @xcite . another line uses lms as black-boxes @xcite @xcite . however, this past work either optimizes over discrete templates (not applicable for the explanation optimization setting) or optimizes over string verbalizations (a search space too large for our setting). ## conclusion we have presented an approach that can search for better-performing explanations for icl starting from a set of seed explanations. our approach first proposes promising candidate combinations of alternative explanations generated using llms, then finds explanation combinations using proxy metrics before using a silver-labeled validation set to select the best candidate. our results highlight the substantial variance in the performance of different sets of explanations, paving the way for future work to further optimize explanations in this paradigm.