L a RS : Latent Reasoning Skills for Chain-of-Thought Reasoning (2024)

Abstract: Chain-of-thought (CoT) prompting is a popular in-context learning (ICL) approach for large language models (LLMs), especially when tackling complex reasoning tasks. Traditional ICL approaches construct prompts using examples that contain questions similar to the input question. However, CoT prompting, which includes crucial intermediate reasoning steps (rationales) within its examples, necessitates selecting examples based on these rationales rather than the questions themselves. Existing methods require human experts or pre-trained LLMs to describe the skill, a high-level abstraction of rationales, to guide the selection. These methods, however, are often costly and difficult to scale. Instead, this paper introduces a new approach named Latent Reasoning Skills (LaRS) that employs unsupervised learning to create a latent space representation of rationales, with a latent variable called a reasoning skill. Concurrently, LaRS learns a reasoning policy to determine the required reasoning skill for a given question. Then the ICL examples are selected by aligning the reasoning skills between past examples and the question. This approach is theoretically grounded and compute-efficient, eliminating the need for auxiliary LLM inference or manual prompt design. Empirical results demonstrate that LaRS consistently outperforms SOTA skill-based selection methods, processing example banks four times faster, reducing LLM inferences during the selection stage by half, and showing greater robustness to sub-optimal example banks.

## introduction large language models (llms) exhibit remarkable capabilities in solving various downstream tasks through in-context learning (icl) @xcite , even without being explicitly trained on the distribution of in-context examples @xcite @xcite @xcite . using in-context learning, llms generate output for an input query by conditioning on a prompt that contains a few input-output demonstrations. reasoning tasks have proven to be particularly difficult for language models and nlp in general @xcite @xcite . in the recent literature, chainof-thought (cot) prompting, an icl method, has been proposed to improve llms on a wide spectrum of reasoning tasks by guiding llms to produce a sequence of intermediate steps (rationale) for generating a (better) final answer @xcite @xcite . the prompts for cot are composed of demonstrations that contain not only input and output, but also the rationales for why the output holds. the core challenge for icl lies in designing effective demonstrations to prompt llms. much evidence has indicated the significant impact of demonstrations on the performance of icl @xcite . to form a prompt, one important setting considers selecting demonstrations from an existing example bank, termed demonstration selection @xcite . while a variety of methods exist in the icl literature for automating this process, cot prompts are distinct in that they include not only questions and answers but also specially-designed rationales. this distinction highlights the importance of rationales in selecting demonstrations for cot prompting. specifically, cot prompting should select demonstrations that illustrate relevant skills within their rationales to effectively address a given question. for instance, in solving math word problems (as depicted in fig. 1 ), a useful rationale involves computing addition to get the correct answer. selecting few-shot examples based on the question similarity (fig. 1a ) might lead to examples showcasing subtraction and generate incorrect rationales. however, skill-based selection (fig. 1b ) can align the skills between examples and the given question, which leads to correct answers guided by relevant rationales. to achieve such a skill-based demonstration selection, @xcite introduces skill-knn, which employs pre-trained llms to generate skill descriptions. then, the few-shot examples are selected based on the embedding of the skill descriptions computed by another pre-trained embedding model. although this approach is straightforward, the llm-generated skill descriptions can be somewhat arbitrary, heavily relying on the manually crafted prompts. this reliance constrains its wider applicability across diverse reasoning tasks. moreover, the approach requires to generate a unique skill description for each example, which limits its scalability to larger example banks. rather than relying on llms, we introduce latent reasoning skill discovery (lars), a new skill-based demonstration selection method. this approach learns skills as latent space representations of rationales through unsupervised learning. the essence of lars lies in a unique formulation for the generation of rationales, which we term the latent skill model. this model, inspired by the principles of topic models (xie et al., 2021a), conditions the generation of a rationale on both a given question and a latent variable, called a reasoning skill. this latent variable embodies a high-level abstraction of the rationales, such as formats, equations, or knowledge. under the skill model formulation, lars utilizes a conditional variational auto-encoder (cvae) to approximate the generation of rationales on a small dataset from the example bank. as a result, two probabilistic models can be learned concurrently: the efficacy of lars is evaluated on four different benchmarks based on five backbone llms with varying scales. the method is also compared with baseline approaches, including an oracle method that assumes access to ground truth rationales. lars consistently outperforms skill-knn and also matches the oracle performance in almost half of the experiments. in addition, lars reduces half of the llm inference, eliminates the need of human prompt design, and maintains better robustness to sub-optimal example banks. a summary of this paper's contribution is as follows: ## formulation in this section, we formally describe the skill model, a new formulation for explaining the generation of rationales in cot reasoning. in section 3.1, the skill model is first introduced to describe the human-generated rationales. then, section 3.2 illustrates how the skill model can be adapted to llm-generated rationales. finally, leveraging the concept of reasoning skill as outlined in the skill model, a new latent-skill-based demonstration selection method is formally described in section 3.3. ## method to enable the skill-based demonstration selection (definition 1), we introduce our approach lars , which involves learning a conditional variational autoencoder (cvae) to approximate p e using the data from the example bank d e . we then outline a practical demonstration selection process aligning with the skill-based selection. the schematic overview of lars (right) and the corresponding demonstration selection process (left) are illustrated in figure 2 . ## experiments this section describes the experimental settings, baselines, metrics, and main results. ## conclusions this paper introduces lars, a novel demonstration selection method designed for cot prompting. lars bases the selection on reasoning skills, which are latent representations discovered by unsupervised learning from rationales via a cvae. based on the experiments conducted across four llms and over four different reasoning tasks, lars manifests comparable performance on selecting effective few-shot examples for cot reasoning while requiring no extra llm inference and saving hours in pre-processing the example bank.