T hink S um: Probabilistic reasoning over sets using large language models (2023)

Abstract: Large language models (LLMs) have a substantial capacity for high-level analogical reasoning: reproducing patterns in linear text that occur in their training data (zero-shot evaluation) or in the provided context (few-shot in-context learning). However, recent studies show that even the more advanced LLMs fail in scenarios that require reasoning over multiple objects or facts and making sequences of logical deductions. We propose a two-stage probabilistic inference paradigm, ThinkSum, which reasons over sets of objects or facts in a structured manner. In the first stage (Think – retrieval of associations), a LLM is queried in parallel over a set of phrases extracted from the prompt or an auxiliary model call. In the second stage (Sum – probabilistic inference or reasoning), the results of these queries are aggregated to make the final prediction. We demonstrate the possibilities and advantages of ThinkSum on the BIG-bench suite of LLM evaluation tasks, achieving improvements over the state of the art using GPT-family models on thirteen difficult tasks, often with far smaller model variants. We also compare and contrast ThinkSum with other proposed modifications to direct prompting of LLMs, such as variants of chain-of-thought prompting. Our results suggest that because the probabilistic inference in ThinkSum is performed outside of calls to the LLM, ThinkSum is less sensitive to prompt design, yields more interpretable predictions, and can be flexibly combined with latent variable models to extract structured knowledge from LLMs. Overall, our proposed paradigm represents a promising approach for enhancing the reasoning capabilities of LLMs.

## introduction large language models (llms; @xcite @xcite can recall a broad range of basic facts, recognize and mimic various forms in language, and efficiently extrapolate analogies in structure and meaning. these abilities allow llms to excel in zero-shot and few-shot tasks formulated as the generation or selection of a likely completion to a prompt. this formulation requires llms to perform fast associative thinking, in which each token of text in the sequence making up the answer is generated or scored in one pass through the model and, other than that, no intermediate information is created or retained. this fast thinking is made possible by the compression of information that is repeated in a variety of ways in large training datasets, within the llm's weights. however, it is increasingly evident that when reasoning, or slow thinking, is required, failure modes of llms are revealed. in our usage, reasoning refers to the sequential manipulation of concepts that can be expressed in language. tasks that require iterative retrieval of rarely stated knowledge, uncertainties over multiple objects or facts, or multiple steps of deduction are difficult even for the most advanced llms @xcite . in a recently designed suite of evaluations, big-bench @xcite , some of the tasks where the gap between machine and human performance is large involve inference sequences with nested counterfactuals (logical deduction), concepts introduced through definitions (conceptual combinations), etc. (see fig. b .1). these are tasks where a human solver's intuitive feeling of '(in)coherence' is insufficient to produce the right answer, and a sequence of thoughts, along with the use of intermediate results, may be necessary to arrive at the solution, particularly when working memory is insufficient. we show several tasks in big-bench that can be addressed by a two-component mechanism, which we name thinksum 1 : ## experiments in this section, we perform case studies on three tasks from the big-bench suite to demonstrate the possibilities of the inference approaches discussed in §2. we also experiment with ten other tasks from big-bench; the best results are summarized in all details of the tasks can be found in the appendix ( §c). comparisons to direct prompting and algorithms that append retrieved or generated tokens to the prompt are given in §3.4. ## related work improvements to llm inference. after the discovery of the in-context learning abilities of llms, there has been an explosion of interest in improving inference with llms in the zero-shot and few-shot setting @xcite @xcite . one approach to improving the reasoning abilities of llms involves appending, or learning to generate, auxiliary knowledge within the prompt @xcite @xcite . recently, more general auxiliary knowledge or chain-of-thought prompting methods have been proposed @xcite @xcite @xcite , including those that allow a control flow external to the main llm @xcite . later, @xcite showed zero-shot chain-of-thought prompting can improve performance on a variety of reasoning tasks. this method does not require any hand-crafted few-shot examples, which is a shared property with thinksum. @xcite observed that a dual-system approach where an associative "system 1" and a logical "system 2" can increase coherence of llms in tasks such as robust story generation and grounded instruction following. the two-step paradigm in thinksum is similar, where "system 1" is the (querying of the llm for) fast thinking, and "system 2" is the probabilistic inference step. brittleness of chain-of-thought prompting. despite the recent success of chain-of-thought approaches, recent studies have raised concerns regarding the limitations of chain-of-thought approaches. @xcite observed that instructive prompts perform similarly with misleading or intentionally irrelevant prompts. additionally, @xcite showed improvements due to few-shot chain-of-thought are not observed in question answering, or natural language inference. more critically, few-shot prompts are highly sensitive to the order in which the samples are provided, the prompt format, and the selection of in-context examples, @xcite . thus, it is crucial to design techniques that are robust to such changes in the prompt. inference as reasoning. iterative inference over llm outputs has been proposed for tackling true/false question answering and commonsense question answering @xcite . @xcite presents a bayesian inference perspective on in-context learning, and @xcite formalizes and unifies existing prompting techniques in a probabilistic framework. our work generalizes such approaches to perform arbitrary probabilistic inference outside of the llm. ## conclusion in this paper we presented thinksum, a two-step probabilistic inference paradigm that reasons over sets in a structured manner. the fast thinking stage of thinksum allows elementary string manipulations as well as natural language prompting, which may enable numerous approaches to solve a natural language task. even with far smaller model variants, thinksum achieves state-of-the-art results on ten difficult tasks in big-bench using gpt-family models. the two-step paradigm allows operating over sets instead of manipulating the prompt itself, preventing sensitivity to prompt format during the probabilistic inference in thinksum, which is performed outside of calls to the llm. as a result, thinksum is more robust to prompt design, yields more interpretable predictions, and can be combined with many probabilistic inference approaches to tackle a diverse set of tasks.