What Are the Odds? Language Models Are Capable of Probabilistic Reasoning (2024)

Abstract: Language models (LM) are capable of remarkably complex linguistic tasks; however, numerical reasoning is an area in which they frequently struggle. An important but rarely evaluated form of reasoning is understanding probability distributions. In this paper, we focus on evaluating the probabilistic reasoning capabilities of LMs using idealized and real-world statistical distributions. We perform a systematic evaluation of state-of-the-art LMs on three tasks: estimating percentiles, drawing samples, and calculating probabilities. We evaluate three ways to provide context to LMs 1) anchoring examples from within a distribution or family of distributions, 2) real-world context, 3) summary statistics on which to base a Normal approximation. Models can make inferences about distributions, and can be further aided by the incorporation of real-world context, example shots and simplified assumptions, even if these assumptions are incorrect or misspecified. To conduct this work, we developed a comprehensive benchmark distribution dataset with associated question-answer pairs that we have released publicly.

## introduction language models (lms) @xcite @xcite are versatile interfaces to knowledge, capable of remarkably complex linguistic tasks. summarization of complex documents @xcite , reasoning over long passages of text @xcite @xcite and zero-shot inference in specialist domains such as medicine @xcite are a few examples that demonstrate their abilities. while performance on primarily linguistic problems, can often be strong, effectiveness on operations that involve numerical reasoning is a domain that language models have struggled with @xcite figure 1 : lms & probabilistic reasoning. models can make inferences about distributions, but can be aided by the incorporation of real-world context, example shots and simplified assumptions, even if these assumptions are incorrect or misspecified. 1 11712 2022). difficulties handling numbers may be due to model pretraining formulations (e.g., using autoregressive next token prediction pretext tasks) or numerical token representations not necessarily being suited to mathematical reasoning @xcite , or simply a limited representation of these types of tasks in the training corpora. nevertheless, some work suggests prompting techniques can substantially improve lm performance on numerical reasoning tasks, indicating that relevant knowledge may already be encoded within these models @xcite . a form of numerical reasoning that is important for interpreting many different forms of data is contextualizing an individual measurement or measurements (a sample or samples) within a population (a distribution). drawing insights from data frequently requires comparing and contrasting a sample from other samples. this is because absolute values in isolation can be hard to interpret, without the context of how probable they are or how close they are to the maximum or minimum values observed across the population. probabilistic reasoning is something that the human brain appears to do @xcite and that is an important component in cognition @xcite . thinking probabilistically is efficient as one does not have to represent every detail of every sample that one observes, and instead can have the data summarized with a small number of parameters that describe the distribution @xcite . research has shown that some probabilistic reasoning processes lead to superior performance; for example, people are more accurate at answering questions about statistical properties when they estimate the full distribution first @xcite . yet, there are limited examples evaluating or improving on the probabilistic reasoning by designing lms that reason over sets @xcite . understanding the distributions is important in many contexts. in population level data it is important when gauging whether an individual behavior is normative (e.g., is sleeping 8 hours normal for a college aged student?). in climatology, inferences about distributions of temperature or precipitation data on a given day of the year at a particular location are important when determining if observed events are typical or abnormal. is a maximum temperature of 35°c likely to be observed in seattle every year? in this work we propose and define a set of proba-bilistic reasoning tasks and use them to evaluate the capabilities of lms -estimating percentiles, drawing samples, and calculating probabilities. next we evaluate the impact of additional real-world context and parametric assumptions (normal distribution) using the task of estimating percentiles (task choice is motivated in section 3). to summarize our research questions: to answer these questions we develop a distribution dataset with associated question-answer pairs that we will release publicly. the dataset includes questions about 12 families of standard, idealized distributions (e.g., normal or power-law distributions) and distributions of real-world data from the domains of population health, climate, and finance. code and additional results for our work can be found here: https://github.com/yahskapar/ llms-and-probabilistic-reasoning . ## defining probabilistic reasoning tasks our probabilistic reasoning benchmark contains three distinct tasks that explore a language model's (lm's) context-free (idealized) understanding of basic, idealized distributions, we describe these tasks as follows: task 1: estimating percentiles. given a distribution, the model is asked for the percentile a sample would appear in. a question is composed of a value (a sample) that is calculated given the target percentile. the answer is expected to be a numerical response from 0 to 100. the target percentiles utilized were n st/@xmath0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 99}. language models responses to these questions are sampled 10 times with a random seed. return the percentile of the value {x} within a normal distribution with mean {y} and standard deviation {z}. task 2: drawing samples. given a distribution the model is asked to draw samples at random from it. a random seed is used for each sample and we repeat this 1000 times per distribution. the language model is explicitly instructed to avoid generating any code or using additional tools to perform the sampling. the answer is expected to be a numerical response. sample a number from the normal distribution with mean {x} and standard deviation {y}. task 3: calculating probabilities. given a distribution the model is asked for the probability a sample from the distribution will fall between two given values. the target probabilities and the corresponding target ranges are computed based on a lower and upper quantile to form examples with different probabilities in the set @xmath1.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}. the answer from the lm is expected to be a numerical response from 0 to 1. calculate the probability that a value falls between {w} and {x} in a normal distribution with mean {y} and standard dev. {z}. in order to evaluate the impact of real-world context on lm probabilistic reasoning, we explore distributions in the real-world that have additional real-world context (e.g., prior knowledge and expectations about us household incomes). we then leverage task 1 of estimating percentiles to evaluate to what degree real-world context impacts performances. task 1 is particularly well suited for this exploration, as we later demonstrate that this task 1 elicits the highest variation in performance across distribution families and the number of incontext examples/shots (further detailed in section 6). in appendix a we provide full prompt templates for our three proposed benchmark tasks, as well as further details regarding template components and real-world distribution prompt templates. ## curating data distributions we use two distinct datasets for the purpose of understanding the probabilistic reasoning capabilities of lms -a dataset of idealized distributions and a dataset of real-world distributions. a visualization of both datasets is shown in figure 2 . ## experimental results we organize our results based on the research questions posed in section 1. alongside a concise answer in bold, we provide discussion based on our analysis of the experimental results. ## conclusion lms are able to answer questions about idealized and real-world distributions, with real-world results suggesting there is some internal representation that enables modeling or interpolation from distribution parameters. the probabilistic reasoning performance of lms varies, with certain distributions (e.g., uniform, normal) having much better performance in contrast to other distributions 8 (e.g., log-normal, skew-normal). within distribution shots and context can improve probabilistic reasoning performance, as can a simplified normal approximation.