Program-Aided Reasoners (Better) Know What They Know (2024)

Abstract: Prior work shows that program-aided reasoning, in which large language models (LLMs) are combined with programs written in programming languages such as Python, can significantly improve accuracy on various reasoning tasks. However, while accuracy is essential, it is also important for such reasoners to “know what they know”, which can be quantified through the calibration of the model. In this paper, we compare the calibration of Program Aided Language Models (PAL) and text-based Chain-of-thought (COT) prompting techniques over 5 datasets and 2 model types - LLaMA models and OpenAI models. Our results indicate that PAL leads to improved calibration in 75% of the instances. Our analysis uncovers that prompting styles that produce lesser diversity in generations also have more calibrated results, and thus we also experiment with inducing lower generation diversity using temperature scaling and find that for certain temperatures, PAL is not only more accurate but is also more calibrated than COT. Overall, we demonstrate that, in the majority of cases, program-aided reasoners better know what they know than text-based counterparts.

## introduction as language models (lms) grow in size and capabilities, several works examine methods to improving their reasoning skills with different styles of prompting @xcite @xcite @xcite . one representative method, chain of thought (cot) reasoning @xcite , takes inspiration from how humans approach problem-solving -by breaking down the problem into a sequence of natural language explanations before arriving at a final answer. furthermore, prompts that enable problem-solving are not limited to natural 1 anonymized code and data are available at https:// anonymous.4open.science/r/code-calibrates-a61d/ . q : 4 people share a room. each month the electricity bill is @xmath0100 each month, we multiply this amount by 12 (months in a year) to find the total cost per year: @xmath11200 we divide the total cost by the number of roommates to find how much each roommate will pay: @xmath21200 the answer is $1200 in this work, we examine the effect of programaided reasoning on calibration. we consider five datasets that cover different reasoning tasks and evaluate the performance of both pal and cot style prompting for openai models @xcite and llama models @xcite with respect to accuracy and calibration. we primarily explore three main research questions : does the consistency of llm generations affect calibration? we examine this by measuring generation diversity and answer space entropy. our results show that program-aided reasoners know what they know even better than standard text-based reasoners with cot. in particular, on openai models, pal exhibits not only superior accuracy but also a consistent enhancement in calibration of about 50%, over cot. interestingly, the consistent improvement of calibration is not observed in llama models. still, we find that adjusting the temperature of sampling (similar to a widely used method of platt scaling (platt et al., 1999), pal improves with respect to accuracy and calibration. we also conduct a detailed analysis of these observations and find a correlation between the similarity of the generated chains-of-thoughts 098 or programs and calibration, which might help ex-099 plain these trends. code and data available here 100 under the apache 2.0 license. 101 2 preliminaries and mathematical 102 formulation 103 2.1 measuring calibration 104 calibration refers to the alignment between the pre-105 ## results we investigate two model types: openai models and llama models along with the two different prompting strategies -pal and cot. ## conclusion in this study, we explore the impact of two distinct prompting styles, namely pal and cot,