Automatic Model Selection with Large Language Models for Reasoning (2023)

Abstract: Chain-of-Thought (CoT) and Program-Aided Language Models (PAL) represent two distinct reasoning methods, each with its own strengths. CoT employs natural language, offering flexibility and interpretability, while PAL utilizes programming language, yielding more structured and rigorous logic. We introduce a model selection method to combine the best of both worlds by employing a large language model (LLM) to dynamically select between them. Our theoretical analysis underscores the feasibility of this method, which is further corroborated by empirical results. Our proposed method demonstrates significant performance improvements across eight reasoning datasets with Codex, ChatGPT, and GPT-4. Additionally, our method is complementary to self-consistency; when integrated, it can further enhance performance while significantly reducing computation costs. Moreover, we achieve new state-of-the-art results on GSM8K and SVAMP, with respective accuracies of 96.8% and 93.7%.

## introduction large language models (llms) have made impressive progresses in numerous fields @xcite @xcite @xcite and are often powerful enough to solve problems through a single unified method. while convenient, this approach tends to ignore the distinct structures and variations among the problems, which would benefit from using different methods. on the other hand, in human society, individuals with different skill sets excel in various roles, leading to a thriving world as a whole. in the case of reasoning, chain-of-thought (cot) @xcite and program-aided language models (pal) (gao et al., 2022; chen et al., 1 our code, data and prompts are available at https:// github.com/xuzhao0/model-selection-reasoning 2022) have emerged as two effective methods that offer different strengths and weaknesses. essentially, cot decomposes a reasoning problem into a series of intermediate steps using natural language, making it more general, flexible, and comprehensible. on the other hand, pal offers solutions via python functions, with its step-by-step programming code ensuring a more rigorous and structured methodology. the external python interpreter further guarantees the computation precision. intuitively, combining the strengths of both models and selecting between them based on their solutions to a specific problem would lead to enhanced performance. however, without access to the ground truth, choosing a better method itself becomes a machine learning problem. in order to select among multiple solutions, previous studies have suggested training a ranker @xcite . while training a dedicated model generally results in improved accuracy, it can also be somewhat cumbersome and entail significant costs. conversely, large language models (llms) have demonstrated good calibration and have been used to assess the accuracy of their own outputs @xcite @xcite . in light of this, we propose leveraging the in-context learning capabilities of llms for model selection. we direct llms to choose between two distinct reasoning models and elucidate their selection rationale. we present a detailed theoretical analysis to validate our approach. the analysis highlights two primary factors impacting the effectiveness of our method: (1) the significance of the difference between the two models, i.e. the distinct distributions of their respective probabilities of correctly solving problems, and (2) the probability of selecting the correct model. particularly, a higher overall performance can be attained when there is a substantial difference between the models being considered, and when there is a high probability of the correct figure 1 : we propose to perform model selection to combine two distinct methods, cot and pal . the figure illustrates an example where pal makes mistakes about crucial information and therefore fails to answer the question correctly. in contrast, cot manages to correctly answer the same question. our selection model successfully chooses the correct solution and provides a brief explanation to support its choice. model being selected. furthermore, our analysis affirms that even without an exceptional model selector, we can still achieve improvement in certain cases. this reinforces our decision to simply employ an llm for model selection. we evaluate our method across eight reasoning tasks, with cot and pal serving as the baseline methods. our method consistently attains performance improvement, when employing codex @xcite , chatgpt, and gpt-4 as backbone llms. its broad applicability is further underscored by performance gains with open-source llama 2 models @xcite . in the context of multiple sample aggregations, our approach significantly reduces the computation costs while achieving notable performance enhancements. moreover, our approach attains new state-of-the-art accuracies of 96.8% and 93.7% on gsm8k @xcite and svamp @xcite , respectively. ## automatic model selection with large language models in this study, we examine reasoning tasks using two baseline models: cot and pal. to tackle complex reasoning tasks, cot leverages an llm to generate several intermediate reasoning steps before arriving at a final answer. due to its reliance on natural language, the reasoning steps are clearly ex-plained. furthermore, natural language supports a broad range of reasoning that may involve common sense and confidence in the reasoning steps. but it might struggle with intricate logical relationships. in contrast, pal takes a structured and accurate approach, breaking down reasoning problems using python code. however, the deterministic program format of pal constrains its generalizability and flexibility. ## analysis in this section, we provide a few analyses to see when and how our method works. ## conclusion we introduce a method that effectively combines two distinct models, cot and pal, by using llms for the model selection. our theoretical analysis supports the feasibility of such model combination, which is validated by empirical results. our method achieves performance improvements across eight datasets with various backbone llms. we reduce the computation cost while attaining notable performance improvement with self-consistency. we set new sota results on gsm8k and svamp. this research represents a significant step towards tapping into the potential of diversity and collaboration among models in llms. in our future work, we aim to expand this framework to more reasoning tasks and possibly other domains. an intriguing concept involves exploring the use of diverse system instructions to elicit varying model behaviors for model combinations. furthermore, a genuine combination of specialized models, each with its strengths and expertise, offers a promising avenue for exploration.