Information Directed Tree Search: Reasoning and Planning with Language Agents

Venue: NeurIPS_cc_2024_Workshop_BDU
Published: 2024-09-06 12:35:59

Abstract: Solving challenging tasks often require agentic formulation of language models that can do multi-step reasoning and progressively solve the task by collecting various feedback. For computational efficiency, it may be advantageous to quantify the information associated with different feedback and guide the search such that the solution can be obtained quickly. To explore this possibility, we take a Bayesian approach and propose an \textit{information directed tree search} (IDTS) algorithm that makes use of in-context learning to approximate the information associated with different feedback. We explore the effectivity of IDTS on challenging tasks involving programming, formal math, and natural language. Interestingly, while we find advantages over simple uniform search methods, the proposed approach is about comparable to MCTS even though it explores different paths. We discuss some possibilities for our findings and highlight open questions for future work.

Main paper: 
 introduction: 
 Large language models (LLMs) have become integral for building autonomous agents that aim to find solutions to challenging tasks. Many such tasks often require multi-step reasoning, are not well specified, or require hierarchical decomposition [19, 54] . In such cases, even though LLMs cannot directly provide the complete answer, they can provide partially correct responses, or answer sub-parts of the prompt. Such responses can be then paired with other source(s) of feedback that can subsequently guide the reasoning of the language models. For instance, if the generated response is a code, then the feedback can be provided by program compilers and unit testing [56, 60] . If the generated response is formal proofs for mathematical statements, then auto-verifiers can be used [14, 58] . Further, language models can also be used to self-critique their responses [60] .However, such sources of feedback vary in terms of their quality. For instance, compilers can only flag incorrect code -they cannot tell what to change in that incorrect code. In contrast, while language model critics can suggest what to change, they might often hallucinate and provide inaccurate feedback. Similarly, humans may want to work with a language model to achieve a goal but may not know themselves what the right steps to reach that goal are. This raises the main question of interest: How do we leverage partially correct response generating language models, with partially correct source(s) of feedback, to find solution to important problems?We cast this as a planning problem with partially correct feedback, and design a new tree search procedure for inference time planning. To be computationally efficient when resources are limited, unlike MCTS [12, 23] that does a naive count-based exploration, we take a Bayesian approach and prioritize search towards feedback that provides higher information gain towards the solution.One-shot generation ✓ ✗ ✗ ✗ Iterative Refinement/CoT ✓ ✓ ✗ ✗ Search (MCTS, best-first, etc.) ✓ ✓ ✓ ✗ IDTS (Proposed) ✓ ✓ ✓ ✓Table 1 : Methods that have single chain of thought are often not robust when the feedback is only partially correct. In contrast, search based methods maintain several solutions and corresponding feedback, and are thus less likely to be derailed in their reasoning if the feedback is not perfect. However, conventional search based methods still explore using count-based statistics, which fail to account for any notion of information associated with each feedback. The proposed IDTS method aims at alleviating all these issues. We discuss related work in more detail in Appendix A.O i = (X i , R i) and A i be the random variables corresponding to observation and action at interaction step i. Specifically, the context X 0 in the initial observation O 0 contains the question, and the action A i is an agent's attempt at the solution and X i+1 contains rich feedback on that action. The goal of the agent is to determine an answer Y ∈ Y to that question within a fixed number of interactions. For example, for coding tasks, X 0 is a coding question, A i is the language model's code, Y is a correct code solution, and O i+1 contains the error messages from compiler and results from unit tests for A i .We define a state s ∈ S to be the history of the interaction so far, i.e., S i := (O 0 , A 0 , O 1 , A 1 , . . . , O i ).Let π : S → T (A) be a policy and let τ π (s) correspond to the trajectory unrolled using policy π starting from a state s. In this work, we build upon pretrained LLMs and thus implicitly leverage side information (e.g., data on the internet) to find the solution Y . Let D denote such data. Let the random variable T t be the tree containing everything observed till the end of the t th iteration. With a slight abuse of notation, we will use a subscript of t to denote implicit conditioning on everything observed so far, i.e., T t and D. For example, P t (Y = y|x) := P(Y = y|x, T t , D).Information Directed Sampling: Given random variables X and Y , the information gain for Y on observing In the bandit setting, let Z(A) := (O, A) be the rich observation (potentially including reward),X is I t (Y ; X) := H t (Y ) -H t (Y |X),and action A = a on executing a. For the optimal action a * , let I t (a * ; Z(a)) corresponds to the information gain towards the agent's belief over the optimal action a * given Z(a). Similar to Bayesian experimental design [43] , information directed sampling [44] or its λ-regularized version [17] selects an action such that it maximizes the performance as well the information gained by the agent,EQUATIONMonte-Carlo Tree Search: A popular planning procedure is MCTS that tracks N t (s, a) and N t (s),i.e., the number of times a particular (s, a) pair and (s) has been chosen during tree traversal till iterate t. Let T t be the tree expanded at the start of a given iterate t. At every iterate t, one of the nodes of T t is chosen to be expanded by selecting action a ∈ A for a given state s ∈ S. Most MCTS algorithms are based on the seminal upper confidence tree (UCT) algorithm [23, 12] that choosesEQUATION)where q is the estimate of the cumulative return [45] , and the term in blue guides the exploration.While (2) provides one approach to balance exploration and exploitation based on counts, not only such discrete notion of counts fail to account for similarities between different states, it also fails to incorporate richer notion of explorations, e.g., even if an agent has taken an action a ∈ A often, it might still want to explore that action more if the information being obtained from doing so is large.The key insight of this work is to leverage ideas from (1) and use information gain to improve the exploration technique for monte-carlo tree search (2) . Particularly, for our problem setup, we let the action a be a complete sentence/solution generated by a LLM, and s be the history of past interactions.Then, if the task is related to coding, we want to prioritize a solution a, which together with its error messages and unit-tests Z(a) provides higher information gain towards the solution a * . (While in principle, maximization over the action set A in intractable, similar to prior work [60, 20] we let A be k sampled completions for any given state s.) In the remaining sections, we discuss how to estimate information gain associated with taking any action a in state s. 
information directed tree search: 
 Central to our idea is a procedure to estimate information gain (IG), such that the proposed idea can scale and be used with LLMs. Here we focus on explaining the core idea of computing IG using in-context learning for a single interaction step. Due to space constraints, in Appendix B, we discuss how to use chain-rule of IG to recursively decompose IG over a sequence of actions under a policy.Recall that a state s is the history of interactions, and let s ′ = (s, a, o) be the subsequent state on enacting a. The information gain of taking an action a and observing the new state s ′ is the reduction in the uncertainty of the agent's belief over the solution Y , i.e., I t (Y ;s ′ ) = H t (Y ) -H t (Y |s ′ ).Computing I t (Y ; s ′ ) requires estimating entropies H t (Y ) and H t (Y |s ′ ), which in turn depend on the probability distributions P t (y) and P t (y|s ′ ) for ∀y ∈ Y, respectively. In conventional RL methods, this requires updating the posterior given the new state s ′ . Specifically, recall P t (y|s′ ) = p(Y = y|s ′ , T t , D), where T t is the tree explored till the iterate t, and let T t+1 = (T t ∪ s ′ ) be the tree for the next iterate after observing s ′ . Let ϕ ∈ Φ be the parameters of the environment model for the problem the agent is facing. Here, the posterior updates entail computingEQUATIONUnfortunately, such posterior updates can be challenging when ϕ is high-dimensional. The challenge is particularly exacerbated because posterior needs to be repeatedly updated for T t+k , where k ≥ 1, as new data is acquired, thereby making prior methods intractable beyond simple/linear settings.Figure 1 : Graphical model for the data generating process.In-context learning for posterior updates: To mitigate this challenge, we build upon the recent insights [49, 27] that draw connections between incontext learning and Bayesian inference. In our setting, in-context learning provides a remarkably simple posterior update for p(y|T t+1 , D). Specifically, consider the model in Figure 1 . Let θ be the parameters of the general world model, and ϕ be the model parameters for the specific problem agent is dealing with, and D is the internet data. We will make the following assumption, Assumption 1. ∀t, I(θ; T t |D) = 0.Assumption 1 states that given the internet-scale data D, a few (since t is usually small) problem specific interactions contained in T t do not provide any more information about the general world model θ. This implies p(θ|T t , D) = p(θ|D). Under this viewpoint we can express p(y|T t+1 , D) as the following,EQUATIONwhere (a) follows because of the model in Figure 1 , and Assumption Advantages: Unlike p(ϕ|T t+1 , D) in (3) that does an explicit update to obtain posterior over ϕ for the underlying environment, in (5) the posterior is over the parameters of the world-model p(θ|D) and thus do not require an explicit update when new data is acquired. Instead, the new data is used in-context p(y|T t+1 , θ) to obtain the desired p(y|T t+1 , D). Not only this avoid any updates to the parameters θ, but also as new data becomes available then p(y|T t+k , D) can also be computed readily for k ≥ 1. This would not have been possible without the in-context learning ability of LLMs.Further, (5) reduces the problem to standard uncertainty estimation for deep learning [16, 1] . Perhaps the most popular technique is to use an ensemble of N models for a Monte-Carlo estimate of the integral in (5) . For extremely large models, ensembles can be created using multiple-low rank adapter instead [31, 24] . For simplicity, we use N = 1 in our experiments. The maximum number of nodes expanded is 10. For MCTS and IDTS, the branching factor is 4. 
empirical analysis: 
 Complete algorithm for the proposed IDTS method and more details about the experimental setup is provided in Appendix C. We run all the experiments using OpenAI GPT models [2] , and the results are presented in Figure 2 . We compare the performances of the above methods on the following domains:Code (Python): This is based on the HumanEval benchmark [10] . We focus on the hard problems by filtering out the ones that can be solved by GPT-3.5-turbo in one-shot. Here, an action corresponds to the entire solution. The feedback consists of results from the synthetically generated (and thus potentially incorrect) unit-tests, error messages from the compiler, and self-critique of the solution given the unit-test results and error messages.We use the MiniF2F benchmark [59] for this task. Each action corresponds to an entire formal proof, and the feedback consists of the error messages from the Lean [36] compiler along with the self-critique of the generated solution.LLF-Bench: [11] This task requires recommending movies to a user whose interests are hidden from the agent. After every suggestion, the domain provides natural language hint regarding how close the recommended movies are to the user's interests. In the feedback, we also incorporate self-critique of the solution based on the response received.While IDTS shows some promise on coding and LLF-bench, the gains over MCTS are not significant at the moment to justify additional complexity associated with IDTS. Math domain stands out, as neither the error messages from Lean compiler had corrective feedback, nor the self-critique provided any meaningful feedback. As such, just ignoring any feedback and sampling diverse solutions emerged to be the best strategy in this domain. 
discussion and future work: 
 While in principle, having access to the true IG should increase the efficiency of the search significantly, it is not feasible at the LLM scale to obtain the true IG. IDTS avoids (repeated) posterior updates using in-context learning, but still requires useful uncertainty measure for LLMs [31, 42] .An important direction for future work is to have a deeper study on the estimation error in the IG computation, as finding other better methods of IG estimation is required before using IG is likely to be helpful with LLMs.Note, however, that search using IG only requires deciding which node to explore. As such, it might only be needed that the relative (not absolute) values of the IG across nodes are accurate. While this could potentially mitigate the IG estimation challenge, assessing the quality of ranking also remains challenging as the true rankings are unknown. Further, expanding > > 10 nodes per tree would provide invaluable insights on the utility of different exploration strategies.A Related WorkChain of thought reasoning [48] and self improvement [55] have shown a lot of success in the past. Building on the success of these tree of thought [52] maintains a diverse set of solutions.Alternatively, planning via tree search based algorithms have also been studied [19, 60] . Extensions of A * search [62] , or even in-context tree traversal [57] , and hierarchical search [40, 54, 47] have also been explored.Several methods have considered solving formal math problems using different variants of MCTS and best-first search [51, 29, 46, 26, 61] , doing proof repair using feedabck (akin to iterative-refinement) [58, 14] , by hierarchical decomposition [50] , or incremental learning [5, 41] . MCTS for language based RL games [20] and code-generation [56] has also been considered.The approaches developed in these are largely complementary to ours, but none of them focus on the topic of how to explore more efficiently in the presence of partially correct feedback.Building upon the initial work on IDS [44] , the core idea has been further expounded by several [30, 18, 17] with a focus on theoretical aspects of the regret associated with various instances of the IDS idea. On the practical side, Zanette and Sarkar [53] considered the tabular setting and aimed at a tractable approximation of the information ratio using a variance based bound for IG. From one perspective, if we could have replaced the information gain term in the proposed IDTS method with a variance based bound, then our method would come closer to using Bernstein based bonuses for tree search [28] .Kirschner and Krause [22] discuss how in Gaussian setting, IG based on Bayesian posterior distribution can be expressed in frequentist terms. Nikolov et al. [37] built upon this to consider the generic/deepRL setting and estimate both the aleatoric (using distributional RL) and epistemic uncertainty (using a q-ensemble). IDS has also been used for doing open/closed loop planning by changing the 'reward' function to consider the information gained about the optimal trajectory by running the desired sequence of actions Mehta et al. [33, 34] . While related, none of them tackle the LLM setting.Our work is also related to works that discuss what information to gather [3] , and how the acquisition of it can be made faster [4] . Similarly, it also related to the work on reformulating POMDPs for optimizing long term value of information when eliciting preferences [7] and query optimization [9, 8, 6] . Several recent works also consider the task of strategically collecting labels for which pair of outputs to query in order to minimize the number of samples need for RLHF training [35, 15, 21, 13] or for in-context learning [32] . Our work complements this direction and considers strategic interaction at the inference time to find a solution to given question, and involves no RLHF.Let X and Y be two random variables. Recall, from the chain rule of entropy:EQUATIONSimilarly, information gain between random variables {X 1 , X 2 } and Y can be written as,I(Y ; X 1 , X 2 ) = H(Y ) -H(Y |X 1 , X 2 ) (7) = H(Y ) -H(Y |X 1 ) + H(Y |X 1 ) -H(Y |X 1 , X 2 ) (8) = I(Y ; X 1 ) + I(Y ; X 2 |X 1 ).(Chain rule of information gain) (9)Recall that p(y|T t+1 , D) from ( 5) was required to compute the information gain I t (Y ; s ′ ),EQUATIONHere, the random variable Y that characterizes the agent's belief over the solution is a sequence of token (Y 0 , Y 1 , . . .) that is generated auto-regressively. This structure can be leveraged to specify an estimator that decomposes the information gain at per token level,EQUATIONwhereP t (Y i |Y Y:i-1) is a random variable because conditioning is on a random variable as well.Further, notice that the two sequence of Y i 's are different, as one is conditioned on s ′ and the other is not. K 1 and K 2 are the lengths of the generated sentences.Estimator Ît (Y ; s ′ ) decomposes information gain such that its computation requires logprobs of the generated tokens, which is accessibly even from proprietary large-language models like GPT [38] .However, when using open-source models, notice that the agent has additional access to the logprobs of the tokens not sampled. These can be leveraged to create the following improved estimator that can have lower variance than Ît (Y ; s ′ ), but still provides an unbiased estimator of I t (Y ; s ′ ).EQUATIONIt can be observed using chain-rule of entropy and tower law of expectations that,EQUATIONlet π be a tree-traversal policy that operates on the tree T t (This policy has a composite nature, where it either selects one of the already expanded branches, or draws a new sample to create a new branch in the tree). For any given node s, let τ π (s) be a random trajectory that could be unrolled using π when starting at node s. Let L t (s) be the set of leaves under the subtree of s in T t . Let C t (s) be the set of immediate children of s, and A t (s) be all the ancestors of s in T t .In the following Theorem 1, we formalize a procedure to recursively estimate the information gain on expanding the sub-tree of a given node s. This has an intuitive form that asserts that the information gain from expanding the subtree of a node s equals the expected information gain from expanding the children of node s.Theorem 1. The information gain from τ t (s) can be expressed as the following recursive form,EQUATIONProof. Consider the following decomposition τ (s) = {s ∪ τ 2 }, where τ 2 is the part of the trajectory τ (s) without node s. Note that the first node in τ 2 will be a s ′ ∈ C t (s).EQUATIONLet I π t (s) be the information gained towards the optimal solution y on unrolling a trajectory τ π (s),given the agent is at state s during iteration t. Also, recall that the sub-script of t denotes implicit conditioning on all the data observed so far (e.g., in the code-gen setting, it denotes conditioning on all the internet data that has been used for model training),EQUATIONOne should consider I π t (h) analogous to the state-value function for a policy π, but instead of being for the long-term return to go, it is for the long-term information gain. Similar to the state-value function, it is also possible to define a Bellman recursion for the information gain (recall that by construct, a state is defined using the history of the trajectory).Theorem 2. Information gain recursion:EQUATIONwhere S ′ is observed following the policy π from state s.Corollary 1. n-step Information gain backupEQUATIONwhere S 0 = s.Proof. Follows by unrolling the recursion in (30) .Proof. Proof of Theorem 2. Without loss of generality, consider s = (O 0 ), i.e., just the starting state, and consider the length of a trajectory to be L, EQUATIONRemark 1. Note that as a 'state' is the history of everything that has occured so far, this makes the decision process resemble an (acyclic) tree, where each 'state' can only be reached through a unique path, and can't be revisited again. This is good, as it is aligned with the problems we aim to resolve.But this may be not so good, because it probably makes learning I (•) much harder. should be 0 as there is no new information to be gained from s.Due to the above challenges in estimating long-term information gain I π t (s), we resort to myopic version, where only one-step information gain backup is used, and I π t (s) is set to 0. This is akin to how even UCT does not take into account uncertainty of Q, it still does one-step/bandit uncertainty.In Algorithm 1 we present the pseudo-code for IDTS. The steps highlighted in blue are the key differences from the typical tree-search/UCT.For different domains, we defined Value(s, τ π (s)) differently. For coding domain, this corresponded to the the fraction of unit test passed by the latest solution available in τ π (s). For LLF-bench, this comprised of the scalar feedback form the environment for how relevant is the latest recommendation to the movie that the user had in mind. For math domain, we set Value(s, τ π (s)) = 0, and treat the task as a pure exploration problem.For all the domains, we set n = 1 for rollout in the Evaluate function. Therefore, τ π (s) = s ′ .With this setting Î(Y ; τ π (s)|T , D), where in the algorithm T represents the current tree, can be equivalently expressed as Ît (Y ; s ′ ), where t is the current iterate.Results for the code, and math domain used gpt-3.5-turbo-0125, and for the LLF-bench gpt-4-turbo-2024-04-09 was used.Algorithm 1: The IDTS algorithm For extremely large models, ensembles can be created using multiple-low rank adapter instead [31, 24] . For our experiments, we simply use M = 1, such that p(y|s ′ , T t , D) ≈ p(y|s ′ , T t , θ).Similarly for p(y|T t , D) ≈ p(y|T t , θ). Computing an estimate information gain I t (Y ; s ′ ) now simply corresponds to estimating the difference in entropy of p(y|s, T t , θ) and p(y|T t , θ).Further, conditioning on the information of the entire tree T t might require very large context window for the LLMs. To avoid such long context, instead of conditioning on the content of all the nodes in T t , we condition only on the ancestor nodes of s ′ . Finally, we also note that that the estimator in (11) can result in negative values because of the sampling error. To avoid this, we clip the minimum value of the information gain to be 0. 
 
 Appendix:  



Reviews: review: title: title: Novel Information-Directed Tree Search for LLM Planning Shows Promise but Needs Further Analysis
main_review: main_review: **Summary**

Information Directed Tree Search (IDTS) is proposed as a new approach for inference-time planning on pretrained LLM responses. As a Bayesian approach, IDTS quantifies the information associated with different feedbacks and prioritizes the tree search towards feedback that provides higher information gain. For information gain estimation, in-context learning is combined with Bayesian inference. The effectiveness of the proposed method is evaluated in three different LLM reasoning tasks.

**Strengths:**
1. The idea of using information-directed sampling within a tree search algorithm for LLM-based problem solving is novel, to my knowledge.
2. The approach is explained in a clear and detailed way, both verbally and theoretically.
3. Experiments are conducted on mainstream LLM tasks, which are meaningful, and baselines are well-selected.
4. Results are promising, especially in tasks with corrective feedback.

**Weaknesses:** 
1. The computational complexity of the proposed method could be compared with MCTS and discussed in the paper more clearly and in more detail. 
2. The performance gain over MCTS is limited, but it is promising. The results could be analyzed in more depth. Why do IDTS and MCTS perform similarly? Under what conditions might IDTS outperform baselines?
title: title: Review for Paper 118
main_review: main_review: This paper develops an Information Directed Tree Search (IDTS) method that leverages feedback on LLM responses to solve reasoning tasks. This is a common and practical application scenario for LLMs. The action is selected based on Information Gain (IG) for tree search. In the process, Bayesian inference and in-context learning are used together for estimation, thus overcoming the high-dimensional space issue that traditional RL frameworks cannot handle. The experiments show that the proposed method performs well on coding and movie recommendation tasks but falls behind the baseline on math tasks. 

A few issues that I would be glad to see resolved:
Some notations and formulas are overly complex and not readable enough to me.
Did you try using feedback from other sources on the math task? This could help determine whether the relatively poor performance is related to the feedback or its nature.

Given that it is a workshop paper with a 4-page limit, I think the authors have conveyed their thoughts and approach well, so I suggest an acceptance.
title: title: review
main_review: main_review: To enhance the quality of feedback sources for autonomous agents, this paper proposes a new algorithm called Information-Directed Tree Search (IDTS), aimed at improving the reasoning and planning capabilities of language models in complex tasks. Since these tasks often involve partially correct feedback, IDTS utilizes a Bayesian approach to prioritize the exploration of paths that can provide higher information gain, thereby increasing search efficiency. It performs well in tasks that require multi-step reasoning.

Weaknesses:
1. Although IDTS has shown some promise in encoding and LLF-Bench tasks, the performance improvement compared to MCTS is not significant. And it fails to clearly explain why the two perform similarly or differ in specific situations. The article should also provide a detailed explanation of the advantages and disadvantages of IDTS compared to other improved methods, such as search methods based on uncertainty estimation or reinforcement learning algorithms.

2. The feedback sources differ across tasks (programming, mathematics, natural language), but the article does not delve into how to dynamically adjust IDTS's search strategy based on the quality or type of feedback.

3. Contextual learning can update the posterior distribution and estimate information gain, but the specific working mechanism of contextual learning, especially its application in multi-step reasoning tasks, has not been explained in detail.

Overall, the method proposed in the article is quite novel, but there are still some details that need further clarification.
title: title: Review of an IDTS for enhancing LLMs reasoning and planning capabilities
main_review: main_review: This paper presents an Information Directed Tree Search (IDTS) algorithm designed to enhance reasoning and planning capabilities of Large Language Models (LLMs) in complex tasks requiring multi-step reasoning and feedback integration.

By incorporating a Bayesian approach, the authors seek to guide the search process toward feedback that offers higher information gain, leveraging in-context learning to approximate this gain without explicit posterior updates. While IDTS shows some advantages over simple uniform search methods, it performs comparably to Monte Carlo Tree Search (MCTS), albeit exploring different paths.
The paper is well-structured and provides a solid theoretical foundation for the proposed method. The authors clearly define the problem, formalize their approach using Bayesian principles, and explain how in-context learning can be utilized to estimate information gain without explicit posterior updates. The inclusion of mathematical formulations and theoretical proofs enhances the clarity of the methodology.
The practical significance of the method is not fully established due to the limited empirical gains observed. The proposed IDTS performs comparably to MCTS but does not significantly outperform it. This raises questions about the added complexity versus the benefits gained.

Pros:
- Introducing information gain into tree search for LLMs is a creative idea.
- The paper provides rigorous theoretical backing, including proofs and formulations using information theory and Bayesian inference.
- The authors openly discuss the limitations of their approach and identify areas for future research.

Cons:
- IDTS does not show significant performance gains over MCTS in the experiments conducted.
- The paper lacks in-depth comparisons with other exploration strategies or uncertainty-guided algorithms. 
- A deeper investigation into the estimation errors of information gain and alternative estimation methods would strengthen the contribution.
- The paper lacks comparisons with other advanced planning or search techniques beyond MCTS and simple uniform search methods. 
    + Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. 326 Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406, 2023.
    + Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In ICLR, 2023b.
    + Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv:2201.11903, 2022.

The paper introduces a promising idea with a solid theoretical basis but falls short in empirical validation. It would benefit from providing more comprehensive experimental results and testing on additional benchmarks.