Paper: PreCoT: Problem Representation Enhances Reasoning in Large Language Models (ICLR-2024)

Reviewer comment (Novelty):
Its technical novelty is somewhat limited, as the idea of initially guiding LLM to extract given information or objectives lacks novelty. In two of the previous works https://aclanthology.org/2023.acl-long.147.pdf and https://arxiv.org/abs/2306.03872, they also involve extracting the given information and (or) objectives of the given problem.

Author response:
**(2) Previous similar works.**
    
We carefully reviewed the papers you suggested and would like to provide what makes our paper different from the previous works.
    
First, we’d like to provide a comprehensive summary of our approach. Our work enhance general solution searching performance of LLMs by grounding this process with a structural interpretation of a problem inspired by human problem-solving. This approach can be applied easily to various solution searching methods such as CoT and LtM.

The papers you mentioned are a sort of solution searching method that involves the given information extraction. They used the extracted information to manipulate specific solving steps, such as filtering out irrelevant information [4] or verifying each reasoning step [5]. In particular, while the prompt used in [4] includes the instruction “understand the problem”, it might not be seen as a specific strategy to improve understanding of a problem. The core of our idea is not to simply extract specific elements of a question but to ground solution searching by structured interpretation of a problem observed in human problem-solving.

Reviewer comment (Experiments):
The problem representation method is only combined with CoT no matter zero-shot or few-shot. It would be better to explore the combination of the problem representation method with more distinct types of solution search methods like decomposition-based methods (e.g., L2M), to validate the generality the proposed problem representation method.

Author response:
**(1) Generality of our method.**

We appreciate your constructive suggestion. We implement the Least-to-Most (LtM) prompting [1] as described in the original paper. Accordingly, we also implement Problem Representation Enhanced LtM (PreLtM) by attaching the problem representation construction stage to LtM. In PreLtM, LtM is grounded on a self-generated problem representation. We evaluated both baselines in the fair evaluation setup described in our paper using PaLM 2. We employed the most benchmarks used by the original LtM paper. Additionally, we added GSM-IC [2] to investigate the effectiveness of problem representation on robustness against irrelevant information you highlighted as our strength.

|  | LtM | PreLtM |
| --- | --- | --- |
| GSM8K | 54.51 | 57.09 (+2.58) |
| GSM-IC | 62.50 | 73.40 (+10.90) |
| DROP (football) | 71.60 | 72.80 (+1.20) |
| DROP (non-football) | 78.80 | 79.80 (+1.00) |
| Last Letters | 50.40 | 48.53 (-1.87) |

The results show that our approach can be incorporated with other solution searching methods, demonstrating the universal applicability and effectiveness. We observe consistent improvement on all arithmetic reasoning tasks, including additional DROP [3] benchmarks consistent with our main results.

For the degradation in last letters concatenation (Last Letters), LLM often fails to concatenate the last letters of the given words in both LtM and PreLtM, while it can successfully extract the last letters of each word. Since our method aims to provide an effective interpretation of a problem rather than directly manipulating specific solution searching steps, the performance improvement may be insignificant for specific tasks where a particular step in the solution searching stage is the bottleneck (e.g., concatenation).

Consistent with our main result, problem representation did not improve the performance on Last Letters in LtM, either. Please note that in zero-shot setting, where the solution structure of this task is not provided to LLMs, our PreCoT significantly boosts the performance on Last Letters (+15.06% in PaLM 2 and +10.00% in GPT-3).

We will update the paper with this result after receiving further feedback.

Reviewer comment (Presentation):
Regarding Figure 1, it would be better to visually distinguish the two stages more clearly. In the current version, "#1 Problem Representation Construction" and "#2 Solution Searching" on the left may be deem as simultaneous inputs of an LLM at the first sight. I recommend referring to Figure 1 in the Least-to-most paper (https://openreview.net/pdf?id=WZH7099tgfM) for inspiration on how to improve the visual separation.

Author response:
**(4) Clarifying Figure 1.**
    
We updated Figure 1 to reflect your suggestion for better visibility. We will follow up if you have further feedback on the presentation.

Reviewer comment (Writing):
In Figure 3, "Math World Problem" should be "Math Word Problems."

Author response:
**(6) About the typo in the title of Figure 3.**
    
Thank you for pointing it out. We corrected the typo.

Reviewer comment (Theory):
The paper repeatedly mentions human problem-solving as a rationale and an inspiration for the approach, but there is a lack of convincing and up-to-date backing for the rather central claim that humans actually decompose problems as suggested in the paper. Despite referring to "accumulated insights", references in section 2 are by no means recent ones, nor would some of them be considered to pertain to cognitive psychology.

Author response:
**(1) The references in Section 2 are not recent ones.**

We believe that a fundamental observation lasts for a long period of time. For instance, recent academic publications cite the references in Section 2 as foundational to their work. In particular, Gick’s model [1], which we mainly referenced, is cited in several studies published in 2023. [2] mentioned this study as a foundational study in human problem-solving. Also, the ideas of references were derived from cognitive psychology in a broad sense, focusing on the problem-solving application.

Reviewer comment (Theory):
Whether seeking inspiration from human problem-solving would actually improve LLM reasoning remains a debated issue and is still unresolved. Some similarities between human and LLM reasoning, such as step-by-step decomposition, tend to be rather simplistic, while more rigorous findings, such as similarity in the influence of content over reasoning (as per the Wason selection task [Dasgupta et al., 2022]), may not scale up to all forms of LLM reasoning.

Author response:
**(2) Whether taking inspiration from human problem-solving can improve LLM reasoning remains a debated issue.**

While many works have been exploring the potential of LLMs on reasoning tasks, borrowing ideas from human behavior studies [3, 4], this remains an open question. In line with this direction, we investigated the effectiveness of problem representation on solution searching in LLMs. Please note that our research does not aim to make LLM reasoning identical to human reasoning. Instead, we focus on adapting observations from human problem-solving that could benefit LLMs to enhance their performance in general reasoning tasks.

**(3) The effectiveness of problem representation may not scale up to all forms of LLM reasoning.**
    
We acknowledge the importance of the scalability of our approach. We also recognize that problem representation might not benefit all forms of LLM reasoning. However, we respectfully point out that due to the vast and diverse nature of reasoning abilities, it is nearly impossible to test our method across every potential reasoning task in a single study. Despite this limitation, we endeavored to conduct as comprehensive an evaluation as possible within practical constraints.
    
We investigated the effectiveness of problem representation in LLMs, focusing on arithmetic, commonsense, and symbolic reasoning tasks. These categories encapsulate the core reasoning abilities typically expected from an LLM and align with the scopes of many existing studies in this domain [3, 5, 6, 7, 8]. To further validate our approach rigorously, we expanded the scale of our evaluation to include 15 benchmarks, which is beyond the usual scope.
    
Evaluating different techniques in this field, including ours, against benchmarks designed to explore intrinsic similarities between human and LLM reasoning could be an intriguing direction for future research. However, the aim of our study is not to investigate these similarities. Instead, we focus on assessing our approach's utility in general reasoning tasks for LLMs.

Reviewer comment (Evaluation):
It is unclear whether the PRECoT method actually works by the hypothesized mechanism of problem decomposition, rather than through a clarification of questions that would facilitate content-based inference through a better specification of relationships between 'variables'.

Author response:
**(6) PreCoT may facilitate content-based inference by better specifying the relationships between variables, rather than by working through the problem decomposition.**
    
The core of our idea is not to extract specific elements of a question but to ground solution searching by structured interpretation of a problem observed in human problem-solving. Actually, “better specification of relationships between variables” is a desirable ability for problem solvers. So we believe that is not the means, but the end.
    
If we've misunderstood something in this part, we'd appreciate clarification.

Reviewer comment (Evaluation):
The claim that PreCOT helps LLM enhance its reasoning doesn’t make sense to me as the LLMs are more or less performing approximate retrieval based on the training data that has been fed into them. When these datasets include not only solutions but also the derivative paths leading to them, methodologies like PreCOT might improve an LLM's ability to generate more probable outputs, thus appearing to solve problems more effectively. However, given the vast and varied nature of the content available on the web, we have no clear idea as to what the entire web contains.

Author response:
**(1) Concerns about reasoning ability of LLMs.**
    
We acknowledge some concerns and different perspectives in the community on whether LLMs have reasoning ability. While many researchers have been exploring the topic to answer the controversial question, it remains open [1]. Our primary goal in this study is not to conclude the debate by firmly insisting that LLMs possess reasoning ability. Instead, we aim to further explore the potential of LLMs in reasoning tasks by applying ideas borrowed from well-established literature on human behavior.

**(2) LLMs are more or less performing approximate retrieval.**
    
Even if we call it retrieval, LLMs should go through some steps to find the desirable information, not just directly spotting it. It involves logically combining necessary information. We used various benchmarks that are designed to evaluate the reasoning ability of LLMs to validate our approach rigorously. Those benchmarks were extensively used in prior studies [2, 3, 4, 5] to assess the specific reasoning abilities of LLMs.

In particular, the mathematical and symbolic reasoning problems we covered in our study cannot be regarded as mere information retrieval tasks. Specifically, GSM-IC [6] incorporates irrelevant information to distract LLMs, and SVAMP [7] requires LLMs to consider the context of a problem rather than relying on simple heuristics.

For example, the question in Figure 2 of our paper inquires about the total gain derived from reselling some magazines. In this case, LLMs must distinguish the relevant information from the question and logically combine these to reach the correct answer.

While we respect and acknowledge your concerns about the reasoning of LLMs, we would like to emphasize that the benchmarks we used are devised to evaluate the reasoning ability [6, 7, 8, 9, 10]. Our method shows competitive performance on them.

Reviewer comment (Evaluation):
My primary concern is that the benchmarks used in the proposed work are susceptible to be part of or mirror the training data of the LLM. While testing these benchmarks is useful, the claims made in the paper can be upheld if there is some form of diagonalization in the benchmarks that are being evaluated.

Author response:
**(3) Primary concern: Benchmarks used in the proposed work are susceptible to be part of or mirror the training data of the LLM.** 
    
The data used to train the LLMs employed in our experiments includes documents only up until mid-2021. This means most of the benchmarks we used, such as GSM8K (released in October 2021) [10], GSM-IC (released in January 2023) [6], and all tasks from Big Bench (released in June 2022) [8, 9], were not part of the training data for these LLMs due to their release dates being subsequent to their data cutoff. Furthermore, the Big Bench tasks explicitly prohibit their inclusion in the pre-training corpora of LLMs.