Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper demonstrates that for simple algorithmic problems (arithmetic), language models can be taught to split problems into multiple subproblems, which can then be fed back to the LM to be solved independently. With very small models this can achieve great performance on these tasks, and it circumvents the limit in context-length of existing transformer models.
strength_and_weaknesses: My main concern with this approach is that the paper does not discuss how to apply this approach to more general problems. The problems considered here are very simple, and an algorithm to solve them needs to be known in order to generate the training data. So, in its current form this does not enable any new abilities as we could use the base algorithm instead of using expensive LMs.

In terms of the insight from a learning point-of-view, I also do not see anything surprising in this work. After decomposing the problem manually (i.e. by writing an algorithm to generate training data), the learning task is pretty simple (for addition, for example, all the model has to do is to extract the last digit from two numbers). So the high accuracy of the resulting method is not that surprising.
clarity,_quality,_novelty_and_reproducibility: The writing style is a bit "flashy", which gives room for miscommunications. For example, the paper opens with the following statement:

"Although neural networks have achieved amazing results on various domains, e.g., images, texts,
audios, videos, games, etc., nearly all of them are classified as System 1 tasks (Kahneman, 2013), ..."

Afaik, this statement is not supported by the literature, and in particular not by the given reference. Following Kahneman's theory, tasks like playing the board game go would certainly require system 2 thinking. IMO, dropping the first paragraph would help the paper.

The paper also makes claims such as "the length of CoT can grow rapidly with the problem’s complexity", without further explanation and I am not even sure what exactly this means.
summary_of_the_review: This paper demonstrates that decomposing simple algorithmic problems can circumvent the limit in context length language models.

However, the technique only seems to work for problems for which we already have algorithms, and it is unclear to me if this could be extended to more general problems. The writing style should be improved.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper experiments with fully supervised training of a LM to produce subsequences needed for solving simple reasoning tasks, motivated by limited context size of LMs.
strength_and_weaknesses: On the positive side, the paper is well written and easy to read.

On the negative side, the solution proposed is formulated as a fully supervised problem, meaning for each task the exact steps needed to solve the problem needs to be annotated. This is not a scalable approach as for realistic real-world applications obtaining detailed annotations for reasoning problems is an expert task which is expensive to scale to large datasets. As a result, outside of tasks such as arithmetic operations where the dataset can be created automatically (and one could argue that you wouldn't use a task specific LM as the paper doesn't deal with multiple tasks or out-of-distribution problems), the applicability of this approach is minimal.
clarity,_quality,_novelty_and_reproducibility: The paper is written clearly.

The novelty is minimal as the solution proposed is rather trivial as it doesn't try to solve the hard problem of reasoning without direct supervision, or other settings such as few-shot/zero-shot/multi-task and formulates the problem as a fully supervised task.

Due to simplicity of approach, I think it's reproducible.
summary_of_the_review: Unfortunately, I don't see much impact from this work in practical terms. The formulation doesn't scale to general reasoning problems, the tasks considered aren't tasks that you would train a specific LM to solve, and the algorithmic solution is very simple/trivial extension. Hence, I'm not able to recommend this paper for publication.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work proposed a model-agnostic framework, Recursion of Thought (RoT), to release the capacity constraint by the maximum size of a single context in language models. RoT teaches a language model to divide and conquer complex problems by recursively creating multiple contexts; therefore, a complex problem could be solved by utilizing multiple contexts. This work also conducts extensive experiments on arithmetic and algorithmic reasoning tasks to show the power of RoT in helping language models solve problems that require hundreds of thousands of tokens. 

Overall, the proposed framework (RoT) is novel and well-motivated, and the authors have conducted multiple experiments which demonstrate the great power of RoT on much more complex arithmetic and algorithmic reasoning problems. One important point missing from this work is how to systematically construct subproblems given any problem in order to teach the model learn the recursion of thoughts. Still, I think this work has great potential to allow language models to achieve better reasoning abilities.
strength_and_weaknesses: Strength:
-

[+] The proposed framework (RoT) is novel and well-motivated.

[+] The work has conducted multiple experiments which demonstrate the great power of RoT on much more complex arithmetic and algorithmic reasoning problems. 

Weakness:
-

[-] The training of RoT requires (non-trivial) human inputs to design proper subproblems.

More specifically, I wonder 

1) what is the limitation of the problems that RoT could solve: for example,

- Does RoT have the ability to learn backtrack? Some problems may require people to modify the earlier part of the answer based on the new observed information.

- Does RoT have the ability to learn a problem where its subproblems have different structures (as recursion usually requires the same structure in the subproblems)?

- What is the performance of RoT on (maybe small-scale) NP-hard algorithmic problems that may not have the divide-and-conquer structure such as TSP？

2) how to design proper subproblems in order to use RoT to train the models?

- What are the criteria of the subproblems for RoT? What kinds of structures do they need to have？ 

- Are there systematic approaches to design subproblems for reasoning tasks in general?

clarity,_quality,_novelty_and_reproducibility: This work has a good and clear presentation of its idea, its specific methodology, and its experiment settings. The quality and originality of the work should meet or exceed the conference standard. My main concern is on how to properly train the RoT framework as it requires people to feed it with hand-designed subproblems. Therefore, it would be great if the authors could explain the limitation of the problems that RoT could solve and how to design proper subproblems in order to use RoT to train the models.
summary_of_the_review: This work proposed a novel model-agnostic framework, Recursion of Thought (RoT), to release the capacity constraint by the maximum size of a single context in language models. RoT teaches a language model to divide and conquer complex problems by recursively creating multiple contexts; therefore, a complex problem could be solved by utilizing multiple contexts. It then conducts extensive experiments on arithmetic and algorithmic reasoning tasks to show the power of RoT in helping language models solve problems that require hundreds of thousands of tokens. 

The proposed framework (RoT) is novel and well-motivated, and this work has a good and clear presentation of its idea, its specific methodology, and its experiment settings. The quality and originality of the work should meet or exceed the conference standard. My main concern is on how to properly train the RoT framework as the training of RoT requires (non-trivial) human inputs to design proper subproblems. Still, I think this work has great potential to allow language models to achieve better reasoning abilities.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper considers chain of thought approach to prompting and the problem of limited context length in current LLMs. The authors propose to use recursion to deal with limited context length and provide experiments on arithmetic and reasoning tasks to show the performance of the proposed approach.

Strength:
- The paper is well-written and easy to follow.
- Limited context length is an important issue for the use of current LLMs.


Weaknesses:
- The idea of using recursion (and splitting into subproblems) to deal with limited context length in LLMs is not new and has been used for other settings such as mathematical theorem proving.
- The authors make general claims about their proposed approach without the idea being general enough as it is hand crafted.

As is, the paper is not ready for publication.

justification_for_why_not_higher_score: The novelty of the paper is below expectation. see meta review for details.
justification_for_why_not_lower_score: The vote is Rejection.
Decision: Reject