Solving Math Word Problems via Cooperative Reasoning induced Language Models (2023)

Abstract: Large-scale pre-trained language models (PLMs) bring new opportunities to challenging problems, especially those that need high-level intelligence, such as the math word problem (MWPs). However, directly applying existing PLMs to MWPs can fail as the generation process lacks sufficient supervision and thus lacks fast adaptivity as humans. We notice that human reasoning has a dual reasoning framework that consists of an immediate reaction system (system 1) and a delicate reasoning system (system 2), where the entire reasoning is determined by their interaction. This inspires us to develop a cooperative reasoning-induced PLM for solving MWPs, called Cooperative Reasoning (CoRe), resulting in a human-like reasoning architecture with system 1 as the generator and system 2 as the verifier. In our approach, the generator is responsible for generating reasoning paths, and the verifiers are used to supervise the evaluation in order to obtain reliable feedback for the generator. We evaluate our CoRe framework on several mathematical reasoning datasets and achieve decent improvement over state-of-the-art methods, up to 9.6% increase over best baselines.

## introduction addressing math problems is a hallmark of human intelligence, which allows reasoning and adapting from limited data. we want neural models to be able to do the same, however, quick and flexible reasoning is challenging to current neural models as they must possess a certain level of prior experience from a limited amount of new data while avoiding overfitting. the rapid growth of largescale pre-trained language models (plms) offers unprecedented potential for this issue, often relying on well-designed trigger prompts @xcite @xcite . although appealing in terms of efficiency, its success relies on memorizing patterns with a sufficiently large number of parameters (≥ 100 billion) @xcite , differentiating it from the fast adaptivity in the human reasoning process. active disciplines like neuroscience and cognitive science attempt to uncover the mechanism of human reasoning, and agree that our learning process is governed by an interaction mechanism, often referred to as system 1 and system 2 @xcite . in particular, system 1 offers fast responses like human instinct, and system 2 performs deliberate reasoning. interactions between them are important for adapting to a continuously changing environment. plms behave more like system 1, according to the above theory, and thus lack the generalization ability in reasoning @xcite . in this work, we explore a new line of zero-shot math problem reasoning, using a human reasoningalike framework with feedback in the solution generation loop as opposed to pure plm-based methods, called cooperative reasoning (core). intuitively, system 1 and system 2 are embodied as generators and verifiers, respectively, and they are defined as follows: generators for generating reasoning paths, and verifiers for supervising the paths' evaluation. specifically, we train a lm beyond the question-answer paradigm by integrating in-the-loop reasoning, i.e., we let the lm output both the answer and the corresponding reasoning process for a given question. meanwhile, we introduce two types of verifiers, including tokenlevel and sentence-level, allowing us to provide feedback in the whole solution generation lifecycle. notice that the solution path is generated by selecting candidate tokens with some probability so that it is tree-alike and much coincides with the tree search process of monte carlo tree search (mcts) @xcite . with this in mind, the verifiers can score tokens along the solution generation process from start to end when using the mcts. therefore, we can use the score to evaluate the quality of the generation process during inferring before finalizing the solution, making timely feedback available for supervising the generation process. with this, the evaluation goes beyond the quality of the final result at the granularity of each reasoning step, extending the supervision from the solution level to the path level. we combine the solution score and the perplexity of its corresponding reasoning path to encourage the overall training towards high-quality augmented solutions while aligning with the reliable reasoning process, aiming to improve generalization ability. our experimentally evaluate core on multiple mathematical reasoning datasets in both zeroshot and fine-tuning settings. core consistently achieves better performance than competing baselines. notably, core has up to 9.6% improvements on multiarith over sota baselines, which are dozens of times larger than our model. in summary, our contributions are as follows. ## cooperative reasoning in this section, we will present the proposed cooperative reasoning framework, core, that enforces system 1 and system 2 mutually cooperating, which includes 3 sequential steps: cooperative training, cooperative inference, and self-thinking. ## discussion although we only fine-tune the language model on gsm8k due to the scarcity of qa datasets annotated with intermediate rationales, zero-shot results on several arithmetic datasets prove that basic reasoning capability is transferable across datasets within the same domain. this observation implies that when it comes to a new domain, we only need to collect a limited number of question-answer pairs with reasoning paths, model's reasoning ability can generalize to other unseen datasets and can be further strengthened by our approach core according to the experimental results. ## conclusions in this work, we mimic the dual system of human cognition to develop an effective reasoning framework for solving the math word problems. the proposed approach is consisting of two ingredients: the generator as system 1 and the verifiers as system 2, and overall reasoning is conducted based on their mutual reinforcement. from the robustness and generalization aspects, core activates superior reasoning ability of lms, and thus outperforms plms that are dozens of times larger.