Exploring Reversal Mathematical Reasoning Ability for Large Language Models (2024)

Abstract: Large language models (LLMs) have presented remarkable capabilities in the wide range of natural language understanding and reasoning tasks. Despite their success, a few works indicate that LLMs suffer from the “reversal curse”, in which LLMs can’t employ the inverted structure “B is A” when they are trained based on “A is B”. To explore the effect of the “reversal curse” for LLMs on complex mathematical reasoning tasks, we present two reversal datasets upon GSM8K and MathQA and verify that LLMs also struggle to solve reversal mathematical problems. We analyze the potential reason and attribute it to the insufficient modeling of the relationship between reasoning steps caused by the left-to-right objective. Consequently, based on the characteristics of multi-step reasoning, we design a novel training method to improve the general and reversal reasoning abilities. Finally, we conduct experiments on four mathematical datasets, and the results demonstrate that our method significantly improves the general reasoning capacities and alleviates the reversal problem. Our datasets and codes are available at https: //github.com/AllForward/ReversalMath.

## introduction with the significant increase in data and model scale, large language models (llms) @xcite @xcite have emerged with their powerful multi-dimensional capabilities, such as longcontext open domain conversation, code assistants @xcite @xcite , instruction following @xcite , particularly in the complex reasoning tasks solved by chain-of-thought (cot) methods @xcite @xcite . ## reversal mathematical datasets construction different levels of mathematical word problems have been proposed to evaluate llms' general mathematical reasoning ability, such as addsub @xcite , multiarith (roy ## methodology in this section, we first introduce the training objective of causal mechanisms widely utilized in llms. after that, we analyze the shortcomings of this objective in multi-step reasoning tasks like mathematical problems and design an additional training objective to alleviate this problem. ## experiments in this section, we evaluate the effectiveness of our method by employing it on mathematical datasets. to compare with current works @xcite @xcite , we follow them and adopt the original test set to present the general reasoning ability. the discussion about the reversal reasoning capacity will be introduced in section 6.1. ## analysis 6.1 evaluation of the reversal ability to further evaluate the effectiveness of our method in improving the reversal reasoning ability, we con- metamath, using backward data augmentation, significantly improves the reversal reasoning ability, which presents the importance of reversal augmented data. after training with our method on the metamath dataset, llama2-7b obtains higher accuracy and outperforms the strong baseline gpt-3.5-turbo on gsm8k-reversal. the above results further evaluate that our method is able to model the relationship between cot reasoning steps better and further improve the reversal reasoning ability. ## conclusion in this paper, we discussed the "reversal curse" in mathematical problems and proposed two reversal datasets based on the gsm8k and mathqa to evaluate whether llms face challenges in reversal mathematical reasoning. we analyzed the potential reason and attributed it to the insufficient modeling of the relationship between reasoning steps. consequently, based on the characteristics of multi-step deduction tasks, a bidirectional training objective is designed to alleviate this problem. finally, we conducted experiments on four mathematical reasoning benchmarks to evaluate the effectiveness of our method and also verify the benefit of our approach on two reversal datasets we constructed. in the future, we will explore other enhanced methods and explore how to apply them in the pre-training stage to improve the general reasoning ability.