Weakly Supervised Formula Learner for Solving Mathematical Problems (2022)

Abstract: Mathematical reasoning task is a subset of the natural language question answering task. Existing work suggested solving this task with a two-phase approach, where the model first predicts formulas from questions and then calculates answers from such formulas. This approach achieved desirable performance in existing work. However, its reliance on annotated formulas as intermediate labels throughout its training limited its application. In this work, we put forward the idea to enable models to learn optimal formulas autonomously. We proposed Weakly Supervised Formula Learner, a learning framework that drives the formula exploration with weak supervision from the final answers to mathematical problems. Our experiments are conducted on two representative mathematical reasoning datasets MathQA and Math23K. On MathQA, our method outperformed baselines trained on complete yet imperfect formula annotations. On Math23K, our method outperformed other weakly supervised learning methods.

## introduction mathematical reasoning is a task where mathematical problems are described in natural language or mathematical symbols. such problems require values, expressions, or other mathematical representations as answers. a naive approach to solving this task is to treat it as an end-to-end token-bytoken predicting problem from questions to answers. however, this approach showed a relatively poor generalization capacity on unseen numbers @xcite . another approach to solving mathematical reasoning task is to adopt a two-phase methodology. in the first phase, specific formulas are predicted for each question. in the second phase, such formulas are calculated under predefined rules to produce the final answers. this approach has been widely applied in recent work and has achieved desirable results in many representative mathematical reasoning datasets @xcite . however, this two-phase solution leads to a reliance on annotated formulas as indispensable labels for training the formula predictor in the first phase. this reliance further results in two major weaknesses. firstly, ground-truth formula annotations are not necessarily prepared for every mathematical problem and dataset. this makes it impossible to extend this solution to datasets without these annotations. secondly, the learning process can be misled when there is noise in the formula annotations. in consideration of this, we are motivated to propose a new learning framework for solving mathematical problems that is not dependent on formula annotations. on the whole, we followed the principle of the two-phase methodology and implemented the two phases with what we call policynet and acttaker. figure 1 is an overview of our learning framework. our main contributions in this work can be summarized as follows: ## experiments our experiments in this work are conducted on two representative mathematical reasoning datasets, mathqa @xcite and math23k @xcite . we report and discuss our findings on them in section 4.1 and 4.2, respectively. ## conclusion this work discussed the issue of formula annotation dependence in existing work on solving mathematical problems. to deal with this issue, we proposed a new learning framework, weakly supervised formula learner. this framework established a mechanism to learn formulas with weak supervision from final answers and enabled a heuristic formula search in the space of possible formulas. in the experiments, our learning framework showed remarkable formula exploring capacity on both mathqa and math23k datasets. particularly, on mathqa, we illustrated that our models trained with formulas discovered in formula exploration outperformed baselines trained with complete yet imperfect formula annotations. on math23k, our learning framework showed more powerful formula exploring capacity than existing weakly supervised learning methods. in view of this evidence, we consider our proposed learning framework a valid and advanced approach for solving mathematical problems with weak supervision from their answers.