G eo DRL : A Self-Learning Framework for Geometry Problem Solving using Reinforcement Learning in Deductive Reasoning (2023)

Abstract: Ensuring both interpretability and correctness is a great challenge in automated geometry problem solving (GPS), and the scarcity of labeled data hinders learning mathematical reasoning from samples. Therefore, we present GeoDRL, a self-learning geometry problem solving framework that integrates logic graph deduction and Deep Reinforcement Learning (DRL) to optimize geometry reasoning as a Markov Decision Process. GeoDRL employs a Graph Neural Network on a Geometry Logic Graph, updating the problem state using a symbolic system. Incorporating DRL into deductive reasoning enables GeoDRL to achieve unsupervised self-learning while maintaining correctness. GeoDRL, through unsupervised learning, exhibits enhanced accuracy in the Geometry3K dataset, improving by 11.1% over previous SOTA methods, and simultaneously boosts efficiency and interpretability.

## introduction automated geometry problem solving (gps) has been a long-standing task in mathematical reasoning research, drawing significant attention from researchers @xcite @xcite @xcite . as depicted in figure 1 , gps can be described as follows: given a geometry problem description which typically consists of a diagram and text, the objective is to produce a flow of solution steps leading to the final answer. solving a geometry problem consists of two key steps: parse and reason, where parsing involves extracting specific interpretation from the question diagram and text and reasoning involves using the interpretation to develop a solution. there has been a long-standing debate between symbolic and probabilistic approaches in mathematical reasoning. in geometry problem solving, * corresponding author the symbolic approaches @xcite @xcite first parse the question diagram and text into logical expressions in a formal symbolic language, and then utilize a predefined set of geometry theorems to solve the problem. these approaches are known for the mathematical rigor, as they construct a symbolic system for logical interpretation and theorem application. in contrast, the probabilistic approaches @xcite @xcite view gps as a special case of text generation with multi-modal input. these data-driven approaches learn mathematical language and solving strategies from problem-solution samples to generate human-readable solutions without prior mathematical knowledge. nevertheless, there are notable limitations to both categories. the symbolic approaches struggle with selecting the appropriate theorem @xcite . the heuristic search strategies tend to involve redundant steps @xcite , which harms the readability of the solution. on the other hand, the probabilistic approaches rely heavily on manually annotated data, which is often scarce and expensive. furthermore, the solutions generated by deep neural network are doubtful in mathematical rigor and correctness @xcite . even an advanced language model like gpt-4 is prone to make arithmetic mistakes @xcite , which is unacceptable in mathematical problem solving. accordingly we propose a self-learning geos framework, geodrl, to addresses these issues. our framework consists of two phases: by introducing reinforcement learning, we construct a self-learning framework that enables unsupervised learning of problem-solving strategies, alleviating the scarcity of labeled data. furthermore, our framework integrates neural network and symbolic system in the deduction, resulting in interpretable step-by-step solutions while maintaining mathematical correctness. our contributions are summarized as follows: glg demonstrates its effectiveness in geometry interpretation modeling by outperforming sequential logical expressions. ## approach this section introduces our proposed geometry problem solving framework, geodrl. the overall architecture is illustrated in figure 2 . here we follow the flow described in 3.1 and introduce the parser and reasoner separately. ## experiments in this section, we conduct experiments on public geometry problem datasets and compare the performance of geodrl against the previous methods. furthermore, we analyze the contribution of different schemes by ablation study and discuss the superiority and limitations of our model with typical cases. ## conclusion and future work in this paper, we propose geodrl, a self-learning geometry problem solving framework by integrating deep reinforcement learning into deductive geometry reasoning, which enables unsupervised learning of problem-solving strategies. we structure geometry information as geometry logic graph to glue discrete geometry literals together. the combination of neural network and symbolic system allows efficient solution while maintaining correctness. experiments demonstrate that geo-drl outperforms state-of-the-art approaches in terms of accuracy, efficiency and interpretability. our work presents a promising idea of integrating reinforcement learning into deductive reasoning for geometry problem solving. we believe that this framework is applicable to a majority of mathematical reasoning tasks. in the future, we intend to expand our framework to broader mathematical domains to become a general-purpose mathematical reasoning framework. limitations 1. error accumulation: incorrect parsing results will affect the reasoner. in our experiments on geometry3k dataset, incorrect parsing results lead to a substantial 21.0% performance drop.