GOLD : Geometry Problem Solver with Natural Language Description (2024)

Abstract: Addressing the challenge of automated geometry math problem-solving in artificial intelligence (AI) involves understanding multi-modal information and mathematics. blackCurrent methods struggle with accurately interpreting geometry diagrams, which hinders effective problem-solving. To tackle this issue, we present the G eometry problem s O lver with natural L anguage D escription (GOLD) model. GOLD enhances the extraction of geometric relations by separately processing symbols and geometric primitives within the diagram. Subsequently, it converts the extracted relations into natural language descriptions, efficiently utilizing large language models to solve geometry math problems. Experiments show that the GOLD model outperforms the Geoformer model, the previous best method on the UniGeo dataset, by achieving accuracy improvements of 12.7% and 42.1% in calculation and proving subsets. Additionally, it surpasses the former best model on the PGPS9K and Geometry3K datasets, PGPSNet, by obtaining accuracy enhancements of 1.8% and 3.2%, respectively.

## introduction automated solving of geometry math problems has gained considerable attention in the ai community recently @xcite @xcite @xcite @xcite . unlike math word problems, geometry math problems involve additional geometry diagrams, necessitating comprehensive reasoning capabilities for understanding multi-modal information (refer to figure 1 for an example of a geometry math problem). as a result, research on automated geometry math problem solving is still in its infancy @xcite . existing approaches for solving geometry math problems utilize neural networks to embed the dia-gram and problem text separately or jointly, resulting in highly generalized models @xcite . however, these methods struggle with accurately capturing the complex relationships within geometry diagrams @xcite . additionally, their vector-based representation of geometric relations is not easily interpretable by humans, posing challenges in identifying whether performance issues are from the relation extraction or the problemsolving component. in a different approach, some studies have successfully translated geometry diagrams into formal languages, enhancing precision and interpretability @xcite @xcite . however, these methods do not separately process relations among geometric primitives and relations between symbols and geometric primitives, which adds difficulty in solving the geometry math problem correctly. moreover, these approaches necessitate specifically designed solvers that take formal languages as input, making them incompatible with prevalent large language models (llms). to address the limitations of existing methods in solving geometry math problems, we introduce the gold model. the gold model converts geometry diagrams into natural language descriptions, aiding in the generation of solution programs for the problems. particularly, the gold model's relation-construction head extracts two types of geometric relations: sym2geo (relations between symbols and geometric primitives) and geo2geo (relations among geometric primitives). this process involves two specialized heads that separately model symbols and geometric primitives within diagrams as distinct vectors. these extracted geometric relations are then converted into natural language descriptions. this not only improves the model's interpretability but also connects geometry diagrams with problem texts. furthermore, since these natural language descriptions meet the input requirements of llms, the gold model is able to utilize the advanced llms as the problem-solving module, efficiently generating solution programs used to solve geometry math problems. to evaluate the effectiveness of the gold model, we conduct experiments on the three latest released datasets: unigeo (comprising calculation and proving subsets) @xcite , pgps9k @xcite , and geometry3k @xcite . the experimental results show the significant performance gains of our gold model compared to state-of-the-art (sota) models. it surpasses the geoformer model, which is the sota model on the unigeo dataset, by 12.7% and 42.1% in accuracy on the unigeo calculation and proving subsets, respectively. additionally, our gold model outperforms the pgpsnet model, the sota model on the pgps9k and geometry3k datasets by 1.8% and 3.2% in accuracy, respectively. these results highlight the superior performance and effectiveness of our proposed gold model compared to existing approaches. the contributions of this work are: (1) we propose the gold model to extract geometric relations from geometry diagrams and subsequently convert these relations into natural languages, which are then utilized for solving geometry math problems. its compatibility with llms is a significant advantage, enabling the gold model to utilize the capabilities of llms to generate solution programs. (2) the gold model separately processes symbols and geometric primitives from the diagrams. this separation design simplifies the extraction of the geometric relations. (3) our gold model demonstrates significant improvements over previous methods across all evaluated datasets, validating the effectiveness of our approach. ## related work early works have explored solving geometry math problems through rule-based approaches @xcite @xcite . recently, with the success of deep learning methods, several works have explored using neural network architectures for automated geometry math problem-solving. approaches such as ngs @xcite utilizing lstm @xcite and resnet-101 @xcite encoded problem texts and geometry diagrams separately. later, methods like dpe-ngs @xcite replaced the text encoder with transformer models. however, these methods struggle to effectively integrate problem texts and geometry diagrams. in response, geoformer @xcite emerged, embedding both diagram and problem text jointly using the vl-t5 @xcite model, treating visuals as additional tokens. despite these advancements, they still struggle to provide precise descriptions of slender, overlapped geometric primitives with complex spatial relationships @xcite , resulting in sub-optimal performance when solving geometry math problems. other approaches typically involve parsing the diagram into formal language and utilizing specific solvers to generate solution programs. recent works like inter-gps @xcite and pgp-snet @xcite employed their parsers to describe the diagram using carefully crafted rules. however, these methods based on predefined rules often lack extensibility, resulting in limited generalization capabilities. to address this issue, our proposed gold model generates natural language descriptions of the diagrams, ensuring compatibility of adopting llms to generate solution programs. ## model our gold model is illustrated in figure 1 . ## conclusion in this work, we have introduced the gold model for automated geometry math problem-solving. gold uniquely converts geometry diagrams into natural language descriptions, facilitating direct integration with llms for problem-solving. a key feature of the gold model is that it separately handles symbols and geometric primitives, simplifying the process of establishing relations between symbols and geometric primitives and relations among geometric primitives themselves. our experiments show that the gold model outperforms the geoformer, the previous sota on the unigeo dataset, with accuracy improvements of 12.7% and 42.1% on the unigeo calculation and proving datasets, respectively. additionally, compared to pgpsnet, the sota for the pgps9k and geometry3k datasets, the gold model shows notable accuracy improvements of 1.8% and 3.2%, respectively, showing our method's effectiveness.