M ath D 2: Towards Disambiguation of Mathematical Terms (2025)

Abstract: In mathematical literature, terms can have multiple meanings based on context. Manual disambiguation across scholarly articles demands massive efforts from mathematicians. This paper addresses the challenge of automatically determining whether two definitions of a mathematical term are semantically different. Specifically, the difficulties and how contextualized textual representation can help resolve the problem, are investigated. A new dataset MathD2 for mathematical term disambiguation is constructed with ProofWiki’s disambiguation pages. Then three approaches based on the contextualized textual representation are studied: (1) supervised classification based on the embedding of concatenated definition and title; (2) zero-shot prediction based on semantic textual similarity(STS) between definition and title and (3) zero-shot LLM prompting. The first two approaches achieve accuracy greater than 0.9 on the ground truth dataset, demonstrating the effectiveness of our methods for the automatic disambiguation of mathematical definitions. Our dataset and source code are available here: https://github.com/sufianj/MathTermDisambiguation.

## introduction mathematical scholarly articles contain highly structured statements, such as definitions, axioms, theorems, and proofs. despite adhering to strict conventions and consistent usage of terminologies, these articles cannot be easily searched or explored through traditional keyword searches. mathematical definitions are rich sources of information. the terms defined therein known as definienda (singular: definiendum) can be automatically extracted. extracted terms can be used to populate a knowledge base (kb), thereby facilitating knowledge discovery. in addition, these terms * these authors contributed equally to this work. to this end, several initiatives have emerged: argot @xcite is a collection of term-definition pairs automatically extracted from mathematical papers, allowing users to retrieve all definitions of a given term, while mathmex @xcite is a recent search engine for mathematical definitions based on the semantic similarity between a user's query and the definition. both projects show promising usage of different word embeddings. existing research in this area focuses on automatically extracting mathematical definitions from scholarly articles @xcite @xcite and disambiguating definienda @xcite . definienda disambiguation involves identifying and connecting terms to their corresponding mathematical definitions in a reference kb. it is particularly challenging when identical terms for the same concept are defined in various ways (e.g., "path") or when polysemous terms (e.g., "block") refer to distinct concepts in various subtopics (see for this study, proofwiki foot_0 serves as the reference list. it is a crowd-sourced online collection of categorized mathematical proofs, including 500 disambiguation pages foot_1 . similar to wikipedia, these disambiguation pages list identical terms, each linking to a dedicated article. the heading of each article is composed of a unique title, appended by the category where the term can be found (e.g. algebra or ## related work the challenges posed by this task are: to address (a), entity linking and sentence similarity approaches for mathematical terms are reviewed. to tackle (b) and (c), transformer models @xcite are employed for their capabilities to produce rich, contextualized representations. contextualized representations produced by bert (bidirectional encoder representations from transformers) @xcite encode the meaning of a word according to its context. this means that polysemous words have several, more accurate representations depending on their location in the sentence. bert is pre-trained on two key tasks: masked language modeling (mlm), where random tokens in a sentence are masked and predicted based on context, and next sentence prediction (nsp), which trains bert to determine whether a sentence follows another in a discourse. pre-training with mlm is widely applied for domain adaptation, especially when there is a dearth of data for fine-tuning @xcite . in addition, fine-tuning bert for specific downstream tasks and domains is straightforward. for instance, by combining bert's output with a classification layer, it has been adapted for mathematical notation prediction @xcite , definiendum extraction @xcite and mathematical statement extraction @xcite . the natural language inferernce (nli) datasets @xcite used by bert's nsp pre-training are related to the task at hand. a piece of supporting evidence is acrobert @xcite , an entity linker that reuses bert for nsp's pre-trained weights and is fine-tuned to link acronyms to their long forms. acrobert outperforms bert and other domain-adapted bert-based models. however, the nature of the bert's pre-training tasks makes it unsuitable for measuring semantic similarity. sentence bert (sbert) foot_2 (reimers and gurevych, 2019) modifies the architecture of bert to produce meaningful sentence embeddings that can be compared using cosine similarity. outof-the-box sbert achieves superior performance across varied classification tasks involving mathematical texts @xcite . in one such task, the proponents measure the similarity of sbert embeddings between an input text and the combination of titles and abstracts of mathematical publications in arxiv foot_3 and zb-math foot_4 to predict the classification code of the respective repositories. in the same vein, this study aims to evaluate the effectiveness of semantic textual similarity in linking definitions to titles. since bert for nsp and sbert require different domain adaptation strategies @xcite , this work first identifies the architecture that performs better for the task. since the release of powerful llms, these models have been applied to various information extraction (ie) tasks, including entity linking. particularly for long-tail entities, llmael @xcite instructed llms to augment the context by expanding entity mentions. the augmented context then serves as additional input to the entity disambiguation component of an ie pipeline @xcite . meanwhile, @xcite attempted to use llms in several ie pipeline components: first by prompting the llm to identify entity mentions (ner), followed by context expansion using prompts reminiscent of llmael's. additional experiments conducted in this paper aim to find out the comparability of the proposed textual similarity models and llms in identifying another example of long-tail entities embodied by mathematical terms. ## methodology mathermatical term disambiguation is formalized as an entity linking task, where the entities refer to the unique article titles in proofwiki. that is, given (1) a definition and an ambiguous definiendum and ## conclusion and future works this work introduces mathd2, a new dataset for mathematical term disambiguation extracted from proofwiki. two entity-linking approaches have been implemented and shown to yield advantages in the usage of contextualized embeddings to differentiate mathematical definitions. the experimental results proved the efficiency and effectiveness of using out-of-the-box sbert. additional experiments with sota llms also show that the proposed models performed better and have fewer computing resource constraints. moreover, error analysis shows the inherent tendency of llms to hallucinate. further work is planned on applying the proposed approaches to scholarly papers. regarding the closed scores of the best models, evaluation with more data and significance tests are planned. in addition, the current approach is to be extended to include document-level representation and citation information to differentiate definitions in scholarly papers. this work also indicates the need for further study on building sentence transformers that benefit from domain-specific mlm and task-related pre-training.