Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Language (2025)

Abstract: Knowledge graphs offer an excellent solution for representing the lexical-semantic structures of lexicographic data. However, working with the SPARQL query language represents a considerable hurdle for many non-expert users who could benefit from the advantages of this technology. This paper addresses the challenge of creating natural language interfaces for lexicographic data retrieval on knowledge graphs such as Wikidata. We develop a multidimensional taxonomy capturing the complexity of Wikidata’s lexicographic data ontology module through four dimensions and create a template-based dataset with over 1.2 million mappings from natural language utterances to SPARQL queries. Our experiments with GPT-2 (124M), Phi-1.5 (1.3B), and GPT-3.5-Turbo reveal significant differences in model capabilities. While all models perform well on familiar patterns, only GPT-3.5-Turbo demonstrates meaningful generalization capabilities, suggesting that model size and diverse pre-training are crucial for adaptability in this domain. However, significant challenges remain in achieving robust generalization, handling diverse linguistic data, and developing scalable solutions that can accommodate the full complexity of lexicographic knowledge representation.

## introduction knowledge graphs (kgs) have emerged as scalable and interoperable resources for organizing and accessing the vast volumes of data produced in our digital age. particularly for lexicographic data, as found in dictionaries, kgs offer an ideal structure for capturing the complex relationships between words, meanings, and linguistic patterns due to the highly interrelated nature of this information @xcite . the preservation and accessibility of lexicographic data is crucial for standardizing language understanding, supporting query in natural language: "what is the gender of apfel in german?" generated sparql query: select ?lexeme ?qitem ?lemma ?qitemlabel where { values ?lemma {'apfel'@de} . ?lexeme wikibase:lemma ?lemma ; wdt:p5185 ?qitem. service wikibase:label { bd:serviceparam wikibase:language 'en' } } figure 1 : conversational lexicography: enabling natural language queries to kgs by automatically generating sparql code, eliminating the need for manual query writing linguistic research, documenting cultural diversity @xcite , and crucially, increasing interoperability in language technology. recent advancements in large language models (llms) have opened new pathways for creating natural language interfaces to kgs, potentially democratizing access to this structured linguistic knowledge @xcite . despite their advantages, kgs remain largely inaccessible to non-technical users due to the specialized knowledge required to query them effectively. currently, accessing information in kgs requires proficiency in a query language, notably sparql, which presents a significant barrier to entry. users must not only master this technical query language but also understand the specific ontologies and data models that structure each kg @xcite . wikidata 1 , a prominent open-source kg, employs a collaboratively developed semantic structure that requires detailed knowledge to navigate effectively. this technical complexity limits the broader utility of kgs, particularly for audiences such as language learners, teachers, and other non-technical stakeholders who could benefit from lexicographic data access @xcite . this paper addresses the significant research gap in creating effective natural language interfaces for lexicographic data retrieval on kgs such as wikidata. to that end, we develop a multidimensional taxonomy that captures the complexity of wikidata's lexicographic data ontology module, systematically categorizing the diverse information requests that may be queried on the kg. additionally, we create a template-based dataset that maps natural language utterances to corresponding sparql queries, designed to reflect the variety of possible information requests identified in our taxonomy. finally, we conduct preliminary experiments using transformer-based language models of modest parameter sizes to generate sparql queries from natural language inputs, as exemplified in figure 1 , evaluating their performance on both seen and unseen utterances to assess the impact of model parameter size and training method. ## related work the translation of natural language queries into sparql has received significant attention in recent years, particularly with the advent of llms and the increasing importance of kgs. this section provides a brief description of datasets, generation techniques and evaluation methods. ## methodology we develop a systematic methodology to map natural language queries to sparql for lexicographic data in wikidata, illustrated in figure 2 . this relies on a taxonomy to generate query templates which are then populated with data instances to create a comprehensive dataset. the dataset is subsequently used to train and fine-tune llms for the sparql generation task. we provide background information about wikidata in appendix b. ## dataset to develop a comprehensive dataset mapping natural language utterances to sparql queries targeting lexicographic data in wikidata, we adopt a template-based approach similar to @xcite based on the taxonomies defined in section 3.1. each data point in our templates consists of three elements: all utterances are in english, though they may reference terms in other languages, e.g., "what is the grammatical gender of 'livre' in french?". the following is an instance in our populated dataset: utterance: where does the word color come from? template_name: q20 query: select ?etonymlexeme ?qitemlanguageoforigin ?etonym ?qitemlanguageoforiginlabel where { values ?lemma {'color'@en} . ?lexeme wikibase:lemma ?lemma ; wdt:p5191 ?etonymlexeme. ?etonymlexeme dct:language ?qitemorigin; wikibase:lemma ?etonym . service wikibase:label { bd:serviceparam wikibase:language 'en' } } 3 nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.4.2 to address the limited diversity inherent in template-based approaches, we decouple semantics from syntax by generating multiple variations of utterance templates while preserving their meaning. this is accomplished by using gpt-4 to generate alternative phrasings with random selection during template population with an example provided in appendix a. ## experiments and results in order to evaluate the effectiveness of various language models in generating sparql queries for lexicographic data on wikidata, we conduct experiments with three strategically selected models: gpt-3.5-turbo as a baseline, and our fine-tuned phi-1.5 and trained gpt-2 models. when evaluated in a zero-shot setting without fine-tuning or training, both phi-1.5 and gpt-2 failed completely, scoring 0 across all metrics, demonstrating that task-specific adaptation is essential for sparql generation with these models. our selection of models prioritizes those with modest parameter counts (1.3b for phi-1.5 and 124m for gpt-2) to demonstrate if effective sparql generation can be achieved without requiring computationally expensive models, making deployment more accessible for resourceconstrained environments. additionally, these models represent different training approaches-gpt-3.5-turbo as a commercial api-based model, phi-1.5 as a recent code-capable model amenable to parameter-efficient fine-tuning, and gpt-2 as a fully trainable smaller model-providing a diverse evaluation spectrum. for each model, we assess performance using the evaluation framework described in section 3.3. the results are summarized in ## conclusion and discussion this paper addresses the challenge of creating natural language interfaces for lexicographic data in kgs. we develop a multidimensional taxonomy capturing the complexity of wikidata's lexicographic data representation based on which we create a template-based dataset with over 1.2 million mappings from natural language utterances to sparql queries. our experiments with gpt-2, phi-1.5, and gpt-3.5-turbo reveal significant differences in model capabilities. while all models perform well on familiar query patterns (pass@1 scores ranging from 0.86 to 0.90), only gpt-3.5-turbo demonstrates meaningful generalization capabilities (pass@3 of 0.57 in the generalization scenario). this suggests that model size and diverse pre-training are crucial for adaptability in this domain. this work is timely and important as kgs continue to grow in complexity, creating an urgent need for accessible interfaces. limitations and future work while our experiments demonstrate promising results with models of modest size, future work should explore more recent reasoning-focused models such as deepseek, qwq, and llama-3, which may offer improved performance for complex sparql generation tasks. additionally, scaling experiments with larger model variants could help establish the relationship between model size and sparql generation capabilities, potentially identifying optimal efficiency-performance tradeoffs for this specific task. as such, future work should focus on improving model generalization through more diverse training data, expanding this approach to other kgs, particularly dbnary @xcite , and conducting user studies to evaluate practical utility for different stakeholder groups in lexicography and linguistics.