Toward Global AI Inclusivity: A Large-Scale Multilingual Terminology Dataset ( GIST ) (2025)

Abstract: The field of machine translation has achieved significant advancements, yet domain-specific terminology translation, particularly in AI, remains challenging. This work introduces GIST, a large-scale multilingual AI terminology dataset containing 5K terms extracted from top AI conference papers spanning 2000 to 2023. The terms were translated into Arabic, Chinese, French, Japanese, and Russian using a hybrid framework that combines LLMs for extraction with human expertise for translation. The dataset’s quality was benchmarked against existing resources, demonstrating superior translation accuracy through crowdsourced evaluation. GIST was integrated into translation workflows using post-translation refinement methods that required no retraining, where LLM prompting consistently improved BLEU and COMET scores. A web demonstration on the ACL Anthology platform highlights its practical application, showcasing improved accessibility for non-English speakers. We address a critical gap in AI terminology resources and fosters global inclusivity and collaboration in AI research.

## introduction the field of machine translation has made significant progress, with state-of-the-art models excelling across diverse tasks @xcite @xcite and demonstrating effectiveness in translating between high-resource and low-resource languages @xcite @xcite . despite these successes, translating domain-specific scientific texts remains a persistent challenge, particularly for terminology translation @xcite @xcite . generalpurpose translation systems often falter in accurately translating specialized terminology, leading figure 1 : direct translations of ai research papers and model cards using google translate generally offer fair quality but often fail to accurately translate ai-specific terminologies, potentially causing confusion or misunderstanding for readers. our work addresses this issue by providing high-quality translations for a wide range of such terms, which can be efficiently integrated posthoc to enhance the initial translations. to loss of critical details @xcite , or worse yet possibly leading to misinterpretations @xcite . in the field of ai, terms such as "coreference resolution" or "explain-away effect," are frequently translated incorrectly or inconsistently, undermining comprehension for global researchers and practitioners @xcite , as illustrated in figure 1 . for non-english readers, who comprise a substantial portion of the global population both within and beyond the ai community @xcite , such inaccuracies in ai research papers and blog translation hinder access to essential knowledge, stifling research innovation and collaboration across linguistic boundaries @xcite : overview of gist creation. ai terminology are extracted from awarded papers using an llm and then combined with existing terminology dictionaries. after data cleansing, translations into five languages are generated via crowdsourcing, and an llm is used to select the best candidate translation. dataset quality is evaluated against two baselines: majority-vote-selected translations and translations from the 60-60 evaluation set, demonstrating the superior quality of gist. to address this, we introduce glossary of multilingual ai scientific terminology (gist), the first large-scale multilingual ai terminology dataset, compiling 5k ai-specific terms from award-winning papers at 18 top-tier ai conferences . using a hybrid approach combining llms for extraction and verification with human expertise for translation, we provide high-quality translations into arabic, chinese, french, japanese, and russian @xcite . to enhance accessibility of our work, we integrate the curated terminology into machine translation pipelines without requiring model retraining. we evaluate three post-translation refinement methods: prompting, word alignment and replacement, and constrained decoding. our experiments reveal that the prompting method effectively integrates the curated dictionary, consistently improving translation quality as measured by bleu @xcite , comet @xcite , chrf @xcite , chrf++ @xcite , and ter @xcite . additionally, we develop a website demonstration that incorporates our dataset into translations of acl anthology research papers, showcasing its practical application for non-english speakers in the ai field. the contributions of this work are summarized as follows: ## related work multilingual terminology datasets the acl 60-60 evaluation sets represent an important effort in translating multilingual terminology from ai presentations, but are limited to just over 250 terms @xcite . scientific terminology datasets are typically built using two approaches. the first relies on human multilingual experts for extracting and translating terminology @xcite . in cases of limited expert availability, human crowdsourcing with aggregation techniques has shown excellent results @xcite . the second approach uses ml and machine translation tools for automatic collection and translation, including log-likelihood comparisons @xcite , machine translationbased data synthesis @xcite @xcite , platform-based terminology linkers @xcite , classifier training @xcite , and prompting llms @xcite . however, fully human-driven approaches can be costly for large-scale multilingual datasets, while fully automated ones often lack accuracy @xcite . to address this, our framework integrates llm-based extraction, human filtering, human translation, llm validation, and merging with existing dictionaries, efficiently and effectively expanding existing terminology datasets. ## gist dataset construction we construct gist, a dataset comprising around 5k english ai terminology and their translations into arabic, chinese, french, japanese, and russian. the basic lexical statistics of the dataset is presented in ## dataset quality assessments to thoroughly evaluate the translation quality in gist, we conduct two additional crowdsourced rating tasks: (1) in section 4.1, we investigate whether using gpt-4o is necessary to select the best translation candidate; (2) in section 4.2, we compare the quality of our translation with the evaluation set from the 60-60 initiative. ## dataset coverage assessment domain coverage we first investigate the ai domains covered by the collected terms in gist and their distributions. we use gpt-4o-mini @xcite to identify the specific ai domains for each term, following the taxonomy proposed by @xcite , which clusters the research domains of ai scholars. figure 9 shows the distribution of the top six ai domains in gist, with terms most frequently categorized into statistics, mathematics, computer science (cs), natural language processing (nlp), data science (ds), and computer vision (cv). we further embed all ai terminology in gist using the all-minilm-l6-v2 model @xcite and apply uniform manifold approximation and projection (umap) @xcite for dimensionality reduction to a two-dimensional space. the visualization in figure 10 reveals that terms in domains such as nlp, cv, statistics, and mathematics form distinct clusters, while terms in cs and ds are more dispersed. terminology coverage we surmise that our awarded paper dataset provides essential coverage of the terms in our terminology dictionary, where many terms are actually repeatedly extracted multiple times from different awarded papers. we present the rarefaction curve in figure 11 , which visually demonstrates the relationship between the number of extracted terms and the dataset size. to statistically validate our claim, we conduct a one-sided one-sample t-test to compare the mean coverage ratio of the terminology dictionary. specifically, we test whether the mean coverage ratio of randomly sampled subsets, constituting 60% of the terminology dictionary, is significantly above 80%. the null hypothesis (h 0 ) assumes that the mean coverage ratio of these subsets is less than or equal to 80%, while the alternative hypothesis (h a ) assumes that the mean coverage ratio exceeds 80%. using 1,000 random samples, the analysis yields a t-statistic of 64.78 and a p-value of 0, rejecting the null hypothesis. these results provide statistical evidence supporting the sufficiency of our dataset in covering the terminology dictionary. refer to appendix b.5 for more details. ## experiments we explore terminology integration approaches that do not need model retraining. we quantitatively evaluate two methods: terminology refinement via post-hoc llm prompting and terminology substitution guided by word alignment. ## website demonstration to facilitate real-world usage of our ai terminology dictionary, we built on the 60-60 initiative by modifying the acl anthology website layout and introducing a new terminology translation feature, foot_5 as illustrated in figure 3 . the acl anthology website was chosen for this demonstration due to its extensive collection of ai-related research papers, making it an ideal platform to showcase the potential impact of our work. in this demonstration, translations of key terms are refined and standardized based on the terminology dictionary we developed, providing more accurate terminology translations.this enhancement represents a step toward improving access to ai knowledge for non-english speakers by offering a consistent and reliable translation system, and broadens the accessibility of ai research to a global audience. ## conclusion we present gist, a large-scale multilingual ai terminology dataset addressing gaps in translating ai-specific terms. combining llm-based extraction and validation with human expertise, it includes 5k english terms with translations into five languages, surpassing the acl 60-60 benchmark. llm prompting proved effective for posthoc terminology integration, improving translation quality across five metrics. we also provide a website demonstration to enhance the accessibility of our work for non-english speakers, supporting equitable ai knowledge access and fostering global collaboration in ai research.