V erba N ex AI at S em E val-2025 Task 2: Enhancing Entity-Aware Translation with W ikidata-Enriched M arian MT (2025)

Abstract: This paper presents the VerbaNexAi Lab system for SemEval-2025 Task 2: Entity-Aware Machine Translation (EA-MT), focusing on translating named entities from English to Spanish across categories such as musical works, foods, and landmarks. Our approach integrates detailed data preprocessing, enrichment with 240,432 Wikidata entity pairs, and fine-tuning of the MarianMT model to enhance entity translation accuracy. Official results reveal a COMET score of 87.09, indicating high fluency, an M-ETA score of 24.62, highlighting challenges in entity precision, and an Overall Score of 38.38, ranking last among 34 systems. While Wikidata improved translations for common entities like “Águila de San Juan,” our static methodology underperformed compared to dynamic LLM-based approaches.

## introduction translating named entities such as proper nouns, geographic locations, and culturally significant references across languages remains a persistent challenge in natural language processing (nlp). this difficulty is particularly evident in the english-to-spanish language pair, where lexical and cultural disparities often hinder accurate translation @xcite . for instance, a literal translation of "the shawshank redemption" as "la redención de shawshank" fails to convey its identity as a well-known film, potentially confusing spanishspeaking audiences. similarly, "eagle of st. john" requires translation to "águila de san juan" to retain its cultural and religious significance rather than an erroneous "águila de jhon." these examples underscore the importance of entity-aware machine translation (ea-mt), the focus of semeval-2025 task 2, which seeks to enhance precision in translating such entities for applications, including content localization, cross-cultural communication, and user-facing services. human translators excel at navigating these nuances by leveraging cultural knowledge and external resources, such as glossaries, to adapt entities appropriately @xcite . for example, rendering "thanksgiving" as "día de acción de gracias" demands understanding its north american cultural context. this task challenges automation without advanced systems. neural machine translation (nmt) has markedly improved fluency in general translation tasks. yet, it often struggles with named entities due to insufficient training data for rare or culturally specific terms and the absence of real-time contextual adaptation @xcite . these shortcomings motivated our participation in semeval-2025 task 2, aiming to improve entity translation accuracy. we propose an ea-mt system that combines marianmt, an efficient model for english-to-spanish translation, with enrichment of 240,432 bilingual entity pairs from wikidata. this approach balances computational scalability, suitable for resource-constrained environments, with precision for culturally significant entities such as movie titles, foods, and landmarks. marianmt's lightweight architecture facilitates fine-tuning with limited resources, while wikidata's structured data addresses data scarcity challenges @xcite . we aim to narrow the divide between human-like cultural adaptation and automated scalability, addressing a pressing need in contemporary nlp. our contributions are threefold: (1) a scalable pipeline for data integration using wikidata, adaptable to various languages and domains; (2) an empirical analysis comparing our static approach to dynamic llm-based systems @xcite ; and (3) insights into the limitations of static ea-mt and the demand for real-time, culturally sensitive adaptation. we intend to release our code and enriched dataset to support further research. this paper is structured as follows: section 2 reviews related work, section 3 details our methodology, section 4 evaluates performance, and section 5 summarizes findings and outlines future directions. ## related work entity-aware machine translation (ea-mt) has emerged as a critical subfield in nlp, addressing the limitations of traditional neural machine translation (nmt) in handling named entities that require cultural and contextual precision @xcite . recent advancements, such as semeval-2025 task 2, introduce specialized metrics like m-eta for entity-specific accuracy and comet for overall translation quality. these developments build upon approaches like yuksel et al.'s dynamic llm-based methods @xcite . unlike these dynamic strategies, our system leverages wikidata as a static knowledge base for entity enrichment. machine translation has evolved significantly over the years. early rule-based systems relied on manually crafted linguistic rules, offering limited scalability and adaptability. statistical machine translation (smt), exemplified by tools like moses, improved upon this by leveraging parallel corpora. however, it struggled with rare entities and contextdependent translations due to their reliance on statistical alignments rather than semantic understanding. the advent of transformer-based nmt models @xcite ) marked a significant leap in translation quality. yet, limitations persist in entity translation, as traditional approaches often lack mechanisms for cultural adaptation and real-time knowledge integration. modern neural approaches like crocoalign @xcite refine sentence alignment, optimizing training data for nmt systems. introducing models like bert @xcite brought pre-trained language representations, further enhancing nmt capabilities. however, challenges persist, particularly in translating rare named entities @xcite . efforts such as mosaico @xcite address data scarcity by providing large-scale, multilingual, semantically annotated corpora. other techniques have contributed to improved entity translation, including entity projection via mt @xcite and denoising pre-training with monolingual data @xcite . our system builds on these advancements by integrating entity enrichment through wikidata. handling named entities remains a significant challenge in nmt. zeng et al.'s extract-and-attend method dynamically extracts entity candidates, reducing errors by up to 35% @xcite . similarly, lee et al. @xcite employ ner post-processing to refine translation outputs, an approach we adapt statically via fine-tuning. our system enhances entity translation by leveraging wikidata to ensure contextual accuracy across languages. cultural adaptation is a crucial aspect of ea-mt. challenges such as preserving culturally significant titles (e.g., breaking bad) align with our focus on entities like águila de san juan (vishwakarma, 2023). wang et al. @xcite highlight the issue of cultural dominance in llms, which we mitigate through the integration of multilingual data in wikidata. named entity recognition (ner) plays a foundational role in this effort, with surveys like li et al. @xcite guiding our enrichment strategy. evaluation remains a significant challenge in ea-mt. traditional metrics like bleu fail to capture entity accuracy, leading to the adoption of m-eta, which reflects the limitations of our static approach. jung et al. @xcite propose fine-grained error analysis for deeper quality assessment. this approach could further refine our evaluation. our work aims to address existing gaps in ea-mt by incorporating structured knowledge bases and static entity enrichment, enhancing translation accuracy and cultural relevance. ## system description our ea-mt system enhances english-to-spanish entity translation through a three-stage process tailored for scalability and precision in semeval-2025 task 2. as detailed below, our methodology adapts a static approach using marianmt and wikidata, explicitly discarding dynamic alternatives due to resource constraints. ## results and analysis we assess performance using semeval-2025 task 2 metrics: comet (general quality), m-eta (entity accuracy), and overall score: ## conclusion despite neural machine translation (nmt) advancements, our results highlight fundamental shortcomings in entity-aware translation when relying solely on static knowledge sources. while our system achieved a comet score of 87.09, demonstrating strong fluency, its m-eta score of 24.62 exposed severe deficiencies in entity precision, ultimately leading to an overall score of 38.38, the lowest among competing systems. these results confirm that a static enrichment approach, even when incorporating a large-scale structured knowledge base like wikidata, is insufficient for handling the complexity of named entity translation. static methods offer scalability and efficiency in low-resource settings (e.g., 4gb gpu) compared to rag's demands, but the comet-m-eta gap shows fluency prioritization over precision, misaligned with ea-mt goals. one of the main issues observed was the rigid dependency on wikidata, which, while useful for well-documented entities, failed to capture emerging terms, domain-specific references, and subtle cultural nuances. the absence of real-time retrieval mechanisms also resulted in translation errors for ambiguous or context-sensitive entities. compared to retrieval-augmented systems @xcite , our approach could not dynamically adjust translations, leading to cases where named entities were either mistranslated or omitted entirely. exploring more curated or updated versions of structured knowledge bases like wikidata could enhance entity translation accuracy. however, the fundamental limitation of static approaches' in-ability to adapt to new or context-specific entities would remain, reinforcing the need for dynamic retrieval methods. another critical limitation was our preprocessing pipeline, which, although effective in text normalization, introduced unintended side effects. for example, the replacement of accented characters (e.g., "águila" to "aguila") compromised entity integrity, further reducing translation accuracy @xcite . moreover, our constrained hardware (nvidia rtx 3050, 4gb) restricted finetuning to only four epochs, potentially limiting the model's ability to leverage the enriched dataset effectively @xcite . traditional approaches, such as rule-based systems and statistical machine translation (e.g., moses), suffer from similar limitations, poor scalability, lack of semantic understanding, and inadequate entity handling, rendering them obsolete for modern ea-mt tasks. our metrics provide clear evidence for the need for alternative solutions. the stark contrast between our m-eta score 24.62 and the top systems' scores (e.g., 90.50 for llama-3.3-70b-instruct + deepseek-r1) indicates a significant gap in entity translation accuracy. in contrast, our competitive comet score (87.09 vs. 95.31 for gpt-4o) suggests fluency is less of a bottleneck. this disparity underscores the inadequacy of static methods for entity-specific tasks. it justifies the adoption of dynamic, retrieval-augmented approaches capable of addressing rare and context-dependent entities. a hybrid approach, like lightweight rag caching frequent entities, could balance efficiency and adaptability.