Zero at S em E val-2025 Task 2: Entity-Aware Machine Translation: Fine-Tuning NLLB for Improved Named Entity Translation (2025)

Abstract: Machine Translation (MT) is an essential tool for communication amongst people across different cultures, yet Named Entity (NE) translation remains a major challenge due to its rarity in occurrence and ambiguity. Traditional approaches, like using lexicons or parallel corpora, often fail to generalize to unseen entities, and hence do not perform well. To address this, we create a silver dataset using the Google Translate API and fine-tune the facebook/nllb200-distilled-600M model with LoRA (LowRank Adaptation) to enhance translation accuracy while also maintaining efficient memory use. Evaluated with metrics such as BLEU, COMET, and M-ETA, our results show that fine-tuning a specialized MT model improves NE translation without having to rely on largescale general-purpose models.

## introduction machine translation (mt) has proven to be significant at enabling cross-cultural and cross-border communication. it is essential in multilingual content creation, border business interaction, and realtime translation communication services. while the contemporary neural machine translation (nmt) models have achieved high fluency and accuracy, they struggle with certain aspects of translation. one of the most difficult parts is the translation of named entities (nes) which are handled comparatively poorly. nes include proper names of people, places, organizations and other cultural references. they pose difficulties due to their rarity in occurrence in natural language, ambiguity, and due to the language-specific variations. entity-aware machine translation (ea-mt) seeks to resolve the complexities faced when translating sentences with named entities. unlike generic mt that depends on the patterns of language and context, ea-mt focuses on entity recognition, retention, and accurate translation. the im-portance of ea-mt is very evident in real-world applications such as news translation, medical and legal document translation, and localization of entertainment content, where small mistakes in ne translation can lead to misinformation or loss of meaning, which would cause problems. handling named entities in translation is a difficult task because entities might not have direct equivalents across all languages. for example, consider the english sentence: an mt model might struggle to translate the above sentence due to several possible reasons: to handle such issues as discussed above, a variety of techniques have been used in mt such as dictionary-based approaches, parallel corpus training, and using external knowledge, in addition to other approaches. traditional approaches utilize large bilingual lexicons or pre-aligned corpora in order to guarantee the correct entity mappings. novel techniques are now using knowledge graphs, entity linking, or explicit annotation to assist models in differentiating between named entities and regular phrases. in this paper, we propose an approach that leverages a silver dataset generated using google translate and fine-tunes the nllb model to enhance entity-aware translation. such a method allows the model to learn important patterns in entity translation. the approach used in this paper also provides flexibility which helps the model to be able to generalize and tackle samples with unseen entities and still have a high accuracy. ## dataset the dataset @xcite provided for the task is a collection of english text data translated into various languages such as italian, spanish, french, etc. the data is present in the jsonl format. fig 1 depicts a sample of evaluation data. the sam-ple has an id, a wikidata_id, a list of entity types present in the sentence, the source language, the target language, the source text in english and the translated target sentence. { " id ": " q850522_0 " , " wikidata_id " : " q850522 " , " entity_types " : [ " movie " ] , " source " : " who are the main characters in the movie little women ?" , " targets " : [ { " translation " : " quines son los personajes principales de la p e l c u l a mujercitas ?" , " mention ": " mujercitas " } ] , " source_locale " : " en " , " target_locale " : " es " } the training data provided has a slightly different format. an example is shown in fig 2 . the training sample contains the source text in english, the target translation in the required language, list of wikidata ids for entities present in the source text and the source of the data sample. prediction data is also provided, which contains predictions by gpt-4o and gpt-4o-mini, which can be used to analyze the performance of proposed systems. { " source " : " did gone with the wind come out before 1940? " , " target " : " via col vento uscito prima del 1940? " , " entities " : [ " q2875 " ] , " source_locale " : " en " , " target_locale " : " it " , " instance_id " : " 826528 e6 " , " from ": " mintaka " } 4 system description 4.1 silver dataset creation as we have mentioned in the previous sections, the provided dataset contains predictions generated using gpt 4o and gpt 4o-mini. however, we wanted to create a different dataset using an expert machine translation system as opposed to using the predictions from a general purpose large language model. this is because of the fact that gpt-4o and gpt-4o-mini are optimized for language modeling over a vast corpus rather than being trained specifically for machine translation. to ensure we have good quality silver predictions which stem from a model specifically trained for machine translation, we made use of the google translate api to translate the sentences in the dataset from the source language to the target language. we call this newly created dataset our "silver dataset". ## conclusion this paper explored the challenge of named entity translation in machine translation, a task where the generic models often fall short. to address this, we created a silver dataset using google translate and fine-tuned the facebook/nllb-200distilled-600m model with lora (low-rank adaptation), enabling a more efficient and specialized approach to tackle the task of entity-aware translation. our evaluation using bleu, comet, and m-eta metrics demonstrated the effectiveness of finetuning to improve ne translation quality without the need to use generalized large language models. while spanish and turkish achieved high scores across both general translation and entity accuracy, languages like japanese and korean displayed weaker bleu scores but better semantic preservation, which can be seen from comet and m-eta scores. overall, our approach shows the strengths of fine-tuning models for named entity machine translation.