Quality-Aware Translation Tagging in Multilingual RAG system (2025)

Abstract: Multilingual Retrieval-Augmented Generation (mRAG) often retrieves English documents and translates them into the query language for low-resource settings. However, poor translation quality degrades response generation performance. Existing approaches either assume sufficient translation quality or utilize the rewriting method, which introduces factual distortion and hallucinations. To mitigate these problems, we propose Quality-Aware Translation Tagging in mRAG (QTT-RAG), which explicitly evaluates translation quality along three dimensions-semantic equivalence, grammatical accuracy, and naturalness&fluency-and attach these scores as metadata without altering the original content. We evaluate QTT-RAG against CrossRAG and DKM-RAG as baselines in two open-domain QA benchmarks (XORQA, MKQA) using six instruction-tuned LLMs ranging from 2.4B to 14B parameters, covering two low-resource languages (Korean and Finnish) and one high-resource language (Chinese). QTT-RAG outperforms the baselines by preserving factual integrity while enabling generator models to make informed decisions based on translation reliability. This approach allows for effective usage of cross-lingual documents in low-resource settings with limited native language documents, offering a practical and robust solution across multilingual domains.

## introduction retrieval-augmented generation (rag) has become a standard approach for large language models (llms) for open-domain question answering tasks by accessing external sources of knowledge @xcite . one core challenge in multilingual rag (mrag) is retrieving relevant documents in a different language that would not de-* equal contribution. ## methodology we address a cross-lingual retrieval scenario where user queries q are posed in medium or low-resource languages l q (e.g., korean, finnish), while the target document collection d predominantly contains documents in high-resource languages l h (e.g., english). our proposed pipeline, shown in figure 2 consists of five sequential modules designed to handle cross-lingual retrieval and generation: (1) retrieval, (2) reranking, (3) language detection & translation, (4) quality tagging, and (5) generation. ## experiments and results in this section, we describe the datasets used in our experiments, the experimental setup, evaluation metrics, and results, followed by ablation studies to analyze the contribution of each component. as baselines, we compare our method against three approaches: (i) base, a retrieval-only system without translation, which relies solely on reranked retrieved documents; (ii) crossrag, which translates all retrieved passages into the query language; and (iii) dkm-rag, which refines translated passages using an llm-based rewriting step. ## discussion we analyze cases where qtt-rag delivers smaller gains in chinese compared to korean and finnish. to formalize this observation, we denote the crosslingual share by in our experiments, the mkqa-zh split has a relatively low cross-lingual share (r @xmath0.0%), whereas the mkqa-ko split shows a much higher cross-lingual share (r @xmath1.7%). this disparity helps explain why qtt-rag's improvements tend to be smaller in chinese than in korean: there are simply fewer instances where translated evidence is involved. more broadly, overall effectiveness also depends on retriever and mt quality, generator backbone, the distribution of retrieved languages, and the evaluation setting. for future work, we aim to expand our evaluation to a wider set of languages to further test the scalability of our approach. we also plan to explore hybrid retrieval strategies, such as deliberately inducing cross-lingual usage via english-only retrieval for non-english queries. ## conclusion we propose qtt-rag, a novel multilingual rag framework that introduces translation quality tagging as an explicit mechanism to mitigate factual distortions and translation-induced errors. unlike prior approaches such as crossrag, which assumes adequate translation quality, or dkm-rag, which relies on rewriting and risk semantic drift, our method preserves the original translated content and supplements it with fine-grained quality scores as metadata. through extensive experiments on two multilingual qa benchmarks (xor-tydi qa and mkqa) across three typologically diverse languages-korean, finnish, and chinese-and six instruction-tuned llms ranging from 2.4b to 14b parameters, we demonstrate that qtt-rag consistently improves character 3-gram recall over strong baselines particularly in low-resource settings (korean and finnish). ablation studies further reveal that quality tagging offers a more reliable default than hard filtering, while still leaving room for filtering-based strategies in specific regimes with abundant in-language evidence.