# Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset

Paul Lerner, François Yvon

<figure><img src="image_1.png" /></figure>

To cite this version:

Paul Lerner, François Yvon. Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset. 2025. hal-05328251

HAL Id: hal-05328251

https://hal.science/hal-05328251v1

Preprint submitted on 23 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Assessing the Political Fairness of Multilingual LLMs:
A Case Study based on a 21-way Multiparallel EuroParl Dataset

Paul Lerner, François Yvon

Sorbonne Université, CNRS, ISIR
75005, Paris, France
lerner@isir.upmc.fr, yvon@isir.upmc.fr

## Abstract

The political biases of Large Language Models (LLMs) are usually assessed by simulating their answers to English surveys. In this work, we propose an alternative framing of political biases, relying on principles of fairness in multilingual translation. We systematically compare the translation quality of speeches in the European Parliament (EP), observing systematic differences with majority parties from left, center, and right being better translated than outsider parties. This study is made possible by a new, 21-way multiparallel version of EuroParl, the parliamentary proceedings of the EP, which includes the political affiliations of each speaker. The dataset consists of 1.5M sentences for a total of 40M words and 249M characters. It covers three years, 1000+ speakers, 7 countries, 12 EU parties, 25 EU committees, and hundreds of national parties.

**Keywords:** Parallel Dataset, Political Fairness, Political Biases, Large Language Models

## 1. Introduction

Parliamentary debates have long been used as a major source of parallel texts in the history of Machine Translation (Brown et al., 1990). Notably, the EuroParl dataset (Koehn, 2005) enabled the study of an increasing diversity of language pairs as the European Union (EU) was growing and played a key part in WMT's shared tasks (Koehn and Monz, 2006; Bojar et al., 2015; Kocmi et al., 2022). Given their content, parliamentary debates are also relevant for various fields, including political sciences (Høyland et al., 2014; Deligiaouri, 2016; Schwalbach, 2024; Sebők et al., 2025). Unfortunately, existing corpora derived from the European Parliament's debates (including the noteworthy EuroParl<fnref n="1" />) share two limitations (see Table 1):

* they are biparallel, i.e., they pair texts in two languages, although speeches were originally translated into *n* languages, which prevents any multilingual comparable analysis;

* the associated metadata is incomplete.

To fill this gap, we build on LinkedEP (Noy et al., 2017) to present 21-EuroParl, a 21-way multiparallel dataset aligned at the sentence level with extensive metadata, including information about the speaker's political affiliation (national and EU party), see example on Figure 1. The dataset contains a grand total of 72,234 instances and spans over three years, from 2009 to 2011 (included). It comprises speeches from 1000+ speakers coming from 27 countries (see Figure 2), 12 EU parties, 25 EU committees, and hundreds of national parties.

<table><caption>Figure 1: A fully worked-out example of 21-EuroParl: a German (DE) source sentence is aligned to translations in 20 other languages. Metadata includes the speaker's political affiliation.</caption><tr><td>speech</td><td>2011-01-19-Speech-3-084-000</td></tr><tr><td>date</td><td>2011-01-19</td></tr><tr><td>speaker</td><td>Eumember_28251 (Jörg Leichtfried)</td></tr><tr><td>EU Committee</td><td>TRAN</td></tr><tr><td>EU Party</td><td>S&D</td></tr><tr><td>National party</td><td>SPÖ_Austria</td></tr><tr><td>Country</td><td>Austria</td></tr><tr><td>Bulgarian (bg)</td><td>В Европа имаме свобода на печата.</td></tr><tr><td>Czech (cs)</td><td>V Evropě máme svobodu tisku.</td></tr><tr><td>Danish (da)</td><td>I Europa har vi pressefrihet.</td></tr><tr><td>German (de)</td><td>In Europa gibt es die Pressefreiheit.</td></tr><tr><td>Modern Greek (1453-) (el)</td><td>Στην Ευρώπη, έχουμε ελευθερία του τύπου.</td></tr><tr><td>English (en)</td><td>In Europe, we have freedom of the press.</td></tr><tr><td>Spanish (es)</td><td>En Europa tenemos libertad de prensa.</td></tr><tr><td>Estonian (et)</td><td>Euroopas valitseb ajakirjandusvabadus.</td></tr><tr><td>Finnish (fi)</td><td>Meillä on Euroopassa lehdistönvapaus.</td></tr><tr><td>French (fr)</td><td>En Europe la presse est libre.</td></tr><tr><td>Hungarian (hu)</td><td>Európában sajtószabadság van.</td></tr><tr><td>Italian (it)</td><td>in Europa vige la libertà di stampa.</td></tr><tr><td>Lithuanian (lt)</td><td>Europoje naudojamės spaudos laisve.</td></tr><tr><td>Latvian (lv)</td><td>Eiropā pastāv preses brīvība.</td></tr><tr><td>Dutch (nl)</td><td>In Europa hebben we persvrijheid.</td></tr><tr><td>Polish (pl)</td><td>W Europie obowiązuje wolność prasy.</td></tr><tr><td>Portuguese (pt)</td><td>Na Europa, existe liberdade de imprensa.</td></tr><tr><td>Romanian (ro)</td><td>În Europa există libertatea presei.</td></tr><tr><td>Slovak (sk)</td><td>V Európe máme slobodu tlače.</td></tr><tr><td>Slovenian (sl)</td><td>V Evropi imamo svobodo tiska.</td></tr><tr><td>Swedish (sv)</td><td>I EU har vi tryckfrihet och pressfrihet.</td></tr></table>

To showcase the potential of 21-EuroParl, we conduct a study on the political fairness of multilingual Large Language Models (LLMs). This question is critical as LLMs are used daily through chatbots by hundreds of millions of users (Milmo et al., 2023; Top Websites Ranking, 2025) and are increasingly embedded in democratic procedures and processes (Small et al., 2023; Tessler et al., 2024; Revel and Penigaud, 2025). However, their biases have so far mostly been assessed through

<footnote n="1">¹Also available via OPUS (Tiedemann, 2012).</footnote>
<table><caption>Table 1: 21-EuroParl compared to related parallel datasets. OL: Original Language.</caption><thead><tr><th>Dataset</th><th># Lang. Pairs</th><th>Sent. Align.</th><th>Multiparallel</th><th>EU Committee</th><th>EU Party</th><th>National Party</th><th>OL</th></tr></thead><tbody><tr><td>LinkedEP</td><td>420</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>MPDE</td><td>42</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td></tr><tr><td>DCEP</td><td>253</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>EuroParl-UdS</td><td>2</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>EuroParl v10</td><td>420</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>Europarl-ST</td><td>12</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>21-EuroParl</td><td>420</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr></tbody></table>

simulated surveys, written in English (Feng et al., 2023; Rozado, 2023; Hartmann et al., 2023; Santurkar et al., 2023; Durmus et al., 2024; Motoki et al., 2024; Potter et al., 2024; Boelaert et al., 2025; Röttger et al., 2024; Ceron et al., 2024). The propose an alternative framing of political biases and ask the following question: *does LLM translation quality systematically vary depending on the political content of the source text?* The answer is not straightforward as existing Machine Translation metrics are not well calibrated across language pairs. We therefore introduce a novel way to aggregate translation scores across multiple language pairs and partisan speeches based on Borda counts. We find systematic differences in scores depending on the source languages, as well as the party affiliation, across several model sizes and families. In summary, our contributions are threefold:

* a new, 21-way multiparallel version of EuroParl, including the political affiliation of speakers (Section 3);

* a new Borda-based method to aggregate rankings across language pairs (Section 4);

* observations regarding the politically unfair translation of LLMs (Section 5).

We make our dataset, code, and translation outputs available for reproducibility<fnref n="2" />.

## 2. Related Work

### 2.1. From Parliamentary Speeches into Multilingual Corpora

The preparation and release of EuroParl based on the processing of speeches at the EP (Koehn, 2005) has had a major impact on machine translation for European languages. Since then, multiple works have, like ours, extended EuroParl with additional information or metadata, such as the original source language or the political affiliation of speakers. For instance, CoStEP (Graën et al.,

<figure><img src="image_3.png" /><figcaption>Figure 2: Countries covered in 21-EuroParl, members of the European Union in 2009-2011 (Kolja21, CC BY 3.0, via Wikimedia Commons)</figcaption></figure>

2014) includes multiple automatic linguistic annotations enabling various types of cross-lingual corpus analyses. However, they are often limited to a small number of languages, see Table 1. EuroParl-UdS (Karakanta et al., 2018) extends EuroParl with metadata about the original source language and the political affiliation of the speaker, focusing on only three languages. MPDE (Amponsah-Kaakyire et al., 2021) augments EuroParl with information about the original source language and about the use of pivot translation through a third language (e.g. FR-EN then EN-DE instead of FR-DE directly). As this information is not explicitly available, they approximate it by considering all translations from 2004 onward as performed through a pivot language. Moreover, they are limited to 7 languages and do not align texts at the sentence level. Iranzo-Sánchez et al. (2020) focus on speech translation and, like us, rely on LinkedEP to collect metadata. However, their study is also limited to a subset of four languages, dubbed Europarl-ST.

The undocumented EuroParl v10 now includes speaker metadata, notably their EU party affiliation.<fnref n="3" /> However, it is lacking the mention of the relevant EU committee as well as the national party

<footnote n="2">²https://github.com/PaulLerner/21-EuroParl</footnote>

<footnote n="3">³https://www.statmt.org/europarl/v10/</footnote>
affiliation. Moreover, it is only biparallel, which prevents comparable analysis across languages.

Related to parliamentary debates, the DCEP dataset covers various documents from the European Parliament, such as press releases (Hajlaoui et al., 2014). However, to avoid overlap with EuroParl, it does not contain the verbatim reports of speeches made in the European Parliament's plenary. Moreover, it is only biparallel and does not include metadata.

## 2.2. Political Biases of LLMs

To the best of our knowledge, this work is the first to leverage Machine Translation as a way to question the political biases of multilingual LLMs. Indeed, most existing works on political biases: (a) focus on monolingual, English-centric LLMs; (b) rely on questionnaires “administered” to LLMs (e.g., querying a model about a degree of agreement with statements such as “*Sex outside marriage is usually immoral*”).

More precisely, we refer to the following works: (Feng et al., 2023; Rozado, 2023; Hartmann et al., 2023; Santurkar et al., 2023; Durmus et al., 2024; Motoki et al., 2024; Potter et al., 2024). Following works have demonstrated the brittleness of the questionnaire approach, as the computation of LLMs' answers is sensitive to details of the prompt format (e.g., using a new line instead of a space), also yielding a lack of overall coherence (Boelaert et al., 2025; Röttger et al., 2024; Ceron et al., 2024). Röttger et al. (2025) differ from these works as they assess the stance of LLMs when prompted to generate texts about debated issues, while still limited to English and a US-centric context. We depart from these works and, instead of measuring political biases of LLMs based on the detected political leaning of the text they produce, we frame it in terms of a *group-fairness analysis*, based on a practical application (Machine Translation), and assess biases over a large set of 420 language pairs. This allows us to also cover a much more diverse set of texts, reflecting the fact that the political spectrum of the European Union is arguably more varied than that of the bipartisan US, which largely maps to a liberal-conservative axis (Ramaciotti et al., 2024; Vendeville et al., 2025). In the EU, other ideological dimensions emerge, e.g., Green-Alternative-Libertarian versus Traditional- Authoritarian-Nationalist (aka GAL-TAN), Urban-Rural, or Pro-Anti Immigration (Jolly et al., 2022).

Our methodology thus borrows from the *fairness* literature (Verma and Rubin, 2018; Czarnowska et al., 2021; Liang et al., 2023; Barocas et al., 2023; Gallegos et al., 2024), as we treat political affiliation as a *protected attribute* which models should not be sensitive to (instead of more commonly-used attributes such as gender (Savoldi et al., 2022)).

<figure><img src="image_4.png" /><figcaption>Figure 3: Illustration of various alignment scenarios: (left) a 1-FR-2-EN alignment may result in a 1-FR-4-DE alignment if each EN sentence is itself aligned to 2 DE sentences; (top, right) although EN-DE is well aligned, FR-EN is not, so we reject the multialignment; (bottom, right) valid multialignment: all bialignments are 1-1 and have a score above the threshold of 0.8</figcaption></figure>

# 3. Preparing 21-EuroParl

## 3.1. LinkedEP

21-EuroParl heavily builds upon LinkedEP (Noy et al., 2017), which represents the European Parliament's debates as a knowledge graph in which speeches, speakers, and political parties are nodes linked by relations.

As LinkedEP is stored in RDF triples, we query it using SPARQL to retrieve, for each speech, all translations as well as their metadata. LinkedEP contains 191,921 speeches, partially translated into 23 languages. However, HR (Croatian) and MT (Maltese) are hardly covered<fnref n="4" /> so we exclude them to arrive at the 21 languages of 21-EuroParl: BG, CS, DA, DE, EL, EN, ES, ET, FI, FR, HU, IT, LT, LV, NL, PL, PT, RO, SK, SL, and SV (ISO 639-1 codes).

Not all the 191,921 speeches are available in 21 languages: we found that many texts tagged as translations in LinkedEP were in fact duplicates of the source speech. After cleaning using fastText's language identification tool (Joulin et al., 2017), we were left with 37,799 speeches that were available in 21 languages. Less than a thousand were then filtered out because of missing metadata. The next important step was to compute a sentence alignment.

## 3.2. Sentence Alignment

Sentence alignment was performed with Bertalign (Liu and Zhu, 2023), which uses LaBSE sentence embeddings (Feng et al., 2022). After initial experi-

<footnote n="4">⁴Croatian only acquired the status of an official language of the EU in 2013. Maltese and Irish were, until recently, subject to a derogatory status, likely due to a lack of trained interpreters/translators, and they were only partially supported. Ireland and Malta also have English as a co-official language.</footnote>
<table><caption>Table 2: Summary of the available metadata in 21-EuroParl</caption><thead><tr><td>Metadata</td><td>Unique Values</td></tr></thead><tbody><tr><td>Speech</td><td>27,873</td></tr><tr><td>Speaker</td><td>1,094</td></tr><tr><td>National party</td><td>272</td></tr><tr><td>EU party</td><td>12</td></tr><tr><td>EU committee</td><td>25</td></tr><tr><td>Country</td><td>27</td></tr></tbody></table>

ments, we replaced Bertalign's original sentence segmenter<fnref n="5" /> by Trankit's<fnref n="6" /> (Nguyen et al., 2021).

Bertalign computes alignments for pairs of languages (bialignments). We first performed English-centric bialignments, from English to the 20 other languages, yielding 20 bialignments per speech. To control the alignment quality and avoid one-to-many alignments propagating into one-to-many (see Figure 3), we only kept examples with one-to-one alignments in the 20 bialignments. Furthermore, we filtered out examples if any of their bialignment had a Bertalign score (LaBSE's cosine similarity between the source and target sentences) of less than 0.8.

The entire process yielded 72,234 examples (1,516,914 sentences) aligned in 21 languages, a reduction of about 76% from the original LinkedEP.

We will make alignments available so that the dataset can easily be extended with less conservative thresholds or one-to-many alignments.

### 3.3. Overview of the dataset

21-EuroParl spans over three years, from 2009 to 2011 (included). While we aligned the speeches at the sentence level, we kept the speech identifier of each example along with the date of the debate and the language in which the speech was originally utilized. Importantly, we also have a unique identifier for the 1000+ speakers in the dataset, along with their political affiliation: national party (and corresponding country), EU party, and EU committee (see Figure 1). The dataset covers 27 countries (see Figure 2), 12 EU parties, 25 EU committees, and hundreds of national parties. These numbers are summarized in Table 2. The entire dataset contains 40,276,205 words<fnref n="7" /> and 248,764,280 characters, a breakdown per language is in Table 3.

We split the dataset chronologically, reserving year 2009 for training, 2010 for validation, and 2011 for testing. All results in this study are computed on

<table><caption>Table 3: Total word and character counts in the 72,234 sentences for each 21-EuroParl language</caption><thead><tr><th>Language</th><th># Words</th><th># Chars</th></tr></thead><tbody><tr><td>BG</td><td>2,057,994</td><td>12,032,635</td></tr><tr><td>CS</td><td>1,778,026</td><td>10,668,881</td></tr><tr><td>DA</td><td>1,949,508</td><td>11,414,059</td></tr><tr><td>DE</td><td>2,027,728</td><td>13,216,642</td></tr><tr><td>EL</td><td>2,140,104</td><td>13,126,027</td></tr><tr><td>EN</td><td>2,088,541</td><td>11,541,038</td></tr><tr><td>ES</td><td>2,240,743</td><td>12,706,854</td></tr><tr><td>ET</td><td>1,524,824</td><td>10,738,616</td></tr><tr><td>FI</td><td>1,487,870</td><td>11,628,885</td></tr><tr><td>FR</td><td>2,213,235</td><td>13,027,959</td></tr><tr><td>HU</td><td>1,802,043</td><td>12,171,311</td></tr><tr><td>IT</td><td>2,027,404</td><td>12,546,078</td></tr><tr><td>LT</td><td>1,651,547</td><td>10,893,475</td></tr><tr><td>LV</td><td>1,682,997</td><td>10,807,953</td></tr><tr><td>NL</td><td>2,119,408</td><td>12,763,061</td></tr><tr><td>PL</td><td>1,781,796</td><td>11,921,243</td></tr><tr><td>PT</td><td>2,182,858</td><td>12,322,265</td></tr><tr><td>RO</td><td>2,102,942</td><td>12,336,509</td></tr><tr><td>SK</td><td>1,777,101</td><td>10,948,052</td></tr><tr><td>SL</td><td>1,792,083</td><td>10,570,973</td></tr><tr><td>SV</td><td>1,847,453</td><td>11,381,764</td></tr></tbody><tfoot><tr><td>Total</td><td>40,276,205</td><td>248,764,280</td></tr></tfoot></table>

<figure><img src="image_5.png" /><figcaption>Figure 4: Composition of the 7th European Parliament, representing EU parties in 21-EuroParl, (Ssolbergj, CC BY-SA 3.0, via Wikimedia Commons)</figcaption></figure>

the test set of 23,386 examples; the split is provided to facilitate future research. The eight EU parties represented in the test set are displayed in Figure 4.

<footnote n="5">⁵https://github.com/mediacloud/sentence-splitter</footnote>

<footnote n="6">⁶github.com/nlp-uoregon/trankit</footnote>

<footnote n="7">⁷Word tokenization is performed using spacy's multi-lingual tokenizer https://spacy.io/models/xx.</footnote>
<table><caption>Table 4: sBLEU scores for Qwen3-8B: source languages in rows and target languages in columns. Scores are comparable within the same column (target language) but not across columns (within the same row, i.e. source language).</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>18.7</td><td>21.7</td><td>22.2</td><td>20.2</td><td>40.0</td><td>33.4</td><td>8.9</td><td>12.0</td><td>27.8</td><td>13.2</td><td>22.4</td><td>13.0</td><td>13.7</td><td>20.3</td><td>17.8</td><td>26.0</td><td>26.3</td><td>18.5</td><td>16.5</td><td>20.2</td></tr><tr><td>CS</td><td>22.0</td><td>-</td><td>20.3</td><td>21.7</td><td>18.8</td><td>36.3</td><td>30.3</td><td>8.6</td><td>11.3</td><td>25.7</td><td>12.4</td><td>21.0</td><td>11.7</td><td>12.4</td><td>19.2</td><td>17.4</td><td>23.3</td><td>23.5</td><td>26.2</td><td>15.6</td><td>18.5</td></tr><tr><td>DA</td><td>21.4</td><td>17.1</td><td>-</td><td>22.9</td><td>18.4</td><td>39.7</td><td>31.5</td><td>8.8</td><td>11.9</td><td>26.9</td><td>12.7</td><td>21.4</td><td>11.4</td><td>12.5</td><td>20.1</td><td>15.8</td><td>24.0</td><td>23.4</td><td>16.7</td><td>14.6</td><td>25.4</td></tr><tr><td>DE</td><td>21.9</td><td>18.3</td><td>22.4</td><td>-</td><td>19.0</td><td>39.2</td><td>32.6</td><td>9.2</td><td>12.1</td><td>27.4</td><td>12.9</td><td>22.2</td><td>11.9</td><td>12.3</td><td>22.5</td><td>16.6</td><td>24.8</td><td>24.8</td><td>18.0</td><td>15.8</td><td>21.2</td></tr><tr><td>EL</td><td>22.6</td><td>15.8</td><td>20.0</td><td>19.7</td><td>-</td><td>35.8</td><td>32.1</td><td>6.3</td><td>10.7</td><td>26.5</td><td>11.5</td><td>21.1</td><td>9.9</td><td>11.1</td><td>19.2</td><td>15.1</td><td>22.5</td><td>22.8</td><td>16.3</td><td>14.6</td><td>18.1</td></tr><tr><td>EN</td><td>29.3</td><td>22.5</td><td>29.0</td><td>29.5</td><td>25.8</td><td>-</td><td>45.1</td><td>10.6</td><td>14.6</td><td>35.9</td><td>14.8</td><td>28.7</td><td>14.3</td><td>15.6</td><td>24.6</td><td>20.7</td><td>34.3</td><td>35.3</td><td>21.9</td><td>19.3</td><td>26.8</td></tr><tr><td>ES</td><td>24.6</td><td>18.6</td><td>22.6</td><td>23.8</td><td>22.0</td><td>44.3</td><td>-</td><td>9.3</td><td>12.3</td><td>31.7</td><td>13.0</td><td>26.1</td><td>12.6</td><td>12.8</td><td>21.2</td><td>17.5</td><td>32.4</td><td>29.4</td><td>18.5</td><td>16.5</td><td>21.0</td></tr><tr><td>ET</td><td>17.8</td><td>14.1</td><td>16.4</td><td>17.2</td><td>14.8</td><td>28.7</td><td>25.3</td><td>-</td><td>10.9</td><td>21.5</td><td>10.9</td><td>16.5</td><td>10.3</td><td>9.3</td><td>16.1</td><td>13.4</td><td>19.7</td><td>18.7</td><td>13.4</td><td>11.9</td><td>15.7</td></tr><tr><td>FI</td><td>18.0</td><td>14.2</td><td>17.5</td><td>18.0</td><td>15.6</td><td>30.2</td><td>26.7</td><td>6.9</td><td>-</td><td>22.8</td><td>11.6</td><td>17.1</td><td>9.6</td><td>9.7</td><td>17.1</td><td>13.2</td><td>20.5</td><td>19.5</td><td>13.7</td><td>12.1</td><td>17.1</td></tr><tr><td>FR</td><td>22.5</td><td>17.4</td><td>20.9</td><td>22.3</td><td>20.2</td><td>39.2</td><td>35.5</td><td>8.4</td><td>11.0</td><td>-</td><td>12.6</td><td>26.0</td><td>11.8</td><td>12.5</td><td>20.8</td><td>16.6</td><td>28.7</td><td>27.1</td><td>16.9</td><td>14.9</td><td>19.6</td></tr><tr><td>HU</td><td>18.5</td><td>14.1</td><td>17.1</td><td>18.2</td><td>15.8</td><td>30.6</td><td>27.2</td><td>7.1</td><td>10.2</td><td>22.7</td><td>-</td><td>18.0</td><td>9.8</td><td>10.1</td><td>16.8</td><td>13.9</td><td>20.2</td><td>19.6</td><td>13.3</td><td>12.4</td><td>16.1</td></tr><tr><td>IT</td><td>20.7</td><td>16.2</td><td>19.4</td><td>18.9</td><td>19.2</td><td>36.0</td><td>33.1</td><td>8.0</td><td>10.5</td><td>28.6</td><td>11.9</td><td>-</td><td>10.9</td><td>11.2</td><td>19.6</td><td>15.7</td><td>26.9</td><td>24.9</td><td>15.8</td><td>13.6</td><td>18.0</td></tr><tr><td>LT</td><td>18.4</td><td>13.7</td><td>15.9</td><td>16.1</td><td>14.7</td><td>28.6</td><td>24.8</td><td>6.8</td><td>10.0</td><td>21.2</td><td>10.2</td><td>16.1</td><td>-</td><td>7.9</td><td>15.6</td><td>13.2</td><td>18.8</td><td>18.0</td><td>13.2</td><td>11.5</td><td>15.2</td></tr><tr><td>LV</td><td>19.6</td><td>15.0</td><td>17.5</td><td>17.5</td><td>15.6</td><td>31.4</td><td>26.9</td><td>6.4</td><td>10.9</td><td>22.9</td><td>11.4</td><td>17.8</td><td>10.8</td><td>-</td><td>17.0</td><td>14.3</td><td>20.4</td><td>19.9</td><td>14.1</td><td>12.4</td><td>16.2</td></tr><tr><td>NL</td><td>19.3</td><td>15.4</td><td>19.3</td><td>21.9</td><td>16.8</td><td>33.7</td><td>28.5</td><td>8.0</td><td>10.5</td><td>25.1</td><td>11.6</td><td>20.1</td><td>10.7</td><td>11.0</td><td>-</td><td>14.7</td><td>21.9</td><td>21.2</td><td>14.6</td><td>13.2</td><td>18.8</td></tr><tr><td>PL</td><td>21.4</td><td>17.2</td><td>19.1</td><td>19.5</td><td>17.4</td><td>30.5</td><td>28.9</td><td>8.4</td><td>10.7</td><td>24.7</td><td>11.7</td><td>19.9</td><td>11.2</td><td>11.8</td><td>18.5</td><td>-</td><td>22.4</td><td>22.4</td><td>17.1</td><td>14.5</td><td>17.8</td></tr><tr><td>PT</td><td>23.8</td><td>17.6</td><td>21.4</td><td>22.5</td><td>21.2</td><td>42.8</td><td>39.4</td><td>8.8</td><td>11.8</td><td>31.6</td><td>12.5</td><td>25.9</td><td>12.0</td><td>12.1</td><td>20.6</td><td>17.4</td><td>-</td><td>28.4</td><td>17.3</td><td>15.5</td><td>20.0</td></tr><tr><td>RO</td><td>24.3</td><td>18.4</td><td>22.0</td><td>22.2</td><td>22.5</td><td>44.1</td><td>37.3</td><td>8.4</td><td>12.0</td><td>31.2</td><td>12.9</td><td>24.5</td><td>11.9</td><td>12.5</td><td>21.0</td><td>17.8</td><td>29.4</td><td>-</td><td>18.0</td><td>15.7</td><td>20.6</td></tr><tr><td>SK</td><td>22.7</td><td>24.8</td><td>20.6</td><td>22.1</td><td>19.2</td><td>37.2</td><td>31.0</td><td>8.2</td><td>11.4</td><td>26.1</td><td>12.5</td><td>21.4</td><td>11.7</td><td>12.1</td><td>19.2</td><td>17.5</td><td>23.7</td><td>24.0</td><td>-</td><td>14.6</td><td>18.6</td></tr><tr><td>SL</td><td>21.7</td><td>19.3</td><td>20.0</td><td>21.2</td><td>18.6</td><td>36.2</td><td>30.5</td><td>7.9</td><td>11.4</td><td>25.5</td><td>12.5</td><td>20.6</td><td>11.5</td><td>11.4</td><td>18.9</td><td>16.5</td><td>23.4</td><td>23.2</td><td>18.7</td><td>-</td><td>18.4</td></tr><tr><td>SV</td><td>20.6</td><td>15.9</td><td>25.7</td><td>21.7</td><td>17.2</td><td>37.5</td><td>29.6</td><td>8.8</td><td>12.1</td><td>25.7</td><td>11.9</td><td>20.7</td><td>10.9</td><td>11.6</td><td>19.6</td><td>15.3</td><td>22.5</td><td>22.6</td><td>15.3</td><td>13.8</td><td>-</td></tr></tbody></table>

# 4. Experiments

## 4.1. Method

As a first study, we would like to assess whether the quality of translations provided by multilingual LLMs varies depending on the EU party,<fnref n="8" /> which may indicate a political bias of the model. We thus generate automatic translations of each speech, considering all possible directions of the 21-way multiparallel dataset, that is $21 \times 20 = 420$ language pairs.

Translation quality is estimated with two standard reference-based metrics: sBLEU<fnref n="9" />, i.e. per-sentence BLEU (Papineni et al., 2002; Chen and Cherry, 2014), and COMET<fnref n="10" /> (Rei et al., 2022). It is however well-known that these metrics are not well calibrated across language pairs (Bugliarello et al., 2020; Zouhar et al., 2024) and that the scores cannot be easily compared. More precisely, COMET scores are not well calibrated across language pairs (Zouhar et al., 2024), likely because it inputs not only the hypothesis and reference, but also the source sentence. sBLEU, while also not well calibrated across target languages due e.g. to tokenizations issues, still enables to compare multiple source languages translating into the same target language (Section 5.2), as it only relies on a reference and a hypothesis in that target language.

Therefore, to measure political fairness based on sBLEU and COMET, we compute a Borda Count of EU parties per language pair (Section 5.3). Borda

Count is a simple, yet effective, way to aggregate rankings. For a given model, metric, and language pair, all EU parties are ranked according to the metric score. The EU party with the lowest score gets 0 points, the second-to-last gets 1 point, etc. until the best one gets 7 points (as there are 8 parties). While initially proposed as a voting system, Borda Counts has been used in Information Retrieval to aggregate rankings (Aslam and Montague, 2001; Bassani and Romelli, 2022). It has also been used more recently for benchmarking models across tasks with different metrics (Colombo et al., 2022) or meta-evaluating metrics across language pairs (Freitag et al., 2023). Our method differs from the usual fairness metrics, e.g. demographic parity or group calibration, that are designed for classification settings (Czarnowska et al., 2021; Barocas et al., 2023).

## 4.2. Implementation

We experiment with two families of posttrained LLMs: Qwen3 (Yang et al., 2025), in two sizes (4B and 8B), and Llama-3.1-8B (Grattafiori et al., 2024).

We originally carried out experiments with the smaller Llama-3.2 (in sizes 1B and 3B) but found the translation scores overall too poor to be reliably compared across parties (for most language pairs, sBLEU scores were below 10, see Appendix A.1). Llama-3.1 is a multilingual model trained with a very large share of English texts, accounting for 92% of the pretraining data (or 75% when excluding code) and 97% English posttraining data (or 82% when excluding code). Little is known about Qwen3's data mix, except that it covers 119 languages.

We use a standard prompt template for zero-shot translation: `"<src lang>:"`

<footnote n="8">⁸We focus only on EU parties, given the exceedingly large number of national parties.</footnote>

<footnote n="9">⁹nrefs:1|case:mixed|eff:yes|tok:13a| smooth:exp|version:2.5.1 (Post, 2018)</footnote>

<footnote n="10">¹⁰wmt22-comet-da</footnote>
<table><caption>Table 5: Borda Count of source language rankings averaged over the 21 target languages (from 0-19, the higher the better), based on sBLEU rankings.</caption><thead><tr><th>Model</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>Fair Model</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td><td>9.5</td></tr><tr><td>Qwen3-4B</td><td>14.9</td><td>11.6</td><td>12.9</td><td>12.9</td><td>6.8</td><td>18.9</td><td>17.1</td><td>1.1</td><td>3.6</td><td>13.1</td><td>3.2</td><td>8.8</td><td>1.2</td><td>4.3</td><td>6.3</td><td>9.1</td><td>12.5</td><td>14.9</td><td>10.5</td><td>7.5</td><td>8.8</td></tr><tr><td>Qwen3-8B</td><td>15.4</td><td>10.9</td><td>12.0</td><td>14.0</td><td>7.3</td><td>18.9</td><td>17.2</td><td>1.5</td><td>2.6</td><td>13.2</td><td>2.7</td><td>8.5</td><td>0.2</td><td>3.5</td><td>5.6</td><td>6.7</td><td>14.4</td><td>15.4</td><td>11.6</td><td>9.4</td><td>9.0</td></tr><tr><td>Llama-3.1-8B</td><td>15.0</td><td>10.9</td><td>9.1</td><td>13.9</td><td>13.8</td><td>18.9</td><td>17.2</td><td>1.3</td><td>2.5</td><td>13.5</td><td>3.1</td><td>9.2</td><td>0.6</td><td>2.6</td><td>5.7</td><td>7.4</td><td>14.4</td><td>13.2</td><td>11.9</td><td>8.0</td><td>7.9</td></tr></tbody></table>

<figure><img src="image_6.png" /><figcaption>Figure 5: sBLEU against COMET scores of the 420 language pairs for each model. A few language pairs are annotated (source → target language) along the quantiles.</figcaption></figure>

<src_text>\n<tgt_long>" where <src_long> and <tgt_long> are replaced by the English name of the source and target languages, respectively, and <src_text> is replaced by the source sentence. For example: "English: In Europe, we have freedom of the press.\nFrench:". This prompt is then formatted as a user query using model-specific templates. We use the recommended decoding parameters, i.e. nucleus sampling with p = 0.9 and temperature τ = 0.6 (Holtzman et al., 2020). Models are implemented using vLLM (Kwon et al., 2023).

Approximately 1,500 V100 GPU hours were used for the experiments.

# 5. Results

## 5.1. Overall Translation Quality

For the 23,386 examples of the test set, we compute sBLEU and COMET scores for the 420 language pairs, giving 9,822,120 observations for each metric. We report the average sBLEU scores of Qwen3-8B per language direction (source language in rows and target languages in columns) in Table 4 (results for all models are reported in Appendix A). This model overall achieves good translation performance, especially for high-resource pairs such as EN-ES where sBLEU scores exceed 40. However, results are not uniform and even for the most favorable case (translating into English), sBLEU scores vary by a wide margin. This is also reflected in Figure 5 where we plot sBLEU against COMET scores for the 420 language pairs for each model. We can see that Qwen3-8B slightly outperforms Llama-3.1-8B and that Qwen3-4B underperforms, but the tendencies across language pairs are largely similar: in fact, the sBLEU scores of Qwen3-4B and Qwen3-8B have a Pearson ρ correlation of .98, as well as that of Qwen3-8B and Llama-3.1-8B. We can also see a strong correlation of the two metrics: the COMET and sBLEU scores of Qwen3-4B have a Pearson ρ correlation of .91, although this number goes down to .77 for Qwen3-8B and Llama-3.1-8B. See (Kocmi et al., 2024) on how to interpret COMET scores.

## 5.2. Language Fairness

Before diving into the political fairness of LLMs, we first study their language fairness, trying to evaluate which language do they translate the worst, and which one do they translate the best. A hasty conclusion would rely on the average of sBLEU/COMET scores across language pairs, to find which language gets the highest score when translating into/out of it. However, COMET scores are not well calibrated across language pairs (Section 4.1). sBLEU, while not well calibrated across target languages, still enables to compare multiple source languages translating towards the same target language.<fnref n="11" /> Therefore, we apply the Borda method described above as follows: for each target language, we rank all source languages and assign 0 points to the worst, up to 19 points to the best. For example in the BG column of Table 4, ET-BG has the worst sBLEU score of 17.8, so it gets 0 points, while EN-BG has the best score of 29.3 so it gets 19 points.

Results are reported in Table 5. As expected, we find that EN is the best translated language,<fnref n="12" />

<footnote n="11">¹¹Assuming that the source texts are similar, as is the case in this section.</footnote>

<footnote n="12">¹²The only target languages where it is not ranked first,</footnote>
<table><caption>Table 6: Borda Count of political parties rankings averaged over the 420 language pairs (from 0-7, the higher the better), based on sBLEU or COMET rankings. The *p* < 0.01 column shows how many language pairs (out of 420) have their performance variance significantly explained by the party variable according to a Kruskal-Wallis test.</caption><thead><tr><th>Model</th><th>Ranking</th><th><i>p</i> &lt; 0.01</th><th>ALDE</th><th>ECR</th><th>EFA</th><th>EFD</th><th>EPP</th><th>NA</th><th>NGL</th><th>S&amp;D</th></tr></thead><tbody><tr><td>Fair Model</td><td>–</td><td>0</td><td>3.50</td><td>3.50</td><td>3.50</td><td>3.50</td><td>3.50</td><td>3.50</td><td>3.50</td><td>3.50</td></tr><tr><td>Qwen3-4B</td><td>sBLEU</td><td>360</td><td>4.98</td><td>3.23</td><td>3.70</td><td>3.06</td><td>5.17</td><td>1.45</td><td>1.41</td><td>4.99</td></tr><tr><td>Qwen3-8B</td><td>sBLEU</td><td>363</td><td>5.13</td><td>3.02</td><td>3.55</td><td>3.08</td><td>5.17</td><td>1.65</td><td>1.18</td><td>5.22</td></tr><tr><td>Llama-3.1-8B</td><td>sBLEU</td><td>304</td><td>4.83</td><td>3.73</td><td>3.63</td><td>2.84</td><td>4.56</td><td>2.20</td><td>1.08</td><td>5.11</td></tr><tr><td>Qwen3-4B</td><td>COMET</td><td>418</td><td>4.51</td><td>3.20</td><td>1.63</td><td>4.49</td><td>6.72</td><td>0.79</td><td>0.94</td><td>5.72</td></tr><tr><td>Qwen3-8B</td><td>COMET</td><td>419</td><td>4.82</td><td>3.41</td><td>1.47</td><td>4.02</td><td>6.64</td><td>1.03</td><td>0.80</td><td>5.82</td></tr><tr><td>Llama-3.1-8B</td><td>COMET</td><td>419</td><td>4.94</td><td>4.15</td><td>1.13</td><td>3.58</td><td>6.37</td><td>1.72</td><td>0.50</td><td>5.61</td></tr></tbody></table>

consistently across models, followed by ES. Unlike what Table 4 would suggest (the ET column has the lowest sBLEU scores), we find that LT is the worst translated language across models, especially for Qwen3-8B where it is almost always ranked last. ET is also among the worst translated languages. A surprising result is that DE and FR, which have a special status within the EP, have a lower rank than SP, but also than BG and RO, reflecting perhaps a small representation of these two languages in the data mix of Llama and Qwen. We also observe, for all models, that all of the 21 target languages have their performance variance significantly explained by the source language variable according to a Kruskal-Wallis test (a non-parametric version of ANOVA; Kruskal and Wallis, 1952) – confirming that some source languages are more difficult to process than others (Bugliarello et al., 2020).

An ideally language-fair translation model, on the contrary, would not have its performance variance explained by the source language. It would have near identical performance for all source languages, i.e. random rankings for each target language, which would result in the same Borda Count for all source languages: 9.5.

## 5.3. Political Fairness

We now turn to the political fairness of LLMs. We apply the same Borda method to rank political parties across language pairs. This can also be done with COMET, as COMET scores are comparable within the same language pair.

Results are reported in Table 6. We find that EPP, S&D, and ALDE get the best translation quality and NA and NGL the worst, across all models and metrics. The preferences are very marked, with EPP nearing 7 points (i.e. always ranked first) and NGL nearing 0 points (i.e. always ranked last),

<table><caption>Table 7: sBLEU scores of Qwen3-8B per target language. Each target language has its performance variance significantly explained by the party variable according to a Kruskal-Wallis test (*p* < 0.01).</caption><thead><tr><th>tgt</th><th>ALDE</th><th>ECR</th><th>EFA</th><th>EFD</th><th>EPP</th><th>NA</th><th>NGL</th><th>S&amp;D</th></tr></thead><tbody><tr><td>BG</td><td>20.83</td><td>21.56</td><td>21.28</td><td>21.59</td><td>21.81</td><td>19.80</td><td>20.88</td><td>21.93</td></tr><tr><td>CS</td><td>17.21</td><td>16.90</td><td>16.80</td><td>17.29</td><td>17.37</td><td>16.47</td><td>15.99</td><td>17.46</td></tr><tr><td>DA</td><td>20.91</td><td>20.35</td><td>20.60</td><td>19.88</td><td>20.55</td><td>19.27</td><td>19.93</td><td>20.52</td></tr><tr><td>DE</td><td>21.35</td><td>21.73</td><td>20.38</td><td>21.00</td><td>21.10</td><td>18.10</td><td>19.70</td><td>21.34</td></tr><tr><td>EL</td><td>18.70</td><td>18.19</td><td>18.21</td><td>18.16</td><td>18.89</td><td>17.29</td><td>17.27</td><td>19.08</td></tr><tr><td>EN</td><td>36.04</td><td>35.65</td><td>34.67</td><td>36.00</td><td>36.35</td><td>34.65</td><td>35.01</td><td>36.60</td></tr><tr><td>ES</td><td>30.81</td><td>30.05</td><td>29.85</td><td>31.39</td><td>32.20</td><td>30.22</td><td>30.66</td><td>31.52</td></tr><tr><td>ET</td><td>8.62</td><td>8.27</td><td>8.67</td><td>7.91</td><td>8.16</td><td>8.14</td><td>7.66</td><td>8.19</td></tr><tr><td>FI</td><td>12.01</td><td>11.42</td><td>11.73</td><td>10.92</td><td>11.45</td><td>10.77</td><td>10.80</td><td>11.47</td></tr><tr><td>FR</td><td>25.99</td><td>25.45</td><td>25.35</td><td>26.27</td><td>27.19</td><td>24.74</td><td>25.68</td><td>26.78</td></tr><tr><td>HU</td><td>12.39</td><td>11.73</td><td>12.82</td><td>11.97</td><td>12.37</td><td>12.19</td><td>11.55</td><td>12.19</td></tr><tr><td>IT</td><td>21.27</td><td>20.15</td><td>20.70</td><td>21.23</td><td>21.78</td><td>19.94</td><td>20.77</td><td>21.50</td></tr><tr><td>LT</td><td>11.86</td><td>11.10</td><td>11.77</td><td>11.40</td><td>11.41</td><td>11.26</td><td>10.55</td><td>11.43</td></tr><tr><td>LV</td><td>11.91</td><td>11.09</td><td>11.88</td><td>11.63</td><td>11.87</td><td>11.24</td><td>10.75</td><td>11.59</td></tr><tr><td>NL</td><td>20.03</td><td>19.54</td><td>19.05</td><td>19.03</td><td>19.34</td><td>18.89</td><td>18.41</td><td>19.79</td></tr><tr><td>PL</td><td>16.12</td><td>15.39</td><td>15.79</td><td>15.59</td><td>16.20</td><td>15.76</td><td>15.22</td><td>16.21</td></tr><tr><td>PT</td><td>24.34</td><td>23.47</td><td>23.64</td><td>24.41</td><td>24.60</td><td>23.76</td><td>22.26</td><td>24.55</td></tr><tr><td>RO</td><td>23.91</td><td>23.07</td><td>23.58</td><td>23.40</td><td>23.86</td><td>23.09</td><td>22.81</td><td>24.13</td></tr><tr><td>SK</td><td>17.03</td><td>16.99</td><td>16.46</td><td>16.50</td><td>17.00</td><td>16.15</td><td>16.08</td><td>17.07</td></tr><tr><td>SL</td><td>14.57</td><td>14.30</td><td>14.28</td><td>13.93</td><td>14.64</td><td>13.88</td><td>13.96</td><td>14.50</td></tr><tr><td>SV</td><td>19.85</td><td>19.57</td><td>18.89</td><td>18.83</td><td>19.18</td><td>18.43</td><td>18.43</td><td>19.27</td></tr></tbody></table>

especially with COMET. The preferences are especially consistent within the same model family: the Borda Count of Qwen3-4B and Qwen3-8B have a Pearson *ρ* correlation of .99, for both sBLEU- and COMET-based scores. Even across model families, results are very consistent, e.g. the Borda Count based on sBLEU of Qwen3-8B and Llama-3.1-8B have a Pearson *ρ* correlation of .96, up to .98 for COMET-based scores. For most language pairs, the party variable significantly explains the variance of performance according to a Kruskal-Wallis test (*p* < 0.01), especially with COMET where only ET-NL, FI-ET, and FI-LV are not significative, across all models. An ideally fair model, on the contrary, would not have its performance variance explained by the party variable. It would have near identical performance for all parties, i.e. random rankings for each language pair, which would result in the same Borda Count for all parties: 3.5.

While Table 6 reveals consistent political bias

<footnote>across all models, are cs and sk, where it is then ranked second (and sk – resp. cs – is ranked first, the two languages being closely related).</footnote>
towards some parties across language pairs, we
also report the per-target-language sBLEU scores
of Gwen3-8B for every EU party in Table 7 (see
other models in Appendix B). For every model, and
target language, the performance variance is sig-
nificantly explained by the party variable according
to a Kruskal-Wallis test (p < 0.01). Although multi-
lingual LLMs are very English-centric, we find that
their political bias follows similar tendencies across
all target languages, e.g. S&D, EPP, and ALDE are
often the best translated parties, and NGL and NA
often the worst (which is reflected in their Borda
Count in Table 6). Computing a Borda ranking from
Table 7 gives very close results to Table 6,<fnref n="13" /> i.e. our
results are consistent regardless if we aggregate
sBLEU scores per language pair or only per target
language.

6. Discussion

Our first study on 21-EuroParl reveals significant dif-
ferences in translation quality of political speeches
across different EU parties. This result provides a
different perspective on the political biases of multi-
lingual LLMs, favoring the majority parties from left,
center, and right over outsiders such as *European
United Left/Nordic Green Left* and *Non-attached
Members*. Our results are consistent across model
sizes and families. Further analyses are needed to
better understand the origin of these differences,
possibly originating either from the pretraining or
posttraining data (Gehman et al., 2020; Matic et al.,
2020; Hovy and Prabhumoye, 2021; Birhane et al.,
2021; Ceron et al., 2025). An other, complemen-
tary explanation, would relate these differences to
differences in topic or style across EU parties. In
any case, these results call for extreme caution
when using automatic translation systems in polit-
ical processes, such as multilingual deliberations
(Cabrera, 2024).¹⁴ Besides political fairness, we
also found, as expected, unfair translation quality
across source languages, with English and Span-
ish being the best translated, while Estonian and
Lithuanian are the worst translated. These differ-
ences may of course be due to differences in train-
ing data but also to the morphological complexity of
these languages (Marco et al., 2022). A limit of our
Borda metric is that it does not take into account the
magnitude of the differences between the sBLEU
or COMET scores. In future work, we could add

ties to the Borda ranking, based on the significance
of the difference (Deutsch et al., 2021; Freitag et al.,
2023).

While we focused on EU parties, future work may
leverage 21-EuroParl to study the biases regard-
ing national parties or countries. A related ques-
tion is the effect of the original language (in which
the speech was uttered), including the well-known
translationese effect (Kurokawa et al., 2009; Lem-
bersky et al., 2012; Volansky et al., 2013; Läubli
et al., 2018; Toral et al., 2018; Zhang and Toral,
2019). This is challenging as parties are not equally
represented in all original languages, so we could
not apply the same Borda method. We found that
excluding the translationese subset (where the tar-
get language is the original language) did not affect
the results of Section 5.3.

Finally, while we focused on LLMs, 21-EuroParl
may prove useful to study other Machine Transla-
tion algorithms, e.g. examplar-based, statistical,
learning or decoding methods.

7. Limitations

Given the scale of our experiments (23,386 exam-
ples times 420 language pairs times 3 models), we
had to rely on automatic metrics to assess trans-
lation quality. However, our results should be con-
firmed using human judgments at a smaller scale
before taking any actions.

Again, because of the scale of experiments, we
could not afford to run the largest models, e.g. the
235B Gwen3 or 405B Llama-3.1. However, our
experiment with Gwen3 shows that, although the
performance overall improves from 4B to 8B, the
“political bias” stays consistent.

21-EuroParl covers three years, from 2009 (as
Bulgaria and Romania only joined EU in 2007,
along with their respective languages) to 2011 (in-
cluded). However, it could easily be extended, pro-
vided an update of LinkedEP. One concern with
evaluating LLMs on data from 2011 is that the data
may have leaked into their training set. In fact, we
did not experiment with translation-savvy models
such as Tower (Alves et al., 2024) nor EuroLLM
(Martins et al., 2024) because they were trained
using WMT's EuroParl. However, we find that only
a very small portion of the automatic translations
exactly match the reference (i.e. have an sBLEU
of 100), e.g. 0.2% for Gwen3-8B, which suggests
very little contamination.

More generally, we emphasize that our study
is limited to a domain, proceedings of parliamen-
tary debates, and a context, the European Union.
Our findings may differ given a different domain
(e.g. social media), political context, or different
languages.

<footnote n="13">¹³For example, in the XX-BG row of Table 7, S&D gets the highest sBLEU score of 21.93 (averaged over all source languages: CS-BG, DA-BG, etc.) so it gets 7 points, and NA gets the worst score of 19.80, so it gets 0 points.

¹⁴As was done during the “Conference on the Future of Europe”. Available online: https://wayback.archive-it.org/12090/20230418091815/https://futureu.europa.eu/.</footnote>
## 8. Ethical considerations

We study the variation of translation quality according to the political affiliation of European parliamentarians in order to assess the political biases of LLMs. Note that the political affiliation of European parliamentarians is of public interest. Therefore, it is available and can be redistributed, as licensed by existing EU regulations (GDPR).

## 9. Acknowledgements

This research was funded by BPI-France under the project AI For Democracy - Democratic Commons, one of seven winners of BPI-France's "Digital Commons for Generative AI" call for projects, conducted as part of the France 2030 investment plan. We thank all the members of the Democratic Commons program at Sorbonne Université, Sciences Po, and make.org for the discussion and feedback on earlier versions of this work, in particular Nazanin Shafiabadi, Jean-Philippe Cointet, Salim Hafid and David Mas. We also acknowledge the contribution of Hal Daumé III who visited our group in the Spring of 2025. Finally, we thank Ziqian Peng for assistance with sentence alignment. This work was performed using HPC resources from GENCI-IDRIS (Grant 2024-AD010615901).

The authors are not affiliated with any political party.

## 10. Bibliographical References

Duarte Miguel Alves, José Pombal, Nuno M Guerreiro, Pedro Henrique Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and Andre Martins. 2024. Tower: An open multilingual large language model for translation-related tasks. In First Conference on Language Modeling.

Javed A. Aslam and Mark H. Montague. 2001. Models for metasearch. In SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, September 9-13, 2001, New Orleans, Louisiana, USA, pages 275–284. ACM.

Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2023. Fairness and Machine Learning: Limitations and Opportunities. MIT Press.

Elias Bassani and Luca Romelli. 2022. ranx.fuse: A python library for metasearch. In CIKM, pages 4808–4812. ACM.

Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963.

Julien Boelaert, Samuel Coavoux, Étienne Ollion, Ivaylo Petev, and Patrick Prág. 2025. Machine bias. how do generative language models answer opinion polls?1. Sociological Methods & Research, 54(3):1156–1196.

Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 workshop on statistical machine translation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 1–46, Lisbon, Portugal. Association for Computational Linguistics.

Emanuele Bugliarello, Sabrina J. Mielke, Antónios Anastasopoulos, Ryan Cotterell, and Naoaki Okazaki. 2020. It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1640–1649, Online. Association for Computational Linguistics.

Luis Cabrera. 2024. Babel Fish Democracy? Prospects for Addressing Democratic Language Barriers through Machine Translation and Interpretation. American Journal of Political Science, 68(2):767–782.

Tanise Ceron, Neele Falk, Ana Barić, Dmitry Nikolaev, and Sebastian Padó. 2024. Beyond Prompt Brittleness: Evaluating the Reliability and Consistency of Political Worldviews in LLMs. Transactions of the Association for Computational Linguistics, 12:1378–1400.

Tanise Ceron, Dmitry Nikolaev, Dominik Stammbach, and Debora Nozza. 2025. What is the political content in llms' pre-and post-training data? arXiv preprint arXiv:2509.22367.

Boxing Chen and Colin Cherry. 2014. A systematic comparison of smoothing techniques for sentence-level BLEU. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 362–367, Baltimore, Maryland, USA. Association for Computational Linguistics.

Pierre Colombo, Nathan Noiry, Ekhine Irurozki, and Stephan Clémençon. 2022. What are the best Systems? New Perspectives on NLP Benchmarking. Advances in Neural Information Processing Systems, 35:26915–26932.
Paula Czarnowska, Yogarshi Vyas, and Kashif Shah. 2021. Quantifying social biases in NLP: A generalization and empirical comparison of extrinsic fairness metrics. *Transactions of the Association for Computational Linguistics*, 9:1249–1267. Place: Cambridge, MA Publisher: MIT Press.

Anastasia Deligiaouri. 2016. Analysis of the political discourse of European parliament political groups during economic crisis (2008–2014). In *24th World Congress of Political Science*.

Daniel Deutsch, Rotem Dror, and Dan Roth. 2021. A Statistical Analysis of Summarization Evaluation Metrics Using Resampling Methods. *Transactions of the Association for Computational Linguistics*, 9:1132–1146.

Esin Durmus, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. Towards measuring the representation of subjective global opinions in language models. In *First Conference on Language Modeling*.

Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT Sentence Embedding. In *Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 878–891, Dublin, Ireland. Association for Computational Linguistics.

Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 11737–11762, Toronto, Canada. Association for Computational Linguistics.

Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frederic Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie, and George Foster. 2023. Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent. In *Proceedings of the Eighth Conference on Machine Translation*, pages 578–628, Singapore. Association for Computational Linguistics.

Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K.

Ahmed. 2024. Bias and Fairness in Large Language Models: A Survey. *Computational Linguistics*, 50(3):1097–1179.

Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating Neural Toxic Generation in Language Models. In *Findings of the Association for Computational Linguistics: EMNLP 2020*, pages 3356–3369, Online. Association for Computational Linguistics.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. *arXiv preprint arXiv:2407.21783*.

Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. 2023. The political ideology of conversational ai: Converging evidence on chatgpt's pro-environmental, left-libertarian orientation. *Available at SSRN 4316084*.

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In *International Conference on Learning Representations*.

Dirk Hovy and Shrimai Prabhumoye. 2021. Five sources of bias in natural language processing. *Language and Linguistics Compass*, 15(8):e12432.

Bjørn Høyland, Jean-François Godbout, Emanuele Lapponi, and Erik Velldal. 2014. Predicting Party Affiliations from European Parliament Debates. In *Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science*, pages 56–60, Baltimore, MD, USA. Association for Computational Linguistics.

Seth Jolly, Ryan Bakker, Liesbet Hooghe, Gary Marks, Jonathan Polk, Jan Rovny, Marco Steenbergen, and Milada Anna Vachudova. 2022. Chapel Hill Expert Survey trend file, 1999–2019. *Electoral Studies*, 75:102420.

Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of Tricks for Efficient Text Classification. In *Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers*, pages 427–431, Valencia, Spain. Association for Computational Linguistics.

Tom Kocmi, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita,
Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel, and Maja Popović. 2022. Findings of the 2022 conference on machine translation (WMT22). In *Proceedings of the Seventh Conference on Machine Translation (WMT)*, pages 1–45, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.

Tom Kocmi, Vilém Zouhar, Christian Federmann, and Matt Post. 2024. Navigating the metrics maze: Reconciling score magnitudes and accuracies. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 1999–2014, Bangkok, Thailand. Association for Computational Linguistics.

Philipp Koehn and Christof Monz, editors. 2006. *Proceedings on the Workshop on Statistical Machine Translation*. Association for Computational Linguistics, New York City.

William H Kruskal and W Allen Wallis. 1952. Use of ranks in one-criterion variance analysis. *Journal of the American statistical Association*, 47(260):583–621.

David Kurokawa, Cyril Goutte, and Pierre Isabelle. 2009. Automatic detection of translated text and its impact on machine translation. In *Proceedings of Machine Translation Summit XII: Papers*, Ottawa, Canada.

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In *Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles*.

Samuel Läubli, Rico Sennrich, and Martin Volk. 2018. Has machine translation achieved human parity? a case for document-level evaluation. In *Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, pages 4791–4796, Brussels, Belgium. Association for Computational Linguistics.

Gennadi Lembersky, Noam Ordan, and Shuly Winter. 2012. Adapting translation models to translationese improves SMT. In *Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics*, pages 255–265.

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang

Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Andrew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. 2023. Holistic evaluation of language models. *Transactions on Machine Learning Research*. Featured Certification, Expert Certification, Outstanding Certification.

Lei Liu and Min Zhu. 2023. Bertalign: Improved word embedding-based sentence alignment for Chinese-English parallel corpora of literary texts. *Digital Scholarship in the Humanities*, 38(2):621–634.

Marion Weller-Di Marco, Matthias Huck, and Alexander Fraser. 2022. Modeling target-side morphology in neural machine translation: A comparison of strategies. *arXiv preprint arXiv:2203.13550*.

PH Martins, P Fernandes, J Alves, NM Guerreiro, R Rei, DM Alves, J Pombal, A Farajian, M Faysse, M Klimaszewski, et al. 2024. Eurollm: Multilingual language models for europe (arxiv: 2409.16235). arxiv.

Srdjan Matic, Costas Iordanou, Georgios Smaragdakis, and Nikolaos Laoutaris. 2020. Identifying Sensitive URLs at Web-Scale. In *Proceedings of the ACM Internet Measurement Conference*, IMC '20, page 619–633, New York, NY, USA. Association for Computing Machinery.

Dan Milmo et al. 2023. Chatgpt reaches 100 million users two months after launch. *The Guardian*, 3:1017–1054.

Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2024. More human than human: Measuring ChatGPT political bias. *Public Choice*, 198(1):3–23.

Minh Van Nguyen, Viet Dac Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021. Trankit: A light-weight transformer-based toolkit for multilingual natural language processing. In *Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations*.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In *Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics*, pages 311–318.

Matt Post. 2018. *A Call for Clarity in Reporting BLEU Scores*. In *Proceedings of the Third Conference on Machine Translation: Research Papers*, pages 186–191, Brussels, Belgium. Association for Computational Linguistics.

Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. 2024. Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters. In *Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 4244–4275, Miami, Florida, USA. Association for Computational Linguistics.

Pedro Ramaciotti, Duncan Cassells, Zografoula Vagena, Jean-Philippe Cointet, and Michael Bailey. 2024. American politics in 3D: Measuring multidimensional issue alignment in social media using social graphs and text data. *Applied Network Science*, 9(1):2.

Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. COMET-22: Unbelabel-IST 2022 Submission for the Metrics Shared Task. In *Proceedings of the Seventh Conference on Machine Translation (WMT)*, pages 578–585, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.

Manon Revel and Theophile Penigaud. 2025. AI-facilitated collective judgements. *Available at SSRN 5167340*.

Paul Röttger, Musashi Hinck, Valentin Hofmann, Kobi Hackenburg, Valentina Pyatkin, Faeze Brahman, and Dirk Hovy. 2025. Issuebench: Millions of realistic prompts for measuring issue bias in LLM writing assistance. *arXiv preprint arXiv:2502.08395*.

Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. 2024. Political compass or spinning arrow? Towards more meaningful evaluations for values and opinions in large language models. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 15295–15311, Bangkok, Thailand. Association for Computational Linguistics.

David Rozado. 2023. *The Political Biases of ChatGPT*. *Social Sciences*, 12(3):148.

Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose Opinions Do Language Models Reflect? In *Proceedings of the 40th International Conference on Machine Learning*, pages 29971–30004. PMLR.

Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2022. Gender bias in machine translation. *Transactions of the Association for Computational Linguistics*, 9(0):845–874.

Jan Schwalbach. 2024. *Mind the context! The role of theoretical concepts for analyzing legislative text data*. *Research & Politics*, 11(3):20531680241277466.

Miklós Sebők, Sven-Oliver Proksch, Christian Rauh, Péter Visnovitz, Gergő Balázs, and Jan Schwalbach. 2025. Comparative european legislative research in the age of large-scale computational text analysis: A review article. *International Political Science Review*, 46(1):18–39.

Christopher T. Small, Ivan Vendrov, Esin Durmus, Hadjar Homaei, Elizabeth Barry, Julien Cornebise, Ted Suzman, Deep Ganguli, and Colin Megill. 2023. Opportunities and risks of llms for scalable deliberation with polis. *CoRR*, abs/2306.11932.

Michael Henry Tessler, Michiel A. Bakker, Daniel Jarrett, Hannah Sheahan, Martin J. Chadwick, Raphael Koster, Georgina Evans, Lucy Campbell-Gillingham, Tantum Collins, David C. Parkes, Matthew Botvinick, and Christopher Summerfield. 2024. AI can help humans find common ground in democratic deliberation. *Science*, 386(6719):eadq2852.

Jörg Tiedemann. 2012. *Parallel data, tools and interfaces in OPUS*. In *Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12)*, pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA).

The Top Websites Ranking. 2025. The top 50 most visited websites for september 2025. Similarweb. Accessed on October 13th 2025. URL archived from October 10th 2025.

Antonio Toral, Sheila Castilho, Ke Hu, and Andy Way. 2018. Attaining the unattainable? reassessing claims of human parity in neural machine translation. In *Proceedings of the Third Conference on Machine Translation: Research Papers*, pages 113–123, Brussels, Belgium. Association for Computational Linguistics.
Antoine Vendeville, Jimena Royo-Letelier, Duncan Cassells, Jean-Philippe Cointet, Maxime Crépel, Tim Faverjon, Théophile Lenoir, Béatrice Mrozoyer, Benjamin Ooghe-Tabanou, Armin Pournaki, Hiroki Yamashita, and Pedro Ramaciotti. 2025. Mapping the political landscape from data traces: multidimensional opinions of users, politicians and media outlets on X. Working paper or preprint.

Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In *Proceedings of the International Workshop on Software Fairness*, FairWare '18, pages 1–7, New York, NY, USA. Association for Computing Machinery.

Vered Volansky, Noam Ordan, and Shuly Wintner. 2013. On the features of translationese. *Digital Scholarship in the Humanities*, 30(1):98–118.

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. *arXiv preprint arXiv:2505.09388*.

Mike Zhang and Antonio Toral. 2019. The Effect of Translationese in Machine Translation Test Sets. In *Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)*, pages 73–81, Florence, Italy. Association for Computational Linguistics.

Vilém Zouhar, Pinzhen Chen, Tsz Kin Lam, Nikita Moghe, and Barry Haddow. 2024. Pitfalls and outlooks in using COMET. In *Proceedings of the Ninth Conference on Machine Translation*, pages 1272–1288, Miami, Florida, USA. Association for Computational Linguistics.

## 11. Language Resource References

Kwabena Amponsah-Kaakyire, Daria Pylypenko, Cristina España-Bonet, and Josef van Genabith. 2021. Do not Rely on Relay Translations: Multilingual Parallel Direct Europarl. In *Proceedings for the First Workshop on Modelling Translation: Translatology in the Digital Age*, pages 1–7, online. Association for Computational Linguistics.

Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin. 1990. A statistical approach to machine translation. *Computational Linguistics*, 16(2):79–85.

Johannes Graën, Dolores Batinić, and Martin Volk. 2014. Cleaning the Europarl corpus for linguistic applications.

Najeh Hajlaoui, David Kolovratnik, Jaakko Väyrynen, Ralf Steinberger, and Daniel Varga. 2014. DCEP -Digital Corpus of the European Parliament. In *Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)*, Reykjavik, Iceland. European Language Resources Association (ELRA).

Javier Iranzo-Sánchez, Joan Albert Silvestre-Cerdà, Javier Jorge, Nahuel Roselló, Adrià Giménez, Albert Sanchis, Jorge Civera, and Alfons Juan. 2020. Europarl-ST: A Multilingual Corpus for Speech Translation of Parliamentary Debates. In *ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pages 8229–8233.

Alina Karakanta, Mihaela Vela, and Elke Teich. 2018. Europarl-uds: Preserving and extending metadata in parliamentary debates. *ParlaCLARIN: Creating and Using Parliamentary Corpora*.

Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In *Proceedings of Machine Translation Summit X: Papers*, pages 79–86, Phuket, Thailand.

Natasha Noy, Astrid van Aggelen, Laura Hollink, Max Kemman, Martijn Kleppe, and Henri Beunders. 2017. The debates of the European Parliament as Linked Open Data. *Semantic Web*, 8(2):271–281.

# A. Results per Language Pair

## A.1. sBLEU

We report average sBLEU results per language pair (source languages in rows and target languages in columns) for each model in Tables 8, 9, 10 and 11. Because the results of Llama-3.2-1B (Table 9) and Llama-3.2-3B (Table 10) are quite poor, we excluded them from the rest of the analysis.

## A.2. COMET

We report COMET results per language pair (source languages in rows and target languages in columns) for each model in Tables 12, 13, and 14.
<table><caption>Table 8: sBLEU scores of Qwen3-4B for each language pair (source language in rows and target language in columns)</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>14.2</td><td>16.8</td><td>18.9</td><td>13.8</td><td>37.5</td><td>29.7</td><td>6.0</td><td>8.6</td><td>25.1</td><td>9.3</td><td>19.3</td><td>8.9</td><td>8.4</td><td>16.5</td><td>13.9</td><td>22.1</td><td>21.1</td><td>10.4</td><td>11.8</td><td>14.8</td></tr><tr><td>CS</td><td>17.1</td><td>-</td><td>16.0</td><td>18.6</td><td>13.0</td><td>34.0</td><td>26.7</td><td>5.8</td><td>7.8</td><td>22.1</td><td>9.3</td><td>17.7</td><td>8.1</td><td>7.7</td><td>15.5</td><td>13.8</td><td>19.6</td><td>19.3</td><td>19.7</td><td>11.1</td><td>14.6</td></tr><tr><td>DA</td><td>16.1</td><td>13.1</td><td>-</td><td>19.0</td><td>11.4</td><td>36.4</td><td>26.9</td><td>6.0</td><td>8.8</td><td>22.7</td><td>9.1</td><td>18.5</td><td>8.2</td><td>8.0</td><td>16.8</td><td>12.0</td><td>20.6</td><td>19.4</td><td>12.3</td><td>10.6</td><td>20.9</td></tr><tr><td>DE</td><td>16.5</td><td>13.9</td><td>17.7</td><td>-</td><td>11.4</td><td>37.1</td><td>28.7</td><td>6.1</td><td>8.7</td><td>23.9</td><td>9.4</td><td>18.8</td><td>8.0</td><td>7.8</td><td>18.7</td><td>12.6</td><td>21.2</td><td>20.0</td><td>4.5</td><td>11.3</td><td>16.6</td></tr><tr><td>EL</td><td>16.9</td><td>11.8</td><td>12.1</td><td>17.3</td><td>-</td><td>34.5</td><td>28.7</td><td>5.4</td><td>7.6</td><td>23.9</td><td>7.2</td><td>18.6</td><td>7.5</td><td>7.3</td><td>15.4</td><td>7.2</td><td>19.1</td><td>19.3</td><td>4.1</td><td>10.4</td><td>13.2</td></tr><tr><td>EN</td><td>22.3</td><td>17.4</td><td>23.9</td><td>25.7</td><td>18.6</td><td>-</td><td>41.6</td><td>7.2</td><td>11.1</td><td>33.2</td><td>11.2</td><td>25.6</td><td>10.0</td><td>9.9</td><td>21.1</td><td>16.7</td><td>31.4</td><td>30.0</td><td>16.0</td><td>13.5</td><td>21.9</td></tr><tr><td>ES</td><td>18.5</td><td>14.2</td><td>18.2</td><td>20.1</td><td>15.8</td><td>42.2</td><td>-</td><td>6.2</td><td>8.5</td><td>29.1</td><td>9.6</td><td>23.2</td><td>8.6</td><td>8.0</td><td>16.9</td><td>13.9</td><td>29.9</td><td>25.2</td><td>12.0</td><td>11.4</td><td>16.5</td></tr><tr><td>ET</td><td>13.1</td><td>9.6</td><td>11.8</td><td>12.8</td><td>10.1</td><td>24.4</td><td>18.2</td><td>-</td><td>6.5</td><td>16.0</td><td>7.8</td><td>12.0</td><td>7.3</td><td>6.3</td><td>11.3</td><td>10.1</td><td>14.2</td><td>13.7</td><td>7.3</td><td>6.7</td><td>11.0</td></tr><tr><td>FI</td><td>13.7</td><td>10.5</td><td>13.3</td><td>14.6</td><td>11.3</td><td>27.7</td><td>22.1</td><td>5.0</td><td>-</td><td>18.6</td><td>8.7</td><td>14.3</td><td>7.1</td><td>6.3</td><td>13.3</td><td>10.4</td><td>16.9</td><td>15.4</td><td>8.2</td><td>7.6</td><td>12.7</td></tr><tr><td>FR</td><td>17.0</td><td>13.2</td><td>16.9</td><td>19.0</td><td>12.7</td><td>37.1</td><td>32.6</td><td>6.0</td><td>7.9</td><td>-</td><td>9.3</td><td>23.1</td><td>8.1</td><td>7.7</td><td>17.0</td><td>12.9</td><td>26.0</td><td>23.0</td><td>4.5</td><td>10.6</td><td>15.6</td></tr><tr><td>HU</td><td>13.6</td><td>10.4</td><td>12.3</td><td>15.0</td><td>10.7</td><td>27.7</td><td>23.2</td><td>4.6</td><td>7.3</td><td>19.5</td><td>-</td><td>15.2</td><td>7.1</td><td>6.3</td><td>12.0</td><td>10.5</td><td>17.6</td><td>15.5</td><td>7.4</td><td>8.8</td><td>11.7</td></tr><tr><td>IT</td><td>15.8</td><td>12.4</td><td>15.6</td><td>17.1</td><td>12.7</td><td>34.2</td><td>30.3</td><td>5.7</td><td>7.2</td><td>26.3</td><td>8.7</td><td>-</td><td>7.7</td><td>7.3</td><td>16.4</td><td>12.0</td><td>23.9</td><td>21.1</td><td>4.7</td><td>10.0</td><td>14.1</td></tr><tr><td>LT</td><td>14.0</td><td>9.9</td><td>10.9</td><td>12.2</td><td>10.0</td><td>24.6</td><td>20.1</td><td>4.5</td><td>7.0</td><td>17.0</td><td>7.3</td><td>13.2</td><td>-</td><td>5.2</td><td>11.8</td><td>9.9</td><td>14.8</td><td>13.8</td><td>6.6</td><td>7.5</td><td>11.1</td></tr><tr><td>LV</td><td>14.8</td><td>10.8</td><td>12.9</td><td>14.4</td><td>11.2</td><td>27.9</td><td>22.3</td><td>4.7</td><td>7.6</td><td>19.1</td><td>8.2</td><td>14.1</td><td>8.3</td><td>-</td><td>12.7</td><td>10.7</td><td>16.9</td><td>15.7</td><td>8.0</td><td>7.0</td><td>11.7</td></tr><tr><td>NL</td><td>14.9</td><td>11.8</td><td>15.8</td><td>15.5</td><td>10.9</td><td>31.8</td><td>25.4</td><td>5.7</td><td>7.9</td><td>21.5</td><td>8.6</td><td>17.4</td><td>7.7</td><td>7.2</td><td>-</td><td>10.7</td><td>19.5</td><td>17.7</td><td>4.9</td><td>9.5</td><td>14.8</td></tr><tr><td>PL</td><td>17.0</td><td>6.5</td><td>15.3</td><td>16.7</td><td>13.0</td><td>32.0</td><td>26.7</td><td>5.6</td><td>8.3</td><td>22.6</td><td>9.1</td><td>17.9</td><td>8.1</td><td>6.0</td><td>15.4</td><td>-</td><td>20.5</td><td>19.1</td><td>13.1</td><td>10.7</td><td>14.4</td></tr><tr><td>PT</td><td>17.3</td><td>12.9</td><td>16.5</td><td>18.1</td><td>14.9</td><td>40.1</td><td>36.6</td><td>5.7</td><td>8.1</td><td>28.9</td><td>7.8</td><td>22.8</td><td>8.1</td><td>7.4</td><td>16.7</td><td>13.0</td><td>-</td><td>23.7</td><td>10.3</td><td>10.5</td><td>15.1</td></tr><tr><td>RO</td><td>18.2</td><td>13.9</td><td>17.1</td><td>19.0</td><td>16.1</td><td>41.4</td><td>33.9</td><td>6.0</td><td>8.5</td><td>28.2</td><td>8.9</td><td>21.7</td><td>8.5</td><td>7.6</td><td>16.6</td><td>13.7</td><td>25.6</td><td>-</td><td>8.9</td><td>11.4</td><td>15.7</td></tr><tr><td>SK</td><td>17.1</td><td>17.9</td><td>14.6</td><td>18.2</td><td>12.9</td><td>34.1</td><td>27.0</td><td>5.6</td><td>7.5</td><td>22.1</td><td>8.8</td><td>18.2</td><td>8.4</td><td>7.9</td><td>15.3</td><td>13.8</td><td>20.1</td><td>19.3</td><td>-</td><td>10.4</td><td>14.3</td></tr><tr><td>SL</td><td>16.6</td><td>14.7</td><td>13.4</td><td>16.6</td><td>12.2</td><td>32.2</td><td>23.2</td><td>4.8</td><td>8.1</td><td>17.7</td><td>8.8</td><td>16.4</td><td>7.9</td><td>7.5</td><td>14.4</td><td>12.9</td><td>18.9</td><td>18.4</td><td>10.3</td><td>-</td><td>13.3</td></tr><tr><td>SV</td><td>15.6</td><td>12.4</td><td>20.4</td><td>18.0</td><td>10.1</td><td>33.7</td><td>24.6</td><td>5.9</td><td>8.8</td><td>21.8</td><td>8.8</td><td>17.6</td><td>7.9</td><td>7.4</td><td>16.3</td><td>11.7</td><td>19.1</td><td>18.7</td><td>11.4</td><td>10.0</td><td>-</td></tr></tbody></table>
<table><caption>Table 9: sBLEU scores of Llama-3.2-1B for each language pair (source language in rows and target language in columns)</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>5.8</td><td>6.8</td><td>10.0</td><td>3.1</td><td>22.8</td><td>16.4</td><td>2.5</td><td>3.5</td><td>13.2</td><td>4.6</td><td>9.6</td><td>2.8</td><td>2.8</td><td>9.1</td><td>6.8</td><td>12.4</td><td>8.7</td><td>3.8</td><td>4.0</td><td>8.4</td></tr><tr><td>CS</td><td>2.0</td><td>-</td><td>5.0</td><td>11.4</td><td>1.9</td><td>24.3</td><td>16.1</td><td>2.1</td><td>3.5</td><td>13.1</td><td>4.2</td><td>8.5</td><td>2.6</td><td>2.5</td><td>8.0</td><td>6.0</td><td>11.0</td><td>6.9</td><td>5.6</td><td>3.8</td><td>7.4</td></tr><tr><td>DA</td><td>2.1</td><td>4.6</td><td>-</td><td>12.1</td><td>3.2</td><td>18.7</td><td>11.7</td><td>2.4</td><td>3.6</td><td>11.7</td><td>4.3</td><td>6.9</td><td>2.4</td><td>2.4</td><td>7.8</td><td>4.8</td><td>9.7</td><td>6.5</td><td>2.6</td><td>2.9</td><td>12.5</td></tr><tr><td>DE</td><td>2.2</td><td>4.7</td><td>5.4</td><td>-</td><td>4.3</td><td>26.5</td><td>17.1</td><td>2.2</td><td>3.9</td><td>14.2</td><td>4.9</td><td>9.2</td><td>2.5</td><td>2.1</td><td>12.7</td><td>5.6</td><td>13.8</td><td>7.4</td><td>2.5</td><td>3.1</td><td>9.9</td></tr><tr><td>EL</td><td>1.9</td><td>3.5</td><td>3.1</td><td>8.5</td><td>-</td><td>21.8</td><td>14.3</td><td>0.8</td><td>1.9</td><td>12.1</td><td>2.2</td><td>8.4</td><td>1.5</td><td>1.3</td><td>6.0</td><td>3.0</td><td>6.8</td><td>5.4</td><td>1.2</td><td>1.8</td><td>4.2</td></tr><tr><td>EN</td><td>3.4</td><td>6.8</td><td>9.8</td><td>16.4</td><td>7.0</td><td>-</td><td>24.9</td><td>2.5</td><td>5.1</td><td>22.9</td><td>6.4</td><td>16.7</td><td>2.6</td><td>2.1</td><td>12.2</td><td>8.5</td><td>19.6</td><td>15.6</td><td>1.7</td><td>4.1</td><td>15.3</td></tr><tr><td>ES</td><td>3.1</td><td>3.8</td><td>4.2</td><td>5.0</td><td>5.1</td><td>22.1</td><td>-</td><td>1.9</td><td>3.3</td><td>18.0</td><td>4.1</td><td>11.6</td><td>2.4</td><td>2.1</td><td>7.1</td><td>4.6</td><td>16.0</td><td>11.1</td><td>2.3</td><td>2.7</td><td>6.8</td></tr><tr><td>ET</td><td>1.8</td><td>2.4</td><td>2.1</td><td>3.5</td><td>1.7</td><td>6.2</td><td>3.9</td><td>-</td><td>2.7</td><td>4.2</td><td>2.7</td><td>2.1</td><td>2.2</td><td>2.3</td><td>3.5</td><td>2.2</td><td>3.4</td><td>2.3</td><td>2.1</td><td>2.2</td><td>2.4</td></tr><tr><td>FI</td><td>1.8</td><td>3.6</td><td>3.9</td><td>7.3</td><td>2.2</td><td>14.6</td><td>10.3</td><td>2.4</td><td>-</td><td>8.4</td><td>3.5</td><td>5.6</td><td>2.3</td><td>2.0</td><td>5.9</td><td>3.5</td><td>7.5</td><td>3.9</td><td>2.3</td><td>2.3</td><td>5.9</td></tr><tr><td>FR</td><td>3.1</td><td>3.7</td><td>3.6</td><td>8.0</td><td>4.9</td><td>17.9</td><td>18.7</td><td>2.0</td><td>3.3</td><td>-</td><td>4.1</td><td>14.2</td><td>2.4</td><td>2.2</td><td>8.2</td><td>5.3</td><td>14.7</td><td>11.8</td><td>2.5</td><td>2.7</td><td>7.0</td></tr><tr><td>HU</td><td>2.0</td><td>3.4</td><td>2.8</td><td>7.2</td><td>2.5</td><td>16.6</td><td>10.9</td><td>1.5</td><td>3.0</td><td>9.2</td><td>-</td><td>5.4</td><td>2.1</td><td>1.8</td><td>6.2</td><td>4.2</td><td>7.3</td><td>4.6</td><td>2.1</td><td>2.6</td><td>4.8</td></tr><tr><td>IT</td><td>2.5</td><td>4.0</td><td>4.0</td><td>7.8</td><td>4.2</td><td>22.5</td><td>19.7</td><td>1.7</td><td>2.4</td><td>17.7</td><td>4.1</td><td>-</td><td>2.3</td><td>2.0</td><td>7.5</td><td>5.1</td><td>15.0</td><td>9.4</td><td>2.1</td><td>2.6</td><td>5.7</td></tr><tr><td>LT</td><td>2.1</td><td>2.7</td><td>3.0</td><td>3.7</td><td>2.1</td><td>6.6</td><td>5.1</td><td>2.1</td><td>2.6</td><td>4.7</td><td>3.2</td><td>3.4</td><td>-</td><td>2.5</td><td>3.9</td><td>3.0</td><td>4.1</td><td>3.2</td><td>2.2</td><td>2.3</td><td>3.6</td></tr><tr><td>LV</td><td>2.0</td><td>2.6</td><td>2.4</td><td>3.7</td><td>1.9</td><td>7.7</td><td>4.2</td><td>2.1</td><td>2.4</td><td>4.8</td><td>3.0</td><td>2.9</td><td>2.3</td><td>-</td><td>3.5</td><td>2.5</td><td>3.9</td><td>3.0</td><td>2.1</td><td>2.2</td><td>3.3</td></tr><tr><td>NL</td><td>3.0</td><td>3.7</td><td>5.0</td><td>14.7</td><td>3.7</td><td>25.6</td><td>15.5</td><td>2.0</td><td>3.6</td><td>13.9</td><td>4.5</td><td>9.4</td><td>2.5</td><td>2.2</td><td>-</td><td>5.0</td><td>12.6</td><td>7.3</td><td>2.3</td><td>2.6</td><td>8.1</td></tr><tr><td>PL</td><td>2.4</td><td>3.7</td><td>4.9</td><td>10.6</td><td>2.7</td><td>23.7</td><td>17.1</td><td>2.2</td><td>3.7</td><td>14.5</td><td>4.5</td><td>10.5</td><td>2.4</td><td>2.3</td><td>8.6</td><td>-</td><td>12.6</td><td>8.1</td><td>3.2</td><td>3.2</td><td>7.9</td></tr><tr><td>PT</td><td>3.1</td><td>4.6</td><td>6.0</td><td>10.5</td><td>4.4</td><td>29.7</td><td>29.0</td><td>2.0</td><td>3.4</td><td>21.7</td><td>4.5</td><td>16.4</td><td>2.5</td><td>2.3</td><td>9.0</td><td>6.5</td><td>-</td><td>12.6</td><td>2.9</td><td>3.0</td><td>8.6</td></tr><tr><td>RO</td><td>1.9</td><td>3.6</td><td>2.7</td><td>7.4</td><td>2.4</td><td>26.4</td><td>19.2</td><td>1.1</td><td>3.2</td><td>17.9</td><td>3.6</td><td>11.5</td><td>2.0</td><td>1.7</td><td>6.6</td><td>5.3</td><td>11.8</td><td>-</td><td>1.9</td><td>2.3</td><td>5.8</td></tr><tr><td>SK</td><td>1.6</td><td>4.0</td><td>2.9</td><td>6.1</td><td>1.3</td><td>18.3</td><td>11.7</td><td>1.5</td><td>2.5</td><td>8.2</td><td>2.7</td><td>5.1</td><td>1.5</td><td>1.7</td><td>4.5</td><td>3.1</td><td>7.5</td><td>3.5</td><td>-</td><td>2.5</td><td>5.0</td></tr><tr><td>SL</td><td>2.0</td><td>3.0</td><td>2.9</td><td>5.9</td><td>2.4</td><td>12.4</td><td>7.5</td><td>1.3</td><td>2.7</td><td>6.9</td><td>3.2</td><td>5.5</td><td>2.1</td><td>2.1</td><td>4.9</td><td>3.7</td><td>5.8</td><td>3.4</td><td>2.5</td><td>-</td><td>5.3</td></tr><tr><td>SV</td><td>2.2</td><td>4.4</td><td>8.6</td><td>10.2</td><td>3.1</td><td>24.0</td><td>11.4</td><td>1.7</td><td>3.4</td><td>12.1</td><td>3.8</td><td>7.7</td><td>2.3</td><td>2.1</td><td>7.8</td><td>3.9</td><td>10.6</td><td>7.7</td><td>2.0</td><td>2.9</td><td>-</td></tr></tbody></table>

<table><caption>Table 10: sBLEU scores of Llama-3.2-3B for each language pair (source language in rows and target language in columns)</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>9.2</td><td>9.2</td><td>14.6</td><td>4.7</td><td>29.6</td><td>23.5</td><td>3.9</td><td>5.1</td><td>19.3</td><td>6.1</td><td>15.6</td><td>3.9</td><td>3.7</td><td>13.8</td><td>8.7</td><td>13.0</td><td>8.1</td><td>4.6</td><td>6.1</td><td>11.9</td></tr><tr><td>CS</td><td>9.2</td><td>-</td><td>11.1</td><td>17.0</td><td>5.3</td><td>32.1</td><td>24.5</td><td>3.7</td><td>6.2</td><td>19.9</td><td>6.3</td><td>16.5</td><td>3.9</td><td>4.0</td><td>15.4</td><td>10.6</td><td>17.7</td><td>14.3</td><td>9.3</td><td>6.8</td><td>14.7</td></tr><tr><td>DA</td><td>7.9</td><td>8.1</td><td>-</td><td>13.3</td><td>3.8</td><td>12.1</td><td>20.3</td><td>3.5</td><td>5.6</td><td>14.6</td><td>4.8</td><td>12.1</td><td>3.9</td><td>3.8</td><td>13.8</td><td>7.7</td><td>10.6</td><td>9.2</td><td>4.4</td><td>5.7</td><td>17.3</td></tr><tr><td>DE</td><td>8.8</td><td>9.2</td><td>12.9</td><td>-</td><td>5.5</td><td>30.1</td><td>23.4</td><td>4.1</td><td>6.2</td><td>18.9</td><td>6.3</td><td>12.9</td><td>4.0</td><td>3.7</td><td>17.4</td><td>7.4</td><td>16.0</td><td>12.0</td><td>4.8</td><td>6.6</td><td>15.1</td></tr><tr><td>EL</td><td>5.6</td><td>7.9</td><td>5.9</td><td>10.3</td><td>-</td><td>29.2</td><td>21.4</td><td>2.3</td><td>3.1</td><td>12.2</td><td>1.8</td><td>12.7</td><td>1.9</td><td>2.1</td><td>9.7</td><td>5.0</td><td>11.6</td><td>3.4</td><td>2.9</td><td>2.5</td><td>6.6</td></tr><tr><td>EN</td><td>14.6</td><td>15.9</td><td>20.5</td><td>25.8</td><td>14.4</td><td>-</td><td>41.5</td><td>5.5</td><td>9.7</td><td>32.7</td><td>12.4</td><td>25.9</td><td>5.7</td><td>6.1</td><td>21.9</td><td>16.4</td><td>31.2</td><td>28.1</td><td>9.7</td><td>10.1</td><td>24.1</td></tr><tr><td>ES</td><td>9.5</td><td>9.6</td><td>9.4</td><td>14.2</td><td>7.1</td><td>26.2</td><td>-</td><td>4.2</td><td>6.3</td><td>22.4</td><td>6.9</td><td>18.0</td><td>4.1</td><td>3.8</td><td>15.6</td><td>9.4</td><td>23.3</td><td>14.7</td><td>5.3</td><td>6.3</td><td>12.6</td></tr><tr><td>ET</td><td>4.2</td><td>4.9</td><td>5.3</td><td>6.9</td><td>2.9</td><td>13.6</td><td>9.9</td><td>-</td><td>4.1</td><td>8.1</td><td>3.1</td><td>5.4</td><td>3.0</td><td>2.3</td><td>7.4</td><td>5.5</td><td>5.9</td><td>4.7</td><td>2.8</td><td>3.4</td><td>6.4</td></tr><tr><td>FI</td><td>6.1</td><td>5.7</td><td>7.8</td><td>8.9</td><td>3.6</td><td>21.7</td><td>15.5</td><td>2.6</td><td>-</td><td>12.8</td><td>4.1</td><td>7.3</td><td>2.7</td><td>2.1</td><td>10.6</td><td>5.7</td><td>9.8</td><td>6.4</td><td>3.5</td><td>3.6</td><td>11.0</td></tr><tr><td>FR</td><td>8.8</td><td>8.4</td><td>9.9</td><td>16.0</td><td>6.5</td><td>23.3</td><td>27.9</td><td>3.7</td><td>5.6</td><td>-</td><td>6.2</td><td>19.2</td><td>4.0</td><td>3.6</td><td>15.2</td><td>7.9</td><td>20.5</td><td>14.5</td><td>5.0</td><td>6.2</td><td>13.5</td></tr><tr><td>HU</td><td>5.9</td><td>6.3</td><td>5.8</td><td>11.8</td><td>2.8</td><td>20.0</td><td>18.3</td><td>3.0</td><td>4.6</td><td>14.9</td><td>-</td><td>10.2</td><td>3.1</td><td>3.2</td><td>11.8</td><td>6.7</td><td>12.5</td><td>8.4</td><td>3.0</td><td>4.5</td><td>9.6</td></tr><tr><td>IT</td><td>8.5</td><td>8.9</td><td>8.4</td><td>12.6</td><td>6.2</td><td>21.4</td><td>26.1</td><td>3.8</td><td>4.6</td><td>21.3</td><td>5.6</td><td>-</td><td>4.0</td><td>3.7</td><td>15.2</td><td>8.8</td><td>20.4</td><td>12.8</td><td>5.2</td><td>6.0</td><td>12.1</td></tr><tr><td>LT</td><td>5.4</td><td>5.0</td><td>4.3</td><td>7.5</td><td>3.7</td><td>13.7</td><td>12.8</td><td>2.8</td><td>4.1</td><td>9.5</td><td>3.2</td><td>6.6</td><td>-</td><td>2.5</td><td>8.4</td><td>5.6</td><td>7.5</td><td>5.4</td><td>2.7</td><td>3.2</td><td>6.6</td></tr><tr><td>LV</td><td>6.1</td><td>6.1</td><td>6.9</td><td>9.5</td><td>4.1</td><td>18.4</td><td>14.9</td><td>3.0</td><td>4.9</td><td>12.4</td><td>4.6</td><td>8.3</td><td>2.5</td><td>-</td><td>9.5</td><td>6.5</td><td>8.6</td><td>5.5</td><td>2.7</td><td>3.6</td><td>8.9</td></tr><tr><td>NL</td><td>8.2</td><td>6.8</td><td>8.7</td><td>15.5</td><td>5.3</td><td>21.3</td><td>20.8</td><td>3.7</td><td>5.4</td><td>17.9</td><td>6.1</td><td>11.5</td><td>3.5</td><td>3.7</td><td>-</td><td>7.6</td><td>13.9</td><td>10.0</td><td>5.0</td><td>5.3</td><td>12.5</td></tr><tr><td>PL</td><td>9.0</td><td>8.1</td><td>9.3</td><td>13.3</td><td>5.8</td><td>24.9</td><td>23.0</td><td>3.7</td><td>5.5</td><td>19.2</td><td>5.0</td><td>14.2</td><td>3.8</td><td>3.7</td><td>13.5</td><td>-</td><td>15.9</td><td>12.4</td><td>4.4</td><td>5.5</td><td>12.6</td></tr><tr><td>PT</td><td>8.9</td><td>8.8</td><td>8.7</td><td>13.9</td><td>6.2</td><td>25.8</td><td>31.0</td><td>3.7</td><td>5.5</td><td>22.1</td><td>4.9</td><td>17.7</td><td>3.8</td><td>3.5</td><td>14.8</td><td>8.9</td><td>-</td><td>11.2</td><td>4.3</td><td>5.2</td><td>10.3</td></tr><tr><td>RO</td><td>10.0</td><td>9.2</td><td>8.8</td><td>16.8</td><td>6.9</td><td>29.4</td><td>30.8</td><td>3.5</td><td>5.6</td><td>25.0</td><td>6.7</td><td>16.9</td><td>3.8</td><td>3.6</td><td>15.6</td><td>10.3</td><td>19.0</td><td>-</td><td>4.5</td><td>4.5</td><td>13.4</td></tr><tr><td>SK</td><td>8.9</td><td>9.8</td><td>9.0</td><td>13.8</td><td>5.0</td><td>28.1</td><td>22.4</td><td>3.6</td><td>5.3</td><td>17.7</td><td>5.0</td><td>13.0</td><td>3.1</td><td>3.4</td><td>13.9</td><td>8.8</td><td>14.9</td><td>11.7</td><td>-</td><td>4.7</td><td>12.4</td></tr><tr><td>SL</td><td>8.0</td><td>7.2</td><td>6.0</td><td>11.7</td><td>4.1</td><td>22.7</td><td>18.1</td><td>3.5</td><td>3.6</td><td>14.8</td><td>4.3</td><td>10.4</td><td>3.3</td><td>2.7</td><td>11.9</td><td>7.7</td><td>10.6</td><td>6.9</td><td>3.1</td><td>-</td><td>9.6</td></tr><tr><td>SV</td><td>7.4</td><td>8.4</td><td>8.2</td><td>10.4</td><td>3.0</td><td>19.5</td><td>18.5</td><td>3.6</td><td>5.9</td><td>14.6</td><td>4.3</td><td>9.1</td><td>3.7</td><td>3.2</td><td>13.8</td><td>6.2</td><td>11.0</td><td>7.6</td><td>3.3</td><td>5.1</td><td>-</td></tr></tbody></table>

<table><caption>Table 11: sBLEU scores of Llama-3.1-8B for each language pair (source language in rows and target language in columns)</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>19.6</td><td>20.4</td><td>21.3</td><td>19.1</td><td>39.3</td><td>32.2</td><td>8.5</td><td>10.5</td><td>27.4</td><td>13.5</td><td>22.1</td><td>10.0</td><td>10.4</td><td>20.9</td><td>18.4</td><td>25.2</td><td>24.8</td><td>15.7</td><td>16.6</td><td>21.3</td></tr><tr><td>CS</td><td>19.5</td><td>-</td><td>19.4</td><td>20.9</td><td>17.6</td><td>35.7</td><td>29.8</td><td>8.2</td><td>10.0</td><td>25.2</td><td>12.5</td><td>20.7</td><td>9.0</td><td>9.2</td><td>20.0</td><td>16.5</td><td>23.0</td><td>21.5</td><td>22.3</td><td>15.0</td><td>20.1</td></tr><tr><td>DA</td><td>17.2</td><td>17.5</td><td>-</td><td>21.5</td><td>16.6</td><td>38.3</td><td>28.3</td><td>7.8</td><td>10.5</td><td>25.3</td><td>12.3</td><td>20.0</td><td>8.3</td><td>9.0</td><td>20.2</td><td>15.3</td><td>21.0</td><td>19.8</td><td>12.7</td><td>12.9</td><td>26.2</td></tr><tr><td>DE</td><td>19.0</td><td>18.5</td><td>20.7</td><td>-</td><td>18.2</td><td>38.6</td><td>31.3</td><td>8.5</td><td>10.9</td><td>26.4</td><td>13.2</td><td>21.0</td><td>9.3</td><td>9.1</td><td>23.2</td><td>16.4</td><td>24.2</td><td>23.2</td><td>14.7</td><td>15.1</td><td>22.3</td></tr><tr><td>EL</td><td>19.7</td><td>18.4</td><td>20.3</td><td>21.4</td><td>-</td><td>40.5</td><td>33.4</td><td>8.2</td><td>10.6</td><td>27.8</td><td>12.6</td><td>22.3</td><td>9.2</td><td>9.4</td><td>20.9</td><td>17.2</td><td>26.9</td><td>25.6</td><td>14.1</td><td>14.9</td><td>21.2</td></tr><tr><td>EN</td><td>26.2</td><td>24.1</td><td>28.2</td><td>29.0</td><td>27.2</td><td>-</td><td>45.3</td><td>10.6</td><td>13.7</td><td>35.9</td><td>16.2</td><td>29.6</td><td>11.7</td><td>12.5</td><td>26.1</td><td>21.9</td><td>35.2</td><td>35.6</td><td>19.0</td><td>19.9</td><td>29.7</td></tr><tr><td>ES</td><td>20.5</td><td>18.8</td><td>21.2</td><td>22.7</td><td>22.0</td><td>44.3</td><td>-</td><td>8.7</td><td>11.2</td><td>30.8</td><td>13.4</td><td>25.8</td><td>9.7</td><td>9.8</td><td>21.8</td><td>17.8</td><td>31.9</td><td>27.7</td><td>14.9</td><td>15.4</td><td>22.0</td></tr><tr><td>ET</td><td>15.0</td><td>13.8</td><td>15.0</td><td>16.1</td><td>14.0</td><td>27.2</td><td>23.6</td><td>-</td><td>9.8</td><td>20.0</td><td>11.4</td><td>15.6</td><td>7.1</td><td>6.9</td><td>15.7</td><td>13.1</td><td>18.4</td><td>15.5</td><td>10.4</td><td>10.2</td><td>16.2</td></tr><tr><td>FI</td><td>15.1</td><td>14.0</td><td>16.2</td><td>16.6</td><td>14.6</td><td>29.1</td><td>24.4</td><td>7.4</td><td>-</td><td>21.1</td><td>11.6</td><td>16.6</td><td>6.5</td><td>6.0</td><td>16.6</td><td>12.9</td><td>19.1</td><td>16.3</td><td>10.6</td><td>10.0</td><td>18.0</td></tr><tr><td>FR</td><td>19.2</td><td>17.6</td><td>19.7</td><td>21.5</td><td>19.8</td><td>38.8</td><td>35.0</td><td>7.9</td><td>10.4</td><td>-</td><td>12.8</td><td>25.7</td><td>9.3</td><td>9.6</td><td>21.3</td><td>17.0</td><td>28.5</td><td>25.9</td><td>13.8</td><td>14.8</td><td>21.2</td></tr><tr><td>HU</td><td>15.5</td><td>14.5</td><td>15.5</td><td>17.6</td><td>14.9</td><td>30.4</td><td>26.3</td><td>6.4</td><td>8.9</td><td>22.2</td><td>-</td><td>17.8</td><td>6.9</td><td>8.0</td><td>17.1</td><td>13.6</td><td>20.9</td><td>17.1</td><td>10.1</td><td>11.1</td><td>17.2</td></tr><tr><td>IT</td><td>17.5</td><td>16.9</td><td>18.4</td><td>19.8</td><td>18.2</td><td>35.8</td><td>32.3</td><td>7.7</td><td>9.9</td><td>28.5</td><td>11.9</td><td>-</td><td>8.7</td><td>8.9</td><td>20.4</td><td>16.1</td><td>26.5</td><td>22.7</td><td>13.0</td><td>13.4</td><td>19.7</td></tr><tr><td>LT</td><td>15.5</td><td>13.3</td><td>14.0</td><td>15.0</td><td>13.3</td><td>26.1</td><td>22.1</td><td>6.5</td><td>8.8</td><td>19.3</td><td>10.6</td><td>15.0</td><td>-</td><td>8.2</td><td>14.5</td><td>13.0</td><td>17.1</td><td>16.0</td><td>9.7</td><td>10.3</td><td>15.3</td></tr><tr><td>LV</td><td>16.2</td><td>14.3</td><td>15.7</td><td>16.2</td><td>14.1</td><td>28.6</td><td>23.5</td><td>6.9</td><td>9.5</td><td>20.5</td><td>11.5</td><td>16.1</td><td>7.5</td><td>-</td><td>15.9</td><td>13.5</td><td>18.5</td><td>17.2</td><td>10.7</td><td>11.1</td><td>16.4</td></tr><tr><td>NL</td><td>16.7</td><td>16.0</td><td>18.4</td><td>21.5</td><td>15.9</td><td>33.3</td><td>27.6</td><td>7.2</td><td>9.7</td><td>24.3</td><td>11.8</td><td>19.4</td><td>8.2</td><td>8.3</td><td>-</td><td>14.6</td><td>21.4</td><td>18.6</td><td>11.9</td><td>11.9</td><td>19.7</td></tr><tr><td>PL</td><td>18.7</td><td>17.7</td><td>17.9</td><td>18.7</td><td>16.7</td><td>33.5</td><td>28.7</td><td>7.6</td><td>9.7</td><td>25.1</td><td>11.9</td><td>19.9</td><td>8.5</td><td>8.9</td><td>18.9</td><td>-</td><td>22.7</td><td>20.8</td><td>13.9</td><td>14.0</td><td>19.1</td></tr><tr><td>PT</td><td>20.1</td><td>18.1</td><td>20.2</td><td>21.9</td><td>21.0</td><td>42.5</td><td>38.0</td><td>8.1</td><td>10.5</td><td>30.8</td><td>12.7</td><td>25.8</td><td>9.3</td><td>9.2</td><td>21.2</td><td>17.4</td><td>-</td><td>25.6</td><td>13.7</td><td>14.4</td><td>21.6</td></tr><tr><td>RO</td><td>20.7</td><td>18.1</td><td>20.2</td><td>22.4</td><td>22.2</td><td>43.2</td><td>36.3</td><td>7.3</td><td>10.1</td><td>30.6</td><td>12.9</td><td>24.3</td><td>8.5</td><td>8.9</td><td>20.8</td><td>17.0</td><td>28.1</td><td>-</td><td>13.3</td><td>13.7</td><td>21.6</td></tr><tr><td>SK</td><td>19.7</td><td>28.5</td><td>19.3</td><td>21.1</td><td>18.1</td><td>35.9</td><td>30.1</td><td>8.3</td><td>10.2</td><td>25.1</td><td>12.8</td><td>20.6</td><td>9.0</td><td>9.3</td><td>19.9</td><td>17.7</td><td>23.0</td><td>22.4</td><td>-</td><td>15.3</td><td>20.2</td></tr><tr><td>SL</td><td>19.2</td><td>19.6</td><td>18.5</td><td>20.2</td><td>17.3</td><td>34.5</td><td>29.0</td><td>7.0</td><td>9.9</td><td>24.4</td><td>12.7</td><td>19.2</td><td>8.1</td><td>7.9</td><td>19.3</td><td>16.3</td><td>22.1</td><td>20.7</td><td>15.5</td><td>-</td><td>19.3</td></tr><tr><td>SV</td><td>16.8</td><td>16.3</td><td>24.1</td><td>20.6</td><td>16.3</td><td>36.8</td><td>27.1</td><td>7.9</td><td>10.6</td><td>24.7</td><td>11.9</td><td>19.8</td><td>8.4</td><td>8.7</td><td>19.7</td><td>15.2</td><td>20.4</td><td>20.9</td><td>12.4</td><td>12.1</td><td>-</td></tr></tbody></table>
<table><caption>Table 12: COMET scores of Qwen3-4B for each language pair (source language in rows and target language in columns)</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>83.7</td><td>81.5</td><td>83.1</td><td>75.6</td><td>88.2</td><td>86.4</td><td>59.3</td><td>75.7</td><td>85.2</td><td>76.5</td><td>86.4</td><td>70.9</td><td>64.7</td><td>81.5</td><td>84.4</td><td>84.4</td><td>85.8</td><td>73.3</td><td>74.7</td><td>80.6</td></tr><tr><td>CS</td><td>81.0</td><td>-</td><td>79.8</td><td>84.4</td><td>75.3</td><td>88.0</td><td>85.3</td><td>56.9</td><td>72.1</td><td>82.4</td><td>75.8</td><td>85.0</td><td>67.4</td><td>61.1</td><td>81.0</td><td>84.4</td><td>83.0</td><td>84.6</td><td>87.4</td><td>74.3</td><td>79.9</td></tr><tr><td>DA</td><td>79.9</td><td>81.5</td><td>-</td><td>82.6</td><td>71.1</td><td>88.3</td><td>83.3</td><td>57.7</td><td>74.6</td><td>81.7</td><td>75.9</td><td>84.9</td><td>68.5</td><td>61.9</td><td>81.1</td><td>81.4</td><td>83.4</td><td>83.2</td><td>75.6</td><td>70.2</td><td>84.4</td></tr><tr><td>DE</td><td>79.8</td><td>82.3</td><td>80.5</td><td>-</td><td>70.8</td><td>88.7</td><td>84.5</td><td>57.6</td><td>73.8</td><td>83.0</td><td>75.4</td><td>85.0</td><td>68.8</td><td>61.5</td><td>82.3</td><td>82.3</td><td>82.4</td><td>82.9</td><td>64.6</td><td>70.3</td><td>80.9</td></tr><tr><td>EL</td><td>79.7</td><td>78.1</td><td>71.6</td><td>81.3</td><td>-</td><td>86.4</td><td>84.6</td><td>57.9</td><td>73.3</td><td>84.4</td><td>71.4</td><td>86.0</td><td>69.5</td><td>62.0</td><td>79.1</td><td>68.1</td><td>79.5</td><td>82.6</td><td>63.8</td><td>70.1</td><td>77.6</td></tr><tr><td>EN</td><td>84.0</td><td>85.7</td><td>84.8</td><td>86.6</td><td>81.0</td><td>-</td><td>89.7</td><td>64.0</td><td>81.0</td><td>88.8</td><td>81.4</td><td>89.4</td><td>74.1</td><td>66.9</td><td>85.5</td><td>85.8</td><td>88.7</td><td>89.8</td><td>80.1</td><td>75.3</td><td>84.7</td></tr><tr><td>ES</td><td>82.3</td><td>82.7</td><td>80.8</td><td>82.2</td><td>78.1</td><td>89.5</td><td>-</td><td>59.6</td><td>74.0</td><td>86.9</td><td>77.5</td><td>88.3</td><td>70.3</td><td>61.6</td><td>81.1</td><td>84.4</td><td>88.3</td><td>87.3</td><td>75.0</td><td>71.8</td><td>79.8</td></tr><tr><td>ET</td><td>77.8</td><td>76.8</td><td>73.7</td><td>77.5</td><td>73.2</td><td>86.3</td><td>77.8</td><td>-</td><td>69.9</td><td>78.7</td><td>71.3</td><td>79.5</td><td>65.3</td><td>60.7</td><td>74.7</td><td>80.8</td><td>78.9</td><td>80.0</td><td>67.4</td><td>63.8</td><td>73.3</td></tr><tr><td>FI</td><td>80.8</td><td>81.1</td><td>78.6</td><td>81.7</td><td>77.3</td><td>89.1</td><td>85.0</td><td>64.9</td><td>-</td><td>83.9</td><td>75.0</td><td>84.4</td><td>66.6</td><td>59.9</td><td>80.0</td><td>83.1</td><td>84.6</td><td>83.7</td><td>73.0</td><td>66.4</td><td>78.8</td></tr><tr><td>FR</td><td>80.9</td><td>81.6</td><td>79.6</td><td>81.8</td><td>72.3</td><td>88.8</td><td>86.6</td><td>59.3</td><td>73.8</td><td>-</td><td>75.7</td><td>87.8</td><td>68.0</td><td>61.4</td><td>81.0</td><td>82.7</td><td>86.0</td><td>85.2</td><td>64.5</td><td>70.5</td><td>79.9</td></tr><tr><td>HU</td><td>79.6</td><td>79.0</td><td>75.1</td><td>82.2</td><td>74.7</td><td>87.8</td><td>84.6</td><td>50.8</td><td>69.7</td><td>83.8</td><td>-</td><td>85.2</td><td>67.6</td><td>59.4</td><td>76.9</td><td>81.8</td><td>83.4</td><td>82.8</td><td>69.3</td><td>67.0</td><td>76.9</td></tr><tr><td>IT</td><td>80.4</td><td>81.2</td><td>78.7</td><td>80.6</td><td>73.8</td><td>88.0</td><td>86.3</td><td>57.6</td><td>69.8</td><td>84.8</td><td>74.6</td><td>-</td><td>68.8</td><td>60.9</td><td>80.0</td><td>81.4</td><td>84.8</td><td>84.8</td><td>64.6</td><td>70.2</td><td>76.5</td></tr><tr><td>LT</td><td>79.0</td><td>76.2</td><td>70.8</td><td>77.0</td><td>73.8</td><td>84.9</td><td>81.5</td><td>57.6</td><td>68.5</td><td>79.4</td><td>70.1</td><td>81.8</td><td>-</td><td>68.5</td><td>75.8</td><td>80.1</td><td>79.4</td><td>79.9</td><td>64.1</td><td>65.3</td><td>74.8</td></tr><tr><td>LV</td><td>80.1</td><td>78.4</td><td>74.4</td><td>79.9</td><td>74.8</td><td>86.7</td><td>82.5</td><td>59.6</td><td>70.4</td><td>82.0</td><td>73.2</td><td>82.7</td><td>69.0</td><td>-</td><td>76.8</td><td>80.8</td><td>81.8</td><td>82.0</td><td>66.5</td><td>65.4</td><td>74.4</td></tr><tr><td>NL</td><td>78.5</td><td>80.3</td><td>78.1</td><td>76.3</td><td>71.0</td><td>87.8</td><td>83.4</td><td>57.1</td><td>72.8</td><td>81.6</td><td>73.8</td><td>84.2</td><td>66.7</td><td>60.9</td><td>-</td><td>78.6</td><td>82.3</td><td>82.9</td><td>62.9</td><td>66.9</td><td>77.8</td></tr><tr><td>PL</td><td>82.3</td><td>85.4</td><td>79.3</td><td>83.4</td><td>76.9</td><td>87.3</td><td>86.1</td><td>58.1</td><td>74.0</td><td>84.9</td><td>76.4</td><td>87.0</td><td>71.1</td><td>64.7</td><td>80.8</td><td>-</td><td>85.4</td><td>86.0</td><td>80.3</td><td>74.8</td><td>80.2</td></tr><tr><td>PT</td><td>81.1</td><td>80.4</td><td>78.8</td><td>80.5</td><td>77.4</td><td>88.7</td><td>88.7</td><td>58.0</td><td>73.5</td><td>86.7</td><td>72.1</td><td>87.8</td><td>68.6</td><td>61.1</td><td>80.6</td><td>82.3</td><td>-</td><td>86.3</td><td>70.7</td><td>70.2</td><td>79.1</td></tr><tr><td>RO</td><td>81.0</td><td>81.7</td><td>79.2</td><td>81.0</td><td>77.5</td><td>89.6</td><td>86.5</td><td>57.0</td><td>72.9</td><td>85.6</td><td>74.4</td><td>86.7</td><td>67.6</td><td>60.3</td><td>80.8</td><td>83.4</td><td>85.2</td><td>-</td><td>70.8</td><td>71.2</td><td>79.2</td></tr><tr><td>SK</td><td>80.3</td><td>89.5</td><td>75.6</td><td>83.5</td><td>75.4</td><td>87.7</td><td>85.0</td><td>56.5</td><td>70.4</td><td>82.2</td><td>73.9</td><td>85.5</td><td>67.0</td><td>61.7</td><td>80.6</td><td>84.7</td><td>83.4</td><td>84.0</td><td>-</td><td>76.3</td><td>79.5</td></tr><tr><td>SL</td><td>79.8</td><td>81.6</td><td>72.5</td><td>80.2</td><td>73.8</td><td>86.9</td><td>80.1</td><td>55.8</td><td>70.1</td><td>74.5</td><td>73.6</td><td>83.2</td><td>66.3</td><td>60.6</td><td>77.4</td><td>82.1</td><td>81.9</td><td>82.7</td><td>73.2</td><td>-</td><td>76.6</td></tr><tr><td>SV</td><td>80.5</td><td>82.0</td><td>83.2</td><td>82.2</td><td>71.4</td><td>87.7</td><td>82.1</td><td>58.2</td><td>74.7</td><td>81.4</td><td>75.4</td><td>85.1</td><td>68.3</td><td>61.8</td><td>81.0</td><td>81.4</td><td>83.0</td><td>83.2</td><td>75.7</td><td>69.9</td><td>-</td></tr></tbody></table>

<table><caption>Table 13: COMET scores of Qwen3-8B for each language pair (source language in rows and target language in columns)</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>89.6</td><td>86.7</td><td>86.3</td><td>85.8</td><td>89.3</td><td>88.0</td><td>75.1</td><td>85.1</td><td>87.2</td><td>85.4</td><td>88.7</td><td>83.7</td><td>82.2</td><td>86.3</td><td>89.9</td><td>87.3</td><td>90.3</td><td>86.5</td><td>84.3</td><td>87.2</td></tr><tr><td>CS</td><td>87.5</td><td>-</td><td>85.8</td><td>87.6</td><td>85.4</td><td>88.9</td><td>87.4</td><td>74.4</td><td>83.6</td><td>86.3</td><td>84.2</td><td>88.1</td><td>81.4</td><td>79.2</td><td>86.8</td><td>89.9</td><td>86.4</td><td>89.2</td><td>91.9</td><td>82.8</td><td>86.7</td></tr><tr><td>DA</td><td>86.9</td><td>88.4</td><td>-</td><td>86.9</td><td>85.6</td><td>89.8</td><td>87.4</td><td>74.8</td><td>84.3</td><td>86.5</td><td>85.3</td><td>87.5</td><td>82.6</td><td>80.1</td><td>85.8</td><td>88.8</td><td>86.5</td><td>87.8</td><td>84.6</td><td>81.3</td><td>89.7</td></tr><tr><td>DE</td><td>87.1</td><td>89.8</td><td>86.5</td><td>-</td><td>85.2</td><td>89.4</td><td>87.2</td><td>75.7</td><td>84.6</td><td>86.9</td><td>85.4</td><td>88.5</td><td>82.7</td><td>79.9</td><td>87.5</td><td>89.8</td><td>86.5</td><td>89.5</td><td>86.4</td><td>82.8</td><td>87.5</td></tr><tr><td>EL</td><td>86.3</td><td>82.4</td><td>84.4</td><td>81.9</td><td>-</td><td>84.9</td><td>85.5</td><td>64.2</td><td>82.0</td><td>85.6</td><td>82.4</td><td>86.4</td><td>77.1</td><td>75.7</td><td>84.2</td><td>86.0</td><td>80.6</td><td>84.4</td><td>82.0</td><td>79.4</td><td>83.8</td></tr><tr><td>EN</td><td>89.4</td><td>90.4</td><td>88.6</td><td>88.7</td><td>88.3</td><td>-</td><td>90.5</td><td>79.4</td><td>88.3</td><td>89.9</td><td>87.5</td><td>90.7</td><td>84.6</td><td>83.4</td><td>88.3</td><td>90.3</td><td>89.8</td><td>92.2</td><td>87.1</td><td>84.3</td><td>89.0</td></tr><tr><td>ES</td><td>88.3</td><td>89.2</td><td>86.0</td><td>86.0</td><td>86.6</td><td>90.0</td><td>-</td><td>76.3</td><td>85.1</td><td>88.7</td><td>85.2</td><td>89.9</td><td>82.7</td><td>79.9</td><td>85.9</td><td>89.7</td><td>89.7</td><td>90.6</td><td>86.1</td><td>82.9</td><td>86.6</td></tr><tr><td>ET</td><td>85.3</td><td>86.9</td><td>83.2</td><td>84.6</td><td>83.8</td><td>88.6</td><td>85.3</td><td>-</td><td>80.6</td><td>85.5</td><td>82.2</td><td>85.5</td><td>78.1</td><td>74.6</td><td>84.3</td><td>88.2</td><td>85.6</td><td>86.9</td><td>81.8</td><td>77.4</td><td>84.1</td></tr><tr><td>FI</td><td>86.6</td><td>88.1</td><td>85.4</td><td>86.2</td><td>85.8</td><td>90.2</td><td>87.8</td><td>72.6</td><td>-</td><td>87.5</td><td>83.5</td><td>86.9</td><td>76.6</td><td>75.2</td><td>86.0</td><td>89.0</td><td>87.4</td><td>88.1</td><td>83.3</td><td>78.9</td><td>86.9</td></tr><tr><td>FR</td><td>87.6</td><td>88.5</td><td>85.3</td><td>86.2</td><td>85.9</td><td>89.6</td><td>88.9</td><td>74.2</td><td>84.1</td><td>-</td><td>85.0</td><td>90.0</td><td>81.9</td><td>80.8</td><td>86.3</td><td>89.2</td><td>88.4</td><td>90.3</td><td>85.3</td><td>81.8</td><td>85.9</td></tr><tr><td>HU</td><td>86.6</td><td>86.6</td><td>84.4</td><td>85.8</td><td>84.8</td><td>89.0</td><td>86.9</td><td>68.8</td><td>80.9</td><td>86.3</td><td>-</td><td>87.5</td><td>78.6</td><td>77.2</td><td>83.6</td><td>88.6</td><td>85.2</td><td>87.8</td><td>81.9</td><td>78.6</td><td>85.0</td></tr><tr><td>IT</td><td>87.1</td><td>88.0</td><td>85.1</td><td>83.6</td><td>85.8</td><td>88.8</td><td>88.6</td><td>73.7</td><td>82.8</td><td>87.8</td><td>84.3</td><td>-</td><td>81.4</td><td>79.3</td><td>85.2</td><td>89.1</td><td>88.0</td><td>87.9</td><td>84.8</td><td>81.1</td><td>85.5</td></tr><tr><td>LT</td><td>86.1</td><td>85.0</td><td>82.4</td><td>83.0</td><td>84.1</td><td>87.2</td><td>85.2</td><td>68.4</td><td>80.6</td><td>84.1</td><td>80.3</td><td>85.8</td><td>-</td><td>76.7</td><td>83.2</td><td>87.5</td><td>84.2</td><td>85.4</td><td>79.3</td><td>75.0</td><td>83.4</td></tr><tr><td>LV</td><td>87.0</td><td>86.8</td><td>83.8</td><td>84.5</td><td>84.2</td><td>88.3</td><td>85.9</td><td>71.8</td><td>81.9</td><td>85.5</td><td>83.2</td><td>87.0</td><td>79.8</td><td>-</td><td>84.7</td><td>88.1</td><td>85.1</td><td>87.4</td><td>87.4</td><td>80.1</td><td>84.3</td></tr><tr><td>NL</td><td>86.2</td><td>87.5</td><td>84.2</td><td>86.9</td><td>84.0</td><td>88.7</td><td>85.9</td><td>73.2</td><td>82.8</td><td>85.9</td><td>83.3</td><td>87.1</td><td>81.2</td><td>79.1</td><td>-</td><td>88.2</td><td>85.2</td><td>87.1</td><td>82.8</td><td>79.2</td><td>85.8</td></tr><tr><td>PL</td><td>87.6</td><td>89.1</td><td>85.2</td><td>86.5</td><td>85.0</td><td>86.8</td><td>87.3</td><td>74.7</td><td>83.3</td><td>86.6</td><td>83.8</td><td>88.1</td><td>80.7</td><td>79.3</td><td>85.8</td><td>-</td><td>86.4</td><td>89.1</td><td>86.8</td><td>82.9</td><td>86.3</td></tr><tr><td>PT</td><td>88.1</td><td>87.7</td><td>85.7</td><td>85.6</td><td>86.2</td><td>89.8</td><td>90.2</td><td>75.6</td><td>84.1</td><td>88.8</td><td>84.8</td><td>90.2</td><td>81.9</td><td>79.3</td><td>85.5</td><td>89.4</td><td>-</td><td>90.3</td><td>84.8</td><td>81.9</td><td>86.4</td></tr><tr><td>RO</td><td>87.7</td><td>88.8</td><td>85.6</td><td>85.1</td><td>86.7</td><td>90.6</td><td>88.9</td><td>71.3</td><td>83.3</td><td>88.4</td><td>84.2</td><td>89.0</td><td>80.0</td><td>78.3</td><td>86.1</td><td>89.5</td><td>88.0</td><td>-</td><td>85.5</td><td>80.8</td><td>86.6</td></tr><tr><td>SK</td><td>87.9</td><td>93.4</td><td>85.8</td><td>87.7</td><td>85.5</td><td>89.1</td><td>87.5</td><td>72.3</td><td>82.8</td><td>86.2</td><td>84.1</td><td>88.2</td><td>79.9</td><td>78.6</td><td>86.6</td><td>90.0</td><td>86.4</td><td>89.1</td><td>-</td><td>82.2</td><td>86.6</td></tr><tr><td>SL</td><td>87.1</td><td>89.3</td><td>85.2</td><td>86.6</td><td>85.3</td><td>88.9</td><td>87.2</td><td>69.4</td><td>82.5</td><td>85.9</td><td>83.8</td><td>88.1</td><td>79.1</td><td>76.1</td><td>85.2</td><td>89.0</td><td>86.4</td><td>88.7</td><td>84.8</td><td>-</td><td>86.0</td></tr><tr><td>SV</td><td>87.0</td><td>88.6</td><td>89.2</td><td>87.0</td><td>85.4</td><td>89.8</td><td>87.3</td><td>75.8</td><td>84.7</td><td>86.3</td><td>84.7</td><td>88.0</td><td>82.0</td><td>79.1</td><td>86.0</td><td>89.2</td><td>86.7</td><td>87.9</td><td>83.7</td><td>80.9</td><td>-</td></tr></tbody></table>

<table><caption>Table 14: COMET of Llama-3.1-8B for each language pair (source language in rows and target language in columns)</caption><thead><tr><th>src/tgt</th><th>BG</th><th>CS</th><th>DA</th><th>DE</th><th>EL</th><th>EN</th><th>ES</th><th>ET</th><th>FI</th><th>FR</th><th>HU</th><th>IT</th><th>LT</th><th>LV</th><th>NL</th><th>PL</th><th>PT</th><th>RO</th><th>SK</th><th>SL</th><th>SV</th></tr></thead><tbody><tr><td>BG</td><td>-</td><td>90.6</td><td>87.4</td><td>85.8</td><td>83.9</td><td>89.0</td><td>87.7</td><td>76.1</td><td>84.5</td><td>86.8</td><td>86.2</td><td>88.7</td><td>78.0</td><td>75.4</td><td>86.8</td><td>90.4</td><td>87.2</td><td>90.0</td><td>84.5</td><td>84.9</td><td>88.7</td></tr><tr><td>CS</td><td>85.7</td><td>-</td><td>87.0</td><td>87.3</td><td>84.3</td><td>88.8</td><td>87.1</td><td>74.1</td><td>83.0</td><td>85.5</td><td>84.5</td><td>88.3</td><td>74.1</td><td>71.0</td><td>87.4</td><td>89.5</td><td>86.2</td><td>88.1</td><td>90.3</td><td>87.5</td><td>89.1</td></tr><tr><td>DA</td><td>83.1</td><td>89.3</td><td>-</td><td>86.1</td><td>82.3</td><td>89.2</td><td>85.3</td><td>73.6</td><td>84.2</td><td>85.3</td><td>85.1</td><td>87.0</td><td>75.2</td><td>72.2</td><td>86.5</td><td>88.5</td><td>84.7</td><td>86.8</td><td>79.7</td><td>79.2</td><td>90.5</td></tr><tr><td>DE</td><td>85.2</td><td>90.4</td><td>86.7</td><td>-</td><td>83.7</td><td>89.1</td><td>86.5</td><td>75.2</td><td>84.7</td><td>85.9</td><td>85.9</td><td>87.8</td><td>76.3</td><td>71.8</td><td>87.9</td><td>89.8</td><td>86.1</td><td>89.1</td><td>82.9</td><td>82.7</td><td>89.1</td></tr><tr><td>EL</td><td>85.5</td><td>89.7</td><td>87.1</td><td>85.2</td><td>-</td><td>89.5</td><td>87.5</td><td>75.4</td><td>84.9</td><td>87.1</td><td>84.6</td><td>88.6</td><td>77.3</td><td>74.1</td><td>86.2</td><td>89.8</td><td>87.2</td><td>90.0</td><td>83.5</td><td>82.6</td><td>88.8</td></tr><tr><td>EN</td><td>88.3</td><td>91.8</td><td>89.5</td><td>88.4</td><td>88.5</td><td>-</td><td>90.3</td><td>82.1</td><td>89.0</td><td>89.6</td><td>89.0</td><td>90.8</td><td>80.4</td><td>77.9</td><td>89.2</td><td>91.1</td><td>89.7</td><td>92.5</td><td>85.3</td><td>85.5</td><td>91.1</td></tr><tr><td>ES</td><td>86.2</td><td>89.7</td><td>86.7</td><td>85.3</td><td>86.1</td><td>89.7</td><td>-</td><td>75.9</td><td>85.6</td><td>88.2</td><td>86.2</td><td>90.0</td><td>76.5</td><td>72.5</td><td>86.7</td><td>90.0</td><td>89.6</td><td>89.8</td><td>82.3</td><td>82.0</td><td>88.5</td></tr><tr><td>ET</td><td>83.3</td><td>87.3</td><td>83.9</td><td>84.0</td><td>82.6</td><td>87.9</td><td>84.8</td><td>-</td><td>80.4</td><td>84.4</td><td>83.7</td><td>85.4</td><td>70.0</td><td>67.5</td><td>84.7</td><td>87.5</td><td>84.8</td><td>84.9</td><td>76.5</td><td>73.7</td><td>86.6</td></tr><tr><td>FI</td><td>84.6</td><td>88.6</td><td>86.4</td><td>85.4</td><td>84.7</td><td>89.7</td><td>86.7</td><td>71.4</td><td>-</td><td>86.4</td><td>84.8</td><td>87.4</td><td>70.2</td><td>67.2</td><td>86.3</td><td>88.4</td><td>86.6</td><td>86.3</td><td>77.9</td><td>75.2</td><td>88.8</td></tr><tr><td>FR</td><td>86.0</td><td>89.1</td><td>86.3</td><td>85.5</td><td>85.2</td><td>89.2</td><td>88.6</td><td>74.1</td><td>85.3</td><td>-</td><td>86.0</td><td>90.1</td><td>76.0</td><td>73.6</td><td>86.9</td><td>89.8</td><td>88.2</td><td>89.9</td><td>81.6</td><td>82.2</td><td>88.1</td></tr><tr><td>HU</td><td>84.1</td><td>87.7</td><td>84.1</td><td>85.4</td><td>83.4</td><td>88.7</td><td>86.6</td><td>66.7</td><td>79.8</td><td>85.8</td><td>-</td><td>87.7</td><td>72.2</td><td>69.6</td><td>84.7</td><td>88.3</td><td>86.4</td><td>86.0</td><td>76.8</td><td>77.6</td><td>87.4</td></tr><tr><td>IT</td><td>84.6</td><td>89.3</td><td>85.7</td><td>84.9</td><td>84.1</td><td>88.4</td><td>88.1</td><td>73.9</td><td>84.1</td><td>87.4</td><td>85.1</td><td>-</td><td>75.4</td><td>72.0</td><td>86.2</td><td>89.7</td><td>87.8</td><td>88.5</td><td>81.5</td><td>80.8</td><td>87.9</td></tr><tr><td>LT</td><td>82.9</td><td>85.6</td><td>82.4</td><td>82.3</td><td>81.2</td><td>85.8</td><td>83.3</td><td>66.9</td><td>80.5</td><td>82.0</td><td>82.5</td><td>84.6</td><td>-</td><td>68.6</td><td>81.8</td><td>87.1</td><td>82.5</td><td>84.4</td><td>73.1</td><td>73.5</td><td>85.5</td></tr><tr><td>LV</td><td>84.0</td><td>86.9</td><td>84.2</td><td>83.6</td><td>81.6</td><td>86.9</td><td>83.8</td><td>68.3</td><td>82.0</td><td>83.6</td><td>83.8</td><td>85.7</td><td>72.3</td><td>-</td><td>84.0</td><td>87.1</td><td>83.6</td><td>85.9</td><td>75.6</td><td>77.0</td><td>86.6</td></tr><tr><td>NL</td><td>83.9</td><td>88.5</td><td>85.1</td><td>86.3</td><td>82.2</td><td>88.2</td><td>85.3</td><td>71.9</td><td>83.2</td><td>85.4</td><td>84.3</td><td>86.8</td><td>74.3</td><td>69.9</td><td>-</td><td>88.0</td><td>84.7</td><td>86.0</td><td>79.2</td><td>77.3</td><td>87.5</td></tr><tr><td>PL</td><td>86.0</td><td>89.9</td><td>86.0</td><td>85.9</td><td>84.1</td><td>87.6</td><td>87.1</td><td>71.8</td><td>83.4</td><td>86.2</td><td>84.9</td><td>88.4</td><td>75.0</td><td>71.2</td><td>86.3</td><td>-</td><td>86.5</td><td>88.5</td><td>82.6</td><td>81.2</td><td>88.5</td></tr><tr><td>PT</td><td>86.4</td><td>89.2</td><td>86.4</td><td>85.3</td><td>85.6</td><td>89.5</td><td>89.8</td><td>74.7</td><td>84.8</td><td>88.3</td><td>85.2</td><td>89.9</td><td>75.4</td><td>71.5</td><td>86.4</td><td>89.4</td><td>-</td><td>88.1</td><td>80.8</td><td>80.2</td><td>88.6</td></tr><tr><td>RO</td><td>86.4</td><td>89.1</td><td>86.0</td><td>86.1</td><td>85.7</td><td>90.2</td><td>88.6</td><td>69.3</td><td>82.8</td><td>87.8</td><td>84.8</td><td>88.9</td><td>72.4</td><td>69.1</td><td>86.2</td><td>88.7</td><td>87.1</td><td>-</td><td>79.7</td><td>77.8</td><td>88.4</td></tr><tr><td>SK</td><td>85.7</td><td>94.2</td><td>86.8</td><td>87.3</td><td>84.3</td><td>88.6</td><td>87.1</td><td>74.3</td><td>83.8</td><td>85.5</td><td>85.7</td><td>88.2</td><td>72.6</td><td>70.9</td><td>87.3</td><td>90.3</td><td>86.3</td><td>88.8</td><td>-</td><td>81.4</td><td>89.0</td></tr><tr><td>SL</td><td>85.2</td><td>90.3</td><td>86.0</td><td>86.0</td><td>83.5</td><td>88.2</td><td>86.4</td><td>68.4</td><td>82.6</td><td>84.9</td><td>84.8</td><td>87.3</td><td>71.5</td><td>66.4</td><td>86.1</td><td>89.0</td><td>85.4</td><td>87.3</td><td>82.1</td><td>-</td><td>88.1</td></tr><tr><td>SV</td><td>83.6</td><td>89.1</td><td>88.9</td><td>86.3</td><td>83.3</td><td>89.4</td><td>85.7</td><td>74.6</td><td>84.5</td><td>85.3</td><td>85.0</td><td>78.5</td><td>75.4</td><td>71.9</td><td>86.4</td><td>89.2</td><td>84.9</td><td>87.6</td><td>79.2</td><td>78.5</td><td>-</td></tr></tbody></table>
## B. Political Fairness per Target Language

We report the per-target-language sBLEU scores of every EU party for all models in Tables 15 and 16.

<table><caption>Table 15: sBLEU scores of Qwen3-4B per target language. Each language has its performance variance significantly explained by the party variable according to a Kruskal-Wallis test ($p < 0.01$).</caption><thead><tr><th>tgt</th><th>ALDE</th><th>ECR</th><th>EFA</th><th>EFD</th><th>EPP</th><th>NA</th><th>NGL</th><th>S&D</th></tr></thead><tbody><tr><td>BG</td><td>15.90</td><td>16.75</td><td>15.89</td><td>16.19</td><td>16.44</td><td>14.75</td><td>15.85</td><td>16.65</td></tr><tr><td>CS</td><td>12.60</td><td>12.46</td><td>12.44</td><td>12.39</td><td>12.69</td><td>12.04</td><td>11.56</td><td>12.76</td></tr><tr><td>DA</td><td>15.96</td><td>15.78</td><td>15.50</td><td>15.09</td><td>15.74</td><td>14.37</td><td>15.09</td><td>15.61</td></tr><tr><td>DE</td><td>17.59</td><td>17.77</td><td>16.39</td><td>17.36</td><td>17.59</td><td>14.67</td><td>16.26</td><td>17.60</td></tr><tr><td>EL</td><td>12.51</td><td>12.31</td><td>12.25</td><td>12.34</td><td>12.93</td><td>11.41</td><td>11.57</td><td>12.92</td></tr><tr><td>EN</td><td>33.19</td><td>32.89</td><td>31.86</td><td>33.34</td><td>34.01</td><td>31.88</td><td>32.21</td><td>33.93</td></tr><tr><td>ES</td><td>26.73</td><td>25.83</td><td>25.66</td><td>27.33</td><td>28.22</td><td>25.99</td><td>26.49</td><td>27.46</td></tr><tr><td>ET</td><td>5.90</td><td>5.81</td><td>6.09</td><td>5.42</td><td>5.56</td><td>5.63</td><td>5.28</td><td>5.63</td></tr><tr><td>FI</td><td>8.50</td><td>8.11</td><td>8.43</td><td>7.69</td><td>8.11</td><td>7.73</td><td>7.76</td><td>8.07</td></tr><tr><td>FR</td><td>22.26</td><td>21.90</td><td>21.82</td><td>22.77</td><td>23.61</td><td>21.05</td><td>22.40</td><td>23.05</td></tr><tr><td>HU</td><td>8.78</td><td>8.28</td><td>9.09</td><td>8.60</td><td>8.99</td><td>8.57</td><td>8.10</td><td>8.72</td></tr><tr><td>IT</td><td>18.23</td><td>17.15</td><td>17.85</td><td>17.95</td><td>18.60</td><td>17.08</td><td>17.91</td><td>18.42</td></tr><tr><td>LT</td><td>8.35</td><td>8.01</td><td>8.44</td><td>7.99</td><td>8.08</td><td>7.96</td><td>7.62</td><td>8.05</td></tr><tr><td>LV</td><td>7.57</td><td>7.30</td><td>7.88</td><td>7.21</td><td>7.40</td><td>7.13</td><td>6.85</td><td>7.30</td></tr><tr><td>NL</td><td>15.98</td><td>15.61</td><td>15.03</td><td>15.36</td><td>15.51</td><td>14.78</td><td>14.65</td><td>15.76</td></tr><tr><td>PL</td><td>12.10</td><td>11.61</td><td>11.87</td><td>11.85</td><td>12.29</td><td>11.78</td><td>11.68</td><td>12.19</td></tr><tr><td>PT</td><td>20.93</td><td>20.02</td><td>20.40</td><td>21.01</td><td>21.19</td><td>20.34</td><td>19.09</td><td>21.12</td></tr><tr><td>RO</td><td>19.34</td><td>18.80</td><td>19.16</td><td>19.52</td><td>19.69</td><td>18.72</td><td>18.56</td><td>19.70</td></tr><tr><td>SK</td><td>9.29</td><td>9.18</td><td>9.24</td><td>8.97</td><td>9.31</td><td>8.74</td><td>8.68</td><td>9.38</td></tr><tr><td>SL</td><td>10.26</td><td>9.98</td><td>10.29</td><td>9.61</td><td>10.08</td><td>9.68</td><td>9.92</td><td>10.09</td></tr><tr><td>SV</td><td>15.22</td><td>14.70</td><td>14.29</td><td>14.52</td><td>14.78</td><td>13.59</td><td>14.20</td><td>14.84</td></tr></tbody></table>

<table><caption>Table 16: sBLEU scores of Llama-3.1-8B per target language. Each language has its performance variance significantly explained by the party variable according to a Kruskal-Wallis test ($p < 0.01$).</caption><thead><tr><th>tgt</th><th>ALDE</th><th>ECR</th><th>EFA</th><th>EFD</th><th>EPP</th><th>NA</th><th>NGL</th><th>S&D</th></tr></thead><tbody><tr><td>BG</td><td>18.13</td><td>19.00</td><td>18.10</td><td>18.60</td><td>18.39</td><td>17.51</td><td>17.94</td><td>18.73</td></tr><tr><td>CS</td><td>17.75</td><td>17.78</td><td>17.58</td><td>17.70</td><td>17.90</td><td>17.29</td><td>16.89</td><td>18.00</td></tr><tr><td>DA</td><td>19.80</td><td>19.61</td><td>19.54</td><td>18.45</td><td>19.17</td><td>18.50</td><td>18.53</td><td>19.22</td></tr><tr><td>DE</td><td>20.86</td><td>21.23</td><td>20.12</td><td>20.33</td><td>20.34</td><td>18.11</td><td>19.22</td><td>20.55</td></tr><tr><td>EL</td><td>17.90</td><td>17.73</td><td>17.11</td><td>17.54</td><td>18.10</td><td>16.55</td><td>16.49</td><td>18.20</td></tr><tr><td>EN</td><td>35.52</td><td>35.23</td><td>34.52</td><td>35.94</td><td>35.80</td><td>34.27</td><td>34.39</td><td>36.06</td></tr><tr><td>ES</td><td>29.52</td><td>28.84</td><td>28.80</td><td>30.26</td><td>30.87</td><td>29.15</td><td>29.48</td><td>30.21</td></tr><tr><td>ET</td><td>8.16</td><td>8.02</td><td>8.30</td><td>7.53</td><td>7.75</td><td>7.99</td><td>7.39</td><td>7.86</td></tr><tr><td>FI</td><td>10.70</td><td>10.61</td><td>10.84</td><td>9.78</td><td>10.23</td><td>10.17</td><td>9.78</td><td>10.27</td></tr><tr><td>FR</td><td>25.23</td><td>24.64</td><td>24.86</td><td>25.58</td><td>26.33</td><td>24.04</td><td>25.27</td><td>25.89</td></tr><tr><td>HU</td><td>12.63</td><td>12.52</td><td>13.12</td><td>12.13</td><td>12.56</td><td>12.68</td><td>11.84</td><td>12.51</td></tr><tr><td>IT</td><td>20.92</td><td>19.92</td><td>20.50</td><td>20.50</td><td>21.10</td><td>19.80</td><td>20.59</td><td>21.00</td></tr><tr><td>LT</td><td>8.91</td><td>8.57</td><td>9.05</td><td>8.53</td><td>8.64</td><td>8.71</td><td>8.01</td><td>8.73</td></tr><tr><td>LV</td><td>9.04</td><td>8.81</td><td>9.12</td><td>8.80</td><td>8.98</td><td>8.81</td><td>8.40</td><td>8.96</td></tr><tr><td>NL</td><td>20.50</td><td>20.16</td><td>19.72</td><td>19.38</td><td>19.63</td><td>19.19</td><td>18.54</td><td>20.02</td></tr><tr><td>PL</td><td>15.95</td><td>15.66</td><td>15.65</td><td>15.71</td><td>16.22</td><td>15.70</td><td>15.21</td><td>16.34</td></tr><tr><td>PT</td><td>23.83</td><td>23.16</td><td>23.27</td><td>23.79</td><td>23.97</td><td>23.48</td><td>21.95</td><td>23.84</td></tr><tr><td>RO</td><td>21.75</td><td>21.46</td><td>21.68</td><td>21.74</td><td>22.04</td><td>21.55</td><td>21.09</td><td>22.15</td></tr><tr><td>SK</td><td>13.70</td><td>13.62</td><td>13.42</td><td>13.36</td><td>13.65</td><td>13.21</td><td>13.00</td><td>13.84</td></tr><tr><td>SL</td><td>13.57</td><td>13.65</td><td>13.56</td><td>13.16</td><td>13.70</td><td>13.10</td><td>13.12</td><td>13.80</td></tr><tr><td>SV</td><td>21.08</td><td>20.89</td><td>20.02</td><td>20.09</td><td>20.40</td><td>20.00</td><td>19.50</td><td>20.56</td></tr></tbody></table>