From Utterance to Vividity: Training Expressive Subtitle Translation LLM via Adaptive Local Preference Optimization
Authors: Chaoqun Cui
Shijing Wang
Liangbin Huang
Qingqing Gu
Zhaolong Huang
Xiao Zeng
Wenji Mao
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Subtitle Translation
Preference Optimization
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose the Adaptive Local Preference Optimization (ALPO) method to improve expressiveness and vividness of subtitle translation.

Abstract: The rapid development of Large Language Models (LLMs) has significantly enhanced the general capabilities of machine translation. However, as application scenarios become more complex, the limitations of LLMs in vertical domain translations are gradually becoming apparent. In this study, we focus on how to construct translation LLMs that meet the needs of domain customization. We take visual media subtitle translation as our topic and explore how to train expressive and vivid translation LLMs. We investigated the situations of subtitle translation and other domains of literal and liberal translation, verifying the reliability of LLM as reward model and evaluator for translation. Additionally, to train an expressive translation LLM, we constructed and released a multidirectional subtitle parallel corpus dataset and proposed the Adaptive Local Preference Optimization (ALPO) method to address fine-grained preference alignment. Experimental results demonstrate that ALPO achieves outstanding performance in multidimensional evaluation of translation quality.

Published as a conference paper at ICLR 2026
FROM UTTERANCE TO VIVIDITY: TRAINING EXPRES- SIVE SUBTITLE TRANSLATION LLM VIA ADAPTIVE LOCAL PREFERENCE OPTIMIZATION
Chaoqun Cuil.2, Shijing Wang', Liangbin Huang", Qingqing Gu", Zhaolong Huang", Xiao Zeng', Wenji Mao
MMAIS, Institute of Automation, Chinese Academy of Sciences
"ccooo of Artificial Intelligence, University of Chinese Academy of Sciences
HHujngg Digital Media & Entertainment Group
*Geely AI lab {cu cuuichacquu2225, wenji menji ) @ia.. ac cn
ABSTRACT
The rapid development of Large Language Models (LLMs) has significantly en- hanced the general capabilities of machine translation. However, as application scenarios become more complex, the limitations of LLMs in vertical domain translations are gradually becoming apparent. In this study, we focus how construct translation LLMs that meet the needs of domain customization. We take visual media subtitle translation as our topic and explore how to train expressive and vivid translation LLMs. We investigated the situations of subtitle translation and other domains literal and liberal translation, verifying the reliability of LLM as reward model and evaluator for translation. Additionally, to train an expressive translation LLM, we constructed and released multidirectional subtitle parallel corpus dataset and proposed the Adaptive Local Preference Optimization (ALPO) method to address fine-grained preference alignment. Experimental results demon- strate that ALPO achieves outstanding performance in multidimensional evaluation translation quality.
INTRODUCTION
In recent years, the rapid advancement of Large Language Models (LLMs) has significantly enhanced the general capabilities of machine translation (Xu et al. 2024c] Enis & Hopkins 2024 Feng et al.) 2024). Representative LLMs such GPT-4 (Achiam al. 2023) and Qwen3 (Yang al. 2025). trained on massive cross-lingual data, have demonstrated exceptional contextual comprehension and generation abilities in multilingual translation tasks. Their translation quality in general domains (e.g.. news, daily conversations) has approached human-level performance (Xu et al. 2024a b). However, as applications penetrate vertical domains, the limitations of LLMs in specialized translation scenarios have become increasingly apparent: critical issues include inadequate consistency in specialized terminology (An ct al. 20244 Huang et al. 2024). deviations from dedustry-sstnddan expressions
(Ling al. 2023 Pang al. 2025), and weak style adaptability (Zhang et al. (024). Consequently, LLMed machine translation research has increasingly focused on addressing domain-speciff customization requirements, such developing iitislationfoocusee LLMs that strictly adhere to
clause formulation norms or medicine-oriented LLMs that precisely align with medical terminology.
In this study, focus on visual media subtitle translation task, which aims to translate the lines in subtitles of visual media programs across genres such as movies, TV series, and documentaries from the source language into the target language. This task plays crucial role in the localization of entertainment programs and in fostering global cultural dissemination and exchange (Federico et al.) 2020; WWu al. 2023) Mhaskar et al. 2024), yet it remains an underexplored subfield in machine translation. Similar aa to literary translation, subtitle translation requires a localized and stylistic liberal translation of program lines to convey the atmosphere, emotions, and tone of the original lines.
Corresponning author.

---

Published as a conference paper at ICLR 2026
However, while LLMs can achieve high translation accuracy, they tend to favor literal translation (as we quantitatively examined in Section 3.2). Therefore, exploring how to train a customized subtitle translation LLM with strong expressiveness and vividness is our primary challenge.
We employ LLM LLMMaasa-Juddee (using LLMs as evaluators) and preference optimization techniques to build customized subtitle translation LLM. We conduct a quantitative investigation into the degree texts across different domains and the liberal translation
of liberal translation in performance of various LLMs in subtitle translation. The findings reveal that: (1) Compared the translation of serious texts requiring high accuracy (literal translation), such as legislation, news, and medicine, subtitle translation tends to favor more liberal translation; (2) In subtitle translation, the translations generated by chat LLMs tend to be more literal compared to those produced by reason LLMSs and humans. We also experimentally validate the reliability of LLMs as reward models and evaluators for subtitle translation, as well as their alignment with human preferences. This enables automated construction of preference data for preference alignment training, thereby enhancing the expressiveness and vividness of subtitle translation LLMs. Additionally, for the task of translating short subtitle lines, which requires fine-grained local preference alignment, we propose a novel preference alignment strategy called Adaptive Local Preference Optimization (ALPO).
In summary, the main contributions of this study are as follows:
We introduce the visual media subtitle translation task and investigate the extent of liberal translation and the reliability of LLLM-as-a-Judge in this domain.
We propose ALPO, a local preference optimization method, provide formal explanation of its effectiveness, and use it to train a subtitle translation LLM with high vividness.
We construct and release multi- directional subtitle parallel corpora based on an efficient bilingual subtitle alignment algorithm to support community research.
Experimental results demonstrate that our 14B LLM, trained with ALPO, achieves significant improvements across multiple dimensions and outperforms SOTA LLMs.
RELATED WORK
2.1 LLM-AS-A-JUDGE
The capacity of LLMs to emulate human reasoning and evaluate specific inputs against a set of predefined rules has paved the way for "LLM--s-a-Judge" (Zheng et al.] [223a] Gu al. (2024). Existing studies demonstrate that I.LMs' scalability, adaptability, and costteefecssveness render them particularly suitable for handling increasing volumes of assessment tasks that humans conventionally
performed (Chen et al. 2024a Li ei al. 2024a] Wang et al. 2024cc Zhu al. 2025). These capabilities are crucial for deploying LLMs flexibly across diverse evaluation scenarios and objectives, driving
their rapid adoption in practical evaluation scenarios (Wang al. 2024b Chen et al. 2024b Khattak
ct al. 2023).
Originally developed for language generation and comprehension tasks, LLMs have evolved sig- nificantly through advanced training methodologies such as Reinforcement Learning from Human Feedback (RLHF) (Ouyang et al. 2022). , enhancing their alignment with human values and reasoning processes. This alignment has al lowed LLMS to transition from generative tasks to evaluative roles. Fundamentally, LLM-as-a-Judge refers the deployment of LLMs assess objects, actions, or decisions according to predefined rules, criteria, or preferences (Kant) 1908; 1987). This framework encompasses a wide range of evaluative roles, including: Graders (Dong et al.) 2023 Luong et al. 2024).
Evaluators/A sssessoss (Li et al.] [2024b] Zhang et al. (2023). Criics Ke et al. [2024] Putta et al. Xiong et al. 2024), Verifiers (Shinn et al. 2023] Wang al. 2024d), Examiners (Bai et al.
2024
2023
Reward/Ranking Models (Luo et al. 2023 Sun et al. 2024
Yang et al. 2024b), etc.
2.2
LANGUAGE MODEL PREFERENCE OPTIMIZATION
Reinforcement learning provides an effective solution for aligning LLMs with human values and controlling text generation (Bender al.] 2021 Bommasani et al.) 2021 Thoppilan et al.] 2022 Taori et al.) 2023) Chiang et al.] 2023 Ji et al. 2023). To this end, RLHF framework based on
human feedback reward models has been established (Ouyang et al. 2022 Christiano al. 2017
2

---

Published as a conference paper at ICLR 2026
MacGlashan et al. 2017; Ziegler et al. 2019 Stiennon al. 2020 Zheng et al. 2023b). However, despite its effectiveness, the complexity, instability, and hyperparameter sensitivity of RLHF remain insufficiently addressed (Andrychowicz method named Direct Preference Optimization (DPO) (Rafailov et al. 2024) simplifies the RI RI.F framework by eliminating the need for explicit reward model construction reinforcement learning procedures, thereby avoiding dependence on reward models. Several variants have subsequently emerged, such SimPO, KTO, and IPO (Meng al. 2024] Azzrr al. 2024; Ethayarajh al. 2024). Nevertheless, when applied to local preference alignment task, these methods still exhibit limitations, including coarse granularity and gradient dilution (see Appendix forr formal specification).
al. 2021 Engstrom et al., 2020 Recently proposed
EMPIRICAL INVESTIGATION
3.1 LLM IS EXCELLENT TRANSLATION EVALUATOR
We aim to verify whether LLMs can achieve high consistency with human preferences in quality assessment across different dimensions of subtitle translation, thereby enabling LLMs to serve as both reward models and evaluators for subtitle translation model alignment training. We begin with a formal definition of LLMM-s--Evllutton
To + 1 (t)),
where ne is the LLM used for evaluation, C is the introduction and instruction in the prompt that guides the LLM's evaluation, is the line under evaluation, (t) represents multiple translations of s, and & denotes the evaluation score given by Te-
We investigated the correlation between human evaluators and LLM evaluators in terms of vividness scores for 10 different translations of 500 lines from our self-constructed Multilingual Subtitle Corpus (MuSC) dataset in Appendix A Each evaluator assigned score from 0 to 100 for the 10 translations of each line under evaluation, and then we computed the average Spearman rank correlation P across evaluators. We present the Spearman rank correlation results Figure []] The detailed experimental settings are provided in Appendix B.2.T The translation directions include en zh, en=de, zh= and zh=th, where Chinese and English, German, and Thai correspond high-, medium-, and low-resource languages, respectively.
Suedeman Rank Correlation
Cal Necpeetf hume
1:03
0.87
0.86
3.53
0.55
1.00
0.26
0.83
0.85
0.22
1.00
0:55
0.87
0.64
0.53
0:87
1.00
0.64
0.50
0.53
0.26
1.00
0.87
0.84
0.27
0:65
1.00
0.89
0.64
0.57
0.29
3.00
0.66
0.22
0.90
0:85
1.00
0.50
0.59
0.33
0:83
0.25
1.00
0.85
0.88
0.66
1:00
0.92
0.04
0:88
1.00
0:85
0.88
0.60
3.59
1.30
0.32
0.87
0.85
0.37
1.00
0:83
0.87
0.65
0.67
1.50
0.20
0.90
0.04
0:92
1.00
Hoheer
Chetk
sno
Hohter
Chile
Desabess
MP%
(a) en=>2b 2h
(b) en=dde
(c) zh=ten
(d) 2h-thh
Figure 1: Investigation of the consistency of multiple evaluators with Spearman rank correlation p.
The results show that the LLM evaluator exhibits high consistency with human evaluators across multiple directions, indicating its ability to handle deviations in cultural and linguistic conventions across languages. Notably, even the Qwen3- 14B model, with 14B parameters, achieves high agree- ment with both human evaluators and other SOTA LLMs across all directions (p 2 0.82). This indicates that 14B model, which incurs relatively low inference cost, can serve as an efficient reward model, forming the foundation of ALPO. Furthermore, Figure presents the Altman plots of Qwen3- 14B and human evaluators for identical line translations in en zh and zh-thh. The results show that the mean difference (MD) between the two is very low, indicating minimal systematic bias, and its limits of agreement (LoA) are within acceptable error margins for scoring on a scale of 100. This further confirms the consistency between the 14B model and human evaluators.
3

---

Published as a conference paper at ICLR 2026
Bland-Alltann Plat: Human va Gwenn3-14B
Bland-Altman Plat: Human Gwen3-148
934
(fem 0.25
Difeeerc (huree -a-----ee-- Bawck-oo 443)
50c 400
Aserage Scree human Qowrd 148)
Avssage Stroe (hamee and Dwerd--441)
(a) er= zh
(b) zh=tth
Figure 2: Bland-Altman plot comparing Qwen3-14B with human evaluator.
3.2 PARALLEL CORPORA ARE ACTUALLY NOT PARALLEL
Literal translation focuses on translation accuracy, striving to preserve the original form, while liberal translation emphasizes conveying meaning and flexibly adjusting the expression (Baker & Saldanha [019] MMndda et al.) 2022). Based on our experience and observation, translations in different domains exhibit varying degrees of liberal translation: domains such subtitle and literature translation often lean towards liberal translation, whereas legislation, medicine, and technical texts typically require highly accurate literal translation.
We designed relevant experiments to quantitatively validate this hypothesis. We investigated the back translation consistency of parallel corpora from different domains. Specifically, for a particular translation direction, we utilized GPT-4o to directly translate the target language translation back to the source language, and then calculated the BLEU and ChrF++ scores between the back-translated text and the original text. Lower scores indicate a higher degree of liberal translation. Detailed experimental settings are provided in Appendix B.2.2
The investigation results in Table indicate that the back-translated texts in domains such visual media, literature, and religion show lower similarity to the original texts compared to those in legislation, news, and medicine. This suggests higher degree of liberal translation. For visual media subtitle translation, translators integrate vidco information into their translations, aiming to convey the atmosphere, emotions, and tone of the original text; this results in greater expressiveness and vividness, sometimes at the cost of sacrificing some degree of accuracy.
3.3 CHAT LLMS FAVOR LITERAL, REASON LLMS EXCEL IN LIBERAL
Generally, LLMS capable of effectively performing liberal translation tend to generate expressive translations. We aim to verify the liberal translation capabilities of different LLMs. Our investigation spans multiple translation directions, where we sampled 2,000 lines from MuSC test set for each direction and prompted various LLMS to produce expressive and vivid translations (see the prompt in Appendix F.I. We then computed the pairwise BLEU similarity between human and multiple LLM-generated translations. The LLMs included are chat models such as GPT-4o (Achiam et al. 2023). Mwen-Max (Yang et al. 2024a), and Claude Opus 4.1 (Anthropic 2025). ) as well as reasoning models like GPT-5 Thinking (OpenAT 2025) and DeepSeek-RI (Guo et al. 2025). Additionally, we incorporated our supervised fine-tuning (SFT) Qwen2.5-14B model trained on MuSC.
The results in Figure indicatee indicate that translations generated by chat models exhibit higher similarity among themselves, suggesting a tendency toward more literal translation. In contrast, translations produced by reasoning models show lower similarity with other translations, demonstrating that these models, through thinking, better adhere to the instruction of liberal translation. This validates that LLM's inference-time scaling can effectively enhance translation performance. The human translation exhibits lower similarity with other translations, indicating its proficiency in liberal translation. We present demos of different translations in Appendix C.2 to visually demonstrate these findings.
4

---

Published as a conference paper at ICLR 2026
Table I: Domain liberal translation investigation. The highest and lowest are blue and orange.
Dataset
Domain
en
en= fr
en=es
BLEU
ChrF++
BLEU
ChrF++
BLEU
ChrF++
OpenSubtitles
Visual Media
15.00
37.08
17.84
37.80
21.60
43.78
Books
Literature
17.22
43.20
21.51
47.14
18.60
43.00
bible-uedin
Religion
22.55
49.57
22.07
48.24
22.31
51.03
DGT
19.85
50.05
22.44
48.10
21.88
52.24
Legislation
JRC-Acquis
27.55
61.95
28.83
69.65
25.16
65.30
News- Cows-Commentary
News
25.00
55.23
24.87
$4.37
35.990
65.84
ECDC
23.T8
60.77
23.86
62.97
27.20
65.39
Medicine
EMEA
25.20
59.36
24.73
61.65
31.46
69.15
Dataset
Domain
de=en
fr=en en
es-en en
BLEU
ChrF++
BLEU
ChrF++
BLEU
ChrF++
OpenSubtitles
Visual Media
15.25
39.41
16.60
40.65
22.18
47.45
Books
Literature
14.50
40.50
20.12
48.29
15.84
43.99
bible-uedin
Religion
15.86
47.14
22.79
51.85
24.55
53.77
DGT
19.70
47.77
23.31
48.91
22.87
52.49
Legislation
JRC-Acquis
27.54
63.43
31.33
64.15
27.68
64.46
News- Coomeentay
News
22.64
54.05
29.34
56.20
41.75
66.35
ECDC
21.09
57.41
29.13
64.81
29.62
64.20
Medicine
EMEA
24.94
60.16
26.24
62.96
29.86
67.09
25.5
28
240
363
242
427
413
060
252
245
60
2:
233
24
402
427
212
111
157
1:2
4244
19.
1611
20:3
27.
35.4
161
21.1
012
27.8
25.1
<hh
(a)
en= zh
(b) en=Ide
(c) zh=en
(d) zh-th
Figure 3: Pairwise BLEU scores between translations.
4 METHOD
4.1 NOTATIONS
bilingual dataset for subtitle translation comprises collections source language lines Cuve and target language lines Legs from multiple programs of genres such as films or TV series, represented by = {fatre Lgg) E Lare x }- The lines multilingual subtitles of the same program are often not in one-to-one correspondence. To address this, have developed bilingual parallel corpus construction algorithm with (NN) complexity; details can be found in Appendix A.2 We will utilize the dataset ID to perform SFT and ALPO training off-the-sheff L.MMs. The input prompt for the LLM includes the context C (comprising introduction and instructions) and set of n source lines {s: i [n]] that need be translated, i.c., & C @ {s:). The response from the LLM includes translations for each line {4 E [n]], i.e., = { See Appendix F. for detailed x and y format.
4.2 OVERALL FRAMEWORK
Obviously, constructing an expressive translation LLM preference optimization task. However, we cannot directly apply general methods (e.g., PPO Ouyang et al. 2022) DPO (Rafailov et al.) 2024)), primarily because the translation each line of subtitle depends on its context. The input x of the SET model Tan must include multiple lines of subtitle {s,), necessitating finer-grainet alignment for each line the response Consequently, outcome-supervised methods like PPO and DPO, which optimize the complete output of LLM,, inadequate for this task. We present experimental
5

---

Published as a conference paper at ICLR 2026
bata Sampling
Prozen Reference LLM
Bane
Policy LLM
peouol
Tipar
Sardingg
Traun
SFT
Irgue 98
Crapel (Tts Tae))
Datcaet
Datoset Tl.b
88
Aligrment
Trunslition
Seareo
Detaset
SFT LLM Fiah
Policy
Stepl SFT
Step2 Segment- wize Sempling
T Scep3 Training
Figure 4: The overall framework of ALPO.
and theoretical validation of this in Appendix C.4 and Appendix [ To address this fine- une-graieed preference optimization task, we propose the process-supervised ALPO method. ALPO introduces novel paradigm that leverages a segment-wiss -wise sampling strategy and fine-grained alignment loss to train high-quality subtitle translation LLM. The overall framework of ALPO is shown in Figure 4
4.3 SAMPLING STRATEGY
We refer to task requiring multi-segment local preference alignment of LLM response as a local preference optimization task. ALPO provides effective paradigm for handling such tasks. We initially divide the parallel corpus D into a demonstration dataset Da and an alignment dataset Dapo (approximately 8:2), using Dst to train SFT model Tsh (see Appendix F.1 for input-output format).
In sampling phase, for x E Dalpo which contains n lines . Sn. For each 8,, using ps = t ................................ (()) prefix, & translations are sampled (k 13 in experiments), i.e., Te at (t/ | p.).
I,
j = 1,....,k resulting in candidate set {t] = 1,...,, }z.. Then, the candidate set is deduplicated; if human reference E is available, it can be added to the candidate set, forming Ti = {
0, J, Subsequently, Qwen3-14h 14B model deployed as an evaluator We (or can regarded a reward model) to evaluate vividness of Tú obtaining the score sequence E, (see Table 2 for
demonstration). Based &,, select superior translation t/ for 8; (randomly chosen from the top 3 scores) to serve as the prefix for the next sampling cycle. Ultimately, for a x E Dalpo- we obtain the sampling result ((x) {(st, ((, &i) ......n.. Algorithm 1 details the entire process.
Table 2: The zh=en evaluation demonstration of the evaluator x..
Line
Evalution
"asttryy" tspaãts, fut Ge" E5. 35 Sabt 13 whar was."
70
distory night replay, but mankind cannot back in time.
85
The histery may
full role,, but
past stays forever looked,
Although histery may repeat icself humana cannos return the past. Bistory often echoss, yet there's a no way for us so tuzn back the clook.
92
4.4 ADAPTIVE ALIGNMENT LOSS
The reward signal acts on segments of model output (process-superviser) rather than the complete response quutoom--supervieet which constitutes the main challenge for vividness enhancement. ALPO enables propess-upeerviee alignment across multiple segments translation LLM's response Specifically, for all lines [s: i 1, ...., n) in an E iDaipo we assign each an adaptive weight u (s))), defined as the product of a gating function 1
(s;) and an importance score 5(x)):
w(x;) 1(s:) 5(s:),
where 1s:)) acts as a gate to determine whether s; participates in optimization. When the sampled translations lack diversity or clear quality distinction, i.e., [Tal < 3 or max(E,) min(&) < 5,

---

Published as a conference paper at ICLR 2026
Algorithm 1 ALPO Sampling Strategy.
Input: SFT model Tst. evaluation LLM ne, alignment dataset aappo sample number k. Output: sampled segment-level candidate set S(x).
for any æ E Dalpo do
// Iterate through the alignment dataset Dalpo
/ Iterate through the subtitle lines in æ. 2: for i = 1 to do
3:
for j = 1 to l: do // Sample multiple translation candidates.
4:
Sample Tan (t, xc, 1
(()) (2-1)-
5: end for
6:
Deduplicate the candidate set {e] j = -.......,
7: (Optional) Add human reference ! to the candidate set, get Ti = (1]
0, ..
8: Evaluate T: by Te. get score sequence E,.
9: Select a superior translation ( (random from top 3) as the prefix of next sampling cycle.
10: end for
11:
end for
return S(x) [(8;, Ti, & 1 n.,
set (8:) ) = 0: otherwise, (x;) = The importance score 0(8;) depends on the diversity of sampled translations. Since lines with richer translations provide more potential for vividness enhancement, we assign higher 0(;;) to lines with a larger number of distinct sampled translations after deduplication:
T
8(st) = 2j=1 IT
We then apply a Tradley-Terry (DPO-style) preference optimization loss achieve fine-grained preference alignment of the translation model:
Calpo(TTs meat) = - Σ w((s) log a 3g log
(o pi) Tee (1()
(o( (!) Te (1 p.)
ps)
Pp)) 3r log
where is the policy model, and Teet is the reference model. The chosen translation { is randomly sampled from the top scored translations &,, while the rejected translation E is selected as the third-loweed lowest scored candidate (excluding the lowest one to avoid overly trivial contrastive pairs). 3 is a hyperparameter controlling sensitivity to reward differences. Since the reward gap between ( and 6 also reflects the diversity of translations, we 3r dynamically follows:
((s,, r(s, (
3 max (rx(s(s,, (00 r(s [5E
(5)
where r(st, ((0) denotes the score of t in &, Moreover, instead of directly setting the prefix in Calpo P; x, { ................................ (1 adopt scheduled prefix mixing strategy to mitigate exposure bias when non-chosen translations are generated during inference. Specifically, with probability X, t/ is appended to the prefix, and with probability 1 - X, translation sampled from T; is appended (in experiments, increased from 0.2 to 0.6 as training progresses), i.e.,
p: = .
f--1 & + Mix(t)0) ,ts ,r, TTA).
EXPERIMENTS
5.1 EXPERIMENTAL SETTINGS
We experiments using customized MuSC dataset, which comprises subtitle corpora from the online video platform Youku for multiple directions, including en= en en ko= zh, zh=en, and zh Each direction includes 100-200 programs across various genres, with 10% of the programs reserved as the test set. See Appendix A for details of MuSC.
We compare model with the following baselines:

---

Published as a conference paper at ICLR 2026
2023) constructs length-controlled translation model (the work most closely related to our newly proposed subtitle translation task).
VideoDubber (Wu et al.
NLLB-3.3B (Costa-jussà et al. 2022) is translation model of Meta AI (>200 languages).
MADLAD-10B (Kudugunta et al.
2024) is translation model of Google (>450 languages).
Google Translate (Google LLC
2023 is multilingual translation service of Google.
GPT-40, Qwen-Max, and DeepSeek- V3.1 (Liu et al. 2024) are current SOTA chat models.
DeepSeek-RI and GPT-5 Thinking are SOTA reasoning models. We use the prompt in Appendix F.T to drive these LLMs for subtitle translation.
Qwen2.5-14B serves as the backbone for ALPO.
Details of the experimental setup are provided in Appendix B dataset are available at httpp:: httppp///tttttub.cmooo/ot//////a http:/uwb..oo//cccueeeeeaaaa/aaa
The source code for ALPO and MuSC
5.2 TRANSLATION QUALITY EVALUATION
Unlike the strict requirements for accuracy in the translation of legal texts and technical documents, subtitle translation focuses more on the localized representation of the translation PPérez-González
2019 Bajeić & Golenko 2024). Specifically, subtitle translation demands not only the conveyance of the original meaning but also expects to reflect the emotions and tone of the original subtitle, aiming for translations that are more fluid, dynamic, and expressive. Given that we have verified the reliability of LLMs as evaluators in Section 3.1 we developed a multidimensional quality evaluation system for subtitle translation based on LLM-as-a-Judde scoring. We primarily assess three dimensions of subtitle translation: (1) Accuracy: whether the translation accurately conveys the original meaning of the subtitle; (2) Naturalness: whether the expression in the translation is natural and fluent, aligning expression
with the grammatical structure and lexical conventions of the target language; (3) Vividness: whether the translation is expressive and successfully conveys the emotions and atmosphere of the original subtitle. We employ DeepSeek- 1,.1, Claude Sonnet 4, and GPT-5 Instant as evaluation models to score subtitle segments in the test set along three dimensions, assigning a score between 0 and 100 for each. Then, we show the average score across all models in Table
Table 3: Multidimensional quality evaluation. The 1st and 2nd best results are denoted as blue and orange. ST for supervised training. ICL for in-context learning.
Medels
Training
Accuracy
Naturalness
en-ede
Vividness
Accuracy
Naturalness
an
Vividness
Accuracy
en Naturalness
Vividness
Gold Reterence
Human
84.8
73.1
14.8
82.6
71.5
VibheDubher
41.5
35.5
45.
48.22
47.3
48.5
46.9
$1.9
29.7
N.LB-3.3B
MADLAD-10B
ST
73.2
75.8
65.6
70.1
59.2
54.5
73.6
76.9
67.8
74.6
57.1
61.8
61.4
59.7
$4.0
55.5
43.7
46.3
Google Translate
89,9
79.7
GPP
"U2.1
86.7
823
Qwen Max
(C)
95.5
89.2
68.9
94.1
89.9
71.6
91.9
B1.1
61.3
DeopSeck-V3.
94,8
89.2
67.4
949
M6.7
72.3
91.2
85.3
63.5
DeepSeakX
'2.
87."
7350
'00
857'
70.8"
GPT-5
(CL(R)
93.6
88.6
72.7
92.2
90.4
75.8
92.4
87.0
71.1
0.4
SPT
67.5
0.66
6.4
"6.2
85
677
56.1
820
5071
Qwen2.5-14B
ALPO
95.4
88.4
7AS
94,1
89.2
78.8
90.6
81.3
76.6
Models
Training
Accuracy
Naturalness
kc
Vividness
Accuracy
2h: Naturalness
Vividness
Accuracy
zhatch Naturalness
Vividness
Gold Reterenes
Human
78.0
77.8
65.8
83.0
B0.3
23.3
76.6
7511
06.3
Vubho
99.6
45.2
48
53.6
$4.8
50.1
34.1
$49
31.5
NLLB-3.3B
20.8
MADLAD- J0B
41,9
12.9
46.7
45.1
38,9
37.6
47.9
50.8
51.0
Google Translate
$4.9
52.8
52.0
70.8
66.3
50.2
55.2
56.2
54.5
"PPT-66
"80.D
703
18.3
837
5466
"8.7"
642
B73
Qwen-Max
(CL,
83.7
61.8
900
85.0
66.8
85.8
DeepSesk.V3.
83.1
82.2
57.2
89.5
63.0
84.6
GPT5
(C)
10.3
84.5
81.6
82.6
R56
65.0
88.
89.1
86.1
857
75.2
75.5"
667.
88.7
83.9
647)
23,11
713F"
Qwen2.5514B
SFT
80.9
63.2
80.1
348
87.3
82.6
Quen2.5-148
ALPO
84.3
X3.3
70.5
88.3
86.8
81.7
91.9
84.7
7442
The results indicate that ALPO and cutting-edge LLMs like GPT-4o significantly outperform tradi- tional translation models like MADLAD, and surpass human translation in accuracy and naturalness. However, human translation excelled in vividness, possibly suggesting that human translations often incorporate more liberal translation by integrating video content. Reason models achieved better vividness score compared to chat models, although its accuracy and naturalness were slightly lower
8

---

Published as a conference paper at ICLR 2026
in some aspects. This aligns with the conclusions in Section 3.3 strated marked improvement not only in vividness but also in accuracy and naturalness compared to the SFT model. ALPO attained the highest vividness scores across all directions and occasionally sur- passed other cutting-edge LLMS in other dimensions, particularly excelling in relatively low-resource language directions such zh and zh=th. For the complete results of quality assessment and the prompt design of the LLM evaluator, please refer to Appendix C.I and Appendix F3
Models trained with ALPO demon-
5.3 HUMAN EVALUATION OF TRANSLATION QUALITY
We present the human evaluation results of the translation quality of ALPO for en zh and zh th in 4 Due to varying subjective preferences among evaluators, we do not adopt a scoring system. Instead, perform pairwise comparisons of different translations assess the win rate metric. We evaluate the ALPO model alongside four baselines across dimensions of accuracy, naturalness, and vividness, and conduct a comprehensive assessment. The human evaluation results consistent with those shown in Table which supports the reliability of our multidimensional evaluation system based on
Specific experimental settings can be found in Appendix B.3
Table 4: Human translation quality evaluation, reporting win rate (win:tie:loss). The winning and losing contrasts are marked in blue and orange, respectively.
Challenger
Competitors
Accuracy
Comprehendve
Accuracy
Comprehenstve
Gold Reference
29:49:23
28:50:22
32:42:28
91:46:23
28:48:24
26:52:22
32:39:29
29:39:22
ALPO (Qwe22.5 5-14B)
SPT Model Tia
26:50:24
81:08:21
38:11:211
37:43:20
26:55:19
27:51:22
19:12:19
90:30:20
GPT-4u
22:54:24
20:57:23
29:51:20
26:54:23
23:57:20
23:51:26
36:37:27
30:45:25
DeepSeck-R1
22:55:23
19:57:24
22:58:20
20:50:21
23:55/22
18:62:20
27:47:24
5.4
ABLATION STUDY
5.4.1 ADAPTIVE STRATEGY
We validate the impact of several adaptive strategies in ALPO on model performance in Table Results show that the gating function 1(s;) has the greatest effect, indicating that the gating mecha- nism effectively reduces noise from low-diversity irrelevant lines (e.g., simple lines such "Good morning.") Moreover, ablating the importance score 0(8;), dynamic 33, and scheduled prefix mixing also leads to varying degrees performance drop, highlighting the necessity of fine- fine-grained control in training.
Table 5: Impact of adaptive strategies.
en
zh Naturalness
Accuracy
Naturalness
Vividness
Accuracy
Vividness
SFT
86.5
82.1
59,2
87.2
82.4
66.0
ALPO
90.6
84.2
76.6
91.9
84.7
74.1
w/o ((s))
88.1
83.2
67.4(19.2)
89.2
83.4
70.3(I3.8)
w/o 1(s,)
89.1
83.4
70.2()6.4)
89.9
83,9
71.2([2.9)
w/o ((s,)
89.4
83.5
72.4([4.2)
90.2
83.1
71.9([2.2)
fixed B (8 = 0.5)
90.0
83.4
74.7(11.9)
89.8
83.4
72.1((2.0)
w/o prefix mixing
89.5
83.7
73.4(J3.2)
88.8
83.7
71.6(12.5)
5.4.2 BACKBONE MODELS
We compare the performance of different backbone models Table The results demonstrate that ALPO enables all backbone models to achieve significant performance improvements compared to the SET model (Qwen2.5-1433)
Notably, the larger Qwen2.5- model with more parameters delivers superior and stable outcomes. LLaMA-3. 1.MMA331153 3-BBBB exhibits underperformance in Chinese-related tasks, resulting in inferior results compared the other two models in both en and en translation directions. Nevertheless, it still achieves measurable improvements the SFT baseline.

---

Published as a conference paper at ICLR 2026
Table Impact of backbone models. The 1st and 2nd best results are marked as blue and orange.
Method
Backbone
Accuracy
envode Naturalnes
Vividnoss
Aucuracy
enooch Naturalnce
Visiduess
Accuracy
shisenn Naturalnos
Visidusss
SFT
K7.7
31.4
04.3
40.3
50.2
63.2
M6.7
$4.0
L.MM
L&R Diibee
2024
9677
87.2
753
35.0
852
72.3
87.2
85.3
77.6
ALPO
GLM4-98 G. pir
93.2
56.1
75.2
M8.
54.
74.4
89.2
85.7
78.2
Qwmn25
14 YTAg 211244
952
15.3
748
98.6
$4.3
76.6
88.3
868
81.6
5.5 ADDITIONAL EXPERIMENTS
perform the following extended experiments in the appendix: In Appendix C. we present full results of quality evaluation with further analysis. 2) In Appendix C.2 show demonstration examples of ALPO translations. 3) In Appendix C.3 We conduct ablation studies for other factors like samping and model size. 4) In Appendix C.4 perform performance evaluation comparing DPO and PPO. 5) In Appendix C.5
we validate the effectiveness of ALPO other application task.
CONCLUSION
In this study, we investigate literal and liberal translation in subtitle translation as well as other domains, and validate the reliability of large language models as evaluators and reward models for translation quality. Building upon this, we propose the Adaptive Local Preference Optimization (ALPO) method for training expressive and vivid subtitle translation. Our experiments, conducted under the established translation quality evaluation framework, validate the effectiveness of ALPO.
ETHICS STATEMENT
This study utilizes subtitle data obtained exclusively from the online video platform Youku. All data employed in this study were acquired with proper authorization and comply with the terms and conditions set forth by Youku. The data were used solely for academic purposes and were handled in a manner that ensures compliance with relevant ethical and legal standards.
No personal sensitive information collected processed in this research. All analyses were performed on accessible content provided by Youku, and no attempts were made to infer or reveal any private identifying information about individuals or entities featured in the materials.
We affirm that this study adheres to the principles of responsible and ethical research and complies with all applicable institutional and regulatory guidelines.
REPRODUCIBILITY STATEMENT
We release all code, training scripts, and the MuSC dataset used in this study to facilitate repro- ducibility and further research. Detailed experimental settings, model configurations, and evaluation protocols are provided in the main text and appendices. The source code and dataset are publicly accessible htttps:: / http:///unneeeeeercca/aa....
ACKNOWLEDGMENNTS
This work is supported in part by the National Natural Science Foundation of China under Grant #72293575, and the Joint Research Project on the Integration of Culture, Science and Technology between Chinese Academy of Sciences and Hunan Province #2024JK4003.
REFERENCES
Atef Odeh AbuSa' aleek. The adequacy and acceptability of machine translation translating the islamic texts. Int. J. Engl. Linguist, 6:185-193, 2016.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint arXiv:2303.08771 2023.
10

---

Published as a conference paper at ICLR 2026
Zhiyu An, Xianzhong Ding, Yen-Chun Fu, Cheng- Chung Chu, Yan Li, and Wan Du. Golden-retrieves High-fidelity agentic retrieval augmented generation for industrial knowledge base. arXiv preprint arXiv:2408.00798, 2024.
Marcin Andrychowicz. Anton Raichuk, Piotr Stañczyk, Manu Orsini, Sertan Girgin, Raphaël Marinier, Leonard Hussenot, Matthieu Geist, Olivier Pietquin, Marcin Michalski, et al. What matters for on-policy deep actor-critic methods? a large-scale study. In International conference learning representations, 2021.
Anthropic. Anthropic's transparency hub: Model report, 2025. URL htttp::/www.antte. com/ hhtp://aaanssaareee/mmmmellooooo
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, pp. 4447 4455. PMLR, 2024.
Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, et al. Benchmarking foundation models with saoaaage-moel-asss as-an-cxaminen Advances in Neural Information Processing Systems, 36:78142-788 127717777 2023.
Martina Bajeié and Dejana Golenko. Applying large language models in legal translation: The state-of-the-art. International Journal of Language Law (JIL), 13:171-196, 2024.
Mona Baker and Gabriela Saldanha. Routledge encyclopedia of translation studies. Routledge, 2019.
Susan Bassnett. Translation studies. routledge, 2013.
Emily M Bender, Timnit Gebru, Angelina McMillan-MMe Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp. 610-623, 2021.
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportuni- ties and risks of foundation models. arXiv preprint wtt.i::211100225555 2021.
Daoyuan Chen, Yilun Huang, Zhijian Ma, Hesen Chen, Xuchen Pan, Ce Ge. Dawei Gao, Yuexiang Xie, Zhaoyang Liu, Jinyang Gao, et al. Data-juicer: A one-stop data processing system for large language models. In Companion of the 2024 International Conference on Management of Data, pp. 120-134, 2024a.
Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Milm-as-a- juduee:: Assessing multimodal llm-as-a-judee with vision-language benchmark. In Forty-first International Conference on Machine Learning, 2024b.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicua. Imsys. org (accessed 14 April 2023), 2(3):6, 2023.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
Christos Christodouloupoulos and Mark Steedman. massively parallel corpus: the bible in 100 languages. Language resources and evaluation, 49:375-395, 2015.
Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. No language left behind: Scaling Guman-centeree machine translation. arXiv preprint 2.X.::220.00667. 2022.
Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. Raft: Reward ranked finetuning for generative foundation model alignment. Transactions Machine Leamning Research, 2023, 2023.
11

---

Published as a conference paper at ICLR 2026
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama herd of models. arXiv preprint urXiv:2407.21783 2w..22477..7778 2024.
Logan Engstrom, Andrew llys, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Aleksander Madry. Implementation matters in deep policy gradients: A case study on ppo and trpo. In International Conference on Learning Representations, 2020.
Maxim Enis and Mark Hopkins. From Ilm to nmt: Advancing low-resource machine translation with claude. arXiv preprint arXiv:2404.1388 13813, 2024.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306 2024.
Marcello Federico, Robert Enyedi, Roberto Barra-Chicote, Ritwik Giri, Umut Isik, Arvindh Kr- ishnaswamy, and Hassan Sawaf. From speech-to-speech translation to automatic dubbing. In Proceedings of the 17th International Conference on Spoken Language Translation, pp. 257-264, 2020.
Zhaopeng Feng, Ruizhe Chen, Yan Zhang. Zijie Meng, and Zuozhu Liu. Ladder: A model-agnostic framework boosting llm-based machine translation to the next level. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 15377-15393, 2024.
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas,
Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406. 12793, 2024.
Google LLC. Google translate, 2023. URL http:://raassate..cm goo.leee..eeggol.eomo com Accessed: 2025-05-15.
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuchao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on lim-as-a-judge. arXiv preprint arXiv:2411.15594, 2024.
Nuno M Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André FT Martins. xcomet: Transparent machine translation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979-995, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 2025.
Kaiyu Huang, Fengran Mo, Xinyu Zhang, Hongliang Li, You Li, Yuanchi Zhang, Weijian Yi, Yulong Mao, Jinchen Liu, Yuzhuang Xu, et al. A survey large language models with multilingualism: Recent advances and new frontiers. arXiv preprint arXiv:2405.I 10936, 2024.
Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, and Xiangang Li. Belle: Be everyone's s large language model engine, 2023.
Immanuel Kant. Critique of pure reason. 1781. Modern Classical Philosophers, Cambridge, MA: Houghton Mifflin, pp. 370-456, 1908.
Immanuel Kant. Critique of judgment. Hackett Publishing, 1987.
Pei Ke. Bosi Wen, Andrew Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng. Yuxiao Dong, Hongning Wang, al. Critiquellm: Towards an informative critique generation model for evaluation of large language model generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13034- 13054, 2024.
Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Mudti-modal prompt learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 191 191122 2023.
12

---

Published as a conference paper at ICLR 2026
Aobo Kong, Wentao Ma, Shiwan Zhao, Yongbin Li, Yuchuan Wu, Ke Wang. Xiaoqian Liu, Qicheng Li, Yong Qin, and Fei Huang. Sdpo: Segment-level direct preference optimization for social agents. arXiv preprint atXiv:250100882.1 2025.
Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. Madlad-400: A multilingual and document-level large audited dataset. Advances Neural Information Processing Systems, 36, 2024.
Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Pengfei Liu, et al. Generative judge for evaluating alignment. In The Twelfth International Conference on Learning Representations. 2024a.
Lijun Li, Bowen Dong. Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. belladbbnech: A hierarchical and comprehensive safety benchmark for large language models. In Findings of the Association for Computational Linguistics: ACL. 2024, pp. 3923-3954, 2024b.
Chen Ling, Xujiang Zhao, Jiaying Lu, Chengyuan Deng, Can Zheng, Junxiang Wang, Tanmoy Chowdhury, Yun Li, Hejie Cui, Xuchao Zhang, et al. Domain specialization as the key to make large language models disruptive: A comprehensive survey. arXiv preprint arXiv:2305.1871 187033 2023.
Pierre Lison and Jörg Tiedemann. Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC" 16), pp. 923-929, 2016
Aiwei Liu, Haoping Bai, Zhiyun Lu, Yanchao Sun, Xiang Kong, Xiaoming Simon Wang, Jiulong Shan, Albin Madappally Jose, Xiaojiang Liu, Lijie Wen, et al. Tis-dpo: Token-level importance sampling for direct preference optimization with estimated weights. In The Thirteenth International Conference on Learning Representations, 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint 1XX1:22412.19437, 2024.
Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. Large language models are superpositions of all characters: Attaining arbitrary role-play via self-alignment. arXiv preprint arXiv:2401.12477 2024.
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583. 2023.
Trung Quoe Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning, arXiv preprint arXiv:2401.08967, 3, 2024.
James MacGlashan, Mark K Ho, Robert Loftin, Bei Peng, Guan Wang, David L Roberts, Matthew Taylor, and Michael Littman. Interactive learning from policy-dependent human feedback. In International conference on machine learning, pp. 2285-22999 PMLR, 2017.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with reference- free reward. arXiy preprint arXiv:2405. 147344 2024.
Shivam Mhaskar, Nirmesh Shah, Mohammadi Zaki, Ashishkumar Gudmalwar, Pankaj Wasnik, and Rajiv Shah. Isometric neural machine translation using phoneme count ratio reward-based reinforcement learning. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 3966-3976, 2024.
Jeremy Munday, Sara Ramos Pinto, and Jacob Blakesley. Introducing translation studies: Theories and applications, Routledge, 2022.
OpenAI. Gpt-5, 2025. URL https:://ppeett www.aaacccooooooooommmmmmmoooo.a comm/ptt-
13

---

Published as a conference paper at ICLR 2026
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Jianhui Pang, Fanghua Ye, Derek Fai Wong, Dian Yu, Shuming Shi, Zhaopeng Tu, and Longyue Wang- Salute the classic: Revisiting challenges of machine translation the age of large language models. Transactions of the Association for Computational Linguisties, 13:73- 95, 2025.
Luis Pérez-González. Audiovisual translation. In Routledge encyclopedia of translation studies, pp. 30-34. Routledge, 2019.
Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.071 199, 2024.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36, 2024.
Eman Rashed Alkatheery. Google translate errors in legal texts: Machine translation quality assess- ment. AWEJ for Translation & Literary Studies, 7(1). 2023.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. meepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Wentao Shi, Mengqi Yuan, Junkang Wu, Qifan Wang, and Fuli Feng. Direct multi-turn preference optimization for language agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.
2024.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634 8652, 2023.
Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. Trial and error: Exploration-based trajectory optimization of Ilm agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 7584- 7600, 2024.
Ralf Steinberger, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaz Erjavec. Dan Tufis, and Dániel Varga. The jrc-aequis: multilingual aligned parallel corpus with 20+ languages. arXiv preprint cs/0609058, 2006.
Ralf Steinberger, Andreas Eisele, Szymon Klocek, Spyridon Pilos, and Patrick Schlüter. Dgt-tm: A freely available translation memory in 22 languages. arXiv preprint arXiv: weXivv:1309.00 309.5226, 2013.
Ralf Steinberger, Mohamed Ebrahim, Alexandros Poulis, Manuel Carrasco-Benitez, Patrick Schlütee, Marek Przybyszewski, and Signe Gilbro. overview of the european union's highly multilingual parallel corpora. Language resources and evaluation, 48:679-707, 2014.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-300 2020.
Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liangyan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. In Findings of the Association for Computational Linguistics ACL 2024, pp. 13088-131 2024.
Richard S Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An eestuuction-folloiinn llma model, 2023.
14

---

Published as a conference paper at ICLR 2026
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng- -ze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint 22X.v2200..08239, 2022.
Jörg Tiedemann. Parallel data, tools and interfaces in opus. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Mehmet Ugur Dogan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis (eds.), Proceedings of the Eight International Conference Language Resources and Evaluation (LREC' 12), Istanbul, Turkey, may 2012. European Language Resources Association (ELRA). ISBN 978-2-95174008-711
Ruiyi Wang. Haofei Yu, Wenxin Zhang, Zhengyang Qi. Maarten Sap, Graham Neubig, Yonatan Bisk, and Hao Zhu. pot:pia-pi: Interactive learning of socially intelligent language agents. arXiv preprint a 203vv22403088715 2024a.
Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi- Yu, Richard Yuanzhe Pang, Maryam Fazel-Zarandi,, Jason Weston, and Xian Li. Self-taught evaluators. arXiv preprint arXiv:2408. 026666666666... 2024b.
Yidong Wang, Zhuohao Yu, Wenjin Yao, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Pandalm: An automatic evaluation benchmark for Ilm instruction tuning optimization. In The Twelfth International Conference on Learning Representations, 2024c.
Zilong Wang, Zifeng Wang, Long Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, et al. Speculative rag: Enhancing retrieval augmented generation through drafting. arXiv preprint 2Xii:24477002233 2024d.
Yihan Wu, Junliang Guo, Xu Tan, Chen Zhang, Bohan Li, Ruihua Song. Lei He, Sheng Zhao, Arul Menezes, and Jiang Bian. Videodubber: Machine translation with speech-aware length control for video dubbing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.
13772-13779, 2023.
Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. Llava-critic: Learning evaluate multimodal models. arXiv preprint arXiv.2410.02712, 2024.
Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. A paradigm shift in ma- chine translation: Boosting translation performance of large language models. In The Twelfth International Conference on Learning Representations, 2024a.
Haoran Xu, Kenton Murray, Philipp Koehn, Hicu Hoang, Akiko Eriguchi, and Huda Khayrallah. X-alma: Plug & play modules and adaptive rejection for quality translation at scale. arXiv preprint arXiv:2410.03115 15, 2024b.
Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: pushing the boundaries of Ilm performance in machine translation. In Proceedings of the 41st International Conference on Machine Learning, pp. 55204-55222 2024c.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024a.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122 2024b.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09330 2025.
Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang, Haifeng Zhang, and Jun Wang. Teken-level direct preference optimization. In Proceedings of the 41st International Conference on Machine Learning, pp.
2024.
15

---

Published as a conference paper at ICLR 2026
Beichen Zhang, Kun Zhou, Xilin Wei, Xin Zhao, Jing Sha, Shijin Wang. and Ji-Rong Wen. Evaluat- petpuuuttttooiinnnniiin math reasoning. Advances in Neural
ing and improving tool-augmented Information Processing Systems, 011:23570-33585 2023.
Ran Zhang, Wei Zhao, and Steffen Eger. How good are Ilms for literary translation, really? literary translation evaluation with humans and llms. arXiv preprint arXiv:2410.1888 18697, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging Ilm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems. 36:46595-46623, 2023a.
Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin, Qin Liu, Yuhao Zhou, al. Secrets of rlhf in large language models part i: Ppo. arXiv preprint arXiv:2307.04964 2023b.
Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried. Graham Neubig, et al. Sotopia: Interactive evaluation for social intelligence in language agents. In The Twelfth International Conference on Learning Representations, 2024.
Lianghui Zhu, Xinggang Wang. and Xinlong Wang. Judgelm: Fine-tuned large language models are scalable judges. In The Thirteenth International Conference on Learning Representations, 2025.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint 19X.::1909.00 08593, 2019.
APPENDIX CONTENTS
Appendix Sections
Contents
Appendix A
Data Source and Dataset Construction
Appendix B
Additional Details of Experimental Settings
Appendix C
Extended Evaluation Experiments for ALPO
Appendix D
Theory: Adaptive Local Preference Optimization
Appendix E
Further Discussions ALPO
Appendix F
Prompts and Instructions in ALPO
A DATASET CONSTRUCTION
In this section, we will introduce the details related to the datasets used for model training.
A.1 DATA SOURCES
In this study, we utilize the MuSC dataset from the online video platform Youku for experiments, which includes source language subtitles and multilingual translated subtitles of the platform' 's programs. presents samples of English and Chinese subtitles from the MuSC dataset (in the ass file format), where the "Start" and "End" columns indicate the start and end times of the subtitle within the program. The translation directions covered in MuSC include en=de, en=fr, en= zh, ko zh, zh=en, and zh. These translation directions encompass both cross-language family and cross-language branch scenarios. Subtitle translation, as data-rich task, makes it casy for leading online video platforms to gather substantial amount of multilingual subtitle corpus. The MuSC dataset includes over 100 programs in each direction, spanning genres such as film, TV series, documentary, and animation over multiple years. Statistics for MuSC dataset provided in Table g
Unlike the translation of serious texts in fields such as legislation and medicine, which require a high degree of accuracy (AbuSa's aleek] 2016 Rashed Alkatheery 2023 Bajčíé & Golenko 2024), subtitle
16

---

Published as a conference paper at ICLR 2026
text in visual media programs is deeply bound to video and audio modal information, allowing for some tolerance of accuracy loss translation. However, it is crucial that the translated subtitle accurately conveys the emotions, tone, atmosphere, and cultural context of the original text. Therefore, human translation does not simply involve direct translation of the original subtitle but rather includes polishing, rewriting, and adaptation to meet the localization needs of the target audience (Bassnett) 2013; Pérez-González) (2019). As a result, the gold reference translations produced by human translators are refined versions of the original text, implying that the parallel corpus used for training is not truly "parallel" (the conclusion addressed in Section 3.2) and there is information asymmetry between the original text and its translation (see Table / for an example). Consequently, while
constructing the training set, each direction's 's program is native-language program.
Table 7: Examples of multilingual subtitles.
Start
End
Text
0:17:33.25
0:17:4..50
Please, 1et me speak!
0:17:39.54
0:17:42.83
As lower house, my voice doesn't carry much weight here.
0:17:42.91
0:17:46.45
mother, have voi that natters deeply. My son isn' = in his right mind.
But
0:17:47.50
0:17:50.25
0:17:51.04
0:17:53.95
His entire life, be's chased an impossible dream.
0:17:54.91
0:17:59:00
what he did was, wh, foolish and unwise.
0:17:59.83
0:18:04.20
But he has guod heart. Please, let him come home.
0:18:04.29
0:18:08.66
A crime Like this can't be overlooked. The boy must be punished. the Ethos calls for banistment, sympathize with young man's s dream to change the world.
0:18:08.75
0:18:12.16
violation
0:18:12.25
0:18:16.25
but I Perhaps in this matter, a lesser sentience may suffice.
0:18:16.83
0:18:20.58
0:17:42.91
0:17:46.50
& & 6 & / 4j Rx if f1 -f
0:17:47.00
0:17:50.25
0:17:51.04
0:17:53.95
+ #
#2
0:17:54.91
0:17:59.33
1
0:17:59.83
0:18:04.25
#
0:18:04.33
0:18:06.70
0:18:06.79
0:18:08.66 66
i3 13
0:18:08.75
0:18:12.20
0:1::12.9
0:18:16.25
12
0:18:16.31
0:18:20.58
88, A
Table 8: Statistics of the datasets. FM, TV, DO, and AN are abbreviations for film, TV series, documentary, and animation, respectively.
Statistic
=n
en Er
en
ko=zh
zh=en
zb=th
cross language family cross language branch
period
2013-2024
2015-2024
2020-2024
2017-2024
2021-2024
2021-2024
# programs
210
207
161
139
225
218
program types
FM,TVANN
FM,TV.AN
FM,TV,NN
TV
FM,TV,DO
FM,TVDD
# lines
1.87M
1.72M
1.02M
1.54M
1.26M
1.21M
total length (h)
1156
1033
852
802
501
479
# avg source token
7,82
7.67
7.77
9.87
6.19
6.08
# avg target token
10.83
10.64
6.41
6.51
8.17
21.11
A.2 BILINGUAL PARALLEL CORPUS CONSTRUCTION
Typically, the multilingual subtitles for the same program do not correspond line by line. This is mainly due to linguistic differences, which may lead to the merger or splitting of original line during human translation. Additionally, subtitle files may contain noise such as scene titles and annotations. To automate the line-by-line alignment of multilingual subtitles, we designed subtitle alignment algorithm that utilizes subtitle timing.
For the source language and target language line sets Cure and Ligs. first determine the row margin M = abs( L - Luul). Then, for each line laee in Lare, we search within a window size 2M around the corresponding index in Ligi for line whose start time differs from that of luee by within 0.7 seconds to serve as figr- The reason for choosing 0.7 seconds as the threshold is that the shortest sentence in various languages (such as "Hello") typically has duration of about 0.7 seconds. Through
17

---

Published as a conference paper at ICLR 2026
this process, can automatically gather a training dataset D fllares ligi) E Lise Ligi) from bilingual subtitles programs. The complexity of this process is (
B ADDITIONAL DETAILS OF EXPERIMENTAL SETTINGS
In this section, we introduce the main settings used in the experiments of this paper. Except for certain experiments with special hyperparameters, the same hyperparameters are adopted to ensure consistency and fairness in experimental comparisons.
B.
MAIN SETTING
We primarily employ the PyTore vllm and Transformer libraries to implement our approach, while utilizing DeepSpeed I for multi-GPU parallel training. All experiments conducted single machine equipped with eight A800 80GB SXM GPUs. The ALPO sampling phase for each translation direction takes approximately 1.5 hours, while the training phase requires about 2 hours. To ensure fairness and consistency in comparisons, we maintain identical irrelevant parameters across the main experiments, ablation studies, and extended experiments wherever possible. All critical hyperparameter settings are presented in Table
Table 9: Hyperparameter configuration in experiments.
Phase
Hyperparameter
Value
Remark
35
Number of lines in prompt r
optimizer
AdamW
SFT
learning rate
le-6
epoch
batch size
96
15
Sampling size
temperature
1.0
top k top p optimizer
40
Sampling generation hyperparameters
0.9
ALPO
AdamW
learning rate
le-6
epoch batch size
Segment-level batch size
We re-implemented the VideoDubber method and utilized paid APIs from Alibaba Cloud, DeepSeek, OpenAl, and Anthropic to obtain experimental and evaluation results for models such as Qwen, DeepSeek, GPT, and Claude. When eliciting translations from these models on the test set via ICL, we employed prompt in Appendix F.1 with a one-shot example to ensure structured output formatting. All LLMs used in our experiments were the latest versions of Sep 1, 2025, specifically: wwen-max-2025-01-2
ReepSeek-V3.. 40-244000 "d GPT-5 and Claude 4/4.
For translation quality evaluation, reserve 10% of each dataset's programs the test set. Due to constraints including paid API costs and inference time, we perform sampling- based evaluation rather than full-scale assessment. From these test programs, we randomly select 2,000 subtitle segments, each containing 35 lines of subtitles, thereby constructing a test set of approximately 70,000 sentences per direction. The final evaluation results represent the average scores across these 2,000 segments.
httess//oppnaa..commtt--5
18

---

The scoring Published as a conference paper at ICLR 2026
B.2 EXPERIMENTS IN EMPIRICAL INVESTIGATION
B.2.1 INVESTIGATION ON LLM-AS-A-JUDGE
In en en= zn and zh t. directions, we investigated the evaluation consistency of different LLMs Te (including DeepSeek-B GPT-5 Thinking, Claude Opus 4.1, and Qwen3-14B) compared to human evaluators multiple translations of lines. Specifically, for each direction, each evaluator scored 500 lines {8; i [n], 1 S < 500} across different translations on a scale of 0- These translations included human references, Google Translate, an SFT model Tan. and multiple LLMs (including DeepSeck-RR -11, GPT-5 Thinking. Qwen3-235B-ANe DeepSeck-V3, Claude Sonnet 4, and Qwen-Max). The SFT model Taa trained using the MuSC dataset. The evaluation LLMs assessed translations based on ICL,, and since lines are typically closely related their contextual environment, also provided the context lines of s; enable the LLM to reference the semantic environment of s;. The prompt is detailed in Appendix F.2 criteria are:
Thinking, GPT-4o,
Principle I (Accuracy): The translation must ensure basic accuracy (Weight: 30%);
Principle 2 (Colloquial Appropriateness): The translation must adhere to the expression habits of human colloquial speech (Weight: 30%);
Principle (Expressive Power): The translation needs to be expressive and vivid (Weight: 40%).
The term "weight" here serves to provide the model with an indicative reference for importance, rather than being utilized for explicit weighting operations.
For the en and zh=tn directions, the human evaluator group consisted of four evaluators (the same four individuals). while the en=de and zh directions each included two evaluators. All evalu- ators were undergraduate master's stu- dents specializing in English, German, or Thai translation, with Chinese as their na- tive language. The human scores were av- eraged across evaluators. In Figure [5] we verify the consistency among the four eval- uators for en zh and zh=en, confirm- ing the reliability of the human evaluation Figure 5: Human assessment consistency verification.
1.00
0.89
0.92
0.91
numan humar Humurz humer
1.00
0.92
0.91
0.91
22
0:90
0.89
0.90
1.00
0.93
1.00
8.90
1.00
1uman human
human human
human 1 human tuman human
(a)
(b) zh=te
results. For each evaluation LLM Te. we shuffled the order of translations four times and took the average across four scores. Each evaluator assigns a score sequence & to multiple translations of 8;- Since the scores from different evaluators often fall into different ranges within the 0-100 scale, we focus only the relative ranking of E rather than the absolute values. Therefore, we evaluate the Spearman rank correlation between different evaluators instead of using metrics such as ICC that sensitive absolute scores.
B.2.2 INVESTIGATION ON BACK-TRANSLATION CONSISTENCY
We investigated the back-translation consistency of parallel corpora from different domains. Specifi- cally, for translation direction, we utilized GPT-4o with a simple prompt in the following text box to translate the target language text back into the source language. We then calculated the BLEU and ChrF++ scores between the back-translated text and the original text. Lower scores indicate a higher degree of liberal translation. We evaluated 1000 sentence pairs per domain. The datasets from different domains involved in the evaluation include:
OpenSubtitles [Lison & Tiedemann 2016) is a collection of translated movie subtitles across 60 languages.
Books (Tiedemann) 2012) is collection of copyright free books in the literature domain.
bible-uedin (Chistodoullupoolls & Steedman 2015) is a multilingual parallel corpus
created from translations of the Bible.
19

---

Published as a conference paper at ICLR 2026
DGT (Steinberger al. 2013) is publicly accessible multilingual translation memory of the Acquis Communautare.
JRC-Acquis (Steinberger et al. 2006) is collection of legislative texts of the European
Union, comprising selected documents written from the 1950s to the present.
News-Commentary (Tiedemann) 2012) is parallel corpus in the news domain provided
by WMT.
ECDC (Steinberger et al. 2014) is translation memory from the European Centre for Disease Prevention and Control.
EMEA Tiedemann 2012) is parallel corpus made from documents of the European Medicines Agency.
The en=de back trccssranslttion prompt demonstration for GPT-4o.
Please literally translate the follewing English text inte German, maintaining the highest degree of literal correspondence without any localized rewriting:
Now the earth was formless and empty. Darkness was on the surface of the deep. God's Spirit was hovering over the surface of the waters.
B.3 HUMAN TRANSLATION QUALITY EVALUATION
We conducted human evaluation of ALPO translation quality in the en and zh=thh directions. For each direction, hired four evaluators, all of whom are undergraduate master's degree professionals specializing in English or Thai translation, and whose native language is Chinese. We assessed the ALPO model against four baselines (gold reference. SFT model Tih. GPT-4o, and DeepSeek-R1) across dimensions of accuracy, naturalness, and vividness, as well as through a comprehensive evaluation. The comprehensive evaluation instructions provided to the evaluators are detailed in Appendix F.3 with similar instructions for other dimensions. There were total of 32 evaluation tasks in the multidimensional assessment across both directions. Fach evaluator was assigned 4 evaluation tasks. In single evaluation task, we provided evaluators with challenger and competitor translations of 400 subtitle segments sourced from the MuSC test set, with each segment consisting of 20 subtitle lines. Evaluators were required to choose the better translation between the two options, or mark them as "no significant difference". The subtitle segments selected for different evaluation tasks were random subsets from the test set.
C EXTENDED EXPERIMENTS
In this section, we will present additional evaluation experiments for ALPO.
C.I FULL EVALUATION OF TRANSLATION QUALITY
We present the full results of the translation quality evaluation in Table 10 Our assessment of accuracy includes results from three LLM evaluators as well as traditional translation evaluation model XCOMET (Guerreiro et al. 2024). The results in Table I0 show that Google Translate achieved substantial performance translation directions within the same language family, such as en=de and en fr. However, its performance was poor for other languages, which might be due to the relatively simpler nature of translations between languages of the same language family. Traditional sequence-to-sequene architectures like MADLAD performed significantly worse than modern LLMs, and even VideoDubber, specifically designed for subtitle translation, did not demonstrate noticeably better performance than NLLB and MADLAD models. The comparison between ALPO and cutting-eds LLMs led to several clear conclusions:
Different LLM evaluators provide consistent ranking of translations by varying models across multiple dimensions, further validating the conclusions in Section 3.1
20

---

Published as a conference paper at ICLR 2026
Despite the receiving lower accuracy and naturalness scores, human translations excelled in the vividness dimension, which could indicate that human translators incorporate more interpretive translations based on the video content.
Reason models scored higher than chat models in vividness but lower in other dimensions, suggesting that reason models adhere more liberal translation instructions compared to chat models. This aligns with the conclusion in Section 3.3 as performing liberal translations can compromise accuracy to some extent.
C.2 TRANSLATION DEMONSTRATION
In Table 11 present the translation demonstrations for the en and zh=en directions including human translations, chat model GPT-4o, reason model GPT-5 Thinking, and our ALPO model. be intuitively observed that the translations by GPT-4o tend more towards literal translation and fail to convey the emotional ambiance and stylistic information of the original subtitle through liberal translation like the other translations do. The model trained with ALPO aligns better with the subtitle translation scenario. which includes:
The translations are more conversational, resembling human character dialogues rather than formal translations.
The wording and phrasing are more vivid, offering expressive articulation.
It does not excessively pursue accuracy, allowing for free translation based on context.
Furthermore, Table illustrates some limitations of LLM translations. LLM translation relies solely on subtitle text and cannot reference information from video context and scenes human translators do. For instance, LLM translates "nature" to á # whereas based on the video context, it should be translated to " ". This issue surpasses what preference optimization techniques can address. Therefore, we believe multimodal machine translation might be valuable direction for advancing subtitle translation research.
C.3 EXTENDED ABLATION STUDY
C.3.1 ALIGNMENT LOss
When simplified, ALPO can be viewed as framework compatible with various plug-and-ple preference optimization losses. Specifically, the ALPO loss can be expressed as:
Lalpo (T5; Teet) (((e.
Σ E((ss)= Cpo(ss)
where Eno can adopt multiple types of preference optimization losses, such as DPO (Rafailov et al. 2024), SimPO (Meng al. 2024), and GRPO (Shao et al. 2024). For the baved GRPO
loss. we compute as:
(8))
No
Pi)
(o( ps)
1 s,, A, - -TKKL(TTl|TTeed (8)
min
Aj, clip
T
Tean ( p:) Tree p.)
mE)
(L(TTolTTTe) Trot(t) (t. pi)
log Trer(t) { Pi)
Aj
1,
(9)
std(E,)
(t p.) To( pp))
where e and v are adjustable hyperparameters, and A, denotes the advantage estimated from the score sequence E,.
We compare the performance of ALPO with different preference alignment losses Epo in Table 12 The experimental results demonstrate that all loss variants achieve substantial improvements over the SFT model, validating ALPO's ' broad compatibility. The DPO loss exhibits stable and superior performance across multiple translation directions. While GRPO effectively utilizes all sampled outputs rather than focusing solely on preference pairs, it only achieves optimal performance the en zh direction.
21

---

Published as a conference paper at ICLR 2026
Table 10: Full quality evaluation results. The 1tt and 2nd best results are marked blue and orange. respectively.
Models
Training
Accuracy
Naturalness Chande
Vividuess Clande
XCOMET
DrepSork
Clande
GPTSS
Avg.
Drepborkk
GPT-5
Asg.
DrepSork
GPT-S
Avg.
Gol Rafertnce ValaDitt
Jaman
$1.4
$3.4
58.5
$5.0
B4.8
$3.6
$5.5
8122
83.6
6444
37.1
77.7
75.1
'661"
-553"
'CC"
-47.3
3.7
365.2
2T.6
"3575"
"42.2"
"47.1"
"50.7"
2
NLLB-3.22
MADI 10-
ST
68.9
70.3
70.0
13.5
67.3
88.1
71.4
66.4
73.2
75.8
68.4
72.2
71.1
75.2
63.0
57.2
65.6
70.1
58.2
55.2
60.4
55.7
58.9
52.6
54.5
59.2
Ghogle Thaalate
88.3
29.0
80.9
91.5
88.3
50.4
$1.7
70.8
80.6
60.9
64.2
62.8
62.6
912"
563
56.5
"6611
35.
35.7
37.5
56.7
36.2
71.3
"8.3"
86.5
Owen Max
(CL
91.7
97.8
97.9
94.7
95.8
59.6
57.8
08.2
58.0
74,0
74.7
68.9
V3.1
90.2
97.3
949
80.8
57.9
09.2
36.5
7422
67.4
544
0422"
"22.8
38.0
35777
33.5
87.4
61.1
74,9
74.
70.0
88.6
64.8
72.7
Qwen2.5-14
SFT
84.7
37.6
91.1
97.5
$3.9
50.6
83.6
56.0
68.2
64.4
Owen? 5-148
ALPO
91.9
98.0
97.0
94.8
95.4
$8.9
20.
87.1
88.4
68.3
78.0
78.1
74.8
Models
Training
Accuracy
Naturalness Chunle
Vividnes Chunde
XCOMET
DrepSeek
Chuude
GPT-5
Avg.
DregSerk
GPT-8
Avg.
DregSeck
GPT-8
Avg.
God Kelennnce
Hamnn
75.4
$4.4
29.0
34.1
B3.5
55.3
30.5
22.7
85.0
63.1
78,6
72.5
74.8
223
363"
367
-49.2"
3.5.5
37.4
47.3
-47.4
"31.4"
"A.
26..5
NLL.8-3.1
MADLAD-I18
ST
62.2
58.1
75.6
76.9
69.6
87.5
73.3
78.7
76.3
73.5
711.
75.0
77.0
71.4
61.7
70.8
74.6
67.8
61.4
55.7
63.3
59.4
57.1
60.7
61.8
57.4
Goople Transbate
$1.4
93.5
05.9
96.7
91.8
84.5
85.8
64.2
84.8
64.3
64.6
64.0
64.5
4P
86.4
97.7
96.3
92.6
93.2
59.5
90.3
56.1
80.8
39.0
74,9
74.7
69.5
Owen-Max
JCL (C)
86.1
97.5
97.3
95.6
94.1
90.4
90.4
50.0
89.9
61.1
76.8
76.8
71.6
DeepSeok-V V3.1
87.6
97.2
96.9
95.9
84.8
90.8
90.6
90.7
90.7
61.7
77.3
77.9
72.3
DespSock
ICL
87.1
96.7
95.1
93.8
82.2
90.4
90.0
75.8
73.4
Qwent 5.148
XPT
727
9006
M.7
86.2
355
36.8
33.6
85.4
37.4
71.7
7400
67.7
Qwen2
85.5
97.9
88.7
89.2
73.7
78.8
Models
Training
Accuracy
Naturaluess Chunle
Vividuess
XCOMET
DrepSeck
Chunde
GPT-S
Axg.
DeepSerk
GPTS
Ang.
DrepSeck
Chaude
GPL-S
Avg.
Gold Referanee
Jaman
79.0
84.3
38.5
$3.5
83.6
$1.5
$3.3
30.5
82.6
63.9
69.7
76.0
71.5
2/15"
1775"
3550
"4h.3"
35.
32.00
517'5
"30.7
"32.5
NLLBB 3.38
MADLAD 108
ST
619
53.5
65.2
61.0
70.4
79.5
44.7
48.2
59.7
61.4
58.3
59.5
56.8
65.5
45.6
42.7
54.0
55.5
47.4
46.3
45.2
46.7
29.5
44.8
43.7
48.3
Google Thansate
79.3
81.4
96.1
30.0
84.2
79.4
85.1
74.6
79.7
52.3
51.8
50.0
54.4
GPT-46
BET
92.1
92.5
$8.6
89.3
83.0
$4.2
79.6
82.3
36.3
57.7
65.3
59.6
Qwen Max
(C)
HK.
95.3
93.5
90.3
91.8
55.8
55.8
$1.6
34.4
19.1
61.7
63.1
61.3
DeepSeek V3.1
85.9
9566
93.6
59.5
91.2
$6.9
56.2
$2.8
05.3
19.5
61.
69.2
63.5
DeepSeek RI
GPT-S
(R)
MK4
869
9677
92.3
92.9
900
90.5
82.4
36.0
83,9
87.0
85.7
68.1
70.0
"A4.3"
74.3
76.6
71.1
"Dean 3.145
BP
RE4
3/3
900
3.4
35.5
355
55,5
820
37.
55.0
Qwen2 5-148
90.6
84.3
76.6
Models
Training
Accuracy
XCOMET
DrepSeck
Clande
GPT-5
Avg.
DeepSeek
Claude
GPT-
Asg.
DeepSeck
Claude
GPT-S
Ave.
Cevl Refernce
Haman
6K.
50.4
38.4
75.0
78.
19.0
74.7
77.8
65.6
58.4
73.5
65.8
35f
1733"
5668"
19.5
"J9.8"
380.2
28.5
4572"
"47.8
"560
"41.1
20.2
MADLAD 10B
SLLB 3:30
ST
30.9
37.2
47.6
39.5
06.1
40.1
21.8
28.5
33.1
44.8
44.7
30.3
29.5
57.3
26.8
18.5
26.1
42.9
27.5
44.4
30.6
55.6
40.2
17.9
48.7
25.4
Google Thuslute
42.4
50.6
76.9
49.6
54.9
49.5
61.8
47.2
52.8
46.3
57.3
52.4
52.0
NES
875
R65
76.
RFF
3.7
76.4
"38.5
"ST.8"
'6.00
Qwen-Mas
(C)
74.5
89.7
S88
$1.7
83.7
$3.4
53.7
S0.5
82.5
57.1
57.6
70.7
61.8
DeepSeek V3.1
73.5
91.9
$7.7
79.1
83.1
84.7
$2.2
79.8
82.2
55.6
49.6
66.4
57.2
GIFTSS
DeepSeck X1
(R)
263
71.1
93.5
36.0
56.4
58.7
75.9
79.5
84.5
79.8
85.0
$4.0
83.2
$2.3
79.6
78.6
82.6
81.6
63.1
64.0
60.0
59.1
72.5
72.7
65.6
65.0
Qwen22 3 140"
3PF
729
843
56.1
71.
60.3
785.5
76.7
31.2
76.1
35.5
47.0
61.0
51.6
Qwen2 5 14B
ALPO
7600
89.
89.8
60.
84.3
83.6
84.7
81.
83.3
67.6
67.5
7663
70.5
Models
Training
Naturalness
Visidmess Clande
XCOMET
DrepSork
Clande
GPT-5
DrepSork
GPT55
DrepSerk
GPT-5
Gol Relernnce
$7.5
63.0
$2.0
60.3
11.9
75.3
VideaDaliet
54.3
53.8
39.2
34.6
33.9
'4.66
50.1
NLLB-3.3B
30.5
36.5
12.7
29.1
26.4
12.5
21.7
26.4
11.5
20.8
MADLAD
49.5
33.3
45.7
30.2
38.9
43.5
31.2
37.8
Transtate
15.6
78.2
79.0
70.0
67.6
61.3
86.3
56.2
49.1
50.2
GPP44
35.2
66.5
Qwen Max
(C)
R64
93.2
92.6
$7.9
90.0
87.0
85.2
$2.7
85.0
63.7
67.2
68.5
66.8
DeepSeok V
8533
92.9
91.4
39.3
89.5
55.5
$4.9
$1.5
84.1
19.0
63.8
66.2
63.0
GPTS
(R)
81.
92.0
01.5
90.9
5006
88.5
88.55
69.1
87.3
356
350
86.8
M4
$4.3
86.1
85.6
60.5
72.7
71..5
75.3
19.7
77.5
75.2
75.5
Qwail 1.1-5
807
BC"
Bf5
"10.2"
3r8
310
75,5
33.5
11.5
'02.7
54.8
Qwen2 5-148
ALPO
85.1
90.9
59.8
82.6
88.3
86.1
87.1
87.2
86.8
79.2
83.0
62.8
B1.7
Models
Training
Accuracy
Naturalness Chande GPT-5
Vividness
XCOMET
DrepSeek Chaude
GPT-5
Avg.
DeepSeek
Avg.
DregSeek
Chande GPT-5
Avg.
NL5
Varafhnto
201
187"
1.57"
"34.1
365
"36.7
17.
54.3"
"45.7
"40.8
"11.0
25.5
NLL.B-1.38
50.2
33.9
40.5
MADLAD-108
45.6
51.4
25.7
47.9
54.7
33.0
50.8
52.7
51.0
Google Translte
52.5
50.1
53.2
53.6
49.2
56.2
53.5
57.6
52.5
54.5
4PT-40
90.5
90.7
88.0
85.1
$3.5
62.6
67.1
73.9
67.9
Qwen
(C)
95.1
91.
94.5
91.3
8539
85.5
05.0
63.5
73.6
08.1
DoopSock-V3.1
848
92.6
80.3
93.1
88.9
$5.0
$3.5
84.6
61.8
66.6
72.8
67.1
GPES
DeepSeek RI
(R)
R22
83.3
92.9
892
88.1
$8.1
90.5
90.8
80.7
87.8
83.5
85.1
$2.4
844
$4.1
$4.0
84.0
83.9
67.3
64.4
73/7
72.3
751
76.2
71.0
73.0
Qwan 5.146
SPC"
9075
87.3
3.76
3TA
8206
'50.5
76.
'72.2
B6.0
Owen2 O.142.5-14B 5-148
AL.PO
850
95.0
92.7
95.1
91.9
$4.3
85.3
84.4
84.7
89.0
75.4
78.1
74.2
22

---

Published as a conference paper at ICLR 2026
Table 11: Multiple translation demonstration of en and zh= High-quality translations and inadeuuate translations are marked in blue and orange, respectively.
Source
Tramiatio
founded verrongeciig nacea.
ES
MARGRRAEERT
RR
2625027
"he" Récane 12 the "aaree of eus"
world. race was rearly
Jest coyed by It
2- A
2822n=
4
4haAR4 BAR8
25221 Sag
AT90 S BARE
Doinoflora lord Roaket and Sate,
ENFFADFFFAL
rhyy hate exth othert
RKTA MAA45E324
Hio millie 1ife, hot chased imoosssble doear,
RRY RXT Hea
B 8-1FL2
& 18121 9-
R: 9 "6
ship
"EE TEA.ACEEE0E
clas.
2.4 #t #
$2. 1AA Rf-
& 2t 24
VE 8+ A C-1
slace entp
Aed:
Tave overboard but"" Xe Davaso hede saved
"EE 132FF2
FEREEAMAI
ATRRTS
AF F
AFA
"RRFFA 555 safe 06 "Frlaane to ohange, hut fortirate'y heve the capacity to clange ous
CKE: 22
Too made ne want see noeld more chan Just black
85:44 21
"M07 her Sa586" 1es86" tfar about
yoursell
ti
to teansform into butterfly. "A 18rg"as as oF praresred aadh other, ve would stay innorall...
and yours
You two ehtnk your better on, but reslly, youtine Just paig
Bay 50u" "Gpp. seam, yourre
Gothan lovea comsincc atoay.
118. SFe "WkaE"" FArRTer couscil. Peuple whhre
15 223
ON 28088288
& 1..6 1.6--88 21
had really atrong
Resuision That
As that susant,
serong tng
desinitlly
ing esot,
Uien, diis steong
At that sorenn,
had strong Runati
that
The iominoua frest
deives avay all
ingorities Bext
calning breath and
Freat's pare Hight,,
enersy.
Freatlight gathers, breath
Fresl illum,
the brocthy
A.4.8.5 *
YSRA Fang
important cecistone life that have Lo malle inseciiateyy wher. sisuatioo
have to ba zade
chese moonted
Ieportant
in life
nade precisnly in
crucial
moseted voments
Pany
raul. be made
nerants in
decistion
175 37500 Chis
20'5
OLÓ 1-
isn't the tize for you tvo to be Re3eyedenyy
Hov's probazly not time for comance.
Iela probably
the Line yui to balk abous 1696.
the right tine
stafe
just yet.
M. 73MR
you by fate. She doe't tain za Eoersstsin this world?
May juin
touetiiz this nos ldl
playchings of fate.
cruelty. Bhy not
Viocleo
foin sa to Rreschzen world?
of up
Beidi
pasns
and Luin Lhis world
jein fozous 2.12
upsids 1oyn 1rn"
2000y 1ha" feel Nace sunsbise anyeace.
pity"
MMGE Shane
this
(2.2 2.
sunlight, will
205 eqein.
suniight-- 11
15, egsin.
evez egsin...
sunshine
niget be unlucky whon cores to seatty
by ny luck
Don's sudge ms
Don't Judge as by my gasbling Julgeest of peopie in 25535
Just beckuse
lusky qubble
ter paozia sudeise charactars.
La precty sharp.
imsLincts
peoplee.
green amluen, blue
May TourExise acand floo
The vorid
green
30000-0000 di FT
to keep, you dafe,
Sha's facing backtasti for trying
And abe's teo you
She took all the
you, werzled sick,
ble to pooteot
verrying
She Endtared purnls
Loas appatise
ROOEN 7o protect you.
you.
appetlle out
She's buan siirrimd
and sha's
for protecting you,
44-2 2. 30
Ih sure ahe has vany erme and legs.
19a1 hard" heads and
Ri RA0 Hard Thiee"
Seace and eix arme
She rust be
scrathing.
"TT "ingele"
advancing stop where you are. The order rust obeyed.
halt army
your Set carp where you stand, Rc insoosstinne
inselictlly
hals your advance, carp issediately. Bismestiene forbidder.
no Riishetteeee allowed.
allowe.
blosming.
bleazing Mostel Buala.
biossing.
in Martal Seaiz.
23

---

Published as a conference paper at ICLR 2026
Table 12: Impact Cpo. The 1st and 2nd best results are denoted as blue and orange.
Method
Accuracy
enstde Naturalmess
Vividness
Accuracy
enee Naturalness
Vividness
Avcurucy
Nuturalness
Vividness
SFT
87.7
83.4
64.4
86.5
$2.1
59.2
85.2
80.1
54.9
DPO
95.2
88.3
74.8
90.6
84.2
76.6
88.3
86.8
81.6
ALPO
SimPO
93.2
87.4
73.9
90.7
83.9
74.3
86.2
84.2
80.9
GRPO
94.6
87.9
74.3
91.2
85.0
77.1
87.2
84,9
81.3
C.3.2 SAMPLING SIZE
The sampling size of ALPO directly affects the diversity of sampled translations. In Figure we examine impact of sampling size k on translation quality in the zh= direction. Results show that as & increases, translation quality improves across multiple dimensions, with the most significant gain observed in vividness, and stabilizes around & = 12. The computation of adaptive hyperparam- eters w(s;) and B3 during ALPO training depends on the number and quality variation of sampled translations. A larger sampling size directly improves the quality of train- ing translations and leads to more suitable w(;)) and 8, Since ALPO loss relies on the relative quality of chosen and rejected translations, performance stabilizes once the sampling size is sufficient. Based on these results, we set 15 all other experiments.
Score
Accuracy Vividness
Figure 6: Impact of sampling size on translation quality.
C.3.3 MODEL SIZE
In Figure 7 investigate the impact of ALPO's backbone model size translation quality. Using the Qwen2.5 series models ranging from 1.5B to 72B ALPO's ' base models, we evaluate translation performance in both en and zh=en directions. Experimental results demonstrate that as model size increases, scores across multiple dimensions consistently improve, though the growth rate gradually diminishes. The 14B or 32B model generally achieves performance comparable to DeepSeck-RI, further validating ALPO's significance in enhancing translation quality. ALPO employs fully autonomous approach to preference labeling of alignment data, enabling cost-effective training of high-quality subtitle translation models. Overall, the 14B base model strikes balanced trade-off between cost and performance. However, as subtitle translation is an offline task, larger-scale models may be adopted when pursuing SOTA performance.
onweh Nat
en Vivi
zh ren Net
05
Zhwen Vivi
DeepSieek
DeepSock R1 onwzh Not
DeepSieek enwech
DecpSock shwen
DeepSeek R1 ahreen Nat
DeepScck zhwee
148
328
/iB
Model Size
Figure Impact of model size on translation quality.
24

---

Published as a conference paper at ICLR 2026
C.3.4 DATA VOLUME
We present the translation quality scores across training steps in Figure B investigate the impact of training data scale. The results demonstrate that as training data volume increases, model accuracy initially rises before declining. naturalness shows modest improvement, while vividness exhibits substantial yet gradually decelerating growth. Although enhanced training data yields higher vividness scores, we observe pronounced hallucination phenomena in the model outputs, which explains the accuracy deterioration. Comprehensive analysis indicates that training for 75 steps (corresponding to 7,000 prompts with a batch size of 96) achieves optimal balance, which serves as the default configuration for all our experiments.
03-
en-2h Acc
en-sch
en-2h
dhren Acc
zhrwen
Training Step
Figure 8: Impact of training data volume on translation quality.
C.4 FURTHER EXPLORATION
C.4.1 VANILLA DPO TRAINING
Training expressive subtitle translation LLM preference optimization problem, want to explore whether vanilla DPO can directly solve this issue. Motivated by this inquiry, we designed and conducted relevant experiments. ALPO achieves significant improvements in translation quality through its segment-wise sampling strategy and fine-grained alignment loss, and we aim to verify the impact of these two components. Following the standard DPO training protocol (Ratailov et al.) 2024), we perform post-training on the SFT model тah. Specifically, we employ either coarse-grained or lie-grained sampling to generate a chosen response y( and rejected response ( for each sample alpo in the alignment dataset, and subsequently optimize the policy model using the standard DPO loss.
The coarse-grained and fine- fine-grained sampling procedures are illustrated in Algorithm 2 and Algo- rithm 3 In the coarse-grained sampling approach, we directly sample k complete responses for given prompt x. Subsequently, a Sowe2-144-Innttreee across these responses. The summation of scores for subtitle lines within each response is calcu- lated, with the highest-scoring response selected as the chosen response y( and the lowest-scoring as the rejected response ()) For fine- rree-graiedd sampling, we perform two separate segment- wise sampling iterations similar to Algorithm for each prompt During these iterations, the sampling of subsequent subtitle lines employs either the highest-scored or lowest-scored existing line as the prefix, ultimately yiclding the segment-we wise sampled chosen response y/() and rejected response (t))
model is utilized to score all subtitle lines
C.4.2 ADVANTAGE-BASE PPO TRAINING
In the literature, is commonly acknowledged that in preference optimization techniques, RLHF (primarily based PPO), despite its intricate procedures, outperforms DPO (Ouyang al. 2022 [Bnder et al. 202T). This motivates our investigation into whether PPO can achieve superior performance to ALPO. To address this, designed and conducted relevant experiments. We adopt
25

---

Published as a conference paper at ICLR 2026
Algorithm 2 Coarse-grained Sampling for Vanilla DPO.
Input: SFT model sst. evaluation LLM Te. alignment dataset alpo sample number k. Output: sampled response pairs set S(z).
for any x E Dalpo do // Iterate through the alignment dataset Daloo
2: for iori=1 to do / Sample multiple candidate responses.
3: Sample ( a:).
4: end for
5: Measure the sum of मe score for each line y in the candidate set {v'1é 1, .,2,....k)
6: Select chosen y( and rejected y
7: end for
8: return S(x) = (
Algorithm 3 Fine-gratd Fine-grained Sampling for Vanilla DPO.
Input: SFT model stt evaluation LLM Te. alignment dataset alpo sample number k.
Output: sampled response pairs set S(z).
for any x E Dalpo do // Iterate through the alignment dataset Dalpo for = n do
2:
// The first sampling cycle used to obtain y
3:
for j 1 to & do Sample (t. (t: xx 1 end for
4:
(c)
5:
6:
Deduplicate the candidate set {t/ j 1.2...... k.. 7: Add human reference E! to the candidate set, get {t! j 0, J.
8: Measure {t/ j = 0, 1....), by Te. get score sequence E
9: Select a chosen translation 1.9 (random from top 3 of &,). end for
11:
.n. for i 1 to do // The second sampling cycle used obtain (
Concatenate { = 1, .., yields (
12:
13:
for j = 1 to & do Sample (t. (0) (21). end for Deduplicate the candidate set {t] j j 1,2.......
14:
15:
16:
Add human reference eQ to the candidate set, get {t] j 0, 1.... Measure {t/ j 0, .....) by Te. get score sequence E,. Select a rejected translation & (random from bottom of &,). 20: end for
17:
18:
21: Concatenate {1 = 1,2, .. yields (
22:
end for
23: return S(x) = (0))
an advantage-based PPO training pipeline (Zheng et al. 2023b). Specifically, we implement the PPO training process through the following steps:
1. Rollout Use Algorithm 4 to sample trajectory for each input Datp.
2. Computing Returns and Advantages - Perform Generalized Advantage Estimation (GAE)
using the value network Vo (base model: Qwen2.5-7BBInstteeee (Yang et al. 2024a))
8, E -Vo6)))))v
n-i-1
(10)
A, =
Σ
(72))6441- 6:+--
1=
26

---

Published as a conference paper at ICLR 2026
3. Updating Policy Network Let Told denote the fixed old policy during sampling. Minimize the clipped loss function (KL divergence constraint between To and Vold omitted below):
(t, Ps) A, clip p4)
(To p4) Tota p.)
Letip (0) Dapo
min
c. c,1+e) A,
Tota(t
(11)
4. Updating Value Network mean squared enror.
Fit (p(p) to the GAE-based estimate 1i = A, + (p()) via
T ((vo()) - f3
Lv (6) Bgp
(12)
5. Repeat for Multiple Epochs policy networks until convergence.
Continuously collect new data and update the value and
Algorithm Sampling for PPO solution.
Input: SFT model Tat., evaluation LLM Te. alignment dataset Dippo Output: sampled trajectory set S(x) = {T).
1: for any z E Dalpo do // Iterate through the DA dataset Dalpo-
2: for io 1 n do // Iterate through the subtitle lines in æ. Sample (t, 1.,,,...., (;-1)
3:
4: Measure i, by Te. get score E..
5: end for
6: Obtain trajectory {(v,, Li E,) | 1-= 1,,, ... , ,n].
7:
end for
8: return S(x) = {T}.
C.4.3 EVALUATION
We evaluated the performance en and zh-ten translation under "Vanilla DPO Training" and Advantage-basse PPO Training" configurations using the Qwen2.5-147 model. In addition, we also compared two token-level DPO-based methods, TDPO (Zeng et al.) 2024) and TIS TIS-DPO (Liu et al.) 2025). Experimental results presented in Table 13 demonstrate that while DPO and PPO training with segment-level sampling strategy achieved moderate performance improvements compared the SFT model, they still exhibited significant gaps compared to ALPO with fine tne-grained alignment loss. ALPO also significantly outperformed the two token-level methods. This validates the effectiveness of both segment- wise sampling strategy and fine-grained alignment loss for the local preference optimization task.
Table 13: Experimental evaluation results of alternative solutions. C for Soarse-grained while F for Fine- -raiggraindd The 1st and 2nd best results are denoted as blue and orange.
Model
Train
Sampling
en zh
zh=een Naturalness
Accuracy
Naturalness
Vividness
Accuracy
Vividness
Gold Reference
83.6
82.9
71.6
82.9
80.2
73.2
GPT-4o
89.5
82.4
59.5
88.6
82.9
64.6
DeepSeck-RI
90.4
85,6
70.7
88.5
85.6
73.6
SFT
86.3
82.1
39.2"
"83.2
80.1
34.9
87.1
63.3
81.1
60.2
DPO
88.2
82.5
69.4
86.3
83.1
68.4
Qwen2.5-14B
PPO
88.3
83,2
70.1
85.9
82.3
70.1
TDPO
87.4
82.7
64.0
86.0
81.3
61.2
TIS-DPO
83,3
ALPO
90.6
76.6
88,3
86.8
81.6
C.5 FURTHER APPLICATION OF ALPO IN MULTI-TURN INTERACTION
C.5.1
OVERVIEW
To validate the generality of the ALPO method, we conducted experiments on another local preference optimization task: social language agent multi-turn interaction (Zhou al. 2024 Wang al. 2024a
27

---

Published as a conference paper at ICLR 2026
et al. 2024 Kong et al. 2025). This task requires LLM agents dynamically adjust generation strategies through multi-turn interactions in an interactive environment, guided by character profiles, contextual scenarios, and private social objectives. The objective is to enable the agent to more effectively accomplish predefined goals (e.g.. persuasion, agreement attainment) within specific social scenarios (e.g., negotiation, collaboration, competition), while simultaneously maintaining improving the relationship between both dialogue parties.
C.5.2 METHOD
For the multi-turn interaction task, we primarily complete preference alignment data sampling through Two steps: 1) Error Location; 2) Preference Data Sampling. For multi-tum dialogue in the training set, we first employ GPT-4o to identify the starting position of erroneous turns, following the same criteria Kong al. (Kong al., y025): 1) The turn is critical to achieving the role's 's goal: 2) There remains room for improvement in the relationship between goal completion and the role.
If both criteria are satisfied, we take the dialogue history preceding this turn as the initial sampling prefix P1, then utilize our LLM agent to sequentially sample 5 single-turn responses for each turn. During the i-th sampling iteration, we employ GPT-4o to determine whether the current turn qualifies as crucial based on the sampled response: responses containing primarily pleasantries or similar content are deemed non-crucial, with their indicator function (p:)) set to O; otherwise, it is set to 1. We then select the response with the highest combined goal and relationship score the preferred response y," and the lowest as the dispreferred response y!. with goal completion prioritized over relationship. The prefix p; for the i-th iteration becomes p,-1, 9 (including interlocutor behaviors, omitted here). Finally, we apply the following simplified ALPO loss function for alignment training:
Calpo (Ta) Tree) = -E EE pr., wh...) IDD
Σ (p.) log (olog
To(y T ps). ( p.) TTee (v² ps))
B log
Toly! Pi)
(13)
where ID denotes the dataset, and both my and Tre are initialized from behavioral cloning (BC) models.
C.5.3 EXPERIMENTS
We utilize 100 out the 410 scenarios from SOTOPIA-T (Wang et al., 2024a) for behavioral cloning (10 role pairs per scenario) and 310 scenarios for alignment (8 role pairs per scenario). We adopt SOTOPIA (Zhou et al. 2024) as the test set, containing 90 scenarios with role pairs per scenario, totaling 450 self-chat tasks and 900 non-self-chat tasks. We evaluate both goal and relationship dimensions on the same baselines, with experimental results presented in Table [4] Baseline results are sourced from Kong et al. (Kong et al. 2023).
Table 14: The performance of various methods on SOTOPIA across the goal and relationship dimensions. The 1st and 2nd best results are denoted as blue and orange. respectively.
Self-Chat
GPT-4o
Methods
Goal
Rel
Goal
Rel
GPI-3.5-turbo
6.38
1.36
7.19
2.05
GPT-4o-mini
6.98
2.11
7.44
2.36
GPT-4-turbo
8.18
2.96
7.92
2.79
GPT-4o
7.90
2.67
7.90
2.67
LLaMA-8B
7.24
7.94
7.70
2.49
LLaMA-SB+BC
7.81
3.05
7.53
2.78
LLMMA-SB--C-PPreeeeedd
7.76
3.05
7.65
2.88
LLAMA-BB-BC+DPO RRfailovv al.
2024
7.95
3.28
7.80
2.97
LLMMA-BB-BC+ETO (Song et al. 2024)
8.29
3.39
8.02
3.03
LLAMA-BB-BC+DMPO (Sh al. 2024
8.28
3.37
8.00
2.98
LLaMA-8B--BC+SDDO KKng et al
2025
8.56
3.69
8.13
3.16
LLAMA-SB-BC+ALPP
8.41
3.56
8.21
3.19
The experimental results demonstrate the effectiveness ALPO on other application tasks. ALPO employs progressive optimization strategy transitioning from local to global optima, while utilizing
28

---

Published as a conference paper at ICLR 2026
an indicator function to adaptively determine the participation of each segment in the loss computation. This approach enables effective handling of multi-ssgmment local optimization tasks.
D THEORY: ADAPTIVE LOCAL PREFERENCE OPTIMIZATION
In this section, we formalize the local preference optimization problem and validate the effectiveness of ALPO and the limitations of general preference optimization approaches. Note that for the sake of expository convenience, the notations used in this section may differ from those in previous sections.
D. LOCAL PREFERENCE OPTIMIZATION PROBLEM
For given input X' (A denotes the input space) the language model ma. assumee it consists of n interrelated segments, i.c., x = (z1, zn). Correspondingly, its output y E y ()' being the output space) is also composed of interrelated segments, i.e., y ("1,'.. ,,), where x; and yi exhibit a one-to-one correspondence. Additionally, the generation of y; is influenced by y1," 9i-1 (note that this influence arises only from the autoregressive nature of language model but also potentially from semantic dependencies between output segments, among other factors), i.e.:
To
σ =
Te( 2, 2, ... .86-1).
(14)
Consider segment-level preference metrie r(x,, Me) (a reward signal other quantitative measure) that evaluates how well the segment output aligns with the optimization objective for its corre- sponding segment input 2,. The goal of the local preference optimization problem is adjust the parameters 0 such that To is optimized to maximize rxvs, y:). This objective is formally expressed as:
l" arg max Expp(x) Byy (1p))
Σ ras w]]]
(15)
8
where p(z) represents the true distribution of the input
x.
D.2 INEFFECTIVENESS OF OUTCOME-SSPERRISSE SUPERVISED PREFERENCE OPTIMIZATION METHODS
We take DPO Rafailov al. 2024 as example illustrate the limitations of general preference optimization methods when addressing local preference optimization problems. When performing local preference optimization, DPO first annotates preferred responses yd (y²" .... . and
non trefrfrfered responses y' = (v/ v/,) corresponding to x (Iı, II)) based on r( V:). These responses inherently satisfy:
(y (x)=
To (v(" xr, v.. ... vY-1), Toly' (x) = II no(y! | x. v/-1). (16)
Then, DPO optimizes my through a single-stage policy optimization approach:
log
To x)) B log T x)) Thet (y
Toly" xx)
Coro Trer) Ex,yy wf -
Bog
(17)
x)
where Tyet denotes frozen reference model. For the contrastive terms in the loss function, compute:
To ( x)) To (v' r) Trer Ty x) Trer
To (v."
x, v.l log
To xr, vi.
log
log (V1"
log
(18)
x))
(1)"
Teer
x vi
1=1
where y
1.-h and vi.e-1 (vf. ... . The limitations of this approach are:
The generation probability of the i-th segment is conditioned on its own preferred or dispreferred prefix (ol or vi J). rather than shared prefix. If y and y exhibit significant divergence in their prefixes (c.g., differing styles or topic branches), the contrast at the i-th segment ceases to be fair comparison under identical conditions.
Lack Fine-Grained Contrast on Aligned Prefixes:
29

---

Published as a conference paper at ICLR 2026
Inability to Apply Training Signals Per Segment: DPO performs holistic preference judgment over complete sequences (v""./)). depriving the model of segment-leve corrective guidance. Specifically, while the model recognizes the global superiority of y" over y', it lacks localized feedhack to optimize decisions at individual segments.
Gradient Dilution and Noise: When the divergence between y" and y² occurs only at specific segments, the global contrast aggregates gradients from numerous irrelevant segments (nearly identical in generation). This drowns critical alignment signals, hindering effective identification and correction of misaligned segments.
In summary, DPO underperforms in local preference optimization tasks due to misaligned generative conditions caused by prefix divergence and the absence of stepwise "correct vs. incorrect" decision learning (a limitation shared by other outcome- supeevise preference optimization losses).
D.3 ADAPTIVE LOCAL PREFERENCE OPTIMIZATION
Unlike vanilla DPO, ALPO labels the preferred response yt and dispreferred response yf for each segment using - as the prefix, instead of generating the entire response sequence. During training, the prefix for the i-th segment is fixed as p, = (z. 91. 0 where Oi is obtained through the scheduled prefix mixing strategy. This ensures that comparisons for each segment made under the same preferred prefix, eliminating unfair competition between "preferred prefix vs. dispreferred prefix". Under the same prefix, the preference alignment loss contrasts To To( ps) and Toly) p4) conditioned p:- Finally, the summation of degment-wiss. losses imposes preference constraints at each segment:
n
(o!" ps) Tiet ( pt) Trer(yd Ps)
To(y! Pi)
Calpo (To; Tret) (a oi JDD
w(p)) log
(B; log
B, log
(19)
where w(p;) and 3; are adaptive hyperparameters that determine the contribution of each segment to the optimization process based specific criteria.
The reason for using scheduled prefix mixing to obtain 01:-1 as the prefix is that, as optimization proceeds, the model tends generate y!", making the post-training distribution more consistent with the - path. Since - is also the prefix path most likely to be followed, this ensures that the model learns to generate the preferred segment y/"" rather than oa under the prefixes it visits most frequently. This aligns with a common principle in reinforcement learning and preference optimization: the state (or prefix) distribution during training should match the distribution most likely be visited by the final policy (on-policy) (Christiano al. 2017 Sutton 2018 2022 Formally, during training we aim to minimize:
Ouyang et al.
Ex--do Lalio p))]
(20)
where dg- denotes the prefix distribution induced by the final model. Since dg- is likely include preferred prefixes upon convergence, selecting 91:1-1 as the prefix essentially samples (or directly uses) states from the preferred path, ensuring consistency with the final distribution dg-
E DISCUSSION
In this section, we provide further discussion on the ALPO method.
E. CONCLUSION
The visual media industry, serving as pivotal medium for human cultural exchange and dissemina- tion, is witnessing automation and industrialization as critical future trends. Against this backdrop, we propose ALPO, novel training paradigm for translation models, by employing techniques including LLM-as---Jude and preference alignment. Through comprehensive experimentation and theoretical analysis, we validate the effectiveness of ALPO. We believe this study will not only positively contribute to the advancement of visual media technologies but also promote research on domain-specific translation models in other fields.
30

---

Published as a conference paper at ICLR 2026
E.2 RECOMMENDATIONS
We would like to offer some developer suggestions for technicians using ALPO:
When employing LLMs for preference annotation, shuffle the order of translations across multiple inference passes and average the results to obtain robust evaluation outcomes.
The training prompt should utilize the language best suited to the selected backbone, e.g., Chinese prompts for Qwen-series models and English prompts for LLaMA-series models.
Our open-source implementation reveals additional processing details, including handling of terminology translation, subtitle segmentation, and semantic integrity preservation. Refer to the source code for specific implementations.
Subtitle texts are inherently multimodal content closely intertwined with the visual and auditory elements of media programs. Videos provide supplementary information about characters, scenes, and plot progression, while audio conveys emotional and tonal cues. Future research should therefore focus on harnessing these multimodal data resources to enhance subtitle translation, e.g., leveraging video and audio modalities enable speaker annotation for subtitle lines, and utilizing audiovisual context to facilitate subtitle translation.
E.3 LIMITATIONS
ALPO has the following limitations:
The visual media data and associated copyrights required for subtitle translation are predom- inantly concentrated within large enterprises, creating data accessibility barriers.
Similar to other SOTA LLMs, ALPO lacks the capability to integrate multimodal information (e.g., video) during the translation process.
F PROMPTS AND INSTRUCTIONS
In this section, we present the input and output formats used for LLM and human evaluators.
F.1 INPUT AND OUTPUT OF TRANSLATION LLM
We illustrate the prompt and response format of the zh- translation model below (similar formats apply other languages). The red text appears only in LLM ICL. The original and translated subtitles of the program are formatted as shown in the text boxes for use in SFT and ALPO model training, as well as other baseline LLM ICL.
The zh=en translation prompt demonstration of translation model.
<INSTRUCTION>
Please translate the following multiple Chinese movie/TV lines into English according to the following requitements:
1. The English translation should be colloquial, easily understandable, and that the language style is consistent with the Chinese lines.
2. The English translation must be expressive and vivid, effectively conveying the atmosphere, emotions, and tone of the original Chinese lines.
3. Output the translation with its line number, ensuring that number lines in the translation matches the number of lines in the Chinese original, without merging any lines.
<EXAMPLE> Original text: 1. -
2.
31

---

Published as a conference paper at ICLR 2026
34. # X H
35. 5) f (
Translaticn results:
1.A1low us moment of eivility. 2.We must talk.
The Arcane stirs within you. 35. away from the Hexgates.
<TASK> According to the requirements, complete the following translation.
Original text:
2 #
34. A B1 #
35. 35 $
Translaticn results:
The zh=en translation response demonstration of SFT model.
1.I heard your prognosis.
2.1'm terribly sorry, boy.
...
34.It's sad truth that
35.those who shine brightest often burn fastest.
F.2 INPUT OF EVALUATION LLM
We present the prompt format for LLM evaluation of vividness in zh=>en translation below (similar formats apply to other languages). When using LLM to assess the vividness of a line, we provide the contextual lines surrounding it.
The zh=en prompt demonstration of evaluation LLM.
<INSTRUCTION>
Please assign vividness score (0-100, integer) to multiple English translations of a Chinese subtitle line. The original Chinese line and its surrounding context will be provided for reference, marked with [Тo be evaluated] and [Context] respectively, The scoring principles for translation vividness are as follows: moderately liberal
Principle (Accuracy): The translation but must maintain reasonable accuracy conveying the criginal meaning without additions or omissions (weight:308);
Principle 2 (Colloquial Appropriateness): translation uses natural spoken language suitable for the subtitle context and effectively conveys the original emotion, atmosphere, and tone (Neight:308);
Evaluate whether the
Principle (Exprcssive Power): Assess whether the translation is emotionally resonant, employs vivid phrasing, and has literary qualities that engage the audience (Neight:408).
Scoring Criteria (Vividness):
109: Exceptional11 expressive and impact translation with rich emotional layers that deeply resonate with the audience. 50: Moderately expressive translation with subdued emoticnal delivery that partially conveys the intended feelings.
0: Inaccurate translation lacking emotional depth or expressive
32

---

Published as a conference paper at ICLR 2026
qualities, failing to evoke any audience connection.
<EXAMPLE>
Chinese original text: [Context] * TRENAS
[Context] * [To be evaluated] / ARAARZAН [Context] ] & * (T
[Context] $. 4 ".. 7
English translation:
Translation A: History repeats, but we can't t go back to what was.
Translaticn B; History often echoes, yet there's a no way for us to turn back the clock.
Evaluation score: ("A": 70, "B": 92, "c"; 85, ....
<TASK>
According to the criterion, complete the following evaluation. Chinese original text: [Context] # #
[Context] $ [To be evaluated] $ [Context] # * # [Context] $ T
MARKNNR
English translation:
Translaticn A: It's a sad truth that those who shine brightest often burn fastest.
Translation B: The heartbreaking truth is that those who shine the brightest tend to burn through their life's s energy all the faster.
Note, you need to output the ratings JSON format: ("A": score, "B": score, "C"; score, ...) Evaluation score (only output the rating, no other content):
F.3 INSTRUCTION AND PROMPT FOR QUALITY EVALUATION
For human evaluation of translation quality, it is essential provide evaluators with clear instructions specifying the evaluation perspectives, criteria, and format. These instructions directly influence the focus and emphasis of evaluators during the quality assessment process. The instructions provided to evaluators are shown in the first text box, while the prompts used for multidimensional automated evaluation with LLMs are presented in the three subsequent boxes.
Instruction for human evaluation of translation quality.
[Evaluation Criteria]
1. Accuracy when evaluateing the accuracy of audiovisus subtitle tle translation, consider the tollowing dimensions: Semantic Equivalence: Evaluate whether the meaning of the original subtitle accurately conveyed the translated version, and if the semantic content of the source subtitle is precisely
33

---

Published as a conference paper at ICLR 2026
expressed in the target subtitle.
Grammatical Correctness: Evaluate the grammatical accuracy the translated subtitle, including sentence structure, tense, voice, and other grammatical aspects.
Terminology Translation: Evaluate whether proper nouns have been accurately translated, maintaining the semantics and centext of the original terms.
2. Naturalness
When evaluateing the naturalness of audiovisual subtitle translation, consider the following dimensions:
:Coherence: Evaluate whether the translated subtitle reads if written by a native speaker of the target language, and evaluate the logical relationships between sentences.
.Readability: Evaluate whether the translated subtitle is easy to read and understand, and if the word choice and expressicns conform the conventions of the target language.
.Fluency: Evaluate whether the translated subtitle flows smoothly, if sentences are vell-censtructed, and if there are any obvious grammatical errors or unnatural expressions.
3. Vividness
When evaluateing the vividness of audiovisual subtitle translation, consider the following dimensions:
Sttylistcc Consistency: Evaluate whether the translated subtitle maintaing the style and Characteristics of the original, including ng consistency
character tone and emotional nuances. *Expressiveness: Evaluate whether the translaticn conveys the essence and atmosphere of the original lines, avoiding mechanical literal translation, thereby making it easier for the audience to understand and find engaging.
Emotion: Assess whether the translation faithfully conveys character's s emotions, aligns with the scene and character centext, and resonates emotionally with the target language audience.
[Task]
For each set of original subtitles, two different translations (A and B) are provided. Please refer to the multiple evaluation dimension speci in the [kvaluation Criteria] to evaluate the two translation
for each set original subtitles. Indicate and в by marking [A is
your evalua vvaluation results for translations betterl, [B is better), or [No significant difference between A and B]. Note that you only need to evaluate the overall performance of each of subtitles, not each individual line of subtitle.
The prompt of en subtitle translation accuracy evaluation.
[Knglish to Chinese Subtitle Translation Accuracy Evaluate Wallitiio]
Please rate the accuracy of the following English to Chinese subtitle translation, using integer scores from 0 to 100. The translation accuracy rating criteria include evaluating whether the Chinese translation accurately conveys the original meaning of the English subtitle. Additionally, pay attention to whether the torminology (c.g., names of people, places, organizations, items, etc.). in the English original subtitle is accurately translated in the Chinese tranalation.
Scoring Criteria (Accuracy):
100: The translation is completely accurate, covers all information, provides coherent translation, and the proper are accurately translated.
50: The translation is mostly accurate, with only minor cmissions
34

---

Published as a conference paper at ICLR 2026
unolear context, and proper nouns partially inaccurately translated, but the overall meaning is still understandable. 0: The translation is severely distorted, misinterprets the main meaning of the original text, and the translation of proper nouns is poor.
Original English Line:
crime like this can't be overlooked. The boy must be punished. A violation of the Ethos calls for banishment, but I can sympathize with young man's dream to change the world.
Yeah, must admit, his tneery intrigues. If dangerous ideas didn't excite the imagination, we would never wander astray.
Chinese Translation: A # H
16
need to output JSON fermat: ("Score": evaluation
score)
Score
(only output the score, no further explanation required):
The prompt of en subtitle translation naturalness evaluation.
[English to Chinese Subtitle Translation Naturalness Evaluation]
Please rate the naturalness of the following English Chinese subtitle translation using an integer score from to 100. The naturalness score of the subtitle translation should consider whether the translated text adequstely takes into account contextual factors, including cultural background and context, and whether it ensures natural and fluent language expressicn that conforms to Chinese grammatical structures and word usage habits. It should be easy to undcrstand and close to the culture and expression habits of the Chinese audience.
Scoring Criteria (Naturalness):
100: The translation fully considers context and cultural background, with smooth and natural language that aligns with Chinese usage habits, and contains no grammatical errors.
lexical
50: The translation considers the context and is basically fluent, but some expressions may be slightly awkward or unnatural, with potential minor gramnatical errors.
0: The translation does not effectively consider the context culture, is not fluent, has many grammatical errors, rigid, and difficult to understand.
Original English Line:
boy must be punished. violation of the Ethos calls for banishment,
crime like this can't be overlcoked.
but can sympathize with young man's s dream to change the world.
Yeah, must admit, his theory intriques. If dangerous ideas didn't excite the imagination, we would never wander astray.
35

---

Published as a conference paper at ICLR 2026
Chinese Translation: i A
ビ / # -
You need to output JSON format: ("Score":
evaluation
score)
Score (only output the score, no further explanation required):
The prompt of en= subtitle translation vividness evaluation.
[English to Chinese Subtitle Translatien Vividness Evaluatien]
Please rate the vividness of the inglish to Chinese subtitle itle translation below, using integer score from 0 to 100. The score for translation vividness does not take into account the accuracy of the translation; it only evaluates whether the translation is expressive, emotionally rich, and more capable of engaging the audience.
Scoring Criteria (iivideess::
100: The translation is highly expressive and Impactful, with rich emotions, capable of strongly moving the audience. 50: The translation has some expressiveness, and the ematicnal expression is relatively flat but still conveys some emction. 0: The translation lacks expressiveness and emotion, failing to evoke any emotional resonance from the audience.
Original English Line:
boy must be punished. A violaticn of the Ethos calls for banishment, but I can sympathize with young man's s dream to change the world.
crime like this can't be overlcoked.
Yeah, must admit, his theory intriques. dangerous ideas didn't excite the imagination, we would never wander astray.
Chinese Translation:
a P & f
1 3
Note: You need to output in JSON format: ("Score"; evaluation score)
Score (only output the score, no further explanation required): :
36


Reviews:
[{"review_id": "l8hsweEwtc", "reviewer": "Reviewer_Wkwm", "summary": "This paper looks at translating subtitles for visual media. They introduce a novel Preference Optimization method (ALPO) that looks at local outputs instead of the full ones. Overall, the paper covers a lot of things in depth, but I have two main questions (in question section below) that I would like to see answered that will really impact my review scores.", "questions": "Normally, PO tasks are done on full sequence outputs which is often touted as a benefit. I guess you could have preferences for partial outputs and that is not a problem, but it seems that in many ways, that is more similar to SFT with a slightly different loss - since you just update at every token autoregressively. Am I understanding this correctly? Regardless, this is a semantic definition that really does not impact the paper too much. My main question is how would a sequence level method (DPO, CPO, KTO, SimPO, etc. - choose one) compare to your method? Essentially, I am asking for another row in Table 3 with one of these.\n\nYour first claim at the end of your introduction states \"We introduce the visual media subtitle translation task for the first time...\" It isn't clear to me that this is the first time subtitle translation is done. You even cite OpenSubtitles in B.2.2. How is your claim here different that OpenSubtitles? Is this just a new dataset? If so, that is fine, but the claim should be about that - and not that this is a novel task. Or am I missing something key to your task that is new?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper has a lot of positive contributions. The analysis of domains on literal vs liberal translations is quite nice. The numerous experiments on recent models encompassing a lot of modern SOTA LLMs. A potentially interesting new PO algorithm. Empirically showing Table 3 that we cannot always beat humans (which makes sense since they likely use more modalities to translate as well). A human evaluation in section 5.3", "weaknesses": "The proposed method is a novel PO algorithm, but the comparisons are only to other models, not other PO methods. The authors claim that local is better, and thus propose ALPO as opposed to using full sequence outputs such as DPO. However, this is never empirically shown (as far as I can tell).\n\nDifferentiating the subtitle translation task from OpenSubtitles (Tiedemann 2016; Lison and Tiedemann 2016). It is mentioned as a core claim, but is only mentioned in section B.2.2", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gRzK1GT6WO", "reviewer": "Reviewer_VFQE", "summary": "This manuscirpt focus on the problem of subtitle translation using LLM. Subtitle translation refers to the machine translation task in the scenarios such as visual media and movie (visual information is not used), where the emotion and atomosphere should be considered. The authors first investigate the subtitle translation by examine exisiting LLMs with comparing the literal and liberal translation in different domain. Then, the authors proposed ALPO, a preference optimization framework that considers the contexts and segmentation specifically needed for subtitle translation. The effectiveness of the method is evaluated by LLM-as-a-judge and human measurement on unclear number of examples.", "questions": "1. Can I understand ALPO as a context-aware sampling (step 2) + DPO (step 3)?\n2.This is related to limitation 3, is it possible to add an ablation experiments that expilictly asking reasoning LLM to consider vividness, emotion and atomospher withouth ALPO optimization?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The manuscript is well-written and I could follow the logic and understand the background.\n2. Comprehensive benchmarks and exisiting LLMs are evaluated in the experiments.\n3. The problem of subtitle translation (maybe in a more general sense, machine translation in diverse tones and scenarios), is an important and emergent problem as SOTA LLMs has now reached a very strong performance on literal translation.", "weaknesses": "1. Although the topic is interesting and problem is important, the evaluation method is a bit weak and lack of soundness. See my questions for specific points below.\n2. The technical challenges of subtile translation are context-dependency and segmentation, etc., which are not discussed in depth. Intead, the point of liberal translation that the authors highlighted is vague in this manuscirpt: for example, there is a gap between lower BLEU and hihgher liberal translation, this could be simply caused by mis-translation from the models. More justification would be helpful in Section 3.2.\n3. In Section 5, the experiments use LLM-as-a-judge to evaluate the models assuming that LLM has already understand the \"emotion\" and \"atomospeher\", then why those LLM still need optimization for those two aspects? Although the BLEU correlation is shown in Section 3 but it's not directly related to the scope in vividness evaluation. \n\n\nMinor comments:\n1. In Figure3 caption: DS-R1 is the abbr. for Deepseek-R1 can be stated, though most people could guess the meaning. In addtion, chat-models and reasoning-models can be somehow visually diffrentiated, as you're comparing the patterns for those two groups.\n2. A previous work about word-level preference reward for machine translation [1], could be related this work as subtitle translation needs sentence-level context.\n\n[1] Word Alignment as Preference for Machine Translation.EMNLP 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7IBQR2OLaU", "reviewer": "Reviewer_w6RR", "summary": "The paper targets subtitle translation, arguing that good subtitles often require “liberal” renderings rather than literal word-for-word output. The authors propose a segment-level preference-optimization setup (ALPO): for each subtitle line, they sample multiple candidates, use an LLM-as-judge to pick local winners/losers, and train the model with a contrastive DPO-style loss; a few engineering choices (segment gating, prefix mixing, adaptive weighting) aim to stabilize learning and encourage fluency. Evaluation emphasizes three dimensions—accuracy, naturalness, and vividness—with both LLM-judged metrics and a small human study. Across several language directions, the ALPO-trained model improves over its SFT base and is competitive with strong chat/reasoning LLMs, particularly on lower-resource directions. Ablations suggest the local preference signal and the auxiliary knobs contribute to the gains.", "questions": "Fine-grained baselines. Why no head-to-head with fine-grained DPO (e.g., token- or segment-level) and span-level MT training (TWA)? Please add at least one strong token-level DPO baseline and a TWA-style span-supervised SFT baseline on the same data.\n\nCredit assignment. How exactly are segment preferences aggregated across a subtitle to affect the full translation? Any cases where segment-level wins yield worse whole-sequence coherence? An ablation on seq- vs seg-only vs mixed would help.\n\nJudge calibration. How sensitive are results to the choice/prompting of the LLM-as-judge? Please report inter-judge agreement, bias by style (literal vs liberal), and a calibration curve vs. human wins. Also, was any method used to overcome the LLM-as-judge biases (e.g. for label bias, ordering bias, etc.) or would that make a difference in the results.\n\nStatistical rigor. For main tables, can you include confidence intervals and a simple permutation/bootstrap test for deltas? Also report the number of items per direction to assess power.\n\nGating and prefix-mixing. These look important. Could you share the exact gating rule, thresholds, and failure cases? What happens if you remove prefix mixing or replace it with random context shuffling?\n\nCost/efficiency. What is the end-to-end training and inference cost vs. your strongest baseline, and the quality-per-dollar or per-second trade-off?\n\nError analysis. Please include a brief breakdown of where ALPO wins/loses (e.g., idioms, wordplay, cultural references, timing constraints). This would make the “liberal translation” claim more concrete.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper tackles a real-world application scenario—subtitle translation often benefits from more liberal renderings—and sets up a clean evaluation around that goal. The method is straightforward and well engineered: segment-level preference optimization with an LLM-as-judge, some human validation, and clear axes (accuracy/naturalness/vividness). The ablations are useful and suggest the components (e.g., segment gating, prefix mixing) actually matter. Empirically, the ALPO-trained model shows consistent gains over its SFT baseline and is competitive with strong chat/reasoning baselines, with gains on lower-resource directions.", "weaknesses": "Novelty and Missing references and baselines:  \n\nThe work feels largely application-driven, and the methodological novelty is unclear beyond packaging known components (segment-level preference optimization with an LLM-as-judge) for subtitles. In particular, there’s no head-to-head comparison against fine-grained preference-learning baselines that operate below sequence level, and skips fine-grained baselines that are directly comparable to a segment-level method. \n\nAt minimum, it should include some of the following methods: \n\nFor fine-grained DPO:\n\n(i) “Token-level Direct Preference Optimization” — Yongcheng Zeng et al., arXiv, 2024\n\n(ii) “TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization” — Aiwei Liu et al., ICLR, 2025 \n\n(iii) “SDPO: Segment-Level Direct Preference Optimization for Social Agents” — Aobo Kong et al., ACL (Long), 2025. \n\nFor MT specifically, a fine-grained SFT baseline like: \n\n“Learning from Others’ Mistakes: Finetuning Machine Translation Models with Span-Level Error Annotations” — Lily H. Zhang et al., ICML, 2025,\n\nshould also be in scope. Adding these would test whether the gains come from segment-/token-level supervision itself rather than the particular packaging here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZIHrRW2su2", "reviewer": "Reviewer_i1xn", "summary": "The paper describes in detail a method to adapt LLMs for subtitle translation. The paper includes a number of preliminary studies that motivate the specific challenges of this task (such as that vividity and naturalness are more important than literalness), and the use of LLM as a judge to assess vividity. It then puts all these insights into practice into a well developed multi-step process, including a new preference optimization method. This is a very well written paper.", "questions": "You point out the contrast between chat and reasoning LLMs. But are you actually using reasoning LLMs to \"think\"? The prompt in the appendix asks just for a straight translation.\n\nIn Figure 4, you start with two differerent datasets - but are they actually different?\n\nThe reinforcement learning approach only tries to optimize vividness, or do I misunderstand something here?\n\nThere seem to be three variations in your preference optimization approach: (1) filtering out of small differences (Eqn2), importance scoring based on size of the set (Eqn3) and a dynamic beta. Which of these things make the method \"adaptive\" - I could not quite connect the name with the approach.\n\nDo I read Table 4 right that your method wins against the human translation on all language pairs / metrics?", "limitations": "", "rating": 10, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Very well written and motivated work, well executed solution.\n\nInsightful analysis into what makes subtitle translation different from more factual translation challenges such as in the news/legal domain.", "weaknesses": "The correlation assessment would benefit from a second human assessment to get a sense of inter-annotator agreement, and how that relates to the correlation numbers you report.\n\nThe method does not deal with other constraints if subtitle translation, such as length limits", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)