SimulPL: Aligning Human Preferences in Simultaneous Machine Translation
Authors: Donglei Yu
Yang Zhao
Jie Zhu
Yangyifan Xu
Yu Zhou
Chengqing Zong
Venue: ICLR 2025 Poster (2025)
Keywords: simultaneous machine translation
simultaneous preference optimization
human preferences
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Simultaneous Machine Translation (SiMT) generates translations while receiving streaming source inputs. This requires the SiMT model to learn a read/write policy, deciding when to translate and when to wait for more source input. Numerous linguistic studies indicate that audiences in SiMT scenarios have distinct preferences, such as accurate translations, simpler syntax, and no unnecessary latency. Aligning SiMT models with these human preferences is crucial to improve their performances. However, this issue still remains unexplored. Additionally, preference optimization for SiMT task is also challenging. Existing methods focus solely on optimizing the generated responses, ignoring human preferences related to latency and the optimization of read/write policy during the preference optimization phase. To address these challenges, we propose Simultaneous Preference Learning (SimulPL), a preference learning framework tailored for the SiMT task. In the SimulPL framework, we categorize SiMT human preferences into five aspects: **translation quality preference**, **monotonicity preference**, **key point preference**, **simplicity preference**, and **latency preference**. By leveraging the first four preferences, we construct human preference prompts to efficiently guide GPT-4/4o in generating preference data for the SiMT task. In the preference optimization phase, SimulPL integrates **latency preference** into the optimization objective and enables SiMT models to improve the read/write policy, thereby aligning with human preferences more effectively. Experimental results indicate that SimulPL exhibits better alignment with human preferences across all latency levels in Zh$\rightarrow$En, De$\rightarrow$En and En$\rightarrow$Zh SiMT tasks.  Our data and code will be available at https://github.com/EurekaForNLP/SimulPL.

Published as a conference paper at ICLR 2025
SIMULPL: ALIGNING HUMAN PREFERENCES IN SI- MULTANEOUS MACHINE TRANSLATION
Donglei Yu Yang Zhao Jie
Jie Zhu', Yangyifan Xu' Yu Zhou' 2; Chengqing Zong' School of Artificial Intelligence, University of Chinese Academy of Sciences
2 State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
3 Graduate School of Translation and Interpretation, Beijing Foreign Studies University (yudonglei20211 zhaoyang201 0 a...cc.cr
jojo-joseephnneeefuu. edu.
yaangyifanxu2022 yu. zhou, chengqing zong] aateiaaa.....
ABSTRACT
Simultancous Machine Translation (SiMT) generates translations while receiving streaming source inputs. This requires the SiMT model to learn a read/write pol- icy, deciding when to translate and when to wait for more source input. Numerous linguistic studies indicate that audiences in SiMT scenarios have distinct prefer- ences, such as accurate translations, simpler syntax, and no unnecessary latency, Aligning SiMT models with these human preferences is crucial to improve their performances. However, this issue still remains unexplored. Additionally, pref- erence optimization for SiMT task is also challenging. Existing methods focus solely on optimizing the generated responses, ignoring human preferences related to latency and the optimization of read/write policy during the preference opti- mization phase. To address these challenges, we propose Simultancous Prefer- ence Learning (SimulPL), preference learning framework tailored for the SiMT task. In the SimulPL. framework, we categorize SiMT human preferences into five aspects: translation quality preference, monotonicity preference, key point preference, simplicity preference, and latency preference. By leveraging the first four preferences, construct human preference prompts to efficiently guide GPT-4/4o in generating preference data for the SiMT task. In the preference op- timization phase, SimulPL integrates latency preference into the optimization objective and enables SiMT models to improve the read/write policy, thereby aligning with human preferences more effectively. Experimental results indicate that SimulPL. exhibits better alignment with human preferences across all latency levels Zh En, De-=En and En-ZZh SiMT tasks. Our data and code will be available hlhttps: //tpthghttttttt// cwm/ Eurekereeekaooo PorkkaaFooNPP/P/ /imuul
INTRODUCTION
Simultaneous Machine Translation (SiMT) (Grissom II et al. 2014 Gu al. 2017 Ma al. 2019) generates translations while receiving the streaming source inputs. Therefore, the SiMT model needs learn not only the translation ability but also read/write policy during training decide whether to wait for the next incoming source token (READ) or to generate a new target token (WRITE) (Grissom II al. 2014 (Alinejad et al. 2021).
The real-time nature of SiMT scenarios leads to unique human preferences from audiences, which has been demonstrated by relevant linguistic studies (Kurz 2001 Zwischenbergeri 2010). On one hand, the audiences prefer translations that are accurate and easy to understand (Moser 1996, Srid- har al. 2013 Dayter 2020): on the other hand, they also prefer translations be delivered without unnecessary latency. Fulfilling these preferences is an important goal for interpreters (Amini et al.)
2013
Kurz 2001) and should also be considered SiMT. However, how to make SiMT models
*Corresponding author.

---

Published as a conference paper at ICLR 2025
align with human preferences remains unexplored. Existing SIMT methods (Ma et al. 2019 Aline-
jad et al., 2021) are primarily trained and evaluated on corpora from the normal offiine machine translation (OMT) task, which do not reflect real SiMT scenarios. Some studies (Chen et al. 2020) Guo et al. 2023) have proposed constructing monotonic references to avoid hallucinations, but they still fail comprehensively consider human preferences.
Furthermore, aligning preferences in the SiMT task presents its own challenges. Existing preference alignment methods (Rafailov et al. 2024| Xu al. 2024a] Ethayarajh al. (024) are designed for tasks such as OMT and question ans answering, which focus solely on optimizing the model's generated responses. In contrast, these methods have limitations in the SiMT context: they do not account for human preferences regarding latency in the SiMT task and fail to consider enhancing the read/ write policy of SiMT models during the preference optimization phase. As a result, these current prefer- ence alignment methods are unsuitable for the SiMT task.
To address these issues, we propose Simultaneous Preference Learning (SimulPL), a preference learning framework tailored for the SiMT task. In the SimulPL framework, based on existing re- search in linguistics and computational linguistics (Moser 1996 Zwischenberger 2010 He et al.) 2016 Cho 2016 Chen et al. 2020 Guo et al. 2023), we categorize human preferences in SIMT scenarios and focus on five aspects: translation quality preference, monotonicity preference, key point preference, simplicity preference, and latency preference. Based on the first four prefer- ences, SimulPI. constructs human preference prompts to effectively guide GPT-4/4o in generating preference data for the SiMT task. During the fine-tuning phase, SimulPL proposes Multi-task Supervised Fine-tuning (MSFT) to jointly train the translation ability and read/write policy of the SiMT model for initial preference alignment. Subsequently. SimulPL employs SimulDPO for fur- ther preference optimization. During the SimulDPO phase, SimulPL integrates latency preference into the optimization objective and enables the SiMT model to further adjust its read/write policy, thereby facilitating more effective aligmment with human preferences. We evaluate SimulPL on test sets with references that we manually revised to align with human preferences. Experimental results demonstrate that SimulPI. achieves higher translation quality across all latency levels. Furthermore, our manual assessment and multi-aspect evaluation indicate that SimulPL exhibits better alignment with human preferences from both the overall perspective and across the categorized five aspects.
To the best of our knowledge, SimulPL is the first preference learning framework for simultancous tasks like SiMT. Our contributions can be summarized as follows:
Our work addresses critical gap in the study of human preferences for SiMT scenarios. We categorize SiMT human preferences into five aspects: translation quality, monotonicity, key points, simplicity, and latency. This categorization enables the construction of human preference prompts to efficiently guide LLMS in generating preference data for SiMT.
We propose SimulPI., a preference learning framework tailored for SiMT scenarios. Un- like existing preference learning methods. SimulPL integrates latency preference into the optimization objective and allows the SiMT model to improve its read/write policy during the preference optimization process, enabling better alignment with human preferences.
Experimental results demonstrate that SimulPL effectively enhances the translation qual- ity across various latency levels. Furthermore, our preference evaluation indicates that SimulPL exhibits better alignment with human preferences.
RELATED WORK
Simultaneous Translation Various SiMT methods introduce different read/write policies. Some approaches propose rule-based fixed policies (Ma et al.) 2019 Elbayad et al.) 2020). while others focus on adaptive policies that adjust dynamically based on the context. These adaptive policies are modeled in various forms, such as multi-head monotonic attention Ma et al. (2020b). Trans- ducer (Liu et al. 2021). information transport model (Zhang & Feng (022), Hidden Markov model (Zhang & Feng 2023), and self-modifying process (Yu et al., 2024h. More recently, some studies Waan et al., [203a]] Agostinelli et al.) 2024 Wang et al.] 2024) have also demonstrated the promis- ing performance of large language models in SiMT tasks. However, these efforts are predominantly validated on OMT datasets. Chen et al. (2020) constructed monotonic secudo-refeennce to reduce unnecessary reorderings. Wann et al. 2023b) generated monotonic references with two-stage beam
2

---

Published as a conference paper at ICLR 2025
Human Perference Prompt
Human Petference
SimaiDPO Training Loss 14 LsmmDDO
speaker speaks, you I. Interppet the speaker's Chinese sentence into flaent English for the andience.
Translation Quality Preference
Monotomicity Preference
don't
information.
Key Point Prefereace
Rolsi (x,82r11)
Only interpret the most important key points the Chinese sentence.
Simplicity Preference
o = (Glog Amelllh X
Latency Preference
(w.1)
3.Enue word your...
(a) Data Construction
(b) MSFT
(e) SimulDPO
O
Segment Signal
0
Senviecee
Semrence
Chesen Sentence
Segmant Lobel
Chosen Senterce
Reject Sentence
LLM
LORA
LLM
LORA
Target
Source
Seuree
Source Senterce
Sertence
Figure 1: Overview of our proposed SimulPL Framework. With the first four preferences, we construct the human preference prompts guide GPT-4/4o generating human- auman-preferred translations. The latency preference is integrated into the preference optimization process.
search. Guo al. (2023 employed RL to balance monotonicity and quality of translations. How- ever, existing work fails to account for real SiMT scenarios and alignment with human preferences.
LLM Alignment Aligning LLMs with human preference has become a crucial research challenge recently. Reinforcement Learning from Human Feedback (RLHF) is one of the key approaches (Ouyang et al. 2022, Bai et al., 2022 Yuan et al. 2023). For stable training and less costs, Rafailov et al. (2024) proposed Direct Preference Optimization (DPO), which directly optimizes LLMs with- out relying on a reward model. Similarly, methods such as CPO (Xu et al., (024a) and KTO (Ftha-) yarajh al. 2024) were introduced to improve DPO. Besides, preference alignment is also widely applicd to enhance specific tasks (Sticnnon et al. 2020 Chen et al. 2024b Yang et al.] (024). Xu al. 2024b) explored using RLHF to improve the translation quality. He al. (2244) utilized the automated evaluation metrics as feedback to enhance translation performance. Nevertheless, exist- ing methods neglect latency preference in the SiMT task and do not improve the read/write policy in the optimization process, both of which negatively impact the alignment in the SiMT task.
PRELIMINARIES
Reward Modeling Existing preference alignment methods typically involve reward modeling and preference optimization. For reward modeling, human-annotated preference dataset (x, y y'') is first constructed, where x represents the input, y is preferred over y', which denoted as y y Subsequently, existing methods (Christiano et al. [0017] Kim et al.) (023)) often train reward model based on the Bradley-Tore -Terryy-TTrry model (Bradley Terry 1952), which is formulated as:
exp(r(x,y"") ply" > y' 1x)x) xx)= exp(r(x,y") exp(r(x,y")) o(exp(r(x.y")) "epp///="")))-xxxxxtxx>>)))" (I)
where (x is the score estimated by the reward model, and a(-) is the logistic sigmoid function.
Preference Optimization Reinforcement learning (RI.) is widely used for preference optimization.
Using signals from a reward model, the LLM can be optimized with the following objective:
max Ex D [r v)] - [ |x) Tree xx]
(2)

---

Published as a conference paper at ICLR 2025
Table 1: Statics of our constructed datasets, We present the reference- COMET scores of our annotated target sentences with GPT-4/4o and the original target sentences.
Our Win
Tio
Cur Losp
7h--
62%
5%
32%
De
50%
8%
34%
Dataset
Size
Ref-free COMET
Fo 2h
69%
15%
20%
train
test.
GPT-4/4o
Origin
20% 40%
60%
50%
100%
Zh
13,491
2,000
79.13
73.72
Figure 2: Human evaluation between our annotated target references and origin target references. Our newly annotated references are more preferred.
Dn
15,717
2,168
78.93
75.02
En +2h
19,967
2,841
80.30
76.97
Additionally, several methods, such DPO (Rafailov et al. 2024), directly conduct preference alignment without a reward model. However, existing preference alignment methods cannot be directly applied to the SiMT task, their optimization objectives do not account for the latency preference and do not adjust the read/write policy in the optimization process.
4 METHOD: SIMULPL
We propose Simultaneous Preference Learning (SimulPL), a preference learning framework tailored for the SiMT task. The overview of SimulPL shown Figure In this framework, we construct human preference prompts based on categorization of SiMT human preferences guide GPT- 4/4o in generating preference data. During the fine-tuning phase, SimulPL introduces Multi-task Supervised Fine-tuning (MSFT) to jointly learn translation ability and the read/write policy for ini- tial preference alignment. During the preference optimization phase, SimulPL. proposes Simultane- Direct Preference Optimization (SimulDPO), which takes latency preference into account and further improves the read/write policy. The details are discussed in the following.
4.1 CATEGORIZATION OF HUMAN PREFERENCE
In real-time SiMT scenarios, the audience exhibits unique human preferences (Kurz 2001 Zwis-
chenberger 2010 Amini et al., (0033). Based existing research in linguistics and computational linguistics, we categorize SiMT human preferences into five aspects:
Translation Quality Preference: Similar to OMT. faithful and fluent translations are also preferred in SiMT (Ma et al. 2019 Miao et al. 202I).
Monotonicity Preference: In the SiMT process, translating monotonically in accordance with the source word order allows for the delivery of translations with minimal pauses (Yang et al. 2023 Chen al. 2020), which is favored by the audience (Macías 2006).
Key Point Preference: According existing research (Moser) 1996. He al. 2016 concise translations that highlight important information points are more appealing than those that provide complete information in the SiMT scenarios.
Simplicity Preference: In real-time SiMT scenarios, the audience prefers sentences with simpler syntactic structures, which are easier to follow (Sridhar et al. 2013 Dayter 2020).
Latency Preference: In real-time settings, the audience prefers translations to be delivered without unnecessary latency (Rennert 2010 Cho 2016).
It is important to note that latency preference differs from the other four preferences, it focuses not on the translation content but rather on reducing delays. Therefore, SimulPL aligns with the first four preferences by improving translation ability, and with the latency preference by enhancing the read/write policy.
4.2 DATA CONSTRUCTION
Annotation of Human- preferre Translation In our categorization, the first four preferences are reflected in the translation content. Therefore, utilize them prior knowledge construct hu-
4

---

Published as a conference paper at ICLR 2025
man preference prompts and leverage GPT-4/4o (Achiam et. 2023) efficiently generate human- preferred translations, denoted as Y The original references, not fully aligned with human pref- erences, are denoted Y'. For the training data, select subsets from three Mitasetts-MIII De-En, WMT22 Zn- and MUST-C En-*Zh for annotation. The complete prompt used for annotation is provided in Appendix [..]] Correspondingly, newstesr2015 De->En, newstest2021 Zh-=En, and Ist-COMON are annotated for evaluation. To ensure the accuracy of the test set, we first use GPT-4/4o to generate drafts, and then manually revise them to produce human-preferred references. Our annotators are all qualified in simultaneous interpretation, ensuring reliable and irustworthy revisions. The statistics for our constructed dataset, along with ref-free COMET scores, are shown in Table [ Notably, we calculate reference-free COMET scores for both kinds of refer- ences, showing that our annotated references match the quality of the originals.
To verify that our constructed translation data aligns with human preference, we randomly sample 100 sentences from each of the three language pairs and conducted a manual evaluation by profes- sional simultaneous interpreters. The results in Figure 2 show that our annotated data achieves higher win rate, indicating stronger alignment with human preference, To further validate our an- notated data quality, we conduct additional comparisons between GPT-generated translations and manually revised translations through human evaluation, along with automatic evaluation from the perspective of the first four preferences. These results are available in Appendix A.2
Prefix Pairs Extraction To enable the SiMT model in learning translation based on source pre- fixes instead complete source sentences, we extract prefix pairs from our annotated sentence pairs using word alignment and add them to the training data. For each sentence pair (X, Y we use awesome-align (Dou & Neubig (02T)) to get the word alignment. For target token y/, we denote the corresponding source token as Ta. and the set of extracted prefix pairs are denoted as:
D.d y() if O S ",,,, then 0 < 0g S |ar]; "ly"|+1 |
(3)
Intuitively, for the given source prefix x, the target prefix y" includes all the translatable content. Similarly, extract prefix pairs from sentence pairs (X, Y obtain D Then, merge D and DJ to create the prefix-level preference dataset:
Dp = xx,,'' " .y') (x,y") E (xy')
(4)
4.3 MULTI-TASK SUPERVISED FINE-TUNING
Based on pre-trained language model Tre SimulPL introduces Multi-task Supervised Fine-tuning (MSFT) to jointly learn translation ability and read/write policy on D for initial preference align- ment. For translation ability, the model learns to generate the target prefix y" from the source prefix For read/write policy, SimulPL adds an extra confidence layer, consisting of linear layer and a sigmoid layer, to make read/write decisions. Specifically, when predicting Vt an additional con- fidence Ci" is estimated by the confidence layer. If t < Iy" the model should predict ci 1, indicating the WRITE decision. Otherwise, if >> ly"",, the model should estimate ctºº = O, which means should stop translating and choose the READ decision. The complete training loss for the MSFT phase is calculated as:
|
by "+1
LMSPT
II 1=1
si" log Tan xx, xx,===-))=
[[t
+ +I( > ( log (1 c(")
(5)
where Weft is initialized with the parameters of Tpre. and II(-) denotes the indicator function. It is noted that we train the model to predict (x"141 = 0, allowing the SiMT model to learn to stop translating at the appropriate position.
4.4 SIMULTANBOUS DIRECT PREFERENCE OPTIMIZATION
After the MSFT phase, SimulPL introduces Simultaneous Direct Preference Optimization (SimulDPO) to further align with human preferences. In the SimulDPO phase, SimulPL integrates the latency preference into the optimization objective and allows the SiMT model to further improve its read/write policy during preference optimization.
5

---

Published as a conference paper at ICLR 2025
First, we modify the optimization objective from Equation
2 to encourage the SiMT model gen- human-preferred preferred translations while satisfying additional latency preference. Specifically, we add output length constraint, which can be expressed follows:
erate human-
max Bx.. (ylx) (xx,))(x))))))))))))))))))))(( - (ToL(TT(y x)]lFrer(y X)] + aB[ly
Th where a a pyyer--paaaeee introduced to control the output length constraint. Unlike áe which only accesses the source prefix Tref provided with the complete source sentence X to generate a more accurate prediction, thus effectively preventing The from diverging. Intuitively, based on the much content possible to minimize
received the SiMT model is encouraged to translate unnecessary latency. Theoretically, we prove that the constraint E|yy aligns with the objective of reducing latency, which is provided in Appendix [B.I Therefore, although similar form to R-DPO (Park et al. 2024), our goals and methods are entirely opposite, as detailed in Appendix
Based on Equation 6
derive the following reward function concerning me and ly]:
Toly
x) X)
y( = B log
+loog ((x olyy
Trefly
where (xx the partition function. The detailed derivation is provided in Appendix B.2
Since both To and nel are initialized from Tan they also possess the ability to estimate confidence. We denote the confidence estimated by my when predicting Yt as ct, leading to the following expres-
sion: (y x) + (mo x,ys x vl) = +yy++1 (To( x,ys =-)) Intuitively,
[
we can also gain that: y = Eh I[t S 1 =
сt- Substituting these two equations into
Equation 7 we obtain the following representation of yxx,)):
ly|+1
ly++1
rx,y)) = B
Σ
co log
To (ye
x,y5 + B log 7(x)
Σ
4
(V
[=1
Then, with the Brarley-Terry model, we can derive the training objective of SimulDPO as follows:
1y"++1
Iy'I+1
SSmalDDO = log (
Σ - Σ
Ta(
x, x
ot (B log
a), * (w,1}
x,y=2-1)
This training loss enables the SiMT model to further improve its read/write policy during the pref- erence alignment phase, and takes the latency preference into account. During training, the SiMT no (o("?" (x,y
model can accurately predict a token that aligns with human preferences (i.e., log matso IX
9). then the SiMT model will learn predict c:" close to avoid latency. Conversely, if the prediction does not align well, cf should be predicted close O. Additionally, since the read/write decisions for yd are not the focus of the optimization, we directly set ef = I[(t S Iy) instead of using the predictions from the SiMT model during training.
4.5 CONFIDENCE BASED POLICY DURING INFERENCE
During inference, SimulPL makes decisions based on estimated c. ef > 0.5, it indicates that the SiMT model can generate target token y/ aligned with human preferences and should choose WRITE; otherwise, it chooses READ. Following Wang et al. (2023a), we introduce the reading length n to control the latency level SimulPI. Specifically, when SiMT model chooses READ, it needs to wait for new source words before making further decisions. In Appendix D we provide more details with Algorithm and analyze the impact of the confidence threshold.
EXPERIMENTS
5.1 EXPERIMENTAL DETAILS
Datasets We validate our method on text-to-text SiMT tasks using our annotated datasets with human-preferred references. For Transformer-hased SiMT' models, first pre-train them the

---

Published as a conference paper at ICLR 2025
20
L15 PFX
1154 PFX
1IMEPOX SFTHDRO
11M PFX SET-1PD
(a) Zh EEn
(h) De->En
(c) En
Figure 3: SacreBLEU against LAAL on Zh En, De and En +Zh SiMT tasks.
303
225
253
5M
SMP
303
IAML
LAAL
JAM
(a) Zh
(h) De->En
(c) En-*Zh
Figure 4: COMET against LAAL on Zh- De and En-+Zh SiMT tasks.
complete OMT training sets and then fine-tune on our annotated data. For LLM-based SiMT mod- els. we begin with SFT on subset of the OMT training data with the size of 100k, followed by additional SFT on our annotated data, so that LLMs can initially learn the translation ability.
Baselines Existing SiMT models primarily include Transformer- and LLM-based architec- tures. We reproduce both types of SiMT models, as detailed in the following:
wait-k (Ma et al. 2019) This Transformer-basse SiMT model first waits for k source tokens. It then repeatedly generates target token and waits for a source token.
SM2 Yuu et al., 2024): This Transformer based SiMT model learns the confidence of cur- rent prediction during training. During inference, it decides whether to wait for an addi- tional source token or output target token based on the confidence.
LLM-PFX-SFT (Wang et al. 2023a): This ALM-based SiMT model utilizes prefix-pairs data for SFT and utilizes incremental decoding during inference.
I.L.-PFX-SFT-DDE Building I.LM-PFX-SFT, further align preferences using our constructed prefix-level preference data through standard DPO method.
Implementation Details We implement Transformer-based SiMT models using Fairseq OOt et al.
2019). For the Zh-IEn and En-*Zh SiMT tasks, use vocabularies of 45,000 for Chinese and 35,000 for English respectively. The De-En task is applied with shared vocabulary of 32,000. For LLM-based SiMT models, following, we choose Blama2-BB-cha (Touvron al. 2023 as the base model and use Simuleval (Ma et al. 2020a) for evaluation. To address vocabulary inconsistencies among SiMT models, we allow word-level read/write operations during testing facilitate accurate latency comparison. More implementation details are available in Appendix
Evaluation Metries We use specific metrics to measure our categorized human preferences respec- tively. For Translation Quality Preference, we utilize SacreBLEU (Posi) 2018) and COMET as the metrics. For Latency Preference, we chose Length- et al. 2022) to avoid misjudging the over-generation phenomena. A higher LAAL indicates greater latency. For Monotonicity Preference, we define Normalized Inversion Rate (NIR) to measure the monotonicity. Specifically, first use awesome-align to obtain word alignment between the
Average Lagging (LAAL) (Papi
7

---

Published as a conference paper at ICLR 2025
Oud Mi
Te
Dut Lasa
2h.4
Do En
B1 2h
Pigg
44.%%
18.4
Hih
Hin
54.5%
38
11.2%
30.6
38.4%
(a) Human evaluation between SimulPL. and LLM-PFX-SFT.
Dur Wi
LorLow
25
69%
100 65% 66% 1106
185
(b) Human evaluation between SimulPI. and I.M.-PFX-SFT-DDO.
Figure 5: Human Evaluation in differenct latency groups Zh-+En, De and En->Zh SiMT tasks. We divide the latency into three levels: low latency (0 S LAAL < 4), medium latency (4 S LAAL < 8), and high latency (LAAL 8). We report the independent evaluation results at each latency group and the overall results across all latency groups
Table Multi-aspect Evaluation different latency groups on Zh, De and En ZZh SiMT tasks. We define Normalized Inversion Rate (NIR), Sentence Length Ratio (SLR) and Dependency Depth (DD) for respectively measuring monotonicity preference, key point preference and simplicity preference. COMET and SacreBLEU are used to measure translation quality preference.
Preference Analysic
Medel:
Zh
De
En
NIR
DD
SLR
COMET
BLBU
NIR
COMET
BLEU
NIR
SLR
COMET
BLEU
Low
ILM-PFX-SFT
5.24
7.60
0.90
24:25
12.05
3.03
5.95
1.32
81.21
29.51
5.21
5.72
2,60
24. 74
19.33
I.M-PFX-SFT-DPO
3.30
7497
0.80
22.67
14.70
1.97
5.76
1.29
81.19
20.20
5.13
5/88
2.55
74.55
19.62
SimuPPI
2.95
6.97
R.54
76.32
23.73
2.69
5.02
1.81
82.34
42,69
4.11
5.46
1196
30.52
38.99
maciius Entency
LEX-SEX-SET
4.65
7.02
0.65
78.76
24.13
3,80
5.97
1.01
$1.86
46.31
7.65
5.43
0.97
$2.42
41.97
LLM-PIX-SIT-DRO
3.89
7.09
0.65
79.14
25.90
3.85
5.94
1.00
$1.79
46,39
7.55
5.48
0.57
$1.87
41.58
SimolPL
4,21
6.91
0.62
79.29
27.37
3,60
5,96
0.99
$6.93
45.03
7.11
5.38
0.95
$2.29
43.88
High Latraryy
ILM-PEX-SFT
5.66
6,82
0.6
80.64
20.81
4.53
5.03
0.98
86.48
$1.41
9.45
5.40
0.91
83.56
44.47
LLM-PFX-SET-DPO
4.60
6.85
I162
81.25
29.47
4.70
5.90
0.97
85.23
51.60
9.34
5.37
0.91
83.08
44.10
StmoiPL
5.59
6.78
0.60
81.00
30,00
4.44
5.92
0.97
86.40
51.58
9.10
5.37
0.92
83.63
44.96
model's output Y and the source sentence X. We denote the source position corresponding to Or as àz. These source positions form sequence Å = [ā1, a.....]. The inversion number (Mannila, 1985) of A, denoted as LA. reflects the monotonicity of /A. A smaller /3, indicates more mono- tonic translation, while a larger IA suggests more reordering in the translation. Since IĄ is affected 100% the metric for monotonicity preference. For Key by |Å,, define NIR -1]
21
Point Preference, we use the Sentence Length Ratio (SLR) as the metric, defined SLR | A smaller SLR indicates that the SiMT model completes the translation with a shorter sentence, aligning better with key point preference. For Simplicity Preference, we use Stanza (Qi et al. 2020) to convert Y into a dependency tree 7, and define the depth of T as Dependency Depth (DD) for evaluation. A smaller DD indicates simpler syntax, which is easier for audiences to follow.
|
5.2 TRANSLATION QUALITY
The SacreBLEU and COMET scores for different SiMT methods are shown in Figure Figure These results show that proposed SimulPL achieves higher translation quality across all latency levels on three language pairs, particularly in low latency level. This indicates that SimulPL better meets the translation quality preference in SiMT scenarios. Since the test used includes
8

---

Published as a conference paper at ICLR 2025
human- -aiinnd references, argue this can also reflect the effectiveness of SimulPL in aligning with other human preferences. We provide detailed numerical results and values for other latency metrics in Appendix F
5.3 PREFERENCE EVALUATION
To validate the effectiveness of SimulPL. in preference alignment, we conduct further human pref- erence evaluation for L.M-PXX-SFT, L.L.-PFX-SFT-DD
and SimulPI. This evaluation includes the overall human evaluation and multi-aspect evaluation. Specifically,
first divide the models' outputs into three latency groups: low latency (0 S LAAL < 4), medium latency (4 S LAAL < 8), and high latency (LAAL 8). For human evaluation, we manually assess 100 sentences sampled from each latency group. Our evaluators are all qualified in simultaneous interpretation and can pro- vide accurate assessments. For multi- aspeit evaluation, we measure the performance these SiMT models in terms of translation quality preference, monotonicity preference, key point preference, and simplicity preference in different latency groups. Detailed analyses are provided as follows.
Human Evaluation The results of the human evaluation are shown in Figure 5 which show that SimulPL achieves higher win rates in all latency groups for these three language pairs. This indicates that SimulPL can generate translations more aligned with human preferences. We attribute this performance improvement to the joint optimization translation ability and read/write policy during the preference alignment process.
Multi-aspect Evaluation The results muti-aspect evaluation are shown in Table SimulPL achieves better alignment across all latency groups for the three language pairs. SimulPL not only maintains high translation quality but also effectively manages monotonicity. key points, and simpler syntactic structures. Under low latency conditions, SimulPL. achieves a better trade-off between latency preference and other preferences and generates better translations.
5.4 ABLATION STUDIES
We conduct ablation studies on SimulPL for the Zh->E SiMT task, with detailed analyses in the following. Additional results and other analyses on De-+En SiMT task are shown in Appendix G
Effect of MSFT To verify the role of MSFT, we evaluate the performance of a SiMT model trained with regular SFT using prefix pairs data, similar LLM-PFX-SFT. The results in Figure that MSFT outperforms SFT, especially in the low latency level. This shows that by explicitly modeling the multi-lask of translation ability and read/write policy, MSFT improves the SiMT performance more effectively and provides better initialization parameters for SimulDPO.
Effect of SimulDPO shown in Figure o SimulPL, which introduces SimulDPO after MSFT phase, achieves higher SacreBLEU scores across various latency levels compared to Only MSFT. This indicates SimulPL further enhances the translation ability and read/write policy during the SimulDPO phase, leading to better performance.
Effect of Tyer(y X) To verify whether the predicted Tyer(y X) in OMT setting can provide more accurate constraint for the training objective, we replace Teet (y X) in SimulPL with the this setting. As shown in Figure 6 the performance of SimulPL trained with m x) obviously declines. We argue this is due to the inaccurate prediction of (f negatively impacts the preference optimization.
probability Fe (y x) and evaluate the performance
5.5
IMPACT OF ON BALANCING ALIGNMENT AND LATENCY DURING TRAINING
into the training loss. As shown in Equation 9 a functions token-level threshold. Since the gradient of ard, does not propagate, we only analyze the impact of act". Specifically, during training, if log .h(
In SimulDPO, a is introduced as a
Nots?" (x.y
38 we consider that no (v²" x,y presents prediction aligning human preferences well, and the SiMT model should learn to predict a higher c/". Conversely, the model should learn a lower cf' when this enhance the model's ability to learn better alignment quality. However, a is set too high, the SiMT model could become overly cautious in translation, leading e!" failing to accurately balance between latency preference
condition is not met. Thus, appropriately increasing the value of

---

Published as a conference paper at ICLR 2025
00
KTO
DR0
176
Figure 6: Ablation Studies of SimulPL framework on Zh-+En SiMT task.
Figure 7: The comparison of SimulDPO with difference a values on Zn SiMT task.
Figure 8: The comparison between different preference optimization methods.
and other preferences. To validate the effect of a we train SimulPI. with different a values and compare their performance 7h-+En task. The results shown in Figure 7 When 10, the SiMT model always estimates low confidence and tends to wait for more tokens before generating translation. On the other hand, when a is too small (a 0), the model's performance decreases to some extent. We will further explore the impact of a in future work.
5.6 GENERALIZATION TO OTHER PREFERENCE OPTIMIZATION METHODS
In the SimulPI. framework, SimulDPO is introduced for further preference optimization, which adapted from DPO. Theoretically, SimulPL has generalization to any preference optimization methods, making them applicable to the SiMT scenarios. validate this generality, we integrate
CPO and KTO into the SimulPL framework, deriving SimulCPO and SimulKTO. Their respective training losses are provided in the following:
141
1'11
CsimulCPO = - log o
Σ ur Σ
w!) - log To |x)
(10)
4 = Be, log (vt xxy aci, * * [w,1}
LSmulkKO - -((x,))
A (x|+1 rt 20), i ~ xx >Io Iyl rt). if
(11)
y)x,y)
where Ay denotes Ay (At) when y is desirable (undesirable). and 20 [ x) m X)].
We evaluate their performances on the Zn-Enn SiMT task. As shown in Figure g both SimulCPO and SimulKTO achieve higher performance compared to CPO and KTO, particularly in low-latency levels. These results indicate the generalization of SimulPL. Additionally, SimulDPO, SimulCPO, and Sim exhibit similar performance, making it difficult to determine which most suitable for SiMT task. Besides preference optimization methods, SimulPL may also generalize to other tasks like simultancous inference (Chen et al. 2024a). We will explore this in future work.
CONCLUSION
We bridge the gap in the study SiMT human preferences and propose SimulPL, a preference learning framework tailored for SiMT task. Drawing from existing research, we categorize prefer- ences SiMT scenarios into five aspects: translation quality, monotonicity, key points, simplicity, and latency. By leveraging the first four preferences, SimulPI. constructs human preference prompts to efficiently guide LLMs in generating preference data for SiMT. During the fine-tuning phase, SimulPL introduces MSFT for initial preference alignment. During the preference optimization phase, SimulPL proposes SimulDPO, integrating latency preference into the optimization objective and further improving the read/write policy. Our experiments indicate that SimulPL achieves bet- ter preference alignment both overall and across each aspect. Additionally, our analysis shows that SimulPI. has generalization other preference optimization methods.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
Acknowledgment The research work has been supported by the Natural Science Foundation of China under Grant No. 62336008 and No. 62476271.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint XXiv:2303.08772 2023.
Victor Agostinelli, Max Wild, Matthew Raffel, Kazi Fuad, and Lizhong Chen. Simul-LLM: A framework for exploring high-quality simultaneous translation with large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume Long Papers). Association for Computational Linguistics, 2024.
Ashkan Alinejad, Hassan S Shavarani, and Anoop Sarkar. Translation-basel supervision for policy generation in simultaneous neural machine translation. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pp. 1734-1744, 2021.
Mansour Amini, Noraini Ibrahim-González,, Leelany Ayob, and Davoud Amini. Quality of inter- preting from users' perspectives. International Journal of Language and Education, 2(1):2013, 2013.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.0580 2022.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4))324-345, 1952.
Chuangtao Chen, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo Ulf Schlichtmann, and Bing Li. Livemind: Low-latency large language models with simultaneous inference. arXiv preprint arXiv:2406. 14319, 2024a.
Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Step-level value preference optimiza- tion for mathematical reasoning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 7889- 7903, Miami, Florida, USA. November 2024b. Association for Computational Linguistics. doi: 10. (0.18653/1/2222.inniin mandinng-eemll.46.3. URL https: httpss//aclaattbooooo..o//////
Findings-emmip.cr 4637
Junkun Chen, Renjie Zheng, Atsuhito Kita, Mingho Ma, and Liang Huang. Improving simul- taneous translation by incorporating pseudo-references with fewer reorderings. arXiv preprint arXiv:2010.11247, 2020.
K Cho. Can neural machine translation do simultaneous translation? arXiv Preprint, CoRR, arXiv: abs/1606.0200 02012, 2016
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing sys- tems, 30, 2017.
Daria Dayter. Strategies in corpus of simultancous interpreting. effects of directionality, phraseo- logical richness, and position in speech event. Meta, 65(3):594-617,
20200. 2020.
Zi-Y Yi Dou and Graham Neubig. Word alignment by fine-tuning embeddings on parallel corpora. In Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2021.
11

---

Published as a conference paper at ICLR 2025
Maha Elbayad, Laurent Besacier, and Jakob Verbeek. Efficient wait-k models for simultaneous machine translation. arXiv preprint arXiv:2005. S000::2005.08895,, 2020.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306 2024.
Alvin Grissom II, He He, Jordan Boyd-Graher, John Morgan, and Hal Daumé III. Don't until the final verb wait: Reinforcement learning for simultaneous machine translation. In Proceedings of the 2014 Conference on empirical methods in natural language processing (EMNLP), pp. 1342- 1352, 2014.
Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Victor OK Li. Learning translate in real-time with neural machine translation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Pupers, pp. 1053-1062, 2017.
Shoutao Guo, Shaolei Zhang, and Yang Feng. Simultancous machine translation with tailored ref- erence. arXiv preprint arXiv:2310. 135888 2023.
He He, Jordan Boyd-Graber, and Hal Daumé III. Interpretese vs. translationese: The uniqueness of human strategies in simultancous interpretation. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 971-976, 2016.
Zhiwei He, Xing Wang, Wenxiang Jiao, Zhuosheng Zhang, Rui Wang. Shuming Shi, and Zhaopeng Tu. Improving machine translation with human feedback: An exploration of quality estimation as a reward model. arXiv preprint arXiv:2401. 2024.
Changyeon Kim, Jongjin Park, Jinwoo Shin, Honglak Lee, Pieter Abbeel, and Kimin Lee. Pref- erence transformer: Modeling human preferences using transformers for rl. arXiv preprint arXiv:2303.00955, 2023.
Tom Koemi, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel, and Maja Popović. Findings of the 2022 conference on machine transla- tion (WMT22). In Philipp Koehn, Loic Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatter- jee, Marta R. Costa-jussă, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz. Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Kocmi, André Martins. Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, and Marcos Zampieri (eds.), Proceedings of the Seventh Conference on Machine Translation (WMT), pp. 1-45, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL (oLthtps::// htttps://aalaathol........
.rg. wmt-1.3 1.1
Ingrid Kurz. Conference interpreting: Quality in the ears of the user. Meta, 46(2):394-409 2001.
Dan Liu, Mengge Du, Xiaoxi Li, Ya Li, and Enhong Chen. Cross attention augmented transducer networks for simultaneous translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 39-55, 2021.
Mingbo Ma. Liang Huang, Hao Xiong, Renjie Zheng. Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, al. Stacl: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3025-3036, 2019.
Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, and Juan Pino. Simuleval: An evaluation toolkit for simultaneous translation. In Proceedings of the EMNLP, 2020a.
Xutai Ma, Juan Miguel Pino, James Cross, Liezl Puzon, and Jiatao Gu. Monotonic multihead attention. In International Conference on Learning Representations, 2020b.
Macarena Pradas Macías. Probing quality criteria in simultaneous interpreting: The role of silent pauses in fluency. Interpreting, 8(1):25-43, 2006.
12

---

Published as a conference paper at ICLR 2025
Heikki Mannila. Measures of presortedness and optimal sorting algorithms. IEEE transactions computers, 100(4):318-3) -25, 1985.
Yishu Miao, Phil Blunsom, and Lucia Specia. A generative framework for simultaneous machine translation. In Proceedings the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6697-6706, 2021.
Peter Moser. Expectations of users of conference interpretation. Interpreting, 1(2):145-178, 1996.
Myle Ott, Sergey Fdunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019 Demonstrations, 2019.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Sara Papi, Marco Gaido, Matteo Negri, and Marco Turchi.
cannot be rewarded: aength-adaptivve average lagging for simultaneous speech translation. In Proceedings the Third Workshop on Automatic Simultaneous Translation, pp. 12-17, 2022.
Ryan Park, Rafael Rafailov, Stefano Ermon, and Chelsea Finn. Disentangling length from quality in direct preference optimization. arXiv preprint arKiv:2403. 19159,, 2024.
Matt Post. A call for clarity in reporting bleu scores. WMT 2018, pp. 186, 2018.
Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning- Stanza: A Python natural language processing toolkit for many human languages. In Proceedings the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2020.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
Sylvi Rennert. The impact of fluency on the subjective assessment of interpreting quality. 2010.
Vivek Kumar Rangarajan Sridhar, John Chen, and Srinivas Bangalore. Corpus analysis of simul- taneous interpretation data for improving real time speech translation. In INTERSPEECH, pp. 3468-3472, 2013.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3022 2020.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.0922 092888, 2023.
Minghan Wang, Jinming Zhao, Thuy-Trang Vu, Fatemeh Shiri, Ehsan Shareghi, and Gholam- reza Haffari. Simultaneous machine translation with large language models. arXiv preprint 2rXiv:2309.06706, 2023a.
Minghan Wang, Thuy- Thuy-Trang Vu, Ehsan Shareghi, and Gholamreza Haffari. Conversational simulmt: Efficient simultaneous translation with large language models. arXiv preprint a0X5v:2402.10552, 2024.
Shushu Wang, Jing Wu, Kai Fan, Wei Luo, Jun Xiao, and Zhongqiang Huang. Better simultaneous translation with monotonic knowledge distillation. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.
2023b.
Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation. arXiv preprint arXiv:2401.084. 2024a.
13

---

Published as a conference paper at ICLR 2025
Nuo Xu, Jun Zhao, Can Zu, Tao Gui, Qi Zhang, and Xuanjing Huang. Advancing translation prefer- ence modeling with rlhf: A step towards cost-effective solution. arXiv preprint arXiv:2402.11525, 2024b.
Fan Yang, Fen Gao, and Kexin Zhang. Impacts of directionality on disfluency of english-chinese two-way sight translation. International Journal of Translation, Interpretation, and Applied Lin- guistics (IJTIAL), 5(1):1-15, 2023.
Wen Yang, Junhong Wu, Chen Wang. Chengqing Zong, and Jiajun Zhang. Language imbalance driven rewarding for multilingual self-improving. arXiv preprint 2rXiv:2410.08964, 2024.
Donglei Yu, Xiaomian Kang, Yuchen Liu, Yu Zhou, and Chengqing Zong. Self- state modeling for simultaneous machine translation. In Lun-Wei Ku, Andre Martins, and Vivek Sriku- mar (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pp. 9781-9795, August 2024.
Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songlang Huang, and Fei Huang. Rrhf: Rank responses align language models with human feedback without tears. arXiv preprint arXiv:2304.05530,, 2023.
Shaolei Zhang and Yang Feng. trffrmatiinn-raassorttto bbasd policy for simultancous translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 992-1013, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
/ttpttpsss//aaaaaiiiaa hww.hhhoooooooooo...oooooeooge/e 2022. emnlp-mainlni 65
Shaolei Zhang and Yang Feng. Hidden markov transformer for simultaneous machine translation. arXiv preprint arXiv:2303.00257 2023.
Cornelia Zwischenberger. Quality criteria simultaneous interpreting: An international vs. na-
tional view. 2010.
14

---

Published as a conference paper at ICLR 2025
A DATASET CONSTRUCTION AND ANALYSIS
A.1 HUMAN PREFERENCE PROMPTS
Based on our categorized SiMT human preferences, we construct human preference prompts, which account for translation quality preference, monotonicity preference, key point preference, and sim- plicity preference, to efficiently guide GPT-4/4o in generating preference data for the SiMT task. Taking Zh SiMT task example, complete human preferences prompts are shown in Figure
System:
You are professional simultaneous interpreter, simulating the scenario of interpreting speaker's Chinese speech into English in real time.
User:
As speaker speaks, you need
Interpett the speaker's Chinese sentence fluent English the audience.
2. don't need to translate information. Only interpret the most important key points Chinese sentence.
Ensure the order of your interpretation must the same with the worl order the Chinese sentences.
U.Use simple, common words, and keep the syntax very easy. Make your interpretation short as possible.
Now, I will provide you with two examples.
Input:
R.
Output:
key points order: 1. 2. 2 3.. 39 EJ @ 3. # M # #5 5. -
interpretation: Last week, losing his Nike sponsorship, Brown said he would start from Miami and train at high school one day per w week.
Input:
3 A fF
Output:
points order 1. HE 2: कu $6 at ti ue 35 3. 3
interuretatio:: flter assuming office Foreign Minister, the chief negotiator for the peace trealy, Motegi held his first negotiations with Russia.
Input:
<souree sentence>
Figure 9: Our constructed human preferences prompts on Zh->En SiMT task.
A.2 FURTHER EVALUATION OF OUR ANNOTATED DATASETS.
We conduct both automated muti-aspect evaluation and additional human evaluation validate the quality of our constructed dataset further. The details are described in the following.
Multi-aspect Evaluation. Similar to Section 5.3
we also use our defined NIR, SLR, and DD to conduct multi-aspect evaluation on the GPT-generated references and the original references. We use Ref-free COMET to assess the translation quality here. The results in Table 3 indicate that GPT-4/4o aligns better with human preferences.
Then, compare the multi-aspect evaluation results of GPT-generated translations and those man- ually revised by interpreters on the test sets. facilitate comparison, we also provide the results for the original references of the test sets. As shown in Table 4. the translations generated by GPT- 4/4o are either superior to or comparable with the original references terms of monotonicity, key points, simplicity, and translation quality. Moreover, these results very close to the manually revised translations. This indicates that the quality of the GPT-generated data is both reliable and aligned well with human preferences.
15

---

Published as a conference paper at ICLR 2025
Table 3: Multi-asppet evaluation on our annotated references and original references.
References
Zh -En
De-En
En-Zh
NIR
DD
SLR
COMET
NIR
DD
SLR
COMET
NIR
DD
SLR
COMET
GPT-4/4o
11.39
5.48
0.55
79.13
9.99
5.61
0.92
78.93
25.20
5.52
0.89
80.30
Origin
45.41
6,34
0.56
73.72
34.37
6.31
1,09
75.02
44.05
6.20
0.94
76.97
Table 4: Multi-aspect evaluation on sets.
References
NIR
DD
SLR
COMET
NIR
DD
SI.R
COMET
NTR
DD
SL.
COMET
GPT-4/4o
6.48
6.55
78.32
5.49
5.71
0.92
80.86
10.65
0.88
81.24
Manually Revised
6.71
6.69
0.52
79.05
5.52
5.84
0.95
81.89
10.47
5.63
0.88
81.42
Origin
79.05
80.78
75.77
Human evaluation. Tollowing/Koent al. (2022) and and et 2024a), we randomly sample 200 sentences from the test set and employ professional interpreters, who are not involved in the revision process, to manually assess their GPT-generated translations and manually-revised translations. This evaluation is conducted on our test sets. The evaluation criteria are follows:
0: The translation is poor and fails to convey any meaningful information.
2: The translation conveys of the speaker's information but misses key points. It also includes unnecessary reordering, leading to unnecessary delays in real-time scenarios, and uses less common expressions that make it harder for the audience quickly understand.
4: The translation conveys all the important information with minimal unnecessary reorder- ing. It uses simple expressions that generally meet the audience's needs, though there are still some minor issues.
6: The translation is perfect interpretation, accurately conveying the speaker's key points while omitting unnecessary details. It uses expressions that align with spoken language conventions, significantly reflecting human preferences in simultaneous interpretation.
The results are shown in Table 5 those are manually revised. Besides, the score distribution shows that most of the GPT-generated translations scored 4 higher. These results suggest that the GPT-generated data aligns well with SiMT human preferences.
The average scores of GPT-generated translations are close to
Table 5: Human evaluation on GPT-generated translations and manually revised translations.
Win-Tie-Lose
Distribution of Scores
References
Average Score
win ratio
ratio
tie ratio
Chinese Enggishh
GPT-4/40
5.37
6.00%
12.00%
82.00% 82.00%
0.50%
5.50% 1.00%
19.00% 19.50%
75.00%
Manually Revised
5.57
12.00%
6.00%
0.00%
79.50%
German- English 80.50%
GPT-4/4o
4.47
2.00%
17.50%
5.50%
19.50%
21.00%
54.00%
Manually Revised
5.01
17.50%
80.50%
2.00%
3.00%
10.00%
20.50%
66.50%
Piglish-Chiness
GPT-4/4o 4.61
1.00%
94.00%
5.00%
3.50%
10.50%
38.00%
48.00%
Manually Revised
4.73
5.00%
94.00%
1.00%
3,50%
8.50%
36.00%
52.00%
B PROOFS AND DERIVATIONS
B.1 PROOF OF EQUIVALENCE BETWEEN OUTPUT LENGTH CONSTRAINT AND LATENCY OPTIMIZATION
To incorporate the goal of reducing latency into the optimization objective, we can directly include latency evaluation metric. Specifically, we integrate Average Lagging (AL.) (Ma al. 2019).
16

---

Published as a conference paper at ICLR 2025
commonly used metric for measuring SiMT latency, into the optimization objective as follows:
max Ex- xx) (x, (
xx] |Ter X)] - E[AL]
(12)
Given the sample (X, Y), when receiving source prefix x, the SiMT model needs output the target prefix y. We assume that the SIMT model can accept source sentence with a maximum length of M and generate target sentence with maximum length of N. Therefore, the following relationship holds:
(13) During simultaneous translation, the prefix pair (x. y) evolves from (zo, 9) (X, Y). For the SiMT process passing by yx, the maximum possible latency occurs when the SiMT model waits for the full input of before starting to generate y, then waits for the complete X before outputting the remaining part of Y. In this situation, AL can be computed as:
x S X M,
by S S
X Y (lyl-1)
1 2N
AL
[
x
(- 2N
+ +1|X -Glyy++ (14)
Y
X
where C1 = 2N
C2
X Therefore, we can derive the following relationship:
2N
D.yD.y (y|x) (xx 3D [ xx)er
(y X)] - [[AL]
(15)
< B.-D.y-ma (y|x) (xx yx] 3L (y xeel X)] + C] - C2
Based on this upper bound, we can optimize the objective in Equation 12 by optimizing the following one:
max Ex Ex-DDyywoll x) ,)[r(x,)) v)] - (Ta (Ter X)] + C,B[ly] C2
(16)
Since C1 and C2 constants, this objective is equivalent to:
max (y [x] - 3D (y x)][Frer(y X)] + oB[[y
(17)
where a is the added hyper-parameter parameter.
B.2 DERIVATION OF THE OPTIMAL SOLUTION FOR REWARD MAXIMIZATION CONSTRAINED BY KI. DIVERGENCE AND LATENCY
Starting from Equation 6 we can conduct the derivation follows:
max Ex y (y [r) - (T x) [ [X]]
aBllyl
To x))
max Ex D By B log
+ alyl]
W0
neat (y X)
( x) 2[х Trerly
(18)
min Ex By
exp( (r(x,)) + alyl)) log 2(x))
((x) ( X) exp( 1 3
=-min Ex [lx [Teel
(r(x,)) alyl))] - log (xx)
Where (xx is the partition function, which is calculated as:
Z(x) Σ Trefly ex)xp( 1 (r(x,)) (((x,)))))))))))) + aly)))
(19)
Based on the property of KI. divergence, we can gain the optimal solution for Equation as
x²
[(x) (y X) exp( (r(x,)) y) + alxl))
(20)
Thus, we can derive the expression of rxx, concerning wo and ly]:
no
x)
r(x,y) y) = =llog
+ Bog ((x) alyl
(21)
Tref
X)
17

---

Published as a conference paper at ICLR 2025
C DISTINCTIONS BETWEEN SIMULDPO AND R-DPO
Firstly, the objectives and methods of R-DPO are entirely opposite to SimulDPO. In R-DPO, Park ct al. (2024) aim to prevent models from generating too long responses and use regularization term of yy to achieve this. In contrast, for the SiMT task, audiences prefer translations with low latency, which requires the SIMT model to translate as much content as possible based on the already received source prefix. To achieve this, SimulDPO introduces "+aly" as additional constraint. important to note that the goal of SimulDPO not to optimize for the length itself, but rather optimize for latency preferences.
Secondly, as shown Equation we use |y| = ΣE 6 to make "+aly" differentiable, allowing gradient signals to be directly propagated to the parameters through backpropagation. In contrast, Park et al. 2024) treats the " "- l" as a margin without further processing.
Iv
D CONFDEENC-BASED POLICY DURING INFERENCE
Algorithm further illustrates the confidence-based policy adopted by SimulPL during inference. As shown in Algorithm 1, in the confidence-based policy, we set the confidence threshold to 0.5 as the basis for read/write decisions. To examine its impact, we compare the performance of SimulPL on the Zh-+En task with different threshold values (7). The results are presented in Table 6 When y is set to small value (-=0.1), the model is allowed to output tokens with low confidence, which results in a decline in translation quality, especially in low latency levels (0 s LAAL < 4). When y is set to higher value (y=0.9), the model imposes stricter constraints on token quality, leading to unnecessary delays. We plan to further explore the impact of in our future work.
Algorithm 1: Confidence-basee Policy In Inference
Input Streaming source prefix x,I = 1,read length n, < (BOS)
Output: Target outputs Y while y # (EOS) do estimate confidence c;; if c 2 0.5 then generate with y
f 1+
else
wait for next source words; update x;
end
end
Table 6: Human evaluation on GPT-generated translations and manually revised translations.
=0.1
0=3
7=0.55
7=0.7
1=0,9
LAAL
SacreBLEU
LAAL
SacreBLEU
LAAL
SacreBLEU
LAAL
SacreBLEU
LAAL
SacreBLEU
3
2.03
20.76
1.99
21.51
1.73
22.51
1.99
22.36
2.04
22.51
3.21
24.89
2.90
24.94
3.11
25.24
7
4.36
26.48
4.36
26.58
4.04
26.76
4.28
26.94
4.45
26.78
6.27
28.17
28.10
5.87
27.97
6.18
28.45
27.81
9.42
29.80
9.08
29.94
9.22
30.13
30.20
20
12.37
30.17
12.35
30.20
12.01
30.05
12.15
30.52
12.46
30.47
ADDITIONAL. IMPLEMENTATION DETAILS
parameerrsmeerrss of the Transformer based SiMT models used in our experiments are shown in Table 7 The hyper-parameters used by SimulPL during the MSFT and SimulDPO phases listed in Table In the training process, we share the LoRA in the MSFT and SimulDPO phases. We use the instruction-followiig format to guide the LIM in completing the SiMT task. Our used prompt
The hyper-
18

---

Published as a conference paper at ICLR 2025
template is shown in Figure 10, For fair comparison between Transformer-batt and LLM-based SiMT models, we following Ma et al. (2020b) to apply greedy search during inference.
F NUMERICAL RESULTS
Tables and respectively present the numerical results of different SiMT models on the Zh En, De- and En -Zh SiMT tasks. In addition to LAAL, also recorded other common latency metrics such as AL (M et al. 2019), DAL (Ma et al. 2020b), and AP (Cho 2016).
G ADDITIONAL ANALYSES ON DE-EN SIMT TASK
In this section, we conduct our analysis experiments on En task. The results are respectively shown in Figure II 12 and [3]
Through these experiments, further validate findings: Both Si and MSFT improve model performance, with more pronounced effect at low latency levels; although the effect is less pronounced in the De-+En task compared the Zh->En task, SimulPL's performance is still influenced by o; SimulPL also generalizes well to other preference optimization methods on the De-+En task.

 <<S>>>>
You are a professional translator.
<<SY S>> S>>
translate the following
<SOURCE LANG> <TARGET LANG>
<OURCE LANG> <SOURCE PREFIX> <IARGET LANG> (/INST] <TARGET PREFIX>
Figure 10: Our prompt template the SimulPL framework and other LLM-based SiMT models.
Only SFT
Sme DP0
(P0
VSFT
Smx
35
10
LMAL
LXAL
The compari- son between different prefer- ence optimization methods on De-+En SiMT task.
Figure 13:
Figure 11: Ablation Stud- ies of SimulPL framework on De- SiMT task.
Figure 12: The comparison of SimulDPO with difference values on De- SiMT task.
19

---

Published as a conference paper at ICLR 2025
Table 7: Hyper-parameters Transformer- based SiMT models in experiments.
Table Hyper-parameters of SimulPL. in our experiments.
Transformer Hyper-parameter
encoder layers
6
encoder attention heads
8
SimulPL Hyper-parameter
encoder embed dim
512
lora_r
64
encoder ffn embed dim decoder layers
1024
LoRA
lora alpha
16
6
Iora dropout
0.1
decoder attention heads decoder embed dim decoder f'nn embed dim
8
512
batch size
1024
micro_ wwto_batch.sste size
32
MSFT
dropout
0.1
learning_rate
2e-4
optimizer
adam
training steps
1000
adam-3
(0.9, 0.98)
0.1
clip-norm
le-7
B batch size
0.1
Ir
5e-4
SimulDPO
scheduler
inverse sqrt
_batch_size
16
varmup-updater
4000
learning -atning-rate rate
2e-6
warmup-init-Ir
1e-7
training steps
400
weight decay
0.0001
label-smoothing
0.1
max tokens
8192
20

---

Published as a conference paper at ICLR 2025
Table 9: Numerical results on Zh-+En SiMT task.
Chinese Chinese-Ennglish
wait-k
BLEU
COMET
LAAL
AL
AP
DAL
1
9.70
56.42
2.02
1.04
0.48
2.53
3
17.88
68.80
4.22
2.92
0.69
4.93
5
21.93
73.90
6,46
4.70
0.88
7.27
7
24.79
75.96
8.48
6.60
1.00
9.34
9
25.73
76.48
10.26
8.54
1.05
11.11
SM²
BLEU
COMET
LAAL
AL
AP
DAL
0.3
21.96
75,43
3.99
2.41
0.75
7.29
0.4
25.87
77.82
5.44
3.65
0.86
9.51
0.5
27.94
78.75
6.80
5.06
0.93
11.73
0.55
28.60
79.06
7.55
5.95
0.97
13.07
0.6
28.83
79.33
8,64
7.17
1.01
14.99
0.65
29.04
79.44
10.23
8.98
1.07
17.33
0.7
29.10
79.55
12.41
11.45
1.13
20.69
LLM-PFX-SFT
BLEU
COMET
LAAL
AL
AP
DAL
3
10.07
71.98
3.96
-13.82
1.24
7.81
5
14.02
76.51
4.30
-14.45
1.12
8.12
7
22.01
78.13
4.59
-4.10
0.85
8.99
26.24
79.39
6.11
0.57
0.84
11.39
15
29.54
80.45
9.21
5.99
0.88
15.20
20
30.08
80.83
12.06
9.53
0.93
18.51
LLM-PFX-SFT DPO
BLEU
COMET
LAAL
AL
AP
DAL
3
12.08
69.39
3.39
-10.22
1.03
6.54
5
17.49
75.95
3.56
-9.87
0.93
7.16
7
24.44
78.44
4.27
-2.35
0.79
8.48
10
27.35
79.84
5.96
2.59
0.80
11.08
15
29.12
81.06
9.14
6.10
0.87
15.12
20
29.82
81.44
12.04
9.45
0.93
18.49
SimulPL
n
BLEU
COMET
LAAL
AL
AP
DAL
3
22.51
75.20
1.73
-3.25
0.68
4.50
5
24.94
77.44
2.90
-1.31
0.70
6.43
7
26.76
78.79
4.04
0.42
0.73
8.32
10
27.97
79.78
5.87
2.42
0.79
11.05
15
29.94
80.85
9.08
6.21
0.86
15.11
20
30.05
81.14
12.01
9.65
0.91
18.49
21

---

Published as a conference paper at ICLR 2025
Table 10: Numerical results on De->En SiMT task.
German->Emm English
wait-k
BLEU
COMET
LAAL
AL
AP
DAL
1
12.77
56.08
0.68
0.06
0.46
1.54
3
31.11
69.52
2.98
1.40
0.87
4.11
5
39.24
76.62
4.98
3.28
1.04
6.14
7
41.42
79.41
7.21
5.05
1.23
8.25
9
42.59
80.36
9.02
6.95
1.32
9.99
SM
BLEU
COMET
LAAL
AL
AP
DAL
0.3
38.76
80.25
4.06
2.43
1.05
7.62
0.4
43.63
82.27
5.21
3.30
1.18
9.27
0.5
44.99
83.22
6.49
4.52
1.29
10.94
0.55
45.79
83.63
7.23
5.25
1.35
11.95
0.6
46.05
83.77
8.06
6.21
1.40
12.97
0.65
46.47
83.99
9.05
7.40
1.45
14.23
0.7
46.82
84.18
10.27
8.85
1.52
15.76
LLM-PFX-SFT
BLEU
COMET
LAAL
AL
AP
DAL
3
19.05
79.46
2.58
-11.35
1.32
4.98
5
40.16
82.95
3.37
-0.16
0.85
6.31
7
44.04
84.25
4.48
1.56
0.86
7.98
48.58
85.47
6.24
3.98
0.90
10.34
15
50.68
86.30
9.22
7.36
0.97
13.65
20
52.13
86.66
12.11
10.56
1.02
16.13
LLM-PFX -AM-PFX-SFT+DOO
BLEU
COMET
LAAL
AL
AP
DAL
3
19.82
79.52
2.46
-9.20
1.25
4.79
5
38.75
82.86
3.16
-0.43
0.85
6.28
7
43.88
84.16
4.49
1.82
0.85
8.03
10
48.90
85.41
6.24
4.16
0.88
10.34
15
50.78
86.18
9.25
7.57
0.95
13.67
20
52.41
86.27
12.13
10.77
1.00
16.14
SimulPL
BLEU
LAAL
AL
AP
DAL
COMET
3
40.81
2.05
-1.43
0.74
4.22
81.15
5
44.57
3.27
0.42
0.78
6.14
83.53
7
46.84
4.45
2.00
0.82
7.90
84.44
10
49.23
6.28
4.12
0.88
10.32
85.41
15
50.86
9.24
7.49
0.95
13.65
86.27
20
52.30
12.15
10.69
1.00
16.13
86.53
22

---

Published as a conference paper at ICLR 2025
Table 11: Numerical results on En +7hh SiMT task.
English Chinese
wait-k
BLEU
COMET
LAAL
AL
AP
DAL
6.37
57.00
0.59
0.36
0.33
1.17
3
26.09
70.28
2.77
1.83
0.75
3.54
5
35.52
76.39
4.80
3.59
0.96
5,52
7
38.46
78.91
6.73
5.45
1.08
7.35
9
40.76
80.00
8.37
7.32
1.12
8.94
SM
BLEU
COMET
LAAL
AL
AP
DAL
0.3
39.46
79.83
4.52
3.38
0.91
6.81
0.4
41.96
81.01
5.31
4.19
0.97
7.77
0.5
42.81
81.64
6.13
5.13
1.01
8.82
0.6
43.27
81.97
7.38
6.53
1.08
10.52
0.65
43.63
82.14
8.30
7.61
1.12
11.92
0.7
43.90
82.22
9.62
9.08
1.16
13.60
LLM-PFX-STE
n
BLEU
COMET
LAAL
AL
AP
DAL
3
7.58
69.86
4.43
-57.94
4.02
7.43
5
30.88
79.61
4.40
-1.91
1.14
7.19
7
40.31
81.87
5.46
3.92
0.92
8.45
10
43.62
82.96
7.29
6.47
0.91
10.55
15
44.47
83.54
10.00
9.48
0.96
13.26
20
44.47
83.58
12.23
11.84
1.00
15.17
LLM-PFX-SFT--DE
BLEU
COMET
LAAL
AL
AP
DAL
3
7.85
69.28
4.42
-54.99
3.87
7.41
5
31.39
79.81
4.26
-0.77
1.04
6.92
7
39.95
81.41
5.47
3.88
0.92
8.48
10
43.21
82.33
7.26
6.45
0.91
10.52
15
44.08
83.05
10.01
9,49
0.96
13.28
20
44.12
83.10
12.23
11.83
0.99
15.17
SimulPL
BLEU
COMET
LAAL
AL
AP
DAL
3
37.26
79.53
2.64
1.24
0.73
4.87
5
40.72
81.51
3.97
2.86
0.79
6.58
7
42.30
81.77
5.32
4.28
0.87
8.33
10
43.86
82.81
7.18
6.37
0.90
10.46
44.83
83.58
9.89
9.33
0.96
13.21
20
45.09
83.68
12.18
11.75
1.00
15.16
23


Reviews:
[{"review_id": "W4HGsqMIjM", "reviewer": "Reviewer_kP65", "summary": "This works present a preference learning framework for simultaneous tasks like SiMT. Particularly, the proposed SimulPL framework categorizes SiMT human preferences into five aspects: translation quality preference, monotonicity preference, key point preference, simplicity preference, and latency preference.", "questions": "1. Since real-time translation is a key requirement of SiMT, LLM-based SiMT may need to re-translate each partial source sentence, as the decoder-only architecture of LLMs does not permit the insertion of additional source tokens without disrupting the positional indexes of the already translated target tokens. I want the authors to justify this issue.\n\n2. Continued on Weaknesses 1, given some GPT-4/4o translations, is that possible to ask the professional interpreter to select the high quality translation? Then a classifier or reward model can be implemented to filter out the translations align most with human.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This is an intriguing question and a useful framework for practical SiMT scenarios. While SiMT has been extensively studied in the research community, most papers have focused on automatic metrics rather than real human preferences. This work proposed 5 human preferences that are closely related to the end-users of SiMT.\n\n2. The proposed optimization objective aligns well with the proposed human preferences. For example, the output length constraint is applied for latency preference. The derived final objective in Eq(9) is further to improve its read/write policy.", "weaknesses": "1. The data construction relies on GPT-4/4o. What is the cost of construction the training data? Does it mean GPT-4/4o is the upper bound of human preference alignment? Or this work is to GPT-4 preference alignment?  Figure 2 only shows the win-rate between GPT-4 translation and original one. Is there a comparison between GPT-4 and professional interpreter?\n\n2. In the experiments, you only tune the hyper-parameter $\\alpha$, and set $c_t$ as 0.5 to control the read/write. In general, tuning the thresholds of $c_t$ may introduce various read/write strategies. Did you try other threshold?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U8xltA4Mp4", "reviewer": "Reviewer_TojP", "summary": "The paper introduces Simultaneous Preference Learning (SimulPL) for the task of Simultaneous Machine Translation. The approach encompasses not only data construction but also model training and inference strategies. However, the novelty of the proposed method appears to be insufficient.", "questions": "please see weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper presents a comprehensive approach to Simultaneous Machine Translation, addressing data construction, model training, and inference.", "weaknesses": "1. The paper lacks a clear emphasis on the innovative aspects of the proposed method. Highlighting specific novel contributions would strengthen the paper.\n\n2. It remains unclear how the quality of the automatic labeling by GPT-4 is ensured, and the consistency rate with human labeling is not provided.\n\n3. The distinction between the uppercase X and lowercase x in equations 6 and 7 needs clarification.\n\n4. According to Figure 6, the improvement of SimulPL over only MSFT is not substantial, raising questions about the effectiveness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4tldYhPmiZ", "reviewer": "Reviewer_qWwc", "summary": "This paper studies the problem of Simultaneous Machine Translation (SiMT). To achieve preference alignment for SiMT models, this paper categorizes human preferences in SiMT scenarios and focus on five aspects: translation quality preference, monotonicity preference, key point preference, simplicity preference, and latency preference. Based on this catogorization, this pape rconstructs human preference prompts with GPT to generate preference data for SiMT. After constructing the data, this paper designs a multi-task supervised fine-tuning stage and a simultaneous direct preference optimization stage to achieve preference alignment for the SiMT model. Experiments on text-to-text SiMT tasks show the effectiveness of the proposed approach.", "questions": "Why most experiments are done between translation of Zh and En? Should possibly incorporate more languages to demonstrate the effectiveness of the approach.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The proposed approach shows performance improvement on the machine translation datasets.\n2. The proposed five aspects for SiMT preference alignment could be useful.\n3. The presentation is clear and easy to understand.", "weaknesses": "1. Heavy reliance on the quality of the GPT-generated preference data, and few effort is done to verify the quality/correctness/trustworthiness of the generated data.\n2. Overall limited novelty for the whole framework, the idea of length penalty term |y| in the proposed SimulDPO loss function come from [1], and nothing much novel is added here.\n3. Scope is very limited to SiMT only, and this method does not seem applicable to more general LLM preference alignment cases.\n4. Some design is arbitrary. For example, in Confidence-Based Policy During Inference, why set the confidence c_t as exactly 0.5? What if your confidence estimation model is biased, e.g., $\\mathbb{E}(c_t)\\neq0.5$?\n5. More experiments and analysis should be done to conduct a comprehensive evaluation. In the current form, experiment analysis are limited.\n\n\nReferences:\n[1] Park, Ryan, et al. \"Disentangling length from quality in direct preference optimization.\" arXiv preprint arXiv:2403.19159 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AzJceDKgvC", "reviewer": "Reviewer_1TVX", "summary": "This paper introduces a new approach to simultaneous interpretation with low latency.", "questions": "Do you have any plans to share your benchmark?\n\nDo you have a github that will make it easy for others to replicate your work?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The experimental results look impressive.\n\nI think there will also be interest in the experimental materials.  Your arguments about offline machine\ntranslation (OMT)  are convincing.  That said, I didn't see any discussion of sharing those materials.\n\nI like the idea of offering the user choices over preferences, but do we need to restrict ourselves to a single choice?  Could you imagine a speech-to-text scenario where you could offer multiple text windows so the user can look at one window when latency is a priority and another window when BLEU is a priority?", "weaknesses": "I have never been that happy with the standard discussion of latency and BLEU scores in this literature.  I have quite a bit of experience with professional interpreters as well as automatic solutions.  According to a number of standard metrics, the automatic solutions are better than professional interpreters, but I know that I am more engaged in the meeting when I have access to an interpreter.  They tell me what I need to know when I need to know it.  In addition, sometimes the speaker uses a metaphor that requires more explanation.    The professionals understand when then need to do more than just translate what the speaker said.\n\nI never liked average latency.  I can't fault you for doing what others have done, but sometimes latency matters and sometimes it doesn't.  The professionals understand this.  They speed up when necessary and slow down when latency is less of a priority.  Moreover, I probably care more about worst case latency or RMS latency than average latency.\n\nThis literature would benefit by including more input from professional interpreters that have a lot more experience in this area than we have.\n\nI would really like to see some data from schools that teach professional interpreters. We should be able to record what students say in the booth when they learn how to do this task.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)