Better Simultaneous Translation with Monotonic Knowledge Distillation
Year: 2023

Abstract:
Simultaneous machine translation (SiMT) presents a unique challenge as it requires generating target tokens before the source sentence is fully consumed. This can lead to the hallucination problem, where target tokens are generated without support from the source sentence. The prefix-to-prefix training data used to train SiMT models are not always parallel, due to divergent word order between the source and target languages, and can contribute to the problem. In this paper, we propose a novel approach that leverages traditional translation models as teachers and employs a two-stage beam search algorithm to generate monotonic yet accurate reference translations for sequence-level knowledge distillation. Experimental results demonstrate the significant improvements achieved by our approach over multiple strong SiMT baselines, leading to new state-of-the-art performance across various language pairs. Notably, when evaluated on a monotonic version of the WMT15 De→En test set, which includes references generated in a more monotonic style by professional translators, our approach achieves even more substantial improvement over the baselines. The source code and data are publicly available for further exploration1.

heading: 1 Introduction
text: Simultaneous machine translation (SiMT) starts to translate with only a partial observation of the source sentence and can present unique challenges compared to full-sentence translation, particularly when employing offline NMT models. Prefix-toprefix (P2P) methods such as the wait-k policy (Ma et al., 2019a) have been developed to narrow the gap between training and inference. However, these methods inherently rely on parallelism at the prefix level, which may not always be present in conventional parallel text.
1 https://github.com/wangshushu0213/
Monotonic-Translation-Generation
Trainset k = 1 k = 3 k = 5 k = 7 k = 9
The parallel text utilized for training offline MT models exhibits a wide range of word reordering between the source and target languages, resulting in non-parallel prefix-to-prefix pairs, as depicted in Figure 1. Table 1 highlights the challenge faced by a wait-k model, which must predict a significant percentage of target tokens without access to the corresponding words in the source prefix across multiple parallel corpora. For example, when training a wait-3 model on the WMT15 De→En dataset, the model needs to anticipate 15.2% of the target tokens during training, exacerbating the hallucination problem during inference.
An alternative approach is to train SiMT models on simultaneous interpretation corpora. However, there are two primary issues. First, the available interpretation training data is scant. Second, due to the real-time nature of simultaneous interpretation, the data tends to be overly simplified, making it less ideal for SiMT models where preservation of information is important. On the other hand, traditional parallel data is abundant. If this data could
2334
be restructured to more closely follow the source word order, it would be more beneficial for SiMT models. This is the idea behind approaches such as (Chen et al., 2021). In line with this direction, we propose a two-stage beam search algorithm to reconstruct the training data, producing accurate yet monotonic translations. This restructured data is then utilized to train the SiMT model using knowledge distillation (KD) (Kim and Rush, 2016).
Similarly, traditional test sets are less ideal for evaluating SiMT models that produce translations in a more monotonic style. To address this, we constructed a new set of human references for the WMT15 De-En test set that more closely follows the source word order. This new reference can provide a more precise measurement of both translation quality and latency in a SiMT setting.
Our primary contributions include:
• We have developed a two-stage beam search algorithm to generate accurate monotonic training data. This algorithm is adjustable for different levels of monotonicity and is capable of leveraging both parallel and monolingual corpora.
• We have curated new human references for the WMT15 De-En test set that is more suitable for evaluating SiMT models. We are pleased to offer these for public access.
• Our empirical results demonstrate that our approach consistently outperforms strong SiMT baselines. We release both code and data to facilitate future research.

heading: 2 Related Works
text: SiMT Policy There are two types of SiMT policies: fixed and adaptive. Fixed policies, such as wait-k in Ma et al. (2019a), first READ k source tokens and then alternately READ/WRITE one token. Elbayad et al. (2020) proposed an efficient multipath training for the wait-k policy to randomly sample k during training.
Adaptive policies make READ/WRITE decisions dynamically. Gu et al. (2016) decides READ/WRITE actions via reinforcement learning. MILk (Arivazhagan et al., 2019) predicts a Bernoulli variable to determine READ/WRITE actions, which is further implemented into transformer architecture MMA (Ma et al., 2019b). Zheng et al. (2020) developed adaptive wait-k through heuristic ensemble of multiple wait-k mod-
els. Miao et al. (2021) proposed a generative framework to generate READ/WRITE decisions. Liu et al. (2021) applies Connectionist Temporal Classification (CTC) by treating the blank symbol as the wait action. Zhang and Feng (2022) develops a READ/WRITE policy by modeling the translation process as information transport and taking the received information as the evidence for READ/WRITE decisions.
Monotonic SiMT Another approach to SiMT is to focus on producing the target as monotonically as possible with the source. Chen et al. (2021) proposed test-time wait-k to produce pseudoreferences which are non-anticipatory. Han et al. (2021) proposed a method of chunk-wise reordering to refine the target sentences in an offline corpus and build a monotonically aligned parallel corpus for SimulMT. Deng et al. (2022) proposed a novel monolingual sampling strategy for SiMT, considering both chunk length and monotonicity. Chang et al. (2022) decomposed the translation process into a monotonic translation step and a reordering step, which rearranged the hidden states to produce the order in the target language. Our method extends (Chang et al., 2022) to include a rescoring stage based on the full sentence to produce more accurate translations.
Knowledge Distillation in NMT Knowledge distillation(KD) approaches (Hinton et al., 2015) aim to transfer knowledge from a teacher model to a student model. Kim and Rush (2016) first applied knowledge distillation to NMT using sequencelevel KD. In terms of online NMT, Zhang et al. (2021b) proposed to use a conventional Transformer as the teacher of the incremental Transformer, and tried to embed future information in the model through knowledge distillation. Ren et al. (2020) proposed to transfer knowledge from the attention matrices of simultaneous NMT and ASR models to a simultaneous speech to text translation system.

heading: 3 Background
text: Offline NMT Offline NMT models typically employ an encoder-decoder framework. The encoder has access to the full source sentence x and maps it into hidden representations. The decoder autoregressively generates each target token yt conditioned on x and the previously generated tokens, as shown in Eq. (1):
p(y|x;θ) = |y|∏
t=1
p(yt|x,y<t;θ) (1)
Simultaneous NMT Simultaneous NMT only has access to part of the source sentence. Let g(t) be a monotonic non-decreasing function of t that denotes the number of source tokens processed by the encoder when generating the target word yt. SiMT uses the source prefix (x1, x2, ..., xg(t)) to predict yt as shown in Eq. (2):
p(y|x;θ) = |y|∏
t=1
p(yt|x≤g(t),y<t;θ) (2)

heading: 4 Monotonic Translation Construction
text: We propose two approaches for creating monotonic pseudo-targets for source sentences in traditional parallel data. This new data is then used to train SiMT models through knowledge distillation (KD).

heading: 4.1 Standard KD
text: A simple approach is to use an offline NMT model as a teacher to translate each source sentence of the parallel training data into a pseudo-target through beam search, as shown in Algorithm 2 in Appendix A. The resulting (source, pseudo-target) data adheres more closely to the source word order, as machine-translated sentences tend to have fewer long-distance reorderings. This data is then used to train SiMT models through sequence-level knowledge distillation (KD) (Kim and Rush, 2016), with the training loss represented in Eq. (3).
Lseq_kd = − log p(ŷ|x;θ) (3)
where ŷ represents the target predicted by the teacher model. Note that this diverges from conventional sequence-level KD training, which also utilizes the training loss over the original references, as the long-distance reorderings in the original data could be detrimental to the SiMT model.

heading: 4.2 Monotonic KD
text: A key drawback of standard KD is that, although the resulting target translations are more monotonic, they still depend on full sentences, and the degree of monotonicity cannot be controlled. To overcome this limitation, we propose a two-stage beam search strategy to produce target translations in a way similar to real-time simultaneous translation, while also preserving the translation quality.
As detailed in Algorithm 1 and depicted in Figure 2, our approach first translates pieces of the source incrementally, akin to a wait-k policy, and then rescores and selects the better partial hypotheses using a full-sentence offline model.
In Stage 1, the streaming source prefix is fed into the offline teacher model to generate the initial b1 partial hypotheses at each beam search step following a wait-k policy. This stage simulates real-time simultaneous translation with incremental input, and ensures that the decoding is based on local information, thereby increasing monotonicity. By defining the desired latency k, the monotonicity level of the partial hypotheses can be controlled.
In Stage 2, we use the teacher model to rescore each of the b1 partial hypotheses conditioned on the full source sentence and only keep the top b2 (b2 < b1) partial hypotheses for the next step in the two-stage beam search process. With this strategy, future information in the source sentence is utilized to improve the quality of top partial hypotheses, while also preserving the local word order dictated by the prefix source.
Note that we can reverse the translation direction and construct more monotonic pseudo-source given the original target through backward translation. However, empirical results show that it is inferior than forward translation for SiMT (see Figure 13 in Appendix E), probably due to the discrepancy between pseudo-source and normal source text.

heading: 5 Experiments

heading: 5.1 SiMT Models
text: We conduct experiments on three representative modeling approaches that have been used for simultaneous machine translation.
Offline MT: a Transformer NMT model (Vaswani et al., 2017) trained on full sentences.
Multipath Wait-k: a wait-k policy model (Elbayad et al., 2020) trained by randomly sampling different k values between batches during training.
ITST: an adaptive read/write policy model (Zhang and Feng, 2022) that formulates the translation process as an optimal information transport problem. To the best of our knowledge, ITST is currently the state of the art method for SiMT.

heading: 5.2 Data
text: We select three datasets of different language pairs that have been used before for investigations of SiMT models.
WMT15 De→En (Callison-Burch et al., 2009) is a parallel corpus with 4.5M training pairs, which
are tokenized and split using 32K BPE merge operations with a shared vocabulary for German and English. We use newstest2013 (3000 sentence pairs) as the development set and report results on newstest2015 (2169 sentence pairs).
CWMT192 Zh→En contains 9.4M sentence pairs in the training set, which are tokenized and split using 32K BPE merge operations for both the source and the target languages. We use the validation set of 956 sentence pairs from BSTC (Zhang et al., 2021a) as the test set.
IWSLT15 En→Vi (Luong and Manning, 2015) contains 133K training pairs. We use TED tst2012 as the validation set (1553 sentence pairs) and TED tst2013 as the test set (1268 sentence pairs). Following the settings in (Ma et al., 2020), we replace rare tokens (frequency < 5) by <unk>. The resulting vocabulary sizes are 17K and 7.7K for English and Vietnamese respectively.
Figure 3 compares AR curves at various k values in both the original and the reconstructed training data with pseudo-targets. Our two KD methods
2 http://nlp.nju.edu.cn/cwmt-wmt/
Algorithm 1: Two-Stage Beam Search Input: x: source sentence
b1: max beam size before rescoring b2: max beam size after rescoring nmax: max hypothesis length k: fixed latency l: source length |x| score(·, ·): scoring function
Output: Best monotonic translation at k 1 // beam format: ⟨score, hypothesis⟩ 2 B0, B ← {⟨0, BOS⟩}, ∅ 3 for i ∈ {1, · · · , nmax} do 4 Bbefore, Bafter ← ∅, ∅ 5 for ⟨s,y⟩ ∈ Bi−1 do 6 if y.last() = EOS then 7 B.add(⟨s,y⟩) 8 continue 9 l = min(i+ k − 1,x.len)
10 for y ∈ V do 11 // score by partial input 12 s← score(x[: l],y ◦ y) 13 Bbefore.add(⟨s,y ◦ y⟩) 14 Bbefore ← Bbefore.top(b1) 15 for ⟨s,y⟩ ∈ Bbefore do 16 // score by oracle input 17 s← score(x,y) 18 Bafter.add(⟨s,y⟩) 19 Bi ← Bafter ← Bafter.top(b2) 20 return B.max()
can effectively reduce the anticipation rate across all language pairs at different k values, with monotonic KD typically resulting in a lower anticipation rate compared to the standard KD. Our experiments are focused on understanding the impact of changes on the translation quality of SiMT models.
To properly evaluate SiMT performance, the test sets should be representative of the characteristics of real-time simultaneous translation, in both content and translation style. In addition to the official test sets described earlier, we choose to adapt the WMT newstest2015 De→En data set for realtime speech translation. We select 500 sentence pairs from this data set and ask professional translators to produce new reference translations, with as much monotonicity as linguistically possible without compromising the translation quality. The detail of this annotation task can be found in the Appendix D.

heading: 5.3 Experimental Setup
text: We use Transformer-base models for the De→En and Zh→En translation directions and Transformersmall mdoels for En→Vi. Our model configurations generally follow the experiment settings detailed in Multipath Wait-k3 and ITST4. For generating pseudo-targets, we use a beam size of 5 in standard KD, and in our two-stage monotonic KD method we set beam sizes b1 = 10 and b2 = 5, with the latency value k set to 7, 7, 6 for De-En, Zh-En, and En-Vi respectively.
For evaluation, we use tokenized case-insensitive BLEU5 for translation quality and Average Lagging (AL, token level) (Ma et al., 2019a) to measure latency.

heading: 5.4 Main Results
text: We first train an offline MT model for each of the three language pairs on the original training data, and then obtain pseudo parallel data and train Multipath Wait-k and ITST models using the regular and monotonic KD methods described in Section 4.
Offline MT Evaluation For each language pair, we train two additional offline models, one for each of the two KD methods. We evaluate these models in both offline and simultaneous scenarios, adopting a simple wait-k policy for the latter. The results6 are presented in Figure 4. The offline mod-
3 https://github.com/elbayadm/attn2d/blob/master/
examples/waitk/README.md 4 https://github.com/ictnlp/ITST
5 https://github.com/moses-smt/mosesdecoder/blob/
master/scripts/generic/multi-bleu.perl 6The results on full sentences (represented by dashed lines) are derived using greedy search. Note that student models trained on KD-produced data can surpass the teacher model in terms of offline BLEU scores. This can be attributed to the fact that the KD data was generated by the teacher model with a beam size of 5. Essentially, the student models are distilled from a teacher model equipped with beam search and thus can perform better than the same teacher model in greedy search.
els perform significantly worse in the streaming scenario, especially when at a low latency, due to the discrepancy between full-sentence training and prefix-to-prefix inference. The two models trained on pseudo-target data exhibit considerable improvements, with an average improvement of more than 2 BLEU points across all latency settings on the De→En test set in particular.
We attribute this improvement to the more monotonic nature of the pseudo data generated through KD. Models trained with this data can better model local source-target relationships, which leads to higher quality translations on partial source inputs. This is reflected in Figure 5, where the mass of cross-attention weights concentrate around the diagonal. Multipath Wait-k We train wait-k SiMT models, following (Elbayad et al., 2020), on the original training data as well as the reconstructed training data with pseudo-target produced by the two KD
However, when both models utilize beam search, the student models are likely to lag behind in performance compared to the teacher model.
methods. As shown in Figure 6, two KD methods are both able to significantly improve translation quality across latency settings. ITST Finally we train ITST models, following Zhang and Feng (2022), to see if our methods can achieve similar improvements with advanced adaptive read/write models. The results are shown in Figure 7. Similarly, we observe overall improvement in translation quality by training ITST models on the pseudo data. As illustrated in the example in Figure 9, the decoding path of the mono-KD trained ITST model is closer to the diagonal and its translation is more faithful and monotonic to the source input.

heading: 5.5 Evaluation on Monotonic Test Set
text: Although the pseudo data constructed by the monotonic KD method has a lower AR, as shown in Figure 3, models trained with the standard KD method typically achieve higher BLEU scores in many cases in Figure 4, 6, and 7. One possibility is that the references in the original test sets were not produced with a focus on simultaneous translation,
and thus can not accurately measure improvement in translation quality of more monotonic translations. To test this hypothesis, we took the first 500 pairs from the De→En test set and commissioned a new set of reference translations that are as monotonic as possible without sacrificing the translation quality. We re-evaluated our De→En models on this monotonic test set and the results are shown in Figure 8. Compared to the previous results on the original test set, the improvement from the monotonic KD method becomes more prominent, on par with the standard KD method or in many cases outperforming. Moreover, the overall improvement from the KD methods also becomes greater on this monotonic test set. Although the monotonic test set is only a subset of the original test set, the same conclusion holds when only comparing results on this subset (see performance of the multipath waitk method on the original subset in Figure 14 in Appendix E).

heading: 5.6 Scaling with Monolingual Data
text: Given that only source sentences are needed for an offline teacher model to produce pseudo-targets, we can expand the KD training data by generating pseudo-targets using monolingual data. We conducted experiments on WMT15 De→En and collected 1 and 4 times of additional pseudo parallel data using the monotonic KD method on German sentences selected from News Crawl articles, excluding sentences longer than 190 characters. The results with the multipath wait-k model are presented in Figure 10. The improvements from more pseudo data suggest that the ability to use a monolingual source corpus is another advantage of our approach.
In Figure 11, we focus on WMT15 De→En and demonstrate how our approach can further advance the current state of the art in SiMT. We take ITST, the current SOTA in SiMT, as our modeling method, and compare with ITST and another recent SiMT method wait-info (Zhang et al., 2022a). For a
fair comparison, we rerun the original ITST and observe a minor performance dip under high latency conditions. The results show that the monotonic KD method combined with additional monolingual data can achieve new state of the art for SiMT.

heading: 5.7 Effects on Hallucination
text: Hallucination, a known issue in machine translation models, presents significant challenges for realtime simultaneous translation. Hallucination Rate (HR%) (Chen et al., 2021) measures the percentage of words in the target output that are hallucinated (see full definition in Appendix C). We compare the HR% of multipath wait-k models trained on the original parallel data or the pseudo data constructed by the KD methods. As shown in Table 2, the monotonic KD method has the lowest HR% across different latency settings. Examples of hallucination in translation results can be found in Table 6 of Appendix E.

heading: 6 Discussions
text: The first beam search stage of our monotonic KD method is equivalent to test-time wait-k inference described in (Chen et al., 2021). This stage, however, may fail to produce accurate rankings of partial hypotheses, given that it relies on offline models for translating partial inputs. The second stage beach search, designed to incorporate full sentence
information, is capable of more accurately scoring and ranking these partial hypotheses. We conducted an analysis on the WMT15 De→En test set to compare the quality of translations produced by test-time wait-k (i.e., monotonic one-stage beam search) and our monotonic two-stage beam search. As shown in Table 3, the rescoring process in the second stage significantly improves translation quality.
Table 4 shows the quality of pseudo-targets generated by standard KD, monotonic one-stage beam search, and monotonic two-stage beam search, measured in BLEU with respect to the original references. Across both De→En and En→Vi, the standard KD achieves the highest BLEU scores, closely followed by the monotonic KD method that uses two-stage beam search. The one-stage only beam search method results in the lowest translation quality among the three approaches, particularly on De→En where the BLEU score is 4 points lower. Figure 12 illustrates the performance of multipath wait-k models trained on the respective training data. The two-stage method consistently outperforms the one-stage method on De→En and is better in most latency settings on En→Vi. It is notable that the one-stage method leads to substantially inferior SiMT models on De→En due to the markedly lower quality of the pseudo-targets.

heading: 7 Conclusion
text: Long-distance reorderings in conventional parallel data can negatively impact the training of simultaneous translation models. To address this problem, we propose a novel two-stage beam search algorithm to generate monotonic yet accurate pseudo translations that are then used to train SiMT mod-
els through sequence-level knowledge distillation. Experiments on three language pairs demonstrate that this method can consistently improve multiple SiMT models and achieve new state of the art performance for simultaneous translation.
Limitations
Our monotonic KD approach requires searching for a hyper-parameter k to strike a balance between monotonicity and translation quality for generating pseudo-targets. The current process requires substantial computational resources to determine the optimal value, which may be different depending on the dataset. More studies are needed to establish an efficient method.

heading: Acknowledgements
text: We would like to thank all the anonymous reviewers for the insightful and helpful comments. This work was supported by Alibaba Research Intern Program, the National Key Research & Development Project of China (2021ZD0110700), the National Natural Science Foundation of China (U19B2043, 61976185), and the Fundamental Research Funds for the Central Universities (226- 2022-00051). This work was done during the first author’s internship at Alibaba DAMO Academy.

heading: A Algorithm of Standard Beam Search
text: Algorithm 2: Standard Beam Search Input: x: source sentence
b: max beam size nmax: max hypothesis length score(·, ·): scoring function
Output: Best hypothesis 1 B0 ← {⟨0, BOS⟩} 2 for i ∈ {1, · · · , nmax} do 3 B ← ∅ 4 for ⟨s,y⟩ ∈ Bi−1 do 5 if y.last() = EOS then 6 B.add(⟨s,y⟩) 7 continue 8 for y ∈ V do 9 s← score(x,y ◦ y)
10 B.add(⟨s,y ◦ y⟩) 11 Bi ← B.top(b) 12 return B.max()

heading: B Anticipation Rate of (Pseudo-)Refs
text: During the training of a simultaneous translation model, an anticipation happens when a target word is generated before the corresponding source word is encoded. To identify the anticipations, we need the word alignment between the parallel sentences.
We use fast-align in our experiments (Dyer et al., 2013) to get a word alignment a between a source sentence x and a target sentence y. It is a set of source-target word index pairs (s, t) where the sth source word xs aligns with the tth target word yt. Formally, a target word yt is k-anticipated (Ak(t, a) = 1) if it aligns to at least one source word xs where s ≥ t+ k:
Ak(t, a) = 1[{(s, t) ∈ a|s ≥ t+ k} ≠ ∅]
The k-anticipation rate (ARk) of an (x, y, a) triple is further defined under wait-k policy:
ARk(x,y, a) = 1
|y|
|y|∑
t=1
Ak(t, a)

heading: C Hallucination Rate of Hypotheses
text: HR is defined to quantify the number of hallucinations in decoding. A target word ŷt is a hallucination if it can not be aligned to any source word.
Formally, based on word alignment a, whether target word ŷt is a hallucination is
H(t, a)=1[{(s, t) ∈ a} = ∅] (4)
Hallucination rate HR is further defined as
HR(x, ŷ, a)= 1 |ŷ| ∑|ŷ| t=1 H(t, a)

heading: D WMT15 De→En Test Set Annotations
text: In order to properly evaluate the quality of SiMT, we expect to remove the long-distance reorderings in the test set. So we ask the professional interpreters to rephrase the references in the test set of WMT15 De→En into simultaneous style. We hired two profession interpreters and spent 888 US dollars in total to get the monotonic test set. The annotation guidelines we provided with them are as follows:
• A monotonic translation should be faithful and fluent, following common practices in professional translation of sentences, without adding, deleting, or substituting meaningful information in the source sentence. The original professional translations are provided for reference only and annotators should feel free to start from scratch, or reuse the original translation and make necessary edits, in order to produce a monotonic translation that is faithful and fluent.
• A monotonic translation should reduce long distance reordering between words and try to emulate the word order in the source language if possible, under the requirement of criterion 1.
• While it can be difficult and time-consuming to come up with the best monotonic translation for a source sentence, we require reasonable effort to create a more monotonic translation that is quantitatively better than the original translation according to criterion 2, unless the original translation is already monotonic.
• There may exist multiple monotonic translations for a source sentence with varying degrees of monotonicity. We require reasonable effort to create a more monotonic translation but it does not need to be the most monotonic translation. We welcome diversity in monotonic translation and would collect multiple versions of monotonic translations from different in-house and external professional translators.

heading: E Additional Training Details and Experimental Results

heading: F Numerical Results
text: The numerical results of the main SiMT systems are presented in table 5 and table 7.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitations
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Section1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
3 B1. Did you cite the creators of artifacts you used? Section1,3
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Section5
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section1
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Section5
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Section5
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section5
C 3 Did you run computational experiments? Section5
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Section5
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section5
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section5
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section5
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section5
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Appendix
3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Section5
3 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Section5
3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Section5, Appendix

heading: Future Work
text: With this strategy, future information in the source sentence is utilized to improve the quality of top partial hypotheses, while also preserving the local word order dictated by the prefix source. Note that we can reverse the translation direction and construct more monotonic pseudo-source given the original target through backward translation. However, empirical results show that it is inferior than forward translation for SiMT (see Figure 13 in Appendix E), probably due to the discrepancy between pseudo-source and normal source text