Addressing the Length Bias Challenge in Document-Level Neural Machine Translation
Authors: Zhang Zhuocheng
Shuhao Gu
Min zhang
Yang Feng
Venue: EMNLP 2023 Findings (2023)
Keywords: Document
Machine Translation
Length Bias

Abstract: Document-level neural machine translation (DNMT) has shown promising results by incorporating context information through increased maximum lengths of source and target sentences. However, this approach also introduces a length bias problem, whereby DNMT suffers from significant translation quality degradation when decoding sentences that are much shorter or longer than the maximum sentence length during training, i.e., the length bias problem. To prevent the model from neglecting shorter sentences, we sample the training data to ensure a more uniform distribution across different sentence lengths while progressively increasing the maximum sentence length during training. Additionally, we introduce a length-normalized attention mechanism to aid the model in focusing on target information, mitigating the issue of attention divergence when processing longer sentences. Furthermore, during the decoding stage of DNMT, we propose a sliding decoding strategy that limits the length of target sentences to not exceed the maximum length encountered during training. The experimental results indicate that our method can achieve state-of-the-art results on several open datasets, and further analysis shows that our method can significantly alleviate the length bias problem.

Addressing the Length Bias Problem in Document-Level Neural Machine Translation
Zhuocheng Zhangh: Shuhao Gu Min Zhang', Yang Feng!..
Key Laboratory of Intelligent Information Processing,
Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS)
?University of Chinese Academy of Sciences, China
School of Future Science and Engineering, Soochow University, China
alangzhuocheeng22eeicc...cca @hugaog511@gmail..oo com
zhangminntt@hotmaill
fengyang@ict.accoo acc.aangicct.ac.cr
Abstract
30
Document-level neural machine translation (DNMT) has shown promising results by incor- porating more context information. However, this approach also introduces length bias prob- lem, whereby DNMT suffers from significant translation quality degradation when decoding documents that much shorter or longer than the maximum sequence length during train- ing. solve the length bias problem, we pro- pose to improve the DNMT model in training method, attention mechanism, and decoding strategy. Firstly, we propose sample the train- ing data dynamically to ensure more uniform distribution across different sequence lengths. Then, introduce length-no norgglieed atten- tion mechanism to aid the model in focusing on target information, mitigating the issue of attention divergence when processing longer sequences. Lastly, we propose a sliding win- dow strategy during decoding that integrates as much context information as possible without exceeding the maximum sequence length. The experimental results indicate that method can bring significant improvements on sev- eral open datasets, and further analysis shows that method significantly alleviate the length bias problem'.
25
(-8150
20
15
Training Length: 64
Training Length: 128
10
Training Length: 256
Training Longth: 512
0
200
400
600
800
1000
Decoding Length
Figure 1: The length bias problem for doc2doc DNMT model. The translation quality degrades significantly as the decoding length deviates from the training length.
2019; Liu al., 2020; Bao al., 2021; Sun al., 2022b), which expands the translation scope from individual sentences to entire documents, has demonstrated exceptional performance, thereby drawing increased attention. For the training doc2doc DNMT model, multiple sentences are assembled into sequences that are close to the predetermined maximum length, enabling the model to learn information from the context much as possible. However, this training strategy can lead to overfitting the maximum length. Sequences that are significantly shorter than the maximum length may be overlooked by the model due to their smaller proportion in the training set. Besides, the model also lacks the ability to handle the sequences that exceed the maximum length, which are not encountered by the model during training. Consequently, the length bias problem results in a significant degradation in translation quality when the length of the decoded sequence deviates from the maximum sequence length, which shown in Figure 1.
1 Introduction
Document-level neural machine translation (DNMT) (Gong et al., 2011; Hardmeier et al., 2013; Garcia et al., 2015; Miculicich et al., 2018; Tan et al., 2019, 9;; Maruf et al., 2019; Zheng et al., 2020; Xu et al., 2020) is proposed to enhance translation quality by leveraging more contextual information. Recently, the document-to-docnee (doc2doc) DNMT model (Junczys-Dowmunt,
"Corresponding author.
'Equal contribution.
Codee is at https//gghhubhccmm//cmp//loooggp

---

Some researchers have made their attempts enhance the length generalization capabili- ties of DNMT model from various perspectives. Some approaches employ data augmentation tech- niques (Junezys-Dowmunt, 2019; Sun et al., 2022b) to mix documents with shorter segments such as sentences paragraphs, thereby augmenting the diversity of sequence lengths in the training set. However, the proposed augmentation method does not necessarily guarantee balanced length distri- bution, the length distribution is still influenced by the training corpus itself. Bao et al. (2021) in- corporates a locality assumption as an inductive bias into the Transformer model, which reduces the complexity of target-to-souree attention. As result, their method allows for the setting of larger maximum lengths, thereby augmenting the model's i ability to handle longer documents. How- ever, this method can only bring limited improve- ments for the short sequences. Besides, the afore- mentioned approaches are still incapable of directly handling sequences that exceed the maximum se- quence length during testing and still require seg- mentation of excessively long test sequences.
combined application yields further performance enhamcements. We conduct experiments on several document-level open datasets and the experimental results indicate that our method can bring signif- icant improvements. Further analysis shows that our method can significantly alleviate the length bias problem.
2 Background
In this section, we will give a brief introduction to the Transformer (Vaswami et al., 2017) model and the doc2doc DNMT model.
2.1 The Transformer
The transformer model is based on the encoder- decoder architecture. The encoder is composed of N identical layers. Each layer has two sublay- ers. The first is multi-head self-attention sublayer, and the second is fully connected feed-forward network. Both of the sublayers are followed by a residual connection operation and a layer normal- ization operation. The decoder is also composed of N identical layers. In addition to the same kind of two sublayers in each encoder layer, the cross- attention sublayer is inserted between them, which performs multi-head attention over the output of the encoder.
Given above, we aim to enhance the capabil- ity of our model to handle both long and short se- quences. Additionally, we seek to enable the model to directly translate sequences that exceed the max- imum length, thereby avoiding information loss caused by segment truncation. To achieve these objectives, have made improvements in the sam- pling of training data, attention weight computa- tion, and decoding strategies. During training, we first sample the sequence lengths and then construct the training data accordingly. This dynamic vari- ation in sequence lengths within different epochs ensures that the model encounters more balanced distribution of sequence lengths during training. Furthermore, we introduce scaling factor during attention computation to ensure that, even as the sequence length increases, the model can still focus on relevant target information and prevent attention divergence. Lastly, when decoding sequences that exceed the maximum length, we employ sliding window decoding strategy which allows for the reiention of more context information while en- suring that the context length remains below the maximum sequence length. These three proposed methods collectively contribute to improving the length generalization capabilities the DNMT model from different perspectives. Moreover, their
The attention mechanism is the core part of the Transformer model, which is computed as:
QK Vdz
Attention Q, K, V) softmax
V
(1)
where Q K and V represent the query, key, and value vectors, respectively. de denotes the dimen- sion of the key vectors. The softmax function is applied to normalize the dot-product similarities between the queries and keys, and the result mul- tiplied by the value vectors obtain the weighted sum.
2.2 The doc2doc DNMT model
The doc2doc DNMT model is to translate the whole document directly. Different from the conventional DNMT model, which translates documents sen- tence by sentence with an additional context en- coder (Tan et al., 2019; Maruf et al., 2019; Yang et al., 2019; Zheng et al., 2020; Xu et al., 2020; Yun al., 2020), multiple sentences will be simul- taneously input into the doc2doc DNMT model for training and decoding. The training data consists of different documents D = Ut {d:). where n

---

Seqquece Leggh
Documents
15
25
8
Sequence Longth
Sequercc Longtn
Sequence Length
Samsing Probabity
Sequence Length
Sequence Longth
Sampled Length
35
12
Sequeecee Length?
Sequence Langth
Segmented Sequence
24
25
8
Figure 3: An example of the segmented sequences.
sequences, directly learning document-level trans- lation is more difficult. Hence, in the initial stages of training, we focus more on training the model on shorter sequences, which improves training stabil- ity and accelerates model convergence. As training progresses, hope increase the probability of sampling longer sequences or documents, allow- ing the model to gradually learn longer contextual dependencies.
Traning Ezoch
Figure 2: An example of the sampling probabilities of different sequence lengths during training.
denotes the number of documents in the training data. Each document dj contains source and tar- get sentences d; = U (i))) where m denotes the number of sentences each document. Besides, special symbols are often inserted within the documents to distinguish between different sen- tences. The training objective can be written
Following the above intuitions, we define the sampling probabilities of different lengths as:
W
b
PI
(3)
ΣΣΣ
Pw X1, Niszy) (2)
Σ
arg max
w
where L denotes the maximum sequence length and wy denotes the sampling weight assigned for different lengths which is defined as = e-!. T is a sampling temperature (Arivazhagan et al.. 2019), which is computed as T = c(ep-2), where ep denotes the current epoch number and y is a hyperparameter, which should be adjusted accord- ing to the dataset. The temperature T varies with the training epoch. An example of the sampling probabilities of different sequence lengths during training is shown as in Figure 2, where is set as 5 and max length is set as 8. We can see from the figure that in the initial stage of training, the probability of short sequence length being sampled is relatively high. In the later stages of training, the probability of different sequence lengths be- ing sampled tends to be equal. Although real training processes, the maximum sequence length is typically much greater than 8, the pattern of the sampling probabilities follows similar trend.
where ly denotes the number of the words in the j-th target sentence of the i-th document. During decoding, documents that exceed the maximum sequence length also segmented, otherwise it will result in significant decrease in translation quality.
3 Method
Our method aims to enhance the length general- ization capabilities of the doc2doc DNMT model, thereby alleviating the length bias problem. To achieve this goal, we have made improvements in three aspects: training data sampling, attention computation, and decoding strategies.
3.1 Dynamic Length Sampling
Dynamic length sampling (DLS) aims to ensure that the model has the opportunity to encounter training sequences of various lengths throughout the training process, thereby facilitating better learning and retention of the ability to translate sequences of different lengths. Therefore, the key challenge of this method lies in determining the sampling probabilities for different sequence lengths. Given that translating complete docu- ments usually involves longer input and output
Specifically, before the training of each epoch begins, first update the probability of each quence length being sampled according to Equa- tion 3. Then, for each document d, in the train- ing set, we sample different sequence lengths [a, .2, .... We segment the documents d into different sequences Sis from left to right

---

Source Sentence
Target Sentence
Context Window;
Contect Window
Context Window
Context Window
Source Sequence
Target Sequence
Dugur
Figure 4: An illustration of the sliding decoding strategy. We set the window size as three sentences in this example for display convenience.
if the segmented length is shorter than the sam- pled length. But if the sampled sequence length is less than the current sentence length of the doc- ument, we will select the current single sentence the input sequence. This overall process can be demonstrated
where denotes the length the attended sequence and s. denotes the average length of sequences the current training epoch. Because the sequence lengths are sampled per epoch by DLS, also changes gradually. It can be demonstrated that incorporating the aforementioned length scale ef- fectively mitigates the issue of entropy divergence in attention results when dealing sequences with different lengths. We have included the proof pro- cess A. During decoding, i set as the value corresponding to the final epoch of the training phase.
Sik
=[Xi,ab: (Xi,a.b.Y,aab),
Xxaabl S Laillyiaal S Like a <<
(4)
s.t.
or
Xx.abll > Vat:/V
Like a b
Different sequences Sy don't overlap with each other. We show an example of the above process in Figure 3. In this example, the document has 4 sentences, with lengths of 15, 25, and 8, re- spectively. The sampled lengths are 35, 1, and 12, respectively. Therefore, the final input sequence we obtained contains three segments, with lengths 24, 25, and 8, respectively. After processing the entire training set, we can obtain the sequences required for the current epoch.
3.3 Sliding Decoding
During training, the DNMT model needs to set a maximum sequence length. However, during the decoding phase, it often encounters documents that exceed this maximum length. Directly decoding such long documents can lead to inferior results, as the model has not been exposed to documents exceeding the maximum length during training. A common approach is to split the long document into shorter segments and translate them separately. subsequently concatenating the translation results. However, such segmentation may result in the loss of contextual information, thereby affecting trans- lation quality.
3.2 Length Aware Attention
The role of the attention mechanism is to retrieve in- formation from the target sequence that is relevant to itself (self-attention) or the current translation (cross-attention)). However, the DNMT model usu- ally needs to handle wide range of context except for the target sequence, which may interfere with the normal operation of the attention mechanism, leading to the divergence of the attention results and consequently deteriorated translation quality for long segments.
To address these issues, we propose a method that utilizes a sliding window for decoding (SD). Specifically, when the length the input sequence smaller than the maximum length, the complete sequence, including the target sentence and the context, is used for translation. However, when the input sequence exceeds the maximum length, we discard the oldest source-side context information from the current time step onwards and no longer employ assist in translation. Simultaneously. the corresponding oldest target-side context infor- mation is also discarded, but will be preserved as part of the translation result. The illustration
Inpired by Chiang and Cholak (2022), we pro- pose the length aware attention (LAA), which adds scaling factor to the original attention computa- tion (Equation to mitigate this issue:
QK
Attention softmax
logl) V,
Vdk

---

of the overall process is shown in Figure 4. If employ beam search stralegy during the decoding process, we retain the candidate with the highest generation probability within the current beam for output and subsequent decoding.
HAN (Tan et al., 2019): This method employs a hierarchical attention mechanism in Trans- former to capture contextual information at sentence-level and word-level.
Flat (Ma et al., 2020): This methods feeds the concatenated sentences into pre-trained BERT to collect the contextualized represen- lations of the sentence being translated.
4 Experiments
4.1 Data Preparation
We conduct experiments on 3 most commonly used English German (En+Dee) translation datasets. The description of the datasets are follows:
LED (Beltagy et al.. 2020); The proposed model in this method is equipped with well designed sparse attention mechanism. We re- produce this by using transformers'.
TED is provided by IWSL72017 (Cettolo et al., 2012), containing talks from TED. We adopt 242016-2017 as the test sets, and the rest for the valid sets.
Doc-Trans (Zhang et al., 2018): This method introduces a new context encoder to represent document-level context. They also propose a two-step training approach to effectively uti- lize abundant sentence-level parallel corpora.
News contains parallel documents extracted from NewsCommentary in domain In our experiments, newstes12015 and new- stest2016 are used for validation and test, re- spectively.
G-Transformer (Bao al., 2021): This method incorporates a locality assumption as an inductive bias into the Transformer model. We train the model with the document-level corpus from scratch (G-Trans) and also pre- train the model with sentence-level corpus and then fine tune the model with the document- level corpus (G-Trans-FT).
Europarl is extracted from Europart v7 (Koehn, 2005) and split using SPEAKER tags. We follow the train/develop/test sets spliting as Maruf et al. (2019).
We use the Moses toolkit (Koehn al., 2007) tokenize other languages. Besides, integrating operations of 32K is performed learn BPE (Sen- nrich et al., 2016). Following Bao et al. (2021); Liu et al. (2020), we set the maximum sequence length as 512 in our main experiments.
MR (Sun et al., 2022b); This method splits each document averagely into different parts for multiple times and collect all the se- quences for training.
ALiBi (Press al., 2021): This method im- proves length generalization by adding static non-learned bias to attention weights. We train the model with the document-level cor- pus from scratch (ALiBi) and also pretrain the model with sentence -eeeen corpus and then fine tune the model with the document-level corpus (ALiBi-FT).
4.2 Systems
The systems used for comparision in our experi- ments are as follows:
Transformer (Vaswani et al., 2017): We have obtained three systems with different train- ing methods based on the Transformer model. The Trans-sent model is trained with the sentence-level corpus. The Trans-doc model is trained with the document-level training corpus. The Trans-FT model fine-tuned based on the Transformer-sen model with document-level corpus.
We applied the proposed methods, including dynamic length sampling (DSL), length aware attention (LAA) and sliding decoding (SD), to the Transformer- doc model (Trans-doc+Ours) and the G- Transformer model (G- Trans+Ours), respec- tively.
Our System:
http:://www..csmaaattcu/ooo/opo/ commentary.html

---

TED
Europari
News
Models
8-BLEU
d-BLEU
S-chuff d-chaF
s-BLEU
d-BLEU
s-cluF d-chaF
s-BLEU
d-BLEU
s-chuff d-cluF
Trans sent
24.12
28,02
30.33
32.45
24,91
26.94
Trans-doc
18.51
25.20
30.86
33.11
21.11
24.02
HAN
23.79
28.17
30.74
32,90
24.22
26.31
30,92
33.04
LED
18.46
24.29
29.90
32.48
12.13
16.63
TranTrran
23.81
27.64
30.74
32.88
24.79
26.77
G-Truns
22,53
25,90
32,02
14.14
23.87
25.90
MR
23,99
28.61
53.73
70.54
31.54
33.75
61.36
69.83
24.79
27.14
54.06
64.33
ALBBi
19.65
26.30
47.22
68.26
29.99
32.54
60.19
69.28
12.67
22,83
36.19
59.91
ALiBi-FT
20.85
27.55
29.65
32.55
69.41
49
Trans-FT
54.70
31.16
33.58
69.91
27.33
53.27
Trans-doc DLS LAA
24.93
28.95
55.18
70.69
31.85
34.32
61.26
70.01
24.57
28.52
53.13
65.47
Trans-doc Our method
24.60
28.43
55.16
70.55
31.63
34,33
60.91
69.93
23.72
27.95
52.50
65.20
G-Trins-FT
25.07
28.86
55.65
70.79
32.38
34,51
62.15
70.32
25.87
27.82
55.71
65.14
G-Trans+ DLS LAA
25.37
29.07
55.76
70.81
32.67
34.81
62.06
70.23
26.70
28.66
55.96
65.24
G-Trans Our method
24.87
28.53
55.34
70.51
32.67
34.82
61.99
70.17
26.56
28.52
55.78
65.09
Table
The experimental results of our proposed method on TED, Europart and News. The best score are shown bold. For our proposed Trans-doc Our and G-Trans Our, the documents translated full unit without segmentation, while for other methods, the documents are segmented according to the maximum sequence length of 512.
Components
Scores
4.3
Main Results
ID
DLS
LAA
SD
S-BLBU
d-BLEU
S-cheF
d-chrF
V
V
31,63
34.33
60.91
69.93
During decoding, the test documents with a length less than the maximum length will be directly in- put into the model. Documents with a length greater than the maximum length will be seg- mented into several shorter sequences accord- ing to the maximum length and input into the model separately (Liu et al., 2020). We gener- ate the translations with beam size of and length penalty
2
V
V
X
31.85
34.32
61.26
70.01
3
*
X
31.86
34.21
61.33
69.84
V
X 32.06
34.37
61.52
69.94
X
31.36
33.72
61.26
69.93
X
31.16
33.58
61.29
69.91
Table 2: The ablation study of our proposed method. We conduct ablation study Europarl with Trans-FT. The marker and indicate the component involved and not involved, respectively. The marker * indicate the length sampling is applied without dynamic adjusting the temperature.
We use the SacreBLEU tool (Post, 2018) to evaluate the output with s- BLEU (sentence BLEU) (Papineni et al., 2002), d-BLEU (document BLEU) (Liu et al., 2020), s- chrF (sentence-chrF) (Popović, 2015) and d-chrF (document-chrF)), respectively. To make our exper- imental results comparable with previous studies (Sun al., 2022b), our BLEU scores are calculated in a case-insensitive manner.
Implementation Details All the systems are implemented as the base model configuration in Vaswani et al. (2017). We train our system on 4 NVIDIA 3090 GPUs by using Adam (Kingma and Ba, 2017) optimizer. Most training param- eters are kept the same with Bao et al. (2021), where the learning rate Ir = 5e - 1, 31 = 0.9, 88 = 0.98. The warmup step is set to 4000 and the label smoothing (Szegedy et al., 2015) value is set to 0. The dropout ratio is set to 0.3 TED and News, and 0.1 on Europarl for its larger scale. During decoding, we set the context window 0.8 of the maximum sequence length used during the training phase to prevent performance degradation caused by overly long target sequences.
The main results are shown in Table 1. In the En-+De translation task, our method outperforms the majority of the comparative systems, when in combination with the conventional doc2doc DNMT (Trans-oo=+DLS+LAAA) Furthermore, our proposed Mlas--doo+LS+LLA. achieves perfor- mance comparable to the best-performing com- parative system G-Trans-FT and surpass MR by large margin. Further improvements observed when integrating our method with G-transformer (G- DG-Ta===DLS+LLA) and it can achieve state- of-the-art performance on datasets. However, when combined with the slide decoding strategy. the performance drops slightly. We suspect that

---

this may be due to the error accumulation problem. On the other hand, the slide decoding strategy aims to solve the length extrapolation problem, and we further explore its advantages in Section 5.2.
and measured their corresponding d-BLEU scores. The results are presented in Figure The results indicate that the baseline system and many com- parison systems experience a significant decrease in d-BLEU score when the decoding length devi- ates from the maximum length used during training (512). In contrast, our method exhibits no signifi- cant decrease in BLEU score. This demonstrates that our approach can enhance the length general- ization performance of the doc2doc model and al- leviate the issue of length bias. In particular, when the decoding length exceeds the training length, the performance of the existing methods suffers from a huge drop. while our proposed slide decoding is able to maintain the high translation quality.
5 Discussion
5.1 Ablation Study
To further understand the impact of each step of our method, we perform further studies by remov- ing certain parts of our method. The results are given in Table 2. Upon comparing the performance of systems 2 and 6, it is evident that removing dynamic length sampling (DLS) significantly de- teriorates the model's performance. This obser- vation validates the importance of DLS and LAA in enhancing the system's performance. Further- more, comparing the results of system 5 and Length Aware Attention (LAA) also demonstrates a notable improvement, indicating that our method can effectively capture the contextual information. Lastly, when comparing system and system 4, we find that dynamic adjust the temperature can fur- ther improve the performance without the need for fine-tuning. Therefore, the above results provide evidence that all of our methods are effective in enhancing the performance of doc2doc DNMT. In addition, to verify the effectiveness of our proposed Length Aware Attention (LAA), we con- ducted statistical analysis of the average entropy of the attention mechanism when translating sen- tences and documents of length 512. As shown in 4, it can be observed that the entropy of the at- tention mechanism is more stable after applying the LAA method, which indicates that after apply- ing the LAA mechanism, the model demonstrates better consistency in handling sentence- level and document-level text. Furthermore, by applying the DLS and LAA method, the entropy of attention when translating the document is lower than that of the FT method, indicating that the model concen- trates more the long-range contexts.
To further comprehend why our approach can enhance the length generalization performance of the model, we perform a visualization of the length distribution of the training data. We visualized the length distribution of the original corpus used for training, the data employed by the MR method, and the data used in the final epoch after incorporating DLS. The results are presented in Figure 6, which demonstrates that our approach achieves a more uniform length distribution. Consequently, our method has the capacity to improve the length gen- cralization performance of the model to a greater extent.
5.3 The Discourse Phenomena
To investigate the translation of discourse phenom- ena, we conduct experiments on ContraPro test suite (Müller et al., 2018), a large contrastive test suite extracted from OpenSubtitles 2018 (Lison and Tiedemann, 2016)., to measure the translation accu- racy of English pronoun "it" into the corresponding German translations "er", "sie" or "es". We employ Europarl as the training set, and the maximum se- quence length is setting to 512. As shown Table 3, compared to random selection, the sentence-level translation model has the ability to infer a portion of the correct answer based on the information within the sentence. However, with the help of contextual information, document-level neural machine trans- lation models outperforms sentence-level basclinc by large margin. Utilizing our proposed DLS and LAA, the error rate of Transformer-doc is further reduced, indicating that our approach can further enhance the capability of the model to capture the contextual information.
5.2 Length Generalization
The main motivation of our approach is to en- hance the length generalization performance of the doc2doc DNMT model, thereby addressing the is- sue of length bias. To assess the effectiveness of our method in achieving this goal, we conduct fur- datasets. We decode the test set using the systems in the main experiments with different maximum lengths
ther analysis based on the English-

---

TED
News
Europar
30
28
25
26
G-BLEU
20
32
24
15
GTrans-FT
CTrans-DLS-LMM-SD
22
10
Transformer-FT
20
10a
10
10
10a
10e
10a
Sequence Length
Sequence Langth
Sequence Length
Figure 5: The length generalization of the different methods. We represent method using solid lines while the baseline mathod using dashed lines.
Baseline
Mulli Resolution
Dynamic Length Sampling
15.0
Parcentage
10.0
5.0
2.5
0.0
100
200
100
200
300
400
500
Length
Length
Length
Figure The length distributions of the training data used by different training strategies. distributions at the maximum length equal to 512.
We collect these
Related Work
Method
ContraPro ACC(%)
6.1 Document-Level Neural Machine Translation
Random
33.33
Trans-sent
52.00
Document-level neural machine translation can be broadly divide into two categorics, in- cluding sentence-to--setennc (sen2sen) approach and documenn-to-docuumel (doc2doc) approach (Maruf et al., 2021). The former feed the con- text as additional information to assist the transla- tion of each single sentence in the document inde- pendently, which is also known as multi encoder method (Lupo et al., 2022). Jean al. (2017) lever- aged additional attention to capture the previous context; Kuang and Xiong (2018) proposed to con- trol the usage of context by gate function; Wang et al. (2017), (Miculicich et al., 201 8), and (Zheng et al., 2020) introduced hierarchical attention net- works to model the contextual information from the documents; Maruf al. (2019) and Martins and Astudillo (2016) designed selective atten- tion network to extract most useful information from the massive context; Yang et al. (2019) pro- posed a query query-guided capsule network to further
Trans-FT
70.58
Dras--A
72.28
Table The results of ContraPro test suit, measured by accuracy.
Method
sentence
document
8
Trans-FT
2.65
4.0
1.35
Trans-DI..
2.51
3.95
1.44
Trans-DLS-LAY DLS-LAA
2.74
3.94
1.20
The average entropy of the attention mecha- nism when translating at sentence and document level. 8 represents the difference in entropy when translating at different granularity.
Table 4:

---

model the relationship between the context words. However, the scarcity of the datasets (Chen al., 2021) and the sparsity of the contextual informa- tion make these model hard to be trained. Lupo et al. (2022) further address this problem by split- ting the sentence into smaller pieces to augment the document-level corpus.
and the results demonstrate a significant improve- ment achieved by our method. Further analysis indicates that our approach can enhance the length generalization capability of the model effectively.
8 Acknowledgement
We would like to express our gratitude the ICT computing platform and the technical service team for providing GPU resources.
Another type of methods fall into doc2doc paradigm, which treats the entire document as a whole unit. Tiedemann and Scherrer (2017) pro- posed that by extending the translation gramularity from sentence to documents the translation become more coherent; Liu et al. (2020) and (Ma et al., 2020) found that the translation quality could be improved by large margin through incorporating pretraining: Bao et al. (2021) suggest that direct training a doc2doc transformer may fail con- verge on small datasets, and proposed to solve this problem by incorporating group attention masks. Similarly, (Sun et al., 2022b) proposed to tackle the same problem by expanding the dataset with a multi-resolution (MR) strategy. On the other hand, this strategy improves the length generalization of the doc2doc models. Compared to the MR strategy, our proposed DLS effectively balances the amount of the text of different lengths. The experimental results shows that our proposed method capable of handling the text with arbitrary length.
Furthermore, we thank the anonymous reviewers for their thorough review and valuable feedback.
Limitations
Although our proposed methods significantly im- prove the translation quality and the length general- ization capability, there still exist some limitations: (I) the slide decoding can not further improve the translation quality as the error accumulation prob- lem of auto-regression model has not been solved; (2) the decoding consumption using SD is slightly higher than the "segment then decoding" method.
References
Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen. Yuan Cao, George F. Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019. Massively multilingual neural machine translation in the wild: Findings and chal- lenges. CoRR, 1e/1900001599
6.2 The Length Bias Problem
Although the length bias problem has not been ex- plore in the field of DNMT, there still exists several studies emphasis on the length extrapolation prob- lem. Press al. (2021) proposed to solve the length extrapolation problem by introducing local assump- tion as the inductive bias in the positional encoding. Following this work, Chi et al. (2022) and Sun et al. (2022a) further proposed new positional encoding methods to overcome this issue. Additionally, (Ru- oss et al., 2023) introduced random positional en- coding training strategy to overcome the length extrapolation problem, and achieved remarkable progress within the field of language modeling.
Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen, and Weihua Luo. 2021. G-transformer for document-level machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing, ACI/JCNLP 2021, I, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 3442-3455.
Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The Long-Doummett Trans- former. 05555 [cs]. ArXiv: 2004.05150.
Mauro Cettolo, Christian Girardi, and Marcello Fed- erico. 2012. WIT3: Web Inventory of Transcribed and Translated Talks. In Proceedings of the 16th Anmual conference of the European Association for Machine Translation, pages 261-268, Trento, Italy. European Association for Machine Translation.
7 Conclusion
In this work, aim to address the issue of length bias in the training of the doc2doc DNMT model. To achieve this objective, we propose several meth- ods, including dynamic length sampling, length aware attention and sliding decoding. We conduct experiments on multiple publicly available datasets,
Linqing Chen, Junhui Li, Zhengxian Gong, Boxing Chen, Weihua Luo, Min Zhang, and Guodong Zhou. 2021. Breaking the Corpus Bottleneck for Context- Aware Neural Machine Translation with Cross-Task Pre-training. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics

---

and the 11th International Joint Conference on Natu- val Language Processing (Volume I: Long Papers), pages 2851-2861, Online. Association for Computa- tional Linguisties.
source toolkit for statistical machine translation. In ACL 2007, Proceedings of the 45th Annual Meet- ing of the Association for Computational Linguistics, June 23-30, 2007, Prague, Czech Republic.
Ta- Ta-Chung Chi, Ting-Hap Han Fan. Peter J Ramadge, and Alexander Rudnicky. 2022. Kerple: Kernelized rel- ative positional embedding for length extrapolation. Advances in Neural Information Processing Systems, 35:8386-8399.
Shaohui Kuang and Deyi Xiong. 2018. Fusing Re- cency into Neural Machine Translation with Inter- Sentence Gate Model. In Proceedings of the 27th International Conference on Computational Linguis- fics. pages 607-617, Santa Fe, New Mexico, USA. Association for Computational Linguisties.
David Chiang and Peter Cholak. 2022. Overcom- ing a Theoretical Limitation of Self-Attention. ArXiv:2202. 12172 [es].
Pierre Lison and Jörg Tiedemann. 2016. Opensub- titles2016: Extracting large parallel corpora from movie and subtitles.
Eva Martínez Garcia, Cristina España-Bonn Bonet, and Lluis Marquez. 2015. Document-leved machine translation with word vector models. In Proceedings of the 18th Annual Conference of the European Association for Machine Translation, EAMT 2015, Antalya, Turkey, May 11 13, 2015. European Association for Ma- chine Translation.
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre- training for neural machine translation. Trans. Assoc. Comput. Linguistics, 8:726-742.
Zhengxian Gong, Min Zhang, and Guodong Zhou 2011. Cache-based deeument-level statistical ma- chine translation. In Proceedings of the 2011 Con- ference Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Conference Centre, Edinburgh, UK, A meet- ing of SIGDAT, Special Interest Group of the ACL, pages 909-919. ACL
Lorenzo Lupo. Marco Dinarelli, and Laurent Besacier. 2022. Divide and Rule: Effective Pre- Fre-Trannin for -owte Multi-Encoder Translation Models. arXiv:2103.17151 [es]. ArXiv: 2103.17151.
Shuming Ma, Dongdong Zhang, and Ming Zhou. 2020. A Simple and Effective Unified Encoder for Document-Level Machine Translation. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3505-3511, On- line. Association for Computational Linguisties.
Christian Hardmeier, Sara Stymne, Jörg Tiedemann, and Joakim Nivre. 2013. Docent: A document-lews level de- coder for phrase-based statistical machine translation. In 51st Annual Meeting of the Association for Com- putational Linguistics, ACI. 2013, Proceddings of the Conference System Demonstrations, 4-9 August 2013, Sofaa, Bulgaria, pages 193-198.
André F. T. Martins and Ramón Fernandez Astudillo. 2016. From Softmax Sparsemax: A Sparse Model of Attention and Multi- Classification. ArXiv:1602.0201 [cs, stat].
Sebastien Jean, Stanislas Lauly, Orhan Firat, and Kyunghyun Cho. 2017. Does Neural Ma- chine Translation Benefit from Larger Context? ArXiv:1704.05133 [cs, stat].
Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. 2019, Selective attention for context-awar neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACI-IILT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 3092-3102 Associa- tion for Computational Linguistics.
Marcin Junczys-Dowmunt 2019. Microsoft translator at WMT 2019: Towards large-scale document-lever neural machine translation. In Proceedings of the Fourth Conference on Machine Translation, WMT
2019, Florence, Italy, August 1-2, 2019 - Volume 2: Shared Task Papers, Day 1, pages 225-233
Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization.
Sameen Maruf, Fahimeh Saleh, and Gholamreza Haffari. 2021. A survey on document- neural machine translation: Methods and evaluation. ACM Comput- ing Surveys (CSUR), 54(2):1-36.
ArXiv:1412: 1412.6980 [es].
Philipp Koehn. 2005. Europarl: A Parallel Corpus for
Statistical Machine Translation. In Proceedings of Machine Translation Summit X: Popers, pages 86, Phuket, Thailand.
Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas. and James Henderson. 2018. Document-level neural machine translation with hierarchical attention net- works. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018, pages 2947-2954. Association for Computational Linguistics.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open

---

Mathias Müller, Annette Rios, Elena Voita, and Rico Sennrich. 2018. A Large-Seuur Scale Test Set for the Evalu- ation of Context-Aware Pronoun Translation in Neu- ral Machine Translation. In WMT 2018, Brussels, Belgium. Association for Computational Linguistics.
Language Processing, EMNLP-JCNNLP 2019, Hong Kong, China, November 3-7, 2019, pages 1576-1585.
Jörg Tiedemann and Yves Scherrer. 2017. Neural machine translation with extended context. arXiv preprint 1rxiv:/708.05443
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- kational Linguisties, July 6-12, 2002, Philadelphia, PA, USA, pages 311-318.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008.
Maja Popović. 2015. chrF:: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392-395, Lisbon, Portugal. Association for Computational Linguistics.
Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. Exploiting Cross-Sentence Context for Neural Machine Translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Lan- guage Processing, pages 2826-2831, Copenhagen, Denmark. Association for Computational Linguis- tics.
Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186- 191, Belgium, Brussels. Association for Computa- tional Linguistics.
Hongfei Xu, Deyi Xiong, Josef van Genabith, and Qi- uhui Liu. 2020. Efficient context-aware neural ma- chine translation with layer-wise weighting and input- aware gating. In Proceedings of the Twenty-Nine -inth International Joint Conference on Artificial Intelli- gence, IJCAI 2020, pages 3933-3940.
Ofir Press, Noah A Smith, and Mike Lewis. 2021. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprint arXiv:2108. 12409.
Anian Ruoss, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Rôbert Csordás, Mehdi Bennani, Shane Legg, and Joel Veness. 2023. Randomized positional encodings boost length generalization of transform- arXiv preprint 23X.::2305.1644.5
Zhengxin Yang, Jinchao Zhang. Fandong Meng. Shubao Gu, Yang Feng, and Jie Zhou. 2019. Enhancing con- text modeling with query-guided capsule network for document-level translation. In Proceedings of the 2019 Conference on Empirical Methods in Natu- rul Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLA 2019, Hong Kong, China, Novem- ber 3-7, 2019, pages 1527-1537. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics, ACL 2016, August 7-12, 2016, Berlin, Ger- many. Volume 1: Long Papers.
Hyeongu Yun, Yongkeun Hwang, and Kyomin Jung. 2020. Improving context-aware neural machine trans- lation using self-attentive sentence embedding. In The Thirty-Foury AAAI Conference on Artificial In- telligence, AAAI 2020, The -hirry-Second Innova- tive Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educa- tional Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 9498-9506.
Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shao- han Huang, Alon Benhaim, Vishrav Chaudhary. Xia Song, and Furu Wei. 2022a. A length-extrapollatbbe transformer. arXiv preprint wtx.::22212.10544444
Zewei Sun, Mingxuan Wang. Hao Zhou, Chengqi Zhao, Shujian Huang. Jiajun Chen, and Lei Li. 20226. Re- thinking leeeelnt-level neural machine translation. In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3537-3548.
Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018. Improving the transformer translation model with document-level context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, Brussels, Belgium, October 31 November 4, 2018, pages 533-542.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zhigniew Wojna. 2015. Re- thinking the Inception Architecture for Computer Vision. A1Xiv:1512.00567 [es].
Xin Tan, Longyin Zhang, Deyi Xiong, and Guodong Zhou. 2019. Hierarchical modeling of global con- text for document-level neural machine translation. In Proceedings of the 2019 Conference Empiri- cal Methods in Natural Language Processing and the 9th International Joint Conference on Natural
Zaixiang Zheng, Xiang Yue. Shujian Huang, Jiajun Chen, and Alexandra Birch, 2020. Towards making the most of context in neural machine translation. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020. pages 3983-3989, ijcai.org.

---

A The Proof of the Length Aware Attention
The entropy of the attention mechanism can be calculated using the following formula":
H4- -
aij log aigj
j=1
Σ
e
Nq, ke (Xqq kj)
log
Aq
kj
==1
11 kg
==1
j=1
(6)
where Hi indicate the attention entropy the i-th token, À is the scale factor, and ai,j is the attention weight. Let 8ig = kj and Pij
M6, we
get:
n AMi,
n
H4: log
c
Pijjiij
j=1
j=1
1
llog + log
Σ j=l
Pij
(7)
According to Mean-field theory, could change the order of computation between the expo- nential function and summation:
H, log + As,
Σ
Sij
(8)
j=1
a
where 8, = Σ 81./n. Considering the proper-
1=1
ties of the softmax function, we can obtain further approximations:
H4 ~ logn A(s; X8maa)
(9)
Thus, we get:
X x log n,
(10)
where À is proportional to log
'Our derivation motivated by the following blog: httpss//ppucs..ccccaaccvvvvvs//p


Reviews:
[{"review_id": "YCXrO11fVL", "reviewer": "Reviewer_J2RQ", "summary": "", "questions": "- A significant amount of previous research ([1], [2]) adopts two-stage training strategy: pre-training at the sentence-level and fine-tuning at the document-level. Therefore, it would be recommended to empirically show if the proposed method is also useful for fine-tuning a pre-trained sentence-level NMT model at the document-level (e.g., Trans-FT + DLS + LAA, G-Trans-FT + LAA). The dynamic length sampling based on curriculum learning (in Figure 2) is questionable. \n\n[2] Voita et al., 2019, When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion\n\n[3] Maruf et al., 2019, Selective Attention for Context-aware Neural Machine Translation", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thorough review and constructive feedback. We are glad that you found our approach to be innovative. In response to the concern raised about our paper, we summarize our response as follows.\n\n## Q1: The proof of entropy divergense problem in attention mechanism is missing.\nThank you very much for pointing out the absence of our appendix section. We apologize for omitting this part during the paper submission, and we would like to assure you that we are taking immediate steps to rectify this oversight in our final paper.\n\nIn addition, to verify the effectiveness of our proposed **Length Aware Attention (LAA)**, we conducted a statistical analysis of the average entropy of the attention mechanism when the model translates sentences and documents of length 512. As shown in the table below, it can be observed that the entropy of the attention mechanism is more stable after applying the LAA method, which indicates that after applying the LAA mechanism, the model demonstrates better consistency in handling sentence-level and document-level text. Furthermore, by applying the DLS and LAA method, the entropy of attention when translating the document is lower than that of the FT method, indicating that the model concentrates more on the long-range contexts. To further demonstrate the effect of Length Aware Attention, we also plan to add the Attention visualization results in the appendix of our paper.\n\n| Model               | sentence | document | Δ    |\n| ------------------- | -------- | -------- | ---- |\n| Transformer-FT      | 2.65     | 4.0      | 1.35 |\n| Transformer-DLS     | 2.51     | 3.95     | 1.44 |\n| Transformer-DLS-LAA | 2.74     | 3.94     | 1.20 |\n\n## Q2: Is the use of longer contexts necessary?\nAlthough many previous studies have shown that the most informative contexts tend to be concentrated within the previous one to three sentences, we contend that this observation may be attributed to the limitations of conventional sent2sent DNMT model in effectively harnessing the entirety of contextual information. Recently, numerous doc2doc DNMT studies have demonstrated its superior capability in leveraging contextual information compared to the sentence-to-sentence approaches [1,2]. In addition, as shown in the following table, our experiment reveals that **even vanilla Transformer can benefit from an increased amount of contextual information** when there is an ample amount of training data.\n\n| Maximum Seq | Europarl7 |        |        |        |\n| ----------- | --------- | ------ | ------ | ------ |\n|             | s-BLEU    | d-BLEU | s-chrF | d-chrF |\n| sent        | 30.33     | 32.45  | 59.99  | 68.11  |\n| 64          | 30.56     | 32.77  | 60.13  | 68.32  |\n| 128         | 30.97     | 33.13  | 60.40  | 68.47  |\n| 512         | 31.16     | 33.58  | 61.29  | 69.91  |\n\n\n[1] Bao et al., 2021, G-transformer for document-level machine translation.\n[2] Liu et al., 2020, Multilingual denoising pre-training for neural machine translation.\n\n## Q3: Why the proposed Slide Decoding can not improve the BLEU further?\nWe propose the **Sliding Decoding (SD)** approach in response to the potential significant incoherence introduced by truncating the entire documents into multiple segments. Our proposed method offers two notable advantages:\n1. Our approach can further **enhance the length generalization performance of the DNMT model**. As shown in Figure 5 in our paper, the SD method significantly improves the BLEU score when the sequence length exceeds the maximum length used during training. In order to ensure a fair comparison, we split all the test sets that were not decoded using the SD method into segments with a maximum length of 512 in the main experiment. As a result, **the superiority of our approach was not fully demonstrated in the main experiments**.\n2. Due to its ability to leverage as much contextual information as possible, our proposed Slide Decoding can further **enhance the fluency of translations**. However, this aspect may not be well-reflected by the BLEU metric. As a result, the advantage of our method was not fully showcased in the experiments.\n\nFurthermore, the **error accumulation problem** may be another reason for the limited improvement in BLEU scores observed in the SD method. We will explore and address this problem in our future work.\n\n## Q4: Whether the proposed methods are useful for fine-tuning a pre-trained sentence-level NMT model at the document-level?\nThe answer to this question is yes. We have conducted series experiments to reveal the effectiveness of our proposed method. As shown in Table 1 in the paper, **our approach significantly outperforms the 2-stage baseline methods** (Transformer-FT, G-Trans-FT). In addition, in Table 2 of the ablation study, both System 4 and System 5 utilize the 2-stage approach, and when compared against System 2 and System 3 which employ the DLS method, they collectively demonstrate the effectiveness of our approach.\n\nTo further demonstrate the effectiveness of our proposed dynamic length sampling method, we simply apply length sampling training stage without dynamic adjusting the probability on a pre-trained sentence-level Transformer model. However, due to time constraints, **we will give our experimental results in the discussion phase**.\n\n## Conclusion\nWe sincerely appreciate your thoughtful feedback and constructive criticisms on our work. We have taken a thorough review of your comments and have taken the time to address each of the points you raised in detail. If our responses align with your expectations, we would greatly appreciate an increase in our score. If you have any concerns or doubts about our response, please feel free to contact us during discussion period.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation of this paper is clear. \n\n2. The three specific training methods are novel.", "reasons_to_reject": "Most importantly, the problems that the authors claim are needed to be well-supported by either logic or experiments. \n\n1. The authors claim that LAA is motivated by the assumption that the document-level NMT model may interfere with the attention mechanism since it needs to handle a wide range of context. However, [1] explains that the actual amount of context needed is not as much as the previous few (1~3) sentences. It might be related to the issue of entropy divergence, which is newly addressed by the authors. The paper is stated that it would explain in an appendix but it doesn't exist. \n\n2. The authors also claim that SD is motivated by the assumption that the contextual information may be lost when decoding longer sequences in segments. Considering [1], it is not expected to be a problem, as the necessary context is preserved even if it is split. \nThe lack of performance gains also supports this suspicion. \n\n[1] Fernandes et al., 2021, Measuring and Increasing Context Usage in Context-Aware Machine Translation", "questions_to_authors": "- A significant amount of previous research ([1], [2]) adopts two-stage training strategy: pre-training at the sentence-level and fine-tuning at the document-level. Therefore, it would be recommended to empirically show if the proposed method is also useful for fine-tuning a pre-trained sentence-level NMT model at the document-level (e.g., Trans-FT + DLS + LAA, G-Trans-FT + LAA). The dynamic length sampling based on curriculum learning (in Figure 2) is questionable. \n\n[2] Voita et al., 2019, When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion\n\n[3] Maruf et al., 2019, Selective Attention for Context-aware Neural Machine Translation", "paper_topic_and_main_contributions": "This paper addresses document-level neural machine translation (NMT). \n\nThe authors introduce the \"length bias\" problem, where the translation quality degrades significantly as the decoding length deviates from the maximum sentence length. \nTo mitigate this bias, they propose a novel document-level NMT training method: 1) Dynamic length sampling (DLS) 2) Length-aware attention (LAA) 3) Sliding decoding (SD). \n\nWhen integrating their proposed method into G-transformer, they show significant improvements on all datasets.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A", "Justification_for_Ethical_Concerns": "N/A"}, "extra_text": {}}, {"review_id": "IBFJvK13bd", "reviewer": "Reviewer_Wr8v", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Your feedback is greatly appreciated, and we would like to address your main points of concern to provide a clearer understanding of our work.\n\n## Q1: Why the proposed Slide Decoding can not improve the BLEU further?\nWe propose the **Sliding Decoding (SD)** approach in response to the potential significant incoherence introduced by truncating the entire documents into multiple segments. Our proposed method offers two notable advantages:\n1. Our approach can further **enhance the length generalization performance of the DNMT model**. As shown in Figure 5 in our paper, the SD method significantly improves the BLEU score when the sequence length exceeds the maximum length used during training. In order to ensure a fair comparison, we split all the test sets that were not decoded using the SD method into segments with a maximum length of 512 in the main experiment. As a result, **the superiority of our approach was not fully demonstrated in the main experiments**.\n2. Due to its ability to leverage as much contextual information as possible, our proposed Slide Decoding can further **enhance the fluency of translations**. However, this aspect may not be well-reflected by the BLEU metric. As a result, the advantage of our method was not fully showcased in the experiments.\n\nFurthermore, the **error accumulation problem** may be another reason for the limited improvement in BLEU scores observed in the SD method. We will explore and address this problem in our future work.\n\n## Q2: Lack of analytical experiments on Length Aware Attention.\nThank you for your constructive advise, and we have incorporated additional experiments on Length Aware Attention as follows. To verify the effectiveness of our proposed **Length Aware Attention (LAA)**, we conducted a statistical analysis of the average entropy of the attention mechanism when the model translates sentences and documents of length 512. As shown in the table below, it can be observed that the entropy of the attention mechanism is more stable after applying the LAA method, which  indicates that after applying the LAA mechanism, the model demonstrates better consistency in handling sentence-level and document-level text. Furthermore, by applying the DLS and LAA method, the entropy of attention when translating the document is lower than that of the FT method, indicating that the model concentrates more on the long-range contexts.\n\n| Entropy of Model    | sentence | document | Δ    |\n| ------------------- | -------- | -------- | ---- |\n| Transformer-FT      | 2.65     | 4.0      | 1.35 |\n| Transformer-DLS     | 2.51     | 3.95     | 1.44 |\n| Transformer-DLS-LAA | 2.74     | 3.94     | 1.20 |\n\n## Q3: Testing on challenging length generalization datasets would better showcase the benefits.\nThanks again for your advise, following [1], we collect ContraPro as **discourse dense dataset** to measure our method. To further increase the challenging of the dataset for length generalization, we truncate the dataset into different length. We evaluate the translation quality using d-BLEU, and the results are shown in the following table.\n\n| Model on discourse dense dataset | d-BLEU |\n| -------------------------------- | ------ |\n| Transformer-FT                   | 19.01  |\n| Transformer-DLS                  | 19.57  |\n| Transformer-DLS-LAA              | 19.90  |\n\nAs shown in the above table, our proposed method performs significantly better than the baseline method, indicating the effectiveness of our method in the context of discourse translation.\n\n\n[1] Post et al., 2023, Escaping the sentence-level paradigm in machine translation.\n\n\n## Q4: There is no comparison to other recent methods that improve length generalization for sequence models.\nWe appreciate your attention to the aspect of length generalization in sequence models. In response to your comment, we would like to address this concern and provide more information regarding our work.\n1. As far as our knowledge extends, **we are the first highlighting the length bias problem in the domain of document-level machine translation**. Our work emphasizes the significance of addressing this problem and has resulted in significant improvements.\n2. In order to provide a comprehensive evaluation and draw comparisons with length generalization efforts in other domains, we have conducted a thorough comparison with the well-recognized ALIBI method [2], known for its exceptional length extrapolation capabilities. \n\n| Model vs dataset       | TED    |        |        |        | Europarl |        |        |        | News   |        |        |        |\n| ---------------------- | ------ | ------ | ------ | ------ | -------- | ------ | ------ | ------ | ------ | ------ | ------ | ------ |\n|                        | s-BLEU | d-BLEU | s-chrF | d-chrF | s-BLEU   | d-BLEU | s-chrF | d-chrF | s-BLEU | d-BLEU | s-chrF | d-chrF |\n| ALIBI-doc              | 19.65  | 26.30  | 47.22  | 68.26  | 29.99    | 32.54  | 60.19  | 69.28  | 12.67  | 22.83  | 36.19  | 59.91  |\n| ALIBI-FT               | 20.85  | 27.55  | 49.54  | 69.71  | 29.64    | 32.55  | 59.76  | 69.41  | 17.37  | 24.02  | 43.49  | 60.79  |\n| Transformer-FT         | 24.31  | 28.48  | 54.70  | 70.51  | 31.16    | 33.58  | 61.29  | 69.91  | 23.96  | 27.33  | 53.27  | 64.98  |\n| Transformer+DLS+LAA    | 24.93  | 28.95  | 55.18  | 70.69  | 31.85    | 34.32  | 61.26  | 70.01  | 24.57  | 28.52  | 53.13  | 65.47  |\n| Transformer+DLS+LAA+SD | 24.60  | 28.43  | 55.16  | 70.55  | 31.63    | 34.33  | 60.91  | 69.93  | 23.72  | 27.95  | 52.50  | 65.20  |\n| G-Trans-FT             | 25.07  | 28.86  | 55.65  | 70.79  | 32.38    | 34.51  | 62.15  | 70.32  | 25.87  | 27.82  | 55.71  | 65.14  |\n| G-Trans+DLS+LAA        | 25.37  | 29.07  | 55.76  | 70.81  | 32.67    | 34.81  | 62.06  | 70.23  | 26.70  | 28.66  | 55.96  | 65.24  |\n| G-Trans+DLS+LAA+SD     | 24.87  | 28.53  | 55.34  | 70.51  | 32.67    | 34.82  | 61.99  | 70.17  | 26.56  | 28.52  | 55.78  | 65.09  |\n\n\n| Model vs sequence length | 32    | 64    | 128   | 256   | 512   | 1024  |\n| ------------------------ | ----- | ----- | ----- | ----- | ----- | ----- |\n| Transformer-FT           | 29.75 | 31.00 | 32.55 | 33.04 | 33.58 | 28.28 |\n| ALIBI-FT                 | 13.46 | 18.90 | 26.87 | 30.66 | 32.55 | 30.54 |\n| Transformer+Our Method   | 33.48 | 33.73 | 33.90 | 34.11 | 34.32 | 34.33 |\n\n\nAs the shown in the first above table, our proposed method outperforms ALIBI on all datasets, indicating that **our method is better suited for DNMT**. Additionally, we compared the length generalization of the ALIBI method. The experimental results within the second table demonstrated that ALIBI exhibits better length extrapolation than the baseline method when decoding sequences exceed the training length, but there still exists a gap between ALIBI and our method. We attribute this difference to the fact that ALIBI's generalization performance relies on local assumptions within the attention mechanism, which contradict the essence of DNMT that utilize contexutal information wherever possible. Furthermore, in the context of encoder-decoder architecture for machine translation, the cross-attention mechanism cannot benefit from this assumption. These two reasons combined contribute to ALIBI's inability to surpass our proposed method in the context of discourse translation.\n\n[2] Press et al., 2021, Train short, test long: Attention with linear biases enables input length extrapolation.\n\n## Conclusion\nWe sincerely appreciate your thoughtful feedback and constructive criticisms on our work. We have taken a thorough review of your comments and have taken the time to address each of the points you raised in detail. If our responses align with your expectations, we would greatly appreciate an increase in our score. If you have any concerns or doubts about our response, please feel free to contact us during discussion period.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The problem being addressed is relevant and the ideas are intuitively motivated to address the length bias problem in document translation.\n2. The methods seem simple to implement on top of standard DNMT models.\n3. The paper is clearly written and easy to follow.", "reasons_to_reject": "1. The sliding window decoding actually seems to hurt performance slightly even though it should help length generalization. Error accumulation may be an issue.\n2. More analysis could be done to understand the effects of the different components, e.g. how attention patterns change with length-aware scaling.\n3. The techniques are evaluated on standard datasets. Testing on challenging length generalization datasets would better showcase their benefits.\n4. There is no comparison to other recent methods (if any) that improve length generalization for sequence models.\n\n---\npost-rebuttal: most concerns have been addressed.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes methods to improve document-level neural machine translation (DNMT) by addressing the length bias problem, where model performance degrades on sentences much shorter/longer than the maximum length seen during training. Experiments across 3 widely-used datasets demonstrate consistent and sizable improvements in BLEU and chrF over competitive baseline DNMT methods. The techniques are also shown to improve length generalization.\n\n\n**Main contributions:**\n1. Dynamic length sampling during training to expose the model to more varied sentence lengths.\n2. Length-aware attention to help focus on relevant target words for long sentences.\n3. Sliding window decoding to handle test sentences exceeding the max length.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jP1TSM6BBK", "reviewer": "Reviewer_VHhW", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Your feedback is greatly appreciated, and we would like to address your main points of concern to provide a clearer understanding of our work.\n\n## Q1: Why the proposed Slide Decoding can not improve the BLEU further?\nWe propose the **Sliding Decoding (SD)** approach in response to the potential significant incoherence introduced by truncating the entire documents into multiple segments. Our proposed method offers two notable advantages:\n1. Our approach can further **enhance the length generalization performance of the DNMT model**. As shown in Figure 5 in our paper, the SD method significantly improves the BLEU score when the sequence length exceeds the maximum length used during training. In order to ensure a fair comparison, we split all the test sets that were not decoded using the SD method into segments with a maximum length of 512 in the main experiment. As a result, **the superiority of our approach was not fully demonstrated in the main experiments**.\n2. Due to its ability to leverage as much contextual information as possible, our proposed Slide Decoding can further **enhance the fluency of translations**. However, this aspect may not be well-reflected by the BLEU metric. As a result, the advantage of our method was not fully showcased in the experiments.\n\nFurthermore, the **error accumulation problem** may be another reason for the limited improvement in BLEU scores observed in the SD method. We will explore and address this problem in our future work.\n\n## Q2: What is the window-size of Slide Decoding in the experiments?\nThank you for meticulously examining our paper. As our motivation is to maximize the contextual information, the windows-size of Slide Decoding is always **aligned with the maximum sequence length during training**, which is equal to 512.\n\n## Q3: How do other related methods perform on this length-bias problem?\nWe appreciate your attention to the length-bias problem. In response to your comment, we would like to address this concern and provide more information regarding our work.\n1. As far as our knowledge extends, **we are the first highlighting the length bias problem in the domain of document-level machine translation**. Our work emphasizes the significance of addressing this problem and has resulted in significant improvements.\n2. In order to provide a comprehensive evaluation and draw comparisons with length generalization efforts in other domains, we have conducted a thorough comparison with the well-recognized ALIBI method [1], known for its exceptional length extrapolation capabilities. \n\n| Model vs dataset       | TED    |        |        |        | Europarl |        |        |        | News   |        |        |        |\n| ---------------------- | ------ | ------ | ------ | ------ | -------- | ------ | ------ | ------ | ------ | ------ | ------ | ------ |\n|                        | s-BLEU | d-BLEU | s-chrF | d-chrF | s-BLEU   | d-BLEU | s-chrF | d-chrF | s-BLEU | d-BLEU | s-chrF | d-chrF |\n| ALIBI-doc              | 19.65  | 26.30  | 47.22  | 68.26  | 29.99    | 32.54  | 60.19  | 69.28  | 12.67  | 22.83  | 36.19  | 59.91  |\n| ALIBI-FT               | 20.85  | 27.55  | 49.54  | 69.71  | 29.64    | 32.55  | 59.76  | 69.41  | 17.37  | 24.02  | 43.49  | 60.79  |\n| Transformer-FT         | 24.31  | 28.48  | 54.70  | 70.51  | 31.16    | 33.58  | 61.29  | 69.91  | 23.96  | 27.33  | 53.27  | 64.98  |\n| Transformer+DLS+LAA    | 24.93  | 28.95  | 55.18  | 70.69  | 31.85    | 34.32  | 61.26  | 70.01  | 24.57  | 28.52  | 53.13  | 65.47  |\n| Transformer+DLS+LAA+SD | 24.60  | 28.43  | 55.16  | 70.55  | 31.63    | 34.33  | 60.91  | 69.93  | 23.72  | 27.95  | 52.50  | 65.20  |\n| G-Trans-FT             | 25.07  | 28.86  | 55.65  | 70.79  | 32.38    | 34.51  | 62.15  | 70.32  | 25.87  | 27.82  | 55.71  | 65.14  |\n| G-Trans+DLS+LAA        | 25.37  | 29.07  | 55.76  | 70.81  | 32.67    | 34.81  | 62.06  | 70.23  | 26.70  | 28.66  | 55.96  | 65.24  |\n| G-Trans+DLS+LAA+SD     | 24.87  | 28.53  | 55.34  | 70.51  | 32.67    | 34.82  | 61.99  | 70.17  | 26.56  | 28.52  | 55.78  | 65.09  |\n\n\n| Model vs sequence length | 32    | 64    | 128   | 256   | 512   | 1024  |\n| ------------------------ | ----- | ----- | ----- | ----- | ----- | ----- |\n| Transformer-FT           | 29.75 | 31.00 | 32.55 | 33.04 | 33.58 | 28.28 |\n| ALIBI-FT                 | 13.46 | 18.90 | 26.87 | 30.66 | 32.55 | 30.54 |\n| Transformer+Our Method   | 33.48 | 33.73 | 33.90 | 34.11 | 34.32 | 34.33 |\n\n\nAs the shown in the first above table, **our proposed method outperforms ALIBI on all datasets**, indicating that our method is better suited for DNMT. Additionally, we compared the length generalization of the ALIBI method. The experimental results within the second table demonstrated that ALIBI exhibits better length extrapolation than the baseline method when decoding sequences exceed the training length, but there still exists a gap between ALIBI and our method. We attribute this difference to the fact that ALIBI's generalization performance relies on local assumptions within the attention mechanism, which contradict the essence of DNMT that utilize contexutal information wherever possible. Furthermore, in the context of encoder-decoder architecture for machine translation, the cross-attention mechanism cannot benefit from this assumption. These two reasons combined contribute to ALIBI's inability to surpass our proposed method in the context of discourse translation.\n\n[1] Press et al., 2021, Train short, test long: Attention with linear biases enables input length extrapolation.\n\n\n## Q4: Lack the statistical analysis for the variation of the sentence length after sampling.\nFirstly, it's important to note that **our sampling method does not directly alter the length of individual sentences**. Instead, it **dynamically adjusts the number of sentences to modify the length of text segments**. This approach was chosen to ensure that the inherent structure and coherence of sentences are preserved while achieving our intended text length variations.\n\nTo better illustrate the impact of our sampling method on sentence length distribution, we have included **Figure 6** in our paper. This figure demonstrates the distribution of segment lengths before and after the application of our sampling method. As depicted in the figure, after implementing our sampling technique, the distribution of segment lengths becomes more uniform. This observation supports the effectiveness of our approach in achieving the desired outcome without compromising the coherence of individual sentences.\n\n\n## Q5: The abstract and the introduction could be further improved.\nThanks for your constructive advise. We will try our best to further improve the writing of this paper. \n\n## Conclusion\nWe sincerely appreciate your thoughtful feedback and constructive criticisms on our work. We have taken a thorough review of your comments and have taken the time to address each of the points you raised in detail. If our responses align with your expectations, we would greatly appreciate an increase in our score. If you have any concerns or doubts about our response, please feel free to contact us during discussion period.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper proposes a simple yet effective method to mitigate the length-bias problem in Doc-NMT, which achieves the sota score in three datasets and significantly mitigates the length-bias problem in this task.\n", "reasons_to_reject": "Although the whole method is effective, the sliding decoding strategy seems cannot be effective as expected, while lacking the declaration of the window-size. And no further corresponding exploration for this failure in Sec.5.2. \nBesides, the performance of other relative methods on this length-bias problem is also important to be analyzed and compared.\nLack the statistical analysis for the variation of the sentence length after sampling.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this paper, the authors focus on the length bias problem in Document-level NMT, which results in significant translation quality degradation in terms of sentence length.\nTo address this issue, the authors optimize the Doc-NMT model in three levels, i.e., training data sampling, length-aware attention, and sliding windows decoding strategy.\nAs shown in their experimental results, the proposed method could achieve the best score on three widely-used datasets and significantly mitigate the length bias problem in the document-level translation task.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Could you exhibit the improvement of your method in the Abstract and Introduction?"}, "extra_text": {}}, {"review_id": "xuS9yN994V", "reviewer": "Reviewer_ap6j", "summary": "", "questions": "A. What is the distribution of series lengths in the data set? Did the similar distributions of these three datasets improve the accuracy across all of them?\nB. How many are the maximum series lengths of each?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your careful consideration of our paper and would like to take this opportunity to respond to the concerns you have raised.\n\n## Q1: How often does the length-bias problem occur?\nThank you for your attention to the aspect of the length-bias problem. This problem is indeed prevalent in real-world scenarios. As depicted in Figure 6, training datasets often exhibit a concentrated distribution of lengths. However, during the decoding phase, users do not limit the translation system to translating only texts of the same length as those in the training set. Instead, users also apply translation systems to sentence-level translation or even longer document-level translation, addressing scenarios that traditional DNMT methods struggle to handle. Hence, we believe this problem presents a valuable opportunity for practical application.\n\n\n## Q2: What is the length distribution of the dataset?\nThanks again for your advice. We will include the original length distribution of these datasets in our final paper. In addition, since these datasets are usually collected from similar sources, their length distributions are concentrated. However, as our method sampling the length of the dataset, the model trained with DLS, LAA and SD can be applied to translation tasks of any length. \n\n## Q3: Typos Grammar Style and Presentation Improvements.\nThank you for your careful examination of our paper. We will try our best to further improve the writing of this paper.\n\n## Conclustion\nWe sincerely appreciate your thoughtful feedback and constructive criticisms of our work. We have taken a thorough review of your comments and have taken the time to address each of the points you raised in detail. If our responses align with your expectations, we would greatly appreciate an increase in our score. If you have any concerns or doubts about our response, please feel free to contact us during the discussion period.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper proposes a combination of three methods. Although each method is simple, they all seem effective for the length bias problem.\nComparison experiments have been conducted on many datasets and comparison systems.\nThe experimental results suggest that the proposed methods are effective.\nThe analysis is well done and shows that the proposed method is effective for both long and short series.", "reasons_to_reject": "The authors describe a problem when decoding sentences that are shorter or longer than the maximum length being trained, but how often this problem occurs should be described.\nThere is no data on the distribution of series lengths in the dataset.\nThe dataset contains a variety of series lengths, and the score of the proposed method may vary greatly depending on the distribution of these series lengths.", "questions_to_authors": "A. What is the distribution of series lengths in the data set? Did the similar distributions of these three datasets improve the accuracy across all of them?\nB. How many are the maximum series lengths of each?", "paper_topic_and_main_contributions": "This paper addresses the length bias problem in document-level translation.\nConventional methods are effective only for long or short sequences and cannot handle both sequences.\nThe authors proposed a method to extend the length generation capability to handle both long and short sequences.\nThey proposed the following three methods.\n1. sampling to learn sequences of various lengths during learning\n2. scaling in Attention by the average length of the series in the training epoch\n3. sliding decoding", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "In Section 2, 'The Transformer' and 'The doc2doc DNMT model' are proper nouns, so we do not need 'The'."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes and addresses a novel problem - the "length bias" problem - that of document level machine translation models struggling to translate documents which are much smaller or larger than those in the training data. 
Using a number of strategies during training (sampling and length-normalised attention) and decoding (sliding decoding) they show strong results on several open datasets and mitigate the length bias problem in the document-level translation task.
One reviewer noted that it would have been better to test their model on datasets which display the "length bias" problem instead of standard datasets and the authors provided a table with some relevant results. The authors also provided a comparison with other recent document level MT methods and their methods perform better. 
There were four reviews and lengthy rebuttals with new results and clarifications which led to one reviewer increasing their score.
Decision: Accept-Findings