Simultaneous Neural Machine Translation with Constituent Label
Prediction
Yasumasa Kano1 Katsuhito Sudoh1,2 Satoshi Nakamura1,2
1
Nara Institute of Science and Technology (NAIST), Japan
2
Center for Advanced Intelligence Project (AIP), RIKEN, Japan{kano.yasumasa.kw4, sudoh, s-nakamura}@is.naist.jp

Abstract
Simultaneous translation is a task in whichtranslation begins before the speaker has finished speaking, so it is important to decide
when to start the translation process. However,deciding whether to read more input words orstart to translate is difficult for language pairs
with different word orders such as English andJapanese. Motivated by the concept of prereordering, we propose a couple of simple decision rules using the label of the next constituent predicted by incremental constituent
label prediction. In experiments on Englishto-Japanese simultaneous translation, the proposed method outperformed baselines in thequality-latency trade-off.

1

Introduction

Simultaneous machine translation is a task in which
the machine starts outputting a translation before
reading the entire input sentence. This task is moredifficult than full-sentence translation because it
translates the initial part of a sentence without thecontext of the latter part. This involves a trade-off
between delay and quality of the translation; using
a longer context should improve translation quality
at the cost of a longer delay, and vice versa. In
practice, we should control the latency so that it’s
not too large, but we may also need to allow a longlatency depending on the situation.
Most of the recent simultaneous translation models (Ma et al., 2019; Arivazhagan et al., 2019; Raffel et al., 2017; Arivazhagan et al., 2019; Ma et al.,2020b; Dalvi et al., 2018; Gu et al., 2017; Alinejad
et al., 2018; Cho and Esipova, 2016; Zheng et al.,2020, 2019; Zhang et al., 2020) are based on neuralmachine translation (NMT), although earlier studies were based on statistical machine translation(Rangarajan Sridhar et al., 2013; Grissom II et al.,2014; Oda et al., 2014, 2015). In simultaneous
NMT, there are two major approaches: those inwhich a latency hyperparameter is given before thetraining and those in which it is given at the timeof inference.
The former approach requires training a model
individually for each pre-defined latency setting,while the latter approach uses a single model fordifferent latency conditions. Most human simultaneous interpreters would not need such long training to slightly adjust latency, while it takes much
more time to learn other languages to develop theirtranslation skill. Therefore, the latter approach is
closer to the learning process of human simultaneous interpreters.
wait-k (Ma et al., 2019) is a simple simultaneous
NMT method of the former approach that waits
k tokens before starting to translate. It also hasvariants within the latter approach called test-timewait-k, in which k is determined at the inference
time. wait-k had better performance than test-timewait-k in that study’s experiments.
There is another method in the latter approachthat uses Meaningful Unit (Zhang et al., 2020).
In this model, chunk-based incremental decoding
is done at inference time by segmentation witha boundary predictor. This model outperformed
baselines of the former approach. They refined
their basic boundary predictor to deal with sentencepairs in which full-sentence translation needs longdistance reordering. However, its training process
is very complicated: It first generates monotonic
translations, fine-tunes the NMT model with them,
then generates an oracle boundary with the model,
and finally fine-tunes a boundary-prediction model
based on BERT (Devlin et al., 2019).
Simultaneous translation is still difficult for language pairs such as English-Japanese, which often require long-distance reordering. To tackle
the reordering problem, we propose an inputsegmentation method for simultaneous translation,using a couple of simple rules and incremental prediction of the label of a syntactic constituent coming immediately after the input existing so far. This

Source sentence
Monotonic translation
Full-sentence translation

I bought a pen.
watashi wa katta pen wo.
watashi wa pen wo katta.

Table 1: Translation from English (SVO) to Japanese (SOV)

Boundary prediction
Simultaneous translation

I / bought a pen.
watashi wa / pen wo katta.

Table 2: Example of English-to-Japanese translation using proposed method with segment-boundary prediction

is not dependent on the trained NMT. Therefore,
once we create it, it is reusable for other models.
Our proposed method is inspired by Head Finalization (Isozaki et al., 2010). Head Finalization reorders words of the source sentence before translating from an SVO (Subject-Verb-Object) language
to an SOV language in full-sentence statistical machine translation. This method moves a syntactic
head into a later position so that the word order ofthe source language (e.g., English) becomes similar
to that of the target language (e.g., Japanese). This
enables us to monotonically translate from English,
which is a typical SVO language, to Japanese, atypical SOV language.
Recent NMT models like Transformer (Vaswani
et al., 2017) works well on reordering in general,so this kind of pre-reordering is not usually used.
However, simultaneous translation monotonicallyreads input words one by one, and therefore thedifference in word order remains a problem. As
shown in Table 1, monotonic translation often becomes unnatural compared to full-sentence translation. The part “bought a pen” should be translated to pen wo katta by reversing the word order.
Therefore, after reading the word “bought,” it isimportant to wait for future words without starting
to translate it. In this case, “I” is the last word that
does not require reordering. This word is regarded
as a segment boundary to start a partial translation.
Table 2 shows an example of our proposed segmentation. Suppose we predict the next constituent
label as a verb phrase (VP) after reading an input
word “I.” This shows the possibility that the next
words should be reordered, so the “I” becomes
the boundary. Once detecting the boundary, NMT
model starts to translate “I” into “watashi wa.” After that, the model restarts to read the remaining
input words, then translates “bought a pen” into
“pen wo katta.” The total output of simultaneoustranslation based on the proposed segmentation is
the same as that of full-translation in this simple

example. By ensuring that the Verb and its Object of the source sentence are included in a single
segment, it is possible to output translation while
maintaining the SOV-like structure of the target
language.
In experiments on English-to-Japanese simultaneous translation, the proposed method outperformed baselines in the quality-latency trade-off.

2

Related work

In statistical machine translation, there are several
approaches to finding boundaries of segments for simultaneous translation. Oda et al. (2014) proposeda method to choose segment boundaries that maximize the BLEU score. Rangarajan Sridhar et al.
(2013) proposed segmentation strategies based on
lexical cues.
In NMT, there have been many studies on simultaneous translation. The amount of latency isdecided either before training or at inference time.
wait-k (Ma et al., 2019) is the simplest variant using
fixed latency: It simply waits for k tokens before
starting translation (Ma et al., 2019). The latency
policy can be learned from a parallel corpus together with an NMT model. MILk (Arivazhagan
et al., 2019) and other approaches (Raffel et al.,2017; Ma et al., 2020b) used a latency-augmented
loss function in training to balance latency and accuracy.
In contrast, the latency policy can be learned
with a pre-trained NMT model, such as test-timewait-k (Ma et al., 2019) and STATIC-RW (Dalvi
et al., 2018). These have fixed policies that wait for
the fixed number of tokens before translation, butthere are other models that learn a more flexible
policy for a given pre-trained NMT model. Somestudies use reinforcement learning to learn an adaptive READ/WRITE policy (Grissom II et al., 2014;
Satija and Pineau, 2016; Gu et al., 2017; Alinejad
et al., 2018). Training by reinforcement learning
can be unstable depending on the condition. Onemethod that does not use reinforcement learning is
wait-if-* (Cho and Esipova, 2016), which translates
and segments jointly to maximize the translation
quality. Zheng et al. (2020) extended wait-k to an
adaptive policy by adaptively choosing the strategy
at inference. There is another method that generates oracle READ/WRITE actions by a pre-trained
NMT model and predicts actions using a neuralnetwork model (Zheng et al., 2019). Meaningful
Unit (Zhang et al., 2020) works along the samelines and has outperformed baselines such as MILk
and wait-k.
With respect to the use of syntactic clues for
simultaneous translation, Oda et al. (2015) proposed a method to incrementally parse an incomplete sentence by predicting unseen syntactic constituents on the right and left side of each segment.
They concatenated the predicted constituents andthe words in a segment and then input the result
into tree2string translation. They decided to wait
for more tokens or output the translation dependingon where the constituents appear in the translation
result.
Our proposed method is based on chunk-based
simultaneous translation using chunk boundary detection with simple rules on next-constituent labels.
It basically segments an input before a verb phrase.
This is much simpler and easier to implement than
the work by Zhang et al. (2020) and Oda et al.
(2015).

3

where g(t) is a monotonic non-decreasing function
representing the number of read source tokens to
output the tth target token.
3.2

Chunk-based Simultaneous Translation

We use chunk-based incremental decoding for our
simultaneous translation model and a full-sentence
NMT model trained in a standard manner. However, at the time of inference, we translate the current prefix upon chunk segmentation while keeping
the previously translated output unchanged.
Suppose we have already translated input
chunks X i−1 = X1 , X2 , ..., Xi−1 into an output prefix also represented by chunks: Ye i−1 =Ye1 , Ye2 , ..., Yei−1 , while translating the next input
chunk Xi into Yei . We restart the translationfrom the beginning using all of the available input
chunks X1i . This is similar to an approach called retranslation that generates translations from scratch
for every new input word (Niehues et al., 2016; Arivazhagan et al., 2020), but we apply forced decoding to Ye i−1 in the output prefix. The probabilityof the prefix Ye i can be denoted as follows:
ppref ix (Ye i |X i ) =pf ull (Ye i−1 |X i ) × pchunk (Yei |X i , Ye i−1 ). (3)The first term is calculated in the same way as thestandard full-sentence NMT in Eq. (1) throughforced decoding, and the second term is decomposed as follows, letting Yei = y1i , y2i , ..., y ie :| Yi |

Proposed Method| Yi |
Ye

Figure 1 shows a step-by-step example of our proposed method described in this section.
3.1

Standard Simultaneous Translation

A standard NMT for full sentences is represented
by the following equation:
pf ull (Y |X) =|Y |YP (yt |X, y<t ),

(1)

i

e i−1

pchunk (Yei |X , Y)=i
P (yti |X i , Ye i−1 , y<t
).

t=1

(4)
This can be more efficient than an incremental
Transformer (Ma et al., 2019) that refreshes theencoder for every input word, since our chunkbased translation refreshes the encoder for every
input chunk, which usually consists of multiple
words.

t=1

where X = x1 , x2 , ..., xn is an input sentence consisting of n tokens and Y = y1 , y2 , ..., ym is apredicted target language sentence consisting of m
tokens.
A simultaneous NMT uses only a prefix of the
input to predict a target language token:
psimul (Y |X) =

|Y |
Y
t=1

P (yt |xg(t) , y<t ),

(2)

3.3

Chunk Segmentation

We use constituent labels for our rule-based chunksegmentation as follows.
3.3.1

Incremental Constituent Label
Prediction
We predict the label of a syntactic constituent coming after a sentence prefix at the current time-step.
We call this process Incremental Constituent Label Prediction (ICLP). Here, we define this next

Result of full-sentence parser
Outputwatashi wa

VPwatashi wa

ICLP
I
g =1

(S (NP (PRP I )) (VP (VBD bought) (NP (DT a) (NN pen))))pen wo katta.
NP
ICLP

NMT

I bought

I

g =2

NN

watashi wa pen wo katta .

.

ICLP

ICLP

I bought a

I bought a pen

I bought a pen .

g=3

g=4

g=5

NMT
(src) I bought a pen.
(prefix) watashi wa

Figure 1: One look-ahead ICLP gives constituent labels. When a boundary is detected based on the label and rules,
NMT starts to translate the source subsequence. The previous translation, which is red in the figure, is used asprefix words for the next translation. EOS is omitted for simplicity in the figure.

segmentation
constituent label
syntax treeYou/ can save time by / doing this
.
VP
VP
NP PP
S
NP
.
(S (NP (PRP You)) (VP (MD can) (VP (VB save) (NP (NN time))(PP (IN by) (S (VP (VBG doing) (NP (DT this))))))) (. .))Table 3: Example result of one look-ahead ICLP with a minimum segment size of one.

constituent as the one coming next to the sentence
prefix in pre-order tree traversal. However, thislabel prediction is not easy without observations
on the next constituent. In this work, we allow one
look-ahead, where we read one more word and predict the label of the constituent starting from thatword. This causes an additional delay by one word
but improves the prediction accuracy. Suppose wehave an input sequence W = [w1 , w2 , ..., w|W | ].
The one look-ahead ICLP predicts the constituent
label ci upon the observation of wi , as follows:ci = argmax p(c0 |w≤i ),

(5)

c0 ∈C

where C is a set of constituent labels. Only a prefix
word subsequence is fed into the ICLP, so previous
label predictions do not affect later ones.
We can train the ICLP model as a multiclass classifier using a set of training instancesin the form of prefix-label pairs. One sentence generates several instances for training
data: (w1 , c1 ), (w1 , w2 , c2 ), (w1 , w2 , w3 , c3 ),(w1 , w2 , w3 , w4 , c4 ), and so on. We implemented
the ICLP model in two different ways using LSTM
(Hochreiter and Schmidhuber, 1997) and BERT
(Devlin et al., 2019).
3.3.2 Segmentation Rules
Table 3 shows an example of a result by the one
look-ahead ICLP. We use one basic and two supplemental rules for chunk segmentation as follows.

• Segment the input coming just before constituents labeled S and VP.
• If the previous label is S or VP, do not segment
the input.
• If the chunk is shorter than the minimum
length, do not segment the input.
In incremental translation from Subject-VerbObject to Subject-Object-Verb, the subject can betranslated before observing the verb coming next,
but the verb should be translated after observing theobject. Therefore, the chunk boundary should be
between the subject and verb, not between verb andobject. To achieve this, we employ a simple rule tosegment a chunk just before VP. We also include
S in the rule just as with VP because S (simple
declarative clause) often appears in the form of aunary branch “(S (VP ...))” as shown in Table 3.
However, in cases such as “can save” in the example, VP occurs again immediately after the segmentation before “can.” The basic rule suggests
segmentation before “save,” but it does not seemappropriate. Therefore, we introduce the minimum
segment size to avoid such over-segmentation asa hyperparameter to control the accuracy-latency
trade-off. If the hyperparameter is larger than one,the chunk segmentation after “You” in the example
does not occur.

4

Experimental setup

4.1

Dataset and preprocessing

We conducted experiments on English-Japanese(En-Ja) translation. We also tried English-German(En-De) translation to investigate the difference inlanguage pairs.
For En-Ja, the model was trained on 17.9 M
sentence pairs from WMT2020 and fine-tuned on
223 K sentence pairs from IWSLT2017. We used5312 sentence pairs for the development set fromdev2010, tst2011, tst2012, and tst2013 of IWSLT.
We evaluated the model on 1442 sentence pairs
from dev2021 of IWSLT.
For En-De, the model was trained on 4.5 M
sentence pairs from WMT2014 and fine-tuned on
206 K sentence pairs from IWSLT2017. We used5589 sentence pairs for the development set fromdev2010, tst2011, tst2012, and tst2013 of IWSLT.
We evaluated the model on 1,080 sentence pairs
from tst2015 of IWSLT.
We tokenized English and German sentenceswith tokenizer.perl in Moses (Koehn et al.,2007) and Japanese sentences with MeCab (Kudo,
2005). For each language pair, we used subwords
based on Byte Pair Encoding (BPE) (Sennrich et al.,2016) with a shared vocabulary of 16 K entries. To
develop the subword vocabulary, we used all of
the in-domain training sentences (IWSLT) and one
million out-of-domain sentences (WMT).
We trained the ICLP models using Penn Treebank 3 (Marcus et al., 1993) for training, excluding a randomly selected one percent of sentences
reserved for the development set. We used NAISTNTT TED Talk Treebank (Neubig et al., 2014) forthe evaluation set. The number of training, development, and test instances (e.g., the number of labels
to be predicted) were 2.8 M, 27.9 K, and 21.9 K,
respectively. Note that multiple ICLP instances are
induced from what a single parse tree generates.
4.2

Model settings

We compared the following four models. All ofthem were based on the Transformer-base (Vaswani
et al., 2017).
wait-k
The range of k is [2, 4, 6,..., 30].
Meaningful Unit
The hyperparameter is p, which is the threshold of the probability of a boundary. The

ranges of p are [0.5, 0.1, 0.15,..., 0.95],[0.99, 0.991, 0.992,..., 0.999], and [0.9991,
0.9992,..., 0.9999]. Monotonic translationof Meaningful Unit was generated from thefine-tuning dataset by the fine-tuned NMT
model. We used their refined Meaningful Unit
method, which improved the translation quality at low latency (Zhang et al., 2020)1 . They
used a two look-ahead boundary predictor in
their experiments. We additionally tried a one
look-ahead predictor because it is not certain
how many future words should be used for thepredictor.
Fixed-size segmentation This simply segments
an input with a fixed length specified by a
hyperparameter f, which means the boundarycomes every f subwords or words. The rangeof f is [2, 4, 6,..., 30] for words and [4, 8,
12,..., 60] for subwords.
ICLP
The hyperparameter is m, which means theminimum number of words in one segment.
The range of m is [1, 2, 3, . . . , 29].
We controlled hyperparameters to adapt to awide range of latency. The hyperparameter is given
both in the training and at the inference time for
wait-k, but it is given only at the inference time
for other models. Therefore, we trained the wait-kmodel for each k while in other approaches a single
NMT model is commonly used.
We used fairseq (Ott et al., 2019) to implement
these models and basically followed the official
baseline for IWSLT 20212,3 to set the hyperparameters. We saved checkpoints every 5000 updates
for pre-training and every 200 updates for finetuning. Other hyperparameters were the same forpre-training and fine-tuning. We stopped training
early with patience 4. The max-tokens for the mini
batch size was 4096, and weights were updated
1

Zhang et al. (2020) removed the monotonic translations
with a lower score than full-sentence translation. However, it israre for a monotonic translation to have a higher score than fullsentence translation. Consequently, few sentences remained inour setting. Therefore, we improved the translation quality by
preventing over-translation instead of removing it. Once thesame words are output four times continuously or the target
length becomes four times longer than the source length, we
expand the source prefix.
2
https://github.com/pytorch/fairseq/
blob/master/examples/simultaneous_translation/docs/enja-waitk.md
3
https://github.com/pytorch/fairseq/
issues/346

4.3

16

BLEU

14

We used SimulEval (Ma et al., 2020a) to evaluate
the quality and latency of simultaneous translation.
BLEU (Papineni et al., 2002) was used to evaluate
quality. We used Average Lagging (AL) (Ma et al.,2019) to evaluate the latency. AL is widely usedand defined by the following equation:
τg (|X|)X1
t−1
g(t) −. (6)τg (|X|)γt=1

τg (|X|) is the decoding step when the source sentence finishes. It counts latency up to the τg (|X|)
th target token predicted just after reading the final
source token. γ is defined as |Y |/|X|. When thesource length |X| equals target length |Y |, AL ofwait-k equals its k. In this experiment, the latency
was calculated on character level for En-Ja, andword level for En-De.

waitk
MU1
MU2
fixed (subword)
fixed (word)
VP (BERT)
VP+S (BERT)
full-sentence

12
10
8
2

4

6

8
AL

10

12

14

16

Figure 2: Scatter plot of BLEU and AL (En-Ja). MU1and MU2 correspond to Meaningful Unit with one andtwo look-ahead respectively.

waitk
MU1
MU2
fixed (subword)
fixed (word)
VP (BERT)
VP+S (BERT)
full-sentence

1.4
1.3
1.2
1.1
1.0
2

4

6

8
AL

10

12

14

16

Figure 3: Scatter plot of length ratio and AL (En-Ja)

4.4

Evaluation

ALg (X, Y ) =

18

ratio

every 4 mini batches. We set the learning rate to
0.0007 and trained the model on a single GPU. The
last three models used the same NMT model. We
used beam search within chunks in a standard way
and chose 1-best hypotheses at the end of chunktranslation. The beam size was four for the chunkbased and full-sentence models. We used greedy
decoding for wait-k.
We implemented two types of ICLP models as
mentioned earlier. For the LSTM-based ICLP, weused two-layered unidirectional LSTMs to encode
an input sentence with a fully connected layer
for the constituent label prediction. The numbersof dimensions for embedding and hidden states
are 512. We tokenized English sentences using
tokenizer.perl in Moses and Byte Pair Encoding (Sennrich et al., 2016) with a vocabulary of
16 K entries. For the BERT-based ICLP, we used a
BERT-based classifier with an additional fully connected layer over the [CLS] token, implemented
using Huggingface transformers (Wolf et al., 2020)
with a pre-trained model bert-base-uncased
and the corresponding subword tokenizer. For bothmodels, the input was a subword sequence, so theconstituent label prediction was made upon theobservation of an end-of-word subword. The following training conditions were commonly applied
to both models: learning rate of 5e-5, training batch
size of 512 instances, checkpoints saved at the endof every epoch, and early stopping with the patience of three epochs.

Results

We illustrate the results of English-Japanese translation in Figure 2. Our proposed method outperformed baselines in a wide range of AL. Most of
the points of the proposed method appear to theupper-left of the other methods, thus showing thebest performance. We compared the use of segmentation rules based on VP and VP+S. The points
shifted to the left by adding S as boundary because it increased the number of boundaries anddecreased latency. Although we tried the different
look-ahead lengths of one and two for the boundary
predictor of Meaningful Unit, our proposed model
outperformed both of these models in a wide rangeof latency.
The difference between wait-k and the models
using the full-sentence translation model was large
in the quality-latency trade-off. Surprisingly, thefixed-size segmentation was also effective. Whenthe segment size was fixed, it did not make a large

1000

1000

800

800
frequency

1200

600
400
200
0

600
400
200

0

10

20

30
40
segment length

50

60

0

70

Figure 4: Segment length distribution of fixed-size segmentation with 16 subwords for AL 7.16 (En-Ja test)

Label
Fixed (16 subwords)
1 look-ahead MU
1 look-ahead ICLP (VP+S)

AL
7.16
7.26
7.23

0

10

20

30
40
segment length

50

60

70

Figure 5: Segment length distribution of one lookahead Meaningful Unit for AL 7.26 (En-Ja test)

BLEU
16.34
16.53
17.22

1200
1000
800

Table 4: BLEU results for AL close to 7

difference in the result, regardless of whether theunit was a subword or a word.

5

Analysis

5.1

Length ratio

frequency

frequency

1200

600
400
200
0

0

10

20

30
40
segment length

50

60

70

Figure 3 shows the length ratios of translation hypotheses and references with different latency parameters. Too large a ratio decreases the BLEU
score and makes the content delivery difficult bothin text (subtitles) and speech (text-to-speech).
The length ratio of wait-k was unstable compared to other models because it was trained individually for each k.
Except for wait-k, the length ratios were large
in the range of small latency, probably due to thecondition mismatch between training and inference.
These NMT models were trained on full sentences,
but they were used to translate short segments in
the inference. Therefore, they tend to output longer
segment translations than expected. Their ratios
gradually decrease as AL increases and the lengthof segments becomes closer to the length of full
sentences.

Figure 6: Segment length distribution of one lookahead ICLP model dividing with label S and VP for
AL 7.23 (En-Ja test)

5.2

5.3

Segment length distribution

Figures 4,5 and 6 show the distributions of source
segment length in the En-Ja test set for which AL
is close to 7.2. Table 4 shows their corresponding
AL and BLEU of each model. The length was

calculated as the number of subwords in a segment,
and the previous segment was concatenated to thenext segment when the previous segment has notranslation output.
Segmentation with fixed size 16 has some segments shorter than size 16 because the sentence
length is not always a multiple of 16.
Compared with ICLP model, Meaningful Unit
has wider distribution, and the most segments consist of two subwords. These short segments haveless context information and can output longer segment translation than expected. This would be oneof the reason why our proposed method outperformed Meaningful Unit.
Controlling latency

In Figures 7 and 8, each plot is labeled by the corresponding value of the hyperparameter of inference.
It is difficult to control latency for Meaningful Unit
as shown in the figure. BLEU scores of hyperpa-

0.9995

BLEU

16

Label
NP
VP
NN
,
PP
S

0.9996

0.99
0.95
0.9

14
12

Precision
0.90
0.89
0.95
0.98
0.85
0.87

Recall
0.94
0.97
0.97
1.00
0.93
0.52

F1
0.92
0.93
0.96
0.99
0.89
0.65

Table 5: Results of label prediction (BERT)
10

MU1
2

4

6

8

10

AL

12

14

Label
NP
VP
NN
,
PP
S

16

Figure 7: BLEU and AL with different chunk segmentation thresholds for Meaningful Unit

15
11 13

17.5
9

7

17.0

17 19 2123252729

Label
NP
VP
NN
,
PP
S

BLEU

16.5
16.0
3
VP (BERT)

1
4

6

8

AL

10

12

14

Figure 8: BLEU and AL with different chunk size
thresholds for the proposed method

rameters from 0.9996 to 0.9999 were also the sameas that of a full-sentence translation model.
In contrast, our proposed method can easily control latency because it uses the minimum chunklength as an intuitive hyperparameter to adjust it.
5.4

How many words to wait

Compared with the fixed-size segmentation model,
our proposed model and Meaningful Unit have adisadvantage in AL, which is caused by the lookahead approach. Despite this disadvantage, ourproposed approach outperformed the fixed-size segmentation in a wide range of AL. This means thebenefit of looking at the future words and finding
a better boundary outweighed the above disadvantage.
5.5

Performance of ICLP

Tables 5 and 6 show the results in precision andrecall of the one look-ahead ICLP models. The
LSTM-based ICLP was better in precision, but the

Recall
0.89
0.94
0.92
1.00
0.94
0.52

F1
0.87
0.92
0.92
0.99
0.86
0.64

Table 6: Results of label prediction (LSTM)

5

15.5

Precision
0.85
0.91
0.93
0.98
0.78
0.84

Precision
0.62
0.75
0.60
0.41
0.50
0.77

Recall
0.85
0.80
0.78
0.34
0.47
0.62

F1
0.72
0.78
0.68
0.37
0.48
0.69

Table 7: Results of label prediction (BERT) without
look-ahead

BERT-based ICLP was better in recall for VP. Figure 9 compares them in the downstream simultaneous translation. The lines connected by dots nearlyoverlapped, so there was no large difference in
BLEU score. LSTM is more efficient than BERT
in incremental processes, so it is suitable for practical usage.
Table 7 shows the results by the ICLP model
without one look-ahead approach. Compared with
Table 5, the scores are much lower. One lookahead approach was important to improve its performance.
5.6

En-De translation

We conducted additional experiments in En-De
translation to investigate the performance in a different language. German is another language withdifferent word order from English especially inverbs and also suffers from the reordering problem.
Figure 10 shows the results. This is almost the opposite of the results of the En-Ja translation. The

JP21H03500.
17.5

References

BLEU

17.0
16.5
16.0

VP (LSTM)
VP (BERT)
full-sentence

15.5
4

6

8

10
AL

12

14

Figure 9: Comparison between the use of LSTM- and
BERT-based ICLP

Ashkan Alinejad, Maryam Siahbani, and Anoop Sarkar.
2018. Prediction improves simultaneous neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language
Processing, pages 3022–3027, Brussels, Belgium.
Association for Computational Linguistics.
Naveen Arivazhagan, Colin Cherry, Wolfgang
Macherey, Chung-Cheng Chiu, Semih Yavuz,
Ruoming Pang, Wei Li, and Colin Raffel. 2019.
Monotonic infinite lookback attention for simultaneous machine translation. In Proceedings of the57th Annual Meeting of the Association for Computational Linguistics, pages 1313–1323, Florence,
Italy. Association for Computational Linguistics.

30

BLEU

25
20

waitk
MU1
MU2
fixed (subword)
fixed (word)
VP (BERT)
VP+S (BERT)
full-sentence

15
10
0.0

2.5

5.0

7.5

AL

10.0

12.5

15.0

17.5

Figure 10: Scatter plot of BLEU and AL (En-De)

proposed boundary decision rules used for En-Ja
translation were not so effective for En-De translation, so we need to find other rules to detect boundaries in En-De translation.

6

Conclusion

We proposed a novel segmentation method for simultaneous translation that uses simple rules and
ICLP. Our proposed method is simple, and it outperformed the baselines in the quality-latency trade-off
in En-Ja translation. On the other hand, the proposed method did not work effectively in En-De
translation due to the smaller word order differences than those in En-Ja translation.
In future work, we expect to extract segmentation rules automatically and apply these rules toother language pairs as well.

7

Acknowledgements

Part of this work was supported by JSPS
KAKENHI Grant Numbers JP21H05054 andNaveen Arivazhagan, Colin Cherry, Wolfgang
Macherey, and George Foster. 2020. Re-translation
versus streaming for simultaneous translation. In
Proceedings of the 17th International Conferenceon Spoken Language Translation, pages 220–227,
Online. Association for Computational Linguistics.
Kyunghyun Cho and Masha Esipova. 2016. Can neural machine translation do simultaneous translation?
arXiv preprint arXiv:1606.02012.
Fahim Dalvi, Nadir Durrani, Hassan Sajjad, andStephan Vogel. 2018. Incremental decoding andtraining methods for simultaneous translation in neural machine translation. In Proceedings of the 2018
Conference of the North American Chapter of the
Association for Computational Linguistics: Human
Language Technologies, Volume 2 (Short Papers),
pages 493–499, New Orleans, Louisiana. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2019. BERT: Pre-training ofdeep bidirectional transformers for language understanding. In Proceedings of the 2019 Conferenceof the North American Chapter of the Association
for Computational Linguistics: Human Language
Technologies, Volume 1 (Long and Short Papers),
pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Alvin Grissom II, He He, Jordan Boyd-Graber, John
Morgan, and Hal Daumé III. 2014. Don’t until
the final verb wait: Reinforcement learning for simultaneous machine translation. In Proceedings ofthe 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1342–
1352, Doha, Qatar. Association for Computational
Linguistics.
Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Victor O.K. Li. 2017. Learning to translate in real-timewith neural machine translation. In Proceedings ofthe 15th Conference of the European Chapter of theAssociation for Computational Linguistics: Volume
1, Long Papers, pages 1053–1062, Valencia, Spain.
Association for Computational Linguistics.
Sepp Hochreiter and J¨urgen Schmidhuber. 1997.
Long short-term memory. Neural Computation.
Hideki Isozaki, Katsuhito Sudoh, Hajime Tsukada, andKevin Duh. 2010. Head finalization: A simple reordering rule for SOV languages. In Proceedingsof the Joint Fifth Workshop on Statistical MachineTranslation and MetricsMATR, pages 244–251, Uppsala, Sweden. Association for Computational Linguistics.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris
Callison-Burch, Marcello Federico, Nicola Bertoldi,
Brooke Cowan, Wade Shen, Christine Moran,
Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra
Constantin, and Evan Herbst. 2007. Moses: Open
source toolkit for statistical machine translation. In
Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion
Volume Proceedings of the Demo and Poster Sessions, pages 177–180, Prague, Czech Republic. Association for Computational Linguistics.
Taku Kudo. 2005. Mecab : Yet another part-of-speech
and morphological analyzer. http://mecab.
sourceforge.net/.
Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng,
Kaibo Liu, Baigong Zheng, Chuanqiang Zhang,
Zhongjun He, Hairong Liu, Xing Li, Hua Wu, andHaifeng Wang. 2019. STACL: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. In Proceedings of the 57th Annual Meeting of the Association
for Computational Linguistics, pages 3025–3036,
Florence, Italy. Association for Computational Linguistics.
Xutai Ma, Mohammad Javad Dousti, Changhan Wang,
Jiatao Gu, and Juan Pino. 2020a. SIMULEVAL: An
evaluation toolkit for simultaneous translation. In
Proceedings of the 2020 Conference on Empirical
Methods in Natural Language Processing: System
Demonstrations, pages 144–150, Online. Association for Computational Linguistics.
Xutai Ma, Juan Pino, James Cross, Liezl Puzon, andJiatao Gu. 2020b. Monotonic multihead attention.
In ICLR 2020.
Mitchell P. Marcus, Beatrice Santorini, and Mary Ann
Marcinkiewicz. 1993. Building a large annotatedcorpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330.
Graham Neubig, Katsuhito Sudoh, Yusuke Oda, Kevin
Duh, Hajime Tsukada, and Masaaki Nagata. 2014.
The NAIST-NTT TED talk treebank. In Proceedings of the 11th International Workshop on Spoken
Language Translation (IWSLT), Lake Tahoe, USA.

Jan Niehues, Thai Son Nguyen, Eunah Cho, Thanh-Le
Ha, Kevin Kilgour, Markus Müller, Matthias Sperber, Sebastian Stüker, and Alex Waibel. 2016. Dynamic transcription for low-latency speech translation. In Interspeech 2016, pages 2513–2517.
Yusuke Oda, Graham Neubig, Sakriani Sakti, Tomoki
Toda, and Satoshi Nakamura. 2014. Optimizing segmentation strategies for simultaneous speech translation. In Proceedings of the 52nd Annual Meeting of
the Association for Computational Linguistics (Volume 2: Short Papers), pages 551–556, Baltimore,
Maryland. Association for Computational Linguistics.
Yusuke Oda, Graham Neubig, Sakriani Sakti, Tomoki
Toda, and Satoshi Nakamura. 2015. Syntax-based
simultaneous translation through prediction of unseen syntactic constituents. In Proceedings of the
53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint
Conference on Natural Language Processing (Volume 1: Long Papers), pages 198–207, Beijing,
China. Association for Computational Linguistics.
Myle Ott, Sergey Edunov, Alexei Baevski, Angela
Fan, Sam Gross, Nathan Ng, David Grangier, andMichael Auli. 2019. fairseq: A fast, extensible
toolkit for sequence modeling. In Proceedings ofthe 2019 Conference of the North American Chapter of the Association for Computational Linguistics(Demonstrations), pages 48–53, Minneapolis, Minnesota. Association for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings ofthe 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia,
Pennsylvania, USA. Association for Computational
Linguistics.
Colin Raffel, Minh-Thang Luong, Peter J Liu, Ron J
Weiss, and Douglas Eck. 2017. Online and lineartime attention by enforcing monotonic alignments. In Proceedings of the 34th International
Conference on Machine Learning-Volume 70, pages
2837–2846.
Vivek Kumar Rangarajan Sridhar, John Chen, Srinivas
Bangalore, Andrej Ljolje, and Rathinavelu Chengalvarayan. 2013. Segmentation strategies for streaming speech translation. In Proceedings of the 2013
Conference of the North American Chapter of the
Association for Computational Linguistics: Human
Language Technologies, pages 230–238, Atlanta,
Georgia. Association for Computational Linguistics.
Harsh Satija and Joelle Pineau. 2016. Simultaneousmachine translation using deep reinforcement learning. In Workshops of International Conference on
Machine Learning, page 110–119.
Rico Sennrich, Barry Haddow, and Alexandra Birch.
2016. Neural machine translation of rare words

with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational
Linguistics (Volume 1: Long Papers), pages 1715–
1725, Berlin, Germany. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob
Uszko-reit, Llion Jones, Aidan N. Gomez, Lukasz
Kaiser, and andIllia Polosukhin. 2017. Attention is
all you need. CoRR, page Vol.abs/1706.03762.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien
Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen,
Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu,
Teven Le Scao, Sylvain Gugger, Mariama Drame,
Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing:
System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
Ruiqing Zhang, Chuanqiang Zhang, Zhongjun He, Hua
Wu, and Haifeng Wang. 2020. Learning adaptive
segmentation policy for simultaneous translation. In
Proceedings of the 2020 Conference on Empirical
Methods in Natural Language Processing (EMNLP),
pages 2280–2289, Online. Association for Computational Linguistics.
Baigong Zheng, Kaibo Liu, Renjie Zheng, Mingbo Ma,
Hairong Liu, and Liang Huang. 2020. Simultaneous translation policies: From fixed to adaptive. In
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2847–
2853, Online. Association for Computational Linguistics.
Baigong Zheng, Renjie Zheng, Mingbo Ma, and Liang
Huang. 2019. Simpler and faster learning of adaptive policies for simultaneous translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language
Processing (EMNLP-IJCNLP), pages 1349–1354,
Hong Kong, China. Association for Computational
Linguistics.