NAIST English-to-Japanese Simultaneous Translation System
for IWSLT 2021 Simultaneous Text-to-text Task
Ryo Fukuda, Yui Oka, Yausumasa Kano, Yuki Yano, Yuka Ko, Hirotaka Tokuyama,
Kosuke Doi, Sakriani Sakti, Katsuhito Sudoh, Satoshi Nakamura
Nara Institute of Science and Technology, Nara, Japan
fukuda.ryo.fo3@is.naist.jp

Abstract
This paper describes NAIST’s system for
the English-to-Japanese Simultaneous Text-totext Translation Task in IWSLT 2021 Evaluation Campaign. Our primary submission
is based on wait-k neural machine translation
with sequence-level knowledge distillation to
encourage literal translation.

1

Introduction

Automatic simultaneous translation is an attractive
research field that aims to translate an input before
observing its end for real-time translation similar tohuman simultaneous interpretation. Starting fromearly attempts using rule-based machine translation(Matsubara and Inagaki, 1997; Ryu et al., 2006)
and statistical methods using statistical machinetranslation (Bangalore et al., 2012; Fujita et al.,2013), recent studies successfully applied neuralmachine translation (NMT) into this task (Gu et al.,2017; Ma et al., 2019; Arivazhagan et al., 2019).
The simultaneous translation shared task in the
IWSLT evaluation campaign started on 2020 withEnglish-to-German (Ansari et al., 2020) speechto-text and text-to-text tasks, and a new language
pair of English-to-Japanese has been included on
2021 only in text-to-text task. English-to-Japaneseis much more challenging than English-to-German
due to the large language difference in addition todata scarsity.
We developed an automatic text-to-text simultaneous translation system for this shared task. We
applied some extensions to a standard wait-k NMTin the training time: sequence-level knowledge distillation and target-side chunk shuffling. However,these techniques showed mixed results in different
latency regimes on the IWSLT21 development set,so we configured the system differently for eachlatency regime. This paper describes the details of
the system and the results on the development sets.

We also describe our another attempt to include
incremental constitutent label prediction that was
not included in the primary system.

2

Simultaneous Neural MachineTranslation with wait-k

Let X = x1 , x2 , . . . , x|X| be an input sequence ina source language and Y = y1 , y2 , . . . , y|Y | be an
output sequence in a target language. Here, the input can be speech or text, but we assume the input
is text because this paper discusses the text-to-text
task. The task of simultaneous translation is totranslate X to Y incrementally. In other words,
each output prediction of Y is made upon partial
input observations of X. Suppose an output prefix subsequence Y1j = y1 , y2 , ..., yj has alreadybeen predicted from prefix observations of the input X1i = x1 , x2 , ..., xi . When we predict the next
0joutput subsequence Yj+1
= yj+1 , ..., yj 0 after fur0ither partial observations Xi+1
= xi+1 , ..., xi0 , theprediction is made based on the following formula:
0

0

j
i
Yj+1
= argmax P (Ŷ |X1i , Xi+1
, Y1j )

(1)Ŷ

where Ŷ is a possible prediction of the subsequence.
In a usual consecutive machine translation, we canuse the whole input sequence X anytime in theprediction of Y . The limitation of available input information is a key challenge of simultaneoustranslation.
Wait-k delays the decoding process in k input
tokens (Ma et al., 2019). The wait-k model translates a token sequence of the source language X
into that of the target language Y as follows.
Hi = Encoder(x1 , . . . , xi+k−1 ),(2)ŷi = Decoder(Hi , ŷ1 , . . . , ŷi−1 ).
The decoder has to predict an output token based on
the attention over an observed portion of the input

tokens. k is a hyperparameter for the fixed delayin this model; setting k larger causes longer delays, while smaller k would result in worse output
predictions due to the poor context information.

3

Knowledge Distillation

Knowledge Distillation (KD) (Hinton et al., 2015)
is a method that uses the distilled knowledge
learned by a stronger teacher model in the learningof a weaker student model. When teacher distribution is q(y|x; θT ), we minimize the cross-entropy
with the teacher’s probability distribution insteadof reference data, as follows:
LKD (θ; θT ) = −|V|Xq(y = k|x; θT ) ×

k=1

log p(y = k|x; θ)(3)where θT parameterizes the teacher distribution.
Sequence-level Knowledge Distillation (SKD),
which gives the student model the output of theteacher model as knowledge, propagates a wide
range of knowledge to the student model and trains
it to mimic its knowledge (Kim and Rush, 2016).
The teacher distribution q(Y |X) is approximatedby its mode q(Y |X) ≈ 1{Y = argmax q(Y |X)},
X∈T

and the loss objectives as follows:
LSKD = −Ex∼dataX

q(Y |X) log p(Y |X)

should consist of more literal and NMT-friendly
translations, therefore the training of the student
model becomes easier than that using the original
parallel corpus. Since we have to train simultaneous translation using less context information thanan offline translation model, the SKD would be
helpful. This is motivated by the recent successof non-autoregressive NMT using SKD (Gu et al.,2018).

4

Target-side chunk shuffling

Chunk shuffling is a kind of data augmentation thatreorders Japanese chunks (called bunsetsu). Our
motivation for this one is to encourage monotonic
IMT utilizing a characteristic of Japanese as anagglutinative language, in which the order of bunsetsu chunks is not so strict. When we have a target
language sequence T = t1 , . . . , t|T | in the training
set, we apply greedy left-to-right chunking to it; T
is divided as a chunk sequence T̄ = C1 , . . . , CQ ,in which each chunk consists of k (i.e., delay hyperparameter in wait-k) tokens Cq = tq1 , . . . , tqk .
Note that the last chunk CQ may be shorter than kaccording to the length of T . Then, we choose to
shuffle or keep the chunks in T̄ with a probability
pr , defined as a hyperparameter. We tried only the
random shuffling with the fixed chunk size of kin this time; More linguistically-motivated chunkreordering would be worth trying as future work.

Y ∈T≈ −EX∼data,Yb =argmax q(Y |X) [log p(y = Yb |X)] (4) 5 Primary systemY ∈T
5.1 Implementation
where p(Y |X) is the sequence-level distribution, Our system implementation was based on the ofand Y ∈ T is the space of possible target sentences. ficial baseline1 using fairseq (Ott et al., 2019) andSKD can be implemented simply by training the
SimulEval (Ma et al., 2020).
student model using (X, Yb ), where Yb is derived
from the teacher model outputs for the source lan- 5.2 Setup
guage portion of the training corpus.
Data All of the models were based on TransWe use SKD for reduction of colloquial expres- former, trained using 17.9 million Englishsions in the spoken language corpus. Such col- Japanese parallel sentences from WMT20 news
loquial expressions are highly dependent on lan- task and fine-tuned using 223 thousand parallelguages and difficult to translate by NMT, which
sentences from IWSLT 2017. During fine-tuning,
usually generates literal translations. Here, we
we examined the effectiveness of knowledge distilfirstly train a teacher, Transformer-based offline
lation and chunk shuffling with several hyperparamNMT model using the training corpus and use it toeter settings and reported the results by the models
obtain pseudo-reference translations in the target
that resulted in the higher BLEU on IWSLT 2021
language. Then, we train a student, Transformer- development set. The text was preprocessed bybased simultaneous NMT model using the pseudo- Byte Pair Encoding (BPE) (Sennrich et al., 2016)
parallel corpus with the original source language
1
https://github.com/pytorch/fairseq/
sentences and the corresponding translation re- blob/master/examples/simultaneous_sults by the teacher model. The pseudo-referencestranslation/docs/enja-waitk.md

BLEU
16.8

AL
-

16
18

11.8
14.69
15.57

7.27
11.47
13.7

14

k=10

12

16
12

14

12

14

k=10

18

20

20

24
22

26 28 30 32
28 30

24

22

32

26

16

10

13.77
13.5
15.22
15.21

7.29
7.28
11.48
13.71

Table 1: In-house results of our systems on IWSLT
2021 En-Ja development set. Superscripts low , medium
and high represent the systems submitted for low-,
medium-, and high-latency regimes, respectively.

for subword segmentation. The vocabulary wasshared over English and Japanese, and its size was
16,000.
Model The hyperparameters of the model almost
followed the Transformer Base settings (Vaswani
et al., 2017). The encoder and decoder were composed of 6 layers. We set the word embedding
dimensions, hidden state dimensions, feed-forward
dimensions to 512, 512, and 2,048, respectively.
We performed the sub-layer’s dropout with a probability of 0.1. The number of attention heads waseight for both the encoder and decoder. The model
was optimized using Adam with an initial learning
rate of 0.0007, β1 = 0.9, and β2 = 0.98, following
Vaswani et al. (2017).
Evaluation To evaluate the performance, we calculated BLEU and Average lagging (AL) (Ma et al.,2019) with SimulEval on IWSLT 2021 development set.
5.3

18

BLEU

SystemofflineBaseline
wait-10
wait-20
wait-30high
Proposedwait-10 + CShuflow
wait-10 + SKD
wait-20 + SKDmedium
wait-30 + SKD

Results on the development set

Table 1 shows the excerpt of system results forthe full-sentence topline (offline), wait-k baselines(wait-k), and our extensions: SKD (+ SKD) andchunk shuffling (+ CShuf).
We tried some different latency hyperparametervalues k = {10, 12, 14, . . . , 32} for comparison.
Figure 1 plots our BLEU-AL results for wait-k
and wait-k+SKD. It shows that the use of SKD
gave some improvements in low-latency settings
with k = {10, 12, 14}, however, the results withlarger k were mixed. These results support ourassumption on the difficulty of the translation into
colloquial expressions discussed in Section 3.

wait-kwait-k+SKD

8
6
6

8

10

12

14

16

Average Lagging

Figure 1: Translation quality against latency for wait-k
and SKD-based wait-k on IWSLT 2021 En-Ja development set. The broken line shows the score of the offlinemodel.

SystemBaseline+ CShuf

pr
0
0.01
0.02
0.03

BLEU
11.80
10.57
13.77
9.87

lenhyp34,37638,25729,36942,296lenref27,891
27,891
27,891
27,891

Table 2: Target-side chunk shuffling result in pr ={0, 0.01, 0.02, 0.03}

We also tried chunk shuffling with different hyperparameter values2 pr = {0, 0.01, 0.02, 0.03}.
Table 2 shows the result using the target-side chunk
shuffling. Here, the chunk shuffling results are only
shown for wait-10. The use of larger latency hyperparameter k did not show remarkable differencesfrom the baseline. Chunk shuffling with pr = 0.02
resulted in the best BLEU and outperformed thebaseline, but the other values 0.01, 0.03 did notwork. These differences should be due to the output length shown in lenhyp column in Table 2; theoutput length became much shorter than the baseline using the chunk shuffling with pr = 0.02. In
contrast, pr = 0.01 and pr = 0.03 increased theoutput length.
Table 3 shows translation examples by the baseline and chunk-shuffling (pr = 0.02). Here,the baseline translation results do not have endof-sentence expressions like で す (desu), ま す(masu), ですよね (desuyone). These differenceswere not straightforward with the chunk shuffling,
but a certain value of pr = 0.02 worked in our
experiment.
The results above suggest that the target-side
2Higher values of pr resulted in much worse results andare not included in this paper.

En-input
Baseline
CShuf
Ja-refEn-input
Baseline
CShuf
Ja-refEn-input
Baseline

CShuf
Ja-ref

I see other companies that say, “I’ll win the next innovation cycle, whatever it takes.”
他 の 会社 が 「 次 の イノベーション サイクル に <unk> 」と 言う の は どんな もの で あれ
他 の 会社 が 「 次 の イノベーション サイクル に 勝 てる 」 と 言う の を 見 ます
私 の 経験 で も 沢山 の 企業 が 同じ よう に 「 何 が なん でも 次 の イノベーション サイクル を 制覇 する 」 と言い 続け て ますShe’s a musical instrument maker, and she does a lot of wood carving for a living.
彼女 は 楽器 の 製造 者 で 木 彫 り を し て 生き て いる 間 に
彼女 は 楽器 の 製作 者 で 木 彫 り を し て い ます
彼女 は 楽器 の 制作 技師 です 木 を 削る こと で 生計 を 立て て い ます
Humans are very good at considering what might go wrong if we try something new, say, ask for a raise.
人間 は 何 が うまく いけ ば 何 が うまく いけ ば 何 が うまく いけ ば 何 が うまく いけ ば 何 が
うまく いけ ば 何 が うまく いけ ば 何 が うまく いけ ば 何 が うまく いけ ば 何 が うまく いけ ば 何 が
うまく いけ ば 何 が うまく いけ ば 何 が うまく いけ ば 何 が うまく いけ ば 何 が 起き て も 何 が 起き て も
何 が 起き て も 何 が 起き て も 何 が 起き て も 何 が 起き て も 何 が 起き て
人間 は 何 が 間違っ て いる の か を 考える の が 得意 です 新しい こと を 試し て み て も いい です よ ね
昇給 を 求める という よう な 何 か 新しい こと を 試みよ う という とき 人 は どう まずい こと に なり 得る か
考える こと に 長け て い ます

Table 3: Translation examples by wait-k baseline and wait-k with chunk shuffling (pr = 0.02).

Systemwait-10 + CShuflow
wait-20 + SKDmedium
wait-30high

BLEU
14.41
16.20
16.19

AL
7.21
11.54
13.83

Table 4: Official results of our submissions on IWSLT
2021 En-Ja test set.

chunk shuffling may work as a perturbation, andwe need further investigation.
5.4

Official results on the test set

Table 4 shows BLEU and AL results on the test
set. The system with the medium latency regime(wait-20 + SKD) worked relatively well; it achived
a comparable BLEU result with wait-30. However,
the results were worse than those of the other teams
by around two points in BLEU in all the latency
regimes.

6

Another attempt: Incremental Next
Constituent Label Prediction

We tried another technique described below in theshared task, but it was not included in our primarysubmission because it did not outperform the baseline. Here, we also describe this for further investigation in future.
For simultaneous machine translation, deciding
how long to wait for input before translation isimportant. Predicting what kind of phrase comesnext is a part of useful information in determining
the timing. In this study, we tried incremental Next
Constituent Label Prediction (NCLP).
In SMT-based simultaneous translation, Oda
et al. (2015) proposed a method to predict unseen
syntactic constituents to determine when to starttrain
2,762,408

dev
27,903eval
21,941

Table 5: Number of NCLP instances.

translation for partially-observed input, using a
multi-label classifier based on linear SVMs (Fanet al., 2008). Motivated by this study, we used aneural network-based classifier using BERT (Devlin et al., 2019) for NCLP. The problem of NCLP
is defined as the label prediction of a syntactic constituent coming next to a given word subsequencein the pre-order tree traversal. In this work, weused 1-lookahead prediction, so the problem wasrelaxed into the prediction of a label of a syntactic
constituent given its preceding words and the firstword composing it. A predicted constituent label
was inserted at the corresponding position in the input word sequence, immediately after its preceding
word. That doubled the length of input sequences.
For subword-based NMT, we applied BPE only
onto words in the input sequences and put dummylabels after subwords other than end-of-word ones,
to order the input in an alternating way.
We used Huggingface transformers (Wolf et al.,2020) for our implementation of NCLP with
bert-base-uncased. We used Penn Treebank
3 (Marcus et al., 1993) for the NCLP training anddevelopment sets, and NAIST-NTT TED Talk Treebank (Neubig et al., 2014) for the NCLP evaluation
set. Table 5 shows the number of training, development, and evaluation instances extracted from thedatasets. Note that we can extract many instancesfrom a single parse tree.
Table 6 shows the results of the 5 most frequentlabels in the NCLP training data. NP and VP arenatural than baseline like these examples. However,many sentences are not informative and missing
details compared to the baseline. We’ll investigate
a more effective way to use NCLP in our future
work.

18
16

wait-kwait-k+NCLP

BLEU

14

32

12

k=10

56
52

40

60

64

48

28

44

7

12
k=10

6
0

2

4

6

8

10

12

14

16

Average Lagging

Figure 2: Translation quality against latency for waitk and NCLP-based wait-k on IWSLT21 En-Ja dev set.
The broken line shows the score of the offline model.

Label
NP
VP
NN
,
PP

Conclusion

22

14

8

26 2830
22 24
32

24

18

16

16

12

20

10

36
14

18 20

Precision
0.90
0.89
0.95
0.98
0.85

Recall
0.94
0.97
0.97
1.00
0.93

F1
0.92
0.93
0.96
0.99
0.89

In this paper, we described our English-to-Japanese
text-to-text simultaneous translation system. We extended the baseline wait-k with the knowledge distillation to encourage literal translation and targetside chunk shuffling to relax the output order inJapanese. They achieved some improvements on
IWSLT 2021 development set in certain latency
regimes.

Acknowledgments
Part of this work was supported by JSPS KAKENHI Grant Number JP17H06101.

References
Table 6: NCLP results on the evaluation set.

important clues of the sentence structure, and their
F1 scores were over 90% on the NCLP evaluation
data.
However, the results by wait-k using NCLP results as its input did not outperform the baseline
wait-k, as shown in Figure 2. We can observe
NCLP-based wait-k gave smaller ALs with thesame latency hyperparameter k. One possible problem of current NCLP-based wait-k is that the lengthof an input length is doubled by the additional
constitutent labels. Since we ran wait-k-based simultaneous NMT for such an augmented input sequence, the decoder using NCLP-augmented input
has roughly half of the information compared to thedecoder using original input if we use the same k.
This forces the decoder to perform very aggressive
anticipation with limited information from an input
prefix.
Table 7 shows the translation input and output
examples of baseline and NCLP. Input sentences include constituents labels. The first example showsthat NCLP could translate “publication” before averb “work” following the Japanese sentence order.
Second example shows NCLP output is constructednaturally in terms of grammar, while the baseline
has repetitive and unnatural phrases. We observed
NCLP sentences are tend to be shorter and more

Ebrahim Ansari, Amittai Axelrod, Nguyen Bach,
Ondřej Bojar, Roldano Cattoni, Fahim Dalvi, Nadir
Durrani, Marcello Federico, Christian Federmann,
Jiatao Gu, Fei Huang, Kevin Knight, Xutai Ma, Ajay
Nagesh, Matteo Negri, Jan Niehues, Juan Pino, Elizabeth Salesky, Xing Shi, Sebastian Stüker, Marco
Turchi, Alexander Waibel, and Changhan Wang.
2020. FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN. In Proceedings of the 17th International Conference on Spoken Language Translation, pages 1–34, Online. Association for Computational Linguistics.
Naveen Arivazhagan, Colin Cherry, Wolfgang
Macherey, Chung-Cheng Chiu, Semih Yavuz,
Ruoming Pang, Wei Li, and Colin Raffel. 2019.
Monotonic infinite lookback attention for simultaneous machine translation. In Proceedings of the57th Annual Meeting of the Association for Computational Linguistics, pages 1313–1323, Florence,
Italy. Association for Computational Linguistics.
Srinivas Bangalore, Vivek Kumar Rangarajan Sridhar, Prakash Kolan, Ladan Golipour, and Aura
Jimenez. 2012. Real-time incremental speech-tospeech translation of dialogs. In Proceedings ofthe 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 437–
445, Montréal, Canada. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2019. BERT: Pre-training ofdeep bidirectional transformers for language understanding. In Proceedings of the 2019 ConferenceEn-input
En-input (with label)
Baseline
NCLP
Ja-refEn-input
En-input (with label)

Baseline

NCLP
Ja-refI won’t work with you until your publication, or your organization, is more inclusive of all kinds of difference.”
I VP won NP &apos;t NNP work PP with NP you SBAR until NP your NN public@@ @@@ ation , , CC or NP yourNN organization , , VP is ADJP more JJ in@@ @@@ clusive PP of NP all NNS kinds PP of NP difference . . : &quot;
出版 まで は 一緒 に 働 か ない し 組織 も すべて の 種類 の 違い を 包 括 し て いる 」
私 は あなた と 仕事 を し ませ ん 出版 さ れる まで は
「 あなた の 出版 物 や 組織 が 多様 性 を 受け入れる まで は ご 一緒 に 仕事 は でき ませ ん 」 と 言う ことも でき ますThose of us who are underrepresented and invited to participate in such projects, can also decline to be included until
more of us are invited through the glass ceiling, and we are tokens no more.
Those PP of NP us SBAR who SQ are VP under@@ @@@ represented CC and VP invited PP to VP participate
PP in NP such NNS projects , , VP can ADVP also VP decline PP to VP be VP included PP until NP more PP of NP us
VP are VP invited PRT through NP the NN glass NN ceiling , , CC and S we VP are NP tok@@ @@@ ens ADVP no
RBR more . .
私 たち は この よう な プロジェクト に 参加 し て いる 人 たち は 私 たち が この よう な プロジェクト に 参加
し て いる 人 たち は ガラス の 天井 を通して 招待 さ れる まで は その 人 たち は その 人 たち の 中 に 含まれる こと を 拒否 する こと が でき ます そして 私 たち は もう トーク ン を 持っ て い ませ ん
私 たち の 代表 で あり 、 招待 さ れ て いる 人 たち は 、 この よう な プロジェクト に 参加 する こと も できます 。
過@@ 小 評価 さ れ て いる 私 たち の よう な 者 が 招待 さ れ た プロジェクト へ の 参加 を 断@@ る こと もでき ます 更に 多く の 者 が ガラス の 天井 を ぬ@@ け 名 ばかり の 女性 参@@ 画 で は なく なる まで はTable 7: Translation examples by wait-k baseline and wait-k with NCLP.

Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, XiangRui Wang, and Chih-Jen Lin. 2008. LIBLINEAR:
A library for large linear classification. The Journalof Machine Learning Research.

Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng,
Kaibo Liu, Baigong Zheng, Chuanqiang Zhang,
Zhongjun He, Hairong Liu, Xing Li, Hua Wu, andHaifeng Wang. 2019. STACL: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. In Proceedings of the 57th Annual Meeting of the Association
for Computational Linguistics, pages 3025–3036,
Florence, Italy. Association for Computational Linguistics.

Tomoki Fujita, Graham Neubig, Sakriani Sakti,
Tomoki Toda, and Satoshi Nakamura. 2013. Simple, Lexicalized Choice of Translation Timing
for Simultaneous Speech Translation.
In Proceedings of the 14th Annual Conference of theInternational Speech Communication Association(INTERSPEECH-2013), pages 3487–3491.

Xutai Ma, Mohammad Javad Dousti, Changhan Wang,
Jiatao Gu, and Juan Pino. 2020. SIMULEVAL: An
evaluation toolkit for simultaneous translation. In
Proceedings of the 2020 Conference on Empirical
Methods in Natural Language Processing: System
Demonstrations, pages 144–150, Online. Association for Computational Linguistics.

Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K. Li, and Richard Socher. 2018. NonAutoregressive Neural Machine Translation. In 6thInternational Conference on Learning Representations (ICLR 2018).

Mitchell P. Marcus, Beatrice Santorini, and Mary Ann
Marcinkiewicz. 1993. Building a large annotatedcorpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330.

of the North American Chapter of the Association
for Computational Linguistics: Human Language
Technologies, Volume 1 (Long and Short Papers),
pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.

Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Victor O.K. Li. 2017. Learning to translate in real-timewith neural machine translation. In Proceedings ofthe 15th Conference of the European Chapter of the
Association for Computational Linguistics: Volume
1, Long Papers, pages 1053–1062, Valencia, Spain.
Association for Computational Linguistics.
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean.
2015. Distilling the knowledge in a neural network.
In NIPS Deep Learning and Representation Learning Workshop.
Yoon Kim and Alexander M. Rush. 2016. Sequencelevel knowledge distillation. In Proceedings of the2016 Conference on Empirical Methods in Natural Language Processing, pages 1317–1327, Austin,
Texas. Association for Computational Linguistics.

Shigeki Matsubara and Yasuyoshi Inagaki. 1997. Incremental Transfer in English-Japanese Machine Translation. IEICE Transactions on Information and Systems, E80-D(11):1122–1130.
Graham Neubig, Katsuhito Sudoh, Yusuke Oda, Kevin
Duh, Hajime Tsukada, and Masaaki Nagata. 2014.
The NAIST-NTT TED talk treebank. In Proceedings of the 11th International Workshop on Spoken
Language Translation (IWSLT), Lake Tahoe, USA.
Yusuke Oda, Graham Neubig, Sakriani Sakti, Tomoki
Toda, and Satoshi Nakamura. 2015. Syntax-based
simultaneous translation through prediction of unseen syntactic constituents. In Proceedings of the
53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint
Conference on Natural Language Processing (Volume 1: Long Papers), pages 198–207, Beijing,
China. Association for Computational Linguistics.

Myle Ott, Sergey Edunov, Alexei Baevski, Angela
Fan, Sam Gross, Nathan Ng, David Grangier, andMichael Auli. 2019. fairseq: A fast, extensible
toolkit for sequence modeling. In Proceedings ofthe 2019 Conference of the North American Chapter of the Association for Computational Linguistics(Demonstrations), pages 48–53, Minneapolis, Minnesota. Association for Computational Linguistics.
Koichiro Ryu, Shigeki Matsubara, and Yasuyoshi Inagaki. 2006. Simultaneous English-Japanese spoken language translation based on incremental dependency parsing and transfer. In Proceedings of the
COLING/ACL 2006 Main Conference Poster Sessions, pages 683–690, Sydney, Australia. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra Birch.
2016. Neural machine translation of rare words
with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational
Linguistics (Volume 1: Long Papers), pages 1715–
1725, Berlin, Germany. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob
Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz
Kaiser, and Illia Polosukhin. 2017. Attention is all
you need. In Advances in neural information processing systems, pages 5998–6008.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien
Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen,
Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu,
Teven Le Scao, Sylvain Gugger, Mariama Drame,
Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing:
System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.