Tree as a Pivot: Syntactic Matching Methods in Pivot Translation
Akiva Miura† , Graham Neubig‡,† , Katsuhito Sudoh† , Satoshi Nakamura††
Nara Institute of Science and Technology, Japan‡
Carnegie Mellon University, USA
miura.akiba.lr9@is.naist.jp gneubig@cs.cmu.edu
sudoh@is.naist.jp
s-nakamura@is.naist.jp

Abstract[X1] 记录 [X2]

[X1] enregistrer [X2]
[X1] record [X2]

Pivot translation is a useful method fortranslating between languages with little orno parallel data by utilizing parallel data
in an intermediate language such as English. A popular approach for pivot translation used in phrase-based or tree-basedtranslation models combines source-pivot
and pivot-target translation models into asource-target model, as known as triangulation. However, this combination is basedon the constituent words’ surface forms
and often produces incorrect source-target
phrase pairs due to semantic ambiguity inthe pivot language, and interlingual differences. This degrades translation accuracy.
In this paper, we propose a approach forthe triangulation using syntactic subtrees
in the pivot language to distinguish pivot
language words by their syntactic roles toavoid incorrect phrase combinations. Experimental results on the United Nations
Parallel Corpus show the proposed method
gains in all tested combinations of language, up to 2.3 BLEU points.1

1

Introduction

In statistical machine translation (SMT) (Brown
et al., 1993), it is known that translation with models trained on larger parallel corpora can achieve
greater accuracy (Dyer et al., 2008). Unfortunately, large bilingual corpora are not readily
available for many language pairs, particularly
those that do not include English. One effective solution to overcome the scarceness of bilingual datais to introduce a pivot language for which paral1
Code to replicate the experiments can be found athttps://github.com/akivajp/wmt2017[X2] [X1] 记录[X1] dossier [X2](a) Standard triangulation method matching phrases
VP
VP[X1] enregistrer [X2]TO

[X1] 记录 [X2]VB

NP

record [X2]

[X1]

NP
NP

[X1] dossier [X2]DT
[X1]

[X2] [X1] 记录NNNPrecord [X2](b) Proposed triangulation method matching subtrees

Figure 1: Example of disambiguation by parse
subtree matching (Fr-En-Zh), [X1] and [X2] arenon-terminals for sub-phrases.
lel data with the source and target languages exists(de Gispert and Mariño, 2006).
Among various methods using pivot languages,
one popular and effective method is the triangulation method (Utiyama and Isahara, 2007; Cohn
and Lapata, 2007), which first combines sourcepivot and pivot-target translation models (TMs)
into a source-target model, then translates using
this combined model. The procedure of triangulating two TMs into one has been examined for
different frameworks of SMT and its effectivenesshas been confirmed both in Phrase-Based SMT(PBMT) (Koehn et al., 2003; Utiyama and Isahara,2007) and in Hierarchical Phrase-Based SMT (Hiero) (Chiang, 2007; Miura et al., 2015). However, word sense ambiguity and interlingual differences of word usage cause difficulty in accurately learning correspondences between source
and target phrases, and thus the accuracy obtainedby triangulated models lags behind that of models

trained on direct parallel corpora.
In the triangulation method, source-pivot andpivot-target phrase pairs are connected as a sourcetarget phrase pair when a common pivot-side
phrase exists. In Figure 1 (a), we show an example
of standard triangulation on Hiero TMs that combines hierarchical rules of phrase pairs by matching pivot phrases with equivalent surface forms.
This example also demonstrates problems of ambiguity: the English word “record” can correspond to several different parts-of-speech according to the context. More broadly, phrases including this word also have different possible grammatical structures, but it is impossible to uniquely
identify this structure unless information about thesurrounding context is given.
This varying syntactic structure will affect translation. For example, the French verb “enregistrer” corresponds to the English verb “record”,
but the French noun “dossier” also corresponds to“record” — as a noun. As a more extreme example, Chinese is a languages that does not haveinflections according to the part-of-speech of theword. As a result, even in the contexts where“record” is used with different parts-of-speech, theChinese word “记录” will be used, although theword order will change. These facts might result in
an incorrect connection of “[X1] enregistrer [X2]”and “[X2] [X1] 记录” even though proper correspondence of “[X1] enregistrer [X2]” and “[X1]dossier [X2]” would be “[X1] 记 录 [X2]” and“[X2] [X1] 记 录”. Hence a superficial phrase
matching method based solely on the surface formof the pivot will often combine incorrect phrase
pairs, causing translation errors if their translation
scores are estimated to be higher than the propercorrespondences.
Given this background, we hypothesize that disambiguation of these cases would be easier if thenecessary syntactic information such as phrase
structures are considered during pivoting. To incorporate this intuition into our models, we propose a method that considers syntactic informationof the pivot phrase, as shown in Figure 1 (b). In thisway, the model will distinguish translation rules
extracted in contexts in which the English symbol string “[X1] record [X2]” behaves as a verbal
phrase, from contexts in which the same string acts
as nominal phrase.
Specifically, we propose a method based on
Synchronous Context-Free Grammars (SCFGs)(Aho and Ullman, 1969; Chiang, 2007), which
are widely used in tree-based machine translation
frameworks (§2). After describing the baseline triangulation method (§3), which uses only the surface forms for performing triangulation, we propose two methods for triangulation based on syntactic matching (§4). The first places a hard restriction on exact matching of parse trees (§4.1)
included in translation rules, while the second
places a softer restriction allowing partial matches(§4.2). To investigate the effect of our proposed
method on pivot translation quality, we perform
experiments of pivot translation on the United Nations Parallel Corpus (Ziemski et al., 2016), whichshows that our method indeed provide significant
gains in accuracy (of up to 2.3 BLEU points), in almost all combinations of 5 languages with English
as a pivot language (§5). In addition, as an auxiliary result, we compare pivot translation using the
proposed method with zero-shot neural machinetranslation, and find that triangulation of symbolictranslation models still significantly outperforms
neural MT in the zero-resource scenario.

2 Translation Framework
2.1 Synchronous Context-Free Grammars
In this section, first we cover SCFGs, which arewidely used in machine translation, particularly hierarchical phrase-based translation (Hiero) (Chiang, 2007). In SCFGs, the elementary structures
used in translation are synchronous rewrite ruleswith aligned pairs of source and target symbols on
the right-hand side:⟨⟩
X → s, t(1)where X is the head symbol of the rewrite rule,and s and t are both strings of terminals and nonterminals on the source and target side respectively. Each string in the right side pair has thesame number of indexed non-terminals, and identically indexed non-terminals correspond to eachother. For example, a synchronous rule could takethe form of:
X → ⟨X0 of X1 , X1 的 X0 ⟩ .

(2)Synchronous rules can be extracted based onparallel sentences and automatically obtained
word alignments. Each extracted rule is scoredwith phrase translation probabilities in both directions ϕ(s|t) and ϕ(t|s), lexical translation probabilities in both directions ϕlex (s|t) and ϕlex (t|s),a word penalty counting the terminals in t, and aconstant phrase penalty of 1.
At translation time, the decoder searches for thetarget sentence that maximizes the derivation probability, which is defined as the sum of the scores of
the rules used in the derivation, and the log of thelanguage model (LM) probability over the target
strings. When not considering an LM, it is possible to efficiently find the best translation for an input sentence using the CKY+ algorithm (Chappelier et al., 1998). When using an LM, the expanded
search space is further reduced based on a limit on
expanded edges, or total states per span, through aprocedure such as cube pruning (Chiang, 2007).
2.2

Hierarchical Rules

In this section, we specifically cover the rules usedin Hiero. Hierarchical rules are composed of initialhead symbol S, and synchronous rules containing
terminals and single kind of non-terminals X.2 Hierarchical rules are extracted using the same phrase
extraction procedure used in phrase-based translation (Koehn et al., 2003) based on word alignments, followed by a step that performs recursive
extraction of hierarchical phrases (Chiang, 2007).
For example, hierarchical rules could take theform of:X → ⟨Officers, 主席团 成員⟩(3)

X → ⟨the Committee, 委员会⟩(4)

X → ⟨X0 of X1 , X1 的 X0 ⟩ .

(5)From these rules, we can translate the input sentence by derivation:

rules, it has the tendency to extract very large translation rule tables and also tends to be less syntactically faithful in its derivations.
2.3 Explicitly Syntactic Rules
An alternative to Hiero rules is the use of synchronous context-free grammar or synchronoustree-substitution grammar (Graehl and Knight,
2004) rules that explicitly take into account thesyntax of the source side (tree-to-string rules), target side (string-to-tree rules), or both (tree-to-tree
rules). Taking the example of tree-to-string (T2S)
rules, these use parse trees on the source language
side, and the head symbols of the synchronous
rules are not limited to S or X, but instead use
non-terminal symbols corresponding to the phrase
structure tags of a given parse tree. For example,
T2S rules could take the form of:
XNP → ⟨(NP (NNS Officers)), 主席团 成員⟩
XNP → ⟨(NP (DT the) (NNP Committee)), 委员会⟩⟨⟩
XPP → (PP (IN of) XNP,0 ), X0 的⟨⟩
XNP → (NP XNP,0 XPP,1 ), X1 X0

XROOT

→
⇒
⇒

⇒

⇒ ⟨X1 of X2 , X2 的 X1 ⟩⇒ ⟨Officers of X2 , X2 主席团 成員⟩⇒ ⟨Officers of the Committee,
委员会 的 主席团 成員⟩
The advantage of Hiero is that it is able toachieve relatively high word re-ordering accuracy (compared to other symbolic SMT alternatives such as standard phrase-based MT) without language-dependent processing. On the other
hand, since it does not use syntactic informationand tries to extract all possible combinations of
It is also standard to include a glue rule S → ⟨X0 , X0 ⟩,S → ⟨S0 X1 , S0 X1 ⟩, S → ⟨S0 X1 , X1 S0 ⟩ to fall backon when standard rules cannot result in a proper derivation.
2

(8)
(9)

.
Here, parse subtrees of the source language rules
are given in the form of S-expressions.
From these rules, we can translate from the parsetree of the input sentence by derivation:∗

S → ⟨X0 , X0 ⟩

(6)
(7)

⟨
⟨
⟨

⟩
XNP,0 , X0⟩

(NP XNP,1 XPP,2 ), X2 X1(NP (NP (NNS Officers) XPP,2 )), X2 主席团 成員⟨(NP(NP (NNS Officers))(PP (IN of), 委员会 的 主席团 成員(NP (DT the)
(NNP Committee))))⟩⟩In this way, it is possible in T2S translation to
obtain a result conforming to the source language’s
grammar. This method also has the advantage thenumber of less-useful synchronous rules extracted
by syntax-agnostic methods such as Hiero are reduced, making it possible to learn more compact
rule tables and allowing for faster translation.

3 Standard Triangulation Method
In the triangulation method by Cohn and Lapata(2007), we first train source-pivot and pivot-target
rule tables as TSP and TP T respectively. Then we
search TSP and TP T for source-pivot and pivottarget rules having a common pivot phrase, andsynthesize them into source-target rules to create
rule table TST :⟨⟩X → s, t ∈ TST⟨ ⟩s.t. X → ⟨s, p⟩ ∈ TSP ∧ X → p, t ∈ TP T .

(10)

For all the combined source-target rules, phrasetranslation probability ϕ(·) and lexical translation
probability ϕlex (·) are estimated according to thefollowing equations:
( )
ϕ t|s
( )
ϕ s|t
( )
ϕlex t|s( )ϕlex s|t=∑p∈TSP ∩TP T=∑

p∈TSP ∩TP T=∑

p∈TSP ∩TP T

=

∑

( )
ϕ t|p ϕ (p|s) ,

(11)

( )
ϕ (s|p) ϕ p|t ,

(12)

( )
ϕlex t|p ϕlex (p|s) , (13)( )ϕlex (s|p) ϕlex p|t . (14)p∈TSP ∩TP T

The equations (11)-(14) are based on the memoryless channel model, which assumes:()( )ϕ t|p, s = ϕ t|p ,)(ϕ s|p, t = ϕ (s|p)

(15)
(16)

. For example, in equation (15), it is assumed thatthe translation probability of target phrase given
pivot and source phrases is never affected by thesource phrase. However, it is easy to come up with
examples where this assumption does not hold.
Specifically, if there are multiple interpretations of
the pivot phrase as shown in the example of Figure
1, source and target phrases that do not correspondto each other semantically might be connected, andover-estimation by summing products of the translation probabilities is likely to cause failed translations.

4

Triangulation with Syntactic Matching

In the previous section, we explained about thestandard triangulation method and mentioned thatthe pivot-side ambiguity causes incorrect estimation of translation probability and the translation
accuracy might decrease. To address this problem, it is desirable to be able to distinguish pivotside phrases that have different syntactic roles or
meanings, even if the symbol strings are exactly
equivalent. In the following two sections, we describe two methods to distinguish pivot phrases
that have syntactically different roles, one basedon exact matching of parse trees, and one based on
soft matching.

4.1 Exact Matching of Parse Subtrees
In the exact matching method, we first train pivotsource and pivot-target T2S TMs by parsing thepivot side of parallel corpora, and store them intorule tables as TP S and TP T respectively. Synchronous rules of TP S and⟨ T⟩P T take the form of
X → ⟨p̂, s⟩ and X → p̂, t respectively, wherep̂ is a symbol string that expresses pivot-side parse
subtree (S-expression), s and t express source andtarget symbol strings. The procedure of synthesizing source-target synchronous rules essentially follows equations (11)-(14), except using TP S insteadof TSP (direction of probability features is reversed) and pivot subtree p̂ instead of pivot phrase
p. Here s and t do not have syntactic information,
therefore the synthesized synchronous rules should
be hierarchical rules explained in §2.2.
The matching condition of this method has
harder constraints than matching of superficial
symbols in standard triangulation, and has the potential to reduce incorrect connections of phrase
pairs, resulting in a more reliable triangulated TM.
On the other hand, the number of connected rules
decreases as well in this restricted triangulation,and the coverage of the triangulated model might
be reduced. Therefore it is important to create TMs
that are both reliabile and have high coverage.
4.2 Partial Matching of Parse Subtrees
To prevent the problem of the reduction of coverage in the exact matching method, we also proposea partial matching method that keeps coverage justlike standard triangulation by allowing connectionof incompletely equivalent pivot subtrees. To estimate translation probabilities in partial matching,
we first define weighted triangulation generalizing the equations (11)-(14) of standard triangulation with weight function ψ(·):( ) ∑∑ ()ϕ t|s =
ϕ t|pˆT ψ (pˆT |pˆS ) ϕ (pˆS |s) ,(17)pˆT pˆS()( ) ∑∑
ϕ (s|pˆS ) ψ (pˆS |pˆT ) ϕ pˆT |t ,ϕ s|t =

(18)pˆS pˆT( ) ∑∑()
ϕlex t|s =ϕlex t|pˆT ψ (pˆT |pˆS ) ϕlex (pˆS |s) , (19)pˆT pˆS()( ) ∑∑
ϕlex (s|pˆS ) ψ (pˆS |pˆT ) ϕlex pˆT |tϕlex s|t =(20)pˆS pˆT

where pˆS ∈ TSP and pˆT ∈ PP T are pivot
parse subtrees of source-pivot and pivot-target
synchronous rules respectively. By adjusting ψ(·),
we can control the magnitude of the penalty for thecase of incompletely matched connections. If wedefine ψ(pˆT |pˆS ) = 1 when pˆT is equal to pˆS andψ(pˆT |pˆS ) = 0 otherwise, equations (17)-(20) areequivalent with equations (11)-(14).
Better estimating ψ(·) is not trivial, and cooccurrence counts of pˆS and pˆT are not available. Therefore we introduce a heuristic estimation
method as follows:ψ(pˆT |pˆS ) = ∑ψ(pˆS |pˆT ) = ∑w(pˆS , pˆT )p̂∈TP T w(pˆS , p̂)· max w(pˆS , p̂) (21)p̂∈TP Tw(pˆS , pˆT )· max w(p̂, pˆT ) (22)p̂∈TSPp̂∈TSP w(p̂, pˆT )0(f lat(pˆS ) ̸= f lat(pˆT ))w(pˆS , pˆT ) =exp (−d (pˆS , pˆT )) (otherwise)d(pˆS , pˆT ) = T reeEditDistance(pˆS , pˆT )

(23)
(24)

where
f lat(p̂)
returnsthesymbolstring of p̂ keeping non-terminals, andT reeEditDistance(pˆS , pˆT ) is minimum costof a sequence of operations (contract an edge,
uncontract an edge, modify the label of an edge)
needed to transform pˆS into pˆT (Klein, 1998).
According to equations (21)-(24), we can assure that incomplete match of pivot subtrees leadsd(·) ≥ 1 and penalizes such that ψ(·) ≤ 1/ed ≤
1/e, while exact match of subtrees leads to a valueof ψ(·) at least e ≈ 2.718 times larger than whenusing partially matched subtrees.

5

Experiments

5.1

Experimental Set-Up

To investigate the effect of our proposed approach,
we evaluate the translation accuracy through pivot
translation experiments on the United Nations Parallel Corpus (UN6Way) (Ziemski et al., 2016).
UN6Way is a line-aligned multilingual parallel
corpus that includes data in English (En), Arabic(Ar), Spanish (Es), French (Fr), Russian (Ru) andChinese (Zh), covering different families of languages. It contains more than 11M sentences foreach language pair, and is therefore suitable formultilingual translation tasks such as pivot translation. In these experiments, we fixed English as thepivot language considering that it is the language
most frequently used as a pivot language. This has
the positive side-effect that accurate phrase structure parsers are available in the pivot language,
which is good for our proposed method. We perform pivot translation on all the combinations of
the other 5 languages, and compared the accuracyof each method. For tokenization, we adopt SentencePiece,3 an unsupervised text tokenizer anddetokenizer, that is although designed mainly forneural MT, we confirmed that it also helps to reduce training time and even improves translation
accuracy in our Hiero model as well. We firsttrained a single shared tokenization model by feeding a total of 10M sentences from the data of all
the 6 languages, set the maximum shared vocabulary size to be 16k, and tokenized all available text
with the trained model. We used English raw text
without tokenization for phrase structure analysis
and for training Hiero and T2S TMs on the pivot
side. To generate parse trees, we used the Ckylark PCFG-LA parser (Oda et al., 2015), and filtered out lines of length over 60 tokens from all
the parallel data to ensure accuracy of parsing andalignment. About 7.6M lines remained. Since Hiero requires a large amount of computational resources for training and decoding, so we decided
not to use all available training data but first 1M
lines for training each TM. As a decoder, we use
Travatar (Neubig, 2013), and train Hiero and T2S
TMs with its rule extraction code. We train 5-gramLMs over the target side of the same parallel data
used for training TMs using KenLM (Heafield,2011). For testing and parameter tuning, we used
the first 1,000 lines of the 4,000 lines test and dev
sets respectively. For the evaluation of translation results, we first detokenize with the SentencePiece model and re-tokenized with the tokenizerof the Moses toolkit (Koehn et al., 2007) for Arabic, Spanish, French and Russian and re-tokenized
Chinese text with Kytea tokenizer (Neubig et al.,2011), then evaluated using case-sensitive BLEU4 (Papineni et al., 2002).
We evaluate 6 translation methods:
Direct:Translating with a Hiero TM directly trainedon the source-target parallel corpus without
using pivot language (as an oracle).
Tri. Hiero:
Triangulating source-pivot and pivot-target
Hiero TMs into a source-target Hiero TM using the traditional method (baseline, §3).
Tri. TreeExact
Triangulating pivot-source and pivot-target
T2S TMs into a source-target Hiero TM using3

https://github.com/google/sentencepieceSource

Ar

Es

Fr

Ru

Zh

Target
Es
Fr
Ru
Zh
Ar
Fr
Ru
Zh
Ar
Es
Ru
Zh
Ar
Es
Fr
Zh
Ar
Es
Fr
Ru

Direct
38.49
33.34
24.63
27.27
27.18
43.24
28.83
27.08
25.10
45.20
27.42
25.84
22.53
37.60
34.05
28.03
20.09
30.66
25.97
21.16

BLEU Score [%]Tri. Hiero Tri. TreeExact(baseline)(proposed 1)
34.20‡ 34.97
29.93‡ 30.68
22.94‡ 23.9422.78‡ 25.1722.97‡ 24.0938.74‡ 39.6226.35‡ 27.25
24.54
25.00
21.65
21.40
40.16
‡ 41.0324.71† 25.24
23.16
23.56
19.82
19.86
34.56
34.96
30.75
† 31.43
24.88
25.07
16.66
17.01
27.84
27.99
23.82
24.34
18.63
‡ 19.58

Tri. TreePartial(proposed 2)‡ 35.94‡ 30.83‡ 24.15‡ 25.07‡ 24.45‡ 40.12‡ 27.41† 25.16† 22.13‡ 41.99‡ 25.64
23.53
20.35‡ 35.62‡ 31.6725.12‡ 17.7328.05† 24.35‡ 19.59

Table 1: Comparison of each triangulation methods. Bold face indicates the highest BLEU score in pivot
translation, and daggers indicate statistically significant gains over Tri. Hiero († : p < 0.05, ‡ : p < 0.01).
the proposed exact matching of pivot subtrees(proposed 1, §4.1).
Tri. TreePartial
Triangulating pivot-source and pivot-target
T2S TMs into a source-target Hiero TM using
the proposed partial matching of pivot subtrees (proposed 2, §4.2).
5.2

Experimental Results

The result of experiments using all combinationsof pivot translation tasks for 5 languages via English is shown in Table 1. From the results,
we can see that the proposed partial matching
method of pivot subtrees in triangulation outperforms the standard triangulation method for all language pairs and achieves higher or almost equal
scores than proposed exact matching method. The
exact matching method also outperforms the standard triangulation method in the majority of thelanguage pairs, but has a lesser improvement thanpartial matching method. In Table 2 we show thecomparison of coverage of each proposed triangulated method. From this table, we can see that theexact matching method reduces several percent in
number of unique phrases while the partial matching method keeps the same coverage with surfaceform matching. We can consider that it is one of
the reasons of the difference in improvement stability between the partial and exact matching methods.
We show an example of a translated sentences
for which pivot-side ambiguity is resolved in thethe syntactic matching methods:
Source Sentence in French:
La Suisse encourage tous les États parties
à:::::::::
soutenir:::le::::::::
travail ::::::::::
conceptuel:::::
que::::
fait
actuellement
le Secrétariat .
:::::::::::::::::::::::
Corresponding Sentence in English:
Switzerland encourages all parties to support
the current conceptual work of the secretariat.
Reference in Spanish:Suiza alienta a todos los Estados partes :aque
apoyen la actual labor conceptual de la:::::::::::::::::::::::::::::::::::::::
Secretaría.
:::::::::

Source

Ar

Es

Fr

Ru

Zh

Target
Es
Fr
Ru
Zh
Ar
Fr
Ru
Zh
Ar
Es
Ru
Zh
Ar
Es
Fr
Zh
Ar
Es
Fr
Ru

Number of source-side unique phrases/words
Tri. TreeExact
Tri. TreePartial
2.580M / 5,072
2.646M / 5,077
2.589M / 5,067
2.658M / 5,071
2.347M / 5,085
2.406M / 5,088
2.324M / 5,034
2.386M / 5,040
1.942M / 5,182
2.013M / 5,188
2.062M / 5,205
2.129M / 5,210
1,978M / 5,191
2.037M / 5,197
1,920M / 5,175
1.986M / 5,180
2.176M / 5,310
2.233M / 5,316
2.302M / 5,337
2.366M / 5,342
2.203M / 5.311
2.266M / 5,318
2.162M / 5.313
2.215M / 5,321
2.437M / 5,637
2.505M / 5,644
2.478M / 5.677
2.536M / 5,682
2.479M / 5,661
2.531M / 5,665
2.466M / 5,682
2.515M / 5,688
1.480M / 9,428
1.556M / 9,474
1.504M / 9,523
1.570M / 9,555
1.499M / 9,490
1,568M / 9,520
1.518M / 9,457
1.593M / 9,487

Table 2: Comparison of rule table coverage in proposed triangulation methods.

Direct:Suiza alienta a todos los Estados partes a que
apoyen el trabajo conceptual que se examinan
en la Secretaría . (BLEU+1: 55.99)
Tri. Hiero:Suiza conceptuales
para apoyar la labor que:::::::::::::::::::::::::::::::::
enestos
momentos
la Secretaría alienta a::::::::::::::::::::::::::::::
todos los Estados Partes . (BLEU+1: 29.74)
Tri. TreeExact:
Suiza alienta a todos los Estados Partes aapoyar la labor conceptual que actualmente la
Secretaría . (BLEU+1: 43.08)
Tri. TreePartial:
Suiza alienta a todos los Estados Partes aapoyar la labor conceptual que actualmente la
Secretaría . (BLEU+1: 43.08)
The results of Tri.TreeExact and Tri.TreePartial
are same in this example.
We find thatthe derivation in Tri.Hiero uses ruleX→
⟨X0 _parties X1 , X1 X0 _Partes⟩4
4

The words emphasized with underline and wavyunderline in the example correspond to X0 and X1 respectively.

causing incorrect re-ordering of phrases
followed by steps of incorrect word selection.5
On the other hand, derivation in
Tri.TreeExact and Tri.TreePartial uses rule X →⟨_tous _les X0 _parties, _todos X0 _Partes⟩6
synthesized from T2S rules with common pivot
subtree (NP (DT all) (NP’ XNNP (NNS parties)).
We can confirm that the derivation improves
word-selection and word-reordering by using thisrule.
5.3 Comparison with Neural MT:
Recent results (Firat et al., 2016; Johnson et al.,2016) have found that neural machine translation
systems can gain the ability to perform translation
with zero parallel resources by training on multiple
sets of bilingual data. However, previous work has
not examined the competitiveness of these methods with pivot-based symbolic SMT frameworks
such as PBMT or Hiero. In this section, we compare a zero-shot NMT model (detailed parametersin Table 3) with our pivot-based Hiero models.
5
For example, the word “conceptuales” with italic face in
Tri.Hiero takes the wrong form and position.
6
The words emphasized in bold face in the example correspond to the rule.

vocabulary size:
source embedding size:
target embedding size:
output embedding size:
encoder hidden size:
decoder hidden size:
LSTM layers:attention type:attention hidden size:
optimizer type:loss integration type:batch size:
max iteration:
dropout rate:
decoder type:

16k (shared)
512
512
512
512
512
1
MLP
512
Adam
mean
2048
200k
0.3
Luong+ 2015

6 Conclusion
In this paper, we have proposed a method of pivot
translation using triangulation with exact or partial matching method of pivot-side parse subtrees.
In experiments, we found that these triangulated
models are effective in particular when allowing
partial matching. To estimate translation probabilities, we introduced heuristic that has no guarantee
to be optimal. Therefore in the future, we plan to
explore more refined estimation methods that utilize machine learning.

Acknowledgments
Part of this work was supported by JSPS
KAKENHI Grant Numbers JP16H05873 andJP17H06101.

Table 3: Main parameters of NMT training

References
Direct NMT is trained with the same data of Direct Hiero, Cascade NMT translates by bridging
source-pivot and pivot-target NMT models, and
Zero-Shot NMT is trained on single shared modelwith pvt ↔ {src,target} parallel data according toJohnson et al. (2016). To train and evaluate NMT
models, we adopt NMTKit.7 From the results
we see the tendency of NMT that directly trainedmodel achieves high translation accuracy even fortranslation between languages of different families, on the other hand, the accuracy is drasticallyreduced in the situation when there is no sourcetarget parallel corpora for training. Cascade is
one immediate method connecting two TMs, andNMT cascade translation shows the medium performance in this experiment. In our setting, whilebilingually trained NMT systems were competitive or outperformed Hiero-based models, zeroshot translation is uniformly weaker. This may
be because we used only 1 LSTM layer for encoder/decoder, or because the amount of parallel corpora or language pairs were not sufficient.
Thus, we can posit that while zero-shot translation
has demonstrated reasonable results in some settings, successful zero-shot translation systems arefar from trivial to build, and pivot-based symbolic
MT systems such as PBMT or Hiero may still be acompetitive alternative.

7

https://github.com/odashi/nmtkit

Alfred V. Aho and Jeffrey D. Ullman. 1969. Syntax
directed translations and the pushdown assembler.
Journal of Computer and System Sciences 3(1):37–
56.
Peter F. Brown, Vincent J.Della Pietra, Stephen
A. Della Pietra, and Robert L. Mercer. 1993. The
Mathematics of Statistical Machine Translation: Parameter Estimation.
Computational Linguistics
19:263–312.
Jean-Cédric Chappelier, Martin Rajman, et al. 1998. A
Generalized CYK Algorithm for Parsing Stochastic
CFG. In Proc. TAPD. Citeseer, volume 98, pages
133–137.
David Chiang. 2007. Hierarchical Phrase-Based Translation. Computational Linguistics 33(2):201–228.
Trevor Cohn and Mirella Lapata. 2007. Machine Translation by Triangulation: Making Effective Use ofMulti-Parallel Corpora. In Proc. ACL. pages 728–
735.
Adrià de Gispert and José B. Mariño. 2006. CatalanEnglish Statistical Machine Translation without Parallel Corpus: Bridging through Spanish. In Proc.
of LREC 5th Workshop on Strategies for developing
machine translation for minority languages. pages
65–68.
Chris Dyer, Aaron Cordova, Alex Mont, and Jimmy
Lin. 2008. Fast, Easy, and Cheap: Construction of Statistical Machine Translation Models with
MapReduce. In Proc. WMT. pages 199–207.
Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan,
Fatos T. Yarman Vural, and Kyunghyun Cho. 2016.
Zero-Resource Translation with Multi-Lingual Neural Machine Translation. In Proc. EMNLP. pages
268–277.

Source

Ar

Es

Fr

Ru

Zh

Target
Es
Fr
Ru
Zh
Ar
Fr
Ru
Zh
Ar
Es
Ru
Zh
Ar
Es
Fr
Zh
Ar
Es
Fr
Ru

Direct
Hiero
38.49
33.34
24.63
27.27
27.18
43.24
28.83
27.08
25.10
45.20
27.42
25.84
22.53
37.60
34.05
28.03
20.09
30.66
25.97
21.16

BLEU Score [%]DirectTri.
NMT TreePartial
38.25
35.94
33.16
30.83
27.00
24.15
30.04
25.07
26.02
24.45
41.83
40.12
30.65
27.41
32.36
25.16
23.28
22.13
44.49
41.99
28.29
25.64
29.10
23.53
23.19
20.35
38.67
35.62
33.26
31.67
31.39
25.12
20.17
17.73
32.69
28.05
27.68
24.35
23.17
19.59

Cascade
NMT
31.62
26.91
21.67
23.70
21.21
31.84
23.60
26.03
18.66
32.93
20.87
23.14
19.71
31.25
27.34
24.25
16.89
26.01
23.35
18.40

Zero-Shot
NMT
8.18
8.57
5.79
5.04
5.22
15.04
7.57
8.62
8.08
14.37
8.77
11.95
3.18
10.42
9.76
9.46
10.38
6.13
7.12
3.21

Table 4: Comparison of SMT and NMT in multilingual translation tasks.

Jonathan Graehl and Kevin Knight. 2004. Training
Tree Transducers. In Proc. NAACL. pages 105–112.
Kenneth Heafield. 2011. KenLM: Faster and Smaller
Language Model Queries. In Proc, WMT. pages
187–197.
Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim
Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda B. Viégas, Martin Wattenberg, Greg
Corrado, Macduff Hughes, and Jeffrey Dean. 2016.
Google’s Multilingual Neural Machine Translation
System: Enabling Zero-Shot Translation. CoRR
abs/1611.04558. http://arxiv.org/abs/1611.04558.
Philip N. Klein. 1998. Computing the Edit-Distance
Between Unrooted Ordered Trees. In Proc. of European Symposium on Algorithms. pages 91–102.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris
Callison-Burch, Marcello Federico, Nicola Bertoldi,
Brooke Cowan, Wade Shen, Christine Moran,
Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra
Constantin, and Evan Herbst. 2007. Moses: Open
Source Toolkit for Statistical Machine Translation.
In Proc. ACL. pages 177–180.
Phillip Koehn, Franz Josef Och, and Daniel Marcu.
2003. Statistical Phrase-Based Translation. In Proc.
NAACL. pages 48–54.

Akiva Miura, Graham Neubig, Sakriani Sakti, Tomoki
Toda, and Satoshi Nakamura. 2015. Improving
Pivot Translation by Remembering the Pivot. In
Proc. ACL. pages 573–577.
Graham Neubig. 2013. Travatar: A Forest-to-String
Machine Translation Engine based on Tree Transducers. In Proc. ACL Demo Track. pages 91–96.
Graham Neubig, Yosuke Nakata, and Shinsuke Mori.
2011. Pointwise Prediction for Robust, Adaptable
Japanese Morphological Analysis. In Proc. ACL.
pages 529–533.
Yusuke Oda, Graham Neubig, Sakriani Sakti, Tomoki
Toda, and Satoshi Nakamura. 2015. Ckylark: A
More Robust PCFG-LA Parser. In Proc. NAACL.
pages 41–45.
Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. BLEU: a Method for Automatic
Evaluation of Machine Translation. In Proc. ACL.
pages 311–318.
Masao Utiyama and Hitoshi Isahara. 2007. A Comparison of Pivot Methods for Phrase-Based Statistical
Machine Translation. In Proc. NAACL. pages 484–
491.
Michał Ziemski, Marcin Junczys-Dowmunt, and Bruno
Pouliquen. 2016. The United Nations Parallel Corpus v1.0. In Proc. LREC. pages 3530–3534.