MWE-sensitive Word Alignment in Factored
Translation Model
Tsuyoshi Okita, Andy Way
Dublin City University, Research Workshop of the Israel
Science Foundation

Table Of Contents

1. EN-JP Corpus
2. MWE-Sensitive Word Aligner on PB-SMT
3. MWE-Sensitive Word Aligner on Factored Translation Model
5. Conclusions and Further Works

2 / 51NTCIR-8 Corpora

I JP-EN Patent Corpus
I
I

Essentially translated from JP into EN (Quality of EN?).
3.2 million sentence pairs.

JP-EN
EN-JP

train set
3,186,284
3,186,284

dev set1,200/2,000
1,200/2,000

test set
1,251
1,119

Table: Parallel corpus size of NTCIR-8

3 / 51NTCIR-8 Corpora
I Unstructured complex sentences

Japanese: この第２のスライドブロック５のスライド移動によって、弾性糸ＳＹが、第１のスライドブロック４の下流側面と前記第２のスライドブロック５の上流側面との間で、確実に把持されるとともに、前記弾性糸ＳＹは、第２のスライドブロック
５の下流側面と下側の固定ブロック６の上流側面とのスライドによって前記カッター刃１０の作用により切断される。
English: due to this slidable movement of the second slide block 5, the elastic yarn sy is reliably held between the downstream side of
the first slide block 4 and the upstream side of the second slide block
5 , and the elastic yarn sy is cut by the operation of the cutter blade
10 due to sliding between the downstream side of the second slide
block 5 and the upstream side of the fixed block 6 at the lower side .
4 / 51NTCIR-8 Corpora

I Translational Omission

Japanese: 従来、スパッタリング用ターゲット（以下、単にター
ゲットと略称する） としてはプレーナ型 （円板状もしくは角板状）のターゲットが広く使用されている。
English: conventional sputtering targets extensively in use are of aplaner type having a circular or square plate-like shape .

5 / 51NTCIR-8 Corpora

I Equations in a sentence

Japanese: 処理７３では、上記の取り込んだ信号Ｖ，θ f ，ｄ／ｄｔ( θ f ） から、目標ヨー角加速度ｄ／ｄｔ( ω T ） を決定する。
English: at a process 73 , target yawing angular acceleration d/dt (.omega. .sub.t) is determined based on the fetched signals v , .theta.f
and d/dt ( .theta.f ) .

6 / 51NTCIR-8 Corpora
Reference number
Japanese: この軸受けユニット（44 ） は、本体支柱（４）に固着した上・下部ブラケット（45 ）（46 ）と、この上・下部ブ
ラケット（45 ）（46 ）に挿通した軸セット（47 ）と、軸セット（47 ）と製品容器（２）間を連結するアーム（48 ）とで構成する。
Japanese: 図１に示すガイド５ とガイドローラ３はこのような案内を行うものであり、以下にその実施例を説明する。
English: the bearing unit 44 comprises upper and lower brackets 45
and 46 fixed on the body pillar 4 , a shaft set 47 inserted in said
upper and lower brackets 45 and 46 , and an arm 48 interconnecting
the shaft set 47 and the product container 2.
English: the guide 5 and the guide rollers 3 shown in fig. 1 aredesigned to provide such guidance , and embodiments thereof will be
described hereinunder .
7 / 51NTCIR-8 Corpora

Many parentheses
Japanese: 次に、直径５ｍｍの多数の空孔（９６ポイント） を有する開閉式の扉１４を開けて、大気中に浮遊している有機ガ
ス１５を基板１０に２４時間吸着させる（ステップＳＡ２） 。
English: then , in step 2 of fig. 4 , the cover 14 is opened for 24 hours
so that gaseous organic substances 15 floating in an atmosphere areadsorbed to the silicon substrate 10 .

8 / 51

Intrinsic Evaluation (JP-EN)
Systems
System combination
HPB-SMT 1
PB-SMT 1
Noise reduction (PB-SMT)
PB-SMT 2+
Preprocessing (PB-SMT)+
HPB-SMT 2Supertag (ENJU) 1
Supertag (ENJU) 2
System combination (unofficial run)

BLEU
27.61∗
26.86∗
26.51∗
24.01
23.91∗
23.82
23.30
20.68
18.27
28.43

#OOV
321
314
194
443
316
194
303
430
426
331

Table: Intrinsic evaluation results (JP-EN). Noted that we trained over
3,200k training corpus for the systems marked with + and over 600k
training corpus for other systems.
9 / 51

Intrinsic Evaluation (EN-JP)
Systems
System combination
HPB-SMT 1
PB-SMT 1
PB-SMT 2+
Noise reduction
Preprocessing (PB-SMT)+
HPB-SMT 2
Context supertag (Base)
Context supertag (Superpair)Context supertag (CCG)
Context supertag (LTAG)
Context supertag (CCG-LTAG)
Context supertag (POS)

BLEU
33.03
32.50
30.53
30.08
29.53
27.93
27.23
26.83
26.45
26.38
26.38
26.22
26.21

Table: Intrinsic evaluation results (EN-JP).
10 / 51

Table Of Contents

1. EN-JP Corpus
2. MWE-Sensitive Word Aligner on PB-SMT
3. MWE-Sensitive Word Aligner on Factored Translation Model
5. Conclusions and Further Works

11 / 51

MWE-sensitive Word Alignment onn PB-SMT

approachIBM
discreference

word alignment
E-stepM-stepothermethods

prior

h-a
corp

feat

MWE-sensitive

12H34[Okita&, 10]

normalN.A.

objectiveno

MWE

GIZA++
Berkeley[Och&, 02][Liang&,06][DeNero&,07][Graca&, 08][Moore, 05][Blunsom&06][Fraser, 07][Marcu&02][Fossum&08][Ma&, 08]
[Lambert&,05]

normalML
jointly trained HMM
syntax-aware HMM
constraint grad asc
N.A.
N.A.
normal
CRF
N.A.
N.A.
N.A.
GIZA++ compatible
GIZA++ compatible
GIZA++ compatible

N.A.
agreement

PostCAT
discrimative aln
CRF
semi-superjoint phrase
delete links
add links
BMWE

12H3456
12H
12H
12H
1d
1d
1d
B
1H34
1H34d
1H34

SVM
N.A.

none
none
none
stochastic
N.A.
noninform
N.A.
none
none
none
none

no
y-n
y-n
no
yes
yes
yes
no
no
no
no

no
no
syn
no
gen
gen
gen
no
syn
syn
no

word lattice[1H34][Dyer&, 08]

GIZA++ compatible

lattice

none

no

no

MAP

agreement
bipartite matchnone
EMD
phrase alignTable: Word / phrase alignment.

12 / 51

N-to-m Mapping Object Problem (1)I (Problem in PB-SMT context) N-to-m mapping objects, such asparaphrases, non-literal translations, and multi-word expressions,
may appear as both noise and as valid training data for wordalignment.

C’ est la vie .

That is life .

C’ est la vie .
It is the life .

C’ est la vie .

C’ est la vie .

This is just the way life is .

Love story .
13 / 51

N-to-m Mapping Object Problem (2)Source Language
to my regret i cannot go today .
i am sorry that i cannot visit today .
it is a pity that i cannot go today .
sorry , today i will not be available

Target Language
i am sorry that i cannot visit today .
it is a pity that i cannot go today .
sorry , today i will not be available
to my regret i cannot go today .

GIZA++ alignment results for IBM Model 4
available pity 1
i NULL 0.667
cannot sorry 0.55
cannot available 0.272
go sorry 0.667
it am 1
am to 1is am 1sorry to 0.33
sorry go 0.667
to , 1, go 1my , 1that regret 0.25
will is 1
cannot regret 0.18
not is 1
visit regret 1
a that 1
regret not 1
pity that 1
be pity 1

today . 1..1
i cannot 0.33
that cannot 0.75

Figure: Paraphrase example: a training corpus consists of four sentencepairs. Results show that only the matching between the colon is correct.
14 / 51

N-to-m Mapping Objects Problem (3)I How phrase extraction works?
C’ est la vie .

C’ est la vie .

Love story .

Love story .
C’ est la vie .
love
story
.

C’ est la vie .
love
story
.

n−to−1 mapping
Symmetrization

C’ est la vie .
love
story
.

1−to−n mapping

Resulted phrase pairs:(love story | C’ est la vie)
(story | vie)
(love story . | C’ est la vie .)(.|.)
15 / 51

N-to-m Mapping Object Problem (4)I Need to stabilize the temper of word alignment by ’grouping’.

C’ est la vie .
loveThis is not stable depending on theword alignment results.

story.
Stabilization
Resulted phrase pairs:(love story | C’ est la vie)
(story | vie)
(love story . | C’ est la vie .)(.|.)

C’ est la vie .

love story.
Resulted phrase pairs:(love story | C’ est la vie)
(love story . | C’ est la vie .)(.|.)
16 / 51

MWE-sensitive Word Alignment (1)Definition (Anchor Word Alignment Problem)
Let (ĕ, f˘) = {(ĕ1 , f˘1 ), . . . , (ĕn , f˘n )} be a parallel corpus. By priorknowledge we additionally have knowledge of anchor words(ê, fˆ) = {(senti , te1 , tf1 , pose1 , posf1 , lengthe , lengthf ), . . .,(sentk , ten , tfn , posen , posfn , lengthe , lengthf )} where senti denotes
sentence ID, posei denotes the position of tei in a sentence ĕi ,and lengthe (and lengthf ) denotes the sentence length of theoriginal sentence which includes ei . Under a given (ĕ, f˘) and(ê, fˆ), our objective is to obtain word alignments. It is notedthat an anchor word may include a phrase pair which forms
n-to-m mapping objects.

17 / 51

MWE-sensitive Word Alignment (2)Statistical MWE extraction method
97|||groupe socialiste|||socialist group|||26|||26
101|||monsieur poettering|||mr poettering|||1|||4
103|||monsieur poettering|||mr poettering|||1|||11
110|||monsieur poettering|||mr poettering|||1|||9117|||explication de vote|||explanation of vote|||28|||26
Heuristic-based MWE extraction method
28|||the wheel 2|||車輪 ２||| 25||| 5
28|||the primary-side fixed armature 13|||１ 次 側 固定 電機 子 １ ３||| 13||| 928|||the secondary-side rotary magnet 7|||２ 次 側 回転 マグネット ７||| 15||| 11
Table: Example of MWE pairs in Europarl corpus (FR-EN) and NTCIR
patent corpus (JP-EN). There are 5 columns for each term: sentence
number, source term, target term, source position, and target position.

18 / 51

MWE-sensitive Word Alignment (3)I We are given two sentence pairs {( that is life . , c‘ est la vie . ),( rosy life, la vie en rose )} and anchor words {(1, life, vie, 3, 4),(2, life, vie, 2, 2)}.
GIZA++ (no prior)
NULL c’ est la vie .

that is life .

NULL la vie en rose

rosy

life

Ours (with prior)
NULL c’ est la vie .

that is life .

NULL la vie en roserosy
19 / 51lifeMWE-sensitive Word Alignment (4)
pair
EN-FR
is NULL
rosy enthat .
life la. c‘that c‘is estlife vie
rosy rose

GIZA++(no prior)
fin ini prior

Ours(with prior)
fin ini prior

1
1
1
1
1
0
0
0
0

0
0
0
0
0
1
1
1
1

.25
.5
.25
.25
.25
.25
.25
.5
.25

0
0
0
0
0
0
0
0
0

.25
.5
.25
.25
.25
.25
.25
.5
.25

.25
.2
.25
0
.25
.25
.25
1
.2

Table: The benefit of prior knowledge of anchor words.

20 / 51

2-step Learning Procedure
I Statistical models often aim to learn from frequent examples only,

and often overlook less frequent but linguistically importantphenomena.
I 2-step learning procedure can capture both frequent andlinguistically important phenomena via (Bayesian) priors. Let
linguistically important phenomenon be learnable in terms of V andnot learnable in terms of W , while let other phenomenon belearnable in terms of W .
I
I

1: Learn linguistically important phenomenon in terms of V .
2: Learn the main phenomenon in terms of W with a priorlearnt in Step 1.

21 / 51

2-step Learning Procedure
I Prior Knowledge 1:

Word alignment with prior knowledge aboutbilingual terminology (IBM Model 4, hidden variable is alignment
function)
1: bilingual terminology: Learn linguistically importantphenomenon in terms of word association.
I 2: Learn word alignment P(e|f ) in terms of word pairfrequencies (with alignment function as hidden variable) withthe prior about an alignment function supplied by bilingual
terminology.
(ei = ti , fj = tj )

 1

0
(ei = ti , fj 6= tj )align(ei |fi , T ) =0(ei 6= ti , fj = tj )
uniform (ei 6= ti , fj 6= tj )I

22 / 51

2-step Learning Procedure
Word alignment implemented by EM algorithm: to replace theM-step with Maximum A Posteriori estimate (p(t) = align(e|f )denotes a prior):
EEXH :

q(a; e, f ) =p(a|e, f ; t)

MMLE :

t 0 = arg max Q(t, t old )tX= arg maxq(a|e, f ) log p(e, f , a; t)t

MMAP :x,zt 0 = arg max Q(t, t old ) + log p(t)t23 / 51

OOV words
I Prior Knowledge 3: Word alignment with prior knowledge about

OOV words.
1: Learn OOV word pairs by first detecting OOV words insource side after decoding parallel corpus, then by translatingthem by hands (transliteration, proper nouns, localization,
symbols, equations and algorithms).
I 2: Learn word alignment P(e|f ) in terms of the word pairfrequencies (with alignment function as hidden variable) withthe prior about an alignment function supplied by OOV wordpairs..
1(ei = ti , fj = tj )0(ei = ti , fj 6= tj )align(ei |fi , T ) =0(ei 6= ti , fj = tj )
uniform (ei 6= ti , fj 6= tj )I

24 / 51

Smoothing Methods for Small Corpus
approach
HPYLM
modified Kneser-Ney
Good-Turing
Kneser-Ney
add-oneinterporated Kneser-Ney
Bayesian smoothing
HPYTM
Good-Turing
Kneser-Ney
others

LM
LM
LM
LM
LM
LM
LM
TM
TM
TM
TM

reference[Goldwater&, 06][Goodman&, 98]

MT context[Okita&,10][Kneser&,06][Peto&,05][Okita&,??][Foster&,06][Foster&,06][Foster&,06]

Table: Smoothing Method for Language Model and Translation Model

25 / 51

Pitman-Yor Process Prior (1)I Generative Model
G |d0 , θ0 , G0 ∼ PY (d0 , θ0 , G0 )


 ∅
...
G

u |d|u| , θ|u| , Gpi(u) ∼ PY (d|u| , θ|u| , Gpi(u) )Xi |Gu ∼ DISCRETE (Gu ),(i = 1, ..., n)
G_0theta_0

G_o

d_0

theta_u

d_u

G_u

X_ui
i=1,...,n_un26 / 51

Pitman-Yor Process Prior (2)I Pitman-Yor process as a prior:Gu ∼ PY (d|u| , θ|u| , Gpi(u) )I(1)

Pitman-Yor process PY (d, θ, G0 ) is distribution over somebase distribution G0 [Pitman, 95], which has two parameters togenerate a power-law distribution,II

d: a discount parameter, andθ: a strength parameter.

u: a given context.
Gu (w ): probability of the current word taking value w .
I pi(u): a function whose parameter is a context u,
I the discount and strength parameters are functions of thelength |u| of the context,II

27 / 51

Pitman-Yor Process Prior (3)I Discount and strength parameters
I First customer sits at the first available table.
I

ith subsequent customer sits at a table drawn from thefollowing distribution (ck : number of customers seated at tablek until now, θ: this controls the similarity between G and G0 ,and t: total number of tables until now):
P(previously occupied table k|Fi−1 )∝ck − dP(the next unoccupied table|Fi−1 )∝θ + dtI Chinese restaurant (Customers enter and seat themselves):
I
I

Number of tables are infinite andEach table has infinite seating capacity.

28 / 51

Pitman-Yor Process Prior (4)I Let pi(u) be the suffix of u consisting of all but the earliest wordII

u is n-gram words
pi(u) is (n-1)-gram words

I Predictive distribution of n-gram probability in HPYLM
I

Recursively calculated as in (2):p(w |h) =Iθ + d · th·c(w |h) − d · thw+
p(w |h0 )θ + c(h)θ + c(h)(2)

p(w |h0 ) is the same probability using a (n-1)-gram context h0 .

29 / 51

Pitman-Yor Process Prior (5)I Incorporation of zero probabilities in decoder.

When we encounter unseen n-gram word in a test sentence,
PB-SMT uses constant zero probabilities for unseen n-gram
word,
I HPYLM should look up different zero probabilities based oncontext, e.g. (n-1)-gram. −→ Just before we do decoding, weupdate LM by supplying zero-probabilities for unseen n-gramword.
I

30 / 51

Pitman-Yor Translation Modelmichael geht davon aus, dass er im haus bleibt

house

dass erdass

erbleibtmichael
geht
davon
aus

,

im haus bleibt
im hausimgeht davon aus

dass
er
im
haus

in the housein theassumesdass er im haus bleibt

mchaelassumes
that
he
will
stay
in
the
house

michaelwill stay in the housewill stayhethat hemichael geht davon aus

that

that he will stay in the housemichael geht davon aus , dassmichaelmichael assumes

michael assumes thatmichael assumes that hemichael assumes that he will stay in the housemichael geht davon aus , dass er

31 / 51

haus

bleibtPitman-Yor Translation Model
Lattice structure (Tree structure if we only accept real lines)
michael assumes that he will stay in the house|| michael geht davon aus, dass er im haus bleibt

michael assumes that he || michael geht davon aus , dass erthat he will stay in the house ||michael assumes that || michael geht davon aus , dasswill stay in the house ||dass er im haus bleibt

im haus bleibtthat he || dass er
michael assumes || michael geht davon aus
2−phrase−grammichael || michaelassumes || geht davon austhat || dass1−phrase−gramin the house || im haushe || erin the || imhouse || haus

will stay || bleibt

Figure: Figure shows a lattice structure of translation model for a toy
example.

32 / 51

Statistics
JP–ENprior knowledge 1prior knowledge 2prior knowledge 3

MWEs
paraphrases
transliteration
proper nouns
localisation
equationssymbols
noise
n

num
120070432135
25928
3408
207
103
13842
19007

Table: Statistics of prior knowledge.

33 / 51

Experimental Results

JP–EN
baseline
MWEs
paraphrases
OOVs
allheuristics

without TM smoothing
21.68
22.48
22.23
22.26
22.93
21.90

with TM smoothing
22.44
22.78
22.44
22.52
23.02
22.49

Table: Results for 200k JP–EN sentences. Heuristics in the last row
shows the result when prior knowledge 1 was added at the bottom of thetranslation model.

34 / 51

Table Of Contents

1. EN-JP Corpus
2. MWE-Sensitive Word Aligner on PB-SMT
3. MWE-Sensitive Word Aligner on Factored Translation Model
5. Conclusions and Further Works

35 / 51

Factored Translation Model

I Factored translation model (Koehn and Huang, 2007; Avramidis

and Koehn, 2008, Koehn, 2010)
I

factor: additional linguistic markup at the word levelImorphological features including case, number, gender,
person, tense, and aspect.

36 / 51

English Factors
Parsing results of “John has come” by Enju [Miyao and Tsujii, 2002]<sentence id=”s0” parse-status=”success” fom=”-0.0221762”><cons id=”c0” cat=”S” xcat=”” head=”c3” sem-head=”c3” schema=”subj-head”><cons id=”c1” cat=”NP” xcat=”” head=”c2” sem-head=”c2” schema=”empty-spec-head”><cons id=”c2” cat=”NX” xcat=”” head=”t0” sem-head=”t0” ><tok id=”t0” cat=”N” pos=”NN” base=”john” lexentry=”[D&lt;N.3sg&gt;]-lxm” pred=”noun-arg0”>
John</tok></cons></cons><cons id=”c3” cat=”VP” xcat=”” head=”c4” sem-head=”c5” schema=”head-comp”><cons id=”c4” cat=”VX” xcat=”” head=”t1” sem-head=”t1”><tok id=”t1” cat=”V” pos=”VBZ” base=”have” lexentry=”[NP&lt;V.have.bse&gt;VP.pap]-sctl-lxmsingular3rd-verb-rule” pred=”aux-arg12” aux=”have” arg1=”c1” arg2=”c5”>has</tok><cons id=”c5” cat=”VP” xcat=”” head=”t2” sem-head=”t2”><tok id=”t2” cat=”V” pos=”VBN” base=”come” lexentry=”[NP.nom&lt;VP.bse&gt;]-lxmperfect-verb-rule” pred=”verb-arg1” tense=”present” aspect=”perfect” voice=”active” aux=”minus” arg1=”c1”>come</tok></cons></cons></cons></sentence>

37 / 51

Japanese Factors

Parsing results of “ジョンが来た”(John has come) by Cabocha [Kudo and Matsumoto, 2003]
0 1D 01 0.00000000
ジョン
ジョン
ジョン
が
ガ
が
* 1 -1O 0/1 0.00000000
来
キ
来る
た
タ
た
EOS

名詞-固有名詞-人名-名
助詞-格助詞-一般
動詞-自立
助動詞

B-PERSON
O
カ変・来ル
特殊・タ

38 / 51

連用形
基本形

O
O

Word Alignment and Phrase Extraction
Word alignment in two directions
null
null
John
has
come

ジョン

が

来

⇒
⇓⇒
⇓
⇓⇒

↓
Phrase extraction
John
|
ジョン

has
が

come
|
来

た

39 / 51

たFactored Translation Model
Translation process (Lemma)Translation process (POS)

た

⇓⇒
⇓
⇓⇒

名詞
noun
aux
verb

来る

が

ジョン
John
have
come

助詞

動詞

助動詞

⇓⇒
⇓
⇓⇒

Translation process (Morphology)
無変化
singularsingular3rd
present perfect

連用形

無変化

基本形

⇓⇒

Generation process

⇓
⇓⇒
ジョン

ジョン,名詞,–が,助詞,無変化
来る,動詞,連用形
た,助動詞,基本形

が

来

た

%
%
%
%

40 / 51

Factored Translation Model
Problematic cases (n-to-m mapping problems including paraphrases, MWEs, non-literal translations)
c’
est
la
vie
that
is
just
the
way
life
is

⇓⇒?

⇓⇒?
⇓⇒?

c’
love
story

is

la

vie
⇓⇒?

⇓⇒?

C’ est la vie .

That

est

life .

C’ est la vie .
It isthe life .

C’ est la vie .

C’ est la vie .

This is just the way life is .

Love story .

41 / 51

Strong Assumptions
1. Correct word correspondence. (Precision ≤ 90%)
2. Decision to separate morpheme(s) from a word. (e.g. Separate‘looks’ into ‘look’ and ‘s’)
Separation of word(s) and morpheme(s): better BLEU score,(adequacy decreases).
I Combination of word(s) with morpheme(s)I

3. Sufficient morphological information for (monolingual) language.
Morphological info (most of the verbs in European language
inflect based on person and number, Japanese verbs inflect
based on aspect).
I Some missing morphological info (No article and gender fornoun phrases in Japanese.)I

42 / 51

Japanese

I Inflection in verbs / adjectives / adverbs
I

Conjugation in six stem forms (imperfective / continuative /terminal / attributive / hypothetical / imperative form) basedon aspect.

I JP noun phrases are accompanied with case particles.
I (Relatively) free word order
I SOV language

43 / 51

Algorithm to Change Assumptions
Overall Algorithm
1. (Morphological predesign)
2. Do segmentation of JP sentence into morphemes by amorphological analyzer.
3. Combine verb and morphemes with attaching case information. By
this construction, we aim at not losing the information bymorphemes).
4. Do word alignment by MWE-sensitive word aligner (Okita et al.,2010) instead of GIZA++.
I

Plugged in prior knowledge about bilingual terminology(Bilingual terminology extraction algorithm)

44 / 51

Experimental Settings (1)I Baseline: Log-linear PB-SMT

GIZA++ [Och and Ney, 2003] of IBM Model 4.
Phrase extraction: grow-diag-final heuristics.
I SRILM [Stolcke, 2002]: 5-gram language model.
I MERT [Och, 2003].
I Moses decoder [Koehn et al., 2007].
I
I

I Corpus
I

NTCIR-8 corpus [Fujii, 2010]. EN–JP. Training corpus 200k
(randomly extracted); development set 1,200 sent; test set
1,119 (EN–JP) / 1,251 (JP–EN).

45 / 51

Experimental Results

I Baseline by the plain factored model: 21.67 BLEU point absolute.
I Separation of a word with morpheme(s) : 18.35 BLEU point

absolute.
I MWE-sensitive word aligner: 22.23 BLEU point absolute.

46 / 51

Conclusions and Further Works

I MWE-sensitive word aligner worked for factored translation model

as well.
I Need more translation pairs and bigger size.

(MWE-sensitive word aligner) Searched alignment spaceshould be enlarged.
I Need some strategy to handle free word order.
I

47 / 51

Acknowledgement
Thank you for your attention.
I This research is supported by the Science Foundation Ireland(Grant 07/CE/I1142) as part of the Centre for Next
Generation Localisation at Dublin City University.
I Irish Centre for High-End Computing.
I Machine Translation Marathon.

48 / 51

MWE-sensitive Word Alignment (7)Algorithm 1 MWE Extraction Algorithm (Step 1 in overall alg)
Given: a parallel corpus and a set of anchor word alignment links:
1. We use a POS tagger to tag a sentence on the SL side.
2. Based on the typical POS patterns for the SL, extract nounphrases on the SL side.
3. Count n-gram statistics (typically n = 1, · · · , 5) on the TL sidewhich jointly occur with each source noun phrase extracted in
Step 2.
4. Obtain the maximum likelihood counts of joint phrases, i.e.
noun phrases on the SL side and n-gram phrases on the TL side.
5. Repeat Steps 1 to 4 reversing SL and TL.
6. Intersect (or union) the results in both directions.

49 / 51

Statistics
observed
1 form2 forms
3 forms
4 forms
5 forms
6 forms

#
911
445
506
270
111
33

%
40%
20%
22%
12%
5%
1%

type
NP
VP
ph (symbols)ph (prefixes)ph (OOVs)ph (conjunctions)ph (attributives)
Adverbial phrases#
1831012259432
68298
66729
66461
65159
59633
33781

Table: Statistics of observed verb forms (left) and number of phrase
types(right) in JP side. In right figures, the inside of parenthesis meansthat the top of the phrase starts with symbols, and so forth.

50 / 51

Statistics

Figure: Statistics of number of nouns in NP.
51 / 51