Given Bilingual Terminology in Statistical Machine Translation:
MWE-sensitve Word Alignment and Hierarchical Pitman-Yor Process-based
Translation Model Smoothing
Tsuyoshi Okita and Andy Way
Dublin City University
Glasnevin, Dublin 9, Ireland

Abstract
This paper considers a scenario when we are given almost perfect knowledge about bilingual terminology interms of a test corpus in Statistical Machine Translation (SMT). When the given terminology is part of a
training corpus, one natural strategy in SMT is to use
the trained translation model ignoring the given terminology. Then, two questions arises here. 1) Can aword aligner capture the given terminology? This issince even if the terminology is in a training corpus,
it is often the case that a resulted translation model
may not include these terminology. 2) Are probabilities in a translation model correctly calculated? In order to answer these questions, we did experiment introducing a Multi-Word Expression-sensitive (MWEsensitive) word aligner and a hierarchical Pitman-Yor
process-based translation model smoothing. Using
200k JP–EN NTCIR corpus, our experimental results
show that if we introduce an MWE-sensitive wordaligner and a new translation model smoothing, theoverall improvement was 1.35 BLEU point absolute and6.0% relative compared to the case we do not introduce
these two.

Introduction
This paper considers a scenario in Statistical Machine Translation (SMT) (Brown et al. 1993; Marcu and Wong 2002;
Chiang 2005; Koehn 2010) when we are given almost perfect knowledge about bilingual terminology in terms of atest corpus. The first motivation is that it is practically likelythat most of the bilingual terminology are already givena priori before we translate a patent. For example, whena Japanese patent is applied to the Japanese Patent Office(JPO), a JP–EN translator has to pick up the English terminology which is often already provided by the bilingual terminology database maintained by the JPO unless it is a newterm. Another example is a translation of scientific books.
The candidate bilingual terminology is often chosen by atranslator before he / she starts translating a book since thishelps to use the same terminology consistently.
When the given terminology is not part of a training corpus, a conventional strategy is to substitute the given terminology in the pre-processing stage and to give the substituted
Copyright c 2011, Association for the Advancement of Artificial
Intelligence (www.aaai.org). All rights reserved.

sentences to a decoder with a small trick not to translate thissubstituted parts further. When the given terminology is partof a training corpus, which is our exact interest, we need tochoose either the strategy which is identical with above ignoring trained translation model or the strategy that we use
the trained translation model ignoring the given terminology.
Now, we would like to pose two questions: 1) Can a wordaligner capture the given terminology? and 2) Are probabilities in a translation model correctly calculated?
The first question is valid since even if the terminology isin a training corpus, a resulted translation model may not include these terminology depending on the case. This is since
the conventional word aligner (Och and Ney 2003) first obtains the 1-to-n mappings in bidirectional ways separately,
and then obtains n-to-m mapping object by symmetrizingthese bidirectional word alignments using the phrase extraction heuristics (Koehn, Och, and Marcu 2003). By thecomputational costs required for the principled approaches
which need to handle the co-occurrence of words in one sideof the languages (Sumita 2000; Marcu and Wong 2002), we
prefer to solve the n-to-m mapping object problem in wordalignment. Among few approaches in this direction, one approach in this line is an MWE-sensitive word aligner which
gives information about alignment links as prior knowledge(Okita et al. 2010). It is expected that the deploymentof this MWE-sensitive word aligner, in the context of thegiven bilingual terminology, will result in the recognition
with higher precision of the terminology in a translation
model. Furthermore, although Okita et al. (2010) only discusses MWEs, this word aligner has potential to incorporate larger category of frequent and less frequent linguistic knowledge such as paraphrases and Out-Of-Vocabulary
words (OOV words). It is also expected that those linguistic knowledge may further help the identification of
the correct terminology in a translation model (Okita 2009;
Okita, Graham, and Way 2010).
The second question is related to the outcomes of wordalignment, together with the following phrase extraction, as
a form of probabilities in a translation model. It is a wellknown fact that the relative frequency (or maximum likelihood estimate) is not an accurate statistics for less frequentphrase pairs since it does not consider zero frequencies. Foster et al. applied various smoothing technique to translation
model (Foster, Kuhn, and Johnson 2006). It is likely that ourconsideration of three kinds of prior knowledge about bilingual terminology will emphasize this phenomenon further
since the resulted phrase pairs become less frequent.
This paper is organized as follows. Section 2 reviews Statistical Machine Translation. Section 3 introduces an MWEsensitive word aligner and three kinds of prior knowledge.
Section 4 mentions the smoothing technique applied to atranslation model based on the hierarchical Pitman-Yor process. Experimental results are presented in Section 5. Section 6 concludes and provides avenues for further research.

Review of Statistical Machine Translation
Let e be an English word, f be a foreign word, ē be an English phrase, ê be an English sentence, P (e|f ) be a lexicaltranslation probability for word e over word f , P (ē|f¯) be atranslation model for phrase ē over f¯, PLM (e) be a language
model for e, and a be an alignment function.
Statistical Machine Translation consists of two steps. In
the first step, for a given sentence aligned parallel corpus,
we obtain three components: a translation model P (f¯|ē),a reordering model d(starti − endi − 1), and a language
model PLM (e). In the second step, for a given test sentence,
we obtains the best translation using these three components
via the noisy-channel model as in (1):¯êBEST = arg max P (fˆ|ê)PLM (e)

(1)

e∈E

=

arg max{ΠIi=1 P (f¯|ē)d(starti − endi − 1)}PLM (e).
e∈E

The widely accepted procedure to obtain a translation model
consists of two steps: a word alignment step (Brown etal. 1993; Och and Ney 2003) and a phrase extraction step
(Koehn, Och, and Marcu 2003). Brown et al. (1993) introduced a generative model which uses an alignment function
as latent variables, and an inference procedure based on an
EM algorithm as in (2):EEXH :

q(z; x) =p(z|x; t)

MMLE : t′ = arg max Q(t, told ) =tX
arg maxq(z|x) log p(x, z; t)
t

(2)

x,z

where t denotes a lexical translation probability t(e|f ) (t isa parameter), and z denotes a latent variable; note that often
t is omitted in word alignment literature but for our purposesin the next section this needs to be explicit. The generativemodels are called IBM Models 1 to 5 whose assumptions
on alignment function and independence assumptions differ.
Further details can be obtained by (Brown et al. 1993; Och
and Ney 2003; Koehn 2010).

Algorithm 1 Prior Knowledge about Paraphrases without
Pivot (Callison-Burch, Koehn, and Osborne 2006)
Given:
Results of word alignment S={(f¯1 , ē1 , a1 ), . . . , (f¯k , ēk , ak )} where k indicates thesize of possible alignment pairs. A set of target side
paraphrases P = {(ē1 |ē2 ), . . . , (ēr−1 |ēr )}. Note that afeature function h(e, f ) denotes an entry in the translation
model (The procedure for the source side paraphrase canbe obtained reversing ‘target‘ and ‘source’.)
Step 1: Augment the baseline translation model with theentry p(f¯2 |f¯1 ) If phrase table entry (ē, f¯1 )¯h(ē, f1 ) =is generated from (ē, f¯2 ) 1(otherwise)

An MWE-sensitive Word Aligner
The word aligner which can incorporate prior knowledge iscalled an MWE-sensitive word aligner (Okita et al. 2010).
This method replaces the maximum likelihood estimate(shown in Equation (2)) with the MAP (Maximum A Posteriori) estimate (shown in Equation (3)).
MMAP :

t′ = arg max Q(t, told ) + log p(t) =tXq(z|x) log p(x, z; t)arg maxtx,z+ log p(t)(3)

Then, the prior log p(t), a probability used to reflect the degree of prior belief about the occurrences of the events, canembed prior knowledge about MWEs.
Let us give information about alignment link between eand f by T = {(sentID, ti , tj , posi , posj ), . . . , }. We use
this information to calculate the prior p(t) = p(t; e, f, T ) for
the given word e and f : this is 1 if e and f have alignment
link, 0 if they are not connected, and uniform if their link is
not known. This is shown in (4):(ei = ti , fj = tj ) 10
(ei = ti , fj 6= tj )p(t; ei , fi , T ) =0(e 6= t , f = t ) (4) uniform (ei 6= ti , fj 6= tj )
i

i

j

j

Then we embed this prior in the M-step of EM algorithm
where we replaced its likelihood estimate with MAP estimate. Although this is for the case of IBM Model 1, IBM
Models 3 and 4 are essentially the same.

Translation Modeling with MWE-sensitive
Word Aligner

First Prior Knowledge: MWEs The first type of priorknowledge is MWEs with their exact bilingual correspondences. Such correspondences in the training sentences canbe incorporated by the prior which we shown in the previous
paragraph.

After introducing an MWE-sensitive word aligner, we will
mention three types of linguistic knowledge, MWEs, paraphrases, and OOV words, which we intend to incorporate tothis word aligner.

Second Prior Knowledge: Paraphrases The second typeof prior knowledge is paraphrases. Recently, various statistical methods are developed which extract paraphrases (Zhaoand Wang 2010). Paraphrases here is assumed to be extracted by the method of (Bannard and Callison-Burch 2005)
where one of the pivot should be identical with the pivot language. Note that (Bannard and Callison-Burch 2005) extract
the most likely alternative phrase ē2 for a given phrase ē1 bypivoting foreign phrase f , as is shown in (5):
ˆē2

=
=
≈

arg max P (ē2 |ē1 )ē2 :ē2 6=ē1X
P (f¯|ē1 )P (ē2 |f¯, ē1 )arg maxē2 :ē2 6=ē1

arg maxē2 :ē2 6=ē1

f¯

XP (f¯|ē1 )P (ē2 |f¯)

(5)

f¯

P
where P (f¯|ē) = count(ē, f¯)/ f¯ count(ē, f¯). Then, thisinformation is converted into the known correspondence
in the training corpus, which is plugged into the MWEsensitive word aligner. Note that Algorithm 1 intend toincorporate paraphrases directly into a translation model,
which is the difference here.
Third Prior Knowledge: OOV Words The third type ofprior knowledge has quite different nature compared to theabove two. We will know the lists of OOV words after theconstruction and the execution of a MT system. Several categories of OOV words are shown below.
• Transliteration related terms.
• Proper nouns: Proper nouns represent unique entities,
such as person’s name, organization’s name, locations
name, signal name (electronics), chemical name (chemistry), and so forth.1• Localization (”l10n”) or internationalization (”i18n”) terminology: these are the matters in computer systems
which support multiple languages.2• Equations and algorithms: these are often embedded in a
sentence without definite boundary.
• Symbols and Encoding related characters.
• Noise: Noise elements are yielded due to the nonexistence in training corpus or the fault in word / phrasalalignment.
Firstly, these tend to appear less frequently in training corpus. Hence, by the statistical methods it is often not easy
to learn their correspondences by statistics. Secondly, theircorrespondences are often not affected by the surrounding
context. Hence, once we know the correspondences, it isfairly easy to be retrieved. Thirdly, it is often difficult toenumerate whole the possible forms, for example the day /time format. In sum, while the detection algorithm are in
1

general not easy to write which performs very well for general corpora, it is fairly easy to write an algorithm which
can perform relatively well for the given corpus. For theproper nouns, a named-entity recognizer aims to detect such
entities (Finkel, Grenager, and Manning 2005). For the localization terminology, it is fairly easy to construct using thewell-developed rule-based software for localization / internationalization industries.
After detecting OOV words together with their counterparts, we could incorporate such prior knowledge into the
MWE-sensitive word aligner.

In English, proper nouns are usually capitalized. In Japanese,
there is no particular rules.
2
This includes day / time format, time zones, formatting of
numbers (decimal separator, digit grouping), currency (including
its symbols and its variation within the same country), weights andmeasures, paper sizes, telephone numbers, addresses, postal codes,
titles, government assigned numbers (social security number in US,
national insurance number in UK).

Translation Model Smoothing
A translation model smoothing method intends to examine
the outcomes of word alignment and phrase extraction interms of their probabilities. This is since it is in general believed that the relative frequencies are better smoothed dueto the data sparseness and ignorance of zero probabilities(Foster, Kuhn, and Johnson 2006). We think that there areat least two cases that smoothing will be fairly effective: acase when a corpus size is relatively small and a case whena corpus includes n-grams whose order are unbalanced. The
case which we examine in this paper is likely to fall among
the latter case. This is since it is likely that the given terminology may include higher n-grams and a language model
may not even include their back-offs.
We consider the statistical smoothing method based on
the hierarchical Pitman-Yor process, which is a nonparametric generalization of the Dirichlet distribution that produces
power-law distributions (Goldwater, Griffiths, and Johnson
2006). First we review the hierarchical Pitman-Yor processbased language model, and then, we introduce a hierarchicalPitman-Yor process-based translation model.

Review of Hierarchical Pitman-Yor LM
HPYLM: Generative Model Hierarchical Pitman-Yor
Language Model (HPYLM) (Goldwater, Griffiths, and Johnson 2006; Teh 2006; Mochihashi, Yamada, and Ueda 2009;
Okita and Way 2010) is constructed encoding the propertyof the power-law distribution.
Let P Y (d, θ, G0 ) denotes a Pitman-Yor process (Pitman
1995), d denotes a discount parameter, θ denotes a strengthparameter, and G0 a base distribution. We define π(u) as thesuffix of u consisting of all but the earliest word in Equation(6) as in (Teh 2006): we see u as n-gram words and π(u)as (n-1)-gram words. Then, we place a Pitman-Yor process
prior recursively over Gπ(u) in the generative model, as isshown in (6):(
Gu |d|u| , θ|u| , Gπ(u) ∼ P Y (d|u| , θ|u| , Gπ(u) )...
(6)G∅ |d0 , θ0 , G0 ∼ P Y (d0 , θ0 , G0 )Note that the discount and strength parameters are functionsof the length |u| of the context, while the mean vector is
Gπ(u) , and the vector of probabilities of the current word
given all but the earliest word in the context.

where chwk is the
Pnumber of customers seated at table k until
now, and th· = w thw is the total number of tables in h.
Hence, the predictive distribution of n-gram probability
in HPYLM is recursively calculated as in (8):p(w|h)=

c(w|h) − d · thw
θ + d · th·+p(w|h′ ) (8)θ + c(h)θ + c(h)′where p(w|h ) is the same probability using a (n-1)-gramcontext h′ . Implementation of this inference procedure relates to the Markov chain Monte Carlo sampling. The simplest way is to build a Gibbs sampler while more efficient
way is to build a blocked Gibbs sampler (Mochihashi, Yamada, and Ueda 2009).

Translation Model Smoothing
An n-gram is often defined as a subsequence of n items froma given sequence where items can be phonemes, syllables,
letters, words or base pairs. Although we can extend thisdefinition of n-gram to the one which includes ‘phrases’,let us use the different term ‘n-phrase-gram’ instead in thispaper, in order not to mix up with the n-gram for words. Fig.
1 shows a typical example of phrase extraction process. In
this process, under the consistency constrained, phrase pairs
are extracted which is depicted in the center. Note that thisfigure is depicted separating the source and the target sides.
Fig. 2 shows the same figure if we depict them in pairs.
The lowest column includes only 1-phrase-grams, the second lowest column includes 2-phrase-grams, and so on. Theline connecting two nodes indicates parent-child relations.
Then, this becomes the lattice structure of the generated
phrase pairs. These generated phrase pairs may have several paths to yield the whole sentences. As is similar with
HPYLM, we can limit this by considering the suffix of asequence, meaning that we can process a sequence always
from left-to-right. Hence, although the natural lattice would
include the dashed lines, the dashed lines can be eliminatedif we impose constraint that we should always read the suffix of this sequence from left-to-right. This constraint makes
the resulted structure a tree. If the resulted structure is a tree,
we can employ the same strategy with HPYLM. The predictive distribution can be calculated by Equation (8) with thereplacement of n-grams with n-phrase-grams.

michael geht davon aus, dass er im haus bleibt

house

dass erdass

erbleibthaus

bleibtmichael
geht
davon
aus

im hausimgeht davon aus,

im haus bleibt

mchaelassumes
that
he
will
stay
in
the
house

michaelassumesdass er im haus bleibt

dass
er
im
haus

in the housein thewill stayhewill stay in the housemichael geht davon aus

that

that hemichael assumes thatmichael assumes that hemichael geht davon aus , dassmichaelmichael assumes

that he will stay in the housemichael geht davon aus , dass er

michael assumes that he will stay in the house

HPYLM: Inference One procedure to do an inference inorder to generate words drawn from G is called Chinese
restaurant process, which iteratively marginalizes out G.
Let h be an n-gram context; for example in 3-gram, thisis h = {w1 , w2 }. A Chinese restaurant contains an infinite number of tables t, each with infinite seating capacity.
Customers, which are the n-gram counts c(w|h), enter therestaurant and seat themselves over the tables 1, . . . , thw· .
The first customer sits at the first available table, while eachof the subsequent customers sits at an occupied table withprobability proportional to the number of customers alreadysitting there chwk − d, or at a new unoccupied table withprobability proportional to θ + d · th· as is shown in (7):chwk − d (1 ≤ k ≤ thw ).
w|h ∼
(7)
θ + d · th· (k = new).

Figure 1: A toy example of phrase extraction process. Resulted phrase pairs can be described as a lattice structure.

Experimental Results
Our baseline was a standard log-linear PB-SMT system
based on Moses. The GIZA++ implementation (Och and
Ney 2003) of IBM Model 4 was used for word alignment. For phrase extraction the grow-diag-final heuristics
described in (Och and Ney 2003) was used to derive the refined alignment. We then performed MERT process (Och2003) which optimizes the BLEU metric, while a 5-gram
language model was derived with Kneser-Ney smoothing
trained with SRILM (Stolcke 2002) on the English side of
the training data. We used Moses (Koehn et al. 2007) fordecoding.
We used NTCIR-8 patent corpus for JP–EN (Fujii et al.
2010). We randomly selected 200k sentence pairs as training corpus. For JP–EN patent corpus, we used 1.2k sentence for development set while we used a test set preparedfor NTCIR-8 evaluation campaign. Japanese side was segmented by Cabocha (Kudo and Matsumoto 2003). Table 1
shows the statistics of each prior knowledge. We prepared
terminology without using external resources but with theinterference of human beings. For the first prior knowledge,
MWEs are extracted by the heuristic MWE-extraction strategy similar to (Kupiec 1993), and then corrected these extracted terminology by hand inspecting the corpora. For thesecond prior knowledge, paraphrases are extracted by themethod described in (Bannard and Callison-Burch 2005).
For the third prior knowledge, OOV word lists were createdin this way. We constructed a PB-SMT decoder, decoded
all the training corpus as well as test corpus, and collected
whole the OOV words from the translation outputs. Then,
we supply the translation counterparts by human beings.3
Table 2 shows our results. Without translation modelsmoothing, the improvement of BLEU by the prior 1 was
0.80 BLEU point absolute, the prior 2 was 0.65 BLEU point
absolute, the prior 3 was 0.58 BLEU point absolute, and the3
Due to the way of segmentation, around 20% of the transliteration terms was not possible to find their counterparts.

Lattice structure (Tree structure if we only accept real lines)
michael assumes that he will stay in the house|| michael geht davon aus, dass er im haus bleibt

michael assumes that he || michael geht davon aus , dass erthat he will stay in the house ||michael assumes that || michael geht davon aus , dasswill stay in the house ||dass er im haus bleibt

im haus bleibtthat he || dass er
michael assumes || michael geht davon aus
in the house || im haus2−phrase−grammichael || michaelassumes || geht davon austhat || dass1−phrase−gramhe || erin the || imhouse || haus

will stay || bleibt

Figure 2: Figure shows a lattice structure of translation model for a toy example.
JP–ENprior knowledge 1prior knowledge 2prior knowledge 3

MWEs
paraphrases
transliteration
proper nouns
localization
equationssymbolsnoise

training
120070432135
25928
3408
207
103
13842
19007

test
3865
—
284
2
2
1
684
175

Table 1: Statistics of prior knowledge.
JP–ENbaselineprior 1prior 2prior 3all 1-3
heuristics

without TM smoothing
21.68
22.48
22.43
22.26
22.95
21.90

with TM smoothing
22.44
22.78
22.64
22.52
23.03
22.49

Table 2: Results for 200k JP–EN sentences. Heuristics inthe last row shows the result when prior knowledge 1 wasadded at the bottom of the translation model.

focus is on the effect of bilingual terminology to the training step: the performance of an MWE-senstive word aligner
and the translation model smoothing method. When we
use the both of these, we obtained the improvement of 1.35
BLEU point absolute and 6.0% relative for this settings. We
obtained the positive results by only the translation modelsmoothing as well. Note that considering the fact that there
are various interventions of human beings including giving
the exact fragments of answer in test sentences, this improvement is rather too small than we expected.
There are several avenues for further research. Firstly,
this paper does not consider syntactical issues. It would
be interesting to see the same results on factored translation
model (Koehn and Hoang 2007) incorporating the deep parsing results, while it is also interesting to extend the MWEsensitive word aligner in order that it can incorporate syntactical features as prior knowledge. Secondly, this paper uses
the in-domain linguistic knowledge, i.e. MWEs and paraphrases are trained within training corpus. We would liketo see the results for an out-of-domain corpus. Similarly,
it would be interesting to see whether the approach of thehierarchical Pitman-Yor process may work as well for theout-of-domain prior knowledge.

Acknowledgments
prior 1 to 3 was 1.27 BLEU point absolute. With translation
model smoothing, the improvement of BLEU compared tothe baseline with no TM smoothing by the prior 1 was 1.10
BLEU point absolute, the prior 2 was 0.96 BLEU point absolute, the prior 3 was 0.85 BLEU point absolute, and theprior 1 to 3 was 1.35 BLEU point absolute. With translation
model smoothing, the improvement of BLEU compared tothe baseline with TM smoothing by the prior 1, 2 and 3 wasrather very small. This shows that an MWE-sensitive aligner
and the translation model smoothing improved the results if
we applied them separately, but the combined effect was notmuch observed unless we incorporate MWEs, paraphrases,
and OOVs together.

Conclusion and Further Studies
This paper considered a scenario when we are given almost
perfect knowledge about bilingual terminology in terms ofa test corpus in Statistical Machine Translation (SMT). The

This research is supported by the Science Foundation Ireland (Grant 07/CE/I1142) as part of the Centre for Next Generation Localisation (http://www.cngl.ie) at Dublin
City University. We would also like to thank the Irish
Centre for High-End Computing and Machine Translation
Marathons.

References
Bannard, C., and Callison-Burch, C. 2005. Paraphrasing
with biingual parallel corpora. Proceedings of the 43nd Annual Meeting of the Association for Computational Linguistics (ACL 2005) 597–604.
Brown, F.; Pietra, V.; Pietra, A.; ; and Mercer, R. 1993.
The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, Vol.19, Issue 2
263–311.
Callison-Burch, C.; Koehn, P.; and Osborne, M. 2006. Improved statistical machine translation using paraphrases. In

Proceedings of the North American Chapter of the Association for Computationa Linguistics (NAACL 2006) 17–24.
Chiang, D. 2005. A hierarchical phrase-based model forstatistical machine translation. In Proceedings of the 43th
Annual Meeting of the Association for Computational Linguistics (ACL-05) 263–270.
Finkel, J. R.; Grenager, T.; and Manning, C. 2005. Incorporating non-local information into information extraction
systems by gibbs sampling. Proceedings of the 43nd Annual Meeting of the Association for Computational Linguistics (ACL 2005) 363–370.
Foster, G.; Kuhn, R.; and Johnson, H. 2006. Phrasetablesmoothing for statistical machine translation. In Proceedings of the Empirical Methods in Natural Language Processing (EMNLP 2006) 53–61.
Fujii, A.; Utiyama, M.; Yamamoto, M.; Utsuro, T.; Ehara,T.; Echizen-ya, H.; and Shimohata, S. 2010. Overview ofthe patent translation task at the NTCIR-8 workshop. In Proceedings of the 8th NTCIR Workshop Meeting on Evaluationof Information Access Technologies: Information Retrieval,
Question Answering and Cross-lingual Information Access
293–302.
Goldwater, S.; Griffiths, T. L.; and Johnson, M. 2006. Contextual dependencies in unsupervised word segmentation.
In Proceedings of Conference on Computational Linguistics / Association for Computational Linguistics (COLINGACL06) 673–680.
Koehn, P., and Hoang, H. 2007. Factored translation models. In Proceedings of the Empirical Methods in Natural
Language Processing (EMNLP 2007) 868–876.
Koehn, P.; Hoang, H.; Birch, A.; Callison-Burch, C.; Federico, M.; Bertoldi, N.; Cowan, B.; Shen, W.; Moran, C.;
Zens, R.; Dyer, C.; Bojar, O.; Constantin, A.; and Herbst, E.
2007. Moses: Open source toolkit for Statistical MachineTranslation. In Proceedings of the 45th Annual Meeting of
the Association for Computational Linguistics Companion
Volume Proceedings of the Demo and Poster Sessions 177–
180.
Koehn, P.; Och, F.; and Marcu, D. 2003. Statisticalphrase-based translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computationa Linguistics (HLT /NAACL 2003) 115–124.
Koehn, P. 2010. Statistical machine translation. Cambridge
University Press. Cambridge. UK.
Kudo, T., and Matsumoto, Y. 2003. Fast methods for kernelbased text analysis. In Proceedings of the Annual Meeting of
the Association for Computational Linguistics (ACL 2003)
24–31.
Kupiec, J. 1993. An algorithm for finding noun phrase correspondences in bilingual corpora. In Proceedings of the
31st Annual Meeting of Association for Computational Linguistics 17–22.
Marcu, D., and Wong, W. 2002. A phrase-based, joint
probability model for statistical machine translation. In Pro-

ceedings of the Conference on Empirical Methods in Natural
Language Processing (EMNLP-2002) 133–139.
Mochihashi, D.; Yamada, T.; and Ueda, N. 2009. Bayesian
unsupervised word segmentation with nested pitman-yor
language modeling. In Proceedings of Joint Conference ofthe 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (ACL-IJCNLP 2009)100–108.
Och, F., and Ney, H. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics
29(1):19–51.
Och, F. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of the 41st Annual
Meeting of the Association for Computational Linguistics
160–167.
Okita, T., and Way, A. 2010. Pitman-Yor process-based language model for Machine Translation. International Journalon Asian Language Processing 21(2):57–70.
Okita, T.; Guerra, A. M.; Graham, Y.; and Way, A. 2010.
Multi-Word Expression-sensitive word alignment. In Proceedings of the Fourth International Workshop On Cross
Ling ual Information Access (CLIA2010, collocated with
COLING2010) 26–34.
Okita, T.; Graham, Y.; and Way, A. 2010. Gap between theory and practice: Noise sensitive word alignment in machinetranslation. In Journal of Machine Learning Research Workshop and Conference Proceedings Volume 11: Workshop on
Applications of Pattern Analysis (WAPA2010) 119–126.
Okita, T. 2009. Data cleaning for word alignment. In Proceedings of Joint Conference of the 47th Annual Meeting of
the Association for Computational Linguistics and the 4thInternational Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (ACL-IJCNLP 2009) Student Research Workshop
72–80.
Pitman, J. 1995. Exchangeable and partially exchangeable
random partitions. Probability Theory and Related Fields
102:145–158.
Stolcke, A. 2002. SRILM – An extensible language modeling toolkit. In Proceedings of the International Conferenceon Spoken Language Processing 901–904.
Sumita, E. 2000. Lexical transfer using a vector-spacemodel. In Proceedings of the 38th Annual Meeting of the
Association for Computational Linguistics (ACL 2000) 425–
431.
Teh, Y. W. 2006. A hierarchical Bayesian language model
based on Pitman-Yor processes. In Proceedings of Joint
Conference of the 44th Annual Meeting of the Association
for Computational Linguistics 985–992.
Zhao, S., and Wang, H. 2010. Paraphrases and applications.
Coling 2010: Paraphrases and Applications–Tutorial notes
1–87.