Identifying Bilingual Synonymous Technical Terms
from Phrase Tables and Parallel Patent Sentences
Bing Liang
Takehito Utsuro
Mikio Yamamoto
Graduate School of Systems and Information Engineering, University of Tsukuba,
Tsukuba, 305-8573, JAPAN

Abstract
In the process of translating patent documents, a bilingual lexicon of technical terms is inevitable knowledge
source. It is important to develop techniques of acquiring
technical term translation equivalent pairs automaticallyfrom parallel patent documents. We take an approach ofutilizing the phrase table of a state-of-the-art phrase-based
statistical machine translation model. In the task of acquiring technical term translation equivalent pairs, we consider
situations where a technical term is observed in many parallel patent sentences and is translated into many translationequivalents. In the task of acquiring technical term translation equivalent pairs, this paper studies the issue of identifying synonymous translation equivalent pairs. First, we collect candidates of synonymous translation equivalent pairs
from parallel patent sentences. Then, we analyze features
for identifying synonymous translation equivalent pairs. Finally, we apply the Support Vector Machines (SVMs) to thetask of identifying bilingual synonymous technical terms,
and achieve the performance of almost 98% precision andover 40% F-measure.
Key Words - Bilingual Lexicon Acquisition, Synonyms,
Phrase Tables

1. Introduction
For both high quality machine and human translation, alarge scale and high quality bilingual lexicon is the mostimportant key resource. Since manual compilation of bilingual lexicon requires plenty of time and huge manual labor,
in the research area of knowledge acquisition from natural
language text, automatic bilingual lexicon compilation havebeen studied. Techniques invented so far include translation term pair acquisition based on statistical co-occurrence
measure from parallel sentences [7], translation term pairacquisition from comparable corpora [2], compositionaltranslation generation based on an existing bilingual lexicon for human use [9], and translation term pair acquisition
by collecting partially bilingual texts through the search engine [3].
Among those efforts of acquiring bilingual lexicon fromtext, [8] studied to acquire technical term translation lexicon from phrase tables, which are trained by a phrasebased statistical machine translation model with parallel
sentences automatically extracted from parallel patent documents. Recently, we further studied to require the acquired technical term translation equivalents to be consistent with word alignment in parallel sentences and achieved
91.9% precision with almost 70% recall. This technique
has been actually adopted by a Japanese organization whichis responsible for translating Japanese patent applications
published by the Japanese Patent Office (JPO) into English, where it has been utilized in the process of semiautomatically compiling bilingual technical term lexicon
from parallel patent sentences. In this process, persons who
are working on compiling bilingual technical term lexicon
judge whether to accept or not candidates of bilingual technical term pairs presented by the system.
Based on the achievement so far, in this paper, we consider situations where a technical term is observed in many
parallel patent sentences and is translated into many translation equivalents. More specifically, in the task of acquiring
technical term translation equivalent pairs, this paper studies the issue of identifying synonymous translation equivalent pairs. First, we collect candidates of synonymous translation equivalent pairs from parallel patent sentences. Then,
we analyze features for identifying synonymous translationequivalent pairs. Finally, we apply the Support Vector Machines (SVMs) [12] to the task of identifying bilingual synonymous technical terms, and achieve the performance ofalmost 98% precision and over 40% F-measure.

2. Japanese-English Parallel Patent Documents
In the NTCIR-7 workshop, the Japanese-English patenttranslation task is organized [1], where parallel patent documents and sentences are provided by the organizer. Thoseparallel patent documents are collected from the 10 yearsof unexamined Japanese patent applications published by
the Japanese Patent Office (JPO) and the 10 years patent
grant data published by the U.S. Patent & Trademark Office(USPTO) in 1993-2000. The numbers of documents are approximately 3,500,000 for Japanese and 1,300,000 for English. Because the USPTO documents consist of only patent

that have been granted, the number of these documents issmaller than that of the JPO documents.
From these document sets, patent families are automatically extracted and the fields of “Background of the Invention” and “Detailed Description of the Preferred Embodiments” are selected. This is because the text of those fields
is usually translated on a sentence-by-sentence basis. Then,the method of [11] is applied to the text of those fields, andJapanese and English sentences are aligned.

3. Phrase Table of an SMT Model
As a toolkit of a phrase-based statistical machine translation model, we use Moses [5] and apply it to the whole
1.8M parallel patent sentences. In Moses, first, word alignment of parallel sentences are obtained by GIZA++ in bothtranslation directions and then the two alignments are symmetrised. Next, any phrase pair that is consistent withword alignment is collected into the phrase table and aphrase translation probability is assigned to each pair. Morespecifically, we construct a phrase table in the direction ofJapanese to English translation, and another one in the opposite direction of English to Japanese translation. In the direction of Japanese to English translation, we finally obtain
76M translation pairs with 33M unique Japanese phrases,
i.e., 2.29 English translations per Japanese phrase on average, with Japanese to English phrase translation probabilities P (pE | pJ ) of translating a Japanese phrase p J into an
English phrase p E . For each Japanese phrase, those multiple translation candidates in the phrase table are ranked
in descending order of Japanese to English phrase translation probabilities. In the similar way, in the phrase table in
the opposite direction of English to Japanese translation, foreach English phrase, multiple Japanese translation candidates are ranked in descending order of English to Japanesephrase translation probabilities.
Those two phrase tables are then referred to when identifying a bilingual technical term pair, given a parallel sentence pair SJ , SE  and a Japanese technical term t J , oran English technical term t E . In the direction of Japanese
to English, given a parallel sentence pair S J , SE  containing a Japanese technical term t J , the Japanese to English
phrase table is referred to when identifying a bilingual technical term pair. From the Japanese to English phrase table,
candidates of translating t J into English which are consistent with word alignment are collected. Then, those English
translation candidates are matched against the English sentence SE of the parallel sentence pair, and those which are
not found in S E are filtered out. Finally, among the remaining translation candidates, t̂E with the largest translation
probability P (t E | tJ ) is selected and the bilingual technical term pair tJ , t̂E  is identified. The precision of identifying bilingual technical term pair here is 91.9%. Similarly,
in the opposite direction of English to Japanese, given a parallel sentence pair SJ , SE  containing an English technicalterm tE , the English to Japanese phrase table is referred towhen identifying a bilingual technical term pair.

4. Developing a Reference Set of Bilingual Synonymous Technical Terms
The following describes the procedure of developing areference set of bilingual synonymous technical terms fromthe whole 1.8M parallel patent sentences and the Japanese
to English / English to Japanese phrase tables. Figure 1
illustrates the whole procedure.
1. First, a initial Japanese noun phrase t 0J is randomly
selected from the Japanese part of the 1.8M parallel
patent sentences.
2. Then, to the initial Japanese noun phrase t 0J , the following “Iteration: Generating Candidates Bilingual
Synonymous Term Pairs” is applied, where the iteration is repeated steps of translation generation fromthe 1.8M parallel patent sentences and the Japanese
to English / English to Japanese phrase tables 1 . Next,
the initial set CBP (t0J ) of candidate bilingual synonymous term pairs is generated as in the left half of Figure 1.
Iteration:

Generating Candidates of
Bilingual Synonymous Term Pairs

1st step Given the input Japanese term t J , collect all
the parallel sentence pairs which contain t J fromthe 1.8M parallel patent sentences. Next, fromeach parallel sentence pair, t J is translated into
English according to the procedure in the previous section, referring to the Japanese to English phrase table. Then, all the bilingual termpairs tJ , tiE  are collected into the initial set
CBP (tJ ) of candidates bilingual synonymous
term pairs2 .
2nd step Similarly, for each English term t E in
CBP (tJ ), collect all the parallel sentence pairs
which contain t E from the 1.8M parallel patentsentences, and translate t E into Japanese, referring to the English to Japanese phrase table.
Then, all the bilingual term pairs t iJ , tE  are
added to CBP (tJ ).
3rd step Similarly, for each Japanese term t J in
CBP (tJ ), collect all the parallel sentence pairs
which contain t J from the 1.8M parallel patentsentences, and translate t J into English, referringto the Japanese to English phrase table. Then,
all the bilingual term pairs t J , tiE  are added to
CBP (tJ ).
4th step Repeat the procedure of the “2nd step”.
1 The number of iteration 6 here is based on our preliminary evaluation,and is decided so that most synonymous bilingual technical terms are generated from the initial Japanese phrase t0J , while the number of candidates
other than true synonyms is minimized. Throughout those steps, we simply
avoid duplicate generation of terms.
2 Throughout the steps from the “1st” to the “6th”, we only keep bilingual term pairs which satisfy the lower bound 6 as well as the upper bound
800 of the co-occurring frequency in the 1.8M parallel patent sentences.

Figure 1. Developing a Reference Set of Bilingual Synonymous Technical Terms
Table 1. Number of Bilingual Technical Terms: Candidates and Reference of Synonyms# of bilingual technical terms for the total 134 seeds
average per seedCandidates


167.7
 CBP (sJ ) = 22,473
of SynonymssJ
Reference of
SBP (sJE ) = 1,680
12.5

SynonymssJE

5th step Repeat the procedure of the “3rd step”.
6th step Repeat the procedure of the “2nd step”.
After the candidate generation iteration, we restrict the
set CBP (t0J ) as having more than or equal to 10 members (i.e., | CBP (t0J ) |≥ 10). In the evaluation of thispaper, out of 4,000 randomly selected initial Japanese
noun phrases and corresponding initial sets CBP (t 0J ),about 350 sets satisfy the lower bound of the numberof members.
3. Next, out of the members of the initial set CBP (t 0J ) of
candidates bilingual synonymous term pairs for the initial Japanese noun phrase t 0J , we select the seed bilingual term pair s JE = sJ , sE  as below:
First, in order to distinguish technical terms and general terms and to select bilingual technical term pairs
as seeds, we assume the candidates of seeds to satisfy
at least one of the following requirements:(a) The co-occurring frequency of the bilingual termpair in the 1.8M parallel patent sentences is lessthan 500.
(b) The character length of the Japanese term is morethan two when it contains kanji (Chinese char-acters) or hiragana (Japanese characters). The
Japanese term consists of more than one morpheme when all of its characters are katakana(Japanese characters for foreign words).
(c) The English term consists of more than one word.
Then, we manually examine the bilingual term pairwith the largest co-occurring frequency in the 1.8M
parallel patent sentences. If the one with the largest cooccurring frequency is appropriate as a pair of technical terms, we select it as seed. Otherwise, we manually
examine all the members of the initial set CBP (t 0J )and select the most appropriate pair as seed. If the initial set CBP (t0J ) does not include any pair of bilingual technical terms, we discard the set CBP (t 0J ) at
this step.
In the evaluation of this paper, out of all the initial sets
CBP (t0J ), for about 29% of the initial sets, we keep
the bilingual term pair with the largest co-occurring
frequency as seed, for about 14% of them, we manually select as seed the pair other than the one withthe largest co-occurring frequency, and for the remaining 57%, we discard the initial sets CBP (t 0J ). It took
about 5.5 minutes on average to manually examine all
the members of each initial set CBP (t 0J ).

4. To the Japanese technical term s J of the seed bilingual technical term pair s JE = sJ , sE , “Iteration:
Generating Candidates Bilingual Synonymous Term
Pairs” is applied. As the result of this iteration, theset CBP (sJ ) of candidates of bilingual synonymous
technical term pairs is generated as in the right half of
Figure 1. Here, we again restrict the set CBP (s J )as having more than or equal to 10 members (i.e.,| CBP (sJ ) |≥ 10). In the evaluation of this paper, about 90% of the sets CBP (s J ) satisfy the lower
bound of the number of members. Finally, we have
134 seed bilingual technical term pairs, where thenumber of bilingual technical terms in total and theiraverage are shown in Table 1.
5. Finally, for each seed bilingual technical term pairsJE = sJ , sE , we manually divide the set CBP (s J )of candidates of bilingual synonymous technical termpairs into SBP (sJE ), those of which are synonymouswith sJE , and the remaining N SBP (s JE ). As in Table 1, the number of bilingual technical terms includedin SBP (sJE ) in total for all of the 134 seed bilingual
technical term pairs is 1,680, which amounts to 12.5
per seed on average.

5. Identifying Bilingual Synonymous TechnicalTerms by Machine Learning
In this section, we apply the SVMs to the task of identifying bilingual synonymous technical terms.

5.1. The Procedure
First, let CBP be the union of the sets CBP (s J ) of candidates of bilingual synonymous technical term pairs for allof the 134 seed bilingual technical term pairs. In the training and testing of the classifier for identifying bilingual synonymous technical terms, we first divide the set of 134 seedbilingual technical term pairs into 10 subsets. Here, for eachi-th subset (i = 1, . . . , 10), we construct the union CBP i ofthe sets CBP (sJ ) of candidates of bilingual synonymous
technical term pairs, where CBP 1 , . . . , CBP10 are 10 disjoint subsets3 of CBP .
As a tool for learning SVMs, we use TinySVM (http://chasen.org/˜taku/software/TinySVM/). As
the kernel function, we use the polynomial (2nd order) kernel. In the testing of a SVMs classifier, we regard the distance from the separating hyperplane to each test instanceas a confidence measure, and return test instances satisfying confidence measures over a certain lower bound only
as positive samples (i.e., synonymous with the seed). In
the training of SVMs, we use 8 subsets out of the whole
10 subsets CBP1 , . . . , CBP10 . Then, we tune the lower
3 Here, we divide the set of 134 seed bilingual technical term pairs into
10 subsets so that the numbers of positive (i.e., synonymous with the seed)/ negative (i.e., not synonymous with the seed) samples in each CBPi
(i = 1, . . . , 10) are comparative among the 10 subsets.

Table 3. Evaluation Results (%)

Baseline (tJ and sJ
are identical, or,tE and sE are identical.)Maximum
SVM
Precision
Maximum
F-measure

Precision

RecallF-measure

67.0

54.3

60.8

97.5

28.7

43.9

73.5

68.1

70.5

bound of the confidence measure with one of the remaining
two subsets. With this subset, we also tune the parameterof TinySVM for trade-off between training error and margin. Finally, we test the trained classifier against anotherone of the remaining two subsets. We repeat this procedureof training / tuning / testing 10 times, and average the 10
results of test performance.

5.2. Features
Table 2 lists all the features used for training and testing of SVMs for identifying bilingual synonymous technical terms. Features are roughly divided into two types:
those of the first type f 1 , . . . , f6 simply represent various
characteristics of the input bilingual technical term t J , tE ,
while those of the second type f 7 , . . . , f16 represent relationof the input bilingual technical term t J , tE  and the seedbilingual technical term pair s JE = sJ , sE .
Among the features of the first type are the frequency(f1 ), ranks of terms with respect to the conditional translation probabilities (f 2 and f3 ), length of terms (f 4 and f5 ),and the number of times repeating the procedure of generating translation with the phrase tables until generating input
terms tJ and tE from the Japanese seed term s J (f6 ).
Among the features of the second type are identity ofmonolingual terms (f 7 and f8 ), edit distance of monolingual terms (f9 ), character bigram similarity of monolingual terms (f10 ), rate of identical morphemes / words (f 11 ),string subsumption and variants for Japanese (f 12 ), identical stems for English (f 13 ), hyphen / space of English terms,(f14 ), compositional translation with an existing bilingual
lexicon4 (f15 ), and translation by the phrase tables (f 16 ).

5.3. Evaluation Results
Table 3 shows the evaluation results for a baseline as well
as for SVMs. As the baseline, we simply judge the input
bilingual term pair t J , tE  as synonymous with the seedbilingual technical term pair s JE = sJ , sE  when tJ andsJ are identical, or, t E and sE are identical. When training / testing a SVMs classifier, we tune the lower bound of
the confidence measure of the distance from the separating
hyperplane in two ways: i.e., for maximizing precision and
4 As the existing Japanese-English bilingual lexicon, Eijiro (http://
www.eijiro.jp/, Ver.79, with 1.6M translation pairs, is used.

for maximizing F-measure. When maximizing precision,
we achieve almost 98% precision where F-measure is over
40%. When maximizing F-measure, we achieve over 70%
F-measure with over 73% precision and over 68% recall.
Table 4 also show examples of improving the baseline by
SVMs.
Table 4 (a) shows the case of correctly judging as “synonym” only by the proposed method. Here, the baseline
judges as “not synonym”, since neither t J and sJ nor tE
and sE are identical. With the proposed method, on theother hand, f 13 returns 1 since “holding” and “hold” havethe same stem. Also, f16 returns 1 since, by the phrase tables, “ホールド回路” can be generated by translating “holding circuit”, and “ 保持回路” can be generated by translating“hold circuit”.
Table 4 (b) shows the case of correctly judging as “notsynonym” only by the proposed method. Here, the baseline
judges as “synonym”, since t E and sE are identical. With
the proposed method, on the other hand, both edit distancesimilarity f9 and character bigram similarity f 10 return 0 for
the Japanese terms “転写器” and “搬送ユニット”. Also,
f15 returns 0 since, by compositional translation with anexisting bilingual lexicon, “ 転写器” cannot be generated bytranslating “transfer unit”, nor “ 搬送ユニット” cannot be
generated by translating “transfer unit”.

6. Related Works
Among related works on acquiring bilingual lexicon
from text, [4] focused on automatic validation of translation pairs available in the phrase table learned by a statistical machine translation model, where their study differswith this paper in that [4] did not study the issue of synonymous bilingual technical terms. [10] is mostly related
to our study, in that they also proposed to apply machinelearning technique to the task of identifying synonymous
bilingual technical terms and that the features of machinelearning studied in [10] are closely related those studied inthis paper. However, [10] studied the issue of identifying
synonymous bilingual technical terms only within manually compiled bilingual technical term lexicon and thus arequite limited in its applicability. Our study in this paper,on the other hand, is quite advantageous in that we startfrom parallel patent documents which continue to be published every year and then, that we can generate candidatesof synonymous bilingual technical terms automatically.
Our study in this paper is also different from previous works on identifying synonyms based on bilingual andmonolingual resources (e.g. [6]) in that we learn synonymous bilingual technical terms from phrase tables of aphrase-based statistical machine translation model trainedwith very large parallel sentences.

7. Conclusion
In the task of acquiring technical term translation equivalent pairs, this paper studied the issue of identifying syn-onymous translation equivalent pairs. We applied the SVMs
to this task and achieved the performance of almost 98%
precision and over 40% F-measure. One of the most important future works is definitely to improve recall. To dothis, we plan to simply introduce a semi-automatic framework, where we employ the strategy of selecting more than
one seeds for each set of candidates bilingual synonymous
term pairs, and automatically identify bilingual technicalterm pairs that are synonymous with one of those seeds.
Then, we improve recall by manually judging whether eachpair of two seeds is synonymous or not. It has been required by the organization mentioned in section 1 to introduce the technique invented in this paper into the task ofsemi-automatically compiling a synonym lexicon of bilingual technical terms for patent translation, which is anotherfuture work.

References[1] A. Fujii, M. Utiyama, M. Yamamoto, and T. Utsuro. Toward
the evaluation of machine translation using patent information. In Proc. 8th AMTA, pages 97–106, 2008.
[2] P. Fung and L. Y. Yee. An IR approach for translating new
words from nonparallel, comparable texts. In Proc. 17th
COLING and 36th ACL, pages 414–420, 1998.
[3] F. Huang, Y. Zhang, and S. Vogel. Mining key phrase translations from Web corpora. In Proc. HLT/EMNLP, pages
483–490, 2005.
[4] M. Itagaki, T. Aikawa, and X. He. Automatic validation ofterminology translation consistency with statistical method.
In Proc. MT Summit XI, pages 269–274, 2007.
[5] P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens,
C. Dyer, O. Bojar, A. Constantin, and E. Herbst. Moses:
Open source toolkit for statistical machine translation. In
Proc. 45th ACL, Companion Volume, pages 177–180, 2007.
[6] D. Lin and S. Zhao. Identifying synonyms among distributionally similar words. In Proc. 18th IJCAI, pages 1492–
1493, 2003.
[7] Y. Matsumoto and T. Utsuro. Lexical knowledge acquisition. In R. Dale, H. Moisl, and H. Somers, editors, Handbook of Natural Language Processing, chapter 24, pages
563–610. Marcel Dekker Inc., 2000.
[8] Y. Morishita, T. Utsuro, and M. Yamamoto. Integrating aphrase-based SMT model and a bilingual lexicon for humanin semi-automatic acquisition of technical term translation
lexicon. In Proc. 8th AMTA, pages 153–162, 2008.
[9] M. Tonoike, M. Kida, T. Takagi, Y. Sasaki, T. Utsuro, and
S. Sato. A comparative study on compositional translation
estimation using a domain/topic-specific corpus collectedfrom the web. In Proc. 2nd Intl. Workshop on Web as Corpus, pages 11–18, 2006.
[10] T. Tsunakawa and J. Tsujii. Bilingual synonym identification with spelling variations. In Proc. 3rd IJCNLP, pages
457–464, 2008.
[11] M. Utiyama and H. Isahara. A Japanese-English patent parallel corpus. In Proc. MT Summit XI, pages 475–482, 2007.
[12] V. N. Vapnik.
Statistical Learning Theory.
WileyInterscience, 1998.

Table 2. Features for Identifying Bilingual Synonymous Technical Terms by Machine Learning
definition( where X denotes J or E,
class
featureand sJ , sE  denotes the seed bilingual technical term pair )f1 : frequency
log of the frequency of t J , tE  within the whole parallel patent sentencesfeatures
f2 : rank of the Japanese term
given t E , log of the rank of t J with respect to the descending orderforof the conditional translation probability P(t J | tE )bilingualf3 : rank of the English term
given t J , log of the rank of t E with respect to the descending ordertechnicalof the conditional translation probability P(t E | tJ )term
f4 : number of Japanese charac- number of characters in t J
tJ , tE 
ters
f5 : number of English words
number of words in t E
f6 : number of times generating the number of times repeating the procedure of generating translatranslation by applying the tion by applying the phrase tables until generating t E or tJ from sJ ,phrase tables
as in sJ → · · · → tJ → tE , or, sJ → · · · → tE → tJ
f7 : identity of Japanese terms
returns 1 when t J = sJ
f8 : identity of English terms
returns 1 when t E = sE
ED(tX ,sX )features
f9 : edit distance similarity of f9 (tX , sX ) = 1− max(|t(where ED is the edit distance
X |,|sX |)for themonolingual termsof tX and sX , and | t | denotes the number of characters of t.)
X )∩bigram(sX )|relation of f10 : character bigram similarity f10 (tX , sX ) = |bigram(t(where bigram(t) ismax(|tX |,|sX |)+1
bilingualof monolingual terms
the set of character bigrams of the term t.)|const(tX )∩const(sX )|technical
f11 : rate of identical morphemes f11 (tX , sX ) = max(|const(t(where const(t) is
X )|,|const(sX )|)term(for Japanese) / words (for the set of morphemes (for Japanese) / words (for English) in thetJ , tE 
English)term t.)andthe f12 : subsumption relation of returns 1 when the difference of t J and sJ is only in their suffixes,
seedstrings / variants relation of or only whether or not having the prolonged sound “ ー”, or only insJ , sE 
surface forms (for Japanese their hiragana parts.
terms )
f13 : identical stem (for English returns 1 when the numbers of constituent words words of t E andterms)
sE are the same, and their corresponding constituents have the samestem.
f14 : hyphen / space (for English returns 1 when the difference of t E and sE is only whether having
terms)
hyphen or space.
f15 : compositional translation returns 1 when s J can be compositionally generated by translatingwith an existing bilingual constituents of tE with an existing bilingual lexicon, or, s E can becompositionally generated by translating constituents of t J with anlexiconexisting bilingual lexicon [9].
f16 : translation by the phrase ta- returns 1 when s J can be generated by translating t E with the phrasebletable, or, sE can be generated by translating t J with the phrase table.

seed sJ , sE  ホールド回路, hold circuit

seed sJ , sE  転写器, transfer unit

Table 4. Examples of Improvement by SVM(a) Correctly Judging as “Synonym” only by SVM
tJ , tE 
ReferenceBaseline 保持回路, holding circuit
synonym not synonym(b) Correctly Judging as “Not Synonym” only by SVM
tJ , tE 
ReferenceBaseline
 搬送ユニット, transfer unit not synonym synonym

SVM (Maximum Precision)synonym

SVM (Maximum Precision)
not synonym