A Comparison of Two Variant Corpora:
The Same Content with Different Sources
Kyonghee Paik , Kiyonori Ohtake , Kazuhide Yamamoto

 



ATR Spoken Language Translation Research Laboratories
2-2-2 Hikaridai“Keihanna Science City”, Kyoto 619-0288 JAPANkyonghee.paik, kiyonori.ohtake  @atr.jp
Nagaoka University of Technology
1603-1 Kamitomioka, Nagaoka City, Niigata 940-2188 JAPAN
yamamoto@fw.ipsj.or.jp
Abstract
In order to investigate the effect of source language on translations, we investigate two variants of a Korean translation corpus. The
first variant consists of Korean translations of 162,308 Japanese sentences from the ATR BTEC (Basic Expression Text Corpus). The
second variant was made by translating the English translations of the Japanese sentences into Korean. We show that the source language
text has a large influence on the target text. Even after normalizing orthographic differences, fewer than 8.3% of the sentences in thetwo variants were identical. We describe in general which phenomena differ and then discuss how our analysis can be used in natural
language processing.

1. Introduction
We compare two corpora in order to investigate the effect of source language effect on translation. In particular,
we present an analysis of paraphrased sentences extracted
from bilingual travel corpora. The corpora used for thisresearch consist of 324,616 Korean sentences. Half of the
Korean sentences (162,308 sentences) were translated fromJapanese, and the other half were translated from English
sentences that match the original Japanese.
Although the two Korean corpora should be equivalent
in meaning, they have different characteristics, since they
were translated originally from such different languages as
English and Japanese. English has a relatively fixed word
order (SVO), and complements such as subject and object
are obligatory. On the other hand, Japanese has a relatively free word order, although it is strongly verb-final,and complements can be freely omitted when their referents are clear from the context. The grammar itself and theset of available grammatical constructions are quite different from each other. It is also obvious that the ways of perceiving and conceptualizing facts, activities and emotional
events reflected in basic vocabulary vary with the language.
In this respect, Korean is closer to Japanese rather than toEnglish.
The following examples show the differences in linguistic structure that are extracted from our two corpora. For
example, (1) comes from the Korean corpus translated fromJapanese 1 and (2) comes from the Korean corpus translated
from English2
(1)

' ,.
 !#
 "$&! % (*
  ) +  - /10 32  .
 
this cable-car-acc take-if hotel-loc go can-decl(2)

 !#
 "$de c 0 
\ ^]`_  b a  Z[2f2 .
cable-car-nom hotel-loc take give future

The Korean translation in (2) does not sound natural, but
the one in (1) does. This translation difference comes fromthe influence of the source language. That is, the two translations reflect their original source language. Even though
the original translation pairs are matched between English
and Japanese, the results of the translation are very different. The differences in translations due to the source text
have not been extensively studied. In this paper, we examine whether a clear difference exists throughout the corpus
and investigate further whether we can use this difference
to improve the quality of machine translation. Furthermore,
we confirm that the differences can be applied to paraphrasing the source or target languages, which may lead to improving the quality of machine translation.

2.

Comparing Two Corpora

We have a basic travel expression corpus (BTEC) that is
a collection of Japanese sentences and their English translations for Japanese travelers (an early version is describedin Takezawa et al. (2001)). This corpus covers such topics
related to travel as “shopping”, “hotel/restaurant reservation”, “airport”, “lost and found” and so on. So far, we havedeveloped four language corpora — Japanese, English, Korean, and Chinese, but we exclude Chinese data since they
are not relevant to the present paper. Henceforth, we callthem Kg (the corpus translated from English) and Kh (thecorpus translated from Japanese). The size of the corpora
and amount of redundancy is shown in Table 1.
Ki
Kj
162,320 162,308
103,051
92,816
36.5%
42.8%
Table 1: Summary of BTEC (Kh & Kg )All
UniqueRedundancy

1
The original Japanese sentence is “ 457698;:<;=>8;?@BA
CEDFHG<I?JLKM47NPORQTSHUWVYX ”(lit. trans. “If you take thiscable car, to go to the hotel is possible.”2
The original English sentence is “This cable car will take you
to the hotel.”

1791

Japanese
162,320
102,247
37.0%

English162,320
97,32640.0%

All sentences
Unique sentences

0-0.1
100
58

0.1-0.2
1,910
1,243

0.2-0.3
11,006
7,876

0.3-0.4
23,126
19,351

Similarity Score
0.4-0.5 0.5-0.6
33,755 34,888
29,053 30,149

0.6-0.7
28,08324,3820.7-0.8
17,40014,946

0.8-0.9
7,693
6,434

0.9-1.0
4,347
3,037

Total
162,308
136,529

0.7-0.8
3
4
2
23
37
4
16
0
34
0
10
149

0.8-0.9
0
0
6
15
11
1
1
0
15
0
0
64

0.9-1.0
18
0
0
1
2
0
1
0
5
0
0
30

Table 2: Distribution of the sentences on similarity score

Category

Phenomenon
Identical
Sentential
Free translation
Type
Mistranslation
Noun
Verb
Lexical
ChoiceInterrogative
Other
Classifier
Syntax
Other
Alphabet
Orthography
NumberNumber of samples

0.1-0.2
0
6
3
0
0
0
1
0
1
1
0
12

0.2-0.3
0
26
8
30
32
2
17
2
20
0
6
78

Similarity Score
0.4-0.5 0.5-0.6 0.6-0.7
0
0
0
13
8
4
7
1
2
110
98
70
193
186
88
9
10
2
115
89
40
11
6
2
109
156
96
7
7
0
18
20
25
290
300
243

0.3-0.4
0
36
6
98
112
11
68
5
71
1
14
192

Table 3: Paraphrasing categories
2.1.

Analysis of Our Corpora

We compared the sentences using the perl module
String::Similarity (Lehman 2000). It returns asimilarity score based on the edit distance (the number ofcharacters that need to be deleted, added or substituted tochange one string into another), normalized to give a score
between 0 and 1 (Myers, 1986). Two completely different
strings have a score of zero, while two identical strings
have a score of one.
There were 136,529 sentences. Their distribution is
given in Table 2. Less than 2% of the sentences were
the same in both corpora. Most sentences were reasonably
similar (0.4–0.8). Very few sentences were identical (lessthan 4%), and even fewer were totally dissimilar (less than0.1%). There were some minor orthographic differences(described in Section 2.2.4.), but even when they were resolved only 8.3% of the sentences were identical.
2.2.

How similar they are: Kg and Kh

Here we present an analysis of how similar the two corpora are. First, we divided linguistic phenomena into four
categories: sentential type (how the sentence was translated), lexical choice (difference due to lexical choice), syntax (difference due to syntax) and orthography (difference
due to orthographic variation). The distribution is shown in
Table 3. A random sample was chosen for each of 9 similarity bands from 0.1–0.2 to 0.9–1.0. The sample size wasproportional to the number of sentences in each band.
2.2.1. Sentential TypeThe more identical sentences are found, then of course
the higher similarity score is. On the other hand, in the low
similarity score we found more free translations. For example, kmnpl o[qst r (trans.“Here you are”) in Kg is given as omu&vfx w omu y[{ z|~}  u`q? (trans. “Will it be okay with this?”) in Kh . At aglance, this is not a good translation, but when you consider
a certain situation, these two translations can be compati-

ble. Analyzing the sentences of similarity score 0.0-0.4, we
find many sentences that are expressed in a different way.
Take the following examples. Both of the sentences mean
“I do not have an appetite.”
(3)
(4)

%
$&%
c
 e>3   )  d  . 3 : 7j+     )  c  - /0 f2  . 4 : Ti >(from similarity 0.3-0.4)

If you want to translate the Korean sentence (3) into
other languages, there is a high chance of translating it
wrongly. It does not give you the real meaning when you
translate it word to word, or phrase to phrase. However, if
we can paraphrase (3) with a simpler expression like (4),then the paraphrased sentence will be easily and correctlytranslated. We can extract these kinds of paraphrases at low
similarity scores from 0.0 to 0.4. As Table 3 shows, theextracted number of free translations is relatively high inthese similarity bands.
Finally, there are a few cases where the two corpora
do not match each other because the Japanese and English
bilingual corpus contain a translation mismatch. However,overall, the number of mistranslations is negligible.
2.2.2. Lexical Choice
The main source of variation is differences in lexical
choice. We divided the phenomena into four classes. Then
we compare the two sentences and check whether any noun,verb or other lexical items such as adjective, adverb and re  plated phrase has been replaced. For example, “   
l m{ ¹  l¡ ¢ £B¦¨¤¥ §ª©¬«®¡­ ¯q±² ° ³µ ¶ ´ ·  u`¸q ” in Kg is compared with “  W º»
 ¢ £¼¤¥¦ ¶ £¼
½ ¿ ®pl ÀÁ  u`¸q ” in Kh , meaning “A single roomm{  ¡ ¾;
with a bath, please.” Comparing the two sentences, we   yoksil “bath” is used instead of  .
 º ¹ yokco
found that  p



ª
§
¬
©

«
­
¶
·
“bath”, which is noun, and ®¡¯q±² ° ³µ´  u ¸q “would like3
Lit. trans. “There is no thought of rice.” It is translated fromthe English sentence “I do not have an appetite.”
4
The sentence was translated from the Japanese sentence “ÂÃO>ÄBÅ .”, meaning “There is no appetite.”1792½ ¿ ®Æl ÀÁ  u ¸q “do me a favor” orto reserve” is used instead of ¾;“please”, which is considered verb alternation. However,the use of different nouns and verbs accounts for about 20%of the variation throughout the corpora.

2.2.3. Syntax
The next most common category is differences in syntax. This includes the deletion or addition of case markers,
scrambling, and changes in voice (active/passive). As we
can see in Table 3, these occur throughout the entire rangeof similarities. The higher the similarity is, the more syntactic changes are found. At the same time, we examine
how often numeral classifiers are used in our corpora since
it is necessary for both Korean and Japanese. We foundthat about 1% of the numeral classifiers are floated. Amongthem, some classifiers are used in one corpus but omitted in
the other. For example, in (5) coffee is counted (a cup ofcoffee), but tea isn’t (some tea). Both choices are valid.
(5)$&% ÈÇmsÉ&ËÍÊ Ì Î Ï (6)É&Õ¬Ô Öd×ÙØ ÈÚ`Ü Û) >Î Ð   !Ò+ Ñ Ó   ? 5 : d ÝÈß Þ  - /0 3à  ? 6 : ji

We found an interesting fact with respect to numeralclassifiers. The kinds of numeral classifiers are far more in
the corpus áRg , whereas the frequency of numeral classifiers are far more in the corpus á h , as shown in Table 4.
Numeral classifier
Type
Token

Kj
320
7,878

Ki
284
8,928

Table 4: Comparison of numeral classifiers in Kh & Kg
2.2.4. Orthography
There are two kinds of orthographic difference. One
arises due to differences in transliterating foreign words
such as place names (e.g. â u`ãqä ¡ å u pikadilli “Piccadilly”
vs â u`ãq&¸u å u pikadili), people’s names and so on. The otheris whether numbers are given using Hindu-Arabic numerals (1,2,3,. . . ) or spelled out in Korean ( «   , omu ,  æ l ....)7 . Therecan also be some minor variation in punctuation (especiallythe use of question marks, which are optional in Japanese),
although none appeared in our samples. After normalizing the differences in spelling out numbers and letters, thenumber of identical sentences more than doubled to 8.3% of
the total. These orthographic differences are not important,
but they are widespread. This highlights the importanceof dealing with orthographic variation in any empirically
based approach.
2.3. Linguistic phenomena between Kh and Kg
In this section we focus on four phenomena with quite
different distributions in Kh and Kg : honorifics, zero pronoun, Kango (Chinese words), and loan words. The details
are shown in Table 5 and discussed below.

2.3.1. Honorifics
Korean has more honorific speech levels than Japanese.
Consider (7) and (8).
(7)

(8)

,) + 0[   ?
ð òñ>pepper exist-honorific/polite“Do you have pepper?” (polite)) +  - /0 3à  ?
,.
ð>òRñ ô óõ
pepper-top exist-honorific/deferential“Do you have pepper?” (deferential)

deferential
polite-nida. -nikka?
-yo.
-yo?
Total
Kg
23,3169,970 33,617 25,481 93,227
Kh
33,351 34,872 21,922
3,082 92,384
Table 6: Difference in honorifics between Kh and Kg
We found that Kh tends to use more deferential honorifics, whereas Kg tends to use more polite honorifics
as shown in Table 6. This distinction is not made in either English or Japanese. Korean has six different speech
level honorifics whereas Japanese has two. In the sense thatJapanese has two lexicalised honorifics showing degree ofpoliteness, we can say that the source language affects thedegree or level of honorifics.
2.3.2. Zero Pronoun
We expect that there are some differences which comesfrom the source language before translation. In English
subjects are obligatory. One of the main characteristics oftranslated Korean sentences from English is that many specific pronouns are used. This is fully reflected in the factthat English strongly express subject/agent of sentences,whereas Korean and Japanese tend to eliminate its counterpart. Furthermore, both Korean and Japanese lack obligatory determiners, whereas English generally requires somedeterminer. Compare (9) and (10).
(9)

(10)

ø&Ëûù ü úý ^b þ " ÿ Ì  c ) e + 0 `2 Ì eÒ+ "3ö ÷.
my-gen friend house-loc stay plan-be VERB“I am planning to stay at my friend’s house.” (K j )
ø&Ë¡
ù üú
b þ  - /10 32  .
Ì e#+ "

friend house-loc stay-VERB“I stay at my friend’s house.” (K i )2.3.3. Kango
As Table 5 shows, Kh has a slightly stronger tendency
to use words containing Chinese characters. Since the ratio of words of Chinese origin in both Korean and Japanese
accounts for more than 50% (Paik and Bond:2001), we believe that if a shared Kango is used in Kh , then it will beeasier to translate into the same word corresponding to Korean.  l (  ) in (12) is a word of Chinese origin that is
also used in Japanese. On the other hand, the phrase of thesentence from Kg is translated into native Korean words
like (11).
(11)

    ô óõ
Ì  % bdÈ2 ?
÷
cosmetics exist place-top where-be(12)“Where are the cosmetics?”
 !   õ Ì  % bdÈ2 ) e + 0 dà  ?
cosmetics selling place-top where-be5

Lit. trans.“How about joining me for a cup of coffee?”
The sentence was translated from the Japanese sentence “ ÅBçèYé?HêBë;QLìÍíTîPQBV;ïEX ”, meaning “How about having (acup of) tea?”
7
In theory, numbers could also be written in Hanja, but this did
not occur in these two corpora.
6“Where is the cosmetics section?”

1793

Phenomenon
Honorific (K i )Honorific (K j )Zero (K i )Zero (K j )Kango (K i )Kango (K j )Loan word(K i )Loan word(K j )0.1-0.2
1
0
0
0
0
0
0
0

0.2-0.3
10
8
3
2
7
2
5
0

0.3-0.4
16
15
8
5
23
10
16
7

Similarity Score
0.4-0.5 0.5-0.6 0.6-0.7
82
101
76
20
32
31
23
28
22
7
15
8
39
54
22
25
35
19
14
15
7
11
9
11

0.7-0.8 0.8-0.9
32
11
9
2
13
4
9
1
19
4
12
0
6
0
3
0
Table 5: Linguistic phenomena causing differences between Kh and Kg

2.3.4. Loan word
By loan words we mean words in daily usage borrowed
from languages other than Chinese. We expected that moreloan words would be used in Kg because most loan words
come from English. However, according to Table 5, the result is the opposite. After examining the source corpora of
Kh , we found that many “Katakana words8 ,” which include
borrowed/foreign words, are used in Japanese sentences.
Almost all of the Katakana words are translated into foreign words in Korean. In contrast, all of the English words
are equally foreign, so for any single word, there is little
pressure for it to be translated into a loan word rather thaninto native Korean.
2.4. Discussion
As we have seen, the source language has a large effecton the translation. Accordingly, it is necessary to reconsider the quality of parallel corpora because they are widelyused in the NLP area. So far, we have taken it for granted
that existing parallel corpus is used when necessary, if there
is any. This is partly because that we do not have enoughparallel corpora throughout the world and partly because
many of the corpora have been created through English.
We expect that many more parallel corpora will be built
not only through English but also through many other languages from now on. Therefore, we need to put emphasison the quality or characteristics of the corpora according to
the application.
In particular, much research into machine translation
uses parallel or comparable corpora without considering
their characteristics. This will lead to different results. For
example, as we observed in this paper, the result of a translation trained on the Kg corpus will be less polite comparedto that of a translation trained on Kh , and it will also produce different results with respect to the linguistic phenomena such as zero pronouns, numeral classifers and so on.
As for the numeral classifier, it is interesting that Kg uses
more classifiers than Kh if we compare unique classifiers.
In many cases, English does not use classifers, whereas
Japanese has to use numeral classifiers for counting objects.
At a glance, this is the opposite to what the existing corpora
show. However, Kg and Kh closely reflect the effects of thesource language. When the numeral classifier is implicitly
expressed, it will be more freely translated. That is why Kg
has more types of numeral classifiers.
8

Japanese has two writing systems based on syllables: Hiragana and Katakana. Katakana is mainly used for writing loan
words and names/places that can’t be written in Kanji.

0.9-1.0
2
0
1
1
0
0
0
0

In addition, we can expect that the optimal translation
strategy may be different between language pairs. In particular, a system translating between Japanese and Korean
needs to put less effort into lexical and syntactic choice andmore into the use of honorifics. On the other hand, a systemgoing between English and Korean has a much harder task,
since it must consider lexical and syntactic choice and zeropronoun resolution in addition to the use of honorifics formachine and human translation.
Furthermore, a pair of corpora such as these is a usefulsource of data on variation within a single language and canreadily be exploited to learn paraphrasing rules. For example, as we have seen in 2.2.1., we can extract free translation
relatively easily and with less cost.

3.

Conclusions

We investigated two variants of a Korean translation
corpus, one based on translations from Japanese and theother from English. We have shown that the source language text has a large influence on the target text in almost
all areas — lexical choice, syntactic structure, use of zeropronouns and honorifics, and even orthographic variation.
One surprising result is how different the corpora were,even after normalizing orthographic differences: fewer than
8.3% of sentences in the two variants were identical. Also,
we should emphasize that the characteristics and the effectof the source languages must be taken into considerationin constructing a pair of parallel corpora for the better machine translation and for the related application.

4. Acknowledgements
This research was supported in part by the Ministry of Public
Management, Home Affairs, Posts and Telecommunications. We
would like to thank Francis Bond of NTT for his useful advice anddiscussion.

5.

References

Lehmann, Marc, 2000. String::Similarity. Perl Module (cpan.
org). (v0.02).
Myers, Eugene, 1986. An O(ND) difference algorithm and its
variations. Algorithmica, 1(2):251–266.
Paik, Kyonghee and Francis Bond, 2001. Multilingual generation of numeral classifiers using a common ontology. In 19thInternational Conference on Computer Processing of Oriental
Languages: ICCPOL-2001. Seoul. 141–147.
Takezawa, Toshiyuki, Satoshi Shirai, and Yoshifumi Ooyama,
2001. Characteristics of colloquial expressions in a bilingual
travel conversation corpus. In ICCPOL-2001. Seoul. 384–389.

1794