Reliable Measures for Aligning Japanese-English News Articles andSentences
Masao Utiyama and Hitoshi Isahara
Communications Research Laboratory
3-5 Hikari-dai, Seika-cho, Souraku-gun, Kyoto 619-0289 Japanmutiyama@crl.go.jp and isahara@crl.go.jp

Abstract
We have aligned Japanese and English
news articles and sentences to make alarge parallel corpus. We first used amethod based on cross-language information retrieval (CLIR) to align the Japaneseand English articles and then used amethod based on dynamic programming(DP) matching to align the Japanese andEnglish sentences in these articles. However, the results included many incorrectalignments. To remove these, we propose two measures (scores) that evaluate
the validity of alignments. The measure
for article alignment uses similarities insentences aligned by DP matching andthat for sentence alignment uses similarities in articles aligned by CLIR. Theyenhance each other to improve the accuracy of alignment. Using these measures,
we have successfully constructed a largescale article and sentence alignment corpus available to the public.

1

Introduction

A large-scale Japanese-English parallel corpus is aninvaluable resource in the study of natural language
processing (NLP) such as machine translation andcross-language information retrieval (CLIR). It isalso valuable for language education. However, nosuch corpus has been available to the public.
We recently have obtained a noisy parallel corpus of Japanese and English newspapers consistingof issues published over more than a decade andhave tried to align their articles and sentences. We
first aligned the articles using a method based on
CLIR (Collier et al., 1998; Matsumoto and Tanaka,
2002) and then aligned the sentences in these articles
by using a method based on dynamic programming(DP) matching (Gale and Church, 1993; Utsuro etal., 1994). However, the results included many incorrect alignments due to noise in the corpus.
To remove these, we propose two measures(scores) that evaluate the validity of article and sentence alignments. Using these, we can selectively
extract valid alignments.
In this paper, we first discuss the basic statistics
on the Japanese and English newspapers. We next
explain methods and measures used for alignment.
We then evaluate the effectiveness of the proposed
measures. Finally, we show that our aligned corpus
has attracted people both inside and outside the NLP
community.

2 Newspapers Aligned
The Japanese and English newspapers used assource data were the Yomiuri Shimbun and the Daily
Yomiuri. They cover the period from September
1989 to December 2001. The number of Japanese
articles per year ranges from 100,000 to 350,000,
while English articles ranges from 4,000 to 13,000.
The total number of Japanese articles is about
2,000,000 and the total number of English articles isabout 110,000. The number of English articles represents less than 6 percent that of Japanese articles.
Therefore, we decided to search for the Japanese articles corresponding to each of the English articles.

The English articles as of mid-July 1996 have tags
indicating whether they are translated from Japanese
articles or not, though they don’t have explicit links
to the original Japanese articles. Consequently, weonly used the translated English articles for the article alignment. The number of English articles used
was 35,318, which is 68 percent of all of the articles. On the other hand, the English articles before
mid-July 1996 do not have such tags. So we used all
the articles for the period. The number of them was
59,086. We call the set of articles before mid-July
1996 “1989-1996” and call the set of articles after
mid-July 1996 “1996-2001.”
If an English article is a translation of a Japanese
article, then the publication date of the Japanese article will be near that of the English article. So wesearched for the original Japanese articles within 2days before and after the publication of each English
article, i.e., the corresponding article of an English
article was searched for from the Japanese articles of
5 days’ issues. The average number of English articles per day was 24 and that of Japanese articles per
5 days was 1,532 for 1989-1996. For 1996-2001, theaverage number of English articles was 18 and thatof Japanese articles was 2,885. As there are many
candidates for alignment with English articles, weneed a reliable measure to estimate the validity ofarticle alignments to search for appropriate Japanese
articles from these ambiguous matches.
Correct article alignment does not guarantee theexistence of one-to-one correspondence between
English and Japanese sentences in article alignment
because literal translations are exceptional. Original
Japanese articles may be restructured to conform to
the style of English newspapers, additional descriptions may be added to fill cultural gaps, and detailed
descriptions may be omitted. A typical example of a
restructured English and Japanese article pair is:
Part of an English article: he1i Two bullet holes were found at
the home of Kengo Tanaka, 65, president of Bungei Shunju, in Akabane, Tokyo, by his wife Kimiko, 64, at around 9 a.m. Monday.
h/e1i he2i Police suspect right-wing activists, who have mounted
criticism against articles about the Imperial family appearing in
the Shukan Bunshun, the publisher’s weekly magazine, were responsible for the shooting. h/e2i he3i Police received an anonymous phone call shortly after 1 a.m. Monday by a caller who
reported hearing gunfire near Tanaka’s residence. h/e3i he4i Police found nothing after investigating the report, but later found abullet in the Tanakas’ bedroom, where they were sleeping at thetime of the shooting. h/e4i

eastern wall of their two-story house at 4 Akabane Nishi, Kitaku, Tokyo.h/j1i hj2i As a result of an investigation, the officers of
the Akabane police station found two holes on the exterior wall of
the bedroom and a bullet in the bedroom.h/j2i hj3i After receiving an anonymous phone call shortly after 1 a.m. saying that twoor three gunshots were heard near Tanaka’s residence, police officers hurried to the scene for investigation, but no bullet holes were
found.h/j3i hj4i When gunshots were heard, Mr. and Mrs. Tanaka
were sleeping in the bedroom.h/j4i hj5i Since Shukan Bunshun, aweekly magazine published by Bungei Shunju, recently ran an article criticizing the Imperial family, Akabane police suspect rightwing activists who have mounted criticism against the recent article to be responsible for the shooting and have been investigating
the incident.h/j5iwhere there is a three-to-four correspondence between {e1, e3, e4} and {j1, j2, j3, j4}, together witha one-to-one correspondence between e2 and j5.
Such sentence matches are of particular interest
to researchers studying human translations and/or
stylistic differences between English and Japanesenewspapers. However, their usefulness as resources
for NLP such as machine translation is limited forthe time being. It is therefore important to extract
sentence alignments that are as literal as possible.
To achieve this, a reliable measure of the validity ofsentence alignments is necessary.

3 Basic Alignment Methods
We adopt a standard strategy to align articles andsentences. First, we use a method based on CLIR
to align Japanese and English articles (Collier etal., 1998; Matsumoto and Tanaka, 2002) and thena method based on DP matching to align Japaneseand English sentences (Gale and Church, 1993; Utsuro et al., 1994) in these articles. As each of thesemethods uses existing NLP techniques, we describe
them briefly focusing on basic similarity measures,
which we will compare with our proposed measuresin Section 5.
3.1

Translation of words
We first convert each of the Japanese articles into
a set of English words. We use ChaSen1 to segment each of the Japanese articles into words. We
next extract content words, which are then translated
into English words by looking them up in the EDR
Japanese-English bilingual dictionary,2 EDICT, and
ENAMDICT,3 which have about 230,000, 100,000,
1Part of a literal translation of a Japanese article: hj1i At about
8:55 a.m. on the 29th, Kimiko Tanaka, 64, the wife of Bungei
Shunju’s president Kengo Tanaka, 65, found bullet holes on the

Article alignment

http://chasen.aist-nara.ac.jp/
http://www.iijnet.or.jp/edr/3
http://www.csse.monash.edu.au/˜jwb/edict.html
2and 180,000 entries, respectively. We select two English words for each of the Japanese words using
simple heuristic rules based on the frequencies ofEnglish words.
Article retrieval
We use each of the English articles as a query andsearch for the Japanese article that is most similar
to the query article. The similarity between an English article and a (word-based English translationof) Japanese article is measured by BM25 (Robertson and Walker, 1994). BM25 and its variants havebeen proven to be quite efficient in information retrieval. Readers are referred to papers by the Text
REtrieval Conference (TREC)4 , for example.
The definition of BM25 is:BM25(J, E) =Xw

(1) (k1 + 1)tf (k3 + 1)qtf

T ∈E

K + tfk3 + qtf

where
J is the set of translated English words of aJapanese article and E is the set of words of an
English article. The words are stemmed and stop
words are removed.
T is a word contained in E.
−n+0.5)w(1) is the weight of T , w(1) = log (N(n+0.5).

N is the number of Japanese articles to be searched.
n is the number of articles containing T .
dl
K is k1 ((1 − b) + b avdl). k1 , b and k3 are parameters set to 1, 1, and 1000, respectively. dl isthe document length of J and avdl is the average
document length in words.

tf is the frequency of occurrence of T in J. qtf isthe frequency of T in E.

To summarize, we first translate each of theJapanese articles into a set of English words. We
then use each of the English articles as a query andsearch for the most similar Japanese article in termsof BM25 and assume that it corresponds to the English article.
3.2 Sentence alignment
The sentences5 in the aligned Japanese and English
articles are aligned by a method based on DP matching (Gale and Church, 1993; Utsuro et al., 1994).
4

http://trec.nist.gov/
We split the Japanese articles into sentences by using simple heuristics and split the English articles into sentences byusing MXTERMINATOR (Reynar and Ratnaparkhi, 1997).
5

We allow 1-to-n or n-to-1 (1 ≤ n ≤ 6) alignments
when aligning the sentences. Readers are referred
to Utsuro et al. (1994) for a concise description of
the algorithm. Here, we only discuss the similarities
between Japanese and English sentences for alignment. Let Ji and Ei be the words of Japanese andEnglish sentences for i-th alignment. The similarity6 between Ji and Ei is:
SIM(Ji , Ei ) =wherel(X) =

co(Ji × Ei ) + 1l(Ji ) + l(Ei ) − 2co(Ji × Ei ) + 2

P
x∈X

f (x)

f (x) is the frequency of x in the sentences.
co(Ji × Ei ) =P(j,e)∈Ji ×Eimin(f (j), f (e))

Ji × Ei = {(j, e)|j ∈ Ji , e ∈ Ei } and Ji × Ei isa one-to-one correspondence between Japanese andEnglish words.

Ji and Ei are obtained as follows. We use ChaSen tomorphologically analyze the Japanese sentences andextract content words, which consists of Ji . We use
Brill’s tagger (Brill, 1992) to POS-tag the English
sentences, extract content words, and use WordNet’s library7 to obtain lemmas of the words, which
consists of Ei . We use simple heuristics to obtain
Ji × Ei , i.e., a one-to-one correspondence between
the words in Ji and Ei , by looking up JapaneseEnglish and English-Japanese dictionaries made up
by combining entries in the EDR Japanese-English
bilingual dictionary and the EDR English-Japanesebilingual dictionary. Each of the constructed dictionaries has over 300,000 entries.
We evaluated the implemented program against a
corpus consisting of manually aligned Japanese andEnglish sentences. The source texts were Japanesewhite papers (JEIDA, 2000). The style of translation
was generally literal reflecting the nature of government documents. We used 12 pairs of texts for evaluation. The average number of Japanese sentences
per text was 413 and that of English sentences was
495.
The recall, R, and precision, P , of the program
against this corpus were R = 0.982 and P = 0.986,
respectively, where
6
SIM(Ji , Ei ) is different from the similarity function usedin Utsuro et al. (1994). We use SIM because it performed wellin a preliminary experiment.
7
http://www.cogsci.princeton.edu/˜wn/

R=

number of correctly aligned sentence pairs
total number of sentence pairs aligned in corpus

P =number of correctly aligned sentence pairs
total number of sentence pairs proposed by program

The number of pairs in a one-to-n alignment is n.
For example, if sentences {J1 } and {E1 , E2 , E3 }are aligned, then three pairs hJ1 , E1 i, hJ1 , E2 i, andhJ1 , E3 i are obtained.
This recall and precision are quite good considering the relatively large differences in the language
structures between Japanese and English.

4

Reliable Measures

We use BM25 and SIM to evaluate the similarity
in articles and sentences, respectively. These measures, however, cannot be used to reliably discriminate between correct and incorrect alignments aswill be discussed in Section 5. This motivated us todevise more reliable measures based on basic similarities.
BM25 measures the similarity between two bagsof words. It is not sensitive to differences in theorder of sentences between two articles. To remedy this, we define a measure that uses the similarities in sentence alignments in the article alignment.
We define AVSIM(J, E) as the similarity between
Japanese article, J, and English article, E:
Pm

AVSIM(J, E) =k=1 SIM(Jk , Ek )mwhere (J1 , E1 ), (J2 , E2 ), . . . (Jm , Em ) are the sentence alignments obtained by the method describedin Section 3.2. The sentence alignments in a correctly aligned article alignment should have moresimilarity than the ones in an incorrectly aligned article alignment. Consequently, article alignmentswith high AVSIM are likely to be correct.
Our sentence alignment program aligns sentences
accurately if the English sentences are literal translations of the Japanese as discussed in Section 3.2.
However, the relation between English news sentences and Japanese news sentences are not literaltranslations. Thus, the results for sentence alignments include many incorrect alignments. To discriminate between correct and incorrect alignments,

we take advantage of the similarity in article alignments containing sentence alignments so that thesentence alignments in a similar article alignment
will have a high value. We define
SntScore(Ji , Ei ) = AVSIM(J, E) × SIM(Ji , Ei )SntScore(Ji , Ei ) is the similarity in the i-th alignment, (Ji , Ei ), in article alignment J and E. When
we compare the validity of two sentence alignmentsin the same article alignment, the rank order of sentence alignments obtained by applying SntScore is
the same as that of SIM because they share a common AVSIM. However, when we compare the validity of two sentence alignments in different articlealignments, SntScore prefers the sentence alignment
with the more similar (high AVSIM) article alignment even if their SIM has the same value, while
SIM cannot discriminate between the validity of two
sentence alignments if their SIM has the same value.
Therefore, SntScore is more appropriate than SIM if
we want to compare sentence alignments in different
article alignments, because, in general, a sentencealignment in a reliable article alignment is more reliable than one in an unreliable article alignment.
The next section compares the effectiveness of
AVSIM to that of BM25, and that of SntScore tothat of SIM.

5 Evaluation of Alignment
Here, we discuss the results of evaluating article andsentence alignments.
5.1

Evaluation of article alignment

We first estimate the precision of article alignments
by using randomly sampled alignments. Next, wesort them in descending order of BM25 and AVSIM
to see whether these measures can be used to provide
correct alignments with a high ranking. Finally, weshow that the absolute values of AVSIM correspondwell with human judgment.
Randomly sampled article alignments
Each English article was aligned with a Japanese
article with the highest BM25. We sampled 100 article alignments from each of 1996-2001 and 19891996. We then classified the samples into four categories: “A”, “B”, “C”, and “D”. “A” means that therewas more than 50% to 60% overlap in the content ofarticles. “B” means more than 20% to 30% and lessthan 50% to 60% overlap. “D” means that there was
no overlap at all. “C” means that alignment was notincluded in “A”,“B” or “D”. We regard alignmentsthat were judged to be A or B to be suitable for NLP
because of their relatively large overlap.

Sorted alignments: AVSIM vs. BM25
We sorted the same alignments in Table 1 in decreasing order of AVSIM and BM25. Alignments
judged to be A or B were regarded as correct. The
number, N, of correct alignments and precision, P,
up to each rank are shown in Table 2.
rank

type
A
B
C
D

lower
0.49
0.06
0.03
0.13

1996-2001
ratio upper
0.59
0.69
0.12
0.18
0.08
0.13
0.21
0.29

lower
0.20
0.08
0.03
0.38

1989-1996
ratio upper
0.29
0.38
0.15
0.22
0.08
0.13
0.48
0.58

Table 1: Ratio of article alignments
The results of evaluations are in Table 1.8 Here,“ratio” means the ratio of the number of articles
judged to correspond to the respective category
against the total number of articles. For example,
0.59 in line “A” of 1996-2001 means that 59 out of
100 samples were evaluated as A. “Lower” and “upper” mean the lower and upper bounds of the 95%
confidence interval for ratio.
The table shows that the precision (= sum of theratios of A and B) for 1996-2001 was higher thanthat for 1989-1996. They were 0.71 for 1996-2001
and 0.44 for 1989-1996. This is because the English articles from 1996-2001 were translations ofJapanese articles, while those from 1989-1996 were
not necessarily translations as explained in Section
2. Although the precision for 1996-2001 was higher
than that for 1989-1996, it is still too low to use themas NLP resources. In other words, the article alignments included many incorrect alignments.
We want to extract alignments which will be evaluated as A or B from these noisy alignments. To
do this, we have to sort all alignments according tosome measures that determine their validity and extract highly ranked ones. To achieve this, AVSIM is
more reliable than BM25 as is explained below.
8

The evaluations were done by the authors. We double
checked the sample articles from 1996-2001. Our second
checks are presented in Table 1. The ratio of categories in thefirst check were A=0.62, B=0.09, C=0.09, and D=0.20. Comparing these figures with those in Table 1, we concluded thatfirst and second evaluations were consistent.

5
10
20
30
40
50
60
70
80
90
100

1996-2001
AVSIM
BM25
N
P
N
P
5
1.00
5
1.00
10 1.00
80.80
20 1.00 16 0.80
30 1.00 25 0.83
40 1.00 34 0.85
50 1.00 39 0.78
60 1.00 47 0.78
66 0.94 55 0.79
70 0.88 62 0.78
71 0.79 68 0.76
71 0.71 71 0.71

1989-1996
AVSIM
BM25
N
P
N
P
5
1.00
2
0.40
10 1.00
40.40
19 0.95
90.45
28 0.93 16 0.53
34 0.85 24 0.60
37 0.74 28 0.56
42 0.70 30 0.50
42 0.60 35 0.50
43 0.54 38 0.47
43 0.48 40 0.44
44 0.44 44 0.44

Table 2: Rank vs. precision
From the table, we can conclude that AVSIM
ranks correct alignments higher than BM25. Its
greater accuracy indicates that it is important totake similarities in sentence alignments into accountwhen estimating the validity of article alignments.
AVSIM and human judgment
Table 2 shows that AVSIM is reliable in ranking
correct and incorrect alignments. This section reveals that not only rank order but also absolute values of AVSIM are reliable for discriminating between correct and incorrect alignments. That is,they correspond well with human evaluations. Thismeans that a threshold value is set for each of 19962001 and 1989-1996 so that valid alignments can beextracted by selecting alignments whose AVSIM islarger than the threshold.
We used the same data in Table 1 to calculate
statistics on AVSIM. They are shown in Tables 3
and 4 for 1996-2001 and 1989-1996, respectively.
type
A
B
C
D

N
59
12
8
21

lower
0.176
0.122
0.077
0.065

av.
0.193
0.151
0.094
0.075

upper
0.209
0.179
0.110
0.086

th.
0.168
0.111
0.085

sig.
**
**
*

Table 3: Statistics on AVSIM (1996-2001)
In these tables, “N” means the number of alignments against the corresponding human judgment.

type
A
B
C
D

N
29
15
8
48

lower
0.153
0.113
0.092
0.076

av.
0.175
0.141
0.123
0.082

upper
0.197
0.169
0.154
0.088

th.
0.157
0.131
0.097

sig.
*
**

Table 4: Statistics on AVSIM (1989-1996)

In summary, AVSIM is more reliable than BM25
and corresponds well with human judgment. By using thresholds, we can extract about 47,000 articlealignments which are estimated to be A or B evaluations.
5.2“Av.” means the average value of AVSIM. “Lower”
and “upper” mean the lower and upper bounds ofthe 95% confidence interval for the average. “Th.”means the threshold for AVSIM that can be used todiscriminate between the alignments estimated to be
the corresponding evaluations. For example, in Table 3, evaluations A and B are separated by 0.168.
These thresholds were identified through linear discriminant analysis. The asterisks “**” and “*” in the“sig.” column mean that the difference in averages
for AVSIM is statistically significant at 1% and 5%
based on a one-sided Welch test.
In these tables, except for the differences in the
averages for B and C in Table 4, all differencesin averages are statistically significant. This indicates that AVSIM can discriminate between differences in judgment. In other words, the AVSIM values correspond well with human judgment. We thentried to determine why B and C in Table 4 were notseparated by inspecting the article alignments andfound that alignments evaluated as C in Table 4 had
relatively large overlaps compared with alignments
judged as C in Table 3. It was more difficult to distinguish B or C in Table 4 than in Table 3.
We next classified all article alignments in 19962001 and 1989-1996 based on the thresholds in Tables 3 and 4. The numbers of alignments are in Table
5. It shows that the number of alignments estimatedto be A or B was 46738 (= 31495 + 15243). We
regard about 47,000 article alignments to be sufficiently large to be useful as a resource for NLP suchas bilingual lexicon acquisition and for language education.

A
B
C
D
total

1996-2001
15491
9244
4944
5639
35318

1989-1996
16004
5999
10258
26825
59086

total
31495
15243
15202
32464
94404

Table 5: Number of articles per evaluation

Evaluation of sentence alignment

Sentence alignments in article alignments have
many errors even if they have been obtained fromcorrect article alignments due to free translation as
discussed in Section 2. To extract only correctalignments, we sorted whole sentence alignments
in whole article alignments in decreasing order of
SntScore and selected only the higher ranked sentence alignments so that the selected alignments
would be sufficiently precise to be useful as NLP
resources.
The number of whole sentence alignments was
about 1,300,000. The most important category for
sentence alignment is one-to-one. Thus, we want
to discard as many errors in this category as possible. In the first step, we classified whole oneto-one alignments into two classes: the first consisted of alignments whose Japanese and English
sentences ended with periods, question marks, exclamation marks, or other readily identifiable characteristics. We call this class “one-to-one”. The
second class consisted of the one-to-one alignments
not belonging to the first class. The alignmentsin this class, together with the whole one-to-nalignments, are called “one-to-many”. One-to-one
had about 640,000 alignments and one-to-many had
about 660,000 alignments.
We first evaluated the precision of one-to-onealignments by sorting them in decreasing order of
SntScore. We randomly extracted 100 samples fromeach of 10 blocks ranked at the top-300,000 alignments. (A block had 30,000 alignments.) We classified these 1000 samples into two classes: The
first was “match” (A), the second was “not match”(D). We judged a sample as “A” if the Japanese andEnglish sentences of the sample shared a common
event (approximately a clause). “D” consisted of thesamples not belonging to “A”. The results of evaluation are in Table 6.9
9
Evaluations were done by the authors. We double checked
all samples. In the 100 samples, there were a maximum of twoor three where the first and second evaluations were different.

range130001 60001 90001 120001 150001 180001 210001 240001 270001 -# of A’s
100
99
99
97
96
92
82
74
47
30

# of D’s
0
1
1
3
4
8
18
26
53
70

Comparison with SIM

Table 6: One-to-one: Rank vs. judgment
This table shows that the number of A’s decreases
rapidly as the rank increases. This means that
SntScore ranks appropriate one-to-one alignments
highly. The table indicates that the top-150,000 oneto-one alignments are sufficiently reliable.10 The ratio of A’s in these alignments was 0.982.
We then evaluated precision for one-to-manyalignments by sorting them in decreasing order of
SntScore. We classified one-to-many into three categories: “1-90000”, “90001-180000”, and “180001270000”, each of which was covered by the range of
SntScore of one-to-one that was presented in Table
6. We randomly sampled 100 one-to-many alignments from these categories and judged them to be A
or D (see Table 7). Table 7 indicates that the 38,090
alignments in the range from “1-90000” are sufficiently reliable.
range
190001 180001 -# of one-to-many
38090
59228
71711

# of A’s
98
8761# of D’s
2
13
39

Table 7: One-to-many: Rank vs. judgment
Tables 6 and 7 show that we can extract validalignments by sorting alignments according to
SntScore and by selecting only higher ranked sentence alignments.
Overall, evaluations between the first and second check wereconsistent.
10
The notion of “appropriate (correct) sentence alignment”
depends on applications. Machine translation, for example,
may require more precise (literal) alignment. To get literalalignments beyond a sharing of a common event, we will selecta set of alignments from the top of the sorted alignments thatsatisfies the required literalness. This is because, in general,higher ranked alignments are more literal translations, because
those alignments tend to have many one-to-one correspondingwords and to be contained in highly similar article alignments.

We compared SntScore with SIM and found that
SntScore is more reliable than SIM in discriminating
between correct and incorrect alignments.
We first sorted the one-to-one alignments in decreasing order of SIM and randomly sampled 100alignments from the top-150,000 alignments. We
classified the samples into A or D. The number of
A’s was 93, and that of D’s was 7. The precision was
0.93. However, in Table 6, the number of A’s was
491 and D’s was 9, for the 500 samples extracted
from the top-150,000 alignments. The precision was
0.982. Thus, the precision of SntScore was higher
than that of SIM and this difference is statisticallysignificant at 1% based on a one-sided proportionaltest.
We then sorted the one-to-many alignments by
SIM and sampled 100 alignments from the top
38,090 and judged them. There were 89 A’s and
11 D’s. The precision was 0.89. However, in Table 7, there were 98 A’s and 2 D’s for samples fromthe top 38,090 alignments. The precision was 0.98.
This difference is also significant at 1% based on a
one-sided proportional test.
Thus, SntScore is more reliable than SIM. Thishigh precision in SntScore indicates that it is important to take the similarities of article alignments
into account when estimating the validity of sentence alignments.

6 Related Work
Much work has been done on article alignment. Collier et al. (1998) compared the use of machine translation (MT) with the use of bilingual dictionary term
lookup (DTL) for news article alignment in Japaneseand English. They revealed that DTL is superior to
MT at high-recall levels. That is, if we want to obtain many article alignments, then DTL is more appropriate than MT. In a preliminary experiment, we
also compared MT and DTL for the data in Table
1 and found that DTL was superior to MT.11 These11
We translated the English articles into Japanese with an MT
system. We then used the translated English articles as queries
and searched the database consisting of Japanese articles. The
direction of translation was opposite to the one described in
Section 3.1. Therefore this comparison is not as objective as
it could be. However, it gives us some idea into a comparisonof MT and DTL.

experimental results indicate that DTL is more appropriate than MT in article alignment.
Matsumoto and Tanaka (2002) attempted to alignJapanese and English news articles in the Nikkei Industrial Daily. Their method achieved a 97% precision in aligning articles, which is quite high. They
also applied their method to NHK broadcast news.
However, they obtained a lower precision of 69.8%
for the NHK corpus. Thus, the precision of their
method depends on the corpora. Therefore, it is notclear whether their method would have achieved ahigh accuracy in the Yomiuri corpus treated in thispaper.
There are two significant differences between ourwork and previous works.
(1) We have proposed AVSIM, which uses similarities in sentences aligned by DP matching, asa reliable measure for article alignment. Previous
works, on the other hand, have used measures basedon bag-of-words.
(2) A more important difference is that we haveactually obtained not only article alignments but also
sentence alignments on a large scale. In addition tothat, we are distributing the alignment data for research and educational purposes. This is the firstattempt at a Japanese-English bilingual corpus.

in this Japanese-English alignment data.

7

Nigel Collier, Hideki Hirakawa, and Akira Kumano. 1998. Machine translation vs. dictionary term translation – a comparison for English-Japanese news article alignment. In
COLING-ACL’98, pages 263–267.

Availability

As of late-October 2002, we have been distributing
the alignment data discussed in this paper for research and educational purposes.12 All the information on the article and sentence alignments are numerically encoded so that users who have the Yomiuri data can recover the results of alignments. The
data also contains the top-150,000 one-to-one sentence alignments and the top-30,000 one-to-many
sentence alignments as raw sentences. The Yomiuri
Shimbun generously allowed us to distribute themfor research and educational purposes.
We have sent over 30 data sets to organizations
on their request. About half of these were NLPrelated. The other half were linguistics-related. A
few requests were from high-school and junior-highschool teachers of English. A psycho-linguist wasalso included. It is obvious that people from both inside and outside the NLP community are interested

8 Conclusion
We have proposed two measures for extracting valid
article and sentence alignments. The measure for article alignment uses similarities in sentences aligned
by DP matching and that for sentence alignment
uses similarities in articles aligned by CLIR. Theyenhance each other and allow valid article and sentence alignments to be reliably extracted from an extremely noisy Japanese-English parallel corpus.
We are distributing the alignment data discussed
in this paper so that it can be used for research andeducational purposes. It has attracted the attention ofpeople both inside and outside the NLP community.
We have applied our measures to a Japanese andEnglish bilingual corpus and these are language independent. It is therefore reasonable to expect thatthey can be applied to any language pair and still retain good performance, particularly since their effectiveness has been demonstrated in such a disparate
language pair as Japanese and English.

References
Eric Brill. 1992. A simple rule-based part of speech tagger. In
ANLP-92, pages 152–155.

William A. Gale and Kenneth W. Church. 1993. A programfor aligning sentences in bilingual corpora. Computational
Linguistics, 19(1):75–102.
Japan Electronic Industry Development Association JEIDA.
2000. Sizen Gengo Syori-ni Kan-suru Tyousa Houkoku-syo(Report on natural language processing systems).
Kenji Matsumoto and Hideki Tanaka. 2002. Automatic alignment of Japanese and English newspaper articles using an
MT system and a bilingual company name dictionary. In
LREC-2002, pages 480–484.
Jeffrey C. Reynar and Adwait Ratnaparkhi. 1997. A maximum entropy approach to identifying sentence boundaries.
In ANLP-97.
S. E. Robertson and S. Walker. 1994. Some simple effective approximations to the 2-Poisson model for probabilistic
weighted retrieval. In SIGIR’94, pages 232–241.

Takehito Utsuro, Hiroshi Ikeda, Masaya Yamane, Yuji Matsumoto, and Makoto Nagao. 1994. Bilingual text matching using bilingual dictionary and statistics. In COLING’94,
12
http://www.crl.go.jp/jt/a132/members/mutiyama/jea/index.html pages 1076–1082.