BUILDING A FREE, GENERAL-DOMAIN PARAPHRASE DATABASE FOR JAPANESE
Masahiro Mizukami, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura
Nara Institute of Science and Technology
8916-5 Takayama-cho, Ikoma-shi, Nara, Japanmasahiro-mi@is.naist.jp

ABSTRACTPrevious works have used parallel corpora and alignment
techniques from phrase-based statistical machine translation
to extract and generate paraphrases. In Japanese, paraphrases
for a number of paraphrase categories or domains have been
extracted by this method. However, most of these resources
focus on a particular phenomenon in Japanese, and there arestill no Japanese paraphrase resources that cover all varietiesof phrases from several domains, and are freely available.
In addition, because Japanese and English vary in grammar
and word ordering, we perform syntax-based preprocessing
to reduce this mismatch and extract paraphrases similar in
quality to those extracted using more similar language pairs.
The data used in creating the Japanese paraphrases is either inthe public domain, or available under the Creative Commonslicense, and spans a variety of genres for wide coverage.

⁆⁚⁛⁥⁛⁥⁓⁦⁗⁥⁦†‵⁗⁕⁛⁗⁥⁦⁧⁠⁗«⁢⁤⁗⁧⁨⁗†Fig. 1. Example of alignment for a language pair with similar
word order and grammar (e.g., English-French).
‵⁓⁞⁞ᅶ↝⁫ӸЭ⁠⁓ ⁗‚ ⇁‛†Ԡ ⇂↖ ⅛Fig. 2. Example of an alignment in English-Japanese.

Index Terms— Paraphrasing, Free Data, General-Domain
1. INTRODUCTION
Paraphrases are alternative ways of conveying the same meaning, and are useful in a number of NLP applications such asmachine translation and question answering [1, 2]. In thispaper, we concern ourselves with building paraphrase resources for Japanese. In Japanese, methods and resources
have been proposed for paraphrasing for a number of categories of paraphrases or domains [3, 4, 5, 6]. However,most of these resources focus on a particular phenomenon
in Japanese, and there are still no broad-coverage and freely
available resources.
In previous research on paraphrasing, methods that use
bilingual corpora have proven successful [7, 8]. In thesemethods, paraphrases for one language (e.g., English) areacquired by treating another language (e.g., French) as anintermediate meaning representation, as described more completely in Section 3. In this paper, we describe a paraphrasing
resource that we constructed for Japanese using a similar
method, with Japanese as our target language and English
serving as the intermediate meaning representation. In contrast, most previous work has focused on using bilingual

129

corpora for language pairs such as English-French, EnglishSpanish and other Germanic and Romance languages inwhich the word order and grammar are similar, as shown in
the example in Figure 1.
In contrast, there is a large divergence in both the wordorder, and the grammatical structure between Japanese andEnglish, as shown in Figure 2. We describe in Section 4 how
we use a syntactic preprocessing method, Head Finalization[9], to help compensate for this difference.
As a target for this method, we collect Japanese-English
bilingual data that is either in the public domain, or available
under the Creative Commons license, as described in Section
5 and use it to create a broad-coverage and freely available
Japanese paraphrase database.1 Finally, we perform an analysis of the database in Section 6, and find that it achievesa standard of accuracy similar to that of previously reported
paraphrasing resources.
2. EXTRACTING PARAPHRASES
We extract paraphrases using Bannard and Callison-Burch’s
bilingual pivoting method [7]. This method is a general1 http://ahclab.naist.jp/resource/jppdb/

ᚪ↞Ҿ૨↚ܱࣙ⁦⁤⁓⁠⁥⁞⁓⁦⁛⁡⁠⁕⁞⁡⁥⁗⁞⁫⁘⁡⁞⁞⁡⁩⁥⁦⁚ ⁗⁡⁤⁛⁙⁛⁠⁓⁞⁦⁤⁓⁠⁥⁞⁓⁦⁛⁡⁠⁛⁥⁖⁛⁘⁘⁛⁕⁧⁞⁦⁘⁡ ⁤⁗※ᅶ↞⁓⁙⁛⁦⁓⁦⁗⁘⁡⁤⁢⁗⁤ ⁓⁠⁗⁠⁦⁤⁗⁨⁗⁞⁤⁫

൨ʁ ↝↟ⅺ ᬳⅽ⇁ Ⴘਦↆ ↕

৆ⅵFig. 4. Example of alignment in standard English-Japanese.
ᎇᚪ↞ᅶ

↚

↞

ᩊↆⅳ

Fig. 3. Phrase-based paraphrases are extracted via bilingualpivoting.
purpose paraphrase extraction method, with the intuition thattwo English strings e1 and e2 that translate to the same foreign
string f can often be assumed to have the same meaning.
In this work, instead of English, we extract paraphrases
for Japanese, over Japanese-English bilingual parallel corpora. In Japanese-English, we can thus pivot over e and extract ⟨j1 , j2 ⟩ as a pair of paraphrases, as illustrated in Figure 3.
We estimate the conditional paraphrase probability P (j2 |j1 )by marginalizing over all shared English translations e:
P (j2 |j1 ) =∑P (j2 |e)P (e|j1 )(1)

eTo calculate these pivoted pairs and probabilities, we needto calculate the conditional probabilities P (j2 |e) and P (e|j1 ).
This is done by first extracting phrase pairs from a JapaneseEnglish corpus according to the standard phrase extraction
algorithm of phrase-based statistical machine translation [10].
For example, translation probability P (e|j) is calculated according to maximum likelihood estimation based on thecounts of each phrase pair c(e, j) and its constituent phrases:c(e, j)P (e|j) = ∑e c(e, j)(2)This method has been shown to accurately extract a diverse set of paraphrases in past research[7].
3. SYNTACTIC PREPROCESSING※⁨⁓• ⁢⁗⁤ ⁓⁠⁗⁠⁦ᅶ↞൨ʁ ↝⁤⁗⁨⁗⁞⁤⁫⁘⁡⁤⁓⁙⁛⁦⁓⁦⁗↟ⅺ ᬳⅽ⇁ Ⴘਦↆ ↕

৆ⅵFig. 5. Example of alignment in head-finalized EnglishJapanese.

sentence to a syntactic structure similar to Japanese before
running alignment and phrase extraction.
In English-Japanese translation, one of the most serious
problems is the difference of grammar and word ordering between the two languages. Isozaki et al. proposed the HF preprocessing method, which helps reduce this syntactic divergence and increase the accuracy of statistical machine translation results [9].
The main element of the method lies in reordering English into a similar order to Japanese. This is done by using
a syntactic parser to parse the English sentence, then moving
the head of all phrases to the end of the phrase, transforming
English into head-final order, similar to Japanese. In addition,
the HF algorithm considers the fact that Japanese contains explicit case markers after the subject and object which are notpresent in English, and inserts pseudo-words that correspondto these particles into the head-finalized English. As thesewill be aligned into Japanese particles, it can be expected thatperforming this processing will also improve the accuracy ofacquiring paraphrases for these particles. We show examplesof alignment in standard English-Japanese and head-finalized
English-Japanese in Figure 4 and 5.
4. PPDB : JAPANESEIn order to use the previously described method, it is necessary to acquire phrase alignments in parallel corpus aspivots between English phrases. In general, these automaticalignments are produced in an unsupervised manner with theGIZA++ toolkit [11]. However, for languages with greatly
different syntax and word order, standard alignment with
GIZA++ has worse performance in comparison to languages
with more similar syntax and word order.
In this paper, we help ameliorate this problem using recent syntactic preprocessing approaches to statistical machinetranslation. Specifically, we use the Head Finalization (HF;[9]) syntactic preprocessing method to change the English

130

Based on this data, we extracted Japanese paraphrases according to the proposed method.We extract alignments froma 1.9M sentence Japanese-English parallel corpus, the detailsof which are shown in Tables 1 and 2. For alignment, we use
GIZA++, with the English side being pre-processed with HF
as mentioned in the previous section. For tokenization, weuse the Stanford Parser2 [17] for English and KyTea3 [18] forJapanese.
2 http://nlp.stanford.edu/software/lex-parser.
shtml
3 http://www.phontron.com/kytea/

Tanaka [12]AOZORA [13]Common Crawl [14]WWWJDIC [15]Kyoto Wiki [16]
Total

Seed
メンバー
member
魏志 倭人 伝 にin the Gishi-wajin-den
論争
argue
突如
suddenlyでbyTable 1. The details of corpus
Sentences Words (en) Words (ja)150k1.4M
2.1M
108k
1.6M
2.5M
821k
13.8M
22.0M
373k
866k
373k
440k
11.5M
11.8M
1.9M
29.2M
38.3M

Licenses
Public Domain
Public Domain
Public Domain
CC BY-SA 3.04
CC BY-SA 3.04
CC BY-SA 3.04

Table 3. Examples of paraphrases with their rough English gloss
Paraphrases
メンバ, 一族, 一員, 員, 会員, 加盟, 会員のmember, family, a member (of), member, membership, member (join), member’s
魏志 倭人 伝 の 記述 に, 魏志 倭人 に, 魏志 倭人 伝 に はAccording to a description in the Gishi-wajin-den, in the Gishi-wajin, in Gishi-wajin-denの 論争, 争議, 紛争, 争い, 討論, 議論’s dispute, dispute, conflict, controversy, discussion, argue
とつぜん, 急 に, 不意 に, 突然
all of a sudden, hastily, abruptly, suddenに より, に よって, に よる, に, のbecause of, depending on, according to, to, ’sϯϬTable 2. The details of the phrase table
Phrases
67.1M
Alignment
GIZA++ [11]Tokenization (en) Stanford Parser [17]Tokenization (ja)Kytea [18]Max phrase length
7 words&ƌĞƋƵĞŶĐǇ΀й΁ϮϱIn paraphrasing, we chose paraphrases where the conditional probability of the target is less than 1% to reduce thenumber of extracted paraphrases with low probability. The
total number of extracted Japanese paraphrases were 10.5Mpairs. We analyze and evaluate the paraphrases, the detailed
results of which are below.

ϮϬϭϱ
ϭϬ
ϱ
Ϭ
ϭ

Ϯ

ϯ
ϰ
ϱ
WŚƌĂƐĞůĞŶŐƚŚ

ϲϳ

Fig. 6. Histogram of every phrase length in the acquired paraphrases.

4.1. Analysis
In the analysis, we evaluated mean phrase length of paraphrases, number of paraphrases provided for single words (asopposed to phrases), number of paraphrases of particles, anda histogram of the phrase lengths of paraphrases.
The mean phrase length of paraphrases is 3.42 words. The
number of paraphrases provided for words is 60.8k, and thenumber of paraphrases for phrases of length two or more is986k. We were able to acquire 134 paraphrases of Japaneseparticles in this method. We show examples of the acquired
paraphrases in Table 3.
We show a histogram of every phrase length of paraphrases in Figure 6.

131

4.2. Evaluation
We evaluate the paraphrases under the same conditions asprevious work [7], with the details as follows. We substituted
candidate paraphrases into 24 sentences which contained
the original phrase, creating a total of 85 unique sentences
through substitution. We had 3 native Japanese speakers produce judgments as to whether the new sentences preserved
the meaning of the original phrase and as to whether theyremained grammatical. Paraphrases that were judged to preserve both meaning and grammaticality were considered to bephrase data is larger than some previous works that createdJapanese paraphrases [3, 4]. And our proposed method wasable to acquire paraphrase data that is large, covers several
domains, and is high quality. We hope that our paraphrase
data will be able to contribute to future studies that requireparaphrases in the Japanese language.

Table 4. Evaluation of the acquired paraphrases
Correct rate [%]Meaning
84.7
Grammar55.3
Both45.8

correct, and examples which failed on either judgment wereconsidered to be incorrect. We show results of the judgement
in Table 4.
The accuracy of paraphrases is 45.8%, almost same withprevious work [7]. The inter-annotator agreement for thesejudgements was measured at κ = 0.60 [19], which is conventionally interpreted as “moderate” agreement [20].
Ignoring the constraint that the new sentences remain
grammatically correct, these paraphrases were judged to havethe correct meaning 84.7% of the time. This indicates that theparaphrases are semantically correct, but may vary in theirsyntactic categories or contexts. These tendencies are similar
to Bannard and Callison-Burch [7].
5. RELATED WORKS

7. REFERENCES[1] Chris Callison-Burch, Philipp Koehn, and Miles Osborne, “Improved statistical machine translation using
paraphrases,” in Proceedings of the Human Language
Technology Conference of the NAACL, Main Conference, New York City, USA, June 2006, pp. 17–24.
[2] Ulf Hermjakob, Abdessamad Echihabi, and DanielMarcu, “Natural language based reformulation resourceand wide exploitation for question answering.,” in Proceedings of the 11th Text Retrieval Conference, 2002.
[3] Kiyonori Ohtake and Kazuhide Yamamoto, “Paraphrasing honorifics,” in Automatic Paraphrasing: Theoriesand Applications (NLPRS2001 Post-Conference Workshop), 2001, pp. 13–20.

With regards to related resources for paraphrasing created using bilingual data, there are English paraphrase data extracted
by the same method [8]. In their work, they use EnglishFrench, English-Spanish and other European language bilingual parallel corpora. Finally, 16.7M paraphrases were extracted from 1G sentences of parallel data. Fujita et al. [21]also extract 28M Japanese paraphrases from 3.2M sentencepairs of Japanese-English patent translation data, althoughthese paraphrases are limited to the patent domain and not
publicly available as a resource.
Considering work on Japanese paraphrasing not limited
to those extracted from parallel data, there is Japanese honorifics paraphrase data [3], action word paraphrases [4] andothers. The Japanese honorifics paraphrase data offers 130k
paraphrases gathered from 50k sentences of monolingual data
covering honorifics. In the action word paraphrase data, 1.1k
paraphrases covering action words such as verbs are included.
In these related works, various types of paraphrases aresuggested, but there are few resources freely available. In addition, there are many paraphrase resources that have somekind of theme, but there are few large-scale and general paraphrase resources. In comparison, our paraphrase data is large,
general, and freely available.

[4] Hiroshi Nakagawa and Hidetaka Masuda, “Extractingparaphrases of Japanese action word of sentence ending
part from web and mobile news articles,” in Information Retrieval Technology, vol. 3411 of Lecture Notes in
Computer Science, pp. 94–105. Springer Berlin Heidelberg, 2005.

6. CONCLUSION[8] Juri Ganitkevitch, Benjamin Van Durme, and Chris
Callison-Burch, “PPDB: The paraphrase database,”in Proceedings of The 2013 Conference of the North
American Chapter of the Association for Computational
Linguistics: Human Language Technologies, Atlanta,Georgia, June 2013, pp. 758–764.

In the end, we were able to acquire 1.2M paraphrases from1.9M sentences of our bilingual parallel corpus. Our para4 CC BY-SA 3.0 : Creative Commons Attribution-ShareAlike 3.0 Unported License

132[5] Stijn De Saeger, Kentaro Torisawa, Jun’ichi Kazama,
Kow Kuroda, and Masaki Murata, “Large scale relation acquisition using class dependent patterns,” in Proceedings of the IEEE International Conference on Data
Mining. IEEE, 2009, pp. 764–769.
[6] Chikara Hashimoto, Kentaro Torisawa, Stijn De Saeger,
Jun’ichi Kazama, and Sadao Kurohashi, “Extracting
paraphrases from definition sentences on the web,” inProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human LanguageTechnologies, Portland, Oregon, USA, June 2011, pp.
1087–1097.
[7] Colin Bannard and Chris Callison-Burch, “Paraphrasingwith bilingual parallel corpora,” in Proceedings of the43rd Annual Meeting on Association for ComputationalLinguistics, 2005, pp. 597–604.

[9] Hideki Isozaki, Katsuhito Sudoh, Hajime Tsukada, andKevin Duh, “Head finalization: A simple reorderingrule for SOV languages,” in Proceedings of the JointFifth Workshop on Statistical Machine Translation andMetricsMATR, Uppsala, Sweden, July 2010, pp. 244–251.
[10] Philipp Koehn, Franz Josef Och, and Daniel Marcu,“Statistical phrase-based translation,” in Proceedingsof the 2003 Conference of the North American Chapterof the Association for Computational Linguistics on Human Language Technology - Volume 1, Stroudsburg, PA,USA, 2003, The 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 48–54.
[11] Franz Josef Och and Hermann Ney, “A systematic comparison of various statistical alignment models,” Comput. Linguist., vol. 29, no. 1, pp. 19–51, Mar. 2003.
[12] Yasuhito Tanaka, “Compilation of a multilingual parallel corpus,” Proceedings of the 2001 Conference of thePacific Association for Computational Linguistics, pp.
265–268, 2001.
[13] Masao
UtiyamaandMayumi
Takahashi,“English-Japanese translation alignment data,”
http://www2.nict.go.jp/univ-com/multi_trans/member/mutiyama/align/.
[14] Jason R. Smith, Herve Saint-Amand, Magdalena Plamada, Philipp Koehn, Chris Callison-Burch, and AdamLopez, “Dirt cheap web-scale parallel text from thecommon crawl,” in Proceedings of the Annual Meetingof the Association for Computational Linguistics, 2013,
pp. 1374–1383.
[15] Jim Breen, “WWWJDIC Japanese dictionary serveruser guide,” http://www.csse.monash.edu.
au/˜jwb/wwwjdicinf.html.
[16] Masao Utiyama and Mayumi Takahashi, “JapaneseEnglish bilingual corpus of Wikipedia’s Kyoto
articles,”
http://alaginrc.nict.go.jp/
WikiCorpus/index_E.html.
[17] Richard Socher, John Bauer, Christopher D. Manning,and Andrew Y. Ng, “Parsing with compositional vector
grammars,” in Proceedings of the Annual Meeting of theAssociation for Computational Linguistics. 2013.
[18] Graham Neubig, Yosuke Nakata, and Shinsuke Mori,“Pointwise prediction for robust, adaptable Japanesemorphological analysis,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers- Volume 2, Stroudsburg, PA, USA, 2011, HLT ’11, pp.
529–533.

133[19] Joseph.L. Fleiss, “Measuring nominal scale agreement
among many raters,” Psychological Bulletin, vol. 76,
no. 5, pp. 378–382, 1971.
[20] J. Richard. Landis and G. Gary Koch, “The measurement of observer agreement for categorical data,” Biometrics, vol. 33, pp. 159–174, 1977.
[21] Atsushi Fujita, Pierre Isabelle, and Roland Kuhn, “Enlarging paraphrase collections through generalizationand instantiation,” in Proceedings of the 2012 Joint
Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Stroudsburg, PA, USA, 2012, EMNLPCoNLL ’12, pp. 631–642.