Chunking and Dependency Analysis ofJapanese Compound Functional Expressionsby Machine Learning
Takehito Utsuro1 , Takao Shime2 , Masatoshi Tsuchiya3 ,Suguru Matsuyoshi4 , and Satoshi Sato4
1Graduate School of Systems and Information Engineering, University of Tsukuba,
1-1-1, Tennodai, Tsukuba, 305-8573, JAPAN
2NEC Corporation, JAPAN3
Computer Center, Toyohashi University of Technology,
Tenpaku-cho, Toyohashi, 441–8580, JAPAN4
Graduate School of Engineering, Nagoya University,
Furo-cho, Chikusa-ku, Nagoya 464-8603, JAPANAbstract. This paper proposes an approach of processing Japanesecompound functional expressions by identifying them and analyzing theirdependency relations through a machine learning technique. The resultsof experimental evaluation shows that, the dependency analysis model
applied to the results of identifying compound functional expressions signiﬁcantly outperforms the one applied to the results without identifying
compound functional expressions.

1

Introduction

In the Japanese language, recognition and semantic interpretation of compound
functional expressions are diﬃcult because it often happens that one compound
expression may have both a literal (in other words, compositional) content word
usage and a non-literal (in other words, non-compositional) functional usage.
For example, Table 1 shows two example sentences of a compound expression“ni tsuite”, which consists of a post-positional particle “ni”, and a conjugatedform “tsuite” of a verb “tsuku”. In the sentence (A), the compound expressionfunctions as a case-marking particle and has a non-compositional functional
meaning “about”. On the other hand, in the sentence (B), the expression simply
corresponds to a literal concatenation of the usages of the constituents: the postpositional particle “ni” and the verb “tsuite”, and has a content word meaning“follow”. When considering machine translation of those Japanese sentences into
English, it is necessary to precisely judge the usage of the compound expression“ni tsuite”.
There exist widely-used Japanese text processing tools, i.e., pairs of a morphological analysis tool and a subsequent parsing tool, such as JUMAN1 + KNP2
12

http://nlp.kuee.kyoto-u.ac.jp/nl-resource/juman-e.html
http://nlp.kuee.kyoto-u.ac.jp/nl-resource/knp-e.html

Table 1. Translation Selection of a Japanese Compound Expression “ni tsuite”
watashi ha kareni tsuite
hanashita(I) (TOP) (he)(about)(talked)(I talked about him.)watashi ha kare nitsuite hashitta(B) (I) (TOP) (he) (ACC) (follow) (ran)(I ran following him.)(A)(1) Correct Dependency Relation by Identifying Compound Functional Expression: “䈫㵥䈚䈩”
with a Case Marking Particle Usage.
☨࿖ 䈲 ⸃᳿ᚻᲑ 䈫 䈚䈩䇮USA-TOP

as a means for solution䊨䉲䉝 䉅
Russia-NOM alsoෳട 䈚䈩 䈇䉎
participate inో᰷቟଻දജᯏ᭴ 䈱of OSCEᵴ↪ 䉕ㅴ⸒䈚䈢䇯

activity-ACC

recommended

Correct English Translation:( As a means of solving the problem, USA recommended the activity of OSCE in which Russia participates.)(2) Incorrect Dependency Relation without Identifying Compound Functional Expression: “䈫㵥䈚䈩”,
which Literally Consists of a Post-positional Particle “䈫” (with) and a Conjugation Form “䈚䈩”
of a Verb “䈜䉎” (do).
☨࿖ 䈲⸃᳿ᚻᲑ 䈫USA-TOP

with a means forsolution䈚䈩䇮䊨䉲䉝 䉅Russia-NOM alsoෳട 䈚䈩 䈇䉎 ో᰷቟଻දജᯏ᭴ 䈱participate inof OSCEᵴ↪ 䉕ㅴ⸒䈚䈢䇯

activity-ACC

recommendedFig. 1. Example of Improving Dependency Analysis of Compound Functional Expressions by Identifying them before Dependency Analysisand ChaSen3 + CaboCha4 . However, they process those compound expressions
only partially, in that their morphological analysis dictionaries list only limited
number of compound expressions. Furthermore, even if certain expressions arelisted in a morphological analysis dictionary, those existing tools often fail in resolving the ambiguities of their usages, such as those in Table 1. This is mainly
because the framework of those existing tools is not designed so as to resolve
such ambiguities of compound (possibly functional) expressions by carefully considering the context of those expressions.
Actually, as a ﬁrst step of studying computational processing of compound
functional expressions, we start with 125 major functional expressions whichhave non-compositional usages [4], as well as their variants (337 expressions intotal). Out of those 337 expressions, 111 have both a content word usage and34

http://chasen.naist.jp/hiki/ChaSen/
http://chasen.org/˜taku/software/cabocha/

⸃᳿

⸃᳿

(solution)(solution)ᚻᲑ(means)䈫䈫(with)(with)

(do)

䈩

䈩

(and)

(and)

ᚻᲑᚻᲑ
(means)

䈫䈚䈩

䈫䈚䈩

(as)

(as)

࡮࡮
࡮

(do)

࡮࡮
࡮

( As a means of solving theproblem, USA recommended theactivity of OSCE in which Russia
participates.)䈚࡮࡮࡮☨࿖䈲⸃᳿ᚻᲑ䈫䈚䈩䇮䊨䉲䉝䉅ෳട䈚䈩䈇䉎ో᰷቟଻දജᯏ᭴䈱ᵴ↪䉕ㅴ⸒䈚䈢䇯䈚⸃᳿(solution)(means)࡮࡮࡮chunking

bunsetsu
segmentation&dependency
analysis࡮࡮࡮ᚻᲑ(means)Identifying
compoundfunctionalexpression

࡮
࡮
࡮

⸃᳿
(solution)

࡮࡮
࡮

࡮
࡮
࡮

morphologicalanalysisby ChaSenbunsetsu
segment

compoundfunctional
expression

dependency
relationFig. 2. Overall Flow of Processing Compound Functional Expressions in a Japanese
Sentencea functional usage5 . However, the pair of JUMAN+KNP is capable of distinguishing the two usages only for 43 of the 111 expressions, and the pair ofChaSen+CaboCha only for 40 of those 111 expressions. Furthermore, the failure in distinguishing the two usages may cause errors of syntactic analysis. For
example, Figure 1 shows an example of improving dependency analysis of compound functional expressions by identifying them before dependency analysis.
Here, Japanese dependency structure is usually deﬁned in terms of the relationship between phrasal units called bunsetsu segments (hereafter “bunsetsus”). A
bunsetsu consists of one or more content words following zero or any numberof functional words. In Figure 1, (1) gives an example of identifying a correct
modiﬁee of the bunsetsu segment including a Japanese compound functional expression “to-shite (as)”, by appropriately detecting the compound functional expression before dependency analysis. On the other hand, (2) of Figure 1 gives anexample of incorrectly indicating an erroneous modiﬁee of the bunsetsu “shite”,
which actually happens if we do not identify the compound functional expression“to-shite (as)” before dependency analysis of this sentence.
Considering such a situation, it is necessary to develop a tool which properly
recognizes and semantically interprets Japanese compound functional expressions. This paper proposes an approach of processing Japanese compound functional expressions by identifying them and analyzing their dependency relations
through a machine learning technique. The overall ﬂow of processing compound
functional expressions in a Japanese sentence is illustrated in Figure 2. First ofall, we assume a sequence of morphemes obtained by a variant of ChaSen withall the compound functional expressions removed from its outputs, as an input
to our procedure of identifying compound functional expressions and analyzing
5In this paper, we take an approach of regarding each of those variants as a ﬁxed expression, rather than a semi-ﬁxed expression or a syntactically-ﬂexible expression [5].

their dependency relations. We formalize the task of identifying Japanese compound functional expressions in a text as a machine learning based chunking
problem [7]. We employ the technique of Support Vector Machines (SVMs) [9]as the machine learning technique, which has been successfully applied to various natural language processing tasks including chunking tasks such as phrase
chunking [1] and named entity chunking. Next, against the results of identifying
compound functional expressions, we apply the method of dependency analysis
based on the cascaded chunking model [2].

2Chunking Compound Functional Expressions by SVMs

This section describes details of formalizing the chunking task using SVMs. In
this paper, we use an SVMs-based chunking tool YamCha6 [1]. In the SVMsbased chunking framework, SVMs are used as classiﬁers for assigning labels forrepresenting chunks to each token. In our task of chunking Japanese compound
functional expressions, each sentence is represented as a sequence of morphemes,
where a morpheme is regarded as a token.
Given a candidate expression, we classify the usages of the expression into twoclasses: functional and content. Accordingly, we distinguish the chunks of the two
types: the functional type chunk and the content type chunk. For representing
proper chunks, we employ IOB2 representation [1]. As for extending SVMs tomulti-class classiﬁers, we employ the pairwise method.
For the feature sets for training/testing of SVMs, we use the information
available in the surrounding context, such as the morphemes, their parts-ofspeech tags, as well as the chunk labels. More precisely, suppose that we identify
the chunk label ci for the i-th morpheme:−→ Parsing Direction −→Morpheme
mi−2 mi−1 mi mi+1 mi+2
Feature set at a position Fi−2 Fi−1 Fi Fi+1 Fi+2
Chunk label
ci−2 ci−1 ciHere, mi is the morpheme appearing at i-th position, Fi is the feature set
at i-th position, and ci is the chunk label for i-th morpheme. Roughly speaking,when identifying the chunk label ci for the i-th morpheme, we use the featuresets Fi−2 , Fi−1 , Fi , Fi+1 , Fi+2 at the positions i − 2, i − 1, i, i + 1, i + 2, as wellas the preceding two chunk labels ci−2 and ci−1 . The detailed deﬁnition of thefeature set Fi at i-th position is given in [7], which mainly consists of morphemes
as well as information on the candidate compound functional expression at i-thposition.
6

http://chasen.org/˜taku/software/yamcha/

3Coping with Compound Functional Expressions inStatistical Dependency Analysis

In general, dependency relations of a Japanese sentence satisﬁes the following
two constraints:
1. Japanese is a head-ﬁnal language. Thus, except for the rightmost one, eachbunsetsu segment modiﬁes exactly one bunsetsu segment among those appearing to its right.
2. Dependencies do not cross one another.
As the framework of statistical dependency analysis of Japanese sentences, weemploy the method of dependency analysis based on the cascaded chunking
model [2]. Unlike probabilistic dependency analysis models of Japanese, the cascaded chunking model of [2] does not require the probabilities of dependencies
and parses a sentence deterministically.
Bunsetsu Representation As we show in Figure 2, a compound functional
expression is identiﬁed as a sequence of several morphemes and then chunked
into one morpheme. The result of this identiﬁcation process is then transformed
into the sequence of bunsetsu segments. Finally, to this modiﬁed sequence ofbunsetsu segments, the method of dependency analysis based on the cascaded
chunking model is applied. Here, when chunking a sequence of several morphemes
constituting a compound functional expression, the following two cases may
exist:(A) As in the case of the example (A) in Table 1, the two morphemes constituting
a compound functional expression “ni tsuite” overlaps the boundary of twobunsetsu segments. In such a case, when chunking the two morphemes into
one morpheme corresponding to a compound functional expression, those
two bunsetsu segments are concatenated into one bunsetsu segment.
kare ni tsuite
kare ni-tsuite=⇒(he)(he) (about)(B) As we show below, a compound functional expression “koto ga aru” overlaps
the boundary of two bunsetsu segments, though the two bunsetsu segments
concatenating into one bunsetsu segment does not include content words. Insuch a case, its immediate left bunsetsu segment, which corresponds to thecontent word part of “koto ga aru”, has to be concatenated into the one of
the compound functional expression “koto ga aru”.
itt ta koto ga aru
itt ta koto-ga-aru=⇒(went)(have been ∼)Next, to the compound functional expression, we assign one of the four grammatical function types (case-marking particle, conjunctive particle, adnominalparticle, and auxiliary verb types) as its POS tag. For example, the compound
functional expression “ni tsuite” in (A) above is assigned the grammatical function type “case-marking particle type”, while “koto ga aru” in (B) is assigned“auxiliary verb type”.

Table 2. Statistics of Data Sets
Usages# offunctional content total sentences
for chunker
training
Kyoto text corpus

1918

1165

3083

2429

5744

1959

7703

38400

Feature Representation As features of dependency analysis, we simply use
those of the Japanese dependency analyzer based on the cascaded chunking
model, namely, the publicly available version of CaboCha [2]. In CaboCha, forthe pair of modiﬁer/modiﬁee bunsetsu segments, head words and their parts-ofspeech tags, inﬂection-types/forms, functional words and their parts-of-speechtags, inﬂection-types/forms, inﬂection forms of the words that appear at theend of bunsetsu segments are used as features. As for features between modiﬁer/modiﬁee bunsetsu segments, the distance of modiﬁer/modiﬁee bunsetsu segments, existence of case-particles, brackets, quotation-marks, and punctuationmarks are used.
The modiﬁcations of the busetsu representations in the previous section ﬁnally cause diﬀerences in the feature representations of CaboCha. For example,
let us compare the feature representations of the modiﬁer bunsetsu segmentsin (1) and (2) of Figure 1. In (1), the modiﬁer bunsetsu segment is “kaiketsushudan-to-shite (as a means for solution)” which has the compound functional
expression “to-shite” in its functional word part. On the other hand, in (2), themodiﬁer bunsetsu segment is “shite”, which corresponds to the literal verb usageof a part of the compound functional expression “to-shite”. In the ﬁnal feature
representations below, this diﬀerence causes the following diﬀerences in headwords, functional words, and POS of the modiﬁer bunsetsu segments:

head word
functional word
POS

(1) of Figure 1(2) of Figure 1
shudan (means)suru (do)to-shite (as)te (and)case-marking particle conjunctive particle
compound functional expression

4

Experimental Evaluation

4.1

Training/Test Data Sets

In the experimental evaluation, we focus on 59 expressions which have balanced
distribution of their usages in the newspaper text corpus and are among the mostdiﬃcult ones in terms of their identiﬁcation in a text. Here, we assume that we
can similarly train chunkers of compound functional expressions other than theTable 3. Evaluation Results of Chunking Compound Functional Expressions (%)
Identifying
Acc. of classifying
functional chunks functional / content
Prec. Rec. Fβ=1chunks
majority ( = functional) 74.6 100 85.5
74.6
Juman/KNP
85.8 40.5 55.0
58.4
ChaSen/CaboCha
85.2 26.7 40.6
51.1
SVM
91.4 94.6 92.9
89.3
Table 4. Accuracies of Identifying Modiﬁer(s)/Modiﬁee of Compound Functional Expressions (%)

baselines CaboCha (w/o FE)
CaboCha (public)
chunker + CaboCha (proposed)
reference + CaboCha (proposed)

modiﬁer modiﬁee
72.5
88.0
73.9
87.6
74.0
88.0
74.4
88.1

selected 59 expressions. For the training of chunking compound functional expressions, we collected 2,429 example sentences from 1995 Mainichi newspaper
text corpus. For the testing of chunking compound functional expressions, aswell as training/testing of learning dependencies of compound functional expressions, we used manually parsed sentences of Kyoto text corpus [3], that are
38,400 sentences selected from 1995 Mainichi newspaper text. To those data sets,
we manually annotate usage labels of the 59 compound functional expressions(details in Table 2).
4.2

Chunking

As we show in Table 3, performance of our SVMs-based chunkers as well as
several baselines including existing Japanese text processing tools is evaluatedin terms of precision/recall/Fβ=1 of identifying functional chunks. Performanceis evaluated also in terms of accuracy of classifying detected candidate expressions into functional/content chunks. Among those baselines, “majority ( = functional)” always assigns functional usage to the detected candidate expressions.
Our SVMs-based chunker signiﬁcantly outperforms those baselines. Some of theerrors should be recovered by simply introducing semantic categories of nouns
as features.
4.3

Analyzing Dependency Relations

Finally, we evaluate the accuracies of judging dependency relations of compound
functional expressions by the variant of CaboCha trained with Kyoto text cor-pus annotated with usage labels of compound functional expressions. This performance is measured through 10-fold cross validation with the modiﬁed versionof Kyoto text corpus. Accuracies of identifying modiﬁer(s)/modiﬁee of compound functional expressions are measured as in Table 4. Here, “CaboCha (w/oFE)” denotes a baseline variant of CaboCha, with all the compound functional
expressions removed from its outputs, while “CaoboCha (public)” denotes thepublicly available version of CaboCha, which have some portion of the compound functional expressions included in its outputs. For the modiﬁer accuracy,
the diﬀerence of “chunker + CaboCha (proposed)” and “CaboCha (w/o FE)” isstatistically signiﬁcant at a level of 0.05. Detailed discussions on the evaluation
results are in [8].

5Concluding Remarks

This paper proposed an approach of processing Japanese compound functional
expressions by identifying them and analyzing their dependency relations througha machine learning technique. The proposed framework has advantages over anapproach based on manually created rules such as the one in [6], in that it requires considerable human eﬀort to manually create and maintain those rules.
Future works include the issue of implementing chunkers and dependency analyzers which incorporate much larger number of compound functional expressions
with training corpus of limited size.

References
1. T. Kudo and Y. Matsumoto. Chunking with support vector machines. In Proc. 2ndNAACL, pages 192–199, 2001.
2. T. Kudo and Y. Matsumoto. Japanese dependency analyisis using cascaded chunking. In Proc. 6th CoNLL, pages 63–69, 2002.
3. S. Kurohashi and M. Nagao. Building a Japanese parsed corpus while improving
the parsing system. In Proc. 1st LREC, pages 719–724, 1998.
4. National Language Research Institute. Gendaigo Hukugouji Youreishu. 2001. (inJapanese).
5. I. Sag, T. Baldwin, F. Bond, A. Copestake, and D. Flickinger. Multiword expressions: A pain in the neck for NLP. In Proc. 3rd CICLING, pages 1–15, 2002.
6. K. Shudo, T. Tanabe, M. Takahashi, and K. Yoshimura. MWEs as non-propositional
content indicators. In Proc. 2nd ACL Workshop on Multiword Expressions: Integrating Processing, pages 32–39, 2004.
7. M. Tsuchiya, T. Shime, T. Takagi, T. Utsuro, K. Uchimoto, S. Matsuyoshi, S. Sato,and S. Nakagawa. Chunking Japanese compound functional expressions by machinelearning. In Proc. Workshop on Multi-Word-Expressions in a Multilingual Context,
pages 25–32, 2006.
8. T. Utsuro, T. Shime, M. Tsuchiya, S. Matsuyoshi, and S. Sato. Learning dependency
relations of Japanese compound functional expressions. In Proc. Workshop on A
Broader Perspective on Multiword Expressions (ACL-2007 Workshop), 2007.
9. V. N. Vapnik. Statistical Learning Theory. Wiley-Interscience, 1998.