SPIRAL CONSTRUCTION OFSYNTACTICALLY ANNOTATED SPOKEN LANGUAGE CORPUS
Tomohiro Ohno†, Shigeki Matsubara‡, Nobuo Kawaguchi‡ and Yasuyoshi Inagaki§
Graduate School of Information Science, Nagoya University †Information Technology Center/CIAIR, Nagoya University ‡Faculty of Information Science and Technology, Aichi Prefectural University §Furo-cho, Chikusa-ku, Nagoya, 464-8603 Japanohno@inagaki.nuie.nagoya-u.ac.jp

ABSTRACT
Spontaneous speech includes a broad range of linguistic phenomena characteristic of spoken language, andtherefore a statistical approach would be effective forrobust parsing of spoken language. Though a largescale syntactically annotated corpus is required forthe stochastic parsing, its construction requires a lotof human resources. This paper proposes a methodof efficiently constructing a spoken language corpus
for which the dependency analysis is provided. Thismethod uses an existing spoken language corpus. A
stochastic dependency parse is employed to tag spoken
language sentences with the dependency structures,and the results are corrected manually. The tagged
corpus is constructed in a spiral fashion where in thecorrected data is utilized as the statistical information
for automatic parsing of other data. Taking this spiral approach reduces the parsing errors, also allowing
us to reduce the correction cost. An experiment using
10,995 Japanese utterances shows the spiral approach
to be effective for efficient corpus construction.
Keywords: Stochastic parsing, Dependency parsing,
Language database, Spoken dialogue corpus

1. INTRODUCTION
A large-scale text corpus for which the syntactic bracketing information is provided plays an important rolein natural language processing. In fact, the various languages’ parse-trees data of written language such asthat used in newspapers and magazines, for instance,
Penn Treebank [6], NEGRA Treebank [13], TIGER
Treebank [8], Prague Dependency Treebank [3], Kyoto corpus [9], EDR corpus [2], etc., have been widely
utilized not only for language parsing, but also for
information retrieval, automatic summarization, ma-chine translation, and so on. In these corpus, the EDR
corpus and the Kyoto corpus are syntactically annotated corpora for Japanese language, and were built by
sufficiently considering various kinds of syntactic features peculiar to Japanese language. On the other hand,turning our attention to those of spoken language, despite the fact that we can enumerate the Switchboard
corpus [5], Verbmobil Treebanks [1], Spoken Dutch
Corpus [11], etc., very few attempts have been madefor Japanese spoken language so far.
Constructing a large-scale syntactically annotatedcorpus of spontaneously spoken language and utilizing
it as the statistical information would be effective fordeveloping a robust spoken language parsing. Sincemanually providing the annotation for a Japanese text
corpus calls for several difficult tasks such as morphological analysis, bunsetsu segmentation, and dependency analysis1 , it therefore requires considerable human resources.
This paper describes spiral construction of a spoken language corpus in which a dependency structure
is given to each utterance. A stochastic dependency
parser is utilized for automatic annotation to construct
the corpus at a lower cost; that is, our approach to corpus construction is to alternately provide the dependency analyses automatically and repair it manually.
The key to this approach is parsing: the parser is basedon statistical information, so the more the learning data
there is, the more precise the parsing. It can be expected that the data would be corrected less in the spiral construction than that in the non-spiral one.
Stochastic dependency parsing was developed for1
A bunsetsu is one of the linguistic units in Japanese, androughly corresponds to a basic phrase in English. A bunsetsu consists of one independent word and more than zero ancillary words.
A dependency is a modification relation between two bunsetsus.

0003 – 00:09:382-00:13:652 F:D:I:I:
今日[today]朝[morning]パン[bread]
食べて
[ate]
お昼は
[lunchtime]
おそばを[soba]
食べたんですよ<H><SB>[ate]0004 – 00:13:995-00:17:328 F:D:I:I:
今晩は[tonight]
何[what]
食べよっかな<SB>[want to eat]

& キョー
& アサ
& パン
& タベテ
& オヒルワ& オソバオ& タベタンデスヨ<H><SB>& コンバンワ& ナニ& タベヨッカナ<SB>

Figure 2: Transcript of in-car dialogue speech

Figure 1: The data collection vehicle (DCV)
the purpose of spiral construction [10], and our approach was evaluated using over 10,000 spoken dialogue turns in a large-scale spoken dialogue corpus. The result has shown it to be effective for efficiently constructing a syntactically annotated spoken
language corpus.
The paper is organized as follows: The next section explains the in-car speech dialogue corpus. Section 3 presents syntactically annotated spoken language corpus. Our corpus construction method andour support tool are described in Section 4 and 5 respectively. The evaluation of our method is reported in
Section 6.

2. CIAIR IN-CAR SPEECH DIALOGUECORPUS
Human-machine speech interface in a moving car is
one of the important applications of spoken language
systems because the conventional models of data input/output such as video display, keyboard and mouse
are not convenient to use while driving a car. The development of an in-car spoken language interface has
to deal with fatal problems such as noise robustness
and distortion of distant speech. Since the backgroundnoise in a moving car is not stationary and consists ofa variety of sounds, a large corpus is required for training acoustic models in the presence of different background noise conditions. Another important issue isthat the in-car speech communication for information

access by the driver has to deal with the continuously
changing environment depending on the factors suchas traffic condition and the distance to the destination.
For a system to understand the environmental condition, it may be helpful to use audio data along withother types of data such as video images of the persons
involved in dialogue, images of the road in front of the
car and vehicle related data such as the angle of thesteering wheel, status of the accelerator and speed of
the car.
The Center for Integrated Acoustic Information
Research (CIAIR), Nagoya University has been collecting large-scale in-car speech dialogues. The main
objectives of this data collection are as follows: 1)
training acoustic models for the in-car speech data under various driving conditions, 2) training language
models of spoken dialogue for different task domains
related to information access while driving a car, and3) modeling communication by analyzing the interaction among different types of multimedia data. In an
ongoing project, a system specially built in a Data Collection Vehicle (DCV), which is shown in Figure 1, hasbeen used for synchronous recording of multi-channel
audio data, multichannel video data and the vehicle related data. About 400 GB of data has been collected byrecording three sessions of spoken dialogue in about a
60-minute drive by each of 200 drivers.
A spontaneously spoken language corpus has been
constructed by transcribing the collected speech data
into ASCII text files by hand in accordance withthe rule of the corpus of spoken Japanese (CSJ)[4]. The corpus is composed of in-car dialogues between drivers and navigators about shop retrieval, driving directions, and so on. An example of a transcript is shown in Figure 2. For advanced analysis, discourse tags are assigned to fillers, hesitations,slips, and so on. Furthermore, each speech is segmented into utterance units by a pause, and their exact start and end times are provided. Other relevant environmental information regarding speaker’sきょう きょう
副詞可能
oday]
朝 朝
副詞可能
[morning]
朝 朝
副詞可能[morning]
食べ 食べる
自立 一段 連用形
て て
接続助詞
[ate]
パン パン
一般
[bread]食べ 食べる
自立 一段 連用形
て て
接続助詞
[ate]
食べ 食べる
自立 一段 連用形
て て
接続助詞
ate]
食べ 食べる
自立 一段 連用形
た た
特殊・タ 基本形
ん ん
非自立
です
です
特殊・デス 基本形
よ よ
終助詞
[ate]
お昼
お昼
副詞可能
は は
係助詞
[lunctime]
お お
名詞接続
そば
そば
一般
を を
格助詞
[soba]
お お
名詞接続
そば
そば
一般
を を
格助詞
[soba]
食べ 食べる
自立 一段 連用形
た た
特殊・タ 基本形
ん ん
非自立
です
です
特殊・デス 基本形
よ よ
終助詞
[ate]
食べ 食べる
自立 一段 連用形
た た
特殊・タ 基本形
ん ん
非自立
です
です
特殊・デス 基本形
よ よ
終助詞
[ate]
なし

((1 ((kyo-> (2 ((asa((2 ((asate((3 ((

verb

particlepan-> (4 ((te((4 (()

particletenone none)))verb

none none)))[

tabe

verb

(

ta

auxiliary-verb

(

n

noun

(

yo

((5 ((wa-> (6 ((o((6 ((

nounonone none)

noun

o

none none)

none none)))prefixnoun

particle-> (7 ((

none none)

noun

(

ta

auxiliary-verb

(

n

noun

(

desu

yo

((7 ((

ta

(

n

(

)

auxiliary-verb none)

none none)))

verb

)

)

auxiliary-verb none

noun

desu

yo

-> (NO ()

none

none none)

particletabe(– Boundaries between bunsetsus– Dependencies between bunsetsus
Here, the specification of the parts-of-speech is inaccordance with that of IPA parts-of-speech in a morphological analyzer called ChaSen [12], the rules ofthe bunsetsu segmentation with those of CSJ [4], andthe dependency grammar with that of the Kyoto Corpus [9]. We have provided the following criteria forthe linguistic phenomena peculiar to spoken language:• There is no bunsetsu on which fillers and hesitations depend. They form dependency structures
independently.
• A bunsetsu whose head bunsetsu is omitted does
not depend on any bunsetsu.

none none)))

tabe

(

• Syntactic information)

none none)

soba

(

none none)

particleo()

none none)))prefixsoba()

none none)))particle()

auxiliary-verb none

particle

ohiru

(

)

none

none none)

desu

(

)

)

particle-> (7 (()

none none)))

verb

tabe

(

)

none none)))

noun

tabe

(

)

none none)))

tabe

(

none none)))[t

none none)))noun-> (4 ((

(

noun

noun

)

none none)

auxiliary-verb none

particlenone none))

)

)

)))

Figure 3: Spoken Japanese sentence annotated by dependency structuresex (male/female), role (driver/navigator), dialogue
task (navigation/information retrieval/...), and noise
conditions (noisy/clear) is provided for each utteranceunit.

3. SYNTACTICALLY ANNOTATEDSPOKEN LANGUAGE CORPUS
Our syntactically annotated spoken language corpus
has been constructed by providing the following information for each of the driver’s utterances in CIAIR
In-car Speech Dialogue Corpus.
• Morphological information– Boundaries between words– Pronunciation, basic form, part-of-speech,
conjugation type, conjugated form of eachword

• The specification for parts-of-speech has been
provided for phrases peculiar to spoken language by adding lexical entries to the dictionary.
• We define one conversational turn as a unit ofdependency parsing. The dependencies might
be over two utterance units, but rarely over two
conversational turns.
Figure 3 shows an example of spoken Japanese
sentences annotated by dependency structure. It illustrates a sequence of dependency relations, each ofwhich consists of a dependency bunsetsu and a headbunsetsu. Each bunsetsu is listed with it’s number andits constituent morphemes.

4. SPIRAL CONSTRUCTION OF THE
CORPUSIn this study, the dependency analyses are provided fora spoken language sentences by correcting the resultsof the stochastic dependency parsing. To correct themefficiently, it is desirable to execute dependency parsing with high precision. Generally speaking, that canbe attained by increasing the corpus data that is used
for acquiring the statistical information.
This section explains the method of spirally constructing spoken language corpus with dependency
structures by splitting the corpus into several sets andcarrying out the annotation incrementally. That is,the data constructed by manually correcting the parsed
data is added to the annotated corpus and utilized asstatistical information for the dependency parsing ofanother data set. Since the statistical information canCIAIRCIAIRIn-car
In-carSpeech
SpeechDialogue
DialogueCorpus
CorpusPick up
Automatic(1) Morphological Analysis(3) Stochastic
Bunsetsu Segmentation(5) Stochastic
Dependency Parsing
SyntacticallyAnnotated Data
Acquire Stochastic Information

Text Data
Manual(2) Modification(Morphology)(4) Modification(Bunsetsu)(6) Modification(Dependency Structure)
Add

SyntacticallySyntacticallyAnnotated
AnnotatedSpoken
SpokenLanguage
LanguageCorpus
CorpusFigure 4: Flow of corpus construction

be automatically acquired from the corrected data, nohuman resources are necessary except to correct theparsed data. The larger the corpus’ scale, the higher
the precision of the stochastic dependency parsing becomes.
4.1. Flow of Corpus Construction
Figure 4 shows the flow of the corpus construction.
The transcripts of spoken language are split into several sets, and the following tasks are executed in orderfor each set:
(1)

The transcribed text is segmented into morphemes, and automatically tagged with respect
to parts-of-speech by ChaSen.

(2) Morpheme segmentation and part-of-speech tagging are corrected manually. Any new expressions unique to spontaneously spoken languageor proper nouns are added to the ChaSen dictionary.
(3) The corrected text is automatically segmented into
bunsetsus based on stochastic bunsetsu segmentation.
(4) The bunsetsu segmentation is corrected manually.

(5) The dependency analysis is automatically provided for the bunsetsu-segmented text based onstochastic dependency parsing [10].
(6) The dependency analysis is corrected manually.
The result of (6) is newly added to the spoken language corpus with dependency analyses. Detailed accounts of the bunsetsu segmentation and dependency
parsing are given below.
4.2. Stochastic Bunsetsu Segmentation
Bunsetsu segmentation is used to decide whether
to insert a bunsetsu boundary between adjoining
morphemes. For example, a sequence of fifteen
morphemes: “kyo-asa-pan-tabe-te-ohiru-wa-o-soba-otabe-ta-n-desu-yo (I had some bread in this morningand soba at lunchtime.)” is segmented into the following seven bunsetsus:kyo(today)/asa(morning)/pan(bread)/tabete(ate)/
ohiruwa(lunchtime)/osobao/tabetandesuyo(ate)The method acquires the stochastic information on
bunsetsu boundaries from a bunsetsu-segmented corpus, and utilizes it to segment a sequence from left
to right. When considering the bunsetsu segmentation between adjoining morphemes mi and mi+1 , themethod uses the following attributes as stochastic information:• The basic forms of mi and mi+1 : hi , hi+1
• The parts-of-speech of mi and mi+1 : ti , ti+1
• The conjugated forms or the detailed parts-ofspeech of mi and mi+1 : si , si+1
The probability that the boundary between mi andmi+1 is the bunsetsu boundary, i.e., that the morphemes are not constituents of the same bunsetsu, iscalculated as follows:
Figure 5: Support tool for corpus correction(1)
P (mi /mi+1 |mi , mi+1 )C(mi /mi+1 , hi , hi+1 , ti , ti+1 , si , si+1 ).
=C(hi , hi+1 , ti , ti+1 , si , si+1 )Here, mi /mi+1 means that there is a bunsetsu boundary between mi and mi+1 , andC is a cooccurrence frequency function.
If
P (mi /mi+1 |mi , mi+1 ) ≥ 0.5, the boundary between mi and mi+1 can be regarded as the bunsetsuboundary.
4.3. Stochastic Dependency Parsing
Our dependency parsing method can robustly parse
grammatically ill-formed linguistic phenomena uniqueto spoken language, e.g. inversion and no head bunsetsu [10]. For a sequence of bunsetsus, B (=b1 · · · bn ), the method identifies the dependency structure S.
The conventional methods of dependency parsing
for a written language have assumed the following
three syntactic constraints: dependencies don’t crosseach other, no dependencies are directed from rightto left, and each bunsetsu except the last one depends
on only one bunsetsu. Considering that there are frequent inversions, fillers, hesitations and slips, we established that a dependency structure only fulfill one
constraint: dependencies don’t cross each other. However, we consider the other two constraints by reflecting the stochastic information.
Assuming that each dependency is independent,
the P (S|B) can be calculated as follows:
P (S|B) =

n


rel

P (bi → bj |B),

(2)

i=1
rel

where P (bi → bj |B) is the probability that a bunsetsu
bi depends on a bunsetsu bj when the sequence of bun-setsus B is provided. The parameter S, which maximizes the conditional probability P (S|B) is regarded
as the dependency structure of B and identified by DP.
rel
Next, we explain the calculation of P (bi → bj |B).
First, the basic form of independent words in a dependent bunsetsu is represented by hi , its parts-of-speechti , type of dependency ri , and the basic form of the
independent word in a head bunsetsu hj , its parts-ofspeech tj . Furthermore, the distance between bunsetsus is depicted as dij , the number of pauses between
them pij , and the location of the dependent bunsetsu li .
Here, if a dependent bunsetsu has an ancillary word,
the type of dependency is the lexicon, part-of-speechand conjugated form of that ancillary word, and if notso, it is the part-of-speech and conjugated form of thelast morpheme. Moreover, the dependent bunsetsu’s
location indicates whether it is the last one of the turn.
By using the above attributes, the conditional probrel
ability P (bi → bj |B) is calculated as follows:rel

P (bi → bj |B)

(3)

rel

∼
= P (bi → bj |hi , hj , ti , tj , ri , dij , pij , li )rel=C(bi → bj , hi , hj , ti , tj , ri , dij , pij , li ).
C(hi , hj , ti , tj , ri , dij , pij , li )Note that C is a cooccurrence frequency function. The
probability of a bunsetsu not having a head bunsetsu
can also be calculated in formula (3) by consideringthat such a bunsetsu depends on itself ( i.e. i = j).

5. SUPPORT TOOL FOR CORPUS
CORRECTION
To reduce the human resources needed for correcting
the parsing errors, we created a graphical user interface. Figure 5 shows an example of the interface.

100
90
Precision rate (%)

The left-hand part of the interface window is used
to correct the parsed morpheme and the bunsetsuboundary. One morpheme is represented by one row,each of which contains buttons that display more information about the morpheme. Part-of-speech, detailed
part-of-speech, conjugation type and conjugated form
are displayed by menu buttons, and the user can correct the data by selecting the appropriate one from themenu bar. The bunsetsu boundary is modified by clicking the button in the left hand corner and changing thecolor of that row.
The right-hand side of the interface window is used
for correcting the dependency structure. Here, the partnot only indicates the bunsetsu and its head bunsetsu,
but also visually displays the dependency relations. If
the head bunsetsu number is “NO,” it implies that thebunsetsu does not have any head bunsetsu.

80
70
60
50
40
0

90

• Non-Spiral Construction: We used only thebasic learning data to parse the entire test data,
after which they were corrected manually.
The results were evaluated by comparing the precision of bunsetsu segmentation and dependency parsing
between the above two construction methods.

Precision rate (%)

We have evaluated our method through an experimenton corpus construction, noting the amount of manuallycorrected automatic parsing results.

• Spiral Construction: We provided the dependency structures from the data 1 to 110 in order,
according to the method described in Section 4.
The completed data with the dependency structures was added to the learning data for the restof the parsing.

40

60

80

100

120

Figure 6: The result of stochastic bunsetsu segmentation95

We performed an experiment on syntactical annotationof a large-scale spoken language corpus using 221 spoken dialogues in the CIAIR speech database [7]. The
data consists of 10,995 dialogue turns, and the lengthof a dialogue turn is 4.1 bunsetsus on average. We
used 10 turns as the basic learning data, while the remaining 10,985 turns were used as the test data. To
examine the effectiveness of the spiral construction,
we equally split the test data into 110 sets (approx.
100 turns each), and for practical convenience, we allocated numbers from 1 to 110 to each set.
We constructed a corpus by the following two
methods:

20

Test data set

6. EVALUATION

6.1. Outline of the Experiment

Spiral Method
Non-spiral Method

85
80

Spiral Method
Non-spiral Method

75
70
65
60
55
0

20

40

60
80
Test data set

100

120

Figure 7: The result of stochastic dependency parsing

6.2. Experimental Results
Figure 6 shows the experimental result for bunsetsu
segmentation. The segmentation conducted with thespiral method attained a level of precision about 46.4%
higher than the non-spiral one. There were 96,675
boundaries between the morphemes included in the
text data, among which 48,969 were correctly judged
by the non-spiral method, and 93,880 by the spiralmethod. In other words, this means that adopting
the spiral method reduced the number of corrections
by 44,911 and that the corrections made by the spiralmethod should be 94.1% fewer than by the non-spiralone.
A comparison of the dependency parsing is shownin Figure 7. The probability of a correct judgment
made with the spiral method is 88.4%, whereas thatwith the non-spiral method is 61.7%. As it waswith bunsetsu segmentation, the spiral method showed
higher precision in ascribing dependency data. Amongthe 45,012 dependencies in the test data, 27,767 werecorrectly ascribed by the non-spiral method, whereas
39,804 were correctly ascribed by the spiral method.
In other words, it means that by adopting the spiralmethod, the number of corrections were reduced by
12,037, a reduction of roughly 69.8% compared with
the non-spiral method.

[3]

J. Hajic, “Building a Syntactically Annotated
Corpus: The Prague Dependency Treebank,” Issues of Valency and Meaning, pp. 106-132, 1998.

[4]

K. Maekawa, H. Koiso, S. Furui and H. Isahara,“Spontaneous Speech Corpus of Japanese,” Proc.
of LREC-2000, No.262, pp. 947-952, 2000.

[5]

M. Meteer, et al., “Dysfluency Annotation Stylebook for the Switch-board Corpus. Linguistic
Data Consortium,” revised by Ann Taylor, 1995,

7. CONCLUDING REMARKSIn this paper,we have proposed a method of spirally constructing a syntactically annotated spoken language corpus based on stochastic bunsetsu segmentation and dependency parsing. The method utilizes the
corpus, which is constructed by manually correcting
the automatic parsing result, as statistical information
to provide the dependency structure for each utterance.
The evaluation using the CIAIR spoken dialogue corpus has shown our approach to be effective in reducing
the human resources necessary to correct the parsing
result.
We expect this approaches effect to increase as thescale of the utilized spoken language corpus grows.
We plan to continue corpus construction and report a
detailed analysis using a larger-scale corpus in anotherpaper.

8. ACKNOWLEDGEMENT
The authors would like to thank Ms. Hitomi Toyamaof the Graduate School of Languages and Cultures,
Nagoya University for her helpful support in correcting the dependency corpus. This work is partially supported by the Grant-in-Aid for COE Research and forYoung Scientists of the Ministry of Education, Science, Sports and Culture, Japan and The Asahi GlassFoundation.

References
[1]

[2]

E.W. Hinrichs, J. Bartels, Y. Kawata, V. Kordoni and H. Telljohann, “The VERBMOBIL Treebanks,” In Zuehlke, W. and SchukatTalamazzini, E. G. (eds.), Proc. of KONVENS2000 Sprachkommunikation, ITG-Fachbericht
161, pp. 107-112, 2000.
Japan Electronic Dictionary Research Institute,Ltd., “The EDR electronic dictionary technicalguide (second edition),” Technical Report TR045, Japan Electronic Dictionary Research Institute, 1995.

ftp://ftp.cis.upenn.edu/pub/treebank/swbd/doc/DFLbook.ps.

[6]

M.P. Marcus, B. Santorini and M.A.
Marcinkiewicz, “Building a Large Annotated Corpus of English: The penn treebank,”
Computational Linguistics, Vol.19, No.2, pp.
313-330, 1993.

[7]

N. Kawaguchi, S. Matsubara，K. Takeda and F.
Itakura, “Multimedia Data Collection of In-Car
Speech Communication,” Proc. of Eurospeech2001, pp. 2027-2030, 2001.

[8]

S. Brants, S. Dipper, S. Hansen, W. Lezius and
G. Smith, “The TIGER Treebank,” Proc. of theWorkshop on Treebanks and Linguistic Theories,
pp. 24-41, 2002.

[9]

S. Kurohashi and M. Nagao, “Building a
Japanese Parsed Corpus while Improving theParsing System,” Proc. of NLPRS-97, pp. 451456, 1997.

[10] S. Matsubara, T. Murase, N. Kawaguchi and
Y. Inagaki, “Stochastic Dependency Parsing of
Spontaneous Japanese Spoken Language,” Proc.
of COLING-2002, Vol.1, pp. 640-645, 2002.
[11] T. van der Wouden, H. Hoekstra, M. Moortgat, B.
Renmans and I. Schuurman, “Syntactic Analysis
in The Spoken Dutch Corpus (CGN),” Proc. of
LREC-2002, pp. 768-773, 2002.
[12] Y. Matsumoto, A. Kitauchi, T. Yamashita and Y.
Hirano, “Japanese Morphological Analysis System ChaSen version 2.0 Manual,” NAIST Technical Report, NAIST-IS-TR99009, 1999.
[13] W. Skut, T. Brants, B. Krenn and H. Uszkoreit, “A Linguistically Interpreted Corpus of German
Newspaper Text,” Proc. of LREC-98, pp. 705711, 1998.