Proceedings of the Third NTCIR Workshop

Unsupervised Acquisition of knowledge about the abbreviation possibilityof some of multiple phrases modifying the same verb/noun
Hiroyuki SAKAI Shigeru MASUYAMA
Department of Knowledge-based Information Engineering
Toyohashi University of Technology
1-1 Hibarigaoka, Tempaku-cho, Toyohashi-shi, Aichi 441-8580, Japansakai@smlab.tutkie.tut.ac.jp, masuyama@tutkie.tut.ac.jp

Abstract
This paper proposes a statistical method of acquiring knowledge about the abbreviation possibility of some of multiple phrases modifying the sameverb/noun. Our method calculates weight values ofmultiple phrases by mutual information based on thestrength of relation between the phrases among themultiple phrases and modified verbs/nouns. Amongphrases modifying the same verb/noun, those having
relatively low weight value are deleted. The evaluation of our method by experiments shows that the precision attains about 74.0% and the recall attains about
43.0%.
Keywords: abbreviation of multiple phrases, corpus, summarization.

1

Introduction

Recent rapid progress of computer and communication technologies enabled us to access enormous
amount of machine-readable information easily. This,
however, has caused so called the information overload problem. Under these circumstances, the necessity for automatic summarization has been increasing
and has been intensively studied recently (see e.g,[6]).
No all-purpose automatic summarization method forsummarizing any type of documents appropriately exists, thus, in many cases, we must combine several
summarization methods as components when we develop an automatic summarization system. Deletionof some unnecessary parts from a sentence is amongsuch important summarization methods[12][3].
In this paper, we propose a statistical learning
method which acquires knowledge about the deletion possibility of some of multiple phrases modifying the same verb/noun in order to summarize a document by deleting unimportant segments from a sentence in the document. As a method of summarization
which deletes unimportant segments from a sentence,© 2003 National Institute of Informatics

Ohtake et al.[7][8] proposes a method which deletes
one of two phrases modifying the same noun by using manually constructed rules. Knight et al.[5] andJing[2] and Takeuchi et al.[11] propose methods which
extract rules for reducing a sentence from aligned
corpus of human-made summaries and their original
manuscripts. As for such application of supervised
learning using aligned corpus, Katoh et al.[4] proposesa method which acquires knowledge about paraphrasing by using an aligned corpus of manuscripts for character broadcasting (i.e., a kind of summaries) and corresponding original TV news manuscripts.
However, these previously proposed methods havethe following drawbacks for practical use.
To make a complete list of rules manually in order to delete unimportant segments from a sentence is a hard task.
Aligned corpus between original manuscripts
and summaries is useful. But it is not necessarily available. Moreover, constructing summaries manually for obtaining such aligned corpus is a time-consuming and costly task.
By these reasons, we propose a statistical learning
method which acquires knowledge from a general corpus (e.g., news paper articles), about the deletion possibility of some of multiple phrases modifying thesame verb/noun to summarize by deleting unimportantsegments from a sentence. Note that multiple phrases
denote some phrases modifying the same verb/noun
and a phrase denotes one of the multiple phrases
throughout this paper unless specified otherwise. An
example is shown as follows. In this example, three:forget)”;phrases modify a verb “wasurereu(
















  "!##$&%$ '

#
(

)  * 
+!&,

Example 1: watasitatiha kessite 1 17wo wasuretehanaranai (: We must never forget “1 17”.),: We),
phrase 1: watasitatiha (: never),phrase 2: kessite (: “1 17”),phrase 3: 1 17wo (: forget),modified verb: wasureru (The Third NTCIR Workshop, Sep.2001 - Oct. 2002

Sakai et al.[9][10] propose statistical methods
which acquire knowledge from a general corpus about
the deletion possibility of adnominal verb phrases andphrases modifying a verb, respectively. The method
proposed in [9] acquires knowledge about the deletion
possibility of adnominal verb phrases. The method
proposed in [10] acquires knowledge about the deletion possibility of phrases modifying a verb. However, the method [9] is applied only to the adnominalverb phrases and the method [10] is applied only to thephrases modifying a verb, the method to be proposedin this paper applies to all phrases among the multiple
phrases.
The method proposed by Ohtake et al.[7][8] deletes
one of two phrases modifying the same noun by using manually-constructed rules and the system YELLOW which implemented the method exhibits a good
performance at TSC1 task B in NTCIR Workshop 2.
However, the method can only be applied to sentences
having structures matching the rules and constructing
such rules manually is a hard task. In contrast, ourmethod deletes some of multiple phrases having unimportant contents by using statistical information obtained from a single corpus. Consequently, more flexibility in deletion of some of multiple phrases is attained. Moreover our method can be applied to not
only multiple phrases modifying the same noun but
also multiple phrases modifying the same verb. In general, the multiple phrases modifying the same verb appear more frequently than those modifying the samenoun (as will be seen in Table 2).
Our method extracts knowledge about deletion ofsome of multiple phrases from a single corpus, e.g.,
news paper articles provided as a machine-readable
form, documents obtained from WWW which are easily available.
We participate in the TSC2 task A in NTCIR Workshop 3 to evaluate our method and show results of thistask in this paper. Moreover, we also evaluate ourmethod by recall and precision in addition to the evaluation by TSC2.
We introduce our method in Sec. 2 and its implementation and experiments for evaluation are illustrated in Secs. 3 and 4, respectively. We show resultsof the TSC2 task A in Sec. 5. We analyze the resultsof the experiments in Sec. 6. Sec. 7 concludes thispaper.

2

Proposed method

Our method is based on the intuition that somephrases among the multiple phrases which are easily
associated with by modified verb/noun can be deleted.
:
For example, in “Example 1”, phrase “kessite(

.0/21

&

never) is easily associated with by modified verb “wasuresu(: forget)”.
To reflect the intuition, we calculate the weightvalue assigned to the strength of the relation between a phrase among multiple phrases and a modified
verb/noun by the phrase, and phrases having relatively
small weight value comparing with other phrases
among the multiple phrases are deleted. And, a phrase
having the largest weight value among the multiple
phrases is not deleted. The weight value assigned tothe strength of the relation is calculated by a formulabased on mutual information and it has small weightvalue if the mutual information has a large value. Thatis, the combination of the phrase among the multiple
phrases and the modified verb/noun, which has a large
mutual information value, is frequently contained in a
corpus. Consequently, such phrases are easily associated with by the verb/noun.
We introduce a deletion method of some of multiple
phrases modifying the same verb in Sec 2.1 and deletion of some of multiple phrases modifying the samenoun in Sec 2.2.

2.1

Deletion of some of multiple phrases modifying the same verb

We introduce our deletion method of some of multiple phrases modifying the same verb. An example ofmultiple phrases modifying the same verb is shown as
follows.














  "!##$&%$ '

#phrase 2: kessite (( : never),phrase 3: 1  17 wo ( 3456  : 1  17),
modified verb: wasureru ( +!&, : forget),Example 2: watasitatiha kessite 1 17 wo wasuretehanaranai (: We must never forget “1 17”.),: We),
phrase 1: watasitatiha (

-

#7

This example shows that verb “wasureru (: forget)” is modified by three phrases, which are watasitati: we), kessite (: never) and 1 17
ha (: 1 17). We define such a structure
wo (of a sentence as the multiple phrases and denote one of,
the multiple phrases as
where,

8:9<;>=
ACB @B#DFEHG

@

.?/21

@

IKJML:NPOQNSRKTIKJML:NPOQNSRKT : the phrase containing word L and modified verb R by relation operator O ,L : a word contained in the phrase IKJML:NPOQNSU T andmodifying verb R ,O : the relation operator showing relation between
word L and verb R ,Proceedings of the Third NTCIR Workshop

L

L

@

@
XYZW[

AVBW@2B DEG @
L
L

L

U

O

L

a b

c>dab
c*d&ab

a b

cHd

O

L

j"k

@

AeBf@gBDhEiG
j"k

@

l

R

O

O

Algorithm for deleting some of multiple
phrases modifying the same verb

Based on the above observations, our method of deleting some of multiple phrases modifying the same verb
is formally described as follows:
Algorithm for deleting some of multiple phrases
modifying the same verb

mnJMIKJoLpNqOqNPRKTrTIKJMLpNqOQNSRKTw JoLnTxHysJoIKJML:NPOQNSRKTrTmnJMIKJML:NPOQNPRsTtTvu
z JML:NPOQNPR T{|}q~) JVJML:NPOQNSR2TrT Nz
JML:NPOQNSRKTvu JM { z JML:JMNPL:OPT NPOQz NSJMRKOQNST R2T ToN{

of one of mul[Step 1:] Weight value
is calculated by the following
tiple phrase
formula :where : frequency of all multiple phrases in the corpus,
w JoLnT : frequency of part of speech of L in the corpus, the part of speech of L is either a noun,a verb, an adjective, a conjunction, an adverb,
or a demonstrative pronoun. However,if thew
partw of speech of L is a noun, JoLnT is set tobe JMLT /2, because the frequency of part ofUUshows the relation between wordand verb .
We define as a relation operator. For example, when
: 1 17), isa phrase is 1 17 wo((wo kaku: wo-case)”, because a caseshown as “
(wo kaku: wo-case)”post-positional particle is “in this phrase. Some relation operators are exemplified
in Table 4, where, the symbol of relation operator isbased on the results analyzed by KNP version 2.0b6
which we employ as a parser.
2.1.1

corpus,z JMOQNSR2T : frequency of phrases which modify verb R
by relation operator O in a corpus,\^]YWY_:`W[LU

z JMLpNqOPT : frequency of phrases which include word L
and modify verbs by relation operator O in theRwhere,
is a word modifying verb . The part ofis either of a noun, a verb, an adjecspeech oftive, an adverb, or a demonstrative. However, an ancillary words are excluded (for example suffixes andpost-positional particles., etc.). For example, when a: 1 17),is thephrase is 1 17 wo(
is a noun,is the semanword 1 17. If word
tic code (feature) of the noun, where, we employ the(Tango taikei: vocabulary system)” in the“(Nihongo goi taikei)”[1]
thesaurus “
as a dictionary of semantic codes (features). However,is a noun which does not have a semantic
if wordis the noun. Moreover, if nouncode (feature),
is a compound noun, noun is replaced with a nouncontained at the end of the compound noun. For exis “Tougou(: unification)”, when noun
ample,
is “Shijo Tougou(: market unification)”.
: market unification)” is(The “Shijo Tougou(:
a compound noun composed of a noun:“Shijo(: unification)”).
market)” and a noun:“Tougou(

(1)
(2)

z JoLpNqOqNPR T : frequency of phrases which include word
L and modify verb R by relation operator O in
the corpus,
 http://www-lab25.kuee.kyoto-u.ac.jp/nl-resource/knp.html

speech of noun is too high comparing to thoseof other parts of speech.

yKJMIKJoLpNqOqNPRKTrT : the number of clauses contained inphrase IKJML:NPOQNSRKT .
[Step 2:] mJoIJML:NPOrNSRKTrT is calculated by the fol-lowing formula :mJMIJML#NSOrPNSRKTrTu 2 ~ M mnJMIJML*mnJMNSIKOrPJMNPL RK TrNPT O  NSRKTrT N (3)l    here, verb R is modified by  phrases, that is, verb R
is modified by IKJML NSO NSRKTMNPSPNSIKJML NPO NSRKT .
l l [Step 3:] Delete phrase IKJML#NSOrPNSRKT having weightvalue mJoIJML*NPOqNSRKTrT smaller than a threshold
value predetermined by trial and error.
2.1.2

Explanation of the Algorithm to delete
phrases modifying the verb

We calculate the weight value assigned to the strengthof the relation between a phrase among multiple
phrases and the modified noun in Step 1. The relative values to a phrase in the multiple phrases are calculated in Step 2. And, the phrases having relatively
small weight value comparing to those of other phrases
are deleted in Step 3. Hence, the phrase having thelargest weight value is not deleted among the multiple
phrases.
is the mutual information between
containing wordand verb
phrasemodified by relation operator . If the mutual information has a large value, the weight value
is small, and as a result, the phrase
tends to be deleted. The combination ofand the modified verb having aphraselarge mutual information value is frequently containedin the corpus and such a phrase is easily associatedwith by verb .

R

JMLpNqOQNSRKTIKJML:NPOQNSRKTmJMIJML:NPOQNSR2TrTIKJML:NPOQNSRKTIKJMLpNqOQNSRKT
R

O

L

R

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

Table 1. Some relation operators (multiple phrases modifying the same verb)
part of speech of
relation operators
noun(ga kaku: ga-case)(wo kaku: wo-case)noun(de kaku: de-case)(ni kaku: ni-case)verb(hukugouzi renyou: verb phrases)
adjective(zyaku renyou: adjective phrases)

L

#k
>¢ k
¤b&¥&¦§
¨>¦§

¡j2k
¡<£ k

L

Table 2. Frequency of part of speech ofpart of speech ofphrases modifying verbs phrases modifying nouns
noun
1277730
77754
verb
17820322217
adjective
53918
12709
adverb70479
2243
demonstrative pronoun
15763
4086
conjunction
19088
108
1615181119117Lw JMLT is the frequency of the part of speech of
L modifying verb R in corpus. Thus, it is difficult
phrases containing word L with large
w JoLntoT delete
value (for example, the part of speech of L
is a noun or a verb). But, it is easy
w to delete phrases
containing word L with small JoLnT value (for example, the part of speech of L is an adjective or a
conjunction
an adverb). Moreover, the value ofw JoLnT changesor withphrases modifying the same verbor phrases modifying the same noun. Thus, degreeof deleting some of multiple phrases changes withphrases modifying nouns or phrases modifying verbs.
when we use 66686 documents
Table 2 showsfrom Nikkei newspaper articles from January 1, tois theJune 31, 1993, as a corpus.
.
number of clauses contained in phrase
is kessite(
For example, when phrase: never),. If the phrase
has a large
value, theis hard to delete because its delephrasetion causes serious information loss. Consequently,
are not deleted byphrases with
our method.

w JoLnTKy JMIKJML:NPOQNSRKTrTIsJoLpNqOqNPRKTIsJoLpNqOqNP| RKT.
/g1
yKJMIJMLpNqOQNSRhTrTpu
IKJML:NPOQNSRKTyKJMIKJML:NPOQNPRsTtTIKJML:NPOQNSRKTyKJMIKJoLpNqOqNPRhTrTs©?ª
¬ 

­ 

« 

:
In addition, phrases modifying verbs “suru(: be), aru(: be)” are not deleteddo), naru(
by our method, because, the phrases modifying theirverbs are hardly associated with by them. Next,
phrases modifying verbs by relation operators shownin Table 3 are not deleted by our method. This is because the phrases containing their relation operators
are a subjective case or often have important contents,
then deletion causes serious information loss.

Table 3. Relation operators which undeleted phrases modify verbs by(ga kaku: ga-case)(wo kaku: wo-case)(ni kaku: to-case)(to kaku: to-case)(mi kaku: null-case)( to: to)#k
° £ kk
2.2

2j k
®¯k
±n² ±

Deletion of some of multiple phrases modifying the same noun

We introduce a deletion method of some of multiple phrases modifying the same noun. An example ofmultiple phrases modifying the same noun is shown as
follows.

³"´µH¶2·"¸5¹ $Hºh»5¼5½*¾K¿"À

Example 3: seiken wo obiyakasu yuuryoku na raibaru ga
sonnzai sinai (: There is no strong rival who threatens theadministration exists.),: who threatphrase 1: seikein wo obiyakasu (ens the administration),: strong),
phrase 2: yuuryoku na (: rival),modified noun: raibaru ( $"'³K´#µ5¶ ·¸5º ¹ »"$ ¼5½-

ÁsÂ>Ã ÄThis example shows that noun “raibaru (: rival)” is modified by two phrases, which are seiken wo: threatening the adminisobiyakasu (: strong). We define
tration) and yuuryokuna (such a structure of a sentence as the multiple phrases,,
and denote the multiple phrases as
where,

Å<ÆG*ÇiÈ&«ÉWÊi¬IKJMLpNqOQNSU T

Proceedings of the Third NTCIR Workshop

IKJML:NPOQNSU T : the phrase containing word L and modifying noun U by “relation operator” O ,L : a word contained in the phrase IKJMLpNqOQNSU T andmodifying noun U ,O : the “relation operator” showing relation between
word L and noun U ,O shows relation between word L and noun U . For
example, when the phrase is seiken wo obiyakasu( ÅÆG#ÇpÈ#« : threatening the administration), the O isshown as “ Ë&ÌW¦:Z (doushi rentai: adnominal verb

phrase)”, because the phrase is an adnominal verb
phrase. Some relation operators are exemplified in Table 4.
2.2.1

Algorithm for deleting some of multiple
phrases modifying the same noun

Based on the above observations, our method of deleting some of multiple phrases modifying the same noun
is formally described as follows:
Algorithm for deleting some of multiple phrases
modifying the same nounof one of[Step 1:] Weight value
is calculated by the folmultiple phrases
lowing formula :mnJMIKJoLpNqOqNPU TrTIKJMLpNqOQNSU T
w JoLnT ysJoIKJML:NPOQNSU TrT
z { JML:NPOQNPU>T
mnJoIKJML:NPOQNSU TrTvu|{ }q~) z JVJML:NPOQNSU TrT N
JMLpNqOQNSU TvuÍÎJM { z JMLpJoNqLpOPT NqOqz NPUJMOQNPT U T T{

Table 5. Relation operators which undeleted phrases modify nouns by(ga kaku: ga-case)(wo kaku: wo-case)(to kaku: to-case)(no kaku: no-case)(mi kaku: null-case)(rensetu: adjoin)( to: to)#k° ®¯kk±n² ±mJMIhJMLpNqO  NSU TrT is calculated by the fol-

[Step 2:]lowing formula :mJMIJML  NPO  NSU TrT
u 2 ~ M mnJMIKJoL*mnJoNSIKOrPJMNPL*U>TtNPT OqNSU TrT N (6)l    here, noun U is modified by  phrases, that is, noun
U is modified by IKJML l NPO l NPU>ToNSPSMNIKJML  NSO  NSU T .
[Step 3:] Delete phrase IsJoL*NSOrPNPU>T having weightvalue mJMIJML#NSOrPNSU TrT smaller than a threshold
value predetermined by trial and error.

(5)

where

z JoLpNqOqNPU T : frequency of phrases which include
word L and modify noun U by relation operator O in the corpus,z JoLpNqOST : frequency of phrases which include word L
and modify verbs by relation operator O in the
corpus,z JoOQNSU T : frequency of phrases which include words
and modify noun U by relation operator O in the
corpus, : frequency of all multiple phrases in a corpus.
w JoLnT : frequency of the part of speech of L in the
corpus.w However, if the partw of speech of L is anoun, JMLT is set to be JMLT /2.
yKJMIKJML:NPOQNPU>TtT : the number of clauses constructing
phrase IKJMLpNqOQNSU T .

Ò<²In addition, phrases modifying nouns “koto(:: thing)” are not deleted by ourthing), mono(
method. Next, phrases modifying verbs by relation
operators shown in Table 5 are not deleted by our
method.

Ó Ô

3
(4)

j2k
Ï k
ÐgÑ

ImplementationÕ | ÕÖØ×We implemented our method for participating in
documents from Mainichi
TSC2 task A. We use
newspaper articles from January 1, to June 31, 1998,
as a document set. Because, our method is applied tothe system developed for participating in TSC2 task A
and the Mainichi newspaper articles are employed asa corpus in TSC2 task A. We employ JUMAN version 3.5 as a morphological analyzer, and KNP version 2.0b6 as a parser. Our method acquires knowledge about the deletion possibility of some of multiple
phrases, and deletes some of multiple phrases by using
the acquired knowledge. The phrases deleted by our
method are exemplified as follows. Note that the multiple phrases deleted by our method are underlined.

$&%$ '  g fi 456* Ks! H

Example 4: watasitatiha kessite 1.17 wo wasureteha
naranai (: We must not never forget “1 17”.)
Example 5: tokuni konngo kyouwakokou ga nigiru afurikahyou wo naniganandemo kakuhosuru totikatta (: In particular,
he promised that the Africa vote which Republic ofCongo has was surely secured.)¾sâ$ Û ¾H$ Ùgè"Úé>Û ê Ügë#ÝHì Þh·:ßg,>à>Ù^á í>î2 ,sã#äWåæ5çï http://www-lab25.kuee.kyoto-u.ac.jp/nl-resource/juman.html

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

Table 4. Some relation operators (multiple phrases modifying the same noun)
part of speech of
relation operatorsnoun(no kaku: no-case)noun(to kaku: to-case)verb(dousi rentai: adnominal verb phrase)
adjective(keihan rentai: adnominal adjective phrase)L÷ø Ûù+ú&2û2ü<,ý"þ $ÿ ò2áÏ k®¯kË>Ì¦Z
ðñ&¦Zé+ó ô*ó2õ2ö

Example 6: chuugoku deno hatuno gorin kaisai ni syuunenn wo miseru pekinn nado... (...: ..., including Beijing which shows a deep attachment to hold the first
Olympic Games in China.)

4

Evaluation by TSC2 in NTCIR Workshop 3

We participate in TSC2 in NTCIR Workshop 3 forevaluation of our method and analyze results of thistask in this paper. Table 6 shows the results of TSC2
task A by the summarization system, which uses our
method as an element constructing it. The system isconstructed by an extraction method of important sentences and our deletion method of unnecessary parts
from a sentence. We employ the extraction method
implemented in a system YELLOW[7][8], which exhibited a good performance at TSC task A in NTCIR
Workshop 2. Table 6 also shows the results of TSC2
by the system YELLOW to compare with our method.

5

Table 7. Result of comparing our methodwith YELLOW
MethodPrecision(%) Recall(%) Deleted
Our method
71.2
49.1
73
Yellow
65.8
48.6
79

Experiments for evaluation

We evaluate our method by precision and recall in
addition to the evaluation by TSC2 in NTCIR Workshop 3. For this purpose, we implement our method byemploying  documents from Nikkei newspaper
articles from January 1, to June 31, 1993, as a corpus
and we choose documents for deleting some of multiple phrases by our method. We manually make a correct data set which shows multiple phrases appropriate
to be deleted among the chosen documents. Note thatthe precision and the recall are defined as follows.
 
 


 


 
 

 
ÕÕÕ Õu zz
y O Q #y u
O

J ww T zz
J T

J KTMNJoLnTMN

where,
freq(A): the frequency of the same phrases shown by
the correct data set with phrases deleted by our
method,freq(C): the frequency of phrases shown by the correct data set,freq(M): the frequency of phrases deleted by our
method.

5.1

Evaluation of our method of deletingsome of multiple phrases modifying thesame noun

The method proposed by Ohtake et al.[7][8] deletes
one of the two phrases modifying the same noun byusing 36 manually-constructed rules and the system
by using the method exhibited a good performance at
TSC1 task B in NTCIR Workshop 2. We compare
our method with the method proposed by Ohtake etal[7][8], and call the method YELLOW. We choose documents from  articles of Nikkei newspaper from January 1, to June 31, 1993. And we manually make a correct data set which shows multiple
phrases modifying the same noun appropriate to bedeleted among the 85 documents. There are 135 multiple phrases modifying the same noun in the 157 documents. Table 7 shows the results of comparing our
method with YELLOW. The threshold values of our
method are adjusted so that the recall of our method
coincides with that of YELLOW. Next, Table 8 showsa part of the results of precision and recall which aretocalculated for threshold values changed from 
on our method.

ÕÕÕ Õ

Î Ö

5.2

Î |

Evaluation of our method of deletingsome of multiple phrases modifying thesame verb

We choose 12 documents from 66686 articles of
Nikkei newspaper from January 1, to June 31, 1993.
And we manually make a correct data set which shows
multiple phrases appropriate to be deleted among the
12 documents. There are 199 multiple phrases modifying the same verb in the 12 documents. Table 9
exemplified a part of the results of precision and recalltocalculated for threshold values changed from on our method.


Î Ö

Î |

Proceedings of the Third NTCIR Workshop

Table 6. The results of TSC2 in NTCIR Workshop 3
System C 20% R 20% C 40% R 40%
System by using our method
2.53
2.87
2.60
2.77
System Yellow
2.67
2.97
2.50
2.77
System by using tf
3.30
3.30
3.20
3.10
manually summarization
2.33
2.20
2.10
2.03

Table 8. Result of recall and precision
at multiple phrases modifying the samenoun
Threshold Precision(%) Recall(%) Deleted
0.12
75.4
40.6
57
0.14
76.7
43.4
60
0.16
76.7
43.4
60
0.18
74.6
44.3
63
0.2
75.0
45.3
64
0.22
73.1
46.2
67
0.24
72.5
47.2
69
0.26
70.4
47.2
71
0.28
70.8
48.1
72
0.3
70.8
48.1
72
0.32
71.2
49.1
73
Average
73.4
45.7
66.2

6

Discussion

As shown in Table 7, the precision and the recallof the deletion of some of multiple phrases by our % and %, respectively.
method attain about
Moreover, Table 7 shows that the precision of our
method is superior to that by the method of YELLOW
when the recall of our method is adjusted to coincidewith that of YELLOW. Thus our method is promising
as a component to delete some of multiple phrases.
We consider that the reason why our method outperforms YELLOW is that 36 deletion rules of YELLOW
do not cover all types of multiple phrases modifying
nouns, as they are manually constructed. In contrast,
our method uses the importance for any phrase among
multiple phrases in the documents. Note that the importance is increased when the weight value assigned
to the strength of the relation between a phrase among
multiple phrases and a modified verb/noun has a large
value. However, YELLOW does not use such kind of
information.

×ªÎ

7

ªÖÎ

Table 9. Result of recall and precision
at multiple phrases modifying the sameverb
Threshold Precision(%) Recall(%) Deleted
0.12
75.0
39.0
72
0.14
76.0
41.2
75
0.16
75.0
41.2
76
0.18
75.3
41.9
77
0.2
75.6
42.6
78
0.22
74.7
42.6
79
0.24
75.3
44.0
81
0.26
73.5
44.0
83
0.28
73.5
44.0
83
0.3
74.1
45.5
85
0.32
74.1
45.5
85
Average
74.7
42.9
79.5

provided in a machine-readable form in order to summarize by deleting unimportant segments from a sentence. Actually, our method deletes some of multiple
phrases which are easily associated with. We evaluate
our method, and we conclude that our method is able
to delete some of multiple phrases appropriately, because the precision and the recall attain about 74.0%
and 43.0%, respectively. Experimental results showthat our method is useful for deleting some of multiple
phrases.

Acknowledgment
We express our gratitude to Nikkei Shinbun, Inc.
and MAINICHI NEWSPAPERS, Inc. who permits us
to use the news paper articles in a machine readableform. The authors are also grateful to Nippon Telegraph and Telephone Corporation for permitting useof ALT-JAWS Ver.2.0, a morphological analyzing library for Japanese, to obtain semantic codes from Goitaikei.

Conclusion
References

We proposed a statistical learning method which
acquires knowledge about the deletion possibility ofsome of multiple phrases from a news paper corpus[1] S.
Ikehara,M.
Miyazaki,S.
Shirai, A. Yokoo, H. Nakaiwa, K. Ogura, Y. Oyama, andThe Third NTCIR Workshop, Sep.2001 - Oct. 2002
Y. Hayashi(editors). Goi-taikei. Iwanami Publishing,
CD-ROM edition, 1999.
[2] H. Jing. Sentence reduction for automatic text summarization. In Proc. of the 6th Conference on Applied
Natural Language Processing, pages 310–315, 2000.
[3] H. Jing and K. McKeown. Cut and paste based text
summarization. In Proc. of the 1st Meeting of theNorth American Chapter of the Association for Computational Linguistics, pages 178–185, 2000.
[4] N. Katoh and N. Uratani. A new approach toaquiring linguistic knowledge for locally summarizingjapanese news sentences. Natural Language Processing, 6(7):73–92 (in Japanese), 1999.
[5] K. Knight and D. Marcu. Statistics-based summarization –step one: Sentence compression. In Proc. of
AAAI2000, pages 703–710, 2000.
[6] I. Mani and M. T.Maybury. Advances in Automatic
Text Summarization. the MIT Press, 1999.
[7] K. Ohtake, D. Okamoto, M. Kodama, and S. Masuyama. Yet another summarization system with two
modules using empirical knowledge. In Proceedingsof the Second NTCIR Workshop Meeting on Evaluation of Chinese & Japanese Text Retrieval and Text
Summarization, pages 331–340, 2001.
[8] K. Ohtake, D. Okamoto, M. Kodama, and S. Masuyama. A summarization system yellow for japanesenewspaper articles. IPSJ Transactions on Databases,
43(SIG2(TOD13)):37–47 (in Japanese), 2002.
[9] H. Sakai and S. Masuyama. Unsupervised knowledge
acquisition about the deletion possibility of adnominal verb phrases. In the Proceedings of Workshop on
Multilingual Summarization and Question Answering
2002 (post-conference workshop to be held in conjunction with COLING-2002), pages 49–56, 2002.
[10] H. Sakai, N. Shinohara, S. Masuyama, and K. Yamamoto. Knowledge acquisition about the abbreviation possibility of verb phrases. Natural Language
Processing, 9(3):41–62 (in Japanese), 2002.
[11] K. Takeuchi and Y. Matsumoto. Acquistion of sentence reduction rules for improving quality of text
summaries. In the Proceedings of the Sixth Natural
Language Processing Pacific Rim Symposium, pages
447–452, 2001.
[12] K. Yamamoto, S. Masuyama, and S. Naito. Green: An
experimental system generating summary of japanese
editorials by combining multiple discourse characteristics. Natural Language Processing, 2(1):39–55 (inJapanese), 1995.