Yet another summarization system with two modules using empiricalknowledge
Kiyonori Ohtake ,Daigo Okamoto,
Mitsuru Kodama,
Shigeru Masuyama
Department of Knowledge-based Information Engineering,
Toyohashi University of Technology,
Toyohashi 441-8580, Japankohtake@slt.atr.co.jp, {okamoto, kodama}@smlab.tutkie.tut.ac.jp,
masuyama@tutkie.tut.ac.jp

Abstract

system is different from that of GREEN which aims to
make natural summaries.

We previously proposed a summarization system,
GREEN, for Japanese newspaper editorials. However,
GREEN is not suitable for summarizing ordinal newspaper articles which are different from newspaper editorials. To participate in subtasks A-1 and A-2 of
TSC (text Summarization Challenge) in NTCIR-2, wedeveloped a new summarization system from scratch
which copes with both ordinal articles and editorials
in a Japanese newspaper.
The new summarization system resulted in goodevaluations: the mean value of all evaluations held
the foremost place among ten systems in subtask A-1
and nine systems in subtask A-2, respectively.
Keywords: Summarization System, Deletion of modifiers, Extracting sentences, Abstracting

The system was evaluated by extraction of important sentences in subtask A-1 and is evaluated by comparing with human-made summaries in subtask A-2.

1

Introduction

We previously proposed a summarization system
GREEN for Japanese newspaper editorials. It chooses
sentences stating opinions as important based on dependency structure, and summarizes by sentences reduction and deletion of noun modifier, etc. However,
GREEN is not suitable for summarizing ordinal newspaper articles, because the news-report-styled newspaper articles are different from newspaper editorials.
Thus, we developed, from scratch, a new summarization system which copes with both ordinal articles
and editorials in Japanese newspaper articles to participate in subtasks A-1 and A-2 of TSC in NTCIR-2.
The new summarization system is designed to avoid
omission of important information, namely to makeinformative summarization. Therefore, the aim of theCurrently with ATR Spoken Language Translation
Research Laboratories.

The system was composed of two components,
an extract-type summarizer for subtask A-1 and an
abstract-type summarizer for subtask A-2.
The extract-type summarizer chooses importantsentences due to a level of importance attached foreach sentence. And it outputs the selected sentences.
Some features on surface information decide thelevel of importance for each sentence. Moreover, different weight are attached according to whether the input is a newspaper article or a newspaper editorial.
The abstract-type summarizer summarizes sentences by deleting multiple modifiers for nouns andillustrations and by paraphrasing. For this purpose, it
employs a parser KNP.
Our new summarization method focuses on multiple modifiers to make natural summary. Mikami
et al.[4] also proposed a method which summarizes
each sentence by deleting noun modifiers. However,it sometimes deletes some modifier whose removalcauses loss of important information. Consequently,
we make our method prudent so that the system does
not delete important information.
In the field of automatic summarization, there
are some researches which only use surface
information[5].
Yamasaki et al.[11] and Wakao
et al.[9] proposed a method of paraphrasing for TV
news manuscript. And Kodama et al.[3] proposed
extraction of summarization knowledge from direct
quotations.
The system also employs a table to paraphrase some
expressions to more concise expressions. Moreover,the system adopts a method of kodama et al.[3] andtry to eliminate the direct quotation.

End expression of a sentence

(want) 
(want)(think)  (consider)!#" (hope) %$'&()"'*+(may),.- (importance) 0/1 (need) 32546 (expectation ) 8796 (regrettable):<;= (attention) 3> ?! (subject) @BAC (should) EDCF% (should)Terms

Table 1. Example of dictionary of opinion sentences

2

System configuration

3.1

This system is implemented on Vine Linux 2.0 using Perl. The system comsists of two components: an
extract-type summarizer and an abstract-type summarizer.
In the extract-type summarizer, some features, main
terms, high frequency words, location information ina paragraph, etc., decide the weight of each sentence,and the predetermined number of sentences are selected from the sentences with the longest weight.
In the abstract-type summarizer, the system selectssentences to suit the predetermined number of characters based on the weights of sentences by the extracttype summarizer. And this part summarizes each sentence with the KNP.

Morphological analyzer

In the morphological analyzer, the system employsa morphological analyzer, JUMAN1 . An analyzed
result is used for determining importance factor ofwords, extraction of main terms and determining importance factor of sentences.

3.2

Determining importance factor of terms

We consider that high frequency words in an articles are strongly related to the author’s opinion. Thus,
these words are important. However, these words areunimportant if they appear in too many other articles.
Thus, the importance factor of each term is decided by
the following expression.
Word frequency in articles

3

Extract-type summarizer

The extract-type summarizer is composed of fivecomponents illustrated in Fig. 1.

Original articles
Morphologicalanalyzer

Lexical Properties
of Japanese

Analyzed results

Deciding importancefactor of terms

Extractionof main terms

Importance factorof terms

Main terms
Deciding importancefactor of sentences
Output of importantsentencessummary result

Figure 1. The outline of the extract-typesummarizer

Importance factor =log(Word frequency in Lexical Properties of Japanese)

This formula is based on the idea of t f G id f ,where the word frequency of Lexical Properties ofJapanese[1] is used instead of id f to reflect the commonness of the term.
The Lexical Properties of Japanese contains frequency counts for terms and characters which appeared in all articles in 14 years (1985 - 1998) of the“Asahi” newspaper. 340,000 words are extracted bymorphological analysis, and the frequencies of occurrence for each word and character are counted.

3.3

Extraction of main terms

We define main terms as nouns, which are implicated in the theme of the article. We can assume thata headline of an article is an ultimate summarizationof the article. In addition, a head line includes main
terms for the article.
Thus, we regard all nouns in the headline as themain terms. In addition, we regard undefined terms
at morphological analysis written in KATAKANA oralphabet as nouns.
In GREEN, proper nouns are considered main
terms, and is also defined by employing a thesaurus,
KADOKAWA RUIGO SHIN JITEN[6]. However, we
1

http://www-lab25.kuee.kyoto-u.ac.jp/nl-resource/juman.html

do not employ any thesaurus in the system, because
we consider that all nouns in the headline are relevantto the theme of the article. In addition, we supposethat each relevance level of the theme is in proportion
to the importance factor of terms. Moreover, we suppose that if a sentence contains some nouns included inother sentences previously judged important, the sentence would be as important as other important sentences.
The system extracts the main terms from different
location depending on whether newspaper articles or
editorials are treated. We consider that the first paragraph of newspaper articles contains important terms,
because an author tends to write important terms in
the first paragraph. We suppose that the first sentence
in a newspaper article and the last opinion sentence inan editorial contain a lot of important terms. Here, anopinion sentence will be defined in subsection 3.4.6.
This is because, the first sentence in a newspaper
article offers new information to readers having few
knowledge about premise information, and the last
opinion sentence in an editorial expresses a conclusion. Thus, for the newspaper articles and editorials,
a noun contained at the location below is defined as amain term.
H

the first paragraph in newspaper articles
H

the first sentence and the last opinion sentence ineditorials

3.4

3.4.3

In news paper articles, the first sentence of an article isimportant[2]. In addition, more important information
tend to be written at the beginning of the article. Thus,
we should attach weights to paragraphs in and aroundthe beginning of the article.
Meanwhile, the last paragraph in a newspaper article contains information that readers consider to beinterest. The system attaches weights to the last paragraph.
3.4.4

The first sentence of each paragraph offers new information to readers who do not have premise information. The last sentence of each paragraph is writtenwith special intention to conclude the paragraph.Thus,
the system attach large weights to the first sentence
and the last sentence of each paragraph.
3.4.5

Unimportant sentence

There are some unimportant sentences in articles suchas titles of paragraphs, supplementary explanations,
etc.
These sentences tend to contain the signs(e.g.,I, J , etc.) peculiar to the newspapers. In addition,
direct quotations can be understood by pre and postcontexts.
We defined sentences containing the signs and the
direct quotations as unimportant sentences and attached small weights on it.
3.4.6

Main terms

If a main term in a sentence is a subject term(having
the nominative case), we attach heavy weights on thesentence.
3.4.2

Location information in a paragraph

Deciding importance factor of sentences

The system decides importance factor of each sentence by employing the following features: the existence of main terms, the importance factor of terms,
a structure of articles, location information in a paragraph, etc. Each parameter is computed by summing up values decided empilically. However, newspaper articles are different from newspaper editorials
in some features, i.e., sentences which state facts are
more important and sentences at the beginning of anarticle are important.
3.4.1

Structure of articles

High frequency word in articles

We suppose that terms having high frequency in anarticle are relevant to insistence of the author. Thus,
sentences containing high frequency terms are important. Therefore, the system gives weights to sentences
if they have high frequency terms.

Opinion sentence and phenomenon sentence

Sentences in articles are classified into two groups[10].
A sentence in the first group tells one’s opinions andthat in the second group tells facts. An opinion sentence is defined as a sentence which expresses author’s
insistence, opinions or hopes. A phenomenon sentence is defined as a sentence which expresses accidents, facts or phenomena.
In editorials, opinion sentences tend to be important. Thus, we adopted different weighting for editorials from newspaper articles.
To extract the opinion sentences, we should pay attention
to expressions of the end of sentences, i.e., 8
K/ 1%L5M  (need) , #N @6A#L5M (should) .
The system extracts opinion sentences by matching topattern of 55 rules, based on a method of GREEN. The
pattern table defines dictionary of opinion sentences.
The dictionary for extracting opinion sentences is
illustrated in Table 1.

3.4.7

Weighting importance factor

The importance factor of each sentence is computedby summing up points that are shown in Tables 2, 3

Condition
Subjects aremain terms
Containing main terms
Containing
high frequency wordFirst sentencesof each paragraph
Last sentencesof each paragraph
Unimportant sentences

Point
Importance factorof terms O 10
Importance factorof terms O 2
Importance factorof terms O 1
20

Condition
First paragraphOpinion sentences
ExceptPoints20
100Table 4. weighting for editorials

4

Abstract-type summarizer

10
Importance factor
of sentence :1/10

Table 2. Weighting in common with newspaper articles and editorials

The abstract-type summarizer is composed of two
components, a component of summarizing a sentence
and a component of selecting sentences. An outline ofthe process in the abstract-type summarizer is shownin Fig. 2.
Original articles

Extract-type summarizer
Condition
First paragraph
Second paragraphThird paragraph
Last paragraphExcept

Point
100
50
20
importance of sentence O
0

Summarizing
sentence

Deciding importancefactor of sentences

Result of summarizing
sentence

Preference rankingof each sentence

10
Important sentence
selection

Table 3. Weighting for newspaper articles
Summary resultand 4(but only an unimportant sentence is divided bypoints attached to it).
These values of weighting parameters are decided
by the heuristics based on the result of DRYRUN.
Thus, we can say that the parameters are suitable for
evaluations in NTCIR-2.

3.5

Output of important sentences

Figure 2. The outline of the abstract-typesummarizer

4.1

In summarizing sentence, the system summarizes
each sentence by six methods based on syntax analysis
by a parser, KNP.
H

Deletion of supplementive explanation

To output important sentences, the system fixes anorder of priority based on the importance factor ofeach sentence, and selects the sentence to the predetermined number of sentences.
H
H

Deletion of expression of direct quotation
Deletion of multiple modifiers for nouns
H

Deciding preference ranking In deciding preference ranking, the system fix an order of the priority inorder of the importance factor of each sentence. But
we decide that the ranking of first sentence in articles
is the first without the importance factor.

Output sentences The system selects sentences inaccordance with established priorities and outputs
them.

Overview of summarizing sentence

H

Deletion of illustration

H

Paraphrasing
Deletion of the head of sentences conjunction

These processings are based on the following concepts.
H

The system summarizes each sentence by deleting parts of prolixity and does not process plural
sentences as a unit.

H

The system aims at not deleting an important information, and retains the naturalness.

Example
2
¡#¢

+

D£%¤+Tw!¥¦

¢

§%¨

¬

«

D­3®¯
·C¸
t_¹

m§.¨

°!±L+U5²
m»
º
#«C¼

ml³j´

q©6ª()«+6

KV¬«+%.qµ¶

Moreover, the system employs a parser KNP without using the verb dictionary of IPAL[8]. This is because, the number of verbs in the dictionary of IPAL
does not reach a practical level for the purpose of theown system, and that verbs in IPAL have different effect on an accuracy of analysis from verbs not in IPAL
if we use KNP with IPAL option.

NÀq} ( Although the former vice president side has submitted
the revocatory written statement to criminal investigation, prosecutors claimed “the fact was accepted in
the criminal-investigation stage”, and they are planning to make the evidence application with precise
timing.
)


4.1.1

§%¨

+

Deletion of supplementive information

There are many supplementive expressions that in circle parentheses are FURIGANA or abbreviation, etc.
in articles.
Thus, the system deletes supplementive expressions
in circle parentheses or can be distinguished fromother portion by mark( J , P6Q , etc.) .
4.1.2

Processing of expression of direct quotation
employing heuristics

An expression of direct quotation is unimportant, andwe can understand a content without it by seeing preand post contexts. The first sentence in direct quotation explains facts, the other sentences states opinions.
Thus, we attach importance to paragraphs at the utterance of opinions. If two or more sentences are included in a direct quotation, the system deletes the first
sentence. However, if the first sentence is connected tothe second sentence by demonstrative, the system does
not delete the first sentence, as doing so makes resulting summaries unnaturally.
Example 1
RS.TVU+W%X5YZ=[\]_^T.` a%bVc'L.d e f6g
hjilkm&n#oV^5L B(qp%r(s!tL+uvxw!yz)"
K5${z|+}*~T,T

!#z*vN

(“It was taken up in mass communications when the rate
rise of a consumption tax was planned with the two
person tripod of our Administrative Vice-Minister and
Mr. Ozawa. So, it will become a lie if I say that I’m
not  interested in the relation with this minister.” )~TV,'qT

_+#

K

¡#¢
¢
D

½j¥¿¾

KV¬
m§.¨
D£%¤+Tw!¥¦q©6ª()«+6
·C¸
m»
t+¹ º
.«V¼½{¥¿¾5NlÀl} (Although the former vice president side has submitted
the revocatory written statement to criminal investigation, prosecutors are planning to make the evidence
application with precise timing. )4.1.3

Deletion of multiple modifiers for nouns

Two or more adnominal forms, which modify one
noun, are defined as a multiple modifiers. In particular, two parts which modify one noun is defined as
double modifiers. In this system, since most multiple
modifiers are double-modifiers, only double-modifiers
are treated. Therefore, the case when three or more
modify a noun, is not treated.
When there is a double modifier, even if one of
the two clauses is deleted, a meaning is assumed notchanging seriously in many cases, and one of adnominal forms is deleted.
However, it is dependent on the kind of each part ofspeech which composes a clause where the former andlatter adnominal part are deleted.
Thus, we employ rules for the elimination of an adnominal part of double modifiers. Each rule consistsof three components, former adnominal part, latter adnominal part and modified.
In the system, if the rules created by the heuristics
matched each part, a latter adnominal part is deleted.
Otherwise, a former adnominal part is deleted.
In addition, rather than the verbs, deleting an adjective preferentially etc., we consider that naturalness is
not spoiled as important.
The rules are illustrated in Table 5.

*#=!_zC5.*v)N

( “It will become a lie if I say that I’m
not interested in the relation with this minister.” )In addition, there is another case that the next expression of a direct quotation is a summary of thequotation[3]. In this case, the sentence expresses astatement that the same opinions are expressed by bothclauses and sentences.
When the statement matches some pattern made byheuristics, we suppose that the sentences are retained
natural, even if it deletes all parts of direct quotations.

The example of deletion of multiple modifiers fornouns is shown below.
Example 3
Á%Â K#Ã
¬ Á%Â
Ë

KCÌ#Í

.Ä

Å_*v

Æ5N{Ç

È(É6*

Ê

«C!}( A political party serves as a light
existence and the chronic distrust to a political party is
whirling.
)Ë KVÌ+ÍÁ Â KÃ¬ Á Â#Ä+ÅZ_*v

«

B¿}

Æ.N¿Ê( A political party serves as a light existence
and the distrust to a political party is whirling. )FormermodifierÎ ÏÐ%Ñ Ò

Lattermodifier¸ Ò Ó ÏModifiedÔÏ(participial adjective
formof verbs)
Î ÏÐ%Ñ Ò(i-adjective)(noun)

ÕVÒ

ÔÏ

Ó

Ï

(participial adjective) (na-adjective)
form of verbs)
6T (no)Ö×(quantity)q (toiu)T (no)“-” means do not care.

(noun)
6T(no)Table 5. Example of rules which deletesa back modifier element

We suppose that the parser KNP have a possibilityof mistaking analysis, and we cope with the mistaken
result.

Processing of the structure with the possibility ofmultiple modifiers Multiple modifiers may not becorrectly detected due to the analysis error of KNP.
In this system, when in the sentence of the structure of “adnominal clause - noun + T - modified”, irrespective of the syntactic-analysis result of KNP, it isregarded as multiple modifiers.
However, if the result by elimination of the adnominal clause is unreadable, we would do nothing. Thisis judged from the noun in “noun + T ”. If the noun
is abstract, the attributive function of “noun + T ” will
not affect the modified. Thus, if the noun is abstract,
we will do nothing. Whether a noun is abstract or not
is judged from a thesaurus, Goi-Taikei[7]. To obtainsemantic codes from the Goi-Taikei, we employed the
ALTJAWS Ver.2.0: a morphological analyzing libraryfor Japanese.

4.1.4
Processing of syntax over a thematic part In syntactic analysis, a modifier part does not accept the relation over a thematic part. Because, such sentence of a
relation is easy to mistake during the analysis by KNP.
In addition, the thematic part is defined as a clauseended by the thematic particle or the collection particle.
Exception handling by the pattern match Wheneach element of double modifiers is the structure
which KNP tends to mistake, exception handling using fine rules is needed.
When double modifier elements matched the rules
created by the heuristics, the system does not delete
the double modifier elements.
The example of a rule which does not delete thedouble modifier elements is shown in Table 6.

Deletion of illustration

Illustration is considered to be a modifier in a broad
sense, and it is assumed that a sentence sense is notchanged by its removal.
This system deletes clauses of “  *!ÜT (nadono)”and clauses of “ (*ÜL (nadode)” which modify averb.
Example 4ÓæÝÞ¬ á â.ã
Lußvà].{z"
a.ä%å
*jÜlT
Ý(çè
¬
Léjêë]#"ì5íîqï{ðT%ñjïÙD
óò
mõZö
^'ô
j (..÷ø­«ù5

ú} ( The boy, the third-year student in a juniorhigh school, was captured on the spot, and arrested
in the act of burglar attempted murder suspicion etc.
It is said that he stated “I thought that a handgun
would be taken.” )


ÝÞ
L

uv)àÙ]¬jz)"¬.Ý5ç6è

L.é5ê]"'BìCímCõöModifiedîûï6ðTCñ5ïBDüò+^CôV(V {)÷ø<l« B(qý} ( The boy, the third-year student
lT (tono) in a junior high school, was captured on the spot, andØ (koto)arrested. It is said that he stated “I thought that a handgun would be taken.” )(verb)(i-adjective)
4.1.5 Paraphrasing¸ Ò Ó Ï6T(i-adjective)(no)
A tedious expression in an article can be summarizedK." (reru)(ga)byparaphrasing of making a briefer expression.
Ú Û Ï
6T
This system paraphrases based on 96 rules created(demonstrative) (no)with the heuristics focusing on expressing briefly in“-” means do not care.
the end of a sentence.
However, the rules are newly created aiming at theTable 6. Example of rules which does notcustomizing for a newspaper article, as the rule credelete
ated by paraphrasing of Wakao et al.[9], Yamasaki etal.[11] were customized for TV news articles.
The Paraphrasing rules are illustrated in Table 7.

Former modifier6T (no)6TD (noha)
Î Ï(verb)
Î Ï

Latter modifier6T (no)* (na)qÙ (toiu)´
%
T (tameno)¸ Ò Ó.Ï

10%
30%
50%
ave

All articles
Recall(%) Precision(%)
33.7
33.7
45.1
45.1
61.2
61.2
46.7
46.7

Only newspapers
Recall(%) Precision(%)
47.8
47.8
49.2
49.2
63.4
63.4
53.4
53.4

Only editorials
Recall(%) Precision(%)
19.6
19.6
41.1
41.1
59.0
59.0
39.9
39.9

Table 8. The evaluation result in subtask A-1

Beforeparaphrasingþ

Afterparaphrasing
þ(will enter)(enters)

#ÿ!oú}%Ø&V*%¿}(also become changing)



#}

z)"'*+%L}(declaration)%L'&+M6¿}(also
an aim)
´



¿}

(changes)(announced that)



¿}

(¿}

(have not been decided)
zl*+j=ÿ}

+}



(aim)
´

z)"5FV}(not decided)
zl*++}(seems that it is not decided)(not decided)Ø¿L6M6¿} dearu)(dakara  dearu)(ni  narudarou)

Ø)}

(koto

(koto)

ÿ5${zL6M6¿}*#ÿ!oú} da)ÿ5${zÿ#}(dakara}(ni)

Table 7. Example of paraphrasing rules

Step 1. Place sentences in the order of importance by the
extract-type summarizer.
Step 2. Adopt the summarization method to all sentences
except paraphrasing.
Step 3. Select sentences in order of importance to makea summary until the total length of summary exceeds the predetermined length of summary.
Step 4. Adopt the summarization method other thanparaphrasing to all sentences again.
Step 5. If the total length of the summary is less than thepredetermined one, the summary would be outputted and processing would be terminated.
Step 6. Apply the paraphrasing by tables.
Step 7. If the total length of the summary is less than thepredetermined one, the summary would be outputted and processing would be terminated.

Step 8. Eliminate the most unimportant sentence, which
was selected as the last sentence at step 3., from
4.1.6 Deletion of the head of sentences conjuncthe summary, and search a suitable sentence,tion
whose adoption satisfies the length constraint inorder of importance from sentences which have
The system does not consider relation among sennot been selected yet.
tences into consideration, in order to extract importantsentence. Therefore, the conjunction at the head ofa sentence does not act its primary role in produced Step 9. If adoption of a sentence causes violation of thelength constraint, i.e., the total length of selectedsummary.
sentences exceed the predetermined length, weThus, all the conjunctions of the head of each senwould adopt the paraphrasing to the sentence, andtence are deleted in this system.
the summary would be outputted and processing
would
be terminated.
4.2 Important sentence selection
In an important sentence selection, the sentence ischosen based on each sentence summarized in the sentence reduction and preference ranking of each sentence. Here, the preference ranking of each sentenceis the value calculated in the preference ranking of theextract-type summarizer.
The algorithm of the important sentence selection
is shown as follows.

Algorithm of the important sentence selection

Step 10. If addition of a sentence satisfies the length constraint, the summary would be outputted and processing would be terminated.

5

Evaluation

We participated in subtasks A-1 and A-2 among
the tasks of TSC(Text Summarization Challenge) in
NTCIR-2, and evaluations on the extract-type summarizer and the abstract-type summarizer were performed.

5.1

Extract-type summarizer

Subtask A-1 evaluated a summary on the basisof the coincidence between the important sentences
which man chose.
The following two formulas were used as evaluation measures.

Recall =The number of texts for which
the subjects judged correctly as relevant
The total number of relevant texts

Precision =The number of texts for which
the subjects judged correctly as relevant
The total number of texts judged
as relevant by subjects

These values are calculated for every rate of a summary(10%,30%,50%).
The averaged results and the results on limitationof newspaper articles and editorials, respectively, areshown in Table 8. Consequently, it turns out that the
precision of important sentence extraction of an editorial is low as a whole, compared with newspaper articles, and in particular, the precision over the editorial
in the case of summarization to 10% is extremely low.
It is mentioned that the parameter about the opinion
sentences is small and description is insufficient in the
dictionary of the opinion sentences as a cause.
By expanding the dictionary of opinion sentences,the precision of the summary to 10% with an editorial is improved to 24.3%, and the whole average is
improved to 47.4%
However, even if it makes the parameter about theopinion sentences heavy and it adopts the opinion sentences compulsorily like GREEN, the improvement in
the precision beyond this is difficult. This means thatthe opinion sentences are not necessarily important ineditorials. Thus, it is necessary to classify opinion sentences more finely to the following two classes: thesentence which expresses the opinion of an author andthat states the opinion on future development. In addition, it is also necessary to give the different weighting.
Moreover, there was an error by the forcible adoption of the first sentence about editorials. The first
sentence in editorials does not necessarily become thewhole outline, which depends on the author. Therefore, it is necessary to adopt the first sentence after
judging whether it shows the whole outline.
As for the newspaper articles, a good result was obtained compared with the editorials. However, when
seen for every article, there was the articles with low
precision. Although important sentence extraction to
the newspaper articles of such important composition
that it is close to the beginning, it may perform inad-

equate extraction to other articles. Therefore, the articles are not simply classified into the editorials andthe newspaper articles, the measure of classificationaccording to the composition of the articles finely isrequired.

5.2

Abstract-type summarizer

For subtask A-2, two kinds of evaluations, subjective evaluation and content-based evaluation, were
performed.
In the subjectivity evaluation, the evaluator (oneperson) read the summaries texts by the system. Then,evaluate and score them in terms of how readable they
are, and how well the content of the text is describedin the summary. The scores are one of 1, 2, 3, and 4
where 1 is the best and 4 is the worst, i.e., the lower
score, the better evaluation is.
In the content-based evaluation, morphologicalanalysis was done to the system results and human summaries, and only content words(morpheme),
which are nouns, verbs, adjectives, and undefinedwords, were selected. Moreover, term weights in eachsummary were calculated by t f G id f measure. Thenthe distance between the document vector of humansummary and a system result were computed by thecosine of the angle between vectors, and we observed
how close the two summaries based the content word.
Moreover, the following are two kinds of correct
answers as a summary: the summary in which human
did free creation (following, FREE), and the summary
which human created by important part extraction (following, PART). Each evaluation was performed by
specifying the predetermined number of words that it
becomes rate of summary 20%, and 40%.
The result of the subjective evaluation is shown in
Table 9Readability 20%
Contents evaluation 20%
Readability 40%
Contents evaluation 40%

Evaluation value
2.53
2.93
2.73
2.77

Table 9. Subjectivity evaluation

FREE20%
FREE40%PART20%PART40%Evaluation value
0.472670
0.648264
0.513655
0.660800

Table 10. Content-based evaluation

In the subjective evaluation, although the summary
to 20% was more readable than the summary to 40%,the content-based evaluation exhibits a little worse result.
As for the content-based evaluation, the summary
to 20% was not similar to a correct answer as a summary from the summary to 40%, and it was with thebad result so that the rate of a summary was low like
subjectivity evaluation.
This is because the sentence is chosen in the
abstract-type summarizer on the basis of the preference ranking searched for in the extract-type summarizer. Therefore, the result in extract-type summarizer
influences subjective evaluation in abstract-type summarizer.
Moreover, precision of FREE is lower than that ofPART. The advanced summary is required in order toraise the precision to FREE. In order to improve precision to FREE, an advanced summary method like thesummary used in case human does a free summary is
required.
5.2.1

Summarization rate

The contribution to summarization rate of the deletion (2129 characters) of redundant part performed in
the abstract-type summarizer to the sentences (22812
characters) extracted by the extract-type summarizer isabout 9%.
In this system, in order to prevent lack of important information, bold deletion was not performed but
a prudent summary was performed. This strategy is
successful and resulted also in good evaluation.
5.2.2

Comparison of the effect of each method

The abstract-type summarizer consists of six methods.
As a result of investigating the number of deletion ofcharacters by each method independently is shown in
Table 11.
There are many deletion characters by the deletion of supplementive explanation among six methods.
This is because circle parenthesis expression is usedabundantly in Mainichi Newspapers. Moreover, thedeletion of supplementive explanation is very effective
because of the additional information like abbreviatedname or the FURIGANA of a difficult Chinese character.
The processing of expression of direct quotation
was the result on the whole being hard to use. This isbecause importance tends to be attached to direct quotation expression and each sentence is hard to adopt inarticles.
The processing of expression of direct quotation isclassified into deletion of a first sentence, and deletionof the whole direct quotation expression. However,most frequently used was deletion of a first sentence.
In order that this method may delete the all sentences

in a parenthesis, it has many deletion characters andtends to maintain naturalness. Therefore, it is an effective method.
On the other hand, deletion of the whole direct quotation expression is used only twice in 30 articles, andthe result is hard to be called effective. It becomes a
subject to consider how the patterns are increased.
Although the deletion of a double modifiers deletedmany characters at once, it had the case where naturalness was spoiled by excessive deletion. Therefore, it isnecessary to decide and control a rule more carefully.
Moreover, in this system, in order to cope with errors of a syntactic-analyzer, the structure of the sentence which is easy to carry out an analysis error took
the object of safe serious consideration of not deleting. But it will be a difficult subject to consider how
to cope with it from now on.
Deletion of illustration is used little number oftimes although its deletion is successful as it maintains
naturalness. This reason is that importance information is not attached to the illustration.
Although only a few characters are deleted at eachapplication of paraphrasing, since it is applicable tomany sentences, it is an effective method in a summary. However, since the table for paraphrasing is created manually, in order to make it applicable to moresentences, it is necessary to extend a table automatically.

5.3

On naturalness

In GREEN, cohesion analysis was performed on thebasis of the abbreviation of a key word or a subject
etc., and the sentence with the above sentence and cohesion took the method of adopting the previous sentence. This is for preventing a summary result frombecoming unnatural.
Since this system considered that extraction of theimportant information rather than the naturalness of asentence, the method, used in GREEN, to retain cohesions on all selected sentences was not taken. Therefore, it is inferior to GREEN from the viewpoint of
naturalness, and may summarize unnaturally.
For example, if the sentence where the object is described to be is not adopted when the demonstrative
is contained in the sentence, it becomes unnatural asa sentence. Moreover, when a theme changes in anarticle and only the extracted sentence is read, there
is also a problem that a relation of sentences becomesambiguous in the portion of relation of a theme.
How to solve such a problem is a future subject.
However, in the subjectivity evaluation, the evaluation
value of readability was about 2.6(average value of
20% and 40%).
This is because minimum disposals, such as deletion of the conjunction in the first of sentences and theforcible adoption of the first sentence, are performed.

Method
Deletion of supplementive explanation
Processing of expression of direct quotation
Deletion of multiple modifier for nouns
Deletion of illustration
Paraphrasing
Deletion of the head of sentences conjunction

Number ofdeletion characters
662
298
729
126
262
52

Rate(%)
31.1
1.4
34.2
5.9
12.3
2.4

Number oftimes to use
335
8
61
6
74
15

Number of meandeletion characters
1.97
37.25
11.95
21.00
3.54
3.47

Table 11. Comparison by the methods

Moreover, the fact that the middle sentence inwhich a demonstrative tends to occur frequently is
hard to adopt is also one of the reasons of weighting
to the first sentence and the last sentence of each paragraph.

6

Conclusion

The summary system customized for both newspaper articles and editorials was introduced. We participated in subtasks A-1 and A-2 of TSC in NTCIR-2.
The evaluated results on the system were the best in
average among all participants for both subtasks A-1
and A-2.
To improve the naturalness by employing a method
to retain cohesion on a produced summary is left for
future subjects.
Acknowledgements
The authors are grateful to Nippon Telegraph andTelephone Corporation for permitting use of ALTJAWS Ver.2.0, a morphological analyzing library forJapanese, to obtain semantic codes from Goi-taikei.

References[1] S. Amano and T. Kondo, editors. the Lexical Properties of Japanese. SANSEIDO Co., Ltd., 1999.
[2] M. Hirai. The Encyclopedia for Writing. SANSEIDO,
1984. (in Japanese).
[3] M. Kodama, A. Kataoka, S. Masuyama, and K. Yamamoto. The automatic extraction of summary knowledge using direct quotation expression. In Proceedingsof The Sixth Annual Meeting of The Association for
Natural Language Processing, pages 241–244, 2000.
(in Japanese).
[4] M. Mikami, S. Masuyama, and S. Nakagawa. A summarization method by reducing redundancy of eachsentence for making captions of newscasting. Journalof Natural Language Processing, 6(6):65–81, 1999.
(in Japanese).
[5] M. Okumura and H. Nanba. Automated text summarization. Journal of Natural Language Processing,
6(6):1–25, 1999. (in Japanese).

[6] S. Ôno and M. Hamanishi. Kadokawa Ruigo Shinjiten(Kadokawa New Thesaurus). Kadokawa PublishingCo., 1981.
[7] S. Ikehara and M. Miyazaki and S. Shirai and A.
Yokoo and H. Nakaiwa and K. Ogura and Y. Oyama
and Y. Hayashi, editors. Goi-taikei. Iwanami Publishing, CD-ROM edition, 1999.
[8] Information technology Promotion Agency(IPA). IPA
Lexicon of the Japanese Language for computers IPAL
(Basic Verbs), 1987. (in Japanese).
[9] T. Wakao, T. Ehara, and K. Shirai. The technique ofthe summary looked at by the title of a television news
program. In IPSJ SIG Notes NL-122-13, pages 83–89,
1997. (in Japanese).
[10] K. Yamamoto, S. Masuyama, and S. Naito. S.GREEN:
An experimental system generating summary ofJapanese editorials by combining multiple discourse
characteristics. Journal of Natural Language Processing, 2:39–55, 1995. (in Japanese).
[11] K. Yamasaki, M. Mikami, S. Masuyama, and
S. Nakayama. The title generation summary by paraphrasing for hearing-impaired person. In Proceedingsof The Fourth Annual Meeting of The Association for
Natural Language Processing, pages 646–649, 1998.
(in Japanese).