Elimination of Multiple Modifiers in Summarization

The results of evaluation

Kiyonori Ohtake and Shigeru Masuyama
Background

Table 1: Example of the elimination rules for double modifiers

Generating a natural and readable summary from asentence.
To eliminate noun modifiers is promising fornatural summary, however, it is not easy to decide
modifiers to be removed.

Elimination of adnominal modifiers
It is difficult to correctly detect adnominal
modifiers from surface information.
Simple eliminations of adnominal modifiers may
cause unnatural and unreadable summaries.
To cope with these points:
1. employ a parser.
2. focus on multiple modifiers.

What is multiple modification?
Two or more adnominal modifiers depend on a noun.
For example (double modifier):  
 (you) (Did you hear this interesting talk?) (interesting) (this) (talk) (hear)

Elimination of multiple modifiers
Did you hear this interesting talk?

Did you hear talk?
Did you hear this talk?
Did you hear interesting talk?

What portion should
we eliminate?
to answer this question

We manually constructed 36 rules,- to avoid unnatural summary,- to avoid wrong elimination caused byparsing errors.
ICCPOL 2001

former modifier (...no)latter modifier

---

---

 (...no)!" (...toiu)  (...no)adnominal clause

adjective

modifiedaction---

Do nothing---

Eliminate the latter (...tono) Do nothing (...no) Eliminate the latter

Summarization system with the method
The summarization system consists of two components:- selecting important sentences;conventional but customized for newspaper articles.
- summarizing each sentence.
+ employ a method of eliminating multiple modifiers;+ employ other five methods to summarize a sentence.
Summarization steps (overview)
1. Decide the importance of each sentence.
2. Summarize each sentence.
3. Select sentences in the order of importance.

Evaluation
We participated in subtask A-2 of TSC (Text
Summarization Challenge) to evaluate thesummarization system.
- TSC is the automatic summarization task ofNTCIR-2 (NII-NACSIS Test Collection forIR Systems-2) workshop.
- NTCIR-2 workshop is a competition-typeworkshop, like TREC.
- TSC chose 30 newspaper articles astargets to be summarized.
- Two summaries, 20% and 40% compression
ratio, were evaluated.
- Nine systems participated in subtask A-2.

Two evaluations were done on subtask A-2.
Subjective evaluation.
- Two points of view: READABILITY and CONTENT- An evaluator scored from 1 (+) to 4 (-).
lengthtype
value (average) ranking
20%
20%
40%
40%

readability
content
readability
content

2.53 (3.16)
2.93 (3.24)
2.73 (3.05)
2.77 (3.12)

1131

Content-based evaluation.
- Compared with human summaries by vector
space method.
- Two kinds of human summary: FREE and PART
lengthtype
value (average) ranking
20%
FREE 0.4727 (0.4418)
140%
FREE 0.6483 (0.6065)
120%
PART 0.5137 (0.4740)
140%PART 0.6608 (0.6342)
1Discussion

compression ratio* =summary length
source length

Compression ratio* of summarization for sentences=91%
Is the proposed method effective?
times toworknum. of charcters
eliminatedElimination of double modifier

61729

Elimination of supplementary
explanations
Elimination of the first sentence
in direct quotations
Elimination of direct quotations

335

662

7277

121

Elimination of illustrations
Paraphrasing

689

126
314

methodThe method may be effective, but we found unnaturaleliminations with‘‘adnominal clause - noun+no - modified’’
Rigid evaluations for each method are left for future work.