Proceedings of the Third NTCIR Workshop

Sentence Extraction by tf/idf and Position Weighting from Newspaper
ArticlesYohei SEKI
Dept. of Informatics, The Graduate University for Advanced Studies (Sokendai)
2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan
Dept. of Integrated Information Technology, Aoyama Gakuin University
6-16-1 Chitosedai Setagaya-ku, Tokyo 157-8572, Japanseki@grad.nii.ac.jp/seki@it.aoyama.ac.jp
Abstract
Recently, lots of researchers are focusing theirinterests on the development of summarization
systems from large volume sources combined withknowledge acquisition techniques such as information extraction, text mining or information retrieval. Some of these techniques are implemented
according to the specic knowledge in the domainor the genre from the source document. In this paper, we will discuss Japanese Newspaper Domain
Knowledge in order to make a summary. My system is implemented with the sentence extraction
approach and weighting strategy to mine from anumber of documents.
Keywords: NTCIR, multi-document summarization, newspaper article structure, and TSC (TextSummarization Challenge).

1 Introduction
There is a year long workshop being held by
the National Institute of Informatics in Japan
called NTCIR-3. We participated in the `Text
Summarization Challenge' (TSC) formalrun in
the spring of 2002: Japanese Summarization
tasks. We created an experimental system for theJapanese Summarization to compute the importance value for each sentence based on Japanesenewspaper terms. Our input data was collected© 2003 National Institute of Informatics

from Mainichi Newspaper articles from 1998 and1999. This included about 230,000 articles. In
this paper, we discuss mainly multi document
summarization6, 7, 9, 10] related problematic issues from several Japanese newspaper articles.
This paper consists of ve sections. We explain
the tasks of TSC in Section 2, and discuss details of our system strategy for sentence selectionin Section 3. Section 4 contains a brief evaluation of our system with TSC problems. In Section
5, another strategy for better results is detailed.
Finally, we present our conclusions in Section 6.

2 Summarization Tasks in TSC
The Text Summarization Challenge (TSC) consists of two tasks. Task A consists of single document summarization problems for two patternsof sizes from 200 to 500 character numbers. Twopattens of sizes corresponds to 20% and 40% condensed rates. Task A has 30 problem source articles. Task B also consists of multi document summarization problems for two patterns of sizes from
125 to 1000 character numbers. It addresses 30
topics as well, which each contain 3 to 17 source
articles. Source article IDs are previously specied. In addition, 1 to 4 keywords are also specied. The formalrun with these tasks was held onfour consecutive days in May, 2002.

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

3 Sentence Extraction Strategy

3.1 Single Document Summarization
My summarization approach is based on a sentence weighting ordering approach. Sentence importance weights are computed in the following.
1. The tf/idf values of nouns in the sentence arebiased.
First, the tf/idf values of all nouns in
the document except some stop words
are computed. The tf/idf value equation is as follows: TermFrequency log(AllDocumentNumbers
DocumentFrequency). According to eachdocument, the sum of all the tf/idf valuesof nouns in the document is computed. The
importance value of a sentence is computedby the sum of tf/idf values of sentences
containg nouns divided by the sum of all
tf/idf values in the document.
2. The weights of phrases included in the Heading are biased.
If a sentence contains phrases in the heading,the number of phrases is divided by the total
phrases in the heading. That value is thenmultiplied by the constant 0.1, and adds tothe sentence weights.
3. Position Weights are biased.
The line number of the sentence in the document divided by the number of all lines in thedocument corresponds to the position value inthe document from 0 to 1. We surveyed 10%
summary data from the Japanese NTCIR2SUMM corpus in the Mainichi Newspaper
from July to November in 1998. The correct summary position weights are shown inFigure 1. These values are multiplied by theimportance value in Step 2. These are theresultant importance values of each sentence.
4. Summarization.
Finally, important sentences whose sum ofcharacters exceeds the restricted character
amount, are eliminated. The remaining sentences are then sorted as they appeared in theoriginal document.

3.2 Multiple Document Summarization
In order to generate one summarization document from multiple articles, one approach is tocompute each sentence's importance weight withineach document. The simplest strategy is to extract important sentences equally from every related document according to the rates of summarization and arrange them chronologically. By
weighing sentence importance with tf/idf value ofa contained lexical set or words in the heading, we
can extract sentences specic to each document.
Another method is weighing each sentence
across the document set. In order to implement
this strategy, the importance value of each sentence is adjusted from 0 to 1 by dividing the sumof tf/idf values contained in each sentence andcomparing sentences' importance values across all
documents.
I take the rst approach. First, the character
numbers required for a summary are divided by
the sum of character numbers from all the source
articles. This value corresponds almost equallyto compress rates. According to these rates, theline number needed for extraction from each document is computed. These sentences are extracted
from every source article by following the single
document summarization strategy. This processproceeds in the document publishing time stamporder, and if the sum of characters contained in
the summary is over the restricted character number, the summary construction process stops. If
sentences are extracted from whole source articles
and the sum of the character numbers is still under the restricted character amount, the subsequent important sentences are extracted one by
one from each article chronologically.

4 Evaluation
TSC results were evaluated with score ranking
with content and readability for 20% and 40% condensed rates. The results of our present system areshown as follows.

Proceedings of the Third NTCIR Workshop

line position/all lines
Distributed Probabilityline position/all lines
Distributed Probability

0 < x  0.1 0.1 < x  0.2 0.2 < x  0.3 0.3 < x  0.4 0.4 < x  0.5
0.17
0.23
0.14
0.08
0.05
0.5 < x  0.6 0.6 < x  0.7 0.7 < x  0.8 0.8 < x  0.9 0.9 < x  1
0.04
0.06
0.04
0.04
0.15
Figure 1. Distributed Probability of Important Sentences

4.1 Task A (Single Document Summarization)
Task A had 30 problems. This task is evaluatedwith human ranking from the best (1 point) to theworst (4 point). My system's evaluation did notvary whether 20% or 40% summary. The dierence between Content evaluation and Readability
evaluation also did not change signicantly. The
total score ranges are shown in Table 1.
4.1.1 Good Results for Single Document Summary
In the summaries with the article ID 990424046,990624052, and 990629039, the long summary got
the rst score. In contrast to human-written summaries, my system's summary include necessary
elements and news background, reason, origin andexplanation. Short summary, however, did notcontain necessary elements in the topic sentence.
In the summaries with the article ID 990402040
and 990403032, the short summary also got therst score. In contrast to human-written summaries which contain many clauses that lackeddetailed information, my system had an eective
combination of facts and opinion.
4.1.2 Problems for Single Document Summary
Some documents include a multi-topic sentence
like "We have three perspectives" in the article
ID 990624050. For this type documents, shortsummaries must contain compact explanations foreach perspective and reduce some background orexplanation elements according to desired summary size. In addition, autobiographic type articles like the article ID 990109032 include someArticleID C 20% R 20% C 40% R 40%
9901090323434990117039
3434990120043
3334990129047
2233990130032
3333990201036
3333990202041
3333990105044
3333990211049
3333990305053
3333990311036
3333990313042
3333990313046
3111990402040
1111990403032
1111990410033
3333990428029
3333990430039
3333990501040
2212990502043
3333990531030
3434990603040
3333990604040
3333990605036
3333990616038
3333990618040
3333990624050
3344990624052
2211990629039
2211990630039
2222avg
2.70
2.73
2.60
2.77
Table 1. Scoring in Task A

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

important events and their reference expressions,but this type importance cannnot be computedfrom term-based weighting. These two types had
better be categorized or discriminated for dierent
summarization strategies, such as multi document
summarization for DUC (Document Understanding Conference)1 .
4.1.3 For Better Single Document Summarization
One of the problematic issues is to cut theabstract important sentence. In Japanese language, this sentence corresponds to the conclusion in the tail part of the article. Conclusionpart in Japanese newspaper articles is sometimes
not equal to introduction, and this fact must be
considered in case of to make a summary. My system's positional weighting approach take it into aconsideration, but a dierent approach may be effective.
One of the good scoring case to make a shortsummary as in 2, 3, 8] is to take more facts thanfacts in the sample summaries. In case of a shortsummary, strategy to select many short sentences
may be more eective. In contrast, in case of along summary, the resultant summary consistedof many short sentences would be hard to read.
In addition, if detailed short modied terms like
"The business in `Oogata village, Akita prefecture'" or "The judgement `last November'" hadbetter not be eliminated. From the viewpoint ofthe construction, many supporting stories of anexperience and concise conclusion would be better than a loose style discussion. These resultsmean sharp and smart construction with real experiences and numeric data would be easy to readfor a summary.

4.2 Task B (Multiple Document Summarization)
Task B had 30 topics to make a summary witha few keywords from multiple article sources. The
result is evaluated as in Task A. My system scores1

http://www-nlpir.nist.gov/projects/duc/index.html

are shown in Table 2.
TopicID C Short R Short C Long R Long
001011440020
22220030
43320040
24440050
22220060
24340070
21430080
23330090
24240100
22330110443301202422013044440140
22430150
33220160
44340170
32240180
34440190
34330200
44
12021044440220
44340230
33330240
1113025032220260
22220270
22120280
21320290
233303003444avg
2.63
2.90
2.80
3.03
Table 2. Scoring in Task B

4.2.1 Good Reults and Problems for Multi Document Summary
The short summaries in the topic ID 0010, 0070,0240, and 0280 got the rst score for readability. These summaries consisted of a few good sentences and consistency is good. The readability
for long summaries in these articles, however, did

Proceedings of the Third NTCIR Workshop

not score well because a topic sentence appeared in STEP 1: `Elaboration' roles are tagged according tothe middle of the discourse. The long summaries
whether a sentence contains heading nounin the topic ID 0200 and 0270 scored well. Inphrases.
these two summaries, several topics from several STEP 2: `Background' roles are tagged according todocuments were well detailed and the topic's dewhether a sentence contains numeric-data or
scription consistency were good. Another problem
time-related suxes (Year, Day, or $).
is sentence ordering from several documents1]. STEP 3: `Prospective' roles are tagged by some auxSome time event expressions in several documents
iliary elements (will, may, or seem) or noun
jumped their chronological ordering.
elements (prospective, assumption or possibility).
STEP 4: `Opinion' roles are tagged by some auxiliary
4.2.2 For Better Multi Document Summarizaelements (want, desire or think), quoting endtion
ing mark, some adjectives and juding auxiliary elements.
In case of multi document summarization, theSTEP
5:
Of the remeining `Elaboration' role sentences,dierent evaluation between a long summary andthe most important sentence is assigned toits short summary is more remarkable. Adding`Main Description' role.
topic start sentences or repetitious referent elements and shifting focus according to the source STEP 6: All sentence importance weights are biased byphrases in `Main Description' sentence.
document might cause the diculties to read fora long summary. In order to avoid this problem,
The combination of `Main Desciption' and `Backsize change process according to the source docuground' or `Prospective' are eective for evaluament or to eleminate or sort repetitious referentstion with the simple weighting approach. My proacross the documents might be required.
posal is to select important sentences with each
In contrast, in case of topicID 0200, a shortrole according to summary size and construct sumsummary has a bad evaluation while its long summary from dierent role sentences with the samemary is not. This is because a proper topic start
topic.
sentece is added into the rst line and a proper
supporting facts is inserted into other middle lines.
6 Conclusions and Future Direction
This means a summary must be constructed fromsource documents according to each sentence poWe tested my experimental summarization syssition and its role. We would try a `sentence-role
tem mainly using pure weighting approach biased
based summarization' technique 8] for multiple
with positional weights and its application to the
document summarization from this ndings.
multi-document summarization based technique.
The results showed that the concluding part could
5 Another Strategy For Better Results
not be extracted from the source document only
by biasing positional weights. In addition, its apWe propose important sentences ltering techplication to multi-document summarization has aniques with sentence roles8]. Some rhetorical sendefect for inserting sentences from several docutence roles for newspaper articles are proposed as
ments. This insertion process caused breaking ain 5]. We try a proposal in Kando4] and tagged
discourse construction. Pure weighting approach
5 specic sentence roles in newspaper articles aslimitation is shown in this result.
`Main Description', `Background', `Elaboration ofIn order to improve my results, some semantic
Main Description', `Prospectives' and `Opinion'.
information for summarization may be required toSentence role tagging algorithm is in the following
reduce the redundancy and to make a construcand this algorithm contains an overriding process
tive summary. In addition, if the assigned threshfor tagged roles.
old for summarization is changed according to theThe Third NTCIR Workshop, Sep.2001 - Oct. 2002

sentences already containing in a summary, better results will follow. In order to determine precise extracted summary, we will try `sentence-role'
based summarization work.

Acknowledgements
We thank the National Institute of Informatics
and organizing members held for NTCIR-3 TSC,
and also thank Patricia Lipska for re-reading theEnglish.

References
1] R. Barzilay, N. Elhadad, and K. R. McKeown.
Inferring strategies for sentence ordering in multidocument news summarization. Journal of Articial Intelligence Research, 17:35{55, 8 2002.
2] B. Boguraev, R. Bellamy, and C. Swart. Summarization miniaturisation: Delivery of news tohand-held. In the 2nd Meeting of the North
American Chapter of the Association for Computational Linguistics (NAACL 2001) Workshop on
Automatic Summarization, pages 99{108, Pittsburgh, June 2001.
3] S. Corston-Oliber. Text compaction for display
on very small screens. In the 2nd Meeting ofthe North American Chapter of the Association
for Computational Linguistics (NAACL 2001)
Workshop on Automatic Summarization, pages
89{98, Pittsburgh, June 2001.
4] N. Kando. Text structure analysis based onhuman recognition: Cases of japanese newspaper articles and english newspaper articles (injapanese). National Center for Science Information Systems Reports, 8:107{126, 1996.
5] E. D. Liddy. Development and implementationof a discourse model for newspaper texts. Inthe Dagstuhl on Summarizing Text for Intelligent
Communication, Saarbruken, Germany, 1995.
6] I. Mani. Automatic Summarization, volume 3 of
Natural Language Processing. John Benjamins,Amsterdam, Philadelphia, rst edition, 2001.
7] K. McKeown and D. R. Radev. Generating summaries of multiple news articles. In the 18th Int.
ACM SIGIR Conf. on Research and Development
in Information Retrieval, pages 74{82, Seattle,
WA USA, July 1995.
8] Y. Seki and N. Kando. Dynamic document generation based on tf/idf weighting. In Mobile
Personal Information Retrieval Workshop at the25th Ann. Int. ACM SIGIR Conf. on Researchand Development in Information Retrieval, pages
57{63, Tampere, Finland, August 2002.
9] G. C. Stein, T. Strzalkowski, and G. B. Wise.
Summarizing multiple documents using text extraction and interactive clustering. In Pacic Association for Computational Linguistics(PACLING-1999), 1999.

10] G. C. Stein, T. Strzalkowski, G. B. Wise, andA. Bagga. Evaluating summaries for multiple
documents in an interactive environment. In 2ndInt. Conf. on Language Resources & Evaluation(LREC2000), 2000.