Generating Summaries of Multiple Technical Articles
Minghui WANG*†, Hediheko TANAKA* and Yixin ZHONG†*Department of Engineering, The University of Tokyo, Japan†The Research Center of Intelligence, Beijing University of Posts andTelecommunications, China
mhwang@mtl.t.u-tokyo.ac.jp

Abstract
The research on document summarization has
been becoming attractive and importantbecause of the explosive increase of thescientific information. In this paper, we try todevelop a system framewo rk that focuses on
performing a summarization of multiple
papers on a specific domain which isconsidered more useful for reader to grasp theoutline of a research domain.

Keywords :Automated
Text
Summarization,
Summarization of Multi-paper, Sentence Extraction,
Citation Information

1 Introduction
The abundance of information and the difficulty toacquire wanted information on a specific topic may be
one of the major problems for a reader in this age of
information overload. Usually, the reader has to go
through Megabytes of data everyday to select articles ofinterest and read the relevant parts of them in order to getneeded materials [Kathleen McKeown and Dragomir R.
Radev (1995), Dragomir R. Radev. (1996)]. Hence, heneeds search and selection services (one of the very
popular topics in present Internet society), as well as for
Summarization facilities.
The automatic summarizer is considered as asoftware system which can produce a synopsis of any
document submitted to it. With few exceptions, automatic
approaches to summarization have primarily addressed
possible ways to determine the most important parts of atext [Luhn,H.P (1958), Daniel Marcu (1997), Eduard
Hovy and ChinYew Lin (1999)]. The so-called summary
produced by most of the currently existing text
summarization systems, including some Web browsers is
actually portions of the text, which is just a Extract. A
truly comprehensive and informative text summary
should combine together various concepts of the originaltext and generate a new text, or called Abstract. Anothermajor obvious problem of the existing summarization
systems is that it only summarizes a single article at atime [Kathleen McKeown and Dragomir R. Radev (1995),
Hietsugu NANBA and Manabu OKUMURA (1999)]. As
we know, especially on technical areas, most of papers

have a very well written abstract with them. So, such a
extract-based summarizer for single article doesn’t lead toany economy of time for the user. On the other hand, areader wants to seize the recent development and newachievement on his specific research field but have notime to read all the published papers. In fact, it is
impossible for a reader to collect and read all papers
published in his interesting areas in this information
overload time. In this situation, summarizing multiple
articles or survey of a specific domain can make it easierfor user to grasp the outlines of the domain.
In this paper, we try to investigate the problem ofsummarization on a set of articles. We use a corpus on the
Neural Network Algorithms collected from Journals andconferences papers in Chinese in which 89 papers areincluded. Firstly, We will analysis and describe somecharacteristics of a human expert on writing a survey fora specific domain in section 2. Section 3 will present our
methodology on summarizing multi-paper in details andexplain the corpus used in the study. Section 4 isconcerned with the system realization and algorithms.
And finally is the conclusion and discussion.

2 Essentials of Summarizing Multi-paper by
Human Expert
It is well known that a survey paper whether on
Magazine and Journals or on the academic conference isoften written by one who has completed a lot of research
work and has enough knowledge in the filed, or who atleast has read many papers in the domain. We call thiskind of person human expert in contradistinction toautomatic machine summarizer. After considering the
behavior on writing a survey paper of a human expert on
his own research field, It is not difficult for us to outline
his cognitive activity or procedure in this task as thefollowing.

(1) Careful consideration of the topic
Who will be the reader of this paper?
Did someone else write the same paper before?
What is the structure of the paper?
Is there any new development achieved recently?
…….

1

(2)Retrieval of papers related to the specific domain
and construction of corpus
Maybe there happened to have many related papers
on his hand. However he also needs to index the papers
especially recently published papers on the area.

(3)Reading carefully and understanding each of theselected papers
This may be a time -consuming work. He needs tocarefully perceive every viewpoint of the authors as wellas the correctness of the experiment result. He needs also
to comprehend or grasp the main idea of each paper.

(4)Extraction of information from papers of thedomainDetection of important fragments from each paper
Recognition of similarity among papers
Discovery of difference among papers(5)Generating new ideas and viewpoint of his own
after fusing together various information from eachpaper
Outline of the new approaches and methodology in thedomain
Review of the main achievements of the domain
Supposition of the future research direction of thedomainHigh point of the existed problem of the current
researches(6)Accomplishment of the final survey paper basedon the above study
Through the overview of human expert based survey
writing activities, we can conclude that: summarization ofmultiple papers should be much more difficult thansummarizing only single article. Firstly, we should take
into account how to collect the target papers forsummarization, which corresponds to (2) of the above
human based activity. Secondly, a multi-paper summary
system should clearly describe the similarity anddifference among papers, or it should be able to extract
useful information exactly from papers like human expert
does in the step (4) of the above procedure. Furthermore,
it is desirable that the output of a multi-paper
summarization system should describe or generate newideas-based sentences or text which cannot be extracted
directly from the original papers corresponding to thestep (5). However, the task to generate such a survey
automatically seems very difficult and impossible atpresent.

3 Corpus and Methodology

3.1 Corpus Description
Summarization research is notorious for its lack ofadequate corpora [Daniel Marcu (1999)]. Today, there
exist only a few small collections of texts whose units
have been manually annotated for textual importance.
Not only the English text but also the other languages
face the problem. This phenomenon is also the trouble of
the whole Natural Language Processing research and has
prevented rapid progress in the field. In order to conduct
study toward multi-paper summarization, We use thecorpus created by the Research Center of Intelligence,
Beijing University of Posts and Telecommunications, as
our main materials. This corpus consists of a collection of
Neural Network Learning Algorithms articles in Chinese,
which were selected from the Chinese Research Journals
and Conference Proceedings in recent years [BUPTTechnical Report (1999)]. There are totally 89 papers on
the corpus. The average length of the texts has about
3,000 Chinese characters. The longest one had about
6,300 Chinese characters. The shortest one has only about
500 Chinese characters. The corpus was manuallymarked with some symbols at the head of the line such asT, T1, N, U, A1, A2, K1, /P, J etc.. T represents Title, N
represents the Name of the author, A represents the
Abstract of the paper given by the author and so on.

3.2 Methodology
All researches on the automatic generation of genericabstract assume that the first task a summarization systemneeds to perform is that of extracting the most important
units in a text [Daniel Marcu (1997), Dragomir R. Radev.
(1996)]. These units can be words, phrasal expressions,clauses, sentence, fragment or paragraphs. Determining
the salient parts is considered to be achievable because
one or more of the following assumption hold:(1)Important sentences in a text contain words that areused frequently [Luhn, 1958] (2)Important sentences
contain words that are used in the title and section
headings. (3)Important sentences are located at the
beginning or end of paragraphs (4)Important sentences
use cue word such as “greatest” and “significant” orindicator phrase such as “the main aim of this paper” and“in this paper we propose…”. (5)Important sentences arelocated at the location of reference areas[Hietsugu
NANBA and Manabu OKUMURA (1999)].
Most current summarization systems use one orseveral above techniques to extract the so-call importantpart based on the powerful computer and then simply
aggregating them together to output as the summarizationof the system. In our multiple text summarization
research, we try to utilize Reference Information
Sentences of each paper as the important part to be
extracted, which was firstly applied for multiple English
texts summarization by [Hietsugu NANBA and Manabu
OKUMURA (1999)] and seems very successful on thetask. This is based on the assumption that ReferenceInformation Sentences contain some information of thesimilarity and difference between the paper and referredpapers. Understanding the essence and difference among
the collected papers of the specific domain would be very
important to generate a survey of multiple papers
according to the observation of human expert-based
survey activity in section 2. Although other informationsuch as the title, keywords etc. may be also very
important to the summary. Therefore firstly, we browse
the text from the beginning to the end to look for thereferred position, and extract fragments of the paper
where the author describes the essence of referred paper
and the difference with his paper. Then with theinformation of reference areas, we can generate multiple
papers summarization through categorizing the types ofreference relationships which will be explained in thefollowing.
Consider the following text in the corpus:
Ｆ：５．ｔｘｔ
Ｔ１：ＢＰ 悚？ 媦懘妛？ 嶼朄揑夵？
Ｎ丗？ 慞涼Ｕ丗峲揤堛妛岺掱尋媶強丆杒嫗丆
１０００９４／ｐ (1) 敧廫擭戙拞婜丆
Ｒｕｍｅｌｈａｒｔ［３］摍 採弌 椆 堦？ 強
？ 媡岦？ 攄 （Ｂａｃｋ Ｐｒｏｐａｇａｔｉｏｎ ？ ？ ？ ＢＰ）揑懡？ 恖
岺恄？ 悚？
（ＡＮＮ）妛？ 嶼朄丆宯？ 揑夝檙 椆 懡？ 悚？
拞？ ？ 尦？ 愙？ 揑妛？ ？ ？ 丆崕暈 椆 憗婜揑？ ？ 姶抦婍 （Ｐｅｒｃｅｐｔｒｏｎ）［４］扅擻？ 峴？ 惈暘？ 揑嬊尷惈丆惉？ ＡＮＮ ？ 揥巎忋揑堦槩棦掱旇［１，２］丅 （２） 扐惀 丆 恖？
嵼尋媶榓？ 梡拞？ ？ 丆 ＢＰ 悚？ 栫桳懘晄懌擵？ 丆斾擛懚嵼嬊晹？ 彫丆妛？ 懍搙枬摍？ ？ 丅夵慞
ＢＰ 悚？
嶼朄栟懘惀妛？ 懍搙惉？ 廳梫揑？ ？ 丅（３）
Ｒｕｍｅｌｈａｒｔ［５］摍恖梡？ 掤搙壛忋？ 惈？ 棃？ 彍戙樍敓悢嬋柺忋揑崅？ 暘検旔柶椆妛？ ？ 掱拞揑怳？ 丆樃帶
壛懍澗？ 丟
Ｌｅ Ｃｕｎｎ［６］慭梡媿？ 曽朄惛？ ？ 惍？ 愙？
埲徚嫀朸嵄嬊晹？ 彫棃？ 摼峏桳澚揑妛？ 丟
Ｌｅｈｍａｎ［７］摍恖梡壛殑惡揑曽朄棃挼弌嬊晹？ 彫丄壛懍澗？ 丟
Ｓｔｏｒｅｔｔａ［８］ 摍恖巊梡憃嬋惓愗敓悢？ 尦棃夵慞妛？
懍搙丟 Ｃａｉｌｌｏｎ［９］摍捠？ 尷惂？ 愙？ ？ 錀？ 棃旔柶妛？ 拞揑怳？ 丆壛懍澗？ 丆摍摍丅（４） ？ 慠 ？ 嵄夵？ 嶼朄嬒嵼晄摨掱搙忋夵慞椆
ＢＰ 悚？ 揑澗？ 惈擻丆扐 懡悢嶼朄揑妛？ 懍搙榓澗？ 惈？ ？ 愙？ 弶？ 揑？ 庢丄？
擖？ 杮揑摿惈渒晀姶丆？ ？ 惈晄懢？ 丆桳嵄嶼朄桼槹鷿壛椆怴揑？ ？ 嶲悢丆？ 棃嶼朄忋怴揑嶲悢？ 壔？
？ 丅 （５） 杮暥 嵼？ ＢＰ 悚？ ？ 峴嶼朄尋媶揑婎？ 忋丆
採弌 崻悩宯？ ？ 嵎帺？ ？ ？ ？ 妛？ 棪榓？ 検場巕揑？ 槩夣懍
ＢＰ 悚 ？ （ＦＢＰ） 妛 ？ 嶼 朄 丆 涹 ？ 懘 ？ 寬 惈（Ｒｏｂｕｓｔｎｅｓｓ）？ 峴椆暘愅斾？ 丅

Mark ‘/p’ means the beginning of a new paragraphwhich was created by the maker of the corpus. In order toexplain our ideas, the bold Mark (1) ｔｏ （５） ａre created
to represent the fragment of the text by ourselves. Now
from the text, We can obtain the following information:
Fragment (1) introduces the theme of the referred papers[1~4]. Fragment (2) points out the problems of thisresearch field. Fragment (3) Describes the methods tosolve the problem by another group of referred papers[5~9]. Fragment (4) further points out the new problem
to solve the problem. Fragment (5) describes the authors’
own method copes with the problem pointed out in
Fragment (2)．
By reading the fragment from (1) ｔｏ（５）， we canunderstand the relationship between this paper and thereferred papers[1~9].We call this kind of fragment in the
text Reference Area. With the information in it, we canalso identify the reason for citation by authors. We
classify the reason for citation into the following three
categories, we call these categories Reference Types.

¦ The Similarity-type reference：
The references to base on other researchers’ theories.

¦ The Difference-type reference：
The reference to compare with related work or to pointout their problems.

¦ The Other-type reference
Refer to the reference other than the Similarity-type
and the Difference-type
From the above Reference Type classification, it isclear that the Similarity-type reference usually represents
the information of similarity (method or viewpoint)
between the paper and the referred paper. While theDifference-type reference describes the information ofdifference between the paper and the referred paper. We
think the reference of the Difference-type is more
important than other, because from reference areas of the
Difference-type, we can obtain the following information:(A) Introduction of previous research(B) Description about the problem of the previous
research(C) The purpose of the authors’ research
In case of the above example, Fragment (1) ａｎｄ（３） corresponds to (A), Fragment (2) ａｎｄ （４）ｃｏｒｒｅｓｐｏｎｄｓ ｔｏ（Ｂ）. And Fragment (5) ｃｏｒｒｅｓｐｏｎｄｓｔｏ （Ｃ） . (A) can be considered as a kind of summary of
the referred paper from the authors’ viewpoint. (A) can
also be regarded as a fragment that describes thesimilarity of research topics between his paper and others.
On the other hand, the problem of previous work and the
purpose of research are described in Fragment (2)，（４）3ａｎｄ （５）． These fragments can be regarded to describe
difference topics between his paper and others. So, It
would be the important information for generating themulti-paper summarization.

4. Multi-paper Summarization by Extracting
Reference Information
As described in the above section, If we assume thesurvey of multiple papers as Detection and extraction of
Similarity and Differences among papers. Therefore,
extracting and displaying reference areas can be a good
support for writing a survey. In this section, we describe
our method to realize the multi-paper summarization
system.

4.1 Extraction of Reference Area
Reference areas can be considered as a set of sentences(or fragment) which have a connection with the sentence
including citation in the paragraph, for example, fragment
1 to ○
5 in the above text. Beside the obvious citation○
information, there are also some cue words to indicate
such a connection between sentences. Those cue words
are considered helpful for reference area extraction.
Examples of cue words are shown below:

1st person pronoun
3rd person pronoun
Indefinite pronoun
AdverbVerb
Special Noun
Negative expression

変？ , 嶌幰…
懠？
…
？ 嵄 ,恖？ …
？ 奜 ,？ ,庡梫 ,？ 堦？
採弌 ,昞柧 ,摼摓…
杮暥 , 嵼杮暥
…
扐惀 ,扐 ,？ 慠 …扐惀

whether it contains the citation information
Step4: If citation mark is found, apply the reference
area extraction rules to determine the fragment
Step5: Identify the type of the reference area.
Step6: If the end of the paragraph is met, go to step 2.
Step7: If the end of the file is met, go to step 1.
Step8: Combine the whole fragment and form a text
file to output.
Step 9: End.

5. Conclusion
The ability to automatically provide summarizationof multiple textual materials in a specific domain will
critically aid in effective use of the Internet in order toavoid overload of information. In this paper, we proposeda prototype system framework to perform thesummarization of multi-paper on the technical domain by
extracting the reference information of each paper in the
corpus. This work is still at early stage. The whole system
hasn’t been completely implemented yet. However, This
seed work will allow us to conduct further study towardautomatic multi-paper summarization.

References
Luhn,H.P (1958). The Automatic Creation of Literature
Abstract. IBM Journal of Research and Development,2(2):159-165,1958.

…

…

4.2 Identification of the Reference TypeIn a reference area, if a negative expression appears
at the beginning of the sentence including citation, thereference area can be considered as the Difference-type.
Similarly, if the expression like “ 杮 暥 嵼…婎 ？ 忋 ”
appears in the sentence including citation, the reference
area can be considered as the Similarity-type ． Therefore
we prepare a list of cue words and make a set of rules forthe reference type identification.

Kathleen McKeown and Dragomir R. Radev (1995),
Generating Summaries of Multiple News Articles. In
thProceeding of 18 annual International ACM SIGIR
Conference on Research and Development in Information
Retrieval, pp.74-82,Washington.
Dragomir R. Radev. (1996). An Architecture forDistributed Natural Language Summarization. In
Proceedings of the 8th International Workshop on Natural
Language Generation, pp.45-48, England.
Daniel Marcu (1997). From Discourse Structures to Text
Summaries. In Proceedings of the ACL’97/EACL’97
Workshop on Intelligent scalable Text Summarization,
pp.82-88.

The flow chart of reference area extraction is shown in
the following.

Hietsugu NANBA and Manabu OKUMURA (1999),
Towards Multi-paper Summarization Using Reference In
formation. The International Joint Conference on
Artificial Intelligence, pp.926-931.

Step 1: Input a text from corpus
Step2: Read one paragraph from the text
Step3: Check the sentence one by one and examine

Eduard Hovy and ChinYew Lin (1999), Automated Text
Summarization in SUMMARIST. In Inderject Mani andMark Maybury, eds, Advances in Automatic Text

Summarization, The MIT Press.
Daniel Marcu (1999). The automatic construction oflarge-scale corpora for summarization research. The 22ndInternational ACM SIGIR Conference on Researchand Development in Information Retrieval (SIGIR'99),
pages 137-144, Berkeley, CA, August.
BUPT Technical Report (1999), Research on the Key
Problems of Some Intelligent Services Provided byComputer Intelligent Network, the China Hi-Tech.
Program 863-317-9601-06-03.

5