Summarizing Search Results using PLSI
Jun Harashima∗ and Sadao Kurohashi
Graduate School of Informatics
Kyoto University
Yoshida-honmachi, Sakyo-ku,
Kyoto, 606-8501, Japan{harashima,kuro}@nlp.kuee.kyoto-u.ac.jp
Abstract
In this paper, we investigate generating
a set of query-focused summaries fromsearch results. Since there may be many
topics related to a given query in the
search results, in order to summarize
these results, they should first be classified into topics, and then each topic
should be summarized individually. In
this summarization process, two types ofredundancies need to be reduced. First,
each topic summary should not containany redundancy (we refer to this problem as redundancy within a summary).
Second, a topic summary should not be
similar to any other topic summary (werefer to this problem as redundancy between summaries). In this paper, we
focus on the document clustering process and the reduction of redundancy between summaries in the summarization
process. We also propose a method using
PLSI to summarize search results. Evaluation results confirm that our method
performs well in classifying search results and reducing the redundancy between summaries.

1

Introduction

Currently, the World Wide Web contains vast
amounts of information. To make efficient use ofthis information, search engines are indispensable. However, search engines generally return*Research Fellow of the Japan Society for the Promotionof Science (JSPS)

only a long list containing the title and a snippet of each of the retrieved documents. While
such lists are effective for navigational queries,
they are not helpful to users with informational
queries. Some systems (e.g., Clusty1 ) present
keywords related to a given query together withthe search results. It is, however, difficult forusers to understand the relation between the keywords and the query, as the keywords are merely
words or phrases out of context. To solve thisproblem, we address the task of generating a setof query-focused summaries from search results
to present information about a given query using
natural sentences.
Since there are generally many topics related to a query in the search results, the taskof summarizing these results is one of, so to
speak, multi-topic multi-document summarization. Studies on multi-document summarization typically address summarizing documents
related to a single topic (e.g., TAC2 ). However
we need to address summarizing documents related to multiple topics when considering thesummarization of search results.
To summarize documents containing multiple
topics, we first need to classify them into topics. For example, if a set of documents related toswine flu contains topics such as the outbreaks ofswine flu, the measures to treat swine flu, and soon, the documents should be divided into these
topics and summarized individually. Note that amethod for soft clustering should be employedin this process, as one document may belong toseveral topics.
12

http://clusty.com/
http://www.nist.gov/tac/

12
Proceedings of the Second Workshop on NLP Challenges in the Information Explosion Era (NLPIX 2010), pages 12–20,
Beijing, August 2010

In the summarization process, two types ofredundancies need to be addressed. First, each
topic summary should not contain any redundancy. We refer to this problem as redundancy within a summary. This problem is wellknown in the field of multi-document summarization (Mani, 2001) and several methods havebeen proposed to solve it, such as Maximum
Marginal Relevance (MMR) (Goldstein et al.,2000) (Mori et al., 2004), using Integer Linear Programming (ILP) (Filatova and Hatzivassiloglou, 2004) (McDonald, 2007) (Takamura
and Okumura, 2009), and so on.
Second, no topic summary should be similar
to any of the other topic summaries. We refer to this problem as redundancy between summaries. For example, to summarize the abovementioned documents related to swine flu, thesummary for outbreaks should contain specificinformation about outbreaks, whereas the summary for measures should contain specific information about measures. This problem is characteristic of multi-topic multi-document summarization. Some methods have been proposedto generate topic summaries from documents(Radev and Fan, 2000) (Haghighi and Vanderwende, 2009), but to the best of our knowledge,
the redundancy between summaries has not yet
been addressed in any study.
In this paper, we focus on the document clustering process and the reduction of redundancy
between summaries in the summarization process. Furthermore, we propose a method using
PLSI (Hofmann, 1999) to summarize search results. In the proposed method, we employ PLSI
to estimate the membership degree of each document to each topic, and then classify the search
results into topics using this information. In thesame way, we employ PLSI to estimate the membership degree of each keyword to each topic,
and then extract the important sentences specific to each topic using this information in orderto reduce the redundancy between summaries.
The evaluation results show that our method performs well in classifying search results and successfully reduces the redundancy between summaries.

Sz
D

Dz
Sz’
W

Dz’

Figure 1: Overview of the proposed method.

2 Proposed Method
2.1 Overview
Figure 1 gives an overview of the proposed
method, which comprises the following four
steps.
Step 1. Acquisition of Search Results Using asearch engine, obtain the search results fora given query.
Step 2. Keyword Extraction Extract the keywords related to the query from the search
results using the method proposed by Shibata et al. (2009).
Step 3. Document Clustering Estimate
themembership degree of each document toeach topic using PLSI, and classify thesearch results into topics.
Step 4. Summarization For each topic, generate a summary by extracting the importantsentences specific to each topic from eachdocument cluster.
In the following subsections, we describe eachstep in detail.
2.2 Step 1. Acquisition of Search Results
First, we obtain the search results for a given
query using a search engine. To be more precise,
we obtain the top N 0 documents of the search engine results. Next, we remove those documentsthat should not be included in the summarization,such as link collections, using a simple filtering
method. For example, we regard any documentthat has too many links as a link collection, andremove it.
In this paper, we write D to denote the search
results after the filtering, and let N = |D|.

13

2.3 Step 2. Keyword Extraction

M-step

We extract the keywords related to a query fromD using the method proposed by Shibata etal. (2009), which comprises the following four
steps.
Step 2-1. Relevant Sentence Extraction For
each document in D, extract the sentences
containing the query and the sentences
around the query as relevant sentences.
Step 2-2. Keyword Candidate Extraction For
each relevant sentence, extract compoundnouns and parenthetic strings as keyword
candidates.
Step 2-3. Synonymous Candidate Unification
Find the paraphrase pairs and the orthographic variant pairs in the keyword
candidates, and merge them.
Step 2-4. Keyword Selection Score each keyword candidate, rank them, and select thebest M as the keywords related to the query.

∑ ∑f req(d, w) p(z|d, w)
p(z) = d ∑w ∑f req(d, w)∑ d w
w f req(d, w) p(z|d, w)p(d|z) = ∑ ∑f req(d0 , w) p(z|d0 , w)
d0
∑ w
f req(d, w) p(z|d, w)p(w|z) = ∑ ∑d00d

We classify D into topics using PLSI. In PLSI,
a document d and a word w are assumed to beconditionally independent given a topic z, andthe joint probability p(d, w) is calculated as follows.
p(d, w) =∑p(z) p(d|z) p(w|z)

AIC = −2L + 2K(N + M )

L=

∑∑
d

(1)

f req(d, w) log p(d, w),(2)w

where f req(d, w) represents the frequency ofword w in document d. L is maximized using
the EM algorithm, in which the E-step and Mstep are given below.
E-step(6)(7)

Furthermore, we select p(z), p(d|z), and p(w|z)
estimated using the selected K as the result of
PLSI.
Next, we calculate the membership degree of
each document to each topic. The membership
degree of document d to topic z, denoted p(z|d),is calculated asp(d|z) p(z)p(z|d) = ∑.
0z 0 p(d|z )zp(z), p(d|z), and p(w|z) are estimated by maximizing the log-likelihood function L, which is
calculated as(5)

The EM algorithm iterates through these steps
until convergence.
First, we give PLSI the number of topics K,
the search results D, and the keywords W as
input, and estimate p(z), p(d|z), and p(w|z),where z is a topic related to the query, d is a document in D, and w is a keyword in W . There is,however, no way of knowing the value of K; thatis, we do not know in advance how many topics
related to the query there are in the search results.
Hence, we perform PLSI for several values of K,
and select the K that has the minimum Akaike
Information Criterion (AIC) (Akaike, 1974), calculated as follows.

In this paper, we write W to denote the extracted keywords.
2.4 Step 3. Document Clustering

w0 f req(d, w ) p(z|d, w )

(4)

(8)

Finally, for each topic, we collect those documents whose membership degree to the topic islarger than the threshold α. If there is a document whose membership degree to multiple topics is larger than the threshold, we classify the
document into each topic.
In this paper, Dz denotes the documents classified into topic z.
2.5 Step 4. Summarizationp(z) p(d|z) p(w|z)
000z 0 p(z ) p(d|z ) p(w|z )p(z|d, w) = ∑(3)

For each topic, we extract the important sentences specific to that topic from each document

14

Figure 2: Algorithm for summarization.

Input: A set of K document clusters {Dz }(z ∈ Z)
Output: A set of K summaries {Sz }(z ∈ Z)
Procedure:
1: for all z ∈ Z
2:while |Sz | < num(z)3:for all s ∈ Dz
4:
calculate s score(z, s, Sz )5:smax = argmaxs∈Dz \Sz s score(z, s, Sz )6:
Sz = Sz ∪ {smax }7:
return Sz

Table 1: Values of c score(w, Sz , s).
w is the subject of sotherwise

w iscontained in Sz
20

w is notcontained in Sz-2
1

to topic z, denoted p(z|w), as w score(z, w).
We use p(z) and p(w|z) estimated using PLSI
in Section 2.4 to calculate p(z|w).
p(w|z) p(z)p(z|w) = ∑0z 0 p(w|z )cluster. Figure 2 gives the algorithm for summarization. When we generate the summary Sz
for topic z, we calculate the importance of sentence s to topic z, denoted as s score(z, s, Sz ),for each sentence in Dz (lines 3-4). Then we extract the sentence smax with the maximum importance as an important sentence, and include
smax in Sz (lines 5-6). When we extract thenext important sentence, we recalculate the importance s score(z, s, Sz ) for each sentence in
Dz except the sentence in Sz (lines 3-4). Thenwe extract the sentence smax with the maximum
importance as an important sentence, and add
smax to Sz (lines 5-6). We continue this processuntil the number of important sentences composing the summary, denoted |Sz |, reaches the number of important sentences extracted for topic z,
denoted num(z) (line 2).
s score(z, s, Sz ) is calculated as follows:(10)

Keywords with high probability in several topics
should have a low membership degree to each
topic. Thus, using p(z|w) as the w score(z, w)
prevents extracting sentences containing such
keywords as important sentences, and it follows
that the similarity between the summaries is reduced. Furthermore, the keywords which are
specific to a topic are supposed to have a highmembership degree to that topic. Thus, usingp(z|w) as w score(z, w) makes it easier to extract sentences containing such keywords as important sentences, and with the result that eachsummary is specific to the particular topic.
c score(w, Sz , s) is a function to reduce theredundancy within a summary, and represents
the importance of a keyword w in a sentences under the condition that there is a set of
extracted important sentences Sz . The values score(z, s, Sz )()
of c score(w, Sz , s) is determined mainly by∑=w score(z, w) × c score(w, Sz , s)whether or not w is contained in Sz . Taw∈Wsble
1 gives the values of c score(w, Sz , s).
(9)For example, if w is contained in Sz , wewhere Ws represents the keywords in sentence s. set c score(w, Sz , s) = 0, else we set
w score(z, w) is a function to reduce the re- c score(w, Sz , s) = 1. In this way, we can exdundancy between summaries, and represents tract the sentences containing the keywords thatthe importance of keyword w to topic z. We can are not contained in Sz as important sentences,use the probability of w given z, denoted p(w|z), and reduce the redundancy within the summary.
as the w score(z, w). This approach fails, how- Note that we make some exceptions to generate
ever, because if there are keywords with a high a coherent summary. For example, even if w isprobability in both topic z and another topic z 0 , contained in Sz , we set c score(w, Sz , s) = 2
the sentences containing such keywords are ex- as long as w is the subject of s. In the same
tracted as the important sentences in both top- way, even if w is not contained in Sz , we set
ics, and it follows that the generated summaries c score(w, Sz , s) = −2 as long as w is the subwill contain redundancy. To solve this problem, ject of s. These values for c score(w, Sz , s) are
we use the membership degree of keyword w empirically determined.

15

Finally, using p(z) we determine the numberof important sentences extracted for topic z, denoted as num(z).
num(z) ={b I × p(z) c ( p(z) ≥ β )
(11)
Imin
( p(z) < β )where I represents the parameter that controls
the total number of important sentences extracted for each topic. The higher the probability
a topic has, the more important sentences we extract. Note that no matter how low p(z) is, we
extract at least Imin important sentences.

3

Experiments

3.1 Overview
To evaluate the proposed method, we recruited
48 subjects, mainly IT workers, and asked themto fill in a questionnaire. We prepared a system implemented according to our method, andasked the subjects to use our system to evaluate
the following four aspects of our method.

2008b). Setting N 0 = 1, 000, we obtained the
top 1, 000 documents in the search results forthe query. In the keyword extraction process,we set M = 100, and extracted 100 keywords
related to the query from the search results. In
the document clustering process, we performed
PLSI for K = 3, 4, 5, and selected the K withthe minimum AIC. We set the initial value ofp(z) = 1/K, and the initial values of p(d|z)
and p(w|z) to random values. The EM algorithm
continued until the increase in L reached just below 1 to achieve convergence. We set α = 1/K.
In the summarization process, we set I = 10,
since the number of important sentences able tobe presented in a browser is about 10. We set
Imin = 2 and β = 0.2, and extracted at least two
important sentences, even if p(z) was very low.
3.3 Validity of the Number of Topics

3.2 System

First, we investigated how well the proposed
method determined the number of topics. In our
method, the number is determined using AIC.
Ideally, we should have manually counted the
topics in the search results, and compared thiswith the number determined using AIC. It was,however, difficult to count the topics, because thesearch results contained 1, 000 documents. Furthermore, it was impossible to count the numberof topics for each query given by each subject.
Thus, in this investigation, we simply asked thesubjects whether they felt the number of topic
summaries presented to them was appropriate ornot, and investigated our method in terms of usability.

Figure 3 shows the system results for the queryswine flu. Our system presents a separate summary for each topic related to a given query. In
Fig.3, the colored words in the summaries are
keywords specific to each topic. If a user clicks
on a keyword, the system presents a list of documents containing that keyword at the bottom of
the browser.
The configuration of our system was as follows. In the acquisition process, the system obtained the search results for a given query using the search engine TSUBAKI (Shinzato et al.,Table 2 gives the results. According to Table
2, 60.4% of the subjects agreed that the numberof topic summaries presented by our system was
acceptable. The average of the number of topics
determined by our method was 3.18 per 1 query.
On the other hand, 33.3% of the subjects said
the number of topic summaries was too low orsomewhat too low. According to these results,
it seems that users are satisfied with the systempresenting about 3 or 4 topic summaries, and our
method determined the desirable number of topics in terms of usability.

• Validity of the number of topics• Precision of document clustering• Degree of reduction in redundancy between
summaries• Effectiveness of the method for presenting
information through summaries
We allowed the subjects to create arbitrary
queries for our system.

16

!"" # $""

"

%

&
'
(

%

)
*

+

,

-

.
&

Figure 3: System results for the query swine flu.

Table 2: Validity of the number of topics.
options(a) definitely too many(b) somewhat too many(c) acceptable(d) somewhat too few(e) definitely too few# subjects
0
3
29
11
5

( % )
( 0.0)( 6.3)(60.4)(22.9)(10.4)

3.4 Precision of Document Clustering
Second, we investigated how precisely the proposed method classified the search results into
topics. To be more precise, we evaluated the reliability of the membership degree p(z|d) usedin the document clustering process. It is generally difficult to evaluate clustering methods.
In our case, we did not have any correct dataand could not even create these since, as mentioned previously, the number of topics is notknown. Furthermore, it is not possible to classify
by hand search results containing 1, 000 documents. Consequently, we did not evaluate ourmethod directly by comparing correct data withthe clustering result from our method, but instead

evaluated it indirectly by investigating the reliability of the membership degree p(z|d) used in
the document clustering process.
The evaluation process was as follows. First,
we presented the subjects with a document d,which was estimated by our system to have ahigh membership degree to a topic z. Strictly
speaking, we selected as d, a document with amembership degree of about 0.9. Next, we presented two documents to the subjects. One wasa document d0 whose membership degree to z
was also about 0.9, and another was a document
d00 whose membership degree to z was about
0.1. Finally, we asked them which document wasmore similar to d3 .
Table 3 gives the results. According to this table, 60.5% of the subjects said d0 was more similar or somewhat more similar. On the other hand,
only 12.6% of the subjects said d00 was moresimilar or somewhat more similar. We see fromthese results that the ability to recognize topics inour system is in agreement to some extent with
3
Naturally, we did not tell them that d0 had a similar
membership degree to d, whereas d00 did not.

17

Table 3: Precision of the estimation p(z|d).
options(a) d0 is definitely more similar(b) d0 is somewhat more similar(c) undecided(d) d00 is somewhat more similar(e) d00 is definitely more similar# subjects
14
15
13
3
3

Table 4: Comparison of df idf (w), p(w|z) andp(z|w).

( % )(29.2)(31.3)(27.1)( 6.3)( 6.3)

options(a) B is definitely less redundant(b) B is somewhat less redundant(c) undecided(d) A is somewhat less redundant(e) A is definitely less redundant
options(a) C is definitely less redundant(b) C is somewhat less redundant(c) undecided(d) A is somewhat less redundant(e) A is definitely less redundant
options(a) C is definitely less redundant(b) C is somewhat less redundant(c) undecided(d) B is somewhat less redundant(e) B is definitely less redundantthe subjects’ ability for recognizing topics; thatis, our method was able to estimate a reliable
membership degree p(z|d). Thus, it seems that
our method using p(z|d) is able to classify search
results into topics to some extent.
3.5 Degree of Reduction in Redundancy
between Summaries
Third, we investigated how well the proposed
method reduced the redundancy between summaries. To be more precise, we used three measures as w score(z, w) to generate summaries
and investigated which measure generated theleast redundant summaries. Generally, methods for reducing redundancy are evaluated using
ROUGE (Lin, 2004), BE (Hovy et al., 2005),
or Pyramid (Nenkova and Passonneau, 2004).
However, the use of these methods require that
ideal summaries are created by humans, and thiswas not possible for the same reason as mentioned previously. Thus, we did not perform
a direct evaluation using the methods such as
ROUGE, but instead evaluated how well our
method performed in reducing redundancy between summaries using the membership degreep(z|w) as w score(z, w).
The evaluation process was as follows. We
used three measures as w score(z, w), and generated three sets of summaries.

# subjects
5
16
15
6
6
# subjects
16
14
6
8
4
# subjects
15
8
10
6
9

( % )
(10.4)(33.3)(31.3)(12.5)(12.5)( % )(33.3)(29.2)(12.5)(16.7)( 8.3)( % )(31.3)(16.7)(20.8)(12.5)(18.8)

Summaries B This set of summaries was generated using p(w|z) as w score(z, w).
Summaries C This set of summaries was generated using p(z|w) as w score(z, w).
We then presented the subjects with three pairsof summaries, namely a pair from A and B, apair from A and C, and a pair from B and C, andasked them which summaries in each pair wasless redundant4 .
The results are given in Tables 4. Firstly, according to the comparison of A and B and thecomparison of A and C, A was more redundant
than B and C. The value of df idf (w) to keyword w was the same for all topics. Thus, using df idf (w) as w score(z, w) made summaries
redundant, as each summary tended to containthe same keywords with high df idf (w). On theother hand, as the value of p(w|z) and p(z|w)
were dependent on the topic, the summaries generated using these measures were less redundant.
Second, the comparison of B and C shows that
48.0% of the subjects considered C to be lessredundant or somewhat less redundant. p(w|z)
was a better measure than df idf (w), but even using p(w|z) generated redundancy between sum-

Summaries A This set of summaries was generated using df idf (w) as w score(z, w),with df idf (w) calculated as ldf (w) ×log(100 million/gdf (w)), ldf (w) representing the document frequency of keyword
w in the search results, and gdf (w) representing the document frequency of keyword w in the TSUBAKI document collection (Shinzato et al., 2008a) comprising
about 100 million documents.

4
Naturally, we did not tell them which summaries weregenerated using which measures

18

Table 5: Comparison of summaries and keywords.
options(a) X is definitely more helpful(b) X is somewhat more helpful(c) undecided(d) Y is somewhat more helpful(e) Y is definitely more helpful# subjects
25
10
3
8
2

( % )
(52.1)(20.8)( 6.3)(16.7)( 4.2)

maries. Because common keywords to a query
have high p(w|z) for several topics, sentences
containing these keywords were extracted as theimportant sentences for those topics, and thus
the summaries were similar to one another. On
the other hand, the keywords’ value for p(z|w)
was low, allowing us to extract the importantsentences specific to each topic using p(z|w) as
w score(z, w), thereby reducing the redundancy
between summaries.
3.6 Effectiveness of the Method for
Presenting Information Using
Summaries
We also investigated the effectiveness of themethod for presenting information through summaries. We asked the subjects to compare twodifferent ways of presenting information and tojudge which way was more effective in terms
of usefulness for collecting information about
a query. One of the methods presented thesearch results with topic summaries generated byour system (method X), and while the anothermethod presented the search results with the keywords included in each topic summary (method
Y).
Table 5 gives the results. 72.9% of the subjects considered the method using summaries tobe more effective or somewhat more effective.
From these results, it appears that the method ofpresenting information through summaries is effective in terms of usefulness for collecting information about a query.

4 Conclusion
In this paper, we focused on the task of generating a set of query-focused summaries fromsearch results. To summarize the search results
for a given query, a process of classifying theminto topics related to the query was needed. In
the proposed method, we employed PLSI to estimate the membership degree of each document
to each topic, and then classified search results
into topics using this metric. The evaluation results showed that our method estimated reliable
degrees of membership. Thus, it seems that our
method is able to some extent to classify search
results into topics.
In the summarization process, redundancy
within a summary and redundancy between summaries needs to be reduced. In this paper, we focused on the reduction of the latter redundancy.
Our method made use of PLSI to estimate themembership degree of each keyword to each
topic, and then extracted the important sentences
specific to each topic using this metric. The evaluation results showed that our method was able
to reduce the redundancy between summaries using the membership degree.
In future, we will investigate the use of
more sophisticated topic models. Although our
method detected the topics related to a query using a simple topic model (i.e., PLSI), we believe that more sophisticated topic models suchas LDA (Blei et al., 2003) allow us to improve
our method.

References
Akaike, Hirotugu. 1974. A new look at the statistical model identification. IEEE Transactions on
Automation Control, 19(6):716–723.
Blei, David M., Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet Allocation. Journalof Machine Learning Research, 3:993–1022.
Filatova, Elena and Vasileios Hatzivassiloglou. 2004.
A formal model for information selection in multisentence text extraction. In Proceedings of COLING 2004, pages 397–403.
Goldstein, Jade, Vibhu Mittal, Jaime Carbonell, andMark Kantrowitz. 2000. Multi-document summarization by sentence extraction. In Proceedings of
ANLP/NAACL 2000 Workshop on Automatic Summarization, pages 40–48.
Haghighi, Aria and Lucy Vanderwende. 2009. Exploring content models for multi-document summarization. In Proceedings of HLT-NAACL 2009,
pages 362–370.

19

Hofmann, Thomas. 1999. Probabilistic latent semantic indexing. In Proceedings of SIGIR 1999,
pages 50–57.
Hovy, Eduard, Chin-Yew Lin, and Liang Zhou. 2005.
Evaluating duc 2005 using basic elements. In Proceedings of DUC 2005.
Lin, Chin-Yew. 2004. Rouge: A package for automatic evaluation of summaries. In Proceedingsof ACL 2004 Workshop on Text Summarization
Branches Out, pages 74–81.
Mani, Inderjeet. 2001. Automatic Summarization.
John Benjamins Publishing Company.
McDonald, Ryan. 2007. A study of global inference
algorithms in multi-document summarization. In
Proceedings of ECIR 2007, pages 557–564.
Mori, Tatsunori, Masanori Nozawa, and Yoshiaki Asada.
2004.
Multi-answer-focused
multi-document summarization using a questionanswering engine. In Proceedings of COLING
2004, pages 439–445.
Nenkova, Ani and Rebecca Passonneau. 2004. Evaluating content selection in summarization: The
pyramid method. In Proceedings of NAACL-HLT
2004.
Radev, Dragomir R. and Weiguo Fan. 2000. Automatic summarization of search engine hit lists. In
Proceedings of ACL 2000 Workshop on Recent advances in NLP and IR, pages 1361–1374.
Shibata, Tomohide, Yasuo Bamba, Keiji Shinzato,
and Sadao Kurohashi. 2009. Web information organization using keyword distillation based clustering. In Proceedings of WI 2009, pages 325–
330.
Shinzato, Keiji, Daisuke Kawahara, Chikara
Hashimoto, and Sadao Kurohashi. 2008a. A
large-scale web data collection as a natural language processing infrastructure. In Proceedingsof LREC 2008, pages 2236–2241.
Shinzato, Keiji, Tomohide Shibata, Daisuke Kawahara, Chikara Hashimoto, and Sadao Kurohashi.
2008b. TSUBAKI: An Open Search Engine Infrastructure for Developing New Information Access Methodology. In Proceedings of IJCNLP
2008, pages 189–196.
Takamura, Hiroya and Manabu Okumura. 2009.
Text summarization model based on maximum
coverage problem and its variant. In Proceedingsof EACL 2009, pages 781–789.

20