ISSN 1346-5597

NII Technical Report

Web Content Summarization Using
Social Bookmarking Service

Jaehui Park, Tomohiro Fukuhara, Ikki Ohmukai, andHideaki Takeda

NII-2008-006E
Apr. 2008

Web Content Summarization
Using Social Bookmarking Service
Jaehui Park
School of Computer Science andEngineering
Seoul National University
Seoul 151-742, Republic of Korea+82-2-880-5517

jaehui@europa.snu.ac.kr

Tomohiro Fukuhara

Ikki Ohmukai1, Hideki Takeda2

fukuhara@race.u-tokyo.ac.jp

101-8430, Japan+81-4-7136-4275

RACE (Research into Artifacts, Center 1Digital Content and Media Sciencefor Engineering)Reserch Division,
2
The University of Tokyo, Kashiwa
Principles of Informatics Research
Chiba 277-8568, Japan
Division+81-4-7136-4275
National Institute of Informatics, Tokyo{i2k,takeda}@nii.ac.jp

ABSTRACTIn this paper, we propose a novel web content summarization
method that creates a text summary by exploiting user feedback(comments, annotations etc.) in the social bookmarking service.
We first analyze the feasibility to utilize user feedback in thesummarization, and then demonstrate how the social summary
which best represents the topics of the web content can begenerated. Performance evaluations on our method are conductedby comparing its output summary with the manual summaries
generated by human evaluators.

Categories and Subject Descriptors

from comments. Furthermore, we demonstrate how thissummarization technique can generate that are of similar qualitythat those produced by human.

2. BACKGROUND
In this paper, we suggest an idea for summarization that is
inspired by the recent emergence of so-called social services. By
encouraging users to submit a note or a tag, they can express theirviews of the contents. The point of all this approach is based
today’s active user role in web environment. There are so many
emerging services that allow users to play more active roles,
eventually enriching or enhancing the original content.

H.3.1 [Content Analysis and Indexing]: Abstracting methods;
H.3.3 [Information Search and Retrieval]: Information filtering,
Selection process

General Terms
Experimentation.

Keywords
Social summarization, Social bookmarking service, user feedback

1. INTRODUCTION
Weblogs (Blogs) provide individual users with an easy way topublish online and others to comment on that. This characteristicof blogs makes them quite different from other internet contents.
This could be used as a medium for social conversation.
Permitting readers to participate the conversation, the blog entry
can have a chance to aggregate related information from so-calledsocial knowledge. Recently, many of internet services such associal bookmarking allow users to gather blog entries and tocomment on it. As these services have matured and grown more
popular, social knowledge which we can exploit will becomerichen and qualified.
In spite of blog entries containing comments, existing studies arelargely focused on the post only. In this paper, we describe asummarization technique using user comments. First, we analyze
comments of blog post in social bookmark service. The focus of
our analysis is on the finding features for blog summarization

Figure 1. The Concept of Social Summarization
Based on user’s active role, enriched data from the social services
such as social bookmark services can be exploited for social
summarization. Social summarization focuses on human’s act
toward information. Figure 1 shows the concept of social
summarization.

3. PREVIOUS WORK
Much work has been done on Weblog Contents summarization. In
this section we give a brief overview of
Nevertheless, very few studies on blog comments and blog postsummarization have been reported [1][2]. The most recent one is[2]. In this approach aim to extract representative sentences froma blog post that best represent the topics discussed among its
comments. This paper shows information hidden in comments.

4. ANALYSIS OF COMMENT
Social bookmark services run as a tool for gathering various
users’ comments. Here we choose representative for this:Del.icio.us and Digg [3][4]. Del.icio.us is a social bookmark
service for storing and sharing web bookmarks. This is one themost popular social bookmark services. Users can tag a bookmark
or put a little note on it. Digg is a community-based news article
site. It combines a social bookmark service and blogging service.
Once one user post a bookmark, other users can simply rate(called diggs) or comment on the article.

4.1.5 Etc.
Spam, slang, non-English, joke, sarcasm, all including irrelevant
information
Categorizing comments, we found some patterns according tosocial bookmark services: Del.icio.us and Digg, Figure 2 shows
the different distribution of comment categories between them.

We examined user note in Del.icio.us and comment in Digg. In
spite that both have characteristics of their own service, we justfocused on the contents of a bookmark and comments on it.
Table 1. Statistics of Experiment Data
Parameters

Value

Number of participant

10

Number of Articles

70

Number of Comments

2504

Data TypeText, Video, Image

Figure 2. Number of comments in Del.icio.us and DiggFirstly, for understanding patterns in comments, we collect 10participants who are graduate students majoring in computerscience. They have read comments on a number of articles duringa week, and 70 articles and 2504 comments were gathered. Table
1 shows the statistics of experiment data. Gathered comments aremanually categorized into 5 groups:

4.1 Categories
4.1.1 Summary
A main summarization feature for this work. Mostly, a collectionof important sentences or sentence fragments from the originalarticle are classified into this category. And paraphrase for thesource content to provide a more concise representation is
included.

4.1.2 Additional Information
Related information from the external sources such as related
links, quotations, comparisons with another post, and so on.

4.1.3 Impression
An expression for describing user’s sentiment when he sees thearticle such as “Awesome”, “nice background design”. Most ofthem includes adjective phrase.

4.1.4 Opinion
An explicit opinion against the article expressing a judgment or
an assessment. This information could be used for enriching theoriginal article in social ways. In a sense, getting the point iscomplicate work even for human.

Figure 3. Pie Chart for Del.icio.us
Like Figure 3, in Del.icio.us, over 62% of comments areclassified into category ‘Summary’. In 14.6% of Etc, there aremany comments written in other languages like Japanese. If we
could understand other languages, the portion for ‘Summary’might be bigger. This result shows that users of Del.icio.us have atendency to summarize the contents of the bookmark. In user page
there are all the bookmarks he/she made and user notes for each.
When users create a bookmark, they put a user note on it. This
page can be exposed to other users, but cannot be altered or addedby them. This means that users have no place to discuss withother users, but more for their own uses such as summarizing
original post concisely. Based on this analysis, comments in
Del.icio.us are thought to be suitable for summarization.

ranking function. And the English stop word dictionary is added
to filter out word has no meaning such as ‘is’, ‘the’ and so on [7].
These improve the original word frequency algorithm
performance notably.
Using this customized tf-idf algorithm we calculate weight ofeach word appeared in comment list. After that, we sum thisweight up for constructing one complete sentence. Here we see
one of the examples.

Figure 4. Pie Chart for Digg
Like Figure 4, in Digg, about 40% of their comments were cannot
be processed. Most of them were jokes and sarcasm. However,comments classified into category ‘Opinion’ occupy over 25%.
This includes useful information supports the original one. This isbecause bookmarked (called dugg) web contents exposed in thepage which every user can add comments. Everyone express theirthought by comments. Some comments like a shrewd criticism ora detail feedback are regarded as a social knowledge of goodquality.
In Digg we found twice more the number of comments than in
Del.icio.us. However, because of good quality of summaries indel.icio.us, we made use of comments in del.icio.us for our
experiments. Based on our comment analysis, we expect that acomment list in the social bookmark services could be very usefulfor blog summarization.

5. EXPERIMENTS
We investigate how comments in social bookmark services arecontribute to the blog summarization. The comment list arecollected from Del.icio.us Today’s popular items. On the average,
there are 25 comments for one bookmark and over 62% ofcomments are good summaries for the bookmarked page. The
final goal of this experiment is finding good set of sentences fromcomment list on the assumption that most of comments indel.icio.us have good summary.
Firstly, we extract words from comment list based on statisticalnatural language processing algorithm. In a collection of comment,
we apply conventional tf-idf weighting algorithm to each word in
a comment. The weight increases proportionally to the number oftimes a word appears in the comment but is offset by thefrequency of the word in the collection. This weighting scheme isoften used by information retrieval field such as search engine [5].
But conventional tf-idf algorithm work at poor performance in our
experiment. This is because comments in the same collection
mostly have a same set of topics. The rank of a word relevant to asmall set of topics falls same as a rank of the word can be used
when talking about almost any topic or any context.
Hence, we employ POS (Part of speech) tagger developed by The
Stanford Natural Language Processing Group [6]. This tool reads
text and assign parts of speech to each word, such as noun, verb,
adjective, etc. Based on the comment analysis before, we
determine that noun, verb, adjective parts will be gained moreweights than others. This selection policy is added into our

Figure 5. Del.icio.us example page
Figure 5 shows comment list (user note in Del.icio.us) page forbookmark which links to blog post titled “Why I’m excited about
the Google Social Graph API – Bokardo”. In this comment list,
there are all 39 sentences. Sentences are classified into categories
as follows, Table 2.
Table 2. Comments classfied into 5 categories

Summary

20

Additional Information

3

Impression

2

Opinion

5

Etc.

9

Knowing that these examples have about 50% summarization
features, we run our algorithm. Among the weighted result
sentences, we choose top-k sentences in Table 3.
Table 3. Top 10 sentences extracted and weightcategorysentence

weightsummary

The Google Social Graph API is a newprogrammingAPI
thatallowsdevelopers
to
exposesocialrelationships embedded in web sites

37.5957

summary

What if you want to combine yourAmazon book history with your friends
lists at Facebook so that you can see
what your friends are reading

25.7348

summary

The Social Graph API helps solve this“silos of information” problem byallowing people to write software that
understands who your friends are

23.4909

additional
information

The truth is that Facebook Amazon oreven Twitter never had a good glimpseof my true social network anyway

23.1096summary

Do you ever feel like your personal
information is spread across the web in
a whole bunch of separate places

22.3561

Etc.

I never gave Facebook my email list
they don’t know anything about myblog and I’m going t

22.1896summary

What does this mean for regular folks
like you and me

18.62

summary

It does this by reading your web site or
blog and making connections between
the social profiles you have18.2541summary

While Google is providing the API
nobody is dependent on them forcreating or storing our relationships

17.4752

summary

The social relationships that the API
exposes are encoded in regular old
HTML using the XFN and FOAF
formats

16.3498

We perform the experiment on 20 bookmark pages which
includes average 18 comments each. The average precision at 20
is 0.54 and the average recall is 0.19. (at a given cu-off rank, therecall value does not really matter.)

6. DISCUSSION
For the example above, the precision at 10 is the value of 0.9 andrecall is 0.45. (When we retrieve more comments to 20 sentences,the precision is 0.7 and recall is 0.7.) This result only makes senseon the assumption that there are enough (over a half) sentences
for summarization in the comment list. We call this‘summarization feature’ in the social services like social
bookmark service. In out experiment, there are over 51%
summary sentences. If there are few related information, thisapproach may perform poor.
Besides, among the low ranked sentences, though we can find agood and more concise one, the algorithm cannot pick that up.
The sentence “Google Social Graph API” is weighted 1.6259. We
think this one is one of the best summaries. There are tendency inour sentence constructing technique that the longer sentence getsthe more weight from extracted word.
On the other hand, in the example of blog titled “What is Web 2.0– Tim Oreilly”, the good summary is made of common word suchas ‘Web’, ‘2.0’ and so on. But to find out this good summary:“This article is an attempt to clarify just what we mean by Web
2.0” , considering ‘Web 2.0’ rather than ‘Web’ and ‘2.0’ is oneof the effective ways. This will be resolved by introducing word
dictionary or the grammatical solution such as linguistic tree
constructing.

In this paper, we have described an approach to blog
summarization, which we call social summarization that uses thecomment list in the social bookmark service such as del.icio.us. Inturn, we have presented the experiment about measuringperformance for this approach. This result clearly highlights thepotential benefits of this summarization technique, which is seen
to produce socialized information from many users, closely
related to a human generated gold-standard.
In our experiments, sentence scores are sometimes very highbecause several words are repeated. Future direction of thisapproach is the consideration of sentence evaluation factors suchas length, word’s position and so on. In addition word extraction
algorithm should be refined for more comprehensive study oncomment pattern.

7. CONCLUSION
This paper presents a blog summarization technique using
comments appeared in the social bookmark service. Based on an
analysis of the comment classification in the social bookmark
services, we found that users have propensity to make a summary
for their bookmark contents. Our proposed solution finds
representative words in comments set by term frequency
weighting, grammatical tagging, and stop-word filtering. Usingsummary generated from the representative words, we evaluatedperformance of our work.

8. REFERENCES[1] Gilad Mishne and Natalie Glance. 2006. Leave a Reply: An
Analysis of Weblog Comments. In Proceedings of the 3rd
Annual Workshop on the Weblogging Ecosystem (The
Edinburgh, The Scotland, April 23-26, 2006)[2] Meishan Hu, Aixin Sun and Ee-Peng Lim. 2007. CommentsOriented blog Summarization by Sentence Extraction. In
Proceedings of the 16th Conference on Information andKnowledge Management (Lisboa, Portugal, November 6-9,
2007)
[3] Del.icio.us. http://del.icio.us. 2008.02.04[4] Digg. http://www.digg.com. 2008.02.04[5] Christopher D. manning and Hinrich Schutze. 1999.
Foundations of Statistical Natural Language Processing, The
MIT Press.
[6] The Stanford POS Tagger.
http://nlp.standford.edu/software/tagger.shtml. 2008.02.06[7] A list of English stop words.
http://en.wikipedia.org/wiki/Stop_words. 2008.02.06