Query-driven Opinion
Summarization
Kam-Fai Wong

Department of Systems Engineering & Engineering
Management
The Chinese University of Hong Kong

Trends in WWWContent
Read OnlyCreation

Management

Web 1.0

Web 2.0
Web 3.08-Jul-2011

2

Ufida SeminarWeb 3.0 Core TechhSocial Networks
Integrated network (3-in-1)
Semantic Net
Ubiquitous and Mobile Computing
Internet Of Things (IOT)
Cloud Computing (IaaS, PaaS and SaaS)
Next Generation Internet (NGI, IPv6)

8-Jul-2011

3

Ufida Seminar

社會3.0現象•e-Engagement (people-centric)•N-Generation•Crowdsourcing•Data is King -> People (relationship) is King• Multi-modal•Monetization

8-Jul-2011

4

Ufida Seminar

Outline
Introduction
Weighting Scheme

Query-driven Opinion Summarization

Evaluation
Conclusion & Future Works
2011/7/29

5

Introduction• With the explosion in the amount of commentaries on current issues
and personal views expressed in weblogs, microblog on the Internet,
there is a need to provide users a summary of opinions. [Kim et al.
2010]

2011/7/29

6

Introduction• Aspect-based opinion summarization (AOS)– to divide input texts (mostly are review data) into aspects (features) andgenerate summaries of each aspect. [Liu et al. 2005]
e.g. Toshiba Satellite L655-S5158• Query-driven opinion summarization (QOS)– to extract an informative summary of opinion expressions about a given
query (also referred as topical opinion), as found in a document
collection. [Dang 2008]
e.g. “What complaints on YouTube do users have?”
2011/7/29

7Introduction• Differences:– AOS is almost about review-type data, the aspects are limited to a list of
predefined or labeled aspects for a same topic (product); while QOS
concerns more on user’s preference, and the query might only focus on
one of the multiple topics presented in the related documents.
e.g. to summarize the negative opinions on YouTube from a number of articles
on Internet Service.

– In AOS, sentiment words are mostly domain-specific and the amount isfixed; while in QOS, general domain sentiment words will occur
frequently across multiple topics.
e.g. Good vs. Delicious

2011/7/29

8

Introduction• One of the fundamental problems in QOS is how to effectively
represent and measure topical opinion so as to precisely select thesentences with salient opinion expression.
• Existing methods:– Three-step approach :1. identify relevant text segments (e.g. sentences or passages) to the query fromthe blogs;
2. re-rank the set of relevant segments by taking sentiment classification intoconsideration;
3. select segments with high ranking and remove redundant text segments.

Most participants in TAC2008 adopted three-step approach. [V. Varma etal. 2008], [Razmara and Kosseim 2008], [Li et al. 2008],[ Seki 2008],[Balahur et al. 2008]2011/7/29

9Introduction

2011/7/2910Introduction• Our Method:– Utilize word pair to represent topical opinion.
– Measure the topical opinion by simultaneously considering thesubjectivity of the topic word and the local relevance of the sentimentword.
– Compute Pointwise Mutual Information(PMI) between sentiment word
and its associated topic within a pair to measure the topical opinion ineach individual word pair.
– Implement the weighted topical opinions into a graph model forsentence ranking and MMR method to generate query-driven summary.
2011/7/29

11

Outline
Introduction
Weighting Scheme

Query-driven Opinion Summarization
Evaluation
Conclusion & Future Works
2011/7/29

12

Formal Definition•Given a document set D={d1,d2,d3,…,dn}, that includes a set of sentences
S={s1,s2,s3,…,sN}, and a specific query Q={q1,q2,q3,…,qz}, where q1,q2,q3,…,qz
are query keywords.

•In addition, we construct a sentiment word lexicon Vo and a topic word lexiconVt .

•We utilize the structure of Query-sentiment word pair pij to denote the topicalopinion, which consists of two elements, one is from Vt, and the other one isfrom Vo. [Li et al, 2010]

e.g. we can extract the word pair <Battery, awful> from the sentence “Batteryis awful lasts for about 1 hour”.
2011/7/29

13

Topical Opinion Weighting• We measure topical opinion based on the following
assumptions:– topic word t1 is more important than topic word t2 when there are morecomments or opinions on t1 than t2.
– sentiment word o1 can be regarded as domain-specific sentiment word
due to different associated targets.
– the associations between topic and sentiment words in different wordpairs vary a lot. [Kim et al. 2009]2011/7/29

14

Topical Opinion Weighting• We measure topical opinion in 2 stages:– we first measure both topic word and sentiment word by computing thegain in selecting a sentence containing the word.
– Based on pairwise representation, we weigh topical opinion by computing
the PMI between the target and sentiment words within a pair.

• Assume that a term t follows the distribution PD(t) on the whole set ofwords, and it also follows another distribution PS(t)(t) on the set ofsentences including t. The higher deviation of PS(t)(t) from PD(t), thehigher the information content of t is.

2011/7/29

15

Topical Opinion Weighting

2011/7/29

16

Topical Opinion Weighting

2011/7/29

17

Topical Opinion Weighting• According to the assumptions, we can assign the topic wordand sentiment word as:
andwhere,• We finally add the associative score to each word pair andcompute the weight of a topical opinion as:

where
2011/7/29

18

Outline
Introduction
Weighting Scheme

Query-driven Opinion Summarization

Evaluation
Conclusion & Future Works
2011/7/29

19

Sentence Ranking• To generate a summary for a specific query, we first select aset of sentences with the topical opinions.
• Graph-based ranking algorithms, such as HITS or PageRank,
have been traditionally and successfully used in citation
analysis, retrieval, summarization. [Erkan et al. 2004]• Intuitively, sentences containing more word pairs with thetopical opinions should achieve a relatively higher ranking.

2011/7/29

20

Sentence Ranking• Based on the PageRank model, we define a graph with nodes
representing relevant sentences and edges connecting 2 sentences
sharing a common word pair.
• We then score all the sentences based on the expected probabilityof a random walker visiting each sentence.
• The jumping probability

from nodewhere

2011/7/29

21

to nodeis given by:Sentence Ranking• All sentences are initialized equally. In each iteration T+1, thescores are updated according to the scores in iteration T.

• Iteration is terminated when the maximum difference between
the scores computed for two successive iterations is lower
than a given threshold (empirically setting as 0.00001). [Li etal. 2009]• Finally, the sentences are ranked by the scores.
2011/7/2922

QOS• We adopt maximal marginal relevance (MMR) method to generate
the summary by incrementally adding the top ranked sentences into
the answer set. [Carbonell and Goldstein 1998]• R is the ranked list of sentences retrieved in the previous step. We
set a relevant threshold, below which it will not be regarded as
candidate sentences.
• The parameter θ lying between [0,1] controls the relative importancegiven to relevance versus redundancy. In our experiments we set
θ=0.5 .
2011/7/29

23

Outline
Introduction
Weighting Scheme

Query-driven Opinion Summarization

Evaluation
Conclusion & Future Works
2011/7/29

24

Experiment Setting• Datasets:– TAC08 dataset is the benchmark data set for the query-driven opinion
summarization track in the Text Analysis Conference 2008 (TAC2008), whichcontains a total number of 2500 documents and 87 opinion queries. [Dang, 2008]– The Opinion Question Answering (OpQA) corpus consists of 98 documents
appeared in the world press and 30 queries. [Wilson, et al 2005]• Sentiment Lexicon:– We use SentiWordNet as the sentiment lexicon, which consists of 4800 negative
sentiment words and 2290 positive sentiment words.

• Topic Word Collection:– The dictionary-based method– The web-based method

2011/7/29

25

Parameter Tuning

M/通用格式

MAP

M/通用格式M/通用格式OpQA

M/通用格式TAC2008
M/通用格式M/通用格式M/通用格式M/通用格式M/通用格式M/通用格式M/通用格式M/通用格式𝛾𝛾=0.8

2011/7/29

26

Experiment 1• In our evaluation, we first test the performance of our proposed
weighting scheme for measuring topical opinion.
• Methods for comparison:––––tf-idf
WordNet: applied the maximum value of a sentiment word in SentiWordNet Lexicon as theweight of sentiment word.
GOSM: proposed to represent topical opinion by word pair, and utilized tf-idf to weigh topicalopinion.
PPM: our proposed method.

• Experimental Metrics:• MAP: Mean Average Precision• Rpre: R-precision• P@102011/7/29

27

Experimental Result 1• Comparison of different weighting schema on TAC08 andOpQA, and the best result in each column is highlighted.

2011/7/29

28

Experiment 2• Different approaches for QOS for comparison:–Baseline 1: This model was achieved the best run in TAC2008 opinion summarization task.
[Varma, et al., 2008]–Baseline 2:This model was achieved 10% improvement over the best run in TAC2008
Opinion QA track. We modified this model to deal with QOS. [Li, et al., 2009]–

OPM: similar with Baseline 2, but use PageRank model for sentence ranking instead.

–

GOSM: This model adopted pairwise representation of topical opinion. We re-designed
GOSM to deal with QOS. [Li, et al., 2010]–PPM: our proposed approaches.

• Experimental Metrics:• Precision• Recall• F-value:
2011/7/29

29

Experimental Result 2• Comparison of different approaches for opinion summarization
on TAC08 and OpQA datasets, and the best F(3) is highlighted.

2011/7/29

30

Outline
Introduction
Weighting Scheme

Query-driven Opinion Summarization

Evaluation
Conclusion & Future Works
2011/7/29

31

Conclusion• We utilize pairwise representation to denote topical opinion.
• A weighting scheme has been proposed to measure thetopical opinion by simultaneously considering the subjectivityof the topic word and the local relevance of the sentimentword.
• Weighted topical opinions were implemented into a graphmodel for sentence ranking and MMR method to generate
query-driven summary which performs well on TAC2008 andOpQA datasets.

2011/7/29

32

Future works• Need of techniques of other research areas– Deeper NLP e.g., discourse analysis, dependency parser, may help tounderstand the meaning of opinion so as to improve the accuracy.

• Need of Standardized Data Set and Evaluation– Current published data set are depending on their own purpose andlack of widely used dataset. [Hu and Liu 2004], [Kim and Zhai 2009],[Ganesan et al. 2010]– Lack of evaluation measures which cover entire opinion summarization
steps is another issue.

2011/7/29

33

References
H. Dang, 2008. Overview of the TAC 2008 opinion question answering and summarization
tasks. In TAC 2008.
G. Erkan and D. R. Radev. 2004. Lexpagerank: Prestige in multi-document text
summarization. In EMNLP 2004.
B. Ernsting, W. Weerkamp, and M. de Rijke. 2007. Language modeling approaches to blog
post and feed finding. In TREC 2007.
A. Esuli and F. Sebastiani. 2006. Sentiwordnet: A publicly available lexical resource for
opinion mining. In LREC.
B. He, C. Macdonald, J. He, I.h Ounis. 2008. An effective statistical approach to blog post
opinion retrieval, In CIKM 2008.
M. Hu, B. Liu, 2004. Mining and summarizing customer reviews, Proceedings of the tenth
ACM SIGKDD 2004.
J. Kim, J. Li, and J. Lee. 2009. Discovering the discriminative views: Measuring term weights
for sentiment analysis. ACL-IJCNLP 2009.
B. Li, L. Zhou, S. Feng and KF Wong. 2010. A unified graph model for sentence-based
opinion retrieval, In ACL 2010.

2011/7/29

34

References
F. Li, Y. Tang, M. Huang, and X. Zhu. 2009. Answering opinion questions with random walks
on graphs. In ACL 2009.
B.Liu, M. Hu, and J. Cheng. 2005. Opinion observer: Analyzing and comparing opinions on
the web. In WWW 2005.
B. Pang and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends inInformation Retrieval
V. Stoyanov, C. Cardie, 2006. Toward opinion summarization: linking the sources,
Proceedings of the Workshop on Sentiment and Subjectivity in Text.
P. Turney, 2002. Thumbs up or thumbs down? semantic orientation applied to unsupervisedclassification of reviews. In ACL 2002.
V. Varma, P. Pingali, R. Katragadda, S. Krishna, S. Ganesh, K. Sarvabhotla, H. Garapati, H.
Gopisetty, V.B. Reddy, K. Reddy, P. Bysani, R. Bharadwaj, 2008. IIIT Hyderabad at TAC
2008, In TAC2008.
T. Wilson, J. Wiebe, and P. Hoffmann. 2005. Recognizing contextual polarity in phrase-level
sentiment analysis. In EMNLP 2005.

2011/7/29

35

Q&A

2011/7/29

36