BackgroundA huge amount of text data on the Web

Collecting Evaluative Expressions
for Opinion Extraction

 Including human opinions

Automatic extraction and classification of humanopinions have been receiving interest
Classifying documents
into recommended or notrecommended (Pang 04,Turney 02)

Computational Linguistics Lab.
Nozomi Kobayashi

Webdocuments

notnotrecommended
recommended recommended
recommended

Subject Attribute

Extracting elements
which constitute theopinionAA
B

price
seat
interior

Valuehighcomfortablepoor(Tateishi 01, Murano03)

Aim

Subject/Attribute/ValueTo develop the automatic method to extract
human opinions

Subject: product name, service name Extraction of three elements〈Subject, Attribute, Value〉
Subject: The subject of evaluation
Attribute: A particular aspect of a Subject
Value: A value of the Attribute
The Prius is quiet andthe seats are very comfortable.
Prius, φ, quiet〉〈Prius, seats, comfortable〉〈

 Jaguar, Opel, Prius, BMW, …

Attribute: constituents, qualitative andquantitative properties
domainAutomobile weight, seat, engine, space
ComputerCPU, speed, usability, design
Value: quantity or quality of the Attribute
domainAutomobileComputer

heavy, elegant, good, exciting
attractive, poor, slow, robust

Related work

ApproachOpinion extraction from Web documents

Semi-automatic method that uses specificcooccurrence patterns

 Using dictionaries and patterns (Tateishi01,Web
Web文書
documents

Murano03) Creating dictionaries manually Attribute/Value expression are domain-dependent
Æ costly

How to reduce the cost of creating a list ofevaluative expressions?

… the handling is excellent ….
〈Attribute〉is 〈Value〉

Approach

Process of collecting Attribute/Value expressions

Semi-automatic method that uses specificcooccurrence patterns

Cooccurrence
Patterns

Web
Web文書
documents… the handling is excellent ….

Attribute is Value
Candidategeneration

Attribute〉is 〈Value〉〈More likely to be an Attribute expression

dictionaries

Documents

The handling is excellent ..
The car is large so that …
… cost is good because ofdictionary

Filter out

excellent

Value
Non-Value
dictionary
dictionary
excellent, large,
good
AttributeNon- Attributedictionary
dictionary
cost
car

handling

Candidateexpressions

human labeler
Candidate selection

×cost, handling, carExperiment (automobile and videogame)

SubjectBMW
dictionary

Resources
Document collections (from review sites)

Semi-automatic collection

Manual collection

WebWeb文書
Judge high-score I bought this car because
documents
expressions
I thought it looked really cool.
(score top 100) It's a very nice looking car,labeler but its performance is useless.

manualsemi-auto

Automobile

Videogame [sentence]

sentence][5,0002,00090,000230,000

Initial Subject dictionary
 Automobile：389 expressions (Jaguar, Opel, Chrysler,…)

System Videogame：660 expressions (Dark Chronicle, Seaman,…)Candidateexpressions

Value
Attributedictionary dictionary

Initial Attribute dictionary： 7 expressions

Tag attribute/valu
expressions using
a tagging tool

 cost, price, service, performance, function, support,design

Initial Value dictionary： 247 expressions

Value
Attributedictionary dictionary

 good, beautiful, bright, favorite, high, ．．

Cooccurrence patterns

Number of expressions collected (Automobile)

Attribute extraction patterns
 Subject no Attribute Attribute ga/ha/mo/no/wo Value
 Value na Attribute

Value extraction patterns
 Subject no Attribute ga/ha/mo/ni/wo Value Attribute ga/ha/mo/ni/wo Value Value na Attribute Value# of expressions collected Subject no Attribute ga/ha/mo/ni/wo Value

Attribute（semi-auto）

Attribute（manual）

Attribute (If Value is adjective)
Time [hours]

Value（semi-auto）
Value（manual）

Attribute（semi-auto）

Value（semi-auto）

Value
Semi-automatic
1000
method is efficient
1000（manual）
Attribute（manual）

Number of expressions collected (Videogame)# of expressions collected# of expressions collected

Number of expressions collected (Automobile)

Value（semi-auto）
Attribute（semi-auto）

Value
Attribute （manual）（manual）

Time [hours]

Time [hours]

Coverage of expressions collected

Discussion (coverage)
The main reason why coverage is not high:Automobile

Semi-auto 349
3676

manual403

Semi-auto
3246

498

manual474

our semi-automatic method doesn’t identify
candidates whose span exceeds the boundary of thebase phrase (bunsetsu)
 Value expressions (30% to 40%)
 Idioms

Semi-automatic collection covered half ofexpressions
Videogamemanually collected
Semi-auto
2399

manual265
520

Semi-auto 347
1970

Attributemanual654

y Nitari yottari (all alike)y Jigen-ga chigau (be on a different level) Attribute expressions (30% to 40%)

 Hierarchical relation
between attribute (A no B)
y body-no ookisa (size of body)

bodysize

Value

Examples of expressions collectedConclusion

Automobile

Opinion extraction from given documents

Shaju
body-no ookisa(weight)(size of body)seijakusei akarui sikichou(quietness) (light in color tone)color

miwakuteki 100%(alluring)(100%)
dame
tsume-ga amai(no good) (easygoing) Extraction of 〈Subject，Attribute，Value〉
 We propose a semi-automatic method to collect

Attribute/Value expressions
 Cooccurrence patterns worked well across different

domainsVideogameCG
chiteki-na senryaku
soudaisutoresu-ga tamaru(CG)(intelligent strategy)(magnificent) (stressful)
free mode kyara-no graphic
comicalki-ga nukenai(graphic of characters)(exciting)※ left: collected only by semi-auto method
right: collected only by manual method Semi-automatic method was more efficient thanmanual method

Future work
Extraction of 〈Subject, Attribute, Value〉using
dictionaries
 Classifying whether the collected value expression

is used to express an “evaluation” in a given
sentence
not evaluation
I bought the 300M brand new off the lot.
The seats are very comfortable.
 Identifying the Subject/Attribute corresponding toa Value
The seats are very comfortable.
300M, seats, comfortable〉〈