A Product Retrieval System
Robust to Subjective Queries
Kenji Sugiki

Shigeki Matsubara

Graduate School of Information Science
Nagoya University
Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan
Email: sugiki@el.itc.nagoya-u.ac.jp

Information Technology Center
Nagoya University
Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan
Email: matubara@nagoya-u.jp

Abstract—In recent years, electronic markets are increasing
rapidly and attracting the attention of customers. In these sites,
people search for products using retrieval systems. They, however,often cannot translate their subjective needs into keyword-based
queries or adapt to the interfaces. In this paper, we describe
a product retrieval system robust to subjective queries. Usinga large amount of consumer reviews, the system allows users
to input natural language queries and retrieves appropriate
products even if the queries are highly subjective. To estimate
the correspondence between a query and a review text, thesystem extracts 3-tuples consisting of a product name/category,its features, and the value from each text using rules based onsyntactic patterns. It calculates each product scores based oncorrespondence rate of 3-tuples and presents ranked relevant
products. In experimental results for a accommodation domain,
it obtained higher average and total precision for 10 queries
compared with a baseline that uses keyword based tf-idf method.
Thus, we confirmed the effectiveness for subjective queries.

a product, customers often use the reviews. Therefore, reviews
are also useful for product retrieval tasks.
We developed a product retrieval system for Japanese accommodation domain. Using large amounts of reviews writtenby bloggers, the system can respond to subjective queries
written in Japanese such as “choshoku-ga baikingu-no yado
(I am looking for a hotel with an all-you-can breakfast.)”.
In retrieval experimental results using about 220,000 reviews
written in Japanese, we confirmed the effectiveness of oursystem.
This paper is organized as follows: Section 2 briefly gives
on overview of our system. Section 3 describes the extraction
process, and Section 4 describes the retrieval process in detail.
Section 5 reports our experiments and results.

I. I NTRODUCTION

Our system allows users to input natural language queries
that may be highly subjective. When inputting queries, thesystem retrieves products matching the query and presents
them in order of relevance scores. To match queries, we use
reviews about products or services. For example, as shownin Figure 1, if the query is about plasma TV and a review
is written about plasma TV “TH-42PX500”, “TH-42PX500”
matches the query.
To determine the degree of correspondence between a queryand reviews, the system transforms these contents into semantic representations, and then computes their correspondence
rate. In this study, we put the information about the queryor the review text into 3-tuples “(object, item, value)”. Whenusers search for the product, they imagine three components:a category, features (items), and values. For example, if there
exists a review that states “gashitsu-ga kire (picture quality isclear)” about plasma TV “TH-42PX500”, the information of
the review is represented as a 3-tuple “(TH-42PX500, gashitsu(picture quality), kire (clear))”. This 3-tuple representation is
identical as that used in related works about opinion extraction
tasks [4], [5], [6], [7], [8], [9]. Many of these studies usedphrasal or syntactic patterns to extract each term or the relation
between the item (or product name) and its value.
These works, however, extracted (subjective) opinion information by only extracting adjective, adverb, and verb as
opinion representations (in our study, value) or constructing

In recent years, such electronic markets as Amazon andeBay have grown rapidly due to increases of Internet users.
These sites deal in vast amounts of products and need toprovide useful search environments for their users. Most sites,
however, just search for prefixed data items such as productnames, categories, and features. Since user needs may be
extremely variable and highly subjective, they often cannot
translate their needs into objective queries. Therefore thesystem can’t easily to accommodate their needs.
On the other hand, there has been a study on a natural
language interface for a product retrieval system in whichusers describe their needs with natural language [1] in an
accommodation domain. In addition, recommendation systems
using a dialogical approach have been proposed [2], [3].
However, these methods have problems transforming natural
language sentences into database query language expressions(e.g., SQL) due to the subjectivity and the variety of natural
language expressions.
In this paper, we propose a product retrieval system robust
to subjective queries written in natural language. We use
product reviews to match the natural language queries andproducts. Recently, electronic market sites generically offer
review services. Additionally, product reviews are often addedby customers in blogs. When customers deciding to purchase

II. P RODUCT R ETRIEVAL S YSTEM U SING R EVIEWS

User query

Dai-gamen-de,
Dai-gamen-de,chideji-nochideji-no
gashitsu-ga
gashitsu-gakire-na
kire-napurazumapurazumaterebi.
terebi.
(PlasmaTV
withbig(Plasma TV with bigscreen
screenandandsupporting
supportingdigital
digitalterrestrial
terrestrial
broadcastingbroadcastingwith
withhigh
highpicture
picturequality.)quality.)Fig. 1.

Plasma TV “TH-42PX500” review

Dai-gamen-de,
Dai-gamen-de,hijo-ni
hijo-nimanzoku-desu.
manzoku-desu.
Toku-niToku-nichideji-nochideji-nogamen-ga
gamen-gakire-dekire-deodoroki-mashi-ta.
odoroki-mashi-ta.
(I(Iwasvery
satisfied
with
the
big
was very satisfied with the big
screen.
screen. II waswassurprised
surprisedIIcan
canwatch
watch
digitaldigitalterrestrial
terrestrialbroadcastings
broadcastingswith
with
clear
clearpicture
picturequality
qualityininparticular.)particular.)

TH-42PX500
TH-42PX500

matching(TH-42PX500,(TH-42PX500, --,,dai-gamen
dai-gamen

I am looking for …

review texts

1. Extraction
Extraction• remove sentences
expressing demand/wish• extract review tuples• process negative expressions• remove stop words

query tuples

query-object,.item, value)(.
.

2. Matching

review tuples

3. Ranking

review-object., item, value)(.
.

Resultretrieval result
1. Product M
2. Product B
3. Product P…

))(TH-42PX500,(TH-42PX500,chideji-gashitsu
chideji-gashitsu,,kirekire(digital broadcasting - picture quality)(digital broadcasting - picture quality)))(high)(high)

Review tuples
Fig. 2.

……..
……..
……

(big screen)(big screen)(TH-42PX500,(TH-42PX500, --,,hijo-manzoku
hijo-manzoku(very(verysatisfied)satisfied)))(TH-42PX500,(digitalbroadcastingbroadcasting--picture
picturequality)
quality),,(TH-42PX500,chideji-gashitsu
chideji-gashitsu(digitalodorokuodoroku(surprised)(surprised)))natural language query

Product P

Example of review matching query

Query tuples(purasuma-terebi(purasuma-terebi(plasma(plasmaTV)TV),,-,-,dai-gamen
dai-gamen(big(bigscreen)
screen)))(purasuma-terebi(purasuma-terebi(plasma(plasmaTV)TV),,chideji-gashitsu
chideji-gashitsu(digital(digitalbroadcastingbroadcasting--picture
picturequality)
quality),,kirekire(high))(high))

Retrieval Step

Extraction Step

Example of matching between query and review tuples

opinion representation dictionaries. On the contrary, our system extracts objective information including facts as well as
opinions. Therefore, since our study does not specialize inopinion representation, it extracts adjectives, verbs, and nouns
as well as adjectives. Here, we determine the 3-tuple of thereview as “(review-object, item, value)” and call it a review
tuple. For example, if a review is writes that “gashitsu-ga kire
(image quality is high)” about plasma TV “TH-42PX500”, thesystem extracts review tuple “(TH-42PX500, gashitsu (image
quality), kire (high))”.
Additionally, the system extracts the 3-tuples from thequery as well as the review. Here, we define the tuple ofthe query as “(query-object, item, value)” and call it a query
tuple. For example, if the user inputs “gashitsu-ga kire-na
purasuma-terebi (Plasma TV with high picture quality)”, thesystem extracts query tuple “(purazuma-terebi (plasma TV),gashitsu (picture quality), kire (high))”. We can judge that
“TH-42PX500”, which is included in the product categoryof plasma TV, matches the query. Figure 2 shows matching
between the tuples extracted from the query and the review
shown in Figure 1.
A system overview is given in Figure 3. The system consistsof the following two components: (1) extracting review tuples
from the review text; and (2) retrieving products that match
the query. In (1) extraction part, the system extracts review
tuples by applying transformation rules based on the syntactic
pattern of modification relations between bunsetsus 1 . In (2)retrieval part, the system transforms a query into query tuples,
calculates the correspondence rate between the query andreview tuples, and then presents products matching the query.
1 A bunsetsu is a Japanese phrasal unit that consists of one or more adjacentcontent words followed by any number of function words.

Fig. 3.

Overview of our product retrieval system

III. E XTRACTION OF O PINION T UPLESIn this section, we describe a method to extract the review tuple “(review-object, item, value)” from the review. We
assume that each review is written about a certain product.
Hence, we decide that the system extracts item-value pairs
in review tuples from the review and determines the products
described by consumers as review objects. To extract itemvalue pairs, we focused on modification relation between
bunsetsus.
We consider the following types of item-value pairs thatappear in review texts:
1) subject-predicate relation“Heya-wa kire-deshi-ta. (The room was clean.)”⇒ (Hotel A, heya (room), kire (clean))“Choshoku-ga insho-teki-deshi-ta. (The breakfast was
impressive.)”⇒ (Hotel B, choshoku (breakfast), insho-teki (impressive))2) modifier-head relation“Totemo shinsetsu-na hoteru-jugyoin-deshi-ta. (She wasvery friendly hotel staff.)”⇒ (Hotel C, hoteru-jugyoin (hotel staff), totemo shinsetsu (very friendly))Hence, we used the patterns based on modification relations
between bunsetsus.
To create rules to transform the review into review tuples,
we investigated the relation between the appearances of items
and values in review texts and the appearance patterns ofmodification relations between bunsetsus.
We confirmed the following frequent patterns: 2 :(1)heya-gaX (The room) → kireY (was clean.)
2 X,Y are bunsetsus that include item or value, and “→” shows themodification relation between bunsetsus.

heya-gaX (The room) → kire-deY1 (was clean) →kaiteki-deshi-taY2 (, and very comfortable)(3)
kire-naY (The nice-looking) → heya-desuX (room)
The following review tuples were extracted from the above
three patterns, respectively:(1)(Hotel A, heyaX (room), kireY (clean))(2)(Hotel A, heyaX (room), kireY1 (clean)),(Hotel A，heyaX (room), kaitekiY2 (comfortable))(3)(Hotel A, heyaX (room), kireY (nice-looking))
Therefore, we created the following three rules. Here, O is
the review-object.
(1)
X→Y⇒ (O,X,Y )(2)
X → Y1 → Y2 ⇒ (O,X,Y1 ), (O,X,Y2 )(3)
Y →X⇒ (O,X,Y )In addition, we created the following rule that includes no
item:
(4)
Y
⇒ (O, - , Y )Example: “shinsetsu-de yokatta-desu. (kindly andgood.)”⇒ (Hotel A, - , shinsetsu (kindly)),(Hotel A, - , yoi (good))
Here “-” shows no element (item or value).
This is because often the item corresponding to the value isomitted and only the value appears in the review text.
To extract additional information, we determined that item
adds bunsetsus including the pattern “noun + no (noun modifier particle)” that modifies item, and also value adds bunsetsus
including adverbs or the pattern “noun + case particle”.
(2)“Hotel agoya” review

Heya-mo hiroku, yokushitsu-no yokuso-mo hiroku, yuttari-to kutsurogeru-nodetaihen kiniitte-imasu (I was able to relax in comfort in the large room, and the bigbathtub was also very fancy. )heya-mo(room) hiroku,(large) yokushitsu-no(bathroom) yokuso-mo(bathtub) hiroku,(big) (inyuttari-to
comfort) kutsurogeru-node(can relax) taihen(very) kiniitte-imasu
dependency tree
(fancy)heya-mo(room) hiroku,(large) yokushitsu-no(bathroom) yokuso-mo(bathtub) hiroku(big) yuttari-to
Item candidates(in comfort) kutsurogeru-node
Value candidates(can relax) taihen(very) kiniitte-imasu
candidate items and values for applying rules(fancy)yokushitsu-no
heya-mo(room) hiroku, (bathroom) yokuso-motaihen(bathtub) hiroku,(very)(large)(big)
yuttari-toyuttari-to
kiniitte-imasu(in comfort)(in comfort)(fancy)kutsurogeru-node
kutsurogeru-node
apply rule 2 (can relax) apply rule 2 (can relax)
apply rule 4(Hotel Nagoya, heya (room), hiroi (large))(Hotel Nagoya, heya (room), yuttari-kutsurogu (relax-comfort))(Hotel Nagoya, yokushitsu-yokuso (bathroom-bathtub) , hiroi (big))(Hotel Nagoya, yokushitsu-yokuso (bathroom-bathtub) ,yuttari- kutsurogu (relax-comfort) )(Hotel Nagoya, - , totemo-kiniiru (very fancy))transform into review tuples

A. Extraction of review tuples
1) Preprocessing: The system divides the Japanese review
text into sentences and gives each sentence a dependency treeusing Japanese dependency parser KNP [10]. Then the systemremoves the sentence including such phrasal expressions fordesire or demand as “˜ shi-te-hoshi (I want someone to ˜)”, “˜nozomashi (I wish ˜)”, and “˜ ba ureshi (It would be nice if
˜)” since users almost never input such queries.
2) Extraction process: The system applies the above 4
rules to dependency trees by giving the following POS pattern
restrictions to the rules:
rule 1 X: “noun + ha/ga/mo (case/dep. particle)”Y : verb, adjective, “verbal noun + suru (aux. verb)”
rule 2 X: “noun + ha/ga/mo (case/dep. particle)”Y : verb, adjective, “verbal noun + suru (aux. verb)”
rule 3 X: “noun + ha/ga/mo/wo/ni/da/desu (case/dep. particle, aux. verb)”
Y : adjective
rule 4 Y : verb, adjective, “verbal noun + suru (aux. verb)”
The rules are applied to each sentence as follows: First,
the system applies rule 2,1 if the sentence has a modification
relation between the item candidate and the value candidate.
Next, rule 3 is applied to these candidates. Finally, if only avalue candidate exists, then rule 4 is applied to it.
An example of the extraction process is shown in Figure
4. First the system parses the sentence and generates theFig. 4.

Example of review tuple extraction

dependency tree. Then the system identifies item and value
candidates and attempts to apply the rules in the order of 2,
1, 3, 4 to these candidates. Finally the system applies rule 2
twice and then applies rule 4. Eventually, the system extracts
five review tuples.
3) Post-processing: The system processes negative facial
expressions and then removes function or stop words. In
the first step, the system unifies negative outward expression
such as “not” and “un-” and adds negative flags to thereview tuple because we want to collectively retrieve such
expressions as “jubun-de-nai (not satisfactory)” , “fu-jubun-da(unsatisfactory)” without retrieving only one expression and to
distinguish negative values from positive values (as outward
expressions).
In the second step, the system removes function or stop
words. We determine that morphemes included in item arenouns and remove other morphemes (prefix, particles etc.).
We also determine that value only includes nouns, verbs, adjectives, adverbs, prefixes, and suffixes and other morphemes
are also removed.
The system then removes words that have almost no
meaning, such as “koto, mono (thing)”, “omou (imagine)”,and “kangaeru (think)” , so the system deals with some
representations as identical.

num.
Q1

IV. P RODUCT RETRIEVALThe system retrieves the product (review-object) thatmatches the query-object of the query tuple by comparing
the item and the value of the query tuple with the review
tuple. Here, query-object is a product class or category suchas notebook PC, car, or hotel.
The system extracts query tuples from the query using thesame method as for the extraction steps. Note that the queryobject of the query tuple is selected by the user from theproduct category. In this paper, we experimented by limiting
product categories to hotel. The head of the query is fixed as“I am looking for a hotel ˜ ”.

EOPmatch :EOPall :EVmatch :EVall :

number of identical bunsetsus
between items of a query and a review
number of all bunsetsus in an item of aquery
number of identical bunsetsus
between values of a query and a review
number of all bunsetsus in a value of aquery

We defined a bunsetsu as the minimum alignment unit.
Based on the correspondence rate, the review-object score
not only can reflect the frequencies of the review tuple
that perfectly matches the query tuple but also reflects thefrequencies of review tuples that only partly matches the query
tuple. We expect that the more a review tuple correspond to
the query tuple, the higher the score of its review-object willbe.
For example, for query tuple qt1 (hotel, taiou (staff accommodation), kaiteki (comfortable)) and review tuple rt1(hotel A, taio (staff accommodation), itsumo kaiteki (always
comfortable)), the correspondence rate cr ate(qt1 , rt1 ) = 1 ×
1 = 1. Meanwhile, for query tuple qt2 (hotel, ryori (meal),

Q2

shokuji-ga washoku-to yoshoku-de eraberu yado
(a hotel that features both Japanese and western
styles)Q3

chekku-auto-jikan-ga osoi yado
(a hotel with a late checkout time)

num.
Q4

subjective queriesquery
furo-ga hiroku-te amenithi-ga jujitsu-shi-te-iru yado
(a hotel with a large bathtub and many thoughtfulamenities)

Q5

heya-no shomei-ga akarui yado
(a hotel with bright room)

Q6

shuhen-ga kansei-na funniki-de, ochitsui-tesugoseru yado
(a hotel in a quiet area where I can have a very
comfortable time)

Q7

konbini-ya resutoran-ga chikaku-te shokuji-ni komaranai yado
(a hotel located near convenience stores orrestaurants)

Q8

ochitsui-te choshoku-ga toreru yado
(a hotel where I can eat breakfast leisurely)

Q9heya-no interia-ga kakucho-takai yado
(a hotel with a high quality interior)

Q10

beddo-ga hiroku-te, negokochi-ga yoi yado
(a hotel with wide beds comfortable)

A. Matching review and query tuples
The system extracts query tuples from the query by the same
method as the extraction process using dependency patterns.
For example, it transforms query “heya-ga kire-de, choshokuga tsui-te, nedan-ga yasui yado (a hotel with clean rooms,
the breakfast is included, and inexpensive.)” into three query
tuples “(hotel, heya (room), kire (clean))”, “(hotel, chosyoku(breakfast), tsuku (included))”, “(hotel, kakaku (price), yasui(inexpensive))”.
Then, the system calculates the correspondence rate between each query tuple and each review tuple to compare theitem-value pair of the query with the review. The correspondence rate is calculated by :⎧ EOPmatch
EVmatch(has a item)⎪⎨ EOPall × EVall
EVmatchc rate(qt, rt) =0.1 × EVall(has no item)⎪⎩0(otherwise)

objective queries
query
chekku-auto-go-mo nimotsu-wo azuka-tte-moraeru
yado(a hotel that keeps baggages after checkout)

Fig. 5.

10 queries used in experimenttotemo manzoku (very satisfied)) and review tuple rt2 (hotelB, ryori (meal), manzoku (satisfied)), correspondence rate
cr ate(qt2 , rt2 ) = 1 × 12 = 12 .
In addition, when the query tuple includes no item, thequery tuple matches the review tuple including no item. Thatis, as shown in the above formula, the system calculates thecorrespondence rate by replacing item parts of the above
formula by 0.1, so that the system retrieves products where
even no item is included in the query tuple.
B. Calculating relevance score
The system calculates the relevance score of each product(review-object) and presents products in the order of these
relevance scores. To calculate the score, we determine that
more appropriate products have the following information:
(1)

(2)

The more customers write identical information, themore authoritative the information is. (like tf (termfrequency))
The fewer products customers write identical information about, the more valuable the information is.
(like idf (invert document frequency))

TABLE I
R ESULTS OF RETRIEVING 10 QUERIES

query
proposal system

Q1
3/3
1.000
7/10
0.700

baseline systemobjective
Q2
Q3
2/9
9/10
0.222
0.900
7/10
8/10
0.700
0.800

Q4
7/10
0.700
6/10
0.600

Q5
10/10
1.000
4/10
0.400

Therefore, the system calculates the relevance score of eachproduct (review-object) by the following formula:
Score(Q, O) =P Fi · IOFi
qi ∈Q

P Fi

=



pfj × c rate(qi , rj )

rj ∈O

IOFi
qi :
oj :
pfi :
ofj :
OF :

OF
= log(+ 1)
ofi + 1

review tuple in query Q
object tuple in object O
frequency that review tuple ik appears
in review-object i
number of review-objects whose correspondence
rate to query tuple j are greater than 0
number of all objects

In this formula, P Fij is calculated by multiplying correspondence rates between review tuple i and query tuple j by
the frequency of the review tuple i. The formula can reflectreview tuples that partly matches the query tuple as well as
review tuples that fully matches. P Fij resembles the frequencyof query tuple j in review-object i and IOFj is logarithm of
the frequency of objects appearing the query tuple j. P Fij
corresponds to the above (1) and IOFj corresponds to theabove (2).
V. R ETRIEVAL EXPERIMENT

Q6
8/10
0.800
5/10
0.800

subjective
Q7
Q8
3/4
4/10
0.750
0.400
7/10
2/10
0.700
0.200

1 Rakuten Travel “The voices of customers”
http://travel.rakuten.co.jp/auto/tabimado bbs top.html

Q10
10/10
1.000
8/10
0.800

total
61/86
0.709
58/1000.580

avg. prec.
0.727
0.580

that index terms are only included nouns, verbs, adjectives,
adverbs, prefixes, and suffixes and considered all reviews ofeach accommodation as one document. It extracts keywords
from natural language query and then retrieves documents(accommodations) using their keywords.
B. Experimental results and Discussion
The result is shown in Table I which represents the precisions of top-10 retrieved accommodations. Compared with thebaseline system, our system obtained higher precisions except
query Q2, and our system’s total and average precision of eachquery are higher.
Our system presents relevant accommodations more properly and potentially could retrieve accommodations, getting
semantic structures between queries and reviews. It, however,has the problem of low recall. It cannot retrieve relevant
accommodations that the baseline system can retrieved as Q1,Q2 because it cannot extract the 3-tuples due to the miss ofapplying extraction rules or dependency parsing.
In contrast, the baseline system achieved higher precision
than anticipated. Since it tend to retrieve accommodations that
include a larger number of reviews and their accommodations
included high frequency keywords, chances are high that theiraccommodations are relevant to queries.
The following are the causes for decreased the precision ofvarious retrieval results:
Q8

A. Outline of experiment
To evaluate the effectiveness of the retrieval method of oursystem, we experimented with an accommodation retrieval
system. Reviews were downloaded from an accommodation
reservation site called Rakuten Travel1 . We used KNP [10]as a Japanese dependency parser and evaluated the system by
precisions of experiment results of 10 queries, shown in Figure
5. For the top-10 accommodations that the system retrieved,
the subject evaluates each accommodation based on checking
all reviews of each accommodation for being relevant/nonrelevant to each query (by considering whether he/her wants
to stay the accommodation or not).
Note that the subject is not one of the authors of this paper.
As a baseline for our experiments, we developed a retrieval system using tf-idf term weighting. We determined

Q9
7/10
0.700
4/10
0.400

Q2

The system could not reflect meanings of the query
very well. That means that since it extracted review tuples “(hotel, - , ochitsuku (leisurely)), (hotel, choshoku (breakfast), toreru (eat))”, each tuples
matched independent reviews. For example, a certain
accommodation included ”kire-na heya-de ochitsuite sugose-mashi-ta. (clean room and had a leisurely
hours.)” , ”goka-na choshoku deshi-ta. (I can eat
lavish breakfast.)” and had no relevant review tothe query. The system should extracted as “(hotel, choshoku (breakfast), ochituku(leisurely) - toreru(eat))” from the query and relevant reviews.
Since the system dropped out meaning “washoku-to
yoshoku-de (both Japanese and western styles)”, it
can not retrieve relevant accommodations well. If it
extracted as “(hotel, washoku (Japanese), eraberu(feature)), (hotel, yoshoku (Japanese), eraberu (feature))” from the query and relevant reviews, it could
match them.

VI. C ONCLUSION
In this paper, we presented a product retrieval system robust
to subjective queries written in natural language. Using vast
amounts of product reviews written by consumers, the system
responded subjectively a variety of representation of user
queries.
Using with the results of an experiment in the hotel domain
with about 220,000 hotel reviews, we confirmed the productretrieval effectiveness of the system. For higher recall, it need
to be applied additional methods (e.g., developing item-value
pair dictionary).
In the future, we must deal with a variety of lexicons withsimilar meanings such as “clean,” “clear,” “nice-looking,” and“good-looking.” Hence, we will examine the retrieval method
using a thesaurus.
R EFERENCES[1] M. Dittenbach, D. Merkl, and H. Berger, A Natural Language Query
Interface for Tourism Information, Proceedings of the 10th International
Conference on Information Technologies in Tourism (ENTER 2003), pp.
152–162, 2003.
[2] J. Chai, V. Horvath, N. Nicolov, M. Stys, and N. Kambhatla, Natural Language Assistant: A Dialog System for Online Product Recommendation,
AI Magazine. Vol. 23, no. 2, pp. 63-75. 2002.
[3] D. Mcsherry, Explanation in Recommender Systems, Artificial Intelligence Review, Vo. 24, Num. 2, pp. 179–197, 2005.
[4] M. Hu and B. Liu, Mining Opinion Features in Customer Reviews,
Proceedings of the 19th National Conference on Artificial Intelligence(AAAI 2004), pp. 755–760, 2004.
[5] B. Liu, M. Hu, and J. Cheng, Opinion Observer: Analyzing and Comparing Opinions on the web, Proceedings of the 14th International World
Wide Web Conference (WWW 2005), pp. 342–351, 2005.
[6] J. Yi, T. Nasukawa, R. Bunescu, and W. Niblack, Sentiment Analyzer:
Extracting Sentiments about a Given Topic using Natural Language
Processing Techniques, Proceedings of the Third IEEE International
Conference on Data Mining (ICDM 2003), pp. 427–434, 2003.
[7] A.M. Popescu and O. Etzioni, Extracting product features and opinionsfrom reviews, Human Language Technology Conference (HLT) / Conference on Empirical Methods in Natural Language Processing (EMNLP),
pp. 339–346, 2005.
[8] N. Kobayashi, Opinion Mining from Web documents: Extraction and
Structurization, Doctoral Dissertation, Nara Institute of Science andTechnology, 2006.
[9] K. Tateishi, Y. Ishiguro, and T. Fukushima, A Reputation Search Engine
That Collects People’s Opinions Using Information Extraction Technology, IPSJ Transaction on Databases, Vol. 45 No. SIG 7 (TOD 22), pp.
115–123, 2004.
[10] KNP version 2.0, http://nlp.kuee.kyoto-u.ac.jp/nl-resource/knp.html