2830

Ordering Concepts Based on Common Attribute Intensity

Tatsuya Iwanari♠ Naoki Yoshinaga♣ Nobuhiro Kaji♥ Toshiharu Nishina♦ Masashi Toyoda♣ Masaru Kitsuregawa♣,★

♠Univ. of Tokyo ♣IIS, Japan ♥Yahoo Japan Corporation ♦Rakuten Inc. ★NII, Japan

Task Setting

Introduction

Ordering concepts from various perspectives
is essential to make decisions in a daily life

Task scheduling Choosing a restaurant

Voting(othersʼ reviews){≻
http://www.mag2.com/p/news/152554

reliable? smart?

important?

★★★

urgent?

，，+★

tasty? cheap?

Vector Open Stock, https://www.vectoropenstock.com/

concepts ordered byattribute intensity

a set of concepts+ an adjective}≻Ordering

Delicious≻

What is thethe ordering maximizing averaged
gold standard? Spearmanʼs ρ against human orderings

Question: Can we derive our views on concept
ordering from what we have written?

Proposal Extract pieces of ordering evidence from social media and integrate them in supervised learning
Train

Delicious + {Featurevector
Train

,

𝒗𝟏 = …
𝒗𝟐 = …
𝒗𝟑 = …

,Test}

Large +{，}

𝒖𝟏 = …
𝒖𝟐 = …
𝒖𝟑 = …

Socialmedia text• Generate a feature vector
for each concept• Learn SVM to minimize# of incorrect orderings oftwo items against gold
depcooc
PMI, PMI,…
+,-≻≻

Evidence≻≻

Example≻ ≻𝑣⃗𝑖 =SVM/SVR

SVM/SVRGold，

Ranking SVM [Joachims ʻ02]

SVR [Drucker et al. ʻ97]
(Support Vector Regression)• Generate a feature vector
for each ordering• Learn SVR to map an ordering
to Spearmanʼs ρ against gold

≻ ≻+,-(1 ≤ 𝑖 ≤ 𝑵)

Ants are so small that elephants canʼt kill them

Ranking SVM 𝑊 [feature weights]

SimileHe is as brave as a lion

Comparative

Elephants are larger than dogs

counts to feature values

0.6

0.2

0.3

𝑝 noun­adjective co­occurencelog
𝑝 noun 𝑝 adjective
depScore𝑓 = 𝑊𝑣𝑖≻

Evaluation

Correlation against gold-standard orderings[Data]• Blog articles (Japanese)- 2005-2013- > 1 million users- ~ 2 billion sentences

Subjective Objective[Queries]• 35 queries (nouns & adj)
- concept to instance nouns- objective to subjective adj[Methods]• Proposed: Ranking SVM / SVR- 4 types of evidence- Linear kernel• Baseline: PMI of noun-adj
co-occurrence [Turney ʼ02]
[Train & Test]• Leave one out cross-validation• Gold: the ordering maximizing
averaged Spearmanʼs ρagainst 7 human orderings
Human orderings
showed high correlation-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
mammal (large)
temple (old)corner store (numerous)
conveyance (fast)
fruit (small)country (warm)
programming (slow)
MLB team (famous)browser (friendly)
animal (lovely)
temple (famous)bird (cute)weather (unpleasant)programming (easy)
country (safe)
fast-food chain (tasty)
fruit (sweet)
corner store (useful)town (fashionable)
appliance (useful)alcohol (delicious)
ﬂower (beautiful)cartoon (amusing)conveyance (comfy)
food (yummy)
automaker (healthy)
jewel (elegant)
instrument (soothing)ﬂesh (preferable)
mammal (clever)
manufacturer (famous)
city (safe)
vegetable (tasty)
coffee shop (likable)
sports (entertaining)

Ranking
Method Baseline
SVM
ave. ρ
0.274
0.441

N!

[regression function]≻≻

Estimated ρ = 𝑓 𝑢𝑖0.3≻

≻0.9

≻

≻

0.6…

e.g.,elephantlarge = PMIcooc,
PMI,…
largelarge

≻𝑓 𝑣⃗

SVR

Co-occurrence Look how large that elephant is!
Dependency…

𝑢𝑖 = …
(1 ≤ 𝑖 ≤ 𝑵!)

SVR
0.366

Human
ave. ρ

0.750

Ablation test for Ranking SVM
Featuresave. ρ
0.441 Simile
All features⊖ co-occurrence⊖ dependency⊖ simile⊖ comparative

0.391
0.407
0.292
0.424

works
well

Aggregating evidence is effective

ExamplesThank your fortaking time!!
Want to see more?
Please visit:
http://www.tkl.iis.utokyo.ac.jp/~nari/ijcai-16/