SNS Retrieval Based On
User Profile Estimation
Using Transfer Learning
From Web Search
Daisuke Kataoka and Keishi Tajima
Kyoto University, Japan1

Background Twitter is useful for opinion mining
Ex) Target marketing of new products

I want to know
students’ opinions
about Smart SpeakerMarketerStudentTwitter1m
Tom
Smart speakeris interesting To know the opinions from students,
the marketer search Twitter with Smart Speaker student
2

Purpose
Tweet Retrieval specifying
topics and user types
Ex) posts about Smart Speaker posted by students

Topic:
User type:

Smart Speaker
studentex) adultmanwoman
TokyoKyoto

 In this way, search by these two conditions
3Problems on Twitter search However, a user type do not appear in tweet itself① search withtopic and user types→ low recall② only topic keywords→ low precision

Smart Speaker student

4yTom
My brother …
… Smart speaker Users do not write their affiliation on each tweet

Smart Speaker

2sLeonard… smart speakerhas problems…
1mChristopher
We bought …
… Smart Speaker…
 including posts by non-student users4Proposal We propose Re-Ranking methodintegrated with user profile estimation
 Search by Topic query Re-rank based on Profile query
student

Smart Speaker
J.K.

3h

My brother …
… Smart speaker

Eddy

4h

Smart Speakeris interesting…

Johnny… I want a new
Smart speaker

5hundergraduate

UpIT engineer

Downmaster’s
student

Up5Approach Regard user profile estimation
as a classification problem
ー Estimate a likelihood of having the target user profileー Give higher ranks to the tweets with higher likelihood

 Judge from the tweets and their context †tweetcontext

Smart SpeakerAneesh5m

Smart speaker …
… is usefulprofilepast tweet

follower/followee’s
profile† D. Kataoka et al., Context-aware Relevance Feedback over SNS Graph Data, WI 20176

Hypothesis 1 on user profile estimation
Search Engine is useful for
obtaining vocabulary about user profiles
 Search engine returns documents relevant to query→ By retrieving by a keyword representing the type of users,
documents describing the target user type are obtained
student

Wikipedia article
about student

Job hunting site
for student

Article on accidents
caused by student7

Hypothesis 2 on user profile estimation
Web search includes more vocabulary
for identifying the target user types
than tweets retrieved through Twitter search
student

Twitter Relevant but
less vocabulary

A
Students are jealous

B
Adults are busier
than students

CI‘d like to go abroadwhile I‘m a student

student

Web

Wikipedia:STUDENT - Wikipedia
A student is primarily a person
enrolled in school l ...

 much morevocabularyStudent Training Modules
Student Training Modules.
This set of modules helps ...

 The more documents
describing
the user typesNI LabVIEW Student Edition
The LabVIEW Student Edition
contains the following soft ...

8

Transfer Learning
Web Search

student

Obtain samples
for training

adult

transferstudent

2.0

school1.5

teacher

1.2

adult

2.3

office

1.4

work

1.3

Train

Classificatio
nModelTwitter Search
Search and Re-ranking
by User-profile estimation

student

1.5

school2.0

teacher

1.2

Predict

Smart Speaker9Training phase Collect positive and negative samples as datasetsand train a classification model① Choose words
describing thetarget user typeand its opposite

Web Search
student

adult② Search③ Each document is
TF-IDF-vectorized⑤ Train(student, adult)(man, woman)（Kyoto, Tokyo, ...）
（soccer, baseball, ...）…

「student」「adult」
labels are given toeach document ④ClassificationModel10

Applying phase Apply the trained model on Tweet Retrieval
ー Input each user vector and obtain prediction score
ー Rank tweets by the likelihood of the target user type
User vector past tweet

profileTopic queryfollower

Smart Speaker

Profile query
student
LikelihoodJohn5m

Smart Speaker …
…

Debra

1hstudent …
… Smart Speaker …

JosephStudent/AdultC
lassification
Model

0.7
0.9

2y

Smart Speaker …
… …

0.3

Reranked
Search ResultDebra

1hJohn5m

Joseph2ystudent …
… Smart Speaker …
Smart Speaker …
…
Smart Speaker …
… …

11How to make a user vector on Twitter † Represent Twitter data as a graph Propagate the word weight Weights for user vector decay through the graph Low weight is assigned to the context far from a tweet
SNS Graphtweet

SpeakerToday Today
1.5
isDB class.

×0.25
author

×0.8

Kyoto
1.5
Kyoto Univ.

Speaker 2.0

CSUniv.
Kyoto

1.50.5class

1.2CS

2.0

Kyoto

0.4×0.25×0.25・・・

follower×0.25

Kyoto
Kyoto 2.0
2nd2nd1.0

 The weight of「Kyoto」is computed as 2.0 × 0.25 × 0.8 = 0.4† D. Kataoka et al., Context-aware Relevance Feedback over SNS Graph Data, WI 201712

Experiments
 Comparison with four methods

3. Web (Proposed method)
1. Twitter (Baseline 1)
2. Twitter Neg (Baseline 2) 4. Web Neg (Proposed method)* In Neg method, negation operator is used on Web search Ex) student -adult
1. Twitter

2. Twitter Neg

3. Web

4. Web Neg

Twitter Search

Twitter Search

Web Search

Web Search

student

adultstudent -adult

adult -student

student

adultstudent -adult

adult -student The search results would not contain the documents

about the opposite user type13

Experiments

student

1

111271

2

121272

3131273

20

...

...

...
10... ...

1280

14

Experiments
 Queries: 5 (topic query) * 6 (profile query) = 30 queries
Occupation

RegionGenderstudent/adult Osaka/Tokyo man/woman
Topic Query

Profile Query

Category

Intention

1

Smart Speaker

student

Occupation

posts about Smart Speakerby students2Osaka Castle

OsakaRegion

posts about Osaka Castle
by the persons from Osaka

3Star WarsmanGender

posts about Star Wars by men
15Experimental Results
ー：Twitter ー：Twitter Neg ー：Web ー：Web Neg

 In the occupation, Web Neg method showed highest nDCG@10,30
nDCG@10 occupation

nDCG@30 occupation
0.6
0.5
0.4
0.3
0.2
0.1
00.6
0.5
0.4
0.3
0.2
0.10

< Top-n results In the region, Web Neg method showed highest nDCG@10,30
nDCG@10 region
0.60.50.4
0.3
0.20.10nDCG@30 region
0.60.50.4
0.3
0.2
0.1016

Experimental Results
ー：Twitter ー：Twitter Neg ー：Web ー：Web Neg

 In the gender, Twitter Neg method showed highest nDCG@10,30
nDCG@30 gendernDCG@10 gender
0.6
0.5
0.4
0.3
0.2
0.1
00.6
0.5
0.4
0.3
0.2
0.1017

ー：Twitter ー：Twitter Neg ー：Web ー：Web Neg
nDCG@10 occupation

nDCG@10 region

nDCG@10 gender
0.6
0.5
0.4
0.3
0.2
0.1
00.6
0.5
0.4
0.3
0.2
0.1
00.6
0.5
0.4
0.3
0.2
0.1
010204080160320 640 1280

10204080160320 640 1280

10204080160

320 640 1280

18

Discussions about effective features
 Examine the effective features of the Random Forest
based on the feature importance calculated by ”gini importance”†
Occupation Models©︎©︎ There are more effective features on Web and Web Neg† Breiman, Friedman, "Classification and regression trees", 1984.

19

Summary & Future works
Summary・We proposed a new tweet ranking method
using transfer learning from Web domain
・Our proposed method achieved higher nDCG
in the occupation and region categories,
but low in the gender category

Future works

20