IBM Research – Tokyo

Mass and Social Media Corpus Analysis
after the 2011 Great East Japan Earthquake

Shosuke Sato, Fumihiko Imamura (Tohoku Univ.)
Michiaki Tatsubori (IBM Research – Tokyo)

WWW 2012 Workshop on SWDM,Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Twitter as a Source of “Major” News
§ 85%+ topics are headline news or persistent news innature [Kawak, WWW 2010]– Major real world events, celebrities, and movies– “We claim that Twitter users tend to talk about topics
from headline news and respond to fresh news.”
Subcritical
Exogenous

7.3%
ephemeralCritical
10^4(e.g. #thoughtsintheclub)

Endogenous

6.9%

54.3%
headline news

10^5(timely breaking news)
10^1

31.5%
persistent news

10^4(e.g. pro sports team)
WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Goal: Focus on Subcritical Topics§ What are Twitter-specific topics?

Subcritical
Exogenous

7.3%
ephemeral

Critical
10^4(e.g. #thoughtsintheclub)

Endogenous

6.9%

10^554.3%
headline news
(timely breaking news)

10^1

31.5%
persistent news

10^4(e.g. pro sports team)Not-specificto Twitter

Might bespecific to Twitter
WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Proposed Algorithm

social media
mass media( social media )( social media )) / idf (t , D)
tf (t , d=
( mass media )( mass media )tf (t , d) / idf (t , D)

WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Preliminary Experiments§ Tweets during March 27-29 out of ones during March16-29– 1258679 tweets / 12944 y! news (March 16-29, 2011)– 261790 tweets / 2335 y! news (March 27-29, 2011)(in Japanese)
WWW 2012 Workshop on SWDM, Lyon, France(in Japanese)
April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Frequently Tweeted Noun Phrases.. should be enough ..
.. this .. strong yen ..
.. truly .. friendly nation ..
.. such .. child ..
.. people .. not ..
.. unplanned .. blackout ..
.. wide-are .. blackout ..
.. serious .. situation ..
.. another .. man ..
.. that .. man ..
.. economical .. damage ..
.. small .. companies ..
.. luxurious .. book ..
.. this .. time ..
.. same .. age ..
.. such .. behavior ..
WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Relatively-Frequently Tweeted Noun Phrases.. should be enough ..
.. such .. child ..
.. truly .. friendly nation ..
.. this .. strong yen ..
.. same .. age ..
.. heavy .. laundry ..
.. wasting .. electric bill ..
.. what .. method ..
.. warm .. air ..
.. such .. behavior ..
.. good .. fluorescent light ..
.. originally .. technician ..
.. wonderful .. challenge ..
.. familiar .. brownout ..
.. bright .. fluorescent light ..
.. important .. issue ..
WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Frequently Used Noun Phrases in Y! News 1/2.. this .. day ..
.. large .. damage ..
.. significant .. damage ..
.. detailed .. assistance ..
.. catastrophic .. devastating ..
.. economical .. issues ..
.. high .. radiation dose ..
.. high .. consistency ..
.. original .. election ..
.. agile .. shakedown ..
.. same .. nuclear power station ..
.. same .. centre ..
.. critical.. situation ..
.. this .. water ..
.. high .. number ..
.. critical .. affairs ..
WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Frequently Used Noun Phrases in Y! News 2/2.. quick .. reconstruction ..
.. high .. value ..
.. strong .. power ..
.. special .. editing ..
.. some sort of .. route ..
.. large .. impact ..
.. that .. result ..
.. same .. association ..
.. right .. information ..
.. high .. operation ..
.. different .. place ..
.. original .. tense ..
.. this .. earthquake ..
.. original .. rep from Japan ..
.. -like .. symptom ..
.. -like .. Radioactive material ..
WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation

IBM Research – Tokyo

Concluding Remarks and Lessons Learned§ Proposed a method for extracting social mediaspecific words/phrases compared with mass media§ Should not use news body - news headlines may
make sense§ Good dictionary is necessary to translate between
mass media (formal) and social media (informal &formal)

WWW 2012 Workshop on SWDM, Lyon, France

April 17, 2012

© 2012 IBM Corporation