World Seer : A Realtime Geo-Tweet Photo Mapping System
Keiji Yanai
Department of Informatics, The University of Electro-Communications
Chofu, Tokyo 182-8585 JAPANyanai@cs.uec.ac.jp
ABSTRACTTwitter is a unique microblog which is diﬀerent from conventional social media in terms of its quickness. Many Twitter’s users send messages to Twitter on the spot with mobile phones or smart phones, and some of them send tweets
with photos and geotags, which can be regarded as being
geotagged photos. Geotagged tweet photos are very usefulto understand what happens currently over the world. Inthe demo, we introduce “World Seer” which is a real-time
geo-tweet photo mapping system. Users can see the latest
geo-tweet photos related to given keywords and areas on theonline maps. The system shows geo-tweet photos not only
on the map, but also on the street-view. In addition, forsome parts of the geo-tweet photos, the system can show
representative photos for the given locations and the given
times employing the GeoVisualRank method which takes
into account both visual features of photos and proximity ofgeotags.

Categories and Subject Descriptors

Figure 1: World Seer: a realtime geo-tweet photo
mapping system. This ﬁgure shows the latest geotweet photos related to “noodle”. You can get toknow what kinds of “noodle” people is eating over
the world now from the geo-tweet photos. In addition, from the street-view, you can get to know whatthe places where “noodles” are eaten are like, too.

H.4 [Information Systems Applications]: MiscellaneousGeneral Terms
Algorithms, Design, Experimentation

Keywords
geotagged photo, geotag, Twitter, geo-tweet

1. INTRODUCTIONTwitter is a unique microblog, which is diﬀerent fromconventional social media in terms of its quickness. Many
Twitter’s users send messages, which is commonly called“tweets”, to Twitter on the spot with mobile phones or smart
phones, and some of them send photos and geotags as wellas tweets. Then, we can regard Twitter as a data source ofgeotagged photos. In fact, we collected about 75 thousand
geotagged photos a day on average in December 2012 via
Twitter Streaming API. We think that Twitter is a promising data source of geotagged of photos, while Flickr has been
the most popular data source of geotagged photos in thePermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for proﬁt or commercial advantage and that copies
bear this notice and the full citation on the ﬁrst page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior speciﬁc
permission and/or a fee.
ICMR ’12, June 5-8, Hong Kong, China
c 2012 ACM 978-1-4503-1329-2/12/06 ...$10.00.
Copyright °research community of multimedia so far. Since the characteristic of Twitter is quickness and on-the-spot-ness, thephotos on Twitter are diﬀerent from the photos on Flickr.
Flickr has many travel photos, while Twitter has many photos related to everyday life such as food, weather and street
scene. Therefore, we think that geo-tweet photos are more
useful to understand what happens currently over the world
than Flickr geo-photos.
We have been collecting geo-photo tweets continuously
since February 2011. Currently, our geo-photo tweet databasehas more than 20 million geo-photo tweets, which includes
ones related to some special events such as the huge earthquake hit in Japan on March 11th, 2011.
To make collection of geo-photo tweets successful, we implemented not only geo-tweet collecting system but also asystem showing geo-tweet photos on the online map in thereal-time way in order to monitor the status of the geo-tweet
collection. Although the original objective of the geo-tweet
mapping system is just monitoring, currently the system has
been extended a lot so that we can get to know what happens
over the world with geo-tweet photos, and then we named
the system “World Seer”.
In the demo, we introduce “World Seer” which is a realtime geo-tweet photo mapping system as shown in Fig.1.
This ﬁgure shows the latest geo-tweet photos selected by thesearch keyword, “noodle”, from which you can get to knowwhat kinds of “noodle” people are eating over the world now.
Users can see the latest geo-tweet photos related to given
keywords and areas on the online maps, and can search morethan 20 million geo-photo tweets stored in our geo-photo
tweet database.

The system shows geo-tweet photos not only on the map,database, or search the database for the tweets corbut also on the street-view. Seeing photos over the streetresponding to the given search keywords and/or theview photos corresponding to the geotag place is also quite
given term. This sub-system is available on the Web 1 .
helpful to know what the place is like. The geo-tweet photo
Geo-tweet photo clustering and ranking system (Fig.3)
shown in the system window and its geo-location on the map
shows geo-photo tweet clusters on the map, and repare switched one after another every ﬁve seconds automatiresentative photos corresponding to each cluster. Imcally.
age clusters are constructed by applying the mean-shift
In addition, for some parts of the geo-tweet photos, theclustering method to a set of 2D geotag vectors (latisystem can show representative photos for the given locatude and longitude), and representative images corretions and the give dates employing the GeoVisualRank method [1]sponding to each cluster are selected by the GeoVisualwhich takes into account both visual features of photos andRank method [1] which is a PageRank-based ranking
proximity of geotags. To do that, the backend part of themethod of geotagged images considering both visualsystem downloads photo JPEG ﬁles and extracts visual feasimilarity and geo-location proximity.
tures in advance. Due to computation loads, currently werestrict photos from which we extract visual features to theGeo-tweet metadata database stores metadata on exgeo-tweet photos tweeted in Japan. An example is shown intracted geo-tweets which contain URLs of images.
Fig.3.
Geo-tweet photo and visual feature database stores image JPEG ﬁles and corresponding BoF vectors.
To obtain tweets from Twitter, we can use two kinds ofTwitter Web API, Twitter API and Twitter Streaming API.
Since we can obtain tweets only for recent ten days at mostvia Twitter API, we are collecting geo-tweets via Twitter
Streaming API. Through Twitter Streaming API, a hugeamount of tweets are broadcasted continuously. In the system, the recording system selects only the tweets which havegeotags and photo URLs from the tweet stream. We havebeen collecting geo-photo tweets continuously since February 2011. Currently, our geo-photo tweet database has morethan 20 million geo-photo tweets. We collected 2.3 million
geo-tweet photos a month last December, which is equivalent to one geo-tweet with a photo per second. The numberof geo-tweet photos a month is increasing gradually.

Figure 2: Overview of the system architecture.

2. OVERVIEW OF THE SYSTEMAs shown in Fig.2, the system consists of the followingﬁve sub-systems and two databases:
Geo-tweet recording system receives tweet stream ﬂowing through Twitter Streaming API, picks up geo-tweets
having photos, and stores them on the metadata databasein a real-time way. It handles only textual data. We
are collecting geo-tweets from whole the world.
Geo-tweet photo downloading system extracts image URLs
from each tweet stored in the metadata database, downloads image ﬁles from Twitter photo aﬃliate sites such
Figure 3: Geo-tweet photo clustering system. The
as Twitpic.com, Instagr.am and yfrog.com., and store
geo-tweet photos in this ﬁgure are recorded at theimage ﬁles into the photo and feature database. Curday of the huge earthquake hit in Japan on Marchrently, photos to be downloaded is limited to the pho11th, 2011.
tos taken in Japan.
Geo-tweet photo feature extraction system extracts SIFT
features, convert them into bag-of-features (BoF) rep3. REFERENCES
resentation based on the pre-computed codebook, and[1] H. Kawakubo and K. Yanai. Geovisualrank: A ranking
store BoF vectors into the photo and feature database.
method of geotagged images considering visual
similarity and geo-location proximity. In Proc. of theGeo-tweet photo mapping system (Fig.1) shows geoACM International World Wide Web Conference, 2011.
tweet photos on the Google Maps and the Google StreetViews.
1
http://mm.cs.uec.ac.jp/geotwphoto/
It fetches the latest geotag tweets from the metadata