DEIM Forum 2017 H8-4

Conditional density estimation for the origin of social media geolocation
Hayate ISO† , Shoko WAKAMIYA† , and Eiji ARAMAKI†† Graduate School of Information Science, Nara Institute of Science and Technology
8916-5 Takayama-cho, Ikoma, Nara 630-0192, Japan
E-mail: †{iso.hayate.id3,wakamiya,aramaki}@is.naist.jp
Abstract Despite the importance of the tweet geolocation, most of the research estimates the user geolocation
through user timeline, not the tweet geolocation, because it is diﬃcult to detect precise geographic coordinates fromonly a single tweet. In light of this, this study centers on estimating the single tweet geolocation through a density
estimation approach. The advantage of density based model expresses the uncertainty of the tweet geolocation as
the spread of the distribution. The proposed model reveals that a high consistency with human judgment, indicating
the practical availability.
Key words Microblogs, Natural language processing, Geographic Identification, Density estimation, Machine
Learning

1. Introduction
The recent rise in popularity and scale of social media hascreated a growing necessity for social media-based applications. One of the strong advantages of social media applications is the availability of various meta information, such as
temporal information [5], [7], spatial information [2], [9] andso on. Although temporal information is widely available,
spatial information is often inaccessible for privacy reasons,causing a diﬃculty of precise location-based applications. Infact, a study reported that fewer than 0.5% of tweets include
GPS information [3], thus motivating many location estimation studies [1], [4].

(a) Nishinomiya is super hot.

(b) I’m really hungry.

Error Distance: 7.7 (km)Error Distances: 77.4 (km)Gold: Hyogo, Human: Hyogo

Gold: Tokyo, Human: UNK

Figure 1: Estimated geographic distribution of two diﬀerent
tweets. “Gold” and “Human” represent the true locationand the human estimation, respectively. The intersection ofdotted line is the true geolocation.

Nowadays, two types of location estimation tasks havebeen tackled; (1) to estimate the most probable user locationfrom a user timeline (the user level geolocation task) and (2)to estimate the most probable geolocation for each tweet (themessage level geolocation task). Although both of the tasks
respectively have their own importance, most studies focus
on the former task because of the task diﬃculty of the lattertask.
Figure 1 shows the typical examples for the tweet and estimated geolocation. Although the former tweet is easy toidentify the correct geolocation, the latter one is hard to that.
To quantitatively evaluate the diﬃculty of the message
level geolocation, we have created a human annotated datathat a worker labels geolocation for each tweet. The data
reveals that only 10% of the tweets are annotated the geolocation by humans (low recall ) (Table 1). In addition, 12%of annotated data were miss judged by humans. It is lower
precision than our expectation. (low precision).

According to this observation, this research employs amodel to represent the certainty of the location as a mixtureof probabilistic distributions. The density based approach
tracts the geolocation uncertainly shown in Figure 1b. Thisdensity-based approach has another practical advantage; itenables the probabilistic values as follows:•posted at Kyoto prefecture with over 50%,

•posted within 50 km around Mount Fuji with over 20%.

These probabilistic values are useful for various social media
services, location dependent advertising optimization, andvoting prediction.
In experiments, we evaluate our model under the hierarchical test dataset based on the human inferences. The experiments also showed a gap between human inference andactual locations.
Note that a recent study [8] was undertaken to estimate

Annotated ratio (%)

p(y|wj ) =Detailed (Prefecture) Rough (Region) Unknown
9.1

1.3

Human89.6Precision (%)

K∑πkj N (y|µkj , Σkj ),k=1where y ∈ R2 represents geographic coordinates (latitudeand longitude). wj represents a word indexed in j, πkj isagreement (%)

PrefectureRegion

the weight as the word wj is assigned to k-th mixture com-

93.6

89.7

74.6

ponent, N (y|µkj , Σkj ) is multivariate Gaussian distribution

Table 1: Human annotation summary.

with mean µkj and covariance matrix Σkj . After estimating
GMMs for each n-gram, the weighted sum of word-specific
GMMs is combined asthe location as a density estimation problem. Although theirmotivations are similar to ours, this research independentlyp(y|x) =J∑πj′ p(y|wj ),considers the word and n-gram for estimating each Gaussian

j=1

Mixture Model (GMM) and combines these GMMs using

where x = {w1 , · · · , wJ } represents n-grams in a tweet, πj′several optimization methods. We introduce a more reasonable method that automatically estimates the parameters for
GMM, weights, means and covariances depending on the feature vectors, which are often used for the document classification.

is a weight of GMM on the word wj .
In the above formula, it is important how to define each
GMM’s weight parameters πj′ . Even if we independently consider the weight πj′ for each word, the output model highly
depends on the locally distributed word in each tweet.

2. Materials

Whereas the WGMM combines the independently estimated group of models to represent tweet distribution, our2. 1 Tweet dataset

approach, Gaussian Mixture Regression, is able to represent

We used a tweet dataset consisting of 554,320 geotagged tweets in Japan (July 15, 2012it through one model.

July 21, 2012).

From the data, we have removed the application generated tweet (Foursquare, etc.). Consequently, we obtained
154,748 tweets consisting of only oﬃcial clients: Twitter for
iPhone/Android. We, then, normalized tweets by replacing
a username to @mention and web link to -URL-. We split
tweets into three sets: training (144,748 tweets), validation(5,000 tweets), and test (5,000 tweets).
2. 2 Human Annotation
To investigate the performance of human estimation, twohumans annotated the location onto the test data (5,000tweets). The human labels have a structure of three levels: “detailed” (Japanese prefecture-level; 47-way), “rough”(region-level; 8-way)（注 1） and “unknown”. To reduce the bias
based on the humans’ familiarity with areas, we permitted
them to search for the location words. The annotated results
are shown in Table 1.

3. 2 Gaussian Mixture Regression
This study applies Gaussian Mixture Regression (GMR)[10] to represent tweets distribution. This enables a uniform
way of handling tweet features for estimating the conditional
distribution p(y|x) conditioned on the tweet feature x.
Our approach does not need to prepare a specific evaluation index for estimating weight πj′ . We can derive theparameters, weights of conditional distribution p(y|x), fromjointly estimated GMM p(x, y).
To obtain the GMR results, we estimate the joint probability distribution p(x, y) of p-dimensional tweet features x
and two-dimensional geographic coordinate y. In this study,
we use the Dirichlet processes to estimate GMM parameters.
The conditional distribution p(y|x) can then be analyticallyderived as follows:

3. Methodp(y|x) = ∫3. 1 Word-specific Gaussian Mixture Model (GMM)∑K

sity estimation, each tweet was converted to n-gram featuresconsisting of the number of n-gram occurrences in a corpus=and geographic coordinates (longitude and latitude) of nword-specific GMM (WGMM) as（注 1）：http://www.japan-guide.com/list/e1001.html

p(x, y)p(x, y)=p(x)p(x, y)dy

πk N (x, y|µk , Σk )= ∑Kk=1πk=1 k N (x|µk,x , Σk,xx )In the existing approach [8] used GMM for geographic den-grams. The GMM applied for each n-gram wj is defined asyK
∑k=1where()πk′ (x)N y|µk,y|x , Σk,y|xPrefectureMethodRegion

Overall

Mean (km) Median (km) Mean (km) Median (km) Mean (km) Median (km)Gaussian Mixture Regression

251

154

242

134

278

214

Elastic Net

272

181

268

181

272

191

Mean location

277

185

273

185

273

187

Table 2: Error Distances between model estimation and true geographic coordinates: These test datasets consist of hierarchical structures; Prefecture ⊂ Region ⊂ Overall. The lower value is the better estimation.

Kanagawa

Tokyo

1008060

Kanagawa
Osaka
Chiba

200

Osaka

sified by humans. We implemented χ2 -test between human

Saitamacorrectly classified data and overall geo-tagged tweet labels.

40# of expected counts# of expected counts

12040

also investigate which prefectures are easily or diﬃcultly clas-

50

14002030

The P -value of χ2 -test is smaller than 0.001 that indicates
Kyoto

Saitama
Kyoto
Miyagi40

For the overall geo-tagged data set, we employ training data.

Chiba
2010

the human inferable tweets diﬀerently distributed with theMiyagi

overall geo-tagged tweet.

6080100120# of human correctly labeled tweets

0140010203040# of human correctly labeled tweets

50

As for exploring which prefecture influenced human deci-Figure 2: The count of each observed prefecture (x-axis) andsions, we visualized the observed count in human correctly

expected ones (y-axis) in the human annotated data set. The

classified data and overall expected counts of geo-taggedright figure is an enlargement of the left one. The prefectures

tweets with standard deviation shown in Figure 2. Whenin blue shaded regions reflect that the amount of human in-

the observed count are larger than the expected value (blueferable tweets is smaller than the overall geo-tagged tweets

shaded region), the number of human inferable tweets are

distribution and vice versa.

larger than the overall geo-tagged tweets distribution andvice versa. The prefectures that belong to the blue shadedµΣ

Σk,xyk,x],], Σk = [ Σk,xxµk = [ µk,yk,yx Σk,yyN (x, y|µk , Σk )N (y|µk,y|x , Σk,y|x ) =,N (x|µk,x , Σk,xx )region tend to be the sightseeing spots. More users state indetail about the visiting places compared with the overall
geo-tagged tweets. Conversely, the prefectures that belong
to unshaded regions tend to be commuter towns. This meansµk,y|x = µk,y + Σk,yx Σ−1k,xx (x − µk,x ),that many users state unrelated content with where that per-Σk,y|x = Σk,yy − Σk,yx Σ−1k,xx Σk,xy ,son is.

πk N (x|µk,x , Σk,xx )πk′ (x) = ∑K
k=1 πk N (x|µk,x , Σk,xx )The key diﬀerence between WGMM and GMR is that theweights of mixture parameters πk′ (x) are flexibly defined depending on the feature vector x.

4. 2 Evaluation
To compare our model with human inference, we evaluate
prediction performance through several human inferable data
set trained by the overall training data set. Note that ourdensity-based model estimates only probability distribution,
not the exact locations. Thus, we employ the mode value ofestimated density as the estimated locations. To calculate
the mode value of GMM, we need numerical optimization.
As an approximation method, we employ the most probable

GMR is useful to represent the tweet as p-dimensional vec-

mean in each mixture component.

tors. As described, we convert the tweet to vector represen-

We measure the mean and median values of error distances

tation learned by fasttext [6]. We set the dimension of thebetween the estimated location and the true tweet geoloca-

vector representation to 100, and compose a vector repre-

tion in the test data. As a baseline method, we use regu-sentation of a tweet by averaging all the word vectors in thelarized linear regression method, Elastic-Net, using the sametweet.

features and Mean locations. We optimize the baseline modelusing the validation set. Our results are presented in Table 2.

4. Results

Although our model performs worse than the baseline model

4. 1 Human Annotation Biases

for the overall test dataset, our model enhances prediction

We first investigate whether human classified tweets andperformance through human inference improvement.

overall geo-tagged ones are distributed in the same way. WeTweetGold

HumanDistanceTrue label

LabelEstimation(km)

probability (%)

36 Kinki God of Fire pilgrimages.

OsakaOsaka

3.2

42.6

Fuji can be seen before summer sunset!

ChibaKanto

250.7

1.06

Finish the work! Let’s carouse all night at Ueno

Saitama

Tokyo

23.7

14.7

Table 3: Comparing the characteristic estimated density examples between human annotation and model inference(a) Gold: Osaka, Human: Osaka(b) Gold: Chiba, Human: Kanto(c) Gold: Saitama, Human: Tokyo

Figure 3: Visualizing the characteristic estimated density examples. The color of the highest probability regions is red.

4. 3 Density-based Representation

coordinates belong to the condensed one in the joint distri-We show some typical density representation of the tweet

bution p(x, y). Therefore, our model properly expresses thein Table 3 and Figure 3. As the first example in Figure 3ashows human label as coincident with the true tweet geolocation in the finest level, prefecture level. In such cases, ourconsistent estimation with human inferred geolocation.

5. Conclusion

model estimates not only that the mode value of estimatedIn this study, we demonstrated that Gaussian Mixture Re-

density is near from the true tweet geolocation, but also thatgression providing a new perspective for estimating tweets’

its variance is small.

geolocations. We discovered that the density-based approach

In contrast to the first example, as for the second exam-

exceeded the conventional approach in the several aspects;

ple in Figure 3b, humans are puzzled when specifying the(1) the proximity of estimated area even for the miss clas-

location and labeled the wider level label, region level. Oursified tweet, and (2) the improvement of the prediction per-model spreads to cover the human designated region.

formance along the improvement of the human inference.

The last example reveals the superiority of density-based
estimation. The tweet’s user stated about “Ueno”, which isa part of Tokyo. Thus, the human estimated the label asTokyo for this tweet. However, the tweet’s user was at theReferencesneighboring of Tokyo; the true tweet geolocation is actually[1] Ajao, O., Hong, J., and Liu, W. A survey of location inference techniques on twitter. Journal of Information Science41, 6 (Dec. 2015), 855–864.
[2] Broniatowski, D. A., Paul, M. J., and Dredze, M. National and local influenza surveillance through twitter: Ananalysis of the 2012-2013 influenza epidemic. PLoS ONE 8,12 (12 2013).
[3] Cheng, Z., Caverlee, J., and Lee, K. You are where
you tweet: A content-based approach to geo-locating twitter users. In Proc. of CIKM 2010.
[4] Han, B., Cook, P., and Baldwin, T. Text-based twitteruser geolocation prediction. Journal of Artificial Intelligence Research 49 (2014), 451–500.
[5] Iso, H., Wakamiya, S., and Aramaki, E. Forecasting wordmodel: Twitter-based influenza surveillance and prediction.
In Proc. of COLING 2016 (2016), pp. 76–86.
[6] Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T.
Bag of tricks for eﬃcient text classification. arXiv preprint
arXiv:1607.01759 (2016).
[7] Paul, M. J., Dredze, M., and Broniatowski, D. Twitter
improves influenza forecasting. PLOS Currents Outbreaks

within Saitama prefecture. Although a conventional classification approach hardly captures the proximity between thetrue tweet geolocation and the estimated area, our densitybased method easily grasps it.
4. 4 Discussion
We have shown that the density-based approach is consistent with human estimation. Whereas conventional methods
aren’t interfered the performance which part of test data isused to predict, our density-based approach dramatically improves the prediction performance. In contrast to the conventional approach, each mixture component contributes adiﬀerent role for modeling the data.

For instance, noisy

tweets belong to the broadly scattered mixture component,and similar tweets in terms of both content and geographic

(2014).
[8] Priedhorsky, R., Culotta, A., and Valle, S. Y. D. Inferring the origin locations of tweets with quantitative confidence. In Proc. of CSCW 2014.
[9] Sadilek, A., Kautz, H. A., and Silenzio, V. Predicting
disease transmission from geo-tagged micro-blog data. In
AAAI (2012).
[10] Sung, H. G. Gaussian mixture regression and classification. PhD thesis, Rice University, 2004.