Population Bias in Geotagged Tweets

Geotagged tweets are an exciting and increasingly popular data source, but like all social media data, they potentially have biases in who are represented. Motivated by this, we investigate the question, ‘are users of geotagged tweets randomly distributed over the US population’? We link approximately 144 million geotagged tweets within the US, representing 2.6m unique users, to high-resolution Census population data and carry out a statistical test by which we answer this question strongly in the negative. We utilize spatial models and integrate further Census data to investigate the factors associated with this nonrandom distribution. We find that, controlling for other factors, population has no effect on the number of geotag users, and instead it is predicted by a number of factors including higher median income, being in an urban area, being further east or on a coast, having more young people, and having high Asian, Black or Hispanic/Latino populations. ‘Geotagged’ or ‘geocoded’ tweets, where users elect to automatically include their exact latitude/longitude geocoordinates in tweet metadata, provide data that are: • High-quality: geotagging is automated, so there are fewer chances of data error such as from user specification (Graham, Hale, and Gaffney 2014; Hecht et al. 2011); • Precise: geotags are down to a ten thousandth of a degree in latitude and longitude; • Richly contextual: geotags are connected to tweets with all their temporal, semantic, and social content; • Easily available, through the Streaming API; • Large: using the Streaming API, a researcher can build a collection of tens of millions of tweets. Unsurprisingly, this makes them an enormously attractive source for studying a wide range of human phenomena (Hong et al. 2012). Existing works have used geotagged tweets to study • mobility patterns (Yuan et al. 2013; Cho, Myers, and Leskovec 2011), • urban life (Doran, Gokhale, and Dagnino 2013; FriasMartinez et al. 2012), Copyright c © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. • transportation (Wang et al. 2014a), • natural disasters, crises, and disaster response (Morstatter et al. 2014; Lin and Margolin 2014; Shelton et al. 2014; Sylvester et al. 2014; Kumar, Hu, and Liu 2014), and • public health (Sylvester et al. 2014; Nagar et al. 2014; Ghosh and Guha 2013) as well as the interplay between geography and • language (Hong et al. 2012; Eisenstein et al. 2010; Kinsella, Murdock, and O’Hare 2011), • discourse (Leetaru et al. 2013), • information diffusion and flows (Kamath et al. 2013; van Liere 2010), • emotion (Mitchell et al. 2013), and • social ties (Stephens and Poorthuis 2014; Takhteyev, Gruzd, and Wellman 2012; Cho, Myers, and Leskovec 2011). Furthermore, maps of geotagged tweets tend to look remarkably similar to maps of population density (figs. 1 and 2; see also Leetaru et al., 2013), even if there are differences at a finer scale (figs. 3 and 4). This naturally leads to the question: are Twitter users who send geotagged tweets (henceforth, ‘geotag users’) randomly distributed over the population? This is a critical question because, if users who elect to geotag are systematically different from people in general, the results of studying geotagged tweets will not have external validity. We used the Twitter API to get a collection of 144,877,685 geotagged tweets from the contiguous US, from which we extract 2,612,876 unique twitter handles. We uniquely assign each handle to a block group, a geographic designation of the US Census Bureau that is the smallest geographic unit for which Census data is publicly available. We then link the counts of unique geotag users per block group to the 2010 Decennial Census population counts per block group, and create a statistical test for the null hypothesis that geotag users are randomly distributed over the US population. We find sufficient evidence to reject this null. Using other Census data, we then use a Simultaneous Autoregressive (SAR) model to test some candidate explanatory factors and investigate what is nonrandom about this distribution. This is, to our knowledge, the first paper to use Standards and Practices in Large-Scale Social Media Research: Papers from the 2015 ICWSM Workshop