Introduction
Recent years have seen a huge boom in the number of different social media platforms available to users. People are increasingly using these platforms to voice their opinions or let others know about their whereabouts and activities. Each of these platforms has its own characteristics and is used for different purposes. The availability of a huge amount of data from many social media platforms has inspired researchers to study the relation between the data generated through the use of these platforms and real-world attributes.
Many recent studies in this field are particularly inspired by the availability of text-based social media platforms such as blogs and Twitter. Text from Twitter microblogs, in particular, has been widely used as data source to make predictions in many domains. For example, box-office revenues are predicted using text from Twitter BIBREF0 . Twitter data has also been used to find correlations between the mood stated in tweets and the value of Dow Jones Industrial Average (DJIA) BIBREF1 .
Predicting demographics of individual users using their language on social media platforms, especially Twitter, has been the focus of many research works: text from blogs and on-line forum posts are utilised to predict user's age through the analysis of linguistic features. Results show that the age of users can be predicted where the predicted and observed values reach a Pearson correlation coefficient of almost $0.7$ . Sociolinguistic associations using geo-tagged Twitter data have been discovered BIBREF2 and the results indicate that the demographic information of users such as first language, race, and ethnicity can be predicted by using text from Twitter with a correlation up to $0.3$ . Other research shows that users' income can also be predicted using tweets with a good prediction accuracy BIBREF3 . Text from Twitter microblogs has also been used to discover the relation between the language of users and the deprivation index of neighbourhoods. The collective sentiment extracted from the tweets of users has been shown BIBREF4 to have significant correlation ( $0.35$ ) with the deprivation index of the communities the users belong to.
Data generated on QA platforms have not been used in the past for predicting real-world attributes. Most research work that utilise QA data aim to increase the performance of such platforms in analysing question quality BIBREF5 , predicting the best answers BIBREF6 , BIBREF7 or the best responder BIBREF8 .
In this paper, we use the text from the discussions on the QA platform of Yahoo! Answers about neighbourhoods of London to show that the QA text can be used to predict the demographic attributes of the population of those neighbourhoods. We compare the performance of Yahoo! Answers data to the performance of data from Twitter, a platform that has been widely used for predicting many real-world attributes. Unlike many current works that focus on predicting one or few selected attributes (e.g. deprivation, race or income) using social media data, we study a wide range of 62 demographic attributes. Furthermore, we test whether terms extracted from both Yahoo! Answers and Twitter are semantically related to these attributes and provide examples of sociocultural profiles of neighbourhoods through the interpretation of the coefficients of the predictive models.
The contributions of this paper can be summarised as follows:
Spatial Unit of Analysis
The spatial unit of analysis chosen for this work is the neighbourhood. This is identified with a unique name (e.g., Camden) and people normally use this name in QA discussions to refer to specific neighbourhoods. A list of neighbourhoods for London is extracted from the GeoNames gazetteer, a dataset containing names of geographic places including place names. For each neighbourhood, GeoNames provides its name and a set of geographic coordinates (i.e., latitude and longitude) which roughly represents its centre. Note that geographical boundaries are not provided. GeoNames contains 589 neighbourhoods that fall within the boundaries of the Greater London metropolitan area. In the remainder of the paper, we use the terms “neighbourhood” or “area” to refer to our spatial unit of analysis.
Pre-processing, Filtering, and Spatial Aggregation
We collect questions and answers (QAs) from Yahoo! Answers using its public API. For each neighbourhood, the query consists of the name of the neighbourhood together with the keywords London and area. This is to prevent obtaining irrelevant QAs for ambiguous entity names such as Victoria. For each neighbourhood, we then take all the QAs that are returned by the API. Each QA consists of a title and a content which is an elaboration on the title. This is followed by a number of answers. In total, we collect $12,947$ QAs across all London neighbourhoods. These QAs span over the last 5 years. It is common for users to discuss characteristics of several neighbourhoods in the same QA thread. This means that the same QA can be assigned to more than one neighbourhood. Figure 1 shows the histogram of the number of QAs for each neighbourhood. As the figure shows, the majority of areas have less than 100 QAs with some areas having less than 10. Only few areas have over 100 QAs.
For each neighbourhood, we create one single document by combining all the QA discussions that have been retrieved using the name of such neighbourhood. This document may or may not contain names of other neighbourhoods. We split each document into sentences and remove those neighbourhoods containing less than 40 sentences. We then remove URLs from each document. The document is then converted to tokens and stop words are removed. All the tokens in all the documents are then stemmed. The goal of stemming is to reduce the different grammatical forms of a word to a common base form. Stemming is a special case of text normalisation. For example, a stemmer will transform the word “presumably” to “presum” and “provision” to “provis”. To keep the most frequent words, we remove any token that has appeared less than 5 times in less than 5 unique QAs. This leaves us with 8k distinct tokens.
To collect data from Twitter, we use the geographical bounding box of London, defined by the northwest and southeast points of the Greater London region. We then use this bounding box to obtain the tweets that are geotagged and are created within this box through the official Twitter API. We stream Twitter data for 6 months between December 2015 and July 2016. At the end, we have around $2,000,000$ tweets in our dataset.
To assign tweets to different neighbourhoods, for each tweet, we calculate the distance between the location that it was blogged from and the centre points of all the neighbourhoods in our dataset. Note that the centre point for each neighbourhood is provided in the gazetteer. We then assign the tweet to the closest neighbourhood that is not further than 1 km from the tweet's geolocation. At the end of this process, we have a collection of tweets per each neighbourhood and we combine them to create a single document. Figure 2 shows the number of tweets per each neighbourhood. As we can see, the majority of neighbourhoods have less than 1000 tweets.
We remove all the target words (words starting with @) from the documents. The pre-processing is then similar to the QA documents. At the end of this process, we obtain 17k distinct frequent tokens for the Twitter corpus.
As we previously explained, each attribute in census data is assigned to spatial units called LSOAs. However, these units do not geographically match our units of analysis which are the neighbourhoods defined trough the gazetteer. A map showing the spatial mismatch is presented in Figure 3 . To aggregate the data contained in the LSOAs at the neighbourhood level, we use the following approach.
Often, when people talk about a neighbourhood, they refer to the area around its centre point. Therefore, the information provided for neighbourhoods in QA discussions should be very related to this geographic point. To keep this level of local information, for each demographic attribute, we assign only the values of the nearby LSOAs to the respective neighbourhood. To do this, we calculate the distance between each neighbourhood and all the LSOAs in London. The distance is calculated between the coordinates of a neighbourhood and the coordinates of each LSOA's centroid. For each neighbourhood, we then select the 10 closest LSOAs that are not further than one kilometre away. The value of each demographic attribute for each neighbourhood is then computed by averaging the values associated with the LSOAs assigned to it. We apply this procedure to all the demographic attributes.
Document Representation
A very popular method for representing a document using its words is the tf-idf approach BIBREF9 . Tf-idf is short for term frequency-inverse document frequency where tf indicates the frequency of a term in the document and idf is a function of the number of documents that a terms has appeared in. In a tf-idf representation, the order of the words in the document is not preserved. For each term in a document, the tf-idf value is calculated as below:
$$\small \text{tf-idf} (d,t) = \frac{\text{tf}\ (d,t)}{\log (\frac{\text{Total number of documents}}{\text{Number of documents containing the term}\ t })} $$   (Eq. 21)
To discount the bias for areas that have a high number of QAs or tweets, we normalise tf values by the length of each document as below. The length of a document is defined by the number of its tokens (non-distinctive words).
$$\small \text{Normalised\ tf} (d,t) = \frac{\text{Frequency of Term t in Document d}}{\text{Number of Tokens in Document d}}$$   (Eq. 22)
Correlation
To investigate the extent to which the text obtained from the two platforms of Yahoo! Answers and Twitter reflect the true attributes of neighbourhoods, we first study whether there are significant, strong and meaningful correlations between the terms present in each corpus and the many neighbourhood attributes through the Pearson correlation coefficient $\rho $ . For each term in each corpus, we calculate the correlation between the term and all the selected demographic attributes. To do so, for each term, we define a vector with the dimension of the number of neighbourhoods. The value of each cell in this vector represents the normalised tf-idf value of the term for the corresponding neighbourhood. For each demographic attribute, we also define a vector with the dimension of the number of neighbourhoods. Each cell represents the value for the demographic attribute of the corresponding neighbourhood. We then calculate the Pearson correlation coefficient ( $\rho $ ) between these two vectors to measure the strength of the association between each term and each attribute.
Since we perform many correlation tests simultaneously, we need to correct the significance values (p-values) for multiple testing. We do so by implementing the Bonferroni correction, a multiple-comparison p-value correction, which is used when several dependent or independent statistical tests are being performed simultaneously. The Bonferroni adjustment ensures an upper bound for the probability of having an erroneous significant result among all the tests. All the p-values showed in this paper are adjusted through the use of the Bonferroni correction.
The number of significantly correlated terms from both Yahoo! Answers and the Twitter with the selected demographic attributes are shown in Table 2 . Note that the number of unique (frequent) words in Twitter (17k) is almost twice as in Yahoo! Answers (8k). The first column shows a demographic attribute and the second column indicates the source, i.e. Yahoo! Answers (Y!A for short) or Twitter. The third column (“All”) shows the total number of terms that have a significant correlation with each attribute (p-value $<0.01$ ). The following columns show the number of terms that have a significant correlation with the attribute with a $\rho $ in the given ranges. The last column shows the number of terms that are significantly correlated with the attribute with a negative $\rho $ . The data source that has the highest number of correlated terms with each attribute is highlighted in bold.
As the table shows, terms extracted from Yahoo! Answers tend to be more related, in terms of the number of correlated terms, to attributes related to religion or ethnicity compared to terms from Twitter. However, for two particular attributes (i.e., Price and Buddhist), the number of correlated terms from Twitter is higher than the ones from Yahoo! Answers . These results collectively suggest that there is a wealth of terms, both in Yahoo! Answers and Twitter, which can be used to predict the population demographics.
In this section, we observe whether the correlations between terms and attributes are semantically meaningful. Due to the limited space, we select three attributes and their relative top correlated terms extracted from Yahoo! Answers (Table 3 ) and Twitter (Table 4 ). We choose the attributes Price and IMD as they show the highest number of correlated terms for both sources. For each source, we then choose one more attribute that has the highest number of strongly correlated terms ( $\rho >0.4$ ), i.e. Jewish% for Yahoo! Anwsers and Buddhist% for Twitter.
We first examine Table 3 and provide examples of semantic similarity between Yahoo! Answers terms and the selected attributes. Words highlighted in bold are, in our view, the ones most associated with their respective attribute. For the attribute Deprivation, the majority of the terms seem to be linked to issues of deprived areas. “Poverty”, “drug”, “victim”, all refer to social issues. “Rundown” and “slum” may be associated with the degradation of the surrounding environment. “Cockney” is a dialect traditionally spoken by working class, and thus less advantaged, Londoners. For the attribute (High) Price, most terms seem to be related to aspects of places which may offer more expensive housing. Terms such as “fortune”, “diplomat”, and “aristocratic” are often associated with wealth. Others seem to reflect a posh lifestyle and status symbol: “townhouse”, “exclusive”, “celeb”, “fashionable”, “desirable”. For the attribute Jewish%, most of the terms seem to reflect aspects of this religion or be linguistically associated with it (i.e., “Jew” and “Jewish”). “Matzo” and “Kosher” are associated with the traditional Jewish cuisine; the former is a type of flat-bread, the latter is a way of preparing food. The “ark” is a specific part of the synagogue which contains sacred texts.
We now examine Table 4 . For the attribute Deprivation, nine words out of ten seem to be linked to more deprived areas. “East”, “eastlondon”, and “eastend”, for example, provide geographical information on where deprivation is more concentrated in London (i.e., East End). Other terms seem to be related to the presence of younger generation of creatives and artists in more deprived neighbourhoods. “Yeah”, “shit”, “ass”, may all be jargons commonly used by this section of population. “Studio”, “craftbeer”, “music” may instead refer to their main activities and occupations. For what concerns (high) “Price”, all the terms seem to relate to aspects of expensive areas, e.g. “luxury”, “classy”, and “stylish”. “Tea”, “teatime”, “delight”, “truffle” seem to relate to social activities of the upper class. For the attribute “Buddhist%”, five terms out of ten are, in our view, associated with neighbourhoods where the majority of people is Buddhist or practise Buddhism. These terms seems to relate to aspects of this religion, e.g. “think”, “learn”, “mind”, etc.
Interestingly, terms extracted from Yahoo! Answers and Twitter seem to offer two different kinds of knowledge. On one side, terms extracted from Yahoo! Answers are more encyclopedic as they tend to offer definitions or renowned aspects for each attribute. “Jewish%” is, for example, related to aspects of the Jewish culture such as “matzo”, “harmony”, and “kosher”. “Deprivation” is associated with social issues such as “poverty” and “drug”, but also with a degraded urban environment (e.g., “rundown”, “slum”). On the other, Twitter words provide a kind of knowledge more related to current sociocultural aspects. This is the case, for example, of the jargon associated with “Deprivation” (e.g., “yeah”, “shit”), or of the culinary habits related to “High Prices” (e.g., “tea”, “truffle”).
Prediction
We investigate how well the demographic attributes can be predicted by using using Yahoo! Ansewrs and Twitter data. We define the task of predicting a continuous-valued demographic attribute for unseen neighbourhoods as a regression task given their normalised tf-idf document representation. A separate regression task is defined for each demographic attribute. We choose linear regression for the prediction tasks as it has been widely used for predictions from text in the literature BIBREF10 , BIBREF11 .
Due to the high number of features (size of vocabulary) and a small number of training points, over-fitting can occur. To avoid this issue, we use elastic net regularisation, a technique that combines the regularisation of the ridge and lasso regressions. The parameters $\theta $ are estimated by minimising the following loss function. Here, $y_i$ is the value of an attribute for the $i$ -th neighbourhood, vector $\mathbf {x}_i$ is its document representation and $N$ is the number of neighbourhoods in the training set.
$$\mathfrak {L} = \frac{1}{N} \sum _{i=1}^N (y_i- \mathbf {x}_{i}^T \theta )^2 + \lambda _1 ||\mathbf {\theta }|| + \lambda _2 ||\mathbf {\theta }||^2$$   (Eq. 25)
To measure the performance of a regression model, residual-based methods such as mean squared error are commonly used. Ranking metrics such as Pearson correlation coefficient have also been used in the literature BIBREF12 , BIBREF2 . Using a ranking measure has some advantages compared to a residual-based measure. First, ranking evaluation is more robust against extreme outliers compared to an additive residual-based evaluation measure. Second, ranking metrics are more interpretable than measures such as mean squared error BIBREF13 . We thus use this method for evaluating the performance of the regression models in this work.
As further performance check, we apply a 10 folds cross-validation to each regression task. In each fold, we use $75\%$ of the data for training and the remaining $25\%$ for validation. At the end, we report the average performance over all folds together with the standard deviation. For each demographic attribute, i.e. target value, training and validation sets are sampled using Stratified Sampling. This is a sampling method from a population, when sub-populations within this population vary. For instance, in London, there are areas with very high or very low deprivation. In these cases, it is advantageous to sample each sub-population independently and proportionally to its size.
The results of the regression tasks performed over the selected set of demographic attributes, in terms of Pearson correlation coefficient ( $\rho $ ), are presented in Table 4 . Results are averaged over 10 folds and standard deviations are displayed in parenthesis.
We can see that on average, performances of Yahoo! Answers and Twitter are very similarly with Yahoo! Answers having a slightly higher performance ( $4\%$ ). Twitter data can predict the majority of the religion-related attributes with a higher correlation coefficient with the exception of population of Jewish%. On the other hand, Yahoo! Answers is superior to Twitter when predicting ethnicity related attributes such as population of White% and Black%. We have seen in Table 2 that Twitter has very few correlated terms with the attributes White (0) and Black (2).
We also observe that IMD and Price can be predicted with a high correlation coefficient using both Yahoo! Answers and Twitter. This can be due to the fact that there are many words in our dataset that can be related to the deprivation of a neighbourhood or to how expensive a neighbourhood is. This is also evident in Table 2 where the number of correlated terms from both Yahoo! Answers and Twitter with these attributes are very high. On the other hand, terms that describe a religion or an ethnicity are more specific and lower in frequency. Therefore attributes that are related to religion or ethnicity are predicted with a lower accuracy.
Table 5 further shows two terms that have the highest coefficients in the regressions models (across the majority of folds) for each attribute and source in the column Terms. These terms are among the strong predictors of their respective attribute. Many of these terms appear to be related to the given demographic attribute (for both Twitter and Yahoo! Answers ) and are also often amongst the top correlated terms presented in Tables 3 and 4 . We follow with some examples. According to the regression coefficients for the attribute Muslim%, neighbourhoods inhabited by a Muslim majority may be located in Mile End, an East London district (i.e., Twitter terms “mileend” and “eastlondon”), see the presence of Asian population and barber shops (i.e., Yahoo! Answers terms “asian” and “barber”). According to the terms for Black%, neighbourhoods with a black majority tend to be located in the southern part of London (i.e., Twitter term “southlondon”) and experience social issues such as presence of criminal groups and drug use (i.e., Yahoo! Answers terms “gang” and “drug”). According to the terms for IMD, more deprived areas seem to be located in the East End of London (i.e., Twitter term “eastlondon”) where the Cockney dialect is dominant (i.e., Yahoo! Answers term “cockney”). Yahoo! Answers and Twitter seem to complement one another in terms of information they provide through the terms associated with each attribute which in most cases are different. One noticeable difference is that Twitter tends to offer geographical information (e.g., “mileend”, “southlondon”, “essex”). On the other hand, terms from Yahoo! Answers sometimes match the name of the attribute (i.e. “asian” and “Jewish”).
In the Appendix, in Tables 6 and 7 , we show the prediction results for a wide range of 62 demographic attributes using Yahoo! Answers and Twitter. For each attribute, we display two terms with the highest coefficient common between the majority of the folds. Attributes are divided into categories such as Religion, Employment, Education, etc. Overall, the results show that Yahoo! Answers performs slightly better than Twitter with an average $1\%$ increase over all the attributes. Wilxocon signed rank test shows that their results are significantly different from each other (p-value $ < 0.01$ ). Outcomes in these tables show that on average, a wide range of demographic attributes of the population of neighbourhoods can be predicted using both Yahoo! Answers and Twitter with high performances of $0.54$ and $0.53$ respectively. While Yahoo! Answers outperforms Twitter in predicting attributes related to Ethnicity and Employment, Twitter performs better when predicting attributes relating to the Age Group, and Car Ownership.
Related Work
The availability of a huge amount of data from many social media platforms has inspired researchers to study the relation between the data on these platforms and many real-world attributes. Twitter data, in particular, has been widely used as a social media source to make predictions in many domains. For example, box-office revenues are predicted using text from Twitter microblogs BIBREF0 . Prediction results have been predicted by performing content analysis on tweets BIBREF14 . It is shown that correlations exist between mood states of the collective tweets to the value of Dow Jones Industrial Average (DJIA) BIBREF1 .
Predicting demographics of individual users using their language on social media platforms, especially Twitter, has been the focus of many research. Text from blogs, telephone conversations, and forum posts are utilised for predicting author's age BIBREF15 with a Pearson's correlation of $0.7$ . Geo-tagged Twitter data have been used to predict the demographic information of authors such as first language, race, and ethnicity with correlations up to $~0.3$ BIBREF2 .
One aspect of urban area life that has been the focus of many research work in urban data mining is finding correlations between different sources of data and the deprivation index (IMD), of neighbourhoods across a city or a country BIBREF16 , BIBREF4 . Cellular data BIBREF17 and the elements present in an urban area BIBREF18 are among non-textual data sources that are shown to have correlations with a deprivation index. Also, flow of public transport data has been used to find correlations (with a correlation coefficient of $r = 0.21$ ) with IMD of urban areas available in UK census BIBREF16 . Research shows that correlations of $r = 0.35$ exists between the sentiment expressed in tweets of users in a community and the deprivation index of the community BIBREF4 .
Social media data has been used in many domains to find links to the real-world attributes. Data generated on QA platforms, however, has not been used in the past for predicting such attributes. In this paper, we use discussions on Yahoo! Answers QA platform to make predictions of demographic attribute of city neighbourhoods. Previous work in this domain has mainly focused on predicting the deprivation index of areas BIBREF4 . In this work, we look at a wide range of attributes and report prediction results on 62 demographic attributes. Additionally, work in urban prediction uses geolocation-based platforms such as Twitter. QA data that has been utilised in this paper does not include geolocation information. Utilising such data presents its own challenges.
Discussion
In this paper, we investigate predicting values for real-world entities such as demographic attributes of neighbourhoods using discussions from QA platforms. We show that these attributes can be predicted using text features based on Yahoo! Answers discussions about neighbourhoods with a slightly higher correlation coefficient than predictions made using Twitter data.
Limitations
Here, we present some of the limitations of our work.
To unify the units of analysis, we take a heuristic approach. We do not cross-validate our results with other approaches. This is because of the lack of work in using non-geotagged text for predicting attributes of neighbourhoods in the current literature.
Our experiments in this paper is limited to the city of London. London is a cosmopolitan city and a popular destination for travellers and settlers. Therefore, many discussions can be found on Yahoo! Answers regarding its neighbourhoods. The coverage of discussions on QA platforms may not be sufficient for all cities of interest.

Question: What do the correlation demonstrate? 
Answer: the extent to which the text obtained from the two platforms of Yahoo! Answers and Twitter reflect the true attributes of neighbourhoods

Question: On Twitter, do the demographic attributes and answers show more correlations than on Yahoo! Answers?
Answer: No

Question: How many demographic attributes they try to predict?
Answer: 62