# **4.1 Group Selection on Social Media**

To examine refugee displacement and return on these platforms, we created parallel processes to identify, collect data from, and analyze accounts, channels, and groups posting public messages about the Syrian conflict on Twitter, Telegram, and Facebook. While this process aimed to limit selection effects to the extent possible, the different natures of the platforms ultimately mean that our samples comprise different populations, and the differences between search functionalities meant there was no single starting point. Underscoring the replicability of our process — and the process’s resemblance to other selection processes common in the use of social media data — we present source specific models in Appendix 9 to show the selection effects inherent in the study of social media users on any singular platform.

We limited our analysis to Arabic language, the predominant language of use among all actors in the conflict.<sup>5</sup> Our final dataset contains messages in Arabic from 657 public channels and groups on Telegram, 2,106 public Twitter accounts, and 2,124 public Facebook groups and pages.

# **4.2 Identifying Location Specific Messages**

The first step of our process was attributing message sets to certain locations. Twitter, Telegram and Facebook’s functionalities make it difficult to verify the location that users post from in all but a few cases. To understand location specific discussion we searched all of the collected messages for 2004 Syrian census locations using string matching on the names in Arabic. We augmented this dataset with locations from the GeoNames API<sup>6</sup> , which includes common misspellings or dialectic spellings of locations.<sup>7</sup> The `Location Mentions` dataset, comprised of the location mentioning messages associated with the REACH data, includes messages from 770 unique locations, 54 of which are mentioned more than 10,000 times, 229 of which have returnees, and 129 of which have IDPs. A heat map showing the distribution of mentions of locations overall, locations with returnees and locations with IDPs by source is in Appendix 8.1, as are maps with locations mentioned by returnee status and year of return. The majority of locations mentioned in the `Location Mentions` dataset are in northeastern Syria in Idlib, Aleppo and northern Ar-Raqqa, an outcome of REACH data emphasis, as can be seen in Section 4 Figure 1. For 218 of these locations return first occurred during the time of data collection, the `Return Date` dataset. Of these locations, 119 have both returnees and IDPs. In the overall REACH data it is also true that most locations with IDPs also have returnees (26% of locations have IDPs and no return). Table 1 shows the size of the datasets and the percentage of messages within that dataset that are from areas with returnees and IDPs. For example, in the main `Location Mentions` dataset there are 2,360,559 messages of which 1,264,871 are from Telegram. Of the total messages, 16% (370,345 messages) are from areas with only return, 5% are from areas with only IDPs, 62% are from areas with neither and 17% are from areas with both. Of

> 5To truncate the dataset to Arabic, we first obtained the language of a message using _langid_ (Lui and Baldwin (2012)), an off the shelf tool for detecting language from text. The tool is based on pre-trained machine learning models and can detect over 90 languages. Before beginning data analysis, we performed basic cleaning on the text data using Nielsen’s stemmer (Nielsen (2017)), but prevented the stemming of proper nouns like key locations and political figures. Next, we removed stopwords using a base list of Arabic stop words and some words in Syrian dialect (my brother, where, why, how, etc.), and the words ‘channel’, ‘subscribe’, ‘Telegram’ and ‘Twitter’. The base list of Arabic stop words is from Mohatahar Arabic Stopword, _Github_ , `https://github.com/mohataher/arabic-stop-words/blob/master/list.txt`

> 6 `http://www.geonames.org/`

> 7We removed any locations which had the same names as governorates to improve granularity. A few top locations which were disproportionately represented due to false positive matches, ‘mil’, ‘san’, and ‘ada’, were removed.

10