# Predicting Location of Tweets Using Machine Learning Approaches
## 3 Data Collection and Preparation
### Data Cleansing  
Twitter texts deviate from regular documents in many forms. First, they are short tweets with a maximum allowable length of 280 characters, which was 140 before 2017. Furthermore, Twitter messages can contain colloquial language, text in various dialects, shortened text (e.g., '\(\xi\)' for '\(\omega\)' On, or '\(\xi\)' for '\(\omega\)' Vocative particle \"O\") or even misspelled words. Furthermore, it allows users to socialize with other users in many ways. Retweeting others' tweets is one way of socializing. Mentioning another user is also a way where users use the symbol \"@\" to mention someone else. Besides, users can add hashtags, consisting of the symbol \"#\" supported by a keyword, to specify the point of their tweet (e.g., '\(\omega\)')\(\models\)'\(\models\)'Riyadh_Season). Ultimately, Twitter documents can hold hyperlinks to external websites. However, these documents, when fetched from Twitter, are in raw noisy and fuzzy format. As we needed to extract tweet text, the documents might include a mixture of different data types, namely, words, numbers, hashtags, URLs, mentions, emojis images, and videos.  
The cleansing process of Twitter documents involved the (i) tokenization of words; (ii) extraction of hashtags and user mentions; and (iii) elimination of stop-words, numbers, URLs, images, videos, symbolic characters, and emojis. This might be universal for all languages. However, Arabic language has extra characteristics, which might have introduced another level of ambiguity. Table 4 illustrates Arabic diacritics, which are usually added to each letter in the word. As they are optional, we eliminated them. However,  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**City** & **Latitude** & **Longitude** & **Class Label** \\ \hline Al Khobar & 26.3039999 & 50.1960237 & 7 \\ \hline Al Udayd & 22.5 & 51 & 8 \\ \hline Al Kharj & 24.148333 & 47.305 & 9 \\ \hline Ar Ar Ar & 130.9815531 & 41.0164788 & 10 \\ \hline Ar Rass & 25.8685205 & 43.5038978 & 11 \\ \hline At Taif & 21.270278 & 40.415833 & 12 \\ \hline Az Zahran & 26.2966528 & 50.1202146 & 13 \\ \hline Baljurshi & 19.859444 & 41.557222 & 14 \\ \hline Baqaa & 27.5 & 42.5 & 15 \\ \hline Bisha & 20 & 42.6 & 16 \\ \hline Boriydah & 26.27657425 & 43.32498065 & 17 \\ \hline Buraydah 1 & 26.331667 & 43.971667 & 18 \\ \hline Dammam & 26.4367824 & 50.1039991 & 19 \\ \hline Jeddah & 21.59734945 & 39.13362779 & 20 \\ \hline Hafar Al Batin 2 & 27.901429 & 45.5283442 & 21 \\ \hline Medina & 24.471153 & 39.6111216 & 22 \\ \hline Ohd Rofida & 19.166667 & 43.166667 & 23 \\ \hline Rafha & 29.6324189 & 43.5178685 & 24 \\ \hline Riyadh & 24.6319692 & 46.7150648 & 25 \\ \hline Sabyaa & 17.333333 & 42.666667 & 26 \\ \hline Sakakah & 29.7851094 & 40.0354435 & 27 \\ \hline Sharoura & 18 & 45.666667 & 28 \\ \hline Tabuk & 27.5 & 37.333333 & 29 \\ \hline Yanbu & 24.0889015 & 38.0666798 & 30 \\ \hline \hline \end{tabular}  
* Boriydah; Class 17, and Buraydah; Class 18, can be slightly confusing. They refer to two distinct places but are not far from each other on the map. 2 This can be found as King Khalid Military City in Google Maps.  
\end{table}
Table 3: _Cont._some diacritics are not as ambiguous as the ones shown in Table 4. There are letters that Twitter users use interchangeably, which should not be the case in the Standard Arabic. For instance, the letter Alif can be written as Alif with Hamza '/' or as a plain Alif '/'. Similarly, the letter Taa can be written as Taa Marbootah '/' or as Haa '-a' because Arabic words are commonly written in different formats for the same terms as indicated in Table 5. In this case, these letters are not always interchangeable. For example, the word '/' means the name \"Ali\" while the word '/' means the preposition word \"On\" and is pronounced in Arabic \"Ala\". Yet, due to the similarity in writing them, it is common on Twitter to use them interchangeably. Finally, data cleansing identified duplicate documents that were needed to be filtered out as well.