# Deep Learning Models for Road Passability Detection during Flood Events Using Social Media Data
## 4 Proposed Solutions
### Algorithm Based on Metadata Only  
As explained in Section 3, each tweet contains 29 different fields, but only 16 of them had non-empty values in at least 90% of the tweets. Therefore, the other 13 features were discarded since they do not contain enough information to give any statistical significant information. Moreover, we discarded the following features: (i) \"_Created at_\", which contained the date in which the tweet was posted. Since the tweets were collected during specific hurricane events (namely, Harvey, Irma and Maria), we considered this field to have a very limited time coverage with the risk of being biased and, therefore, not useful. Specifically, the development set contains tweets from 38 different days. (ii) \"_Extended entities_\", which contains structural information about the tweet, such as the icon and image sizes, their urls and ids and, therefore, it does not provide any relevant information; (iii) \"_Id_\" and \"_Id str_\" fields are automatically generated to guarantee uniqueness to the tweet thus, do not contain any meaningful information; (iv) \"_Truncated_\" contains a constant value, which is equal for each tweet in the development set; (v) \"_Source_\" and \"_User_\": contained features pertinent to Twitter and the user profile, such as \"id\", \"profile image url\", \"friends count\", which is information not relevant to our purposes. Additionally, we verified that the development set rarely contained multiple posts from the same user: this lack of information prevented the extraction of data for determining a possible positive (or negative) influence toward our goals. As for the \"_Lang_\" feature, since most of the tweets were in English and all the other languages were very minoritary, we transformed it into a binary value \"_originally_en_\" to state whether the language of the tweet was English. To ensure that all features would contribute equally to the loss function used to train our proposed approaches, we normalized the features \"_Favorite count_\" and \"_Retwect count_\" between 0 and 1, which we named \"_favorited_norm_\" and \"_retweeted_norm_\", respectively. Finally, we also discarded the features corresponding to \"_Favorited_\" and \"_Retwected_\" since they are subsumed by the former ones. To determine a correlation between the normalized fields: \"_favorited_norm_\", \"_is_quote_status_\", \"_originally_en_\", \"_possibly_sensitive_\" and \"_retweeted_norm_\" and the task at hand, we built a point-biserial correlation matrix between each feature and the \"ER\" and \"ERP\" ground truth using the Pearson correlation coefficient. As seen on the point-biserial correlation matrix from Figure 3, none of the features have a very strong correlation with the ground truth; however, we decided to keep the fields \"_favorited_norm_\", \"_originally_en_\" and \"_retweeted_norm_\" since they are the highest correlated features. We expected the text written by the user (\"_Text_\") and the hashtags of the tweet (\"_Entities_\") to be the most informative features, which we concatenated, obtaining a single sentence. To help the training, we translated all the texts into English, tokenized the words, filtered stopwords (i.e., emojis, urls, special characters, articles, conjunctions) and lemmatized the sentence. Finally the sentences were transformed into a matrix using a word embedding initialized with GloVe [26] weights, transforming each word into a vector of 200 dimensions. To be processed by a neural network, the matrices generated from _Text_ and _Entities_ have been standardized to have the same number of word vectors--sentences shorter than 30 words (the maximum length of a processed sentence in the dataset) have been filled with zero padding. As other state-of-the-art works [51], the 30 \(\times\) 200 matrices have been fed in a Bidirectional Long Short-Term Memory (BiLSTM) network. Then, the output was concatenated with the _extra fields_ and fed into two parallel fully-connected (FC) layers with a softmax classifier, one per task.