# Deep Learning Models for Road Passability Detection during Flood Events Using Social Media Data
## 4 Proposed Solutions
### Algorithms Based on Image Only
#### 4.2.1 Ensemble Image Base Architecture  
For this solution, we considered both tasks as two, separated, two-class classification problems. Since performance is prioritized over computation and operational time, we created an ensemble of networks, using 9 state-of-the-art networks: InceptionV3, Xception, VGG16, VGG19, InceptionResNetV2, MobileNet, DenseNet121, DenseNet201, NaSNetLarge. Since the dataset was too small to train the networks from scratch, we pre-trained all the networks on ImageNet [35], freezed  
Figure 4: Architecture of the neural network to process metadata. The input matrix, composed by stacked word embeddings representing the tweeted text and hashtags is processed by a Bidirectional Long Short-Term Memory network (BiLSTM). Its output is concatenated with other metadata information representing whether: (i) the tweet has been favorited, (ii) the tweet was originally written in English, (iii) it was retweeted. Then, two Fully Connected (FC) layers are dedicated to deal with each task: the FC on the top will determine the Evidence of Roads (ER), while the other one will determine the Evidence of Roads Passability (ERP). The classification is obtained by thresholding their output. Finally, to guarantee consistent classifications, the output of the ERP classifier is combined with the output of the ER classifier by a logical AND operation. Figure 3: Correlation matrix between the ground truth features, “_ER_” (Evidence of Roads) and “_ERP_” (Evidence of Roads Passability) as two separate binary values, with the numerical features that were not discarded. the parameters from the first half of the network and fine-tuned the parameters from the second half. All nine models were separately trained for both classification problems--one network for the classification of road passability and one for the classification in positive or negative road passability. The networks trained on the detection of positive or negative road passability were trained using only the images with evidence of road passability according to the ground truth. Moreover, in order to prevent overfitting, the dataset was randomly divided into training (75%) and validation (25%) sets. The validation set was used to prevent the networks to overfit to the training set--after training the networks in the training set, we stored the models that performed better in the validation set. Finally, in order to prevent overfitting while exploiting the whole dataset, we performed cross-validation using 5 different train-validation folds. Each fold was generated using a random split of the development set into 75% train and 25% validation, this means that there is some overlapping between splits. Each network was trained in each fold and for each task separately, resulting in a total of 90 networks, 45 convolutional neural networks for each task (or 5 networks per network architecture and task). The network architecture is shown in Figure 5a. The output of each network is a number between 0 and 1 which represents the probability of the picture containing evidence of road passability and whether the road has a positive or a negative passability, respectively. In order to ensemble the results of all the networks, we decided to allocate the same weight to each sub-model and we applied two different aggregation methods. On the one hand, we applied a classical average aggregation prediction. On the other hand, we applied a combination of an average aggregation with a majority voting aggregation. These two aggregation methods are defined, respectively, by  
\[\text{pred}_{1}(p_{1},\ldots,p_{n})=(\overline{p}>0.5),\]  
and \(\text{pred}_{2}(p_{1},\ldots,p_{n})=\)  
\[\left\{ \begin{array}{ll}1&\text{if }\left(\overline{p}>0.45\text{ and } voting(p_{1},\ldots,p_{n})\geq\frac{n}{2}\right)\text{ or }\\ &\left(\overline{p}>0.5\text{ and } voting(p_{1},\ldots,p_{n})>\frac{n}{2}-2\right),\\ 0&\text{otherwise,}\end{array}