## 5 Results
### Classification  
In this study, binary classification and multi-class classification have been done. Binary classification is used to classify messages into two classes of damage and non-damageso that their results can finally be used to estimate the earthquake damage. Multi-class classification is used to categorize messages to increase post-crisis situational awareness and to monitor the process of changing conditions over time and to determine the concentration of various topics in different locations.  
For classification, the pre-processed data were divided into training and test data. Then, based on the training data of each trained classification model and then using the test data, the accuracy of the prediction models were evaluated.  
* Binary classification  
For binary classification, the final dataset comprised 26,942 tweets that were manually labeled to produce the training dataset of 5038 (including 4031 non-damage and 1007 damage) tweets. This dataset was divided into two sections. 70% of them were used for training and 30% of them are used for testing. Precision, accuracy, recall, Kappa, and F-measure were used for evaluating the performance of binary classification. Table 1 demonstrates the performance of Naive Bayes, SVM, and deep learning binary classification.  
In Table 1, precision and recall were obtained from the average of two damage and non-damage classes by equal weight. The results of Table 1 show that the SVM algorithm performed better in all the indices and Naive Bayes algorithm performed poorly in all the indices. The deep learning algorithm also performed moderately in all of the indicators. However, its results were closer to SVM than to Naive Bayes.  
* Multi-class classification  
For multi-class classification, [PERSON] et al. [16] datasets, including 14,006 earthquake-labeled tweets (Napa and Nepal earthquake labeled datasets ([[https://crisisnlp.qcri.org/](https://crisisnlp.qcri.org/)]([https://crisisnlp.qcri.org/](https://crisisnlp.qcri.org/))), accessed on 16 October 2019) were used to create the training dataset. This dataset was split into two categories--60% of them were applied for training and 40% of them were applied for testing. The results show SVM classifier accurately identified damage-related messages. Precision, accuracy, recall, Kappa, and F-measure were used for evaluating the performance of multi-class classification algorithms. Table 2 demonstrates the performance of Naive Bayes, SVM, and deep learning multi-class classification.  
In Table 2, precision and recall were obtained from the average of all five classes by equal weight. The results of Table 1 show that the SVM algorithm performed better in all the indices and naive Bayes algorithm performed relatively poorly in all the indices. The deep learning algorithm also performed moderately in all of the indicators.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Binary Classification** & **Accuracy** & **F-measure** & **Recall** & **Precision** & ** Kappa** \\
**Algorithm** & & & & & \\ \hline Naive Bayes & 71.22\% & 63.03\% & 64.91\% & 61.25\% & 0.249 \\ \hline Deep Learning & 84.74\% & 76.17\% & 75.64\% & 76.71\% & 0.520 \\ \hline SVM & 89.30\% & 81.22\% & 79.08\% & 85.61\% & 0.634 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance of Naive Bayes, SVM, and deep learning binary classification.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Multi-Class Classification** & **Accuracy** & **F-Measure** & **Recall** & **Precision** & ** Kappa** \\
**Algorithm** & & & & & \\ \hline Naive Bayes & 79.81\% & 78.46\% & 81.33\% & 75.79\% & 0.662 \\ \hline Deep Learning & 86.76\% & 83.43\% & 80.26\% & 86.86\% & 0.762 \\ \hline SVM & 90.25\% & 81.22\% & 88.58\% & 93.26\% & 0.825 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance of Naive Bayes, SVM, and deep learning multi-class classification.