# Social Media Analytics for Disaster Response: Classification and Geospatial Visualization Framework
## 4 Results
### Social Media Text Classification
#### 4.1.3 Experimental Results  
Table 2 shows the results for the weighted F1 score of text classification. Among all the evaluated models, ModernBERT achieved the highest weighted F1 score, surpassing the performance of other transformer-based architectures. This result underscores the advancements brought by ModernBERT's architectural innovations, which contribute to its superior ability to capture complex patterns within disaster-related social media data. However, it is worth noting that, for certain disaster events, switching between different models did not yield significant performance improvements. This observation suggests that the limitations in classification accuracy may be influenced by the quality and granularity of the dataset's annotations, rather than being influenced solely by model architecture.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Disaster** & **Class** & **BERT** & **RoBERTa** & **DistilBERT** & **BigBird** & **ALBERT** & **ELECTRA** & **ModernBERT** \\ \hline
2016 Ecuador & & & & & & & \\ Earthquake & 8 & 0.823 & 0.868 & 0.853 & 0.859 & 0.832 & 0.870 & **0.900** \\ \hline
2016 Canada Wildfires & 8 & 0.754 & 0.786 & 0.780 & 0.751 & 0.759 & 0.780 & **0.842** \\ \hline
2016 Italy & & & & & & & \\ Earthquake & 6 & 0.823 & 0.872 & 0.851 & 0.843 & 0.818 & 0.867 & **0.900** \\ \hline
2016 Kaikoura & & & & & & & \\ Earthquake & 9 & 0.731 & 0.741 & 0.732 & 0.736 & 0.730 & **0.752** & 0.746 \\ \hline
2016 Hurricane & & & & & & & \\ Matthew & 9 & 0.751 & 0.775 & 0.768 & 0.759 & 0.750 & 0.767 & **0.796** \\ \hline
2017 Sri Lanka Floods & 8 & 0.686 & 0.729 & 0.718 & 0.708 & 0.690 & 0.721 & **0.845** \\ \hline
2017 Hurricane & 9 & 0.730 & 0.759 & 0.747 & 0.739 & 0.739 & 0.748 & **0.791** \\ Harvey & & & & & & & \\ \hline
2017 Hurricane & 9 & 0.705 & 0.717 & 0.719 & 0.704 & 0.712 & 0.719 & **0.733** \\ Irma & & & & & & & \\ \hline
2017 Hurricane & & & & & & & \\ Maria & 9 & 0.706 & 0.725 & 0.722 & 0.710 & 0.714 & 0.720 & **0.767** \\ \hline
2017 Mexico & & & & & & & & \\ Earthquake & 8 & 0.834 & 0.852 & 0.847 & 0.842 & 0.836 & 0.837 & **0.915** \\ \hline
2018 Maryland Floods & 8 & 0.684 & **0.764** & 0.708 & 0.690 & 0.699 & 0.756 & 0.751 \\ \hline
2018 Greece Wildfires & 9 & 0.702 & 0.742 & 0.735 & 0.711 & 0.715 & 0.732 & **0.836** \\ \hline
2018 Kerala Floods & 9 & 0.723 & 0.741 & 0.728 & 0.736 & 0.733 & 0.739 & **0.821** \\ \hline
2018 Hurricane & & & & & & & \\ Florence & 9 & 0.755 & 0.776 & 0.763 & 0.760 & 0.759 & 0.770 & **0.810** \\ \hline
2018 California & & & & & & & & \\ Wildfires & 10 & 0.741 & 0.768 & 0.757 & 0.758 & 0.749 & 0.756 & **0.804** \\ \hline
2019 Cyclone & & & & & & & & \\ Idai & 10 & 0.733 & 0.789 & 0.759 & 0.749 & 0.740 & 0.791 & **0.844** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Weighted F1 score of text classification (best results are bolded).  
Despite these constraints, ModernBERT, as the most recent evolution of BERT-based models, demonstrated consistently strong performance across the majority of disaster-related datasets. Its advanced training techniques and architectural refinements allowed it to outperform earlier models, making it particularly effective in processing social media data for disaster response. These results showcase the potential of ModernBERT for enhancing situational awareness and facilitating timely decision-making in disaster management systems.