# Social Media Analytics for Disaster Response: Classification and Geospatial Visualization Framework
## 3 Methods
### Models  
This study concentrates on multiclass classification tasks using transformer-based models to enhance text classification for disaster-related social media posts. Specifically, we employed BERT [20], RoBERTa [21], DistilBERT [24], BigBird [25], ALBERT [23], ELECTRA [22], and ModernBERT [26] to train and evaluate the effectiveness of our classification system.  
* BERT: BERT is a transformer-based model designed to understand the context of words by simultaneously analyzing both their left and right surroundings. It is pre-trained on large corpora using two objectives: masked language modeling and next sentence prediction. This dual-training strategy allows BERT to capture complex relationships between words, making it highly effective for a wide range of natural language processing tasks, including sentiment analysis, question-answering, and text classification.
* RoBERTa: RoBERTa builds upon BERT's foundation by refining the pretraining process. It eliminates the next sentence prediction task and instead focuses on training with larger batches and more data for longer durations. These adjustments enhance the model's ability to capture subtle linguistic patterns, leading to an improved performance across various benchmarks and making it more robust for complex text classification tasks.
* DistilBERT: DistilBERT is a condensed version of BERT that prioritizes efficiency while retaining much of the original model's performance. It uses a technique known as knowledge distillation, where a smaller model learns from a larger, pre-trained model. This results in a faster, lighter model that is particularly suitable for applications with limited computational resources, such as mobile devices or real-time processing systems.  
* BigBird: BigBird extends the capabilities of traditional transformer models by introducing a sparse attention mechanism, allowing it to process much longer sequences without dramatically increasing computational costs. This innovation makes BigBird especially suitable for tasks that involve lengthy documents or sequences, such as document summarization, genome analysis, or processing large-scale social media streams during disasters.
* ALBERT: ALBERT enhances BERT's efficiency by reducing the number of parameters through cross-layer parameter-sharing and factorized embeddings. Despite being a smaller model, ALBERT achieves competitive results by emphasizing sentence-order prediction during pretraining. Its memory-efficient design makes it well-suited for environments where computational resources are constrained while still delivering strong performance on various NLP tasks.
* ELECTRA: ELECTRA introduces a novel pretraining method that diverges from the traditional masked language modeling used in BERT. Instead, it uses a generator-discriminator framework, where a discriminator learns to differentiate between real tokens and those generated by a smaller model. This approach leads to more sample-efficient training, allowing ELECTRA to achieve high performance with reduced computational costs, making it an effective solution for large-scale text classification tasks.
* ModernBERT: ModernBERT represents the latest evolution in BERT-based architecture, incorporating advanced optimization strategies and training techniques. It leverages innovations such as dynamic masking and improved pretraining objectives to enhance both efficiency and accuracy. ModernBERT is designed to achieve superior results on contemporary NLP benchmarks, making it highly effective for complex text classification tasks in dynamic environments such as disaster response systems.  
By evaluating and comparing these advanced transformer-based models, this study seeks to determine the most effective method for the classification of social media posts. This allows emergency response teams to rapidly analyze incoming data and respond appropriately to disaster situations, improving the efficiency and effectiveness of crisis management operations.