## 5 Discussion

Table 4: _Cont._understand the meaning and context of language, aiding in distinguishing offensive language from non-offensive language based on the underlying meanings conveyed by words. Similarly, our third approach employed BERT, ALBERT, and ELECTRA, and each model was chosen for its specific strengths in capturing contextual information and handling offensive language nuances [72]: BERT's bidirectional training objective, ALBERT's parameter-efficient modifications, and ELECTRA's novel pre-training task; all models contributed to enhancing the models' ability to understand the context in which offensive language occurs, thereby improving offensive language detection accuracy. Algorithm 8 demonstrates the process for OLD (Offensive Language Detection) on social networks, including steps for preprocessing, feature extraction, and training using traditional machine learning, deep learning, and transformer-based models. We opted for BERT, ALBERT, and ELECTRA BERT due to their consistent performance across various NLP tasks, indicating their efficacy in handling complex language understanding tasks. Their pre-trained representations learned on large text corpora facilitate efficient transfer learning to offensive language detection, requiring minimal annotated data for fine-tuning. These transformer-based models excel in capturing contextual nuances, crucial for accurately detecting offensive language. Their attention mechanisms enable them to consider both local and global context, enhancing their understanding of offensive language usage patterns. Moreover, their sub-word tokenization techniques effectively handle non-standard language elements like slang and misspellings commonly found in offensive content. This ensures robust performance even in the presence of out-of-vocabulary words. **Input:**  
1. Training dataset containing text samples labeled with offensive or non-offensive categories
2. Pre-trained GloVe embeddings
3. Machine learning models (e.g., SVM, Random Forest, Logistic Regression)
4. Transformer-based models (e.g., BERT, ALBERT, ELECTRA BERT)
5. Deep learning models (e.g., neural networks)  
**Output:**  
1. Trained classifier for OLD  
**Steps:**  
1. Preprocess the text data:  
1. Tokenize the text into words or subwords. 2. Remove stop words and punctuation. 3. Perform lemmatization or stemming. 2. Extract features from the text data:  
1. Compute TF-IDF vectors for each text sample. 2. Generate Word2 Vec embeddings. 3. Perform sentiment analysis. 4. Calculate FKRA/FRE readability scores. 5. Use GloVe embeddings to represent words. 3. Concatenate the extracted features, excluding GloVe embeddings. 4. Train traditional machine learning models on the concatenated feature representation. 5. Train deep learning models (e.g., neural networks) using GloVe embeddings. 6. Fine-tune pre-trained transformer-based models (BERT, ALBERT, ELECTRA BERT) and used as preprocessing and encoder with 1D CNN, Neural Network on OL Classification  
7. Evaluate the performance of each model. 8. Output the trained classifier for OLD. The widespread adoption of BERT, ALBERT, and ELECTRA BERT in both research and industry circles provides access to abundant resources, pre-trained models, and fine-tuning strategies tailored for offensive language detection tasks. While our current study demonstrates strong performance across multiple approaches on the Davidson dataset (with F1-scores of 0.89 for feature fusion with ML models and GloVe with DL models, and 0.90 for transformer-based models), we recognize certain limitations in the statistical validation of our results. Specifically, our study does not yet include comprehensive statistical analyses such as F-tests and N-tests that would enhance the robustness of performance comparisons. Additionally, metrics such as the mean and standard deviation of performance results, confidence intervals, and statistical significance tests between different models were not computed.