## 1 Introduction  
Despite the myriad of platforms for public expression expansion, the internet is seemingly rife with prejudice, and freedom of speech often serves as a veil for insidiousness on social media. The increase in online toxicity has led to the need for stronger detection methods to create a safer (and less toxic) internet. The best approaches to offensive language detection are heavily based on machine learning (ML), deep learning (DL), and transformer-based models. Fine-tuning pre-trained language models, such as BERT, has been proven to effectively improve the ability to detect abusive content [1]. However, ensuring that these models are explainable and reliable is still a challenge. Some recent approaches have suggested incorporating logical rule-based methods into neural frameworks to provide explainability [2], and others have used data augmentations based on believed symmetries of the data to improve the models overall generalization and explainability [3].  
In marginalized communities, the effects of offensive speech can be profound, especially on adolescents with autism, where AI-enabled virtual companions have been developed to help users become aware of and combat cyberbullying [4]. Moreover, cross-lingual learning methods have also been investigated to enhance hate speech detection in multiple languages, revealing the impact of transfer learning techniques on multilingual toxicity [5]. For instance, when dealing with languages previously mentioned (e.g., Spanish), linguistic features, as well as transformer-based architectures, have been studied [6], and the combination of multiple features has been identified as a crucial aspect that is looked into in order to improve the models. Overall, techniques such as zero-shot and few-shot learning have been applied to increase the adaptability of these models in a multilingual setting, alleviating the extensive requirements for labeled data [7].  
More recent literature has methodologies such as ring hybrid methodologies, e.g., using added bidirectional encoder-decoder architectures, which have yielded promising results in this domain of interest [8]. Following suit, optimization-driven approaches, including hybrid CNN-based classifiers, have also shown enhancement in classification performance for abusive comments [9]. The efficacy of transformer-based models for trait-based classification tasks has been further solidified through extensive analysis of multimodal classification for cyberbullying detection [10]. To create synthetic but realistic training samples, data augmentation strategies and, most notably, contrastive self-supervised learning have been suggested to improve cyberbullying detection [11]. Transfer learning methods were additionally validated and beneficial to datasets pertaining to Twitter, increasing the classification of hate speech in social media [12].  
These advancements aside, the detection of offensive language in Roman Urdu is a relatively unexplored area. Roman Urdu, a widely adopted script for Urdu on digital channels, poses a challenge with its varying standards of writing, code-mixing, and blurred grammar. This study tackles these challenges by applying and evaluating traditional ML models (Random Forest, Support Vector Machines, Naive Bayes, and Logistic Regression), deep learning models (CNN and Bi-LSTM), and state-of-the-art transformer-based models (LLaMA 2 and ModernBERT). Moreover, we utilize Low-Rank Adaptation (LoRA) for the efficient tuning of transformer models, ensuring the best performance with minimal computational resources. In our comparative analysis, we compare how well these models perform in identifying offensive language in Roman Urdu social media comments, providing insights into their portability and real-world usability.