# Machine Learning- and Deep Learning-Based Multi-Model System for Hate Speech Detection on Facebook
## 2 Literature Review
Developing standardized and diverse datasets is crucial for improving the effectiveness of machine learning models. To detect hate speech, numerous studies have investigated the performance of ML and DL models in different scenarios. Findings indicate that deep learning models, particularly LSTM and CNN, consistently outperform ML classifiers [31]. For instance, a recent study showed that LSTM achieved 96% accuracy, outperforming SVM and Random Forest models [30]. However, DL models require large amounts of labeled data and significant computational resources, which limits their widespread adoption for low-resource languages such as Roman Urdu. Hate speech detection models must balance accuracy with ethical considerations [32]. Automated systems are prone to biases, particularly when trained on imbalanced datasets or biased labeling practices [30]. Some studies have highlighted the risk of over-censorship, where AI models incorrectly classify non-hate speech as offensive. Ensuring fairness, transparency, and unbiased model training is critical in the development of effective hate speech detection systems. Hate speech detection has garnered much attention in the past few years, thanks to the spread of offensive content on social media. In the early days, the majority of the approaches were based on classic machine learning classifiers (such as SVM, Naive Bayes, and Logistic Regression) with surface-level features like TF-IDF or Bag-of-Words. Since the introduction of deep learning, models such as CNN and LSTM have shown better performance because their structure can capture the contextual semantics. Morerecently, another group of models, those designed in the transformer paradigm, have exceeded prior art by using contextual embeddings and a large scale of pretraining, e.g., BERT, RoBERTa, XLM-R. There are several studies [33] that have investigated multilingual and cross-lingual hate speech detection, most of which, however, are restricted to high-resource languages, such as English, Arabic, or Spanish. Although these models are highly generalizable, they do not cope well with the informal, noisy style of user-generated content in low-resource languages. A limited number of studies have been conducted for Roman Urdu because it does not have a standard orthography, has linguistic inconsistencies, and has hardly any annotated datasets available. In [34], a BiLSTM, BigRU model for hate speech detection in Roman Urdu with FastText embeddings, was based on a small and non-cross-domain dataset. The authors employed BERT+CNN-gram and handcrafted features but did not investigate large language models or explainable AI techniques. Also, most of the existing work does not consider the intricacy of code mix and the cultural connotations of Roman Urdu phrases. We aim to address this gap by employing QLoRA-optimized LLMs (e.g., LLaMA3, Mistral) on translated Roman Urdu data. The focus of future research should be on developing more robust datasets, improving model interpretability, and integrating explainable AI (XAI) techniques for better decision-making [4]. Additionally, multilingual and cross-lingual hate speech detection approaches can help address the challenges of low-resource languages. Exploring the role of transformer-based architectures such as BERT and RoBERTa could further enhance performance, although these models require substantial computational power [35]. Despite significant advancements in hate speech detection, research on Roman Urdu remains underdeveloped due to linguistic variability, limited datasets, and the lack of standard NLP tools. Existing studies have primarily focused on either traditional ML classifiers or deep learning models in isolation, often overlooking the potential of hybrid approaches that combine both techniques for enhanced performance. Moreover, previous works have struggled with imbalanced datasets and contextual ambiguity, limiting their real-world applicability.