Selam@ D ravidian L ang T ech 2024:Identifying Hate Speech and Offensive Language (2024)

Abstract: Social media has transformed into a powerful tool for sharing information while upholding the principle of free expression. However, this open platform has given rise to significant issues like hate speech, cyberbullying, aggression, and offensive language, negatively impacting societal well-being. These problems can even lead to severe consequences such as suicidal thoughts, affecting the mental health of the victims. Our primary goal is to develop an automated system for the rapid detection of offensive content on social media, facilitating timely interventions and moderation. This research employs various machine learning classifiers, utilizing character N-gram TF-IDF features. Additionally, we introduce SVM, RL, and Convolutional Neural Network (CNN) models specifically designed for hate speech detection. SVM utilizes character Ngram TF-IDF features, while CNN employs word embedding features. Through extensive experiments, we achieved optimal results, with a weighted F1-score of 0.77 in identifying hate speech and offensive language.

## introduction in the digital age, social media plays an important role in online communication, allowing users to create and share material while also giving accessible means to express their views and thoughts on anything at any time @xcite . however, with the advent of social media, platforms such as youtube, facebook, and twitter not only aided in information sharing and networking, but they also became a place where people were targeted, defamed, and marginalized based solely on their physical appearance, religion, or sexual orientation @xcite social media platforms have become more integrated with this digital era and have impacted various people's perceptions of networking and socializing @xcite . not only human beings, the hate content can corrupt the chatbots as well. microscoft's chatbot 'tay' which was developed to engage people through casual and playful conversation started using filthy language, which it learned from the conversation with people. the chatbot was unable to understand and avoid the hate content. so the detection of hate speech in tweets and social media sites has important applications in chatbot building, content recommendation, sentiment analysis, etc. india being a diverse country in terms of its culture and language has a huge population using code-mixed language in social media. around 44% of the indian population speak hindi. so the usage of hindi-english code-mixed language is very high on twitter and facebook. it is mainly seen among bilingual and multilingual communities. code-mixing is the usage of certain words, phrases, or morphemes of one language in other languages. this influence allowed different users to communicate via various social media platforms using a mix of texts. nlp technology has advanced rapidly in many applications, including machine translation @xcite , @xcite although considerable progress has been achieved in identifying offensive english language and hate speech, most research has mostly concentrated on identifying the abusive and offensive language in monolingual settings. this subject still appears to be at a very early stage of research for under resourced languages such as tamil, malayalam, and kannada, which lack tools and datasets @xcite , @xcite . the task at hand involves identifying hate or offensive content in telugu code-mixed text, with annotations made at the comment or post level. given the complexity of language mixing in the telugu context, where expressions may span multiple sentences, a tailored approach is crucial. in response, we deploy a convolutional neural network (cnn) @xcite to enhance the effectiveness of detection methods. this research not only contributes to hate speech detection but also addresses the specific challenges posed by code-mixed expressions in telugu. the subsequent sections delve into the deployment of cnn in our methodology, present experimental results, and discuss the implications of our approach. ## related work currently, solving nlp problems in code-mixed data is getting attention from many researchers. social platforms are inundated with hate speech and offensive content, necessitating swift filtration @xcite . however, manual filtering is nearly impossible due to the immense volume of incoming posts. this issue has garnered significant attention from the research community. numerous studies have focused on predicting offensive and hate speech posts on social platforms, encompassing various languages. however, a majority of these studies have predominantly utilized english languages. languages detecting offensive content in social media comments is not a novel concept for the english language @xcite . several systems have also been developed for languages other than english, such as hindi, germany @xcite , @xcite . however, there is limited research focused on identifying offensive content in low-resource dravidian languages such as tamil, malayalam, and kannada @xcite . the study proposes a method for identifying offensive language in code-mixed kannada-english, malayalam-english, and tamil-english language pairs sourced from social media. @xcite the authors advocate for a multi-label classification approach, leveraging an ensemble of indicbert and generic bert models, to recognize and mitigate offensive content on social media platforms. @xcite , @xcite addressing a multi-label classification challenge with various sub-categories, the system employs tokenization of the data before model training. the calculation of class-wise confidence scores, subsequently combined into an output vector, facilitates effective classification. across the malayalam-english, tamil-english, and kannada-english datasets, the achieved f1 scores are 0.54, 0.72, and 0.66, respectively. this research task is outlined in the study conducted by @xcite . introduced is a potent multiclass abusive detection model, showcasing an impressive accuracy and f1-score of 0.99 on a well-balanced dataset. the model detected abusive comments within tamil and telugu english code-mixed text, and employed the tf-idf technique in conjunction with svm, as demonstrated by @xcite . ## datasets the dataset employed in this research is sourced from the hate and offensive language detection in telugu code-mixed text (hold-telugu) dataset provided by dravidian-langtech@eacl @xcite the data look below in 1 the data set contains telugu-english code-mixed language comments, along with the labels and text ids. the labels indicate whether the comment is offensive language or hate speech since there were only 2 category comments in the training set and no such labeled comments found in the test set, we have discarded those hate or offensive labeled samples from our corpus. hence, we have viewed this as a binary classification in this dataset, we have a total of 6930 telugu tweet comments, of which tried 5355. the remaining 1575 samples were part of the testing set we have tried to find a suitable representation for telugu language text data and also studied the effect of stemming and stop word removal by applying various methods. the details of the proposed methodology are presented in the following sections. the described methodology is extensively detailed in this section. figure 2 illustrates the comprehen-sive flowchart outlining the process of identifying hate speech and offensive tweets. the section commences with an overview of the dataset,1 followed by an in-depth exploration of the proposed models. we used the dravidianlangtech dataset to carry out our experimentations. each comment in the dataset is labeled as offensive (hate) or not offensive (not hate). we experimented with different conventional classifiers such as (i) support vector machine (svm), (ii) naïve bayes (nb), (iii) random forest (rf), and (iv) convolutional neural network (cnn. we employed character n-gram tf-idf features in conventional machine learning classifiers and the dnn model. in the case of cnn, we used a 100-dimensional word embedding vector to represent each word of the tweet. in our case, we fixed the maximum word length for comments to 10000 words. as in our dataset, we found most of the comments were less than 100 words, due to this we chose the maximum length of 1000 words for the experiments. it means we curtailed the words of the comments that have more than 42 words and padded for the comments that have less than 100 words. it means a comments matrix with a (42 × 100) dimension is passed through the cnn network to extract features from the comments to classify them into hate and not hate classes. a convolutional neural network is a multi-layered neural network with a unique design that is used to recognize complex data features. we implemented one layered convolutional neural network with 128 filters of 3-gram to extract features from the text and then this feature is passed through the max pooling operation and the activation function to get the feature map and this feature map is then used by the dense layer to classify tweets into hate and not-hate classes. 4.1 conclusion show in figure 3 the results underscored the exceptional success of the cnn model, achieving a weighted f1 score of 0.7711. this emphasizes cnn's ability to effectively balance accurate identification of offensive or nonoffensive content while minimizing false positives and negatives. although svm, naive bayes, and random forest demonstrated commendable performances, the superior performance of the cnn model highlights the significance of deploying deep learning techniques for intricate tasks such as identifying hate and offensive speech. the visual representation through the bar graph further emphasized the comparative performance, with the cnn model standing out due to its higher weighted f1 score. looking ahead, the integration of machine learning and deep learning models, alongside ongoing parameter tuning for the cnn-based model, presents promising avenues for enhancing the efficiency of hate speech identification in comments.