MIT - KEC - NLP @ D ravidian L ang T ech- EACL 2024: Offensive Content Detection in K annada and K annada- E nglish Mixed Text Using Deep Learning Techniques (2024)

Abstract: This study presents a strong methodology for detecting offensive content in multilingual text, with a focus on Kannada and Kannada-English mixed comments. The first step in data preprocessing is to work with a dataset containing Kannada comments, which is backed by Google Translate for Kannada-English translation. Following tokenization and sequence labeling, BIO tags are assigned to indicate the existence and bounds of objectionable spans within the text. On annotated data, a Bidirectional LSTM neural network model is trained and BiLSTM model’s macro F1 score is 61.0 in recognizing objectionable content. Data preparation, model architecture definition, and iterative training with Kannada and Kannada- English text are all part of the training process. In a fresh dataset, the trained model accurately predicts offensive spans, emphasizing comments in the aforementioned languages. Predictions that have been recorded and include offensive span indices are organized into a database.

## introduction the research focused on comments that were combined kannada and english in order to investigate language processing processes for identifying unacceptable content in multilingual situations. in the first phase, which was called data preparation, a rich set of unacceptable spans in kannada comments were chosen. subsequently, the dataset was transformed and transliterated by google translate, facilitating an extensive examination of patterns in the usage of foul language. during the labeling process, text was separated into tokens, such as words or subwords, using sequencing and tokenization. the offensive spans were classified as beginning (b), inside (i), or outside (o) by the innovative bio tagging approach. this made it possible to assess the offensive context inside the language context more precisely. a bidirectional long short-term memory (bil-stm) neural network was employed in this study to detect objectionable content in comments that were written in both kannada and english. by addressing code-mixed text and distinct linguistic patterns, the natural language processing approach improved the detection of such information in a range of linguistic situations. the research was focused on identifying incorrect information within mixed kannada and kannada-english comments, which contributed to the growing interest in dravidian languages. it recognized the value of applying specialist techniques to successfully negotiate the nuances of diverse linguistic contexts. using keywords and knowledge from earlier research, the goal was to make nlp applications more inclusive and culturally sensitive. after training was finished, the computer confidently moved on to new datasets, tokenizing text and correctly identifying problematic spans. this step focused on the concept's adaptability and generalizability to different language situations. the model's work was arranged into a tsv file at the export results phase, signifying the conclusion of the ground-breaking inquiry. the offensive content that was discovered required more examination and in-depth analysis, which was made possible by these forecasts that included offensive span indices. this study addresses the problem of multilingualism in natural language processing by providing a systematic way to find offensive gaps in comments. it combined state-of-the-art technology with a flow diagram that offered insights and strategies for overcoming language obstacles in order to lower language barriers and promote safety and inclusivity in the online environment. ## related works the multilingual analysis was development @xcite of a transformer-based technique for identifying inappropriate language in code-mixed tamil text. however, @xcite concentrated on named entity recognition (ner) in tamil, demonstrating the variety of nlp applications in this language.further, @xcite translation explored sentiment analysis in tamil using deep learning approaches. along with tamil, there have been significant efforts done in kannada language processing. kannada text summarization challenges were the primary focus @xcite , @xcite concentrated on kannada part-of-speech tagging, highlighting the diverse range of kannada natural language processing (nlp) problems. morphological study and applications of tamil text @xcite . additionally, in @xcite a deep learning-based machine translation system for the english language was developed for the indian language (amarappa and sathyanarayana, 2015) a multinomial naïve bayes (mnb) classifier was used for the kannada named entity recognition and classification (nerc). in 2019 @xcite indo-european neural machine translation system. moreover, using tamil tweets, @xcite predicted the sentimental reviews of tamil movies. these initiatives shows the increasing interest in and need for dravidian language-specific methodologies. for effective nlp applications, customized methods are needed due to the challenges inherent in these languages, such as code-mixed text and unique linguistic structures. ## problem and system description the dataset was shown in ## conclusion a multilingual approach was employed in the study to pinpoint problematic spans in kannada and kannada-english literature. the bidirectional lstm model's macro f1 score of 61.0 was made possible by extensive pre-processing of the data. because of its flexibility, the model was able to be used with new datasets to predict offensive spans in situations with a mixture of kannada and english as well as pure kannada. an important advancement in natural language processing was made by this work, which tackled the issue of culturally acceptable material filtering in online communication.