NITK - IT _ NLP @ T amil NLP - ACL 2022: Transformer based model for Toxic Span Identification in T amil (2022)

Abstract: Toxic span identification in Tamil is a shared task that focuses on identifying harmful content, contributing to offensiveness. In this work, we have built a model that can efficiently identify the span of text contributing to offensive content. We have used various transformer-based models to develop the system, out of which the fine-tuned MuRIL model was able to achieve the best overall character F1-score of 0.4489.

## introduction as far as social media and entities involved in content moderation are concerned, identifying offensive content is critical. however, most of these companies employ content moderators for determining and mitigating offensive content, but they are frequently swamped by their volume @xcite . small firms cannot utilize human moderators because of the cost, and hence they turn off their comment sections fully. code-mixing is the mixing of various linguistic units from two or more languages in a conversation or even in a single utterance. when the indian perspective is considered, english is primarily influenced by all indian languages, including dravidian languages like tamil, malayalam, and kannada @xcite . hence this has become a part of different conversations in social media. many recent works address the whole comment classification as offensive or not but do not consider the span of text that makes it offensive. identifying this span of text will further help moderators who deal with these contents. offensive span identification is a shared task organised as a part of dravidianlangtech @acl-2022 foot_0 . they had two subtasks, supervised offensive span identification and semi-supervised offensive span identification, where we were given annotated as well as non-annotated data. the task was to identify the offensive span of text content. in this paper, we have used multilingual transformer-based models and local interpretable model agnostic explanations (lime) @xcite to identify the span of text. the paper is presented as follows; section 2 explains the dataset, section 3 is about the methodology used, section 4 explains the experiments and results, which follows the conclusions and future scope. ## related works offensive language identification is one of the widely explored problems. most of the work on offensive language identification tasks is of classification type rather than identifying the span of texts. recent works @xcite @xcite have explored various transformer-based models and some @xcite have made an ensemble of different ones which are focused on classification task. offensive span identification is in its developing stage, @xcite was the first to introduced a shared task and offensive span dataset. ## dataset description two subtasks were given on the codalab competition website foot_2 for offensive span identification in tamil, namely supervised offensive span identification and semi-supervised offensive span identification. the dataset @xcite had training and testing sets for both tasks, which are retrieved from youtube, whose details are given in ## methodology we had two subtasks as a part of this shared task on offensive span identification. the first task was to use a supervised method to identify offensive span of text in the data, and the second task was to use a semi-supervised method to do the same. we used the supervised method, which uses the transformer-based model to train the data for classifying offensive content. this trained model is used to predict whether the given comment is offensive or not. on top of this, we further examine the results on each class individually using input perturbation-based explanation method involving local interpretable model agnostic explanations (lime) @xcite . here we are training the model for offensive content identification from comment text. we have used data from the hasoc-2021 shared task foot_3 @xcite , which contains comments extracted from youtube annotated with labels 'offensive' and 'not-offensive.' the comments are preprocessed, tokenized, and fed to the pretrained model and further fine-tuned to make predictions. ## experiments and result we have fine-tuned the pre-trained models for the hasoc 2021 data using the hyperparameters mentioned in we initially trained our model with m-bert for the code-mixed offensive data. this model is used to predict the test data given for identifying offensive/harmful content. the final prediction is interpreted using lime, which will give a score for each of the words contributing to the offensive and non-offensive contents. those words contributing to the offensive texts are extracted and predicted for the given task and its span in the whole comment. we employed a similar procedure for the electra and muril model; because of the better representation for the indian context, muril was able to give a more reasonable prediction and we got first position for the same which is given in example comment text [offensive] : @user ungotha ku rate ellam illa ,free eh othukittu irukan da ungotha thevidiya mundaiyae. example comment text [not-offensive]: ellarum saptacha ..??? *** : yen sapadu vangi kuduka poriya ?? unaku rt dhana venum .... straight ah kelu !! ama..!! tag #tag. ## conclusions and future scope social media is the primary source from which people use to get information. hence, the companies that handle these need to moderate content so that the offensive and harmful content are not propagated. in this paper, we have explored the transformer-based model along with lime interpretations to identify the span of harmful content in comments. google's muril achieved the best result from different models, which came first in the leaderboard for the shared task. in the future, we would like to explore more on the code-mixed data and develop improved solutions to this problem.