Findings of the Shared Task on Offensive Language Identification in T amil, M alayalam, and K annada (2021)

Abstract: Detecting offensive language in social media in local languages is critical for moderating user-generated content. Thus, the field of offensive language identification in under-resourced Tamil, Malayalam and Kannada languages are essential. As the user-generated content is more code-mixed and not well studied for under-resourced languages, it is imperative to create resources and conduct benchmarking studies to encourage research in under-resourced Dravidian languages. We created a shared task on offensive language detection in Dravidian languages. We summarize here the dataset for this challenge which are openly available at https://competitions.codalab.org/competitions/27654 , and present an overview of the methods and the results of the competing systems.

## introduction the dawn of social media helped to bridge the gap between the borders and paved the way for the people to communicate or express their opinions more easily than any other time in human history @xcite . but with this advent of social media, the platforms like youtube, facebook, and twitter not only helped information sharing and networking but also became the place that target, defame and marginalize people merely on the basis of their physical appearance, religion or sexual orientation @xcite @xcite . social media has moulded itself into its specialised tool with which the people are verbally threatened and cornered not for what they have done but for who they are @xcite . undoubtedly, the infiltration depth and width of this 'digital wonder' equipped the erstwhile 'invisible and socially paralysed' communities to play cards, if not extravagantly noticeable @xcite . research in hate speech detection @xcite or offensive language detection @xcite using natural language processing (nlp) has significantly improved in recent years. however, the work on underresourced languages is still limited @xcite . for example, under-resourced languages such as tamil, malayalam, and kannada lack tools and datasets @xcite @xcite . recently, shared sentiment analysis for tamil and malayalam by @xcite and offensive language identification in tamil and malayalam by @xcite paved the wave for more research on dravidian languages. tamil, malayalam and kannada belong to the dravidian language family and are spoken by some 220 million people in the indian subcontinent, singapore and sri lanka @xcite . it is essential to create nlp systems such as hate speech detection or offensive language detection in local languages since most user-generated content is in such languages. the dravidian language family is reported as being approximately 4500 years old by bayesian phylogenetic analysis of cognate-coded lexical data @xcite . the dravidian languages scripts are first attested in the 580 bce as tamili 1 script inscribed on the pottery of keezhadi, sivagangai and madurai district of tamil nadu, india @xcite 2 by tamil nadu state department of archaeology and archaeological survey of india. the modern dravidian languages have its own scripts evolved from tamili scripts. the scripts are alpha-syllabic, belonging to a family of the abugida writing systems that are partially alphabetic and partially syllable-based 1 also called damili or dramili or tamil-brahmi 2 keeladi-book-english-18-09-2019.pdf @xcite . the writing system of tamili was explained in the old grammar text tolkappiyam which dates are various proposed between 9th century bce to 6nd century bce @xcite @xcite and in the jaina work samavayanga sutta and pannavana sutta, these two jain works date to 3rd-4th century bce @xcite . up until around the 15th-century ce, malayalam was the west coast dialect of tamil @xcite . due to geographical isolation by the steep western ghats from the main speech community, the dialect eventually evolved into a distinct language by the 15th century @xcite . the first literary work in malayalam was ramacaritam ("deeds of rama") was written using tamil grantha script which was used to write pali, prakrit and foreign words in tamil @xcite . the first printed work in an indian language and script was a roman catholic catechism translated by henrique which is thambiran vanakkam (doctrina christam en lingua malauar tamul in portuguese) on 20 october 1,578 ce at quilon, venad (present day kerala) @xcite . similarly, kannada also split from tamil by 9th century ce; for example, taayviru is a term of kannada origin and is found in a tamil inscription from the 4th-century ce. the kappe arabhatta record of 700 ce is the oldest extant form of kannada poetry in the tripadi metre, it is based in part on kavyadarsha @xcite ). however, user-generated content is often adopted to roman script for typing due to historical reason and modern computer keyboard layouts. hence, the majority of user-generated data for these dravidian languages are code-mixed @xcite . due to the growth of social media platforms across the world and the possibility of writing content without any moderation, users write content in multilingual code-switching without grammatical restrictions and using non-native scripts @xcite . in linguistics, code-switching is switching between two or more language in the same utterance. this explosion of code-mixed user-generated content in the social media platforms aroused interest in nlp. in this paper, we dicuss about the offensive language identification shared task for tamil, malayalam and kannada languages and the participant's submissions. ## task description offensive language identification for dravidian languages at different levels of complexity were developed following the work of @xcite . it was customized to our annotation method from three-level hierarchical annotation schema. a new category not in intended language was added to include comments written in a language other than the dravidian languages. annotations decision for offensive language categories were split into six labels to simplify the annotation process. sample of not-offensive comments in our dataset provided for the participants is given below with the corresponding english glosses.. this is an example of offensive targeted individual. mr. ghibran is a popular music composer in tamil films, and the author tries to insult him. english: "except the presence of disgusting manju, everything is super in this movie" this is an example of offensive targeted individual. mrs. manju warrier is a popular actress in south indian films, and the author tries to discredit her by clearly mentioning that her presence is disgusting. this is an example of offensive untargeted comment. this text have inadmissible language without targeting anyone. ## results and discussion the offensive language identification shared task was organized for three languages tamil, malayalam, and kannada. many participants connecting to all three languages had submitted their solutions as described in the previous section. here in this section, we'll be highlighting the results of all three 6 shows the overall results and teams which are placed in the top three positions. the team hate-alert @xcite ) came at the first position in tamil and malayalam languages and second in kannada as well, and the team indicnlp@kgp @xcite ) came second in tamil, first in malayalam, and third in kannada. the team sj-aj (jayanthi and gupta ( 2021 )) came at the first position in kannada and third and second in tamil and malayalam respectively.as per the f1-scores apart from malayalam, the other two languages got scores around 0.75, whereas 0.97 was the former's score. as the vocabulary size of malayalam is more, it has produced better results compared to tamil (which is having the maximum), and kannada, where vocabulary is less but has got similar results to that of tamil. ## conclusion we presented the results of the first shared task on offensive language identification in tamil, malayalam, and kannada relying on a thoroughly annotated data set based on human judgements. the task setup provided an opportunity to test model in multilingual scenarios along with code-mixing phenomenon. however, several teams reach high performance in all three languages. we found out that the type of embedding strongly influence the results. we hope this task and dataset will intrigue and facilitate further research on under-resourced dravidian languages.