Findings of the Shared Task on Multi-task Learning in D ravidian Languages (2022)

Abstract: We present our findings from the first shared task on Multi-task Learning in Dravidian Languages at the second Workshop on Speech and Language Technologies for Dravidian Languages. In this task, a sentence in any of three Dravidian Languages is required to be classified into two closely related tasks namely Sentiment Analyis ( SA ) and Offensive Language Identification ( OLI ). The task spans over three Dravidian Languages, namely, Kannada, Malayalam, and Tamil. It is one of the first shared tasks that focuses on Multi-task Learning for closely related tasks, especially for a very low-resourced language family such as the Dravidian language family. In total, 55 people signed up to participate in the task, and due to the intricate nature of the task, especially in its first iteration, 3 submissions have been received.

## introduction the term "social media" provides a channel through which people engage in interactive communities and networks by creating, sharing, and exchanging thoughts and information. it has received users from almost all generations and all around the world @xcite . users can interact and connect with others and form communities through social media. it allows users to share their ideas, views and information openly on various topics. this gives license to the users to write hateful and offensive comments sometimes. people come from a variety of racial backgrounds and hold a diversity of belief systems. this can often cause conflict of opinions during their interactions on social media platforms @xcite @xcite . due to the covid 19 pandemic, the internet community has become more popular than it has ever been. the amount of false narratives and derogatory remarks shared on online platforms has risen exponentially. a large number of social media users share malicious posts despite understanding that they are infringing on their rights to free expression @xcite . sentiment analysis is a text mining task that identifies and extracts personal information from source material, allowing a company/researcher to better understand the social sentiment of its brand, product, or service while monitoring online conversations. multi-task learning (mtl) is a practical approach to improving system performance by utilising shared characteristics of tasks @xcite . the goal of mtl is to use learning multiple tasks at the same time to improve system performance @xcite . because sa and oli are essentially sequence classification tasks, we were motivated to conduct the shared task, due to the recent developments in large language modeling. kannada and malayalam are dravidian languages that are widely spoken in south india and are also official languages in the states of karnataka and kerala @xcite @xcite @xcite . tamil is an official language in tamil nadu, india, as well as sri lanka, singapore, malaysia, and other parts of the world. dravidian languages are morphologically rich; with code-mixing, processing these languages becomes even more difficult, and they are under-resourced @xcite @xcite @xcite @xcite @xcite . significant minority speak tamil in the four other south indian states of kerala, karnataka, andhra pradesh, and telangana, as well as the union territory of the andaman and nicobar islands @xcite @xcite @xcite . it is also spoken by the tamil diaspora, which may be found in malaysia, myanmar, south africa, the united kingdom, the united states, canada, australia, and mauritius. tamil is also the native language of sri lankan moors. tamil, one of the 22 scheduled languages in the indian constitution, was the first to be designated as a classical language of india @xcite @xcite . tamil is one of the world's longest-surviving classical languages. the earliest epigraphic documents discovered on rock edicts and "hero stones" date from the 6th century bc. tamil has the oldest ancient non-sanskritic indian literature of any indian language @xcite . the shared task on mtl in dravidian languages investigates whether it is beneficial to train models using mtl, as obtaining extensive annotated data for under resourced languages is difficult. additionally, sa and oli have discourse properties in common @xcite . the lack of large labelled data for user-generated code-mixed datasets motivated the selection of these tasks. past studies have shown us that the benefits to mtl are two folds, namely, reducing the space/time complexity, and the ability for the model to learn from each other tasks @xcite . our dataset contains a wide range of code-mixing, from simple script mixing to morphological mixing. the task is to determine the polarity of sentiment and offensiveness in a code-mixed dataset of tamil-english, malayalam-english, and kannada-english comments or posts. this paper presents an overview of the task description, dataset, description of the participating systems, analysis, and provide insights from the shared task. ## dataset the dravidiancodemix dataset @xcite is the primary resource of the shared task. it comprises of over 60,0000 manually annotated comments scraped from youtube. additionally, dravidiancodemix spans three languages in the dravidian language family, namely, kannada, malayalam, and tamil. the kannada code-mixed dataset has 7,273 comments, while the malayalam and tamil codemixed datasets have 12,711 and 43,349 comments, respectively. following the removal of repetitive sentences, figure 1 shows the class-wise distribution of the datasets which will be split into train, validation, and test sets. ## evaluation, results and discussion the submissions were primarily evaluated using major classification metrics such as macro averaged and weighted average precision, recall, and f1-score. we predominantly used macro averaged f1 score to rank the teams because it identifies the f1 score to every label and calculates their unweighted mean. mtl in its essence is a very challenging problem, especially when we focus this aspect on lowresourced language family such as dravidian languages (kannada, malayalam, and tamil). ## conclusion in its first iteration, the shared task on mtl for dravidian languages opened up new avenues for research in low-resource multi-task learning. the task involved multiple languages, namely, kannada, malayalam, and tamil. this overview article analyzed the systems that were submitted to the shared task. the main inference from the participants is that mtl is a very challenging problem, especially for morphologically rich languages and all participants performed single task learning and aggregated the outputs.