Findings of the Shared Task on Multimodal Social Media Data Analysis in D ravidian Languages ( MSMDA - DL )@ D ravidian L ang T ech 2024 (2024)

Abstract: This paper presents the findings of the shared task on multimodal sentiment analysis, abusive language detection and hate speech detection in Dravidian languages. Through this shared task, researchers worldwide can submit models for three crucial social media data analysis challenges in Dravidian languages: sentiment analysis, abusive language detection, and hate speech detection. The aim is to build models for deriving fine-grained sentiment analysis from multimodal data in Tamil and Malayalam, identifying abusive and hate content from multimodal data in Tamil. Three modalities make up the multimodal data: text, audio, and video. YouTube videos were gathered to create the datasets for the tasks. Thirty-nine teams took part in the competition. However, only two teams, though, turned in their findings. The macro F1-score was used to assess the submissions

## introduction analyzing insights from social media data with several modalities-text, audio, and video-is known as multimodal social media data. text data from sources like facebook posts, youtube comments, and tweets make up conventional social media data. the primary goal of social media data analysis is to extract useful information from text data. however, a study published in @xcite considers the variety of content posted on social media sites. multiple modalities in the data can be analyzed to provide a more thorough knowledge of user behaviour, attitudes, and trends. it is possible to think of the features retrieved from audio and video data as extra information that improves the text features of the input data. pitch, tone, and the video's facial expressions can all be used to fine-tune the elements used to identify various viewpoints and expressions more accurately. to analyze and identify various data categories, multimodal social media data analysis integrates methods from several fields, such as computer vision (cv), speech processing, and natural language processing (nlp). the multimodal social media data analysis in dravidian languages (msmda-dl) at the fourth workshop on speech, vision, and language technologies for dravidian languages (dravidianlangtech-2024) at eacl 2024 has three tasks: multimodal sentiment analysis in tamil and malayalam; multimodal detect abusive language in tamil; and multimodal detect hate speech in tamil. the primary goal of the shared task is to motivate researchers and academicians worldwide to join and submit their methods and findings to help research in languages with limited resources, such as tamil and malayalam. the shared task on multimodal social media data analysis in dravidian languages (msmda-dl) is summarized in this work. additionally, the results of the submitted models for the three subtasks are discussed in this study. the shared task was hosted on the codalab foot_0 . all enrolled participants received access to training and validation data to construct their models. the test data without labels were later shared to use the developed models to forecast the future. thirty-nine teams signed up for the two subtasks. only two teams, though, turned in their findings. ## literature review users on many social media sites write messages and comments in their native tongue and codemixed languages. as a result, machine learning models developed using monolingual datasets are inappropriate for classifying abusive language or deciphering the emotions present in languages with mixed coding. however, scientists are making strides toward creating systems with codemixed datasets. one major problem is, as mentioned above, the process of gathering and annotating data. ## description of the subtasks the three subtasks -multimodal sentiment analysis in tamil and malayalam, multimodal abusive language identification in tamil and multimodal hate speech detection in tamil -as well as the dataset utilized, are covered in this section. there are two tasks in the "multimodal sentiment analysis in tamil and malayalam" subtask: one in tamil and one in malayalam. both tasks are modelled as a multiclass classification task with. the subtask "multimodal abusive language detection in tamil" is a binary class classification task, whereas the third task, "multimodal hate speech detection in tamil", is a multiclass classification task. all of the tasks mentioned above included text, audio, and video data, and participants could use any combination of modalities to create their models. ## system description we received two submissions; only one team participated in all three tasks. each team could submit up to three runs. the run with the highest macro f1 score was considered when creating the rank list. below are the system descriptions that were submitted for the shared tasks. the team @xcite participated only in abusive language detection. the team used con-vlstm for the video dataset, and for the audio dataset, they used bilstm and multinomial naive bayes for the text dataset. they did not use any external data to build the model. this team developed three models. from these models, they counted the majority-based result for the final output. ## conclusion the results of the shared task on multimodal social media data analysis in dravidian languages (msmda-dl) are presented in this study. the task dataset was made up of transcripts and audio that went along with videos that were gathered from youtube. thirty-nine people signed up for each of the three subtasks. nevertheless, only two teams turned in their predictions for the test data that the participants were given. to create the rank list and evaluate the performance of the submitted forecasts, we employed the macro f1-score.