Wit Hub@ D ravidian L ang T ech-2024:Multimodal Social Media Data Analysis in D ravidian Languages using Machine Learning Models (2024)

Abstract: The main objective of the task is categorised into three subtasks. Subtask-1 Build models to determine the sentiment expressed in multimodal posts (or videos) in Tamil and Malayalam languages, leveraging textual, audio, and visual components. The videos are labelled into five categories: highly positive, positive, neutral, negative and highly negative. Subtask-2 Design machine models that effectively identify and classify abusive language within the multimodal context of social media posts in Tamil. The data are categorized into abusive and non-abusive categories. Subtask-3 Develop advanced models that accurately detect and categorize hate speech and offensive language in multimodal social media posts in Dravidian languages. The data points are categorized into Caste, Offensive, Racist and Sexist classes. In this session, the focus is primarily on Tamil language text data analysis. Various combination of machine learning models have been used to perform each tasks and do oversampling techniques to train models on biased dataset.

## introduction the digital age has fundamentally transformed our information landscape, with social media emerging as a dominant force shaping how we interact and engage with the world. while its benefits are undeniable, the rapid spread of online hate has become a pressing concern, posing significant threats to trust, democracy, and societal cohesion. unrestricted access to post any data that may be offensive or abusive is a very important con of social media. this issue is particularly acute in dravidian languages, where the lack of dedicated tools and resources exacerbates the impact of negativity in social media. to address this challenge, significant research efforts have been directed towards developing advanced models capable of effectively detecting and categorizing various forms of harmful content in online spaces. this paper delves into the development of such models within the context of tamil text data, focusing on three critical tasks: through these tasks, the research presented in this paper highlights the immense potential of multimodal nlp and deep learning techniques in analyzing the complexities of communication within tamil text data. the developed models offer practical solutions for combating misinformation, fostering trust, and promoting healthy online spaces. the model traverses through different models employed for each model, such as lstm, knearest neighbors, linear regression, multinomial naive bayes and others and explains the purpose for each method. firstly the methodology and data is analysed, which includes model description, previous models and disadvantages,then an overview of obtained results, limitations, and conclusion with the findings. the datasets that have been used are @xcite , @xcite , @xcite . the overview of the shared task is given in findings of the shared task on multimodal social media data analysis in dravidian languages (msmda-dl) @xcite . ## related work banerjee (banerjee et al., 2020), has used an autoregressive xlnet model to perform sentiment analysis on code-mixed tamil-english and malayalam-english datasets. dravidianmultimodality: a dataset for multimodal sentiment analysis in tamil and malayalam is a paper @xcite , where the product or movies review videos were downloaded from youtube for tamil and malayalam. next, the captions were created for the videos with the help of annotators and the videos were labelled for sentiment. in the paper (ofli et al., 2020), they are doing analysis on social media data using multi modal deep learning for disaster response. they have used cnn, image modality and others. by using these models, they performed 2 tasks, informativeness classification task and humanitarian classification task with f1 score 84.2 and 78.3 respectively. ## experimental result and performance analysis the prescribed models had been submitted to codalab and the runs on the test data had been submitted. the submissions were evaluated and and the results are as in ## limitations the model is trained only over a small sample of training data. there may be various other data that should be included for better performance of the model. and also, as linguistic trends change, the model may be ineffective over time. we need measures to prevent that too. ## conclusions in conclusion, various models have been trained and tested for each subtask. these models have been trained only under a specific small dataset and are adapted to it. the models prescribed are best adapted to the small training datasets and are proven to produce a good f1 score for each task. the model can be improved by including online learning techniques and reinforcement learning to adapt to new data and trends and thus have an enhanced performance.