SSN -Nova@ LT - EDI 2024: Leveraging Vectorisation Techniques in an Ensemble Approach for Stress Identification in Low-Resource Languages (2024)

Abstract: This paper presents our submission for Shared task on Stress Identification in Dravidian Languages: StressIdent LT-EDI@EACL2024. The objective of this task is to identify stress levels in individuals based on their social media content. The system is tasked with analysing posts written in a code-mixed language of Tamil and Telugu and categorising them into two labels: “stressed” or “not stressed.” Our approach aimed to leverage feature extraction and juxtapose the performance of widely used traditional, deep learning and transformer models. Our research highlighted that building a pipeline with traditional classifiers proved to significantly improve their performance (0.98 and 0.93 F1-scores in Telugu and Tamil respectively), surpassing the baseline as well as deep learning and transformer models.

## introduction stress is a complex and multifaceted psychological and physiological response to challenges or demands, often characterized by a state of heightened arousal and a perceived inability to cope with the stressor @xcite . in the contemporary era, psychological stress is widely acknowledged as a major factor contributing to a variety of health issues and mental disorders. the complexities of modern life, marked by rapid technological changes, societal expectations, and economic pressures, have made stress a pervasive issue. people from various backgrounds are experiencing the adverse effects of persistent stress, leading to various health challenges and mental health issues @xcite . the dynamic evolution of social networks has prompted a widespread trend wherein individuals extensively employ various social media platforms as primary channels for expressing their thoughts and emotions. this shift in communication patterns emphasizes the growing reliance on these platforms as primary channels for expressing perspectives and feelings. notably, amidst this surge, people increasingly turn to social media to vent about their stress @xcite , highlighting the importance of identifying and addressing these expressions to support the mental well-being of users. various approaches have been crafted for the analysis of physiological data with the goal of stress identification @xcite . the feasibility of identifying the stress of the users through the analysis of their social media activities, such as tweets has been substantiated @xcite . however, the current state of machine learning models reveals a gap in addressing stress detection in dravidian languages, such as tamil and telugu due to the dearth of well-annotated datasets and proficiently trained models specific to these linguistic contexts. this deficit poses a significant challenge in developing accurate algorithms for identifying stress within the nuanced expressions inherent in dravidian language structures. our paper's sequence is as follows -in section 2, we navigate through existing publications focusing on text classification tasks in low-resource languages. section 3 undertakes an analysis of the dataset distribution. our proposed model's methodology is outlined in section 4. section 5 examines the performance metrics of our solutions. ## related work extensive research in the field of sentiment analysis and text classification has predominantly centred around languages with latin scripts, such as english and spanish @xcite @xcite . not much research has been reported in other languages with some notable ones including arabic @xcite and dravidian languages @xcite . to address the major issues regarding dra-vidian code-mixed and code-switched datasets, transformer-based models like m-bert, distil-bert, xlm-roberta, and muril, which are pretrained on a large corpus of multiple indian languages, have proven to outperform deep learning (dl) models @xcite . however, aiming to explore and leverage the applications of state-of-the-art technology including transformers and deep-learning in low-resource languages, @xcite implemented ensemble techniques with the experimental outcomes of the weighted ensemble framework outperforming state-of-the-art models by achieving 0.802 and 0.933 weighted f1-score for malayalam and tamil code-mixed datasets. similar results were produced using a pre-trained multilingual-bert model with convolution neural networks @xcite . withal, taking the limited availability of annotated data in dravidian languages such as tamil and telugu @xcite , traditional machine learning models have outperformed state-of-the-art technology in numerous similar ventures due to their ability to learn linear features from smaller datasets @xcite . ## dataset analysis the task has been bifurcated based on language into tamil and telugu. the provided labels for the data were "stressed" and "non stressed". the data distribution is provided in ## results and analysis on analysis of the precision of each model implemented following the different vectorisation techniques, tf-idf has emerged as the most efficient. this could plausibly be due to tf-idf's sparse representation of the document-term matrix, emphasizing the importance of terms based on their frequency and rarity across the corpus. this is beneficial when dealing with low-resource scenarios where datasets are limited, as it helps capture distinctive features. word2vec, while powerful, relies on distributed representations and might face challenges in low-resource scenarios where the model struggles to capture diverse semantics due to limited data. this may also affect combination vectorisation techniques as it may introduce complexity and degrade the performance. however, the outlier in this hypothesis is the superiority of the word2vec vectoriser when utilised in the cnn-lstm model due to its lowerdimensional embeddings and continuous vector representations for words that capture semantic relationships and contextual information, making it more suitable for deep learning models. deep learning techniques have proven to perform significantly well with longer-length sentences @xcite . however, most social media comments tend to be much shorter, enabling better performance than traditional models. hence, we hypothesise that an ensemble of traditional classifiers employing probabilistic and deterministic classifiers would produce better results than deep learning models not only due to the linear relationship of the data but also the usage of shorter sentences. parallel to both conventional deep learning methodologies and traditional approaches, the m-bert transformer model achieved an accuracy of 99.93 on the tamil validation dataset. nevertheless, despite its extensive multilingual training, theoretically providing the transformer with a competitive advantage, an investigation revealed that the model exhibited signs of overfitting. this occurred despite deliberate efforts to mitigate overfitting by reducing the model's complexity and implementing data augmentation techniques to address class imbalances in the dataset. this revelation prompted a reevaluation of our methodology, prompting an exploration of alternative strategies to fortify the resilience of our model. ## conclusion in conclusion, our research on stress identification for dravidian languages has yielded insightful findings across various vectorisation techniques and modelling approaches. the analysis of model accuracy points to tf-idf as the most efficient vectorisation technique. the discrepancy in the performance of the m-bert transformer raises questions about its adaptability in specific linguistic contexts, emphasising the importance of thorough validation and optimisation procedures even in well-established transformer models. furthermore, the study underscores the nuanced dynamics between the nature of the data and model performance. in light of the discerned constraints of the m-bert transformer model, our investigation reveals that social media comments favour traditional models, leading us to propose an ensemble technique, leveraging both probabilistic and deterministic classifiers to outperform deep learning and stand-alone classifiers. the voting classifier emerged as an enticing alternative, not only combatting the challenge of overfitting but also elevating the overall efficacy of our model.