CUET _ DUO @ S tress I dent_ LT - EDI @ EACL 2024: Stress Identification Using T amil- T elugu BERT (2024)

Abstract: The pervasive impact of stress on individuals necessitates proactive identification and intervention measures, especially in social media interaction. This research paper addresses the imperative need for proactive identification and intervention concerning the widespread influence of stress on individuals. This study focuses on the shared task, “Stress Identification in Dravidian Languages,” specifically emphasizing Tamil and Telugu code-mixed languages. The primary objective of the task is to classify social media messages into two categories: stressed and non stressed. We employed various methodologies, from traditional machine-learning techniques to state-of-the-art transformer-based models. Notably, the Tamil-BERT and Telugu-BERT models exhibited exceptional performance, achieving a noteworthy macro F1-score of 0.71 and 0.72 , respectively, and securing the 15^{th} position in Tamil code-mixed language and the 9^{th} position in the Telugu code-mixed language. These findings underscore the effectiveness of these models in recognizing stress signals within social media content composed in Tamil and Telugu.

## introduction along with the hectic pace of contemporary life, stress has become an unavoidable force impacting the mental well-being of humans. it is a complicated emotional state produced by multiple events that might inspire displeasure, rage, or worry. recognizing and resolving stress in its early stages is crucial since persistent stress may lead to devastating diseases, including depression @xcite . recent surveys indicate that 48% of gen z individuals experience depression symptoms, often triggered by the pervasive impact of social media. issues like the fear of missing out heightened concerns about judgment, and increased insecurity further contribute to stress levels @xcite . this highlights the need for efficient stress detection and support methods within online platforms. global stress statistics emphasize the importance of proper stress management, impacting various aspects of people's lives, from businesses and educational institutions to family contexts @xcite . automatic stress detection provides an effective solution to address this global health crisis, offering help and resources to individuals dealing with stress-related challenges. this research addresses the problem of stress identification in tamil and telugu code-mixed languages. this proposed study consists of the following key contributions: ## related work while various studies have studied stress detection in english and other high-resource languages, attention to low-resource languages like tamil and telugu has been sparse @xcite . chauhan et al. (2017) conducted a study using electrocardiogram data to analyze mental stress. they employed discrete wavelet transform for preprocessing and feature extraction techniques. nijhawan et al. (2022) used the application of unsupervised topic modeling using latent dirichlet allocation has facilitated the identification of emotions in online user data. this approach has proven effective in analyzing stress or depression, which achieved a high detection rate. another study (jadhav et al., 2019) focused on social media stress detection using textual data, highlighting the effectiveness of combining bilstm with an attention mechanism. dreaddit, a corpus of 190k reddit posts with 3.5k labeled for stress identification, was introduced by (elsbeth, 2019). few studies (li and liu (2020), oryngozha et al. (2023)) demonstrated high accuracy rates in stress identification through the application of conventional and neural supervised learning techniques on the dreaddit dataset. ahuja and banga (2019) focused on exam pressure and recruitment stress frequently ignored factors and aimed to determine the extent of stress experienced by college students. the researchers utilized four classification algorithms (lr, nb, rf, and svm) with a dataset comprising 206 student records from the jaypee institute of information technology. their study yielded the highest accuracy for svm. in another study conducted by @xcite , the relationship between users' stress states and their friends on social media was investigated using a large-scale real-world social platform dataset. researchers enhanced transformer-based models, including bert and mentalbert, by incorporating extra-linguistic data for depression and stress detection in social media @xcite . their approach involved a multimodal adaptation gate for combined embeddings, inputting data into a bert (or mentalbert) model, and model calibration through label smoothing @xcite . the study highlighted the robustness of transformer-based models like roberta, xlnet, and bert in stress tests but also identified fragility and unexpected behaviors, suggesting potential directions for further advancements in the field. ## task & dataset descriptions the task organizers curated a standardized dataset for identifying stress-related statements in tamil and telugu code-mixed social media texts. this effort aims to develop a system that proficiently recognizes stress expressions within a given social media text. the dataset is derived from the organizers' corpus @xcite , categorized into stressed (st) and non stressed (nst). ## methodology the suggested methodology encompasses assessing diverse feature extraction techniques, integrating ml and dnn, and exploring various transformer-based architectures. the comprehensive approach aims to explore the effectiveness of different strategies in addressing the challenge of stress identification in the specified linguistic context. figure 1 illustrates an overall outline of the stress identification technique in tamil and telugu code-mixed texts. ## results and analysis the lr displays competitive performance across ml models, reaching an accuracy of 0.72, a balanced recall of 0.57, and a macro f1-score of 0.64 for the tamil dataset. dt excels in recall (0.94), resulting in a higher macro f1-score (0.71), whereas nb displays high recall (0.99) but poorer accuracy, generating a macro f1-score of 0.68. the dl model gets a competitive macro f1-score of 0.70. among transformers, m-bert and tamil-bert demonstrate comparable performance, with macro f1-scores of 0.68 and 0.71, respectively. for telugu code-mixed language, lr obtains a moderate accuracy of 0.66, paired with a reduced recall, resulting in a macro f1-score of 0.27. decision tree stands out with solid recall (0.91) and a large macro f1-score of 0.70. naive bayes displays excellent recall (0.97) but poorer accuracy, pro-viding a macro f1-score of 0.70. cnn+bilstm delivers balanced accuracy, recall, and the greatest macro f1-score (0.71). transformer models, m-bert and telugu-bert, demonstrate decent performance, with telugu-bert (0.72) marginally beating m-bert (0.70) in the macro f1-score. the comparison analysis underlines the different performance of models in stress identification tasks for tamil and telugu code-mixed languages. lr and dt demonstrate different strengths in tamil, whereas in telugu, dt and cnn+bilstm do exceptionally well. the transformer models exhibit competitive performance but with variances in efficacy throughout the two languages. ## error analysis the stress detection performance of the bert model in both tamil and telugu code-mixed languages demonstrates excellent accuracy in recognizing stressed situations, with a large true positive count in both datasets. however, a substantial difficulty develops in the form of false positives, indicating examples incorrectly categorized as stressed, particularly within an environment of class imbalance when non-stressed instances outweigh stressed ones. figures 2 and 3 illustrate the performance of tamil-bert and telugu-bert models concerning the confusion matrix for the stress identification task. figure 2 illustrates the tamil-bert model's robust performance, accurately classifying 365 not stressed and 361 stressed samples out of 650 and 400, respectively. despite this, precision is limited, with 285 not stressed samples misclassified as stressed and 9 stressed samples misclassified as not stressed, indicating susceptibility to false positives, particularly in identifying not stressed samples. in figure 3 , the telugu-bert model demonstrates strong performance, correctly tagging 371 not stressed and 378 stressed samples out of 650 and 400, respectively. however, precision is limited, with 279 not stressed samples misclassified as stressed and 22 stressed samples misclassified as not stressed. this highlights a vulnerability to false positives, especially in identifying not stressed samples. figure 4: few examples of predicted outputs by the best performing model (tamil-bert) figure 5: few examples of predicted outputs by the best performing model (telugu-bert) figures 4 and 5 illustrates some correct and incorrect predicted outcomes by the best-performed models (tamil-bert and telugu-bert). ## conclusion this work presented a comprehensive study of stress detection within the code-mixed languages of tamil and telugu by exploiting various ml, dl, and transformer-based models. remarkably, the transformer model tamil-bert emerges as a remarkable performer, achieving the most significant macro f1 score of 0.71 in the context of tamil. meanwhile, in the domain of telugu, the leading model is telugu-bert, exhibiting a substantial macro f1 score of 0.72. future endeavors may involve the integration of culturally sensitive features, thereby enhancing the effectiveness of stress detection in social media interactions within specific linguistic contexts.