Discourse Mode Categorization of B engali Social Media Health Text (2023)

Abstract: The scarcity of annotated data is a major impediment to natural language processing (NLP) research in Bengali, a language that is considered low-resource. In particular, the health and medical domains suffer from a severe paucity of annotated data. Thus, this study aims to introduce BanglaSocialHealth, an annotated social media health corpus that provides sentence-level annotations of four distinct types of expression modes, namely narrative (NAR), informative (INF), suggestive (SUG), and inquiring (INQ) modes in Bengali. We provide details regarding the annotation procedures and report various statistics, such as the median and mean length of words in different sentence modes. Additionally, we apply classical machine learning (CML) classifiers and transformer-based language models to classify sentence modes. We find that most of the statistical properties are similar in different types of sentence modes. To determine the sentence mode, the transformer-based M-BERT model provides slightly better efficacy than the CML classifiers. Our developed corpus and analysis represent a much-needed contribution to Bengali NLP research in medical and health domains and have the potential to facilitate a range of downstream tasks, including question-answering, misinformation detection, and information retrieval.

## introduction with the increasing popularity of social media, various types of online content generated by vast numbers of people have become available. the health and medicine-related data are no exception, accumulating at a high pace as more and more people are using social media for health-related queries and discussions @xcite . in fact, nowadays, by possessing ample amounts of health-related information, social media has become one of the prominent data sources for health-related research. people all over the world use online health forums to acquire medical information. besides, people share their experiences regarding diseases, symptoms, and related matters to help other patients. due to the importance of medical and health text mining, the nlp community has organized a series of open challenges focusing on biomedical entity extraction and classification @xcite . the importance of social support in online health forums has been discussed in many earlier studies @xcite . as individuals seek support and information regarding various health-related issues in health and well-being forums, it is imperative to analyze them for a better understanding of user needs and to provide required support @xcite . for example, @xcite , in their study, classified covid-related health text into the following four categories: i) emotional support given, ii) emotional support sought, iii) informational support given, and iv) informational support sought. another health text classification task was performed by @xcite , where the authors classified health-related text into narratives and non-narrative categories. a study related to the identification of informative health posts was conducted by @xcite . categorizing health-related text on social media into distinct discourse modes can be beneficial for a range of downstream natural language processing (nlp) tasks, as each mode has specific roles in health support and discussion. for instance, usergenerated questions can provide insights into the outbreak of the disease over time @xcite and can facilitate the development of social support chatbots that cater to the needs of individuals seeking healthcare-related assistance @xcite . user narratives or experiences can reveal valuable information on disease symptoms and severity. in addition, it can help to find peers with similar experiences @xcite . it is imperative to analyze the suggestions or information-related discourse shared by peer users to detect the dissemination of disinformation and misinformation @xcite . in english and a few other major languages, various health-related corpora are publicly available @xcite . however, although bengali (also known as bangla) is one of the most spoken languages in the world foot_0 , such resources are typically not available @xcite . with the growing popularity of telemedicine and the availability of health and medical-related data written in bengali, creating resources for developing an nlp-based health system in bengali is a pressing necessity. to comprehend and automatically categorize health-related text, it is essential to have at least a moderate amount of annotated data. hence, in this study, we introduce a discourse mode annotated health corpus, the first of its kind, for the low-resource bengali language. the dataset is created by retrieving publicly available healthrelated texts from a number of social media health forums. the retrieved text data are tokenized into sentence-level and annotated with four types of discourse modes: narrative (nar), informative (inf), suggestive (sug), and inquiring (inq). the final corpus contains around 2000 sentences annotated by one of the four types of sentence modes. the details of the annotation procedure and various statistics of the sentence modes are provided. in addition, we present a baseline evaluation by employing multiple ml classifiers for the automatic categorization of the sentence modes. we observe that top classical ml classifiers and deep learning-based models demonstrate similar efficacy for the classification tasks. ## results and discussion we evaluate the performances of various classifiers based on 5-fold cross-validation and report f1 scores. the results indicate that the lr classifier yields the best performance among the four traditional ml classifiers by achieving an f1 score of around 0.78. svm performs similarly and achieves an f1 score of about 0.76. the tree-based methods show comparatively inferior performances. we observe that the performance of cml classifiers is affected by the class distribution of the dataset. all classifiers yield better results for the narrative (nar) mode as nar mode represents the highest number of samples (42%) in the dataset. although the transformer-based multilingual lan-guage model yields slightly better performance than the cml classifiers, the improvement is not significant compared to cml classifiers, which can be attributed to the limited amount of labeled data. with more labeled data incorporated, the improvement may be higher as transformer-based models have shown state-of-the-art performances for various nlp tasks across languages. ## summary and future work developing an effective framework for analyzing social media health data has substantial practical applications. however, such tools require annotated data which is hardly available in a low-resource language like bengali. therefore, we introduce a bengali health corpus created from several bengali social media health pages. we report detailed annotation guidelines and procedures for the annotation. moreover, we provide various statistics of four types of discourse modes in the annotated corpus. we make the corpus publicly available for the researchers. future work will focus on enlarging the size of the annotated corpus. besides, we will investigate how to leverage cross-lingual resources from other languages, such as english, to improve the performance of this classification task.