Deciphering psycho-social effects of Eating Disorder : Analysis of R eddit Posts using Large Language Model( LLM )s and Topic Modeling (2024)

Abstract: Eating disorders are a global health concern as they manifest in increasing numbers across all sections of society. Social network platforms have emerged as a dependable source of information about the disease, its effect, and its prevalence among different sections. This work lays the foundation for large-scale analysis of social media data using large language models (LLMs). We show that using LLMs can drastically reduce the time and resource requirements for garnering insights from large data repositories. With respect to ED, this work focuses on understanding its psychological impacts on both patients and those who live in their proximity. Social scientists can utilize the proposed approach to design more focused studies with better representative groups.

## introduction eating disorders (ed) are an area of increasing concern even as it continues to be under-reported and under-researched. eating disorders are recognized by signs and symptoms which are published in the fifth edition of the diagnostic and statistical manual of mental disorders (dsm-5). clinical symptoms of ed most often manifest as anorexia nervosa, bulimia nervosa, binge eating disorder, or arfid which is defined as an eating or feeding disturbance so pervasive that the person is unable to meet appropriate nutritional needs, resulting in significant weight loss, nutritional deficiency, dependency on nutritional supplements, or interference in social functioning. in @xcite , it was reported that there is an alarming rise in ed across the world. according to the paper, an estimated 5.5-17.9% of young women and 0.6-2.4% of young men have experienced a dsm-5 eating disorder by early adulthood in the western world. studies from other parts of the world also show similar trends. a study conducted by nimhans in india in 2018 reported that eating disorders (ed) affect 6.5% of adolescent girls in india and an estimated 2-3% of the population as a whole. national eating disorders association foot_0 reinforces this and states that though eating disorders impact people of all genders, ages, races, religions, ethnicities, sexual orientations, body shapes, and weights, the diversity is not represented in published research, which has most often relied on surveys of targeted groups. this lack of inclusion implies that the true impact of ed is still not well understood. with the advent of hyper-specific online communities, those suffering from eating disorders (eds) have access to social network platforms where they can share their experiences anonymously and seek support from each other as well as from therapists, counselors, and caregivers. reddit, a popular online forum, hosts many such communities centered around eating disorders, with the largest one being r/eatingdisorders. these platforms are good sources of data since they are used by patients as well as those who live in proximity. hence unlike targeted surveys which analyze information from affected individuals only, social media platforms also give insights into the psychological stress and trauma experienced by other groups of people like parents, partners, siblings, and friends of affected individuals. in this paper, we present a study that exploits large language models (llm) and topic modeling for analyzing large volumes of reddit posts, to obtain insights about ed and its impact on social relationships. while traditional topic modeling techniques like latent dirichlet allocation (lda) perform a global analysis of large repositories, llms are good at generating summaries and answering questions through contextual analysis of individual articles, like posts. however, being gen-erative in nature, the answers to different questions have to be further semantically analyzed to identify global patterns. most of the earlier work on analyzing social networks for similar purposes has been done on much smaller datasets. the research contributes to the growing body of literature on the intersection of mental health, social media, and natural language processing. since the posts are made available publicly after anonymization and careful removal of all sensitive information, these kinds of studies do not violate any ethical norms. it reinforces the potential of online platforms as a resource for studying eating disorders. it also establishes the role of llms in facilitating largescale automated analysis in social science as stated in @xcite . the focus of our work is on analyzing the psychological state of people suffering from ed and those who live in proximity, using both statistical technical techniques like lda and llms. consequently, the key research questions were formulated as follows: the rest of the paper is organized as follows. section 2 presents an overview of earlier work in the area. section 3 presents statistics about the data. section 4 presents the analytical pipeline, along with details of each module. this is followed by results and discussions. ## related work eating disorders (eds) are complex psychiatric disorders with a range of associated mental and physical health symptoms. a meta-review article published in 2021 @xcite analyzed thirty-three studies published across the world and concluded that the prevalence of eating disorders might have been underestimated thus far. combined analyses showed that the lifetime prevalence of eds in western countries was 1.89%, and was high at 2.58% in females. most of these studies did not use the dsm-5 criteria for their analyses. it also proposed that new diagnostic criteria should be used to comprehensively assess all types of eating disorders. in a study from finland @xcite , the prevalence of lifetime dsm-5 eating disorders among the finnish population was estimated to be 17.9% for females and 2.4% for males, and 10.5% across genders. a meta-review on ed-related research in india was presented in @xcite , and also states that cultural context should be included in the analysis, with the help of culturally sensitive instruments used for diagnosis, and generating locally relevant epidemiological data about eating disorders from community as well as hospital settings. the role played by social network platforms in the health and wellness pursuits of people was comprehensively analyzed in @xcite . the article states that social networking sites (snss) have evolved into an informal source of health education which influences peoples' health choices. analysis revealed that the discussion majorly focuses on the promotion of idealized bodies, healthy foods, diets, and exercise. while this raises concern about the influence of social media on mental health, especially promoting body image concerns, eating disorders, and psychological distress, it is also acknowledged that social media can be alternately used to encourage wellness through promoting self-acceptance, intuitive eating, and lifeenhancing movements. this work critically eval-uates both the positive and negative role of social networks concerning health and wellness content. prior to this, a small sample study on content analysis from a pro-ed community on reddit was presented in @xcite . this study delved into the task of identifying the expression of behaviors aligned with ed symptoms and support for these behaviors within the social network discussion. the paper presented results based on an analysis of four weeks of topic-specific discussion threads on a small dataset of 125 threads created for the purpose. it reported statistics on behaviors consistent with ed, and how the platforms provided support. topic modeling using latent dirichlet allocation (lda) was employed to identify topics in eating disorder-specific social media content in @xcite . the paper employed social network analysis using a linear network autocorrelation model to analyze communication patterns and the most influential users. twitter, another popular social network platform was studied by @xcite to assess social media engagement among individuals with ed. the focus of this was to develop an automatic approach to assess public perceptions about ed and ed-related behaviors. a machine learning approach was developed to identify ed-relevant tweets, and content analysis using topic modeling was implemented to unearth potential ed-related factors including behaviors, thoughts, and mental status. ## dataset details and data preparation we scraped data from three reddit communities: r/eatingdisorders, r/eatingdisordersover30, and r/edanonymous, spanning from 2020 to 2023, among which the most prolific one was r/eatingdisorders. a total of 20,918 posts and 58,228 comments posted from 2021 to 2023 were collected from this site. upon generating a timebased chart with the number of monthly posts, we found a significant increase in the numbers over this period. the most substantial spike occurred between 2021 and 2022 as shown in figure 1 , and continued consistently into 2023. for this paper, we have presented insights extracted from the posts that appeared in the years 2022 and 2023, since these would present the most contemporary issues. the results presented are based on an analysis of more than 16.5 k posts, with 6700 from 2022 and 10450 posted in 2023. text cleaning and prefigure 2: unsupervised content processing pipeline using traditional nlp techniques and llms processing included the removal of html tags, urls, special symbols, etc. the title and text were merged into one body for analysis. ## methodology figure 2 describes the processing pipeline, which is generic and can be applied for analysis of any large-scale repository. starting with an aggregate analysis of content, we move towards an analysis of individual posts. the first step towards aggregate analysis is topic modeling. statistical topic modeling techniques like latent dirichlet allocation (lda) leverage an unsupervised statistical approach to discover hidden semantic patterns from a large text collection to automatically identify topics that exist inside it. topics are represented by clusters of similar words within a body of text. though topic modeling is effective in obtaining a bird's eye view of content from a large repository, the words assigned to topics by themselves do not obtain much semantic information. to alleviate the problem, we have exploited the generative powers of an llm to generate a semantically coherent description of the topic from the representation words. this step is referred to as topic naming. the topic summaries generated at this step can be used to compare topics against each other. this has been utilized later to compute topic trends based on the percentage presence of similar topics across the years. the aggregate analysis is followed by an individual analysis of posts. since the intent of analyzing any repository is to obtain statistics about certain key concepts present in the repository, the objective of this step is to establish the presence or absence of the concepts in each post. this step is akin to "coding" in the social sciences, which is an analytical process by which data elements are "categorized" to facilitate analysis. traditionally this is an expensive knowledge-intensive task. in recent times, there have been efforts to use the large language models (llms) for this task, as these are capable of performing various language processing tasks without additional task-specific training. these models, termed as zero-shot llms, can also reliably classify and explain certain social phenomena @xcite . it is this aspect of a large language model, that we have exploited to use it as a computational social science tool. details for each of these sub-tasks are presented in the following subsections. ## results: findings and discussion we first present an aggregated analysis of the content obtained in terms of topics and topic trends using lda. a short description of each topic was generated by feeding the top 10 topic words to llm. to compare the topic strengths for two years, topic alignment was done using bert-based similarity scores among the short descriptions across the years. the unique topics, their names, and trends across the years found by lda are presented in figure 4 . while the most dominant topic of 2022 was "weight gain and image-related issues", the conversation around this substantially dropped in 2023. the topics rather shift towards those which are about "seeking help", sharing experiences on "relapse and recovery", "disordered eating", and "eating patterns and appetite" along with a discussion on handling "relationship issues" and advice on handling "social interactions". there are also topics centered around extreme feelings of guilt, shame, concern, and fear which appear as a major topic. these emotions will be discussed in more detail later. a topic centering around "cleanliness" discussing changes in personal cleanliness routines and organizational approaches to cleanliness featured in 2022, but not in 2023. this could be due to the immediate aftermath of the pandemic. the changes in topic trends indicate that the platform is indeed emerging as a forum where ed patients form a mutual support group. figure 5 presents the most frequently occurring topics in the posts as identified by the llm using prompt 1. comparing these with figure 4 shows that while both the results show similar trends, the llm-identified topics are broader in nature, more like themes rather than topics. an issue faced with llm themes was the wording differences in the answers generated for the posts. these themes had to be combined based on the words that appeared in them. the similarity of the topics and themes obtained by two different unsupervised methods increases confidence in the results obtained. the most commonly occurring theme is that of personal struggle faced by patients of ed, especially during recovery, as they frequently get into relapse. this is followed by concerns about body image which is at the center of triggering ed in people. impact on mental and physical health is also a recurring theme. the topics reveal finer nuances of these topics as weight-related issues for body image, guilty feelings along with anger, fear, and anxiety as chief mental health problems. few posts had explicit mentions of the age and gender of the author or the patient of ed as "x/g" where x indicated age and g indicated gender. 208 posts in 2022 and 351 posts in 2023 had such mentions. figure 6 shows their relative distrifigure 6: demographic profiles of users who explicitly mention age. gender -indicates trends and not estimates emotion analysis results were obtained from the output of prompt 2. analysis shows that the emotions present in the posts are predominantly negative, with concern, frustration, confusion, fear, and desperation dominating over others. figure 7 shows that the relative presence of these emotions is fairly consistent over time. such a high presence of negative emotions is consistent with the dsm 5 diagnostic disorders which mention that individuals who develop eating disorders are at increased risk of developing various co-occurring mental health concerns, including anxiety, depression, obsessivecompulsive disorder (ocd), and post-traumatic stress disorder (ptsd). a manual evaluation of the llm-assigned emotion was done for 100 posts, 50 from among those tagged with frustration and another 50 tagged with guilt and shame. the evaluation was done by a psychology expert, who had no prior exposure to the data, but was explained about the process and informed that the tags were assigned by a language modeling tool. for the first set on frustration, the accuracy of llm tagging was found to be 90%, for which the expert agreed that the post exhibited frustration. for the remaining posts, the expert marked the emotions as confusion, obsession, etc. and not frustration. for the posts marked with guilt, the expert agreed fully with the llm tagging. on inspection of the posts, it was found that all these posts had explicit mentions of guilty, guilt, shame. evaluations at a larger scale are being planned. however, the preliminary results do establish the effectiveness of using llms in large-scale data analysis for mental health from social networking sites. this also establishes the possibility of deploying llms to set up semi-supervised and distant-supervised learning frameworks, by providing a quick approach to generate training data, which is reasonably clean, and not expensive to initiate. figure 8 presents an analysis of answers generated in response to prompt 3. it extracts the relationship of the authors of a post to its subject who is mentioned in it as the patient suffering from ed. these results are from the 2023 collection. as expected, around 70% of the authors are the patients themselves. however, there are a substantial number of posts that are written by friends and acquaintances, parents, partners, and siblings. friends and acquaintances include colleagues, roommates, family friends, etc. siblings are mentioned either as such or as brothers or sisters. the partner tag was assigned to authors who were identified by the llm as "one in a romantic relationship with the person suffering from ed" or a spouse, girlfriend, boyfriend, or "someone in a close relationship with the patient". responses to prompt 4 provide deeper insights into the psychological impact of ed on the patient when the author is "self", as well as on other people when the author of the post is a friend, parent, partner, colleague, or sibling. these results are presented below. ## conclusion in this paper, we have presented a large-scale study on reddit posts related to eating disorder. we have shown how large language models like gpt can be used effectively to gain insights about facts and figures using their power to summarize and draw inferences against targeted prompts. the most significant insight gained through this study is about people who live in proximity to the subject of eating disorder, and use the platform to convey their distress. the psychological stress undergone by this group which includes parents, siblings, friends, and partners is not much reported. though the known effects of ed are found in the data, the study reveals that the platforms are primarily used for advice, support, and seeking empathy from cousers. the results are highly encouraging. our plan includes devising a way to methodically evaluate the llm-generated answers. the ultimate intent is to build a framework for analysis of mental healthrelated posts, that can expedite the whole process of insight generation as well as strategically plan interventions and support mechanisms.