Lived Experience Not Found: LLM s Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use (2025)

Abstract: Adverse Drug Reactions (ADRs) from psychiatric medications are the leading cause of hospitalizations among mental health patients. With healthcare systems and online communities facing limitations in resolving ADR-related issues, Large Language Models (LLMs) have the potential to fill this gap. Despite the increasing capabilities of LLMs, past research has not explored their capabilities in detecting ADRs related to psychiatric medications or in providing effective harm reduction strategies. To address this, we introduce the **Psych-ADR** benchmark and the **A**dverse **D**rug Reaction **R**esponse **A**ssessment (**ADRA**) framework to systematically evaluate LLM performance in detecting ADR expressions and delivering expert-aligned mitigation strategies. Our analyses show that LLMs struggle with understanding the nuances of ADRs and differentiating between types of ADRs. While LLMs align with experts in terms of expressed emotions and tone of the text, their responses are more complex, harder to read, and only 70.86% aligned with expert strategies. Furthermore, they provide less actionable advice by a margin of 12.32% on average. Our work provides a comprehensive benchmark and evaluation framework for assessing LLMs in strategy-driven tasks within high-risk domains.

## introduction adverse drug reactions (adrs) 1 caused by psychiatric medications are a leading cause of hospitalizations among individuals with mental health conditions, accounting for 51.9% to 91.8% of cases, as reported in previous studies @xcite . with nearly 70% of individuals worldwide having limited access to mental health professionals, many patients increasingly 1 adr is defined as an appreciably harmful or unpleasant reaction, resulting from an intervention related to the use of a medicinal product, which predicts hazard from future administration and warrants prevention or specific treatment, turn to social media platforms such as reddit to share experiences and seek advice @xcite @xcite . yet, around 35% of posts on mental health-related subreddits go unanswered, leaving many without adequate support @xcite . further, while social media offers a platform for seeking assistance with resolving adr queries, responses are frequently provided by individuals lacking expertise, raising concerns about the reliability of the information shared @xcite . hence, as conversational ai platforms (such as chatgpt) gain prominence, more individuals are turning to these systems for healthcare-related queries, including those about psychiatric medication and adrs. given the current limitations of healthcare systems and social media platforms, alongside the growing capabilities of llms in mental healthrelated tasks @xcite @xcite , llms have the potential to bridge the gap in online discussions by providing high-quality, contextual responses to adr queries related to psychiatric medications. while previous studies have focused on detecting adrs using deep learning methods, these efforts have primarily addressed or alteration of the dosage regimen, or withdrawal of the product. @xcite . to answer the above stated research questions, we present psych-adr benchmark and adverse drug reaction response assessment (adra) framework. the proposed psych-adr benchmark includes 239 reddit posts, labeled across two hierarchical levels for adr detection and multiclass classification along with expert-written responses to queries. the proposed framework evaluates llm-generated responses against those of medical experts, focusing on four assessment axes: (a) text readability, (b) emotion and tone expression, (c) alignment of harm-reduction strategies, and (d) actionability of suggested strategies. rq1 results show that both adr detection and adr multiclass classification are challenging tasks, with the top model in a few-shot setting achieving f1 scores of 75.38 and 76.69 in respective tasks. we observed that all models exhibited a "riskaverse" nature, leading to a false-positive rate of over 70%. additionally, models struggled with nondose-related and time-related adrs, with gpt-4 turbo misclassifying 51% and 50% of these instances, highlighting difficulties in grasping nuanced adr types. for rq2, llm-generated responses were significantly harder to read than expert-written responses. in contrast, there was no observed significant difference in emotional or tone alignment between llm and expert responses. however, the best model (openbiollm-70b) achieved only 70.86% alignment with expert harm-reduction strategies, and llms provided 12.32% less actionable advice on average. given the observations, our research has important realworld implications. the proposed benchmark provides a resource for evaluating llms on tasks involving the interaction between mental health conditions and psychiatric medications. the proposed framework hold practical utility for policymakers, practitioners, and healthcare professionals to assess llm performance, especially in strategy-driven tasks in high-risk domains. we have provided code in a repository here foot_1 . ## data collection and curation we begin by providing the details of the data collection and filtering pipeline. we used publicly available data in english from reddit spanning a one-year period (january 2019 -december 2019) obtained from pushshift reddit dataset @xcite . while the broad timeframe ensures large enough data before filtering, the specific period also (a) predates the use of generative ai in day-to-day lives and (b) the knowledge cutoff for all llms used in our evaluation. this allows for a fairer comparison between human experts and llms and also ensures minimal presence of machine-generated content on reddit. following the past work @xcite @xcite , we selected 10 subreddits that focus on mental healthrelated issues or provide a platform for users to ask medical queries (such as r/depression and r/askdocs; see appendix a for the complete list). to extract relevant posts, we compiled a set of 297 fda-approved psychiatric medications provided by @xcite . further, to detect expressions of adverse symptoms in post titles and texts, we employed healthe @xcite , a specialized named entity recognizer for identifying healthcare and medical entities. by combining the psychiatric medication names with the entities given by healthe, we obtained 19,252 reddit posts. filtering based on mentions of psychiatric medications and adverse symptoms provides a rich sample to extract posts that strictly discuss symptoms caused by psychiatric medications, which is the focus of our study. to specifically filter out posts expressing concerns of adverse drug reactions (adrs), we prompted gpt-3.5 using definitions and specific conditions identified in previous research while also including insights from co-authors who are medical experts @xcite ; see appendix b and c for complete list of criteria and exact prompts. based on the annotations by gpt-3.5, we obtained 6,108 reddit posts expressing adr and 11,999 expressing no adr (rest were deleted). next, we discuss human validation of the labels for constructing the psych-adr benchmark. the annotation task proved to be challenging for the annotators, with an average time of ∼7.2 minutes taken to annotate each post due to the complexity and subjectivity inherent in detecting adverse drug reactions. all three annotators agreed on the labels for 48% of the posts. to address the disagreements, we conducted a second round of annotations in which all three annotators collaboratively resolved disagreements, resulting in the final set of labels (details in appendix d). finally, 11 posts were discarded due to their lack of relevance to adrs, resulting in a final benchmark comprising 239 annotated posts. ## model selection & implementation we conduct our analysis for the research questions with a total of 9 proprietary and openweights llms. for proprietary models, we evaluate gpt-4o (openai-gpt-4o, 2024), gpt-4 turbo @xcite evaluate llama-3.1 405b instruct-turbo, llama-3.1 70b instruct-turbo (meta-llama3.1, 2024) and specialized medical llms -llama3-med42-v2 70b @xcite and llama3-openbiollm 70b (ankit pal, 2024). the choice of these models stems from their reported performance in different general-purpose and medical benchmarks @xcite @xcite . previous studies have recommended lower temperatures for detection and labeling tasks to ensure more consistent outputs, while higher temperature values aid in more flexible generation @xcite . accordingly, for the adr detection and multiclass classification tasks we set the temperature @xmath0.6 for the response generation tasks. beyond the taskspecific temperature variations, the settings were kept consistent across all the llms. additional details regarding the models, evaluation setup, and compute are provided in appendix e. ## rq1: detecting adverse drug reaction for this task, we evaluated llms on detecting expressions of adverse drug reactions using the psych-adr benchmark. the evaluation involved two separate tasks: (1) identifying the presence or absence of concerns related to adrs in the 239 reddit posts, and (2) classifying the type of adr into one of five pre-defined categories for the 133 instances labeled as expressing adrs in psych-adr benchmark. in both tasks we evaluated models using the zero-shot and few-shot variants of the chain-of-thought (cot) prompting @xcite . detailed prompts and classification criteria are provided in appendix b and c. due to the wide variety of medications and symptoms in psych-adr benchmark, we eval-uated two different example sampling strategies for few-shot prompting. for this, we generated text embeddings for each reddit title and post using text-embedding-3-large (openai-textemb-3-large, 2024). using cosine similarity, we retrieved the five most-similar and five least-similar posts for each example. ## rq2: alignment between human and ai feedback evaluation of long-form text generation is an open problem and involves many challenges like isolating the stylistics from the semantics. however, in the context of responses to adr queries, we propose abstracting out the llm generations and ground-truth expert responses to four key components -(1) emotion and tone, (2) text readability, (3) harm reduction strategy, and ( 4 ) actionability of proposed strategies. via this abstraction to key components, our alignment evaluations focus on specific aspects that contribute towards an ideal response to adr queries. we explain the importance of these components below and the methodology for evaluation. emotional and tone alignment: emotional intelligence is regarded as a key factor in healthcare, fostering strong therapeutic relationships that drive meaningful change @xcite . therefore, llm-generated responses should align with expertwritten responses in tone and expressed emotion. to assess this, we used empath @xcite , a widely-used lexicon-based tool, focusing specifically on 8 relevant emotional and tonal categories identified from prior literature @xcite . we analyzed the distribution of these categories in llm-generated and expert responses, and quantified their differences using kullback-leibler (kl) divergence to measure alignment of expressed emotions and tone in the llm and expert responses. text readability alignment: past studies have shown that health literacy is strongly correlated with patient outcomes @xcite . a major factor contributing to lower health literacy is the communication barrier between patients and healthcare providers, which often arises from the complexity of medical text, including the writing style and choice of terminology @xcite . hence, the responses produced by llms should be easily readable and be of comparable to that of the expert-written responses. to assess this, we used smog index @xcite , a popular readability index to assess health literacy material. harm reduction strategy alignment: in cases of adverse reactions to psychiatric medications, suggesting safe medical interventions is crucial to prevent further harm. we operationalized these interventions using harm reduction strategies (hrs) @xcite , aimed at minimizing the negative effects of medications that one is reliant on. ideally, llms should propose strategies that align with the expert's responses. to compare the harm reduction strategies suggested by llms and experts, we took inspiration from methods for entailment and factuality evaluation in long-form texts @xcite @xcite . first, we extracted atomic hrs from llm responses by prompting gpt-4o (openai-gpt-4o, 2024). since some extracted strategies were redundant, we used a fewshot approach to combine those that suggested the same overall approach but differed in specific details to get the final set of hrs for each response (examples in we then evaluated alignment of hrs for each llm-expert response pairs using two methods. first, we used alignscore @xcite , a widely-used unified text alignment method providing a score between 0 and 1 based using a finetuned roberta-large model @xcite . we computed alignscore for each strategy from the llm response against the expert response. we obtained a response-level alignscore by averaging the scores across all hrs for the response. second, for a more interpretable alignment score, we prompted gpt-4o with in-context examples to reason and classify if a strategy is aligned with the expert's response. we computed a responselevel gpt-4o score by computing the percentage of aligned hrs over total number of hrs. these two approaches ensured robustness by covering both a continuous alignment score and a binary gpt-4o alignment label. we conducted another round of human evaluation for the gpt-4o score, where four annotators annotated 40 responses, achieving a 95% correlation with gpt-4o's score and reasoning. prompts for llm-based tasks are presented in to operationalize the quantification of actionability alignment, we prompted the gpt-4o model using carefully selected in-context learning examples and chain-of-thought prompting. the gpt-4o model considers the adr post made by the user and assigns a binary label to each harm reductions strategy based on whether or not the target sub-dimension of actionability is present in the strategy (0: absent; 1: present). to validate the labels assigned by the gpt-4o model, the medical experts reviewed the rationales generated for detecting each of the sub-dimensions of actionability in 100 harm reduction strategies, and agreed with 91 of them for practicality, 94 for relevance, 82 and 89 for specificity and clarity, respectively. overall, the extent of the agreement between experts and gpt-4o rationales reinforced the validity of the labels assigned to the 4 sub-dimensions of actionability. following this, for responses generated by the llms, we computed the fraction of harm reduction strategies that are aligned with the hrs and also demonstrate presence of a certain subdimension of actionability. for instance, for the practicality dimension, the llm-generated hrs are scored as: it is worth emphasizing that the constraint of only considering aligned hrs within the llmgenerated responses enforces a penalty for generating unaligned hrs while computing actionability. since expert responses are inherently always aligned, their hrs do not undergo such a penalization. we present the average scores for the 4 sub-dimensions and their aggregate as the overall actionability score in ## conclusion and future work in this work, we proposed the psych-adr benchmark and adra framework for evaluating the alignment of llms with experts on responding adr queries caused due to psychiatric medication use. in our rq1 analysis, even the best models achieved only 77.41% accuracy in detecting adr and 74.44% accuracy in detecting the type of adr. our rq2 analysis further revealed that while models align with experts on expressed emotions and tone of the text, they struggle in important areas like readability, alignment of harm reduction strategies with expert knowledge, and suggesting actionable interventions. our work can inspire future work to adopt a more holistic approach for evaluating models, emphasizing the integration of "lived experience" alongside expert knowledge.