W ell D unn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions (2024)

Abstract: Language Models (LMs) are being proposed for mental health applications where the heightened risk of adverse outcomes means predictive performance may not be a sufficient litmus test of a model’s utility in clinical practice. A model that can be trusted for practice should have a correspondence between explanation and clinical determination, yet no prior research has examined the attention fidelity of these models and their effect on ground truth explanations. We introduce an evaluation design that focuses on the robustness and explainability of LMs in identifying Wellness Dimensions (WDs). We focus on two existing mental health and well-being datasets: (a) Multi-label Classification-based MultiWD, and (b) WellXplain for evaluating attention mechanism veracity against expert-labeled explanations. The labels are based on Halbert Dunn’s theory of wellness, which gives grounding to our evaluation. We reveal four surprising results about LMs/LLMs: (1) Despite their human-like capabilities, GPT-3.5/4 lag behind RoBERTa, and MedAlpaca, a fine-tuned LLM on WellXplain fails to deliver any remarkable improvements in performance or explanations. (2) Re-examining LMs’ predictions based on a confidence-oriented loss function reveals a significant performance drop. (3) Across all LMs/LLMs, the alignment between attention and explanations remains low, with LLMs scoring a dismal 0.0. (4) Most mental health-specific LMs/LLMs overlook domain-specific knowledge and undervalue explanations, causing these discrepancies. This study highlights the need for further research into their consistency and explanations in mental health and well-being.

## introduction according to the national institute of mental health @xcite , over 20% of us adults have experienced mental illnesses, prompting the government to allocate $280 billion to address unmet mental health service needs @xcite . this highlighted the need to leverage ai (particularly lms/llms) for mental health, as they can potentially decrease costs and increase the accessibility of mental health services. however, vigilance is crucial regarding the potential risk of lms/llms arising from low-confidence predictions and correct predictions with wrong explanations. motivated by this longer-term goal of safe deployment of nlp-based mental health systems, we propose evaluation schemes examining the consistency in lm's attention (and llm's attention where the attention is accessible) with ground-truth explanations foot_0 and confidence in predictions. our insight is that a model's attention in disagreement with physician assessment is unlikely to be accepted, regardless of predictive accuracy. indeed, such a scenario implies the model has learned some shortcut or correlative signal instead. we present an evaluation framework, acronymized as welldunn, which examfine-tuning of general purpose and domain-specific lms for extracting data representations, followed by (2) feeding them into a feed-forward neural network classifier (ffnn). two loss functions assess lms' robustness: sigmoid cross-entropy(sce) and gambler's loss(gl). singular value decomposition (svd) and attention-overlap (ao) score assess the explainability. in: input, and out: output. welldunn benchmarking box: this middle rectangle highlights the components of the benchmark system, which includes steps of (1) fine-tuning and (2) ffnn classifier, as well as robustness and explainability components. the left and right dotted rectangles grouped the components for the multiwd and wellxplain tasks, respectively. in the case of task 1, the input (text post) is fed into the multiwd task, and the model produces an output (prediction) in terms of various wds like pa, ia, sa, etc. for task 2, the input (text post) is also fed into the wellxplain task, which produces output (prediction) along with corresponding explanations. note that in the instruction (training), we provide both input and output, but in the evaluation (test), we provide the input. ## welldunn: methodology and evaluation domain-specific and general-purpose lms: we consider two distinct categories of models for the task of wd identification -general models and domain-specific models. the general models under consideration include bert @xcite , roberta @xcite , xlnet @xcite , and ernie @xcite . additionally, we incorporate domain-specific models, namely clinicalbert @xcite , mental-bert @xcite , and psychbert @xcite , to further explore their applicability within the mental health domain. ## experiments, results, and analysis we employed the general architecture, depicted in figure 2 , consisting of two crucial steps applicable to four general and three domain-specific models. step 1: we independently utilize each of the seven models to extract a representation for the input data. step 2: this representation is fed into a fully connected neural network classifier, which determines the aspect or dimension of the input. ## conclusion and future work welldunn introduces a demanding pair of datasets for the ai for social impact community working on mental health. through thorough benchmarking on domain-specific and generalpurpose lms, we've highlighted the disparities between prediction accuracy and attention, underscoring the need for a transparent classifier rooted in clinical understanding. second, despite the expectation that gambler's loss would enhance performance by avoiding predictions for low-confidence samples, we observed a significant drop in performance for the multiwd dataset. third, the ao scores show that attention explanations are not closely aligned with the ground truth. further experiments were conducted to thoroughly analyze the datasets and confirm these findings refer to