Identifying Medical Self-Disclosure in Online Communities (2021)

Abstract: Self-disclosure in online health conversations may offer a host of benefits, including earlier detection and treatment of medical issues that may have otherwise gone unaddressed. However, research analyzing medical self-disclosure in online communities is limited. We address this shortcoming by introducing a new dataset of health-related posts collected from online social platforms, categorized into three groups (No Self-Disclosure, Possible Self-Disclosure, and Clear Self-Disclosure) with high inter-annotator agreement (_k_=0.88). We make this data available to the research community. We also release a predictive model trained on this dataset that achieves an accuracy of 81.02%, establishing a strong performance benchmark for this task.

## introduction self-disclosure is a communicative act that helps people develop close relationships @xcite through reciprocal sharing of personal information, promoting maintenance of trust and security @xcite . it is defined as the "process of making the self known to others" @xcite , often by sharing one's personal thoughts, opinions, or experiences. for example: in addition to facilitating social bonds, selfdisclosure in general produces a wide variety of health benefits and plays a critical role in successful treatment of many physical and psychological * authors contributed equally. first, we establish the novel task of medical selfdisclosure detection, and create a 6,639-instance dataset comprised of public online social posts covering a wide range of mental and physical health issues, annotated with graded (no self-disclosure, possible self-disclosure, and clear self-disclosure) labels. we release this dataset to the research community to facilitate easy replication of our work, as well as rapid entry to this new task by others. next, we compare a suite of classical machine learning and neural network approaches (including lstm-, cnn-, and transformer-based models) for this task, finding that neural approaches typically outperform classical machine learning models. our highestperforming model, a bert-based model fine-tuned for the medical self-disclosure task, achieves an accuracy of 81.02%, establishing a strong performance benchmark for this novel task. finally, we find that our highest-performing model outperforms the best existing (general) categorical self-disclosure model @xcite , retrained on our new med-ical self-disclosure dataset and fine-tuned for this task, by relative percentage increases of 41.81%, 32.63%, 66.60%, and 49.76% for accuracy, precision, recall, and f1-measure, respectively. this provides empirical support that detecting medical self-disclosure is a distinct task with unique linguistic nuances, making it impractical to simply apply existing non-medical self-disclosure models to the medical domain with expectations of similarly high performance. in the long term, it is our hope that high-performing medical self-disclosure models can be deployed in clinical settings to support overburdened healthcare workers in understanding, diagnosing, and treating patients' health issues. ## related work self-disclosure detection has been the focus of prior work in psychology @xcite @xcite and computer science @xcite @xcite . however, research examining self-disclosure in online health discourse specifically has been limited. existing work in this domain shows that detecting self-disclosure in the areas of health and wellness can be beneficial @xcite , with patients often preferring to engage in interviews with computers rather than humans and also providing more candid and honest answers to computers @xcite . thus, detecting illness may be an easier process when taking into account patients' virtual disclosures @xcite . in fact, @xcite relied on self-reported diagnosis when examining linguistic trends in a wide range of mental health conditions on twitter. 1 most computational work on self-disclosure detection has taken place in the general domain, and specifically on tweets. @xcite presented a computational framework for automatically detecting self-disclosure using text mining techniques applied to twitter conversations, and walton and @xcite investigated the roles of gender and social identities and their influences on self-disclosure on twitter by adult users. outside of twitter, @xcite also focused on detecting selfdisclosure in news commentaries using dependency parsing and named entity recognition. while these studies involve social posts, they do not specifically focus on health. @xcite presented a simple neural network with three classes (no sd, low sd, and high sd) to predict self-disclosure of mental wellness in reddit foot_1 posts. their highestperforming approach, a perceptron-based model, achieved an accuracy of 78.4%. balani and de choudhury's work is the closest existing work to ours; however, although mental wellness may be a significant interest when identifying self-disclosure in health domains, limiting work to this precludes other critical health concerns such as psychosomatic @xcite or physical ailments. we address the limitations of prior work in automated self-disclosure detection by including an extensive range of mental and physical health concerns in our dataset. like balani and de choudhury (2015), we consider three self-disclosure categories (in contrast to, e.g., the two classes employed by @xcite ). this facilitates a more precise prediction, and focusing on medical self-disclosure specifically helps to (a) validate the distinction between medical and other types of self-disclosure when building automated models for the task, and (b) develop techniques attuned to the latter. ## methods to demonstrate efficacy and learnability of our dataset, we created a suite of classification models for comparative analysis. this offered the parallel opportunity to identify a strong performance benchmark for this task. we describe our preprocessing techniques and modeling algorithms below. ## evaluation we evaluated the performance of all models using accuracy, precision, recall, and f1-measure, following prior work on self-disclosure detection @xcite . we provide the results from three separate experiments in the following subsections. in §5.1, we compare performance between the binary and multinomial classification settings. in §5.2, we compare performance between our svm, nb, lstm, blstm, 1d-cnn, and distilbert models for the multiclass setting. finally, in §5.3, we provide external validation for our highest-performing multinomial model by comparing it to the baseline and balani and de choudhury's highest-performing model. ## discussion although balani and de choudhury's model worked well for their setting, we found that it did not transfer well to our task. it may be that detecting medical self-disclosure inherently carries extra levels of complexity. for example, identifying firstperson pronouns could be a decisive indicator of general self-disclosure, whereas for medical selfdisclosure, self-identifiers would also need to be accompanied by medical terms, some of which may be obscure @xcite . to further disentangle the performance of our highest-performing model, we computed the number of true positives for each class separately, shown alongside per-class accuracy in to develop a further understanding of the linguistic patterns associated with clear self-disclosure, possible self-disclosure, and no self-disclosure instances, we computed the log odds ratio with an informative dirichlet prior @xcite for words in these classes to assess which words were most strongly correlated with each, and plot them in figures 1 , 2 , and 3. the plots support our hypotheses. the words most closely associated with possible self-disclosure have much lower ratios in general than the words most closely associated with clear self-disclosure or no self-disclosure, suggesting that this class is characterized by fewer strong cues indicating membership. furthermore, while the words closely associated with clear self-disclosure are a mix of personal pronouns, medical terms, duration, and narrative descriptors, the words most closely associated with possible self-disclosure are mostly about others and family, or being scared and in search of hope and support. words closely associated with no self-disclosure are less personal or narrative, and more indicative of support or general health interest. ## conclusion in this work, we introduced a novel medical selfdisclosure dataset containing 6,639 instances collected from public online social platforms. instances in this dataset are triple-annotated with high inter-annotator agreement (κ=0.88) for no self-disclosure, possible self-disclosure, and clear self-disclosure. we evaluated a wide range of classical machine learning and neural classifiers (including lstm-, cnn-, and transformer-based models) to assess their efficacy at learning to predict medical self-disclosure. we examined both a simpler binary classification setting and a more challenging multinomial setting, finding that the highest-performing model in both cases was a fine-tuned distilbert model. we compared our best-performing model to the best existing categorical model for self-disclosure detection @xcite , finding that our model outperformed that model by a wide margin for the task of detecting medical self-disclosure (relative percent increases of 41.84% and 49.76% for accuracy and f1-measure, respectively). our findings pave the way for subsequent experiments with other models, moving the dial a necessary step forward by establishing a strong performance benchmark. in the future, we hope to explore medical self-disclosure in the context of goaloriented dialogue systems, resulting in downstream benefits for both physicians and patients. we make our dataset available to interested researchers to foster further progress on this emerging research task.