Measuring the Language of Self-Disclosure across Corpora (2022)

Abstract: Being able to reliably estimate self-disclosure – a key component of friendship and intimacy – from language is important for many psychology studies. We build single-task models on five self-disclosure corpora, but find that these models generalize poorly; the within-domain accuracy of predicted message-level self-disclosure of the best-performing model (mean Pearson’s r=0.69) is much higher than the respective across data set accuracy (mean Pearson’s r=0.32), due to both variations in the corpora (e.g., medical vs. general topics) and labeling instructions (target variables: self-disclosure, emotional disclosure, intimacy). However, some lexical features, such as expression of negative emotions and use of first person personal pronouns such as ‘I’ reliably predict self-disclosure across corpora. We develop a multi-task model that yields better results, with an average Pearson’s r of 0.37 for out-of-corpora prediction.

## introduction interpersonal exchanges are a core component in human relationships. they are determined by intimacy, which in turn is characterized by the willingness of the involved parties to self-disclose @xcite . in general, self-disclosure can be defined as "revealing intimate information about one's self" @xcite . note that selfdisclosure, which often involves revealing embarrassing facts about oneself that are considered violations of social norms ("i flunked my exam." or "i have a growth on my butt"), is different from revealing personally identifiable information (pii). selfdisclosure encompasses the sharing of thoughts, aspirations, feelings, likes and dislikes, while pii, such as date of birth or social security number, is used to unambiguously identify a person. unlike self-disclosing, sharing pii does not necessarily suggest an intimate relationship between two people. figure 1 : two sentences from the med and the emp-con data set. in our work, we predict the associated self-disclosure and assess the most important features in both sentences, which are highlighted in the examples. nlp researchers have labeled a variety of data sets with self-disclosure or some approximation of self-disclosure such as "intimacy", which is more accurately viewed as being a property of the relationship between two people than of an utterance. in this paper, we build models to predict self-disclosure from text, and assess how well these models generalize across five different corpora. we find that they mostly generalize poorly, but that there are some reliable linguistic markers of selfdisclosure. we draw on multiple corpora labeled for selfdisclosure: conversations from an online breast cancer support community @xcite ; annotated conversational turns @xcite ; medical posts from patient.info and reddit @xcite and posts from the r/offmychest and the r/casualconversations subreddits @xcite . the labels on these data sets vary both in terms of how self-disclosure is defined, and in their scaling (e.g., 0/1 or 1-5 likert scales), complicating the analysis. ## background and related work people reveal information about themselves to form and maintain personal relationships @xcite . as an essential part of interpersonal communication, self-disclosure can have positive and negative effects on the person disclosing, which are reinforced in an online environment. risks resulting from revealing private information can encompass a loss of privacy @xcite , a negative impact on identity and self-presentation @xcite , and negative consequences caused by context collapse, i.e. the disclosure to an unintended audience, that is especially prevalent on social media @xcite . on the other hand, disclosing private information can lead to increased social expression, social validation and perceived intrinsic rewards @xcite . self-disclosure can be influenced by a variety of factors including anonymity, cultural norms, personality, loyalty and mutual trust @xcite . these have an impact on the risk/benefit dynamic in revealing personal information online. @xcite have formulated a functional model of self-disclosure to capture these conflicting dynamics and allow for a more holistic understanding of self-disclosure by showing how people try to maximize their benefits when disclosing private information. self-disclosure is a determining factor in the level of intimacy between people. on an individual level, it has been shown that intimate relationships are an important resource for inter-and intrapersonal growth @xcite . they strengthen a person's sense of belonging and self-worth (rawlins, 2017) and provide a source of emotional support as well as a safe space for self-exploration @xcite . through these mechanisms, self-disclosure can positively influence a person's mental health @xcite , improving their feeling of connectedness to others, a primary human need @xcite . for example, @xcite showed that intimate relationships, which are dependent on self-disclosure, lead to better competence, sociability and self-esteem as well as less self-reported depression and anxiousness, compared to reference groups with less intimate connections. the steady rise of social media usage led to an increase in the availability of publicly disclosed 'private' information. this is especially interesting given that self-disclosure has been found to be higher online compared to face-to-face communication @xcite , partially because sharing to larger audiences is facilitated in an online context @xcite . in the light of these developments, we use social networking sites (sns) data to identify and subsequently predict self-disclosure in online posts. in previous works, self-disclosure was predicted in different contexts using unsupervised, semi-supervised and supervised models. @xcite used unsupervised learning to detect the voluntary disclosure of private information in tweets. they investigated how self-disclosure was impacted due to the covid-19 pandemic and found a significant shift towards support-seeking and supportiveness. in addition, @xcite developed a semi-supervised self-disclosure topic model to automatically detect self-disclosure in tweets, with the aim of analyzing its effects on subsequent conversations. they find a significant positive correlation between self-disclosure and conversation length as well as frequency. furthermore, @xcite investigated how publicness influences self-disclosure in health support groups by applying a supervised model based on the linguistic inquiry and word count (liwc) as well as other linguistic features and word embeddings to assess the level of positive and negative self-disclosure. considering the broader concept of intimacy, pei and jurgens (2020) designed a computational framework to study the expression of intimacy in questions. they predicted intimacy using a semi-supervised model, showing that it is an impactful dimension in language that is influenced by social settings. our study differs in that we aim to understand self-disclosure across different platforms and contexts. we contribute to previous efforts (e.g. @xcite ) by focusing on the specific prediction of self-disclosure in order to assess well-being and mental health from social data. as such, we are not limited to one sns but rather aim to develop a supervised model that generalises across multiple platforms. we further compare the performance of roberta-, liwc-, lda-and emolex-based models to show which linguistic features are predictive of self-disclosure. finally, given that we find that single-task models are insufficient, we develop a multi-task model across all available data sets to assess self-disclosure. this is an innovative approach that has not yet been pursued in this realm to the best of our knowledge. ## data sets to develop a general model to detect the degree of self-disclosure in messages, we gathered five data sets, trained models on them, and tested the performance of these models across all data sets. the available data sets offer a challenge in that they all have different labels, including 'self-disclosure', 'intimacy', and 'emotional disclosure'. these labels differ both in the instructions provided to the annotators (there is no consistent definition of selfdisclosure used in computational linguistics) and in their scales. the fact that some labels are binary and others are on 1-to-3, 0-to-5, or 1-to-7 likert scales complicates the analysis. we thus evaluate the accuracy of models by looking at the correlation of the prediction with the true label, allowing us to see e.g. how accurately a prediction of a 1-to-5 label estimates a binary label. ## features each of the above-mentioned data sets have been used to train discriminative, supervised machine learning models to correlate linguistic characteristics with the perceived presence of self-disclosure. in this section, we present the features we took into consideration. ## results we now discuss the quantitative results and their implications. since we found in the analysis that the int data set does not generalize well due to the fact that it only consists of questions, we focus on the four remaining data sets in our analysis and only report the int results in the appendix. ## limitations & ethical considerations several limitations of our study should be taken into account when considering results in a wider context. a key issue in building a general selfdisclosure models was the differing labels based on differing definitions of self-disclosure across the data sets considered. (this is a common problem in computational social science, where constructs such as "happy" or "liberal" are often measured using widely different measures, see @xcite for more information). it needs to be taken into account that we assume in our paper that the different notions of self-disclosure across the in addition, we have not studied how selfdisclosure prediction differs among different cultures, genders and races. specifically, it is unclear how well our recommended general self-disclosure model applies to specific subgroups. for example, women tend to self-disclose more and express more emotional content than men @xcite . whether this suggests that different models of selfdisclosure would be helpful for men and women is less clear. similarly, the amount of self-disclosure varies widely across settings and cultures. how this affects models is similarly unclear. these variations should be studied in a subsequent research project. secondly, our training corpora included mostly native english speakers and hence might not generalize well to non-native speakers. finally, self-disclosure detection could be used for unethical targeting, e.g. in the context of insurance companies who want to discriminate on prices for people who don't self-disclose much, given that self-disclosure can influence relationships and subsequently the mental health of a person. the application of our model for such usages is strongly advised against. ## conclusion self-disclosure is a determining factor of the quality of interpersonal relationships, where closer friendships include more self-disclosure @xcite . furthermore, the amount of self-disclosure on a platform should also strongly affect how much information can be extracted about personality and emotion from language written on that platform; linkedin, for example, should show less self-disclosure than facebook. motivated by these observations, we studied to what extent self-disclosure can be predicted by looking at lexical features. many aspects of language indicate self-disclosure. the expression of negative emotions and the use of first person pronouns are particularly predictive. models trained on different data sets with different annotations of selfdisclosure generalize poorly across corpora. our best performing model, a roberta-based linear multi-task model trained on on all our data sets, is available at https://github.com/tea-shaped/selfdisclosure-model .