Mental State as Mixtures of Emotions
This experiment shows a possible novel approach to create explainable mental state embeddings based on emotions, expanding the conversational model’s role from empathetic response generation to emotion analysis and diagnostic tool. The approach involves leveraging the fine-tuned model from the previous experiment, primarily as an emotion classifier. The idea is to extract emotional embeddings, used as markers that are indicative of specific mental disorders from specialized corpora, in this case from social media interactions in mental health forums. The goal is to later compare the distinctive emotional profiles obtained in this experiment to the profiles obtained from users interacting with the model in a conversation.

For this experiment,
considering the lack of professionally labeled data, various datasets are gathered from Reddit, a social news website and forum where content is socially curated and promoted by site members through voting. It must be acknowledged that the data obtained from Reddit or other social media platforms may not accurately represent the broader population with mental illnesses, as it only captures those who choose to discuss their experiences online.

Reddit is organized into
forums known as “subreddits”. Each subreddit focuses on a specific topic, interest, or theme, creating a unique community within the broader Reddit platform. In Table 3 all the considered subreddits obtained from Low et al. ( 2020 ) are reported (for a more in-depth discussion we refer to Section E in Appendix), together with a graphical visualization in Figures 5 and 6 in Appendix of the relative mental state embeddings based on emotions obtained by processing 1,000 posts from each subreddit.

In the approach described in this section, embeddings for each mental disorders were generated by processing posts from the respective subreddits. The methodology involves the empathetic conversational model obtained in the previous experiment. However, rather than responding with both emotion and a reply, the posts are segmented into individual phrases. For each phrase, the model predicts a set of 10 emotions. These predicted emotions are then aggregated across all posts by summation and subsequently normalized. This process results in a characteristic emotional distribution profile for each mental disorder.

This experiment yielded interesting results: the obtained emotion embeddings show significant differences across a spectrum of Reddit communities. Also, similarities across related disorders were to be expected. For instance, depression and suicide, or addiction and alcoholism,
show consistent similarities. Overall, these distributions can provide insights into how individuals discussing their experiences with similar conditions might perceive and express their feelings. In Figure 4 (A) , we observe the mental disorder representations in a two-dimensional reduced space after applying t-SNE on top of the emotional embeddings. We can observe how mental disorders such as alcoholism , addiction , and eating disorder are close in the embedding space, as are depression and loneliness or schizophrenia and post-traumatic stress disorder (PTSD) . We can also observe interesting properties of our representations, for example, by summing the depression and schizophrenia’s embeddings, a new representation can be obtained, that is very close, in the embedding space, to bipolar.

The assumption that mentally distressed individuals show identifiable skewed patterns of emotions must be addressed, by first establishing a normal distribution for comparison. For this, the Daily Dialogue Dataset Li et al. ( 2017 ) , a high-quality multi-turn open-domain English dialogue dataset, was chosen as a control group. On average there are around 8 speakers turns per dialogue with around 15 tokens per turn where people discuss their daily lives, the whole training set was used to extract the embedding for this dataset.

The order of the emotional features in the embeddings can be arbitrary. But as an example, for clarity and visual comparison, the control group embeddings and the depression embeddings can be ordered according to what are most commonly considered positive and negative emotions: Figure 4 (B-C) clearly shows the contrast in emotional profiles, emphasizing the marked disparities in how emotions are manifested and experienced by those within the reddit depression community, exhibiting an extremely skewed distribution towards negative emotions, compared to individuals engaging in daily dialogues.