# Classifying Sensitive Issues for Patients with Neurodevelopmental Disorders

Torben Wallbaum, Tim Claudius Stratmann, Susanne Boll

► To cite this version:

Torben Wallbaum, Tim Claudius Stratmann, Susanne Boll. Classifying Sensitive Issues for Patients with Neurodevelopmental Disorders. 17th IFIP Conference on Human-Computer Interaction (IN-TERACT), Sep 2019, Paphos, Cyprus. pp.107-114, 10.1007/978-3-030-29381-9_7. hal-02544534

HAL Id: hal-02544534

https://inria.hal.science/hal-02544534

Submitted on 16 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Classifying Sensitive Issues for Patients with Neurodevelopmental Disorders

Torben Wallbaum<fnref n="1" />, Tim Claudius Stratmann<fnref n="1" />, and Susanne Boll<fnref n="2" />

**Abstract.** ADHD has an estimated worldwide prevalence of 2-3 % and is one of the most frequent neurodevelopmental disorders. Many problems in an ADHD-patient's life arise from the lack of self-management abilities and social interaction with others. While medication is considered the most successful treatment for disorders such as ADHD, patients often seek support in therapeutic sessions with trained therapists. These aim to strengthen self-awareness of symptoms, emotional self-regulation, and self-management. However, sharing personal insights can be a burden for patients while therapists would benefit from understanding important issues a patient is facing. Our work aims to support therapy for patients and therapists by providing classification of digital diary entries for therapy sessions while protecting patients privacy. Additionally, we provide insights into important issues and topics including their affective interpretation for patients suffering from ADHD.

**Keywords:** ADHD · Social communities · Classification · Text analysis

## 1 Introduction & Prior Work

One of the most commonly diagnosed psychiatric disorders for children as well as adolescents is ADHD (Attention-deficit-hyperactivity disorder). The prevalence of ADHD in adults is around 2-3% [20], often starts in childhood and persists in up to 50% into adulthood. In recent years an increase in diagnosis, as well as medication, can be found in multiple sources (e.g., [5,19]). Core symptoms of the disorder are inattention, hyperactivity, and impulsivity, however patients suffering from various combinations of these and further symptoms. A common instrument in ADHD diagnosis and therapy are self-reporting scales (such as ADHD Rating Scale-IV [12]), which are assessing behavioral patterns that are considered ADHD risk factors. The use of such self-reporting questionnaires is often criticized, due to the lack of a missing specific situational context. Patients need to imagine and predict their own behavior for situations described within these assessments. Because of a missing self-awareness of their own reaction, patients may have difficulties in evaluating oneself objectively or may differ in their ability for introspection. Previous works have suggested that additional behavioral data could allow for more objective measures.

<footnote n="1">¹ OFFIS - Institute for Information Technology, Oldenburg, Germany</footnote>

<footnote n="2">² University of Oldenburg, Oldenburg, Germany</footnote>

<footnote>`{torben.wallbaum,tim.stratmann}@offis.de`</footnote>

<footnote>susanne.boll@uni-oldenburg.de</footnote>
In previous works Sonne et al. have suggested a framework to support therapy with technological on-body systems [17]. Based on this, concepts and systems have been presented which support self-management of psychological disorders for young adults unobtrusively in everyday life. Sensing modules are used to detect symptoms that are relevant by using unobtrusive sensors that continuously collect data about a person's movements [13,18], heart-rate variability [15,7], eye-movements [10], contextual parameters e.g., body temperature [4] and in-situ experience sampling [11]. Current moods and self-evaluations are queried through a mobile device in appropriate moments.

One other successful measure is the use of retrospective or emotional diaries, that provide patients a save space to collect and reflect experiences from their day-to-day life. This method is used often by therapists when applying techniques from cognitive behavioral therapy (CBT) [3]. Essentially, cognitive behavioral therapy aims to change behavior by identifying negative and distorted thinking patterns. These diaries provide a tool for monitoring feelings of anxiety, fear, hurt, anger, shame, guilt, or sadness as well as when and where these feelings were experienced. This successful form of therapy emphasizes the link between thoughts, feelings, and behavior. However, some patients might feel uncomfortable to later share their experiences with therapists during sessions due to privacy reasons and therefore miss the chance to get helpful feedback or learn helpful strategies specific to their own personal circumstances. At the same time therapists could benefit from these inputs to better understand common issues and links between subjects, involved people, and emotional interpretations. By gaining knowledge from these experiences, therapists can improve the quality of their therapy over time.

To address these shortcomings of traditional diaries, we propose a support system that aims to classify digital diary entries using text-analysis and detect the overarching topic of a diary entry. In addition, our approach performs a sentiment analysis of each diary entry to understand how a patient emotionally rates specific topics personally e.g. social interactions are perceived negatively. The privacy conserving overarching topics based on a patient's personal experiences might be shared with therapists, to provide a basis for future therapy sessions and indicate changes in topics important to patients for long-term self-reflection. Patients can decide if or when to share diary entries with therapists. If only privacy conserving keywords and sentiments are shared, therapists can use them as a conversation starter during which more details might be revealed by the patient. The reuse of diary entries in therapy session might additionally encourage patients to keep engaged in regularly documenting important things, emotional states and reflections of their day-to-day experiences.

With this work, we aim for two main goals: (1) Discover and understand important issues and topics including their affective interpretation for patients suffering from ADHD or similar neurodevelopmental disorders. (2) Support therapy for patients and therapists by providing in-situ classification of digital diary entries collected to provide important topics for therapy sessions. To understand important topics and gather create a dataset for training a classification model,
we collected posts from an active online community. Following we describe our
approach.

2 Crowdsourcing Topics for Classification

2.1 Data Mining & Analysis

We collected our initial dataset from the ADHD Subreddit<sup>3</sup>, which is an active
community, of at that time about 221,382 users, to share and discuss topics
related to ADHD. We collected the 1000 TOP posts of ALL times (upvoted
by users of the community). After cleaning up and removing duplicates, the
dataset resulted in 998 posts from 823 different authors. We choose the ADHD
Subreddit community because of two main attributes: First, similar to diary
entries, posts vary in length and grade of detail and second, users often share
important subjects with the community in a way they would collect them in a
diary, such as:

[..]I feel nervous or numb mostly and can't think clearly most of the time. I don't particularly feel like a pleasant human being and I have quite a disheveled past. Luckily nothing too horrible but just many many SO many fuck ups and experiences and broken relationships that feel like a weight on my back. [..]

We first aimed to retrieve overarching topics from the dataset itself. Based on initial topics from a literature review e.g. problems at work or with loved ones, we used techniques from NLP to understand the data corpus. We analyzed the existing overall text corpus with regard to word frequencies, clustering as well as term correlations. After cleaning the dataset and removing stop words (english and custom ADHD-related ones), we used stemming and removed white spaces. Based on the corpus we created a tf-idf weighted document-term-matrix ([14]) with 10% sparse terms removed as basis for further analysis. While we analyzed the corpus with different clustering algorithms (hierarchical clustering, K-means), we could not find any particular interesting correlations. However, we will continue our analysis in the near future, in cooperation with experts from the medical domain to further search for extending topics of interest.

## 2.2 Classification of Posts

We included three experts (therapists, medical professionals) to help us identify
general important topics and issues that often or regularly occur during therapy
sessions with patients and are known from professional literature. Together with
the experts, we identified 14 topics (listed in Figure 2a for the following three
overarching areas: *ADHD-related*, *Work*, *Private Life / Social Skills*. The topics
for relationships and planning occur in two areas, for private and professional
life accordingly.

<footnote n="3">³ (https://www.reddit.com/r/ADHD/; last retrieved: 06-28-2018)</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1: Snapshot of the crowdsourcing website, to categorize social media postings into one of fifteen overarching topics.</figcaption></figure>

In a next step, we built a website to crowdsource the mappings of overarching topics for each entry of our dataset to retrieve labels for a later model training (shown in Figure 1). The website was accessible for crowd-workers hired through a crowdsourcing service (We hired 100 crowd-workers through https://www. prolific.ac/ with an incentive of about $8/hour) Workers were presented one post at a time, which they had to read and choose an appropriate overarching topic for. Each categorization tasks consisted of 10 posts and took about 15-30 min per worker. We asked workers to categorize by 1 in 14 categorize or select *other* if none of the available options seemed appropriate. Figure 2a shows the frequencies of topics identified as a result of the crowdsourcing task for our dataset.

# 3 Analysis & Model

## 3.1 Sentiment-Analysis of Topics

To understand affective interpretation of topics, we used sentiment analysis for our dataset. Each post (categorized into one of the overarching topics by the crowd-workers) was analyzed using the VADER Sentiment Analysis<fnref n="4" />. VADER is a lexicon and rule-based sentiment tool, specifically trained to classify sentiments expressed in social media [6]. It was already successfully used for similar data in

<footnote n="4">⁴ (https://github.com/cjhutto/vaderSentiment; last retrieved 01-07-2018)</footnote>
related research [1,16]. The resulting overall distribution of sentiments (53.9% positive, 13.1% neutral, 33.0% negative) is shown in Figure 2b. The median associated sentiment for each separate topic is represented in Figure 3. For six topics the median classification is neutral, nine correspond to positive, none of the topics is mostly associated with negative feelings. Similar to a bias towards a positive sentiment of the posts from Reddit, we expect a positive bias in later diary entries as these are a way to get help by outsiders (either other users or a therapist using the diaries).

<table><thead><tr><th>Topic</th><th>Count</th></tr></thead><tbody><tr><td>comorbidity</td><td>15</td></tr><tr><td>education</td><td>98</td></tr><tr><td>emotions</td><td>104</td></tr><tr><td>focus</td><td>147</td></tr><tr><td>hyperactivity</td><td>49</td></tr><tr><td>inbodyment</td><td>85</td></tr><tr><td>interests</td><td>40</td></tr><tr><td>medication</td><td>82</td></tr><tr><td>planning_private</td><td>74</td></tr><tr><td>planning_work</td><td>43</td></tr><tr><td>relationship_private</td><td>79</td></tr><tr><td>relationship_work</td><td>10</td></tr><tr><td>rewards</td><td>22</td></tr><tr><td>therapy</td><td>40</td></tr><tr><td>other</td><td>110</td></tr><tr><td>Total</td><td>998</td></tr></tbody></table>

(a) Distribution of topics.

<figure><img src="image_3.png" /></figure>

(b) Distribution of sentiments.

<caption>Fig. 2: Distribution of topics classified from our sourced reddit postings and distribution of sentiments (positive, neutral, negative) over all analyzed posts.</caption>

## 3.2 Model-Training for Classification

For training the model to classify in-situ diary entries from patients into over-arching topics we used a supervised learning approach. As a basis, we used our existing dataset including 9,647 words and the fifteen topic labels created by the crowd-workers. Following we used the FastText [9] library to train our classification model as it proofed to be well suited in related research [2,8]. We trained our model for 500 epochs with a learning rate of 0.5. For testing the trained model we used 20% of our dataset per topic for testing (to prevent underrepresented topics). As we are currently implementing the mobile application for in-situ diaries, we have not yet evaluated the overall performance of the model with regard to classification accuracy. However, our first tests showed a precision of 0.215 as well as a recall of 0.215 (performs better than pure chance=0.06). While the classification accuracy can be further improved e.g. by using n-grams to set words into context, we see first promising results in our current results.
<figure><img src="image_4.png" /><figcaption>Fig. 3: Overarching topics and their associated sentiment (entries colored in green represent a positive sentiment, while yellow related to neutral).</figcaption></figure>

# 4 Next Steps & Research Agenda

Our next step is to experiment further with learning approaches for diary entry classification. Furthermore, we are currently completing the development of the first digital diary prototype for Android, while also planning a field test with patients. In this field evaluation, we want to test our Application (see digital diary for patients Figure 4a and overview screen including classified entries with sentiment rating for therapists Figure 4b) against a baseline *analog* diary during therapy sessions. We aim to better understand (1) how patients and therapists can be supported during therapy sessions, (2) how good our approach performs
in comparison with existing techniques and (3) how we can improve our classification model to be further generalized for real-life applications with patients suffering from other neurodevelopmental disorders.

<figure><img src="image_5.png" /><figcaption>Fig. 4: Using a smartphone application, the patients are enabled to keep a diary. Each entry can be hidden from the therapist, if the patient is choosing to do so. Therapists can see diary entries for each patient individually. Each entry is presented including a) keywords, b) a color coded sentiment (red=negative, green=positive, grey=neutral) as well as the entries text, if not hidden by the patient.</figcaption></figure>

## References

1. Althoff, T., Clark, K., Leskovec, J.: Large-scale analysis of counseling conversations: An application of natural language processing to mental health. Transactions of the Association for Computational Linguistics 4, 463–476 (2016). https://doi.org/10.1162/tacl_a_00111, https://doi.org/10.1162/tacl_a_00111

2. Badjatiya, P., Gupta, S., Gupta, M., Varma, V.: Deep learning for hate speech detection in tweets. In: Proceedings of the 26th International Conference on World Wide Web Companion. pp. 759–760. WWW ’17 Companion, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2017). https://doi.org/10.1145/3041021.3054223, https://doi.org/10.1145/3041021.3054223

3. Beck, J.S.: Cognitive behavior therapy: Basics and beyond. Guilford press (2011)

4. Bijlenga, D., van der Heijden, K.B., Breuk, M., van Someren, E.J., Lie, M.E., Boonstra, A.M., Swaab, H.J., Kooij, J.S.: Associations between sleep characteristics, seasonal depressive symptoms, lifestyle, and adhd symptoms in adults. Journal of Attention Disorders **17**(3), 261–275 (2013)

5. Brault, M.C., Lacourse, É.: Prevalence of prescribed attention-deficit hyperactivity disorder medications and diagnosis among canadian preschoolers and school-age children: 1994–2007. The Canadian Journal of Psychiatry **57**(2), 93–101 (2012)
6. Gilbert, C.H.E.: Vader: A parsimonious rule-based model for sentiment analysis of social media text. In: Eighth International Conference on Weblogs and Social Media (ICWSM-14). Available at (20/04/16) http://comp. social. gatech. edu/papers/icwsm14. vader. hutto. pdf (2014)

7. Griffiths, K.R., Quintana, D.S., Hermens, D.F., Spooner, C., Tsang, T.W., Clarke, S., Kohn, M.R.: Sustained attention and heart rate variability in children and adolescents with adhd. Biological psychology **124**, 11–20 (2017)

8. Jha, A., Mamidi, R.: When does a compliment become sexist? analysis and classification of ambivalent sexism using twitter data. In: Proceedings of the Second Workshop on NLP and Computational Social Science. pp. 7–16. Association for Computational Linguistics, Vancouver, Canada (Aug 2017). https://doi.org/10.18653/v1/W17-2902, https://www.aclweb.org/anthology/W17-2902

9. Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 (2016)

10. Matsuo, Y., Watanabe, M., Taniike, M., Mohri, I., Kobashi, S., Tachibana, M., Kobayashi, Y., Kitamura, Y.: Gap effect abnormalities during a visually guided pro-saccade task in children with attention deficit hyperactivity disorder. PloS one **10**(5), e0125573 (2015)

11. Murphy, P., Schachar, R.: Use of self-ratings in the assessment of symptoms of attention deficit hyperactivity disorder in adults. American Journal of Psychiatry **157**(7), 1156–1159 (2000)

12. Pappas, D.: Adhd rating scale-iv: Checklists, norms, and clinical interpretation. Journal of psychoeducational assessment **24**(2), 172–178 (2006)

13. Pina, L., Rowan, K., Roseway, A., Johns, P., Hayes, G.R., Czerwinski, M.: In situ cues for adhd parenting strategies using mobile technology. In: Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare. pp. 17–24. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering) (2014)

14. Ramos, J., et al.: Using tf-idf to determine word relevance in document queries. In: Proceedings of the first instructional conference on machine learning. vol. 242, pp. 133–142 (2003)

15. Rukmani, M.R., Seshadri, S.P., Thennarasu, K., Raju, T.R., Sathyaprabha, T.N.: Heart rate variability in children with attention-deficit/hyperactivity disorder: a pilot study. Annals of neurosciences **23**(2), 81–88 (2016)

16. Song, K., Feng, S., Gao, W., Wang, D., Yu, G., Wong, K.F.: Personalized sentiment classification based on latent individuality of microblog users. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)

17. Sonne, T., Marshall, P., Obel, C., Thomsen, P.H., Grønbæk, K.: An assistive technology design framework for adhd. In: Proceedings of the 28th Australian Conference on Computer-Human Interaction. pp. 60–70. ACM (2016)

18. Sonne, T., Obel, C., Grønbæk, K.: Designing real time assistive technologies: A study of children with adhd. In: Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction. pp. 34–38. ACM (2015)

19. Toh, S.: Datapoints: Trends in adhd and stimulant use among children, 1993-2003. Psychiatric Services **57**(8), 1091–1091 (2006)

20. de Zwaan, M., Gruß, B., Müller, A., Graap, H., Martin, A., Glaesmer, H., Hilbert, A., Philipsen, A.: The estimated prevalence and correlates of adult adhd in a german community sample. European Archives of Psychiatry and Clinical Neuroscience **262**(1), 79–86 (2012)