AI in mental health or ‘choose your adventure’ self-help books?
Tags: Emotional Intelligence, AI, Therapy, Mobile App Development, Mental Health

by Rashmi Rogers

There is much evidence that chat bots built, with their varying levels of artificial intelligence, struggle to demonstrate therapeutic effectiveness in mental health. Study after study, all in the last two years, go on to provide less than stellar reviews, such as:

There is an apparent paucity of published evidence of the effectiveness of chatbots (4).

So far, the research conducted on chatbots does not allow for strong conclusions about the usability and efficacy of mental health chatbots or their outcomes (3).

Chatbots’ limited capacity to re-create human interactions and to offer tailored treatment, combined with the currently lack of access to mental health services in real time, raise the question of whether chatbots could do harm to users (4).

While the overall research in the field remains limited, a 2019 study of the efficacy of smartphone apps as standalone psychological interventions found they “cannot be recommended based on the current level of evidence” (2).

Dr. Darcy, founder of Woebot correctly adds a disclaimer for the chatbot “As smart as I may seem, I’m not capable of really understanding what you need,” and likens the program to a “choose your adventure self-help book.”

Yet, apps like Wysa, Replika, Joy, Youper, Woebot and Tess (the list goes on) are seeing thousands of downloads, high ratings on all stores and glowing reviews from users. So, even a conversation that is stunted, lacking in context and unnatural, is well-received by the population that is engaging with it.

“When you look at the analytics and engagement rates with chatbots, users actually seem to like and stick with them, [which is notable] because in the world of digital mental health products, engagement is notoriously poor.” Notes clinical research psychologist Stephen Schueller of One Mind PsyberGuide.

From a technology standpoint, chatbots, similar to other conversation agents (like Siri and Alexa), provide a ‘more natural medium’ for communication. They process all text using natural language processing modules and provide the user with guidance and responses. The first such attempt was made in 1966 by Dr. Weizenbaum, at MIT, when he created an empathetic conversation agent, ELIZA. For example, the user would say, “He says I’m depressed much of the time”, to which ELIZA would respond, “I am sorry to hear you are depressed.” Dr. Weizenbaum was uncomfortable with how users seemed to develop a strong personal connection with ELIZA and went to become a vocal critic of AI. But the idea of anthropomorphizing a seemingly intelligent bot was here to stay.

On one end of the spectrum of intelligent chat agents, Woebot uses minimal deep learning and provides scripted answers, which leads the user through a series of self-help questions. These CBT modules have great merit and if the user chooses to engage with the app, and they can provide meaningful change. But these are crafted by humans (a series of questions and modules) and do not involve any machine learning. The only time AI is called into action is when deciphering free text from the user, which is also limited in its capability. The chatbot is not able to have natural conversations or grasp the nuances of users’ life history and current circumstances that may be at the root of mental health difficulties.

Replika, on the other end of the spectrum, uses neural networks to learn from each interaction and become more like its user over time. But its ability to ‘learn’ and become more like a user comes with the trade-off that it quickly loses the ability to engage in any therapeutic techniques, making it simply a digital friend.

All other apps lie somewhere in between on this spectrum, with their ability to have conversations with the human is limited by the complexity and context of the user’s text. Most are based on hand-crafted rules, that resemble a medical expert system. There is on-going research to build supervised learning models to understand thinking errors, emotions and situations (in a CBT context), and provide a more relevant chatbot (4). We are in the early stages of truly using artificial intelligence in the context of mental health digital agents.

We turn our attention to the user again. Ideally, efficacy of an intervention system should align well with usability and user experience. Especially in mental health, where outcomes of treatment rely so heavily on engagement, it is critically important that the perceived benefit of an intervention is tied back to its scientific therapeutic basis. The most well-established research lies in decades of evidence behind diagnoses and therapeutics of rich face-to -face conversations, engaged with a person well-qualified in this field. There is no conclusive, far-reaching, or controlled study that provides evidence of effectiveness of digital mental health apps or chat bots in reducing depression, anxiety, worry (and other illnesses) in a heterogenous population.

The high touch, high tech model can provide such a confluence, at least for now. Keeping a qualified therapist in the loop, while increasing the user interaction with chat bots and digital mental health apps, may be one way to increase engagement in a meaningful way. As we are transitioning into a society that is boldly de-stigmatizing mental wellness, we are also coming to terms with our limited capabilities to deal with it. I imagine a system based on the best of evidence-based practices with strong support from therapeutic technology platforms.

In a way, it becomes a continuum of support where we become at better at recognizing our symptoms and are motivated to use tools to take care of ourselves. Imagine a chat bot that is like Replika but with the therapeutic knowledge of Woebot. Such a system would be able to provide individualized and meaningful support and at the same time engage with a therapist as needed. It is important to recognize the current shortcomings of technology in the mental health space so we do not blindly place our faith in a method that is in its infancy.

The limitations may not be from our lack of understanding technology — deep learning is successfully used in many industries. Rather it comes from the relatively new application space of mental health, where we are just beginning to scratch the surface of efficacy, usability, engagement, and most importantly, ethics. Keeping the experts in the care loop can potentially be our best path forward.

References

1. Weisel, K.K., Fuhrmann, L.M., Berking, M. et al. Standalone smartphone apps for mental health — a systematic review and meta-analysis. npj Digit. Med. 2, 118 (2019) (https://doi.org/10.1038/s41746-019-0188-8)

2. https://gizmodo.com/your-ai-chatbot-therapist-isn-t-sure-what-it-s-doing-1843608786

3. Dosovitsky G, Pineda BS, Jacobson NC, Chang C, Escoredo M, Bunge EL. Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage. JMIR Form Res. 2020;4(11):e17065. Published 2020 Nov 13. doi:10.2196/17065 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7695525/)

4. Kira Kretzschmar, Holly Tyroll, Gabriela Pavarini , Arianna Manzini and Ilina Singh (2018). Can Your Phone Be Your Therapist? Young People’s Ethical Perspectives on the Use of Fully Automated Conversational Agents (Chatbots) in Mental Health Support (https://journals.sagepub.com/doi/pdf/10.1177/1178222619829083)

5. Rojas-Barahona, Lina & Tseng, Bo-Hsiang & Dai, Yinpei & Mansfield, Clare & Ramadan, Osman & Ultes, Stefan & Crawford, Michael & Gasic, Milica. (2018). Deep learning for language understanding of mental health concepts derived from Cognitive Behavioural Therapy. (https://www.aclweb.org/anthology/W18-5606.pdf)