Towards Development of an Automated Health Coach (2022)

Abstract: Human health coaching has been established as an effective intervention for improving clients’ health, but it is restricted in scale due to the availability of coaches and finances of the clients. We aim to build a scalable, automated system for physical activity coaching that is similarly grounded in behavior change theories. In this paper, we present our initial steps toward building a flexible system that is capable of carrying out a natural dialogue for goal setting as a health coach would while also offering additional support through just-in-time adaptive interventions. We outline our modular system design and approach to gathering and analyzing data to incrementally implement such a system.

## introduction it is well-known that eating a balanced diet and engaging in regular moderate-to-vigorous physical activity (mvpa), among other healthy behaviors, promotes better health and reduces the risk of cardiovascular disease and other chronic illnesses @xcite . however, people may struggle to develop and integrate healthier behaviors on their own @xcite . health coaching is a behavioral health intervention that is demonstrably effective in improving motivation and confidence and is strongly associated with behavior change @xcite @xcite . health coaches utilize behavioral theories and evidence-based strategies in a clientcentered approach to help clients set goals that are challenging yet achievable, supported by action plans and coping plans that include strategies to overcome barriers like lack of time or poor weather @xcite . as such, goals and dialogue are highly specific and tailored to the client. however, human health coaching is limited by coach availability, cost to potential clients, and the retrospective nature of the feedback @xcite . most attempts to automate this process thus far have been mostly limited to theoretical studies or systems with pre-scripted, non-tailored dialogues, if there is any interactivity at all (op den @xcite @xcite @xcite . while some successfully demonstrate the acceptability of automated systems, even with scripted interaction, the feedback also identifies a user desire for increased tailoring with regards to timing and response to collected user data or context during coaching sessions @xcite . another type of intervention, just-in-time adaptive intervention (jitai), leverages technology to monitor a user's state and deliver support at a time when it is most needed and the user is most receptive to act upon it in the moment @xcite @xcite . for instance, this may include nudges like "walk around the block an extra time" if the user is out for a walk. to our knowledge, no system has yet combined automated, interactive coaching with real-time knowledge of user progress and jitai. to this end, we aim to build an automated system capable of helping clients set achievable goals through interactive discussion and support them in achieving those goals. we focus on physical activity (pa) coaching, but this infrastructure is modular and extendable to other health coaching areas in which goals can be clearly defined or real-time user context leveraged, including stress management or adapting to a prescribed diet. in this paper, we present our preliminary work in building a dialogue and messaging system for an application capable of coaching a user to set and achieve goals and provide useful just-in-time messaging. we will contextualize the messaging components within the greater system architecture we are building upon in section 3 and then detail the approach we've taken to build our proposed dialogue and messaging component in sections 4 and 5. finally, we will describe our plans for future experiments and evaluations. ## related work the rise in popularity and availability of wearable technology and biometric sensors offers the opportunity to create similarly theoreticallydriven, evidence-based behavioral interventions @xcite @xcite @xcite @xcite @xcite . however, the resulting apps generally do not adhere to the american college of sports medicine (acsm) recommendation of 150 minutes per week of moderate-intensity aerobic physical activity or 75 minutes per week of vigorousintensity aerobic physical activity @xcite @xcite . they lack guidance establishing realistic and appropriate behavioral goals, do not assist users in modifying goals over time, display messages that are not personalized, and do not account for contextual or situational barriers, such as weather and emotional states, that can significantly influence physical activity intentions and behavior @xcite @xcite @xcite . active work on jitai systems emphasizes their basis in behavioral theory, user relevance, and actionable feedback @xcite . however, most do not truly account for context or barriers and instead use simple, canned messages delivered at preset moments @xcite @xcite . more recently, @xcite and @xcite demonstrated that individualized, contextualized jitai nudges are significantly better received than non-jitai nudges. some automated coaching systems exist (op den @xcite , but most are limited in interactivity, and efficacy varies. many dialoguedriven health coaching systems are largely theoretical @xcite , although a few extend to more practical implementations with varying degrees of tailoring and interactivity @xcite . several other embodied conversational agents were included in a recent survey @xcite . of these, most rely on scripted dialogue selection, and the others provide limited text interaction at best, lacking the flexibility to adequately tailor to users' unique goals and values. a detailed comparison between textbased coaching and human health coaching was performed in @xcite , demonstrating the feasibility of an automated system with a wizard-of-oz setup. while participants appreciated the automated coaching system, they lamented the lack of tailoring and context sensitivity. some analysis work has been done on counseling dialogues @xcite @xcite and, more recently, on coaching dialogues @xcite . this latter work is ongoing, but focuses primarily on post-dialogue smart goal summarization and health coach assistance rather than interactivity. thus, to our knowledge, no one has yet developed a system to conduct a dynamic, flexible, and interactive coaching dialogue. human coaching dialogue adheres to a specific procedural structure and language. generation for dialogue in this context has added constraints that increase its complexity compared to other domains. while the intents and data content will be provided by the respective messaging policies, it is also important to incorporate personalization @xcite @xcite , empathy @xcite , and additional constraints for a health domain, as well as constrained generation @xcite @xcite and style transfer @xcite . ## automated coaching system the automated coach brings together ideas and solutions in human behavioral theory, physical activity monitoring, cooperative multi-agent architectures, and natural language processing to build an integrated approach to reducing sedentary behavior while increasing users' overall physical activity. the automated coach is designed to ultimately take over certain basic tasks from a traditional human coach: it will be able to meet with users on a weekly basis to negotiate goals and talk about the user's progress. a number of schemas exist for creating a well-defined goal; we discuss these in section 4. the system may also leverage user qualities or other initially-provided information to quickly adapt to users' preferred intervention strategies. unlike a traditional human health coach, it will be available for user-initiated coaching sessions at any time and will be able monitor the user's real-time goal progress. this will allow the automated coach to better support the user, as well as allow it to send personalized just-in-time messaging when needed. unlike typical one-size-fits-all solutions for justin-time interventions such as "remind the user to walk at 10 min before the hour if they have not yet reached 250 steps," which do not work particularly well @xcite , the action plan and user preferences are compiled into a context-(state-) sensitive strategy enumerating possible conditions under which the coach should nudge with particular message content (remind, congratulate, suggest, or otherwise interact with) their user. positive (or negative) reactions to these nudges are used to adaptively learn a better nudging strategy by taking into account both timing and message content. the automated coaching system is built on a multi-agent architecture @xcite for privacy and scalability, and further extended by our contributions toward coaching domains with an eye toward individualization, data integration, domain flexibility and (agent) behavior transferability @xcite . as can be seen in figure 1 , each user is allocated two personalized agents: one autonomous cloud-based agent running continuously on a server that is responsible for data collection, learning, dialogue understanding, and generation; and an app-based agent on the user's smartphone that handles local data collection from the user's smartwatch, user dialogue interface, and summary graphics. although we have previously built-out versions of this using the fitbit charge platform, true jitai is not possible with that due to lack of real-time sensor access @xcite . our current version uses the apple watch platform, which allows for an on-watch app that can be configured by the phone agent to detect specific, context-sensitive intervention events (prolonged inactivity, walking, stress) for jitai. a centralized dashboard agent (not pictured) is also used for running trials to give at-a-glance access to current participant status. agents are able to process data and communicate with each other concurrently. processing can be triggered by communications, state/sensed context/goal changes, and also pre-scheduled agent behaviors. just-in-time notifications are triggered directly on the watch or phone (depending on the type), and do not require dialogue responses. dialogues can be initiated by the phone agent at scheduled times, or by the user. we frame the coaching dialogue as a taskoriented dialogue. however, most task-oriented dialogues consist of rigidly defined simple tasks (e.g., booking a flight or negotiating a price) defined by a few parameters that the system needs to elicit from the user to complete the task. dialogue policy, which determines each system intent (e.g., request information, offer a suggested value) and directs the dialogue, is similarly simplistic and limited: a task is complete when the parameters have been filled and an operation or query happens successfully (e.g., a flight is successfully booked). parameters can be modified or updated until the task is completed. additionally, there is no need for information to carry over from session to session; once a price is agreed upon, for instance, the task is complete, and there are no further exchanges on the subject. health coaching instead centers around a reflective discussion to achieve a more loosely-defined objective: setting a well-defined goal with realistic strategies for completing it. the dialogue is completed when the goal and strategies are not only fully defined by their parameters, but sufficiently motivated and supported to improve the user's success. the latter is accomplished not through filled parameters, but a series of reflective questions to ensure the user has thought their goal through thoroughly. this goal is then revisited at the subsequent coaching session, where a new goal may be set or a new coping plan may be created to assist the client in overcoming unforeseen barriers. additionally, understanding barriers or support systems requires some representation of a health coach's world knowledge. to ensure that the top-down approach aligns with practice and data, we also examined coaching dialogues. due largely to patient privacy and protection, few publicly available datasets exist within the health coaching domain. there is one recently released dataset containing health coaching dialogues conducted via sms text message @xcite . this dataset is tagged with a twolevel labeling structure. one level covers stages and phases, breaking down the overall weekly dialogue into goal setting and goal implementation stages, which further break down into phases such as refining, anticipating barriers, negotiation, and follow up. additionally, they identify smart goal components, which break down a goal into specificity, measurability, attainability, realism, and time-bound components. this dataset released after development on our system had begun, and the coaching paradigm is different from our face-to-face data. due to their curtailed nature, text messages often lack certain nuances, context, and cues compared to verbal interaction @xcite . messages tend to be more curt and elaborate less, which affects the style of questions that need to be used to elicit the same information. discussion of action plan, barriers, and coping strategies is thus unsurprisingly significantly more limited in this dataset, which focuses more on the goal parameterization. however, since our target coaching format is also text message-like, it will still be crucial for designing a text message coaching session, as well as for training the natural language understanding components described briefly below in section 5.1. to ensure that our automated system is rooted in core health coaching concepts and behavior change theory, we examined coach training materials and guidelines provided by our health coaching team or publicly available online. these included outlines as well as coaching roleplay videos. based on these materials, we developed a dialogue model described below in section 5.2. this model was further refined by examining data collected through a tangential, developmental study. we will refer to this data as dataset 1. ## dialogue and messaging system in this section, we detail the dialogue and messaging model and how the natural language interfaces will be built upon it. we will focus primarily on the dialogue model, as the one-way jitai messaging is largely driven by the multi-agent architecture described previously and requires no interaction and much less tailoring of wording than the eventual dialogue system. the overall dialogue system architecture is shown in figure 2 . we chose a traditional, modularly built dialogue system over an end-to-end neural network because the latter is unable to handle the level of complexity, control, and constraint that a health coaching system requires. this system is a modified dialogue state architecture. the dialogue schema and models are hierarchical. the modularity of this system allows for an evolving implementation. the current policy and generation modules are fully rule-based, which allows us to ground the overall structure of the dialogue in theory and coaching protocols. however, these will be incrementally swapped for dynamic, data-driven, learned implementations as the rest of the dialogue system develops to support them. ## formative evaluation the first iteration of our dialogue and messaging system will be both derived (as described above) and evaluated on dataset 1. we are aware that evaluating a system on the data that was used to derive it will bias it. however, since this will only be the rudimentary dialogue and messaging system, we do not believe the risk to be too great, since we will be further refining and evaluating the system with further trials. the evaluation needs to be separated into two parts: evaluating the nlu component and evaluating the message generation. we mentioned above that basing our text-based system on the speech-based dataset 1 will affect the message generation by forcing us to make two assumptions: (1) the messages in text-based conversation need to be more direct and (2) the user is less likely to elaborate on little input. these two assumptions will also affect the evaluation of our generated messages since we cannot evaluate on whether the two messages (one from dataset 1 and one generated by our system) are equivalent. instead, we will need to evaluate on whether both messages ask the user for the same information given the same prompt. evaluating the nlu component could also be complicated due to the same assumptions. in this case, since our system is expecting more direct messages, the nlu component would expect that the user's response would be more straightforward. however, we can see what the nlu component can correctly recognize and this could be a worsecase situation. in addition, we can evaluate it on whether it reacts correctly to a message. therefore, we will be evaluating the system on whether it correctly identifies the parameters it is expecting and on whether the policy correctly prompts the message generation. ## next steps once we have a working dialogue and messaging system, we plan to lead two trials in order to evaluate and improve the system: trial alpha and trial beta. trial alpha. in this trial, we will be generating a dataset we plan to release and evaluating our twoway dialogue. as with the collection of dataset 1, we will once again be collecting data from real user-human health coach interaction. this time, however, all interactions will be text-based and the human coach will have the same information as our automated coach will. we expect the data labelling will function similarly as it did for dataset 1 (as described in the section 4.1). however, we hope that the data will be much cleaner and much more catered to the text domain. as previously mentioned, there was another dataset released in 2022 by @xcite , but it does not cover barrier resolution and strategy negotiation. therefore, we believe this dataset of labelled data will be very helpful in improving future health coaching research. as far as evaluating our system goes, we will not need to evaluate our system around base assumptions like we will have to do for the formative evaluation. therefore, the evaluation will be focused on four factors: (1) given two messages (one from dataset 2 and one generated by our sys-tem), is the content of both equivalent?, (2) is the language from the generated messages appropriate for a health coach? (3) are the parameters the nlu component is expecting reasonable?, and (4) does the nlu component correctly identify the parameters, and does the policy correctly prompt the message generation? to answer all these questions we will be using both standard metrics, such as bleu-4 @xcite , and human health coach manual evaluation. trial beta. this trial is the first time that users will be using our system. it will serve to evaluate both our two-way dialogue and our jitai messaging. by this point we hope to assess (1) how users respond jitai messages and timing, (2) how users respond to our automated coach as opposed to the human coach in two way dialogue, (3) how successful the nlu component is at properly understanding the user, and (4) how successful the automated coach is when compared to the human coach. the goals and focus of this trial are subject to change based on the results of trial alpha. ## conclusion increasing engagement in mvpa and reducing sedentary behavior is a national priority for improving cardiovascular health. while wearable pa monitors show promise in initiating pa change, they do not assist the user in updating their pa goal, nor do they provide personalized messaging to assist the user in overcoming barriers to pa. human coaching, following sound theoretical models of behavior change, has been demonstrated to be effective, but is hard to scale and misses the potential of "just-in-time" behavior suggestions and encouragement, as the coach is not always readily available. our automated coaching system is an integrated system to provide personalized, evidencebased, just-in-time feedback as well as interactive coaching including goal (re)negotiation, targeted at increasing pa and reducing risk for cardiovascular disease. our system focuses on pa, but this infrastructure is modular and extendable to other health behaviors, including stress management and sleep hygiene.