Beyond calories: evaluating how tailored communication reduces emotional load in diet-coaching (2022)

Abstract: Dieting is a behaviour change task that is difficult for many people to conduct successfully. This is due to many factors, including stress and cost. Mobile applications offer an alternative to traditional coaching. However, previous work on apps evaluation only focused on dietary outcomes, ignoring users’ emotional state despite its influence on eating habits. In this work, we introduce a novel evaluation of the effects that tailored communication can have on the emotional load of dieting. We implement this by augmenting a traditional diet-app with affective NLG, text-tailoring and persuasive communication techniques. We then run a short 2-weeks experiment and check dietary outcomes, user feedback of produced text and, most importantly, its impact on emotional state, through PANAS questionnaire. Results show that tailored communication significantly improved users’ emotional state, compared to an app-only control group.

## introduction an unhealthy diet poses a serious threat to an individual's health. research showed that a poor diet kills more people than smoking @xcite and that obesity has tripled since 1975. 1 coaching through human experts is one of the most effective ways to improve diet @xcite , but it can be too expensive for disadvantaged groups, adding to other costs associated with a healthy diet @xcite @xcite . e-health apps are a cheaper alternative, although there is mixed evidence about their effectiveness @xcite @xcite . compared to experts, apps often show sub-optimal communication. typically apps focus on data presentation (e.g.: charts), limiting the use of text to short and fixed messages. this could be the reason why previous apps evaluation focused primarily on diet outcomes. there has been little work on effective communication for dieting tools: this should be addressed as it plays a big role in engagement and adherence @xcite . dieting habits are also known to be influenced by emotional state @xcite @xcite , yet no prior work on diet-apps investigated communication's role in this. in this paper, we implement an advanced communication strategy and investigate its effect on emotional state in the context of diet coaching apps. we exploit affective-nlg, text-tailoring and persuasive communication techniques to create weekly diet reports. reports are implemented as an additional layer on top of a standard diet app, augmenting its communicative capability. we then proceed to evaluate our system in a short experiment. we compare participants that used the report-augmented app, with an app-only control group. unlike previous work, we do not focus our human evaluation on dietary outcome only. we inspect communication adequacy through user feedback on a variety of measures including readability and accuracy. as a novel contribution we evaluate if our reports improved participant's affective state. we adopt a validated psychometric tool, the panas questionnaire @xcite , to analyse the behaviour of both groups on a weekly basis. participants who received our report experienced significantly more positive emotions and fewer negative ones. we also observe the opposite behaviour in the control group. in section 2 we expose the common limits of diet apps under the functional, communication and psychological aspect. we also briefly describe sota hello dan18777, you told us that you want to gain some weight, so we wrote this report especially for you. your calorie intake could use some improvement: there was an occasional lack of food (generally you ate about a third less than your target). it was a bit better the previous time, we're sure you can do it again! friday looks like the most problematic day (you ate about half of your target). it seems that sodium and protein intake needs a bit of improvement. your sodium consumption was about half of your target. of the foods you ate, "spinacina" was the highest in sodium. it would be better to correct this as sodium deficiency can lead to cramps. also, your protein consumption was about half of your target. last week it was better and we know you can do it again! "pizza" was the food you ate which had the most protein. keep in mind that protein deficiency can be responsible for muscle loss. in communication-based systems for diet-coaching. in section 3 we detail our approach to augmenting diet-coaching apps communication, and describe the implemented features. we present our experiment methodology in section 4, and discuss the results in section 5. in section 6 we sum up our conclusions and present our future research directions. finally, in section 7 we detail the procedure through which we ensured ethical compliance for our experiment. ## related work today people can access lots of diet tools, but both academic and commercial products show some common limits. some of these are purely functional: missing features that negatively impact on effectiveness. this includes low accuracy @xcite , fixed suggestions @xcite or the excessive use of humans in the loop @xcite 23 . low accu-2 www.rise.us 3 https://www.noom.com/ racy is an obvious limit to the app's effectiveness; fixed suggestions overlook customisation and potential dangers (like allergies, user taste and religious food dogmas); major use of human experts nullifies apps' usefulness in the first place. however, these problems can be solved by expanding the tool-set of features and evaluating dietary outcomes. but if we consider the behavioural component of dieting, we raise different problems, for example at communication and psychological level. previous research showed that behaviour change benefits more from advanced communication @xcite @xcite than from factual text. diet apps @xcite @xcite , however, do not follow this logic and favour data presentation, through visual features (like charts, color codes and tables) @xcite . at communication level, used text is typically short, fixed and lacks informativeness @xcite . a first way to improve the communication of diet apps could be the use of finetuned, domain-specific nlg, combined with texttailoring @xcite and persuasive communication @xcite @xcite . this is motivated by the relationship between personalisation and engagement in diet apps @xcite , and the role of persuasion in behaviour change @xcite . additionally, nlg has been used in various healthcare domains @xcite @xcite , including some work in nutrition. shed @xcite is a tailored dietsystem that exploits nlg to propose alternative meal plans in real time. initial inspection of user acceptance showed it as a promising system for further evaluation. a conceptual diet-recommender system has been proposed @xcite , focusing on reinforcement learning for linguistic personalisation. other work @xcite presented a nlg-based persuasive reasoner to address dietary guidelines violations. evaluation showed the appropriateness of presented feedback, and its effectiveness in reducing the amount of violations compared to canned text. madi-man @xcite ) is a persuasive diet-coaching system, developed to convince the user to opt for an healthier diet. evaluation in both controlled and uncontrolled scenario revealed that users appreciated the presence of both visualisations and text, and confirmed its persuasiveness. while these works evaluated the use of persuasion and dietary outcomes, we note that tailoring involved only data analysis (e.g.: custom meal plans) and not textual features. moreover, previous research did not inspect whether the adopted communication techniques had an effect on users' emotional state. this aligns with previous evidence that diet-apps rarely consider this element @xcite . we know from nutrition research @xcite @xcite ) that user's emotional/affective state influences eating habits, causing various issues including calorie excess @xcite , emotional @xcite and binge @xcite eating. the importance of this factor is also confirmed by previous research of the matter in other domains such as cognitive behaviour therapy @xcite , mental well-being @xcite , substance abuse @xcite or emotional support in public speaking @xcite . to the best of our knowledge, this is the first work in nlg for nutrition that investigates the influence of the system on affective state. ## augmenting diet apps communication we implement an nlg report generator for dietcoaching based on our previous work @xcite foot_1 , and use it to augment the communication strategy of a traditional diet app. we use myfitnesspal (mfp) @xcite as data source. the execution flow can be seen in figure 1 . the report is tailored based on various preferences. users were asked to specify: username and motivation are injected in the report, to make it feel more personal, while the other elements are used for content selection and tailoring. reports are further enriched with the following insights: ## experiment setup we evaluated the effect of our reports on the diet and emotional state of users in a 2-weeks experi-ment. a total of 81 participants were recruited (see section 7 for details). population demographics can be seen in figure 3 . participants were trained in using mfp and asked to log their meals through the app for the following 2 weeks. they were then randomly split into two groups: "report group" (@xmath0. participants in report group received one report at the end of each week, while control group could only see the insights provided in mfp. about 60% of the participants (from both groups) agreed to fill-in a weekly panas questionnaire @xcite that we used to monitor their emotional/affective state. panas consists of 20 mixed positive (e..g: "attentive", "proud", "strong" etc...) and negative (e.g.: "hostile", "guilty", "scared" etc...) words. users score @xcite or ham-a @xcite would have been too focused on specific aspects. we also avoided combining multiple tools as this could have been too tiring for participants, leading to inaccurate results. finally, at the end of the experiment, participants were asked to evaluate the report by scoring eight likert-7 questions that can be seen in figure 6 . participants were also given the chance to express an open comment about the system. we let participants from the control group read one single report at the end of the experiment to let them express their feedback as well. through this setup, we inspected the following research hypotheses: hypothesis 1 (h1): participants in report group improved their diet (in terms of caloric and nutritional intakes) more than control group. hypothesis 2 (h2): participants in report group improved their positive affect score more than control group. hypothesis 3 (h3): participants in report group improved their negative affect score more than control group. while h1 is comparable to classic diet-coaching evaluation, we introduce h2 and h3 as a novel investigation of the communicative potential of these tools, related to users' emotional state. for h1 we check the initial distance between mfp goals (for calories and nutrients) and user intake. then, we verify if, at the end of week 1 and week 2, participants got closer to said goals. for nutrients, we consider the two most unbalanced ones (those that could be seen in the report). for h2 and h3 we monitor weekly panas scores (pa and na) for each group. since no group had access to reports when completing panas at the end of the first week, we use this value as a starting point. then, we check differences at the end of week 2 and overall (from the start of the experiment). ## results and discussion in terms of dietary outcomes, we obtained mixed results, but none of these were significant. in fact, the majority of participants improving calories and the first most unbalanced nutrient were in report group, but a chi-squared test revealed no significance (see on the other hand, panas analysis gave us more interesting results. initially, we verified through a t-test that the two groups shared similar initial pa (average @xmath1.1, @xmath2.96) and na (average @xmath3.7, @xmath4.51). then, we checked how scores changed for both groups. pa and na were checked at week 1, week 2 and across the whole experiment. the report group showed bigger improvements, both in terms of pa and na (see figure 5 ). the report group showed (through t-test and sidak's p-value adjustment) a significantly bigger improvement for pa on the second week (@xmath5.04) and for na across the whole experiment (@xmath6.04). generally, the report group improved both scores more than the control group in any other situation, but only in these two cases the p-values were statistically significant. these results tell us that the report group tended to experience significantly: final feedback (figure 6 ) was mostly positive. the lowest scores belong to the help in changing diet, which could also be related to the experiment duration. when given the chance to express a comment on the system, many participants asked for charts and graphical elements which could have improved understanding. this result aligns with previous research @xcite @xcite , suggesting that a combination of visual features and textual communication could be the most effective approach. ## conclusion in this paper we evaluated the effects of augmented communication in diet-apps using affective nlg, tailoring and persuasive communication techniques. unlike previous work in evaluating diet-coaching systems, we did not look only at di-etary outcomes. since diet is influenced by psychological factors we introduced a novel evaluation by adopting a validated psychometric tool (panas). we inspected whether our reports could play a role in improving users' affective state. our hypotheses were confirmed, as we found that participants who read the report experienced more positive emotions and fewer negative ones. we also saw the opposite in most cases for the control group. our work has shown that improved communication can reduce the impact of emotional load on dieting. most importantly, we showed how important it is to consider the psychological component when designing, developing and evaluating communication systems, in diet-coaching and other domains. we could not see an effect on diet itself, which encourages us to run a longer trial (one month or more) in future, to further assess the effectiveness of our communication strategy. however, we ran just a basic assessment on the psychological side. we plan to expand our evaluation procedure by combining multiple tools and scales. as our previous work pointed out, stress is one particular factor that could be worth monitoring @xcite , so this is one of the main directions we intend to follow. we also could not run any kind of ablation test. this leaves us with the conclusion that our approach did work, but without any insights on how different elements (affective nlg, persuasion or text tailoring) contributed. based on the feedback from users, more than just text is required to improve the system. we leave this as future work. still feedback was largely positive with regards to textual features and comprehension. we note that the questions were not accompanied by rigorous definitions of "readability", "accuracy" and others. users expressed feedback based on their own personal idea of these concepts and this raises questions regarding the reliability of the results. we consider the overall uniformity of ratings as an indicator that all participants had a "common" definition of the proposed concepts. still this uncertainty contributes to a well-known problem in human evaluation @xcite , so we commit to more rigorous and uniform metric definitions in future.