When LLM s Can’t Help: Real-World Evaluation of LLM s in Nutrition (2025)

Abstract: The increasing trust in large language models (LLMs), especially in the form of chatbots, is often undermined by the lack of their extrinsic evaluation. This holds particularly true in nutrition, where randomised controlled trials (RCTs) are the gold standard, and experts demand them for evidence-based deployment. LLMs have shown promising results in this field, but these are limited to intrinsic setups. We address this gap by running the first RCT involving LLMs for nutrition. We augment a rule-based chatbot with two LLM-based features: (1) message rephrasing for conversational variety and engagement, and (2) nutritional counselling through a fine-tuned model. In our seven-week RCT (n=81), we compare chatbot variants with and without LLM integration. We measure effects on dietary outcome, emotional well-being, and engagement. Despite our LLM-based features performing well in intrinsic evaluation, we find that they did not yield consistent benefits in real-world deployment. These results highlight critical gaps between intrinsic evaluations and real-world impact, emphasising the need for interdisciplinary, human-centred approaches.

## introduction every day, individuals make over 200 food-related decisions @xcite . with sedentary lifestyles becoming increasingly common @xcite and global health issues on the rise @xcite , scalable interventions are needed. digital health technologies via mobile devices offer accessible solutions @xcite . in parallel, advances in fine-tuned language models enabled the generation of human-like responses for many practical applications @xcite . this resulted in a general hype and trust-especially among laypeople and companies-in the potential of this technology @xcite . this also applies to nutrition: llms look promising for tasks like meal recommendation, providing dietary advice, and general domain understanding @xcite @xcite . randomised controlled trials (rcts) are required by domain experts before any real-world deployment @xcite @xcite , as they are the foundation of evidence-based medicine and give objective measures of real-world impact. however, past evaluations of llms in nutrition are intrinsic only. no evidence has been collected regarding the impact of llms in realworld nutrition tasks. this includes sustained diet coaching, where users receive feedback on improving their dietary habits @xcite , or nutritional counselling, where tailored empathetic support helps users address more complex dietary issues @xcite . we conduct the first extrinsic evaluation of an llm-enhanced chatbot for these two tasks. we start from a rule-based chatbot capable of scanning users' food diaries to provide tailored insights. then, we integrate two llm-based features: (1) a rephrasing module and (2) a nutritional counselling model. the former varies the base templated responses to make communication more engaging, while the latter provides tailored support, comfort, and suggestions for users' specific dietary concerns. in a seven-week rct with 81 participants, we compare three groups: a group using the full set of features (insights+rephrasing+counselling), an intermediate group (insights+rephrasing), and a base group using only the rule-based chatbot (insights only). we measure dietary outcomes, emotional well-being, and engagement. ethics details are presented in section d. based on our results, the "promise" of llms in nutrition falls short in the real world: the llmbased features had little to no effect on any of the measures we consider. our study provides critical insights into the effectiveness of llms in nutrition, the safe deployment of these models, and the interdisciplinary challenges of applying them to sensitive domains. ## related work digital health interventions can improve accessibility, cost-effectiveness, and patient-centred care @xcite @xcite . for example, telemedicine allows remote consultations @xcite , while wearable devices enable continuous health monitoring @xcite . mobile health interventions can be highly effective in terms of user adherence @xcite @xcite @xcite . in nutrition, chatbots have emerged as a promising tool to promote healthy eating habits, offering food intake tracking @xcite , educational content, and motivational messaging @xcite @xcite . recent advancements in pre-trained language models can further expand their capabilities in this domain, but not without shortcomings. recent llms can generate meal plans and dietary recommendations based on user needs, but their performance decreases in more complex cases @xcite @xcite . beyond meal guidance and recommendations, ai chatbots are being increasingly explored for their potential to provide empathetic support towards behaviour change. negative emotions can lead to poorer nutritional choices @xcite and tailored advice can mitigate this mental load @xcite . chatbots can be approachable, calming, and display adequate therapeutic skills @xcite @xcite . this can enhance engagement in sensitive contexts like body image, eating disorders, and relationship counselling. artificial empathy and personalised interactions from chatbots @xcite can help users in pursuing healthier habits. the therapeutic promise of chatbots also raises important ethical questions. there is still need for deeper integration of empathy, mental health sensitivity, and patient safety into chatbot design @xcite . when evaluated by experts, ai nutritional support is often scientifically sound but potentially outdated, inaccurate @xcite , or even harmful for more complex cases @xcite . while accuracy is important, user engagement also plays a critical role to the success of digital health interventions in nutrition. when users lose interest in using the chatbots, this causes a rapid decrease in dietary adherence, and eventually result in early drop-out @xcite @xcite . user satisfaction and motivation are usually pursued through personalisation, communication, visual elements @xcite @xcite , or social support mechanisms @xcite . for all of the above aspects, rigorous extrinsic evaluation is needed, to move chatbots from experimental prototypes to deployable healthcare assistants @xcite . yet, existing evaluations are synthetic @xcite , focused on textual characteristics or accuracy against standardised benchmarks @xcite . our work is, to our knowledge, the first @xcite randomised controlled evaluation of llm-delivered diet coaching and nutritional counselling over an extended deployment. ## chatbot development we extend the rule-based diet-coaching chatbot by @xcite , which is deployed on the telegram messaging platform and designed to deliver personalised nutritional insights based on users' food diary on myfitnesspal (evans, 2017), a popular and freely available calorie-counting app. the core functionality of the chatbot involves delivering insights in two forms: (1) basic insights (figure 2a ), which are simple recaps of a user's dietary intake-calories and nutrients, and (2) advanced insights (figure 2b ), which present an extended textual description with some corresponding data visualisation. for this work, we extend the chatbot foot_1 with two llm-powered features: rephrasing to vary the templated responses, and nutritional counselling through fine-tuned models. figure 1 illustrates (a) basic insights into all monitored nutrients for a single day. (b) advanced insights comparing the average calorie intake and toughest day from the first half of the month to the other half. the architecture and flow of our chatbot. its main objective is to improve long-term diet adherence, emotional well-being, and user engagement. ## chatbot trial we conducted a seven-week randomised controlled trial to test the chatbot's real-world impact from may to june 2024. participants were recruited through social media, flyers, and direct outreach across multiple locations, resulting in a demographically diverse cohort. exact demographics are outlined in section f. through an onboarding process, users were taught how to install the required apps, connect their accounts, access the chatbot, and use it. we successfully onboarded 87 participants, and 81 of them completed the full seven-week duration. dropouts occurred across all study groups, primarily due to time constraints or lack of engagement. these early exits were balanced across conditions comfort: it's understandable to turn to food as a coping mechanism for stress. reframing: something positive you could consider about this is that you are aware of the connection between stress and eating habits, and can take steps to address it. suggestion: starting from tomorrow, you could try practicing mindfulness techniques such as deep breathing or meditation to help manage stress and reduce cravings for food. participants were randomly assigned to one of three groups: baseline (@xmath0, with templated insights from the food diary only; rephrased (@xmath1, with llm-rephrased responses; full (@xmath2, with llm-rephrased messages and nutritional counselling. throughout the trial, participants logged their meals daily via myfitnesspal, engaged with the chatbot on telegram five or more times per week, and completed a weekly emotional wellbeing questionnaire using the positive and negative affect schedule (panas) @xcite (more details in section 5.2). participation in the trial was incentivised through weekly online gift vouchers, with a doubled reward in the final week, and adherence was monitored through myfitness-pal logs and conversation history. while we en- best and worst ∆ highlighted. the trial faced several technical challenges, including a temporary disruption in the chatbot's access to myfitnesspal data (beyond our control), a necessity to retrain the nutritional counselling model, and rare bugs that made the chatbot unusable for short periods of time (typically one hour or less, and fixed promptly). we discuss these in section e. ## conclusion as of today, there is an increasing trust in applying llms to healthcare and its related sub-domains, including nutrition. however, these models are typically evaluated intrinsically, on unrealistic or simulated tasks, and mostly relying on metrics. in this work, we present the first objective evaluation of the real-world impact of llms in nutrition. we integrated them into a chatbot for diet coaching and nutritional counselling, and ran a seven-week rct on a population of 81 participants. our results quickly point out the limits of intrinsic llm evaluation. we found no consistent improvement across our metrics. although some statistically significant improvements emerged, these appear spurious in the larger context of our trial. based on our results, these models are unable to effectively promote dietary adherence, reduce the emotional load of dieting, provide empathetic help through counselling, or simply boost user engagement. we conclude that, at the current time, there is no evident benefit in applying llms to nutrition in the setup we investigated. while our findings are specific to this domain and trial configuration, they serve as a step towards real-world evaluation in healthcare, highlighting how llms might (or might not) affect user outcomes in chatbot-driven nutritional counselling. our overall research underscores the importance of critical evaluation of llms in health-focused applications. although these models are often heralded for their potential to deliver dynamic and personalised interactions, our findings caution against adoption without rigorous real-world validation. we hope our results will shed light on the need for real-world assessments beyond benchmarks.