The POWeR of looking into the black box
Authors: Emily Brindal
Journal: The Lancet Diabetes & Endocrinology (2016)

Translational studies using ecologically valid components are necessary to solve real-world problems such as obesity. In The Lancet Diabetes & Endocrinology, Paul Little and colleagues1Little P Stuart B Hobbs RFD et al.An internet-based intervention with brief nurse support to manage obesity in primary care (POWeR2): a pragmatic, parallel-group, randomised controlled trial.Lancet Diabetes Endocrinol. 2016; (published online July 26.)http://dx.doi.org/10.1016/S2213-8587(16)30099-7Google Scholar offer an example of such research in their evaluation of the Positive Online Weight Management Plus (POWeR+) intervention—a web-based behavioural weight management program. The investigators assessed the efficacy of POWeR+ with either brief face-to-face nurse support (POWeR+F) or brief remote nurse support (POWeR+R) compared with an active control strategy (evidence-based dietary advice with two brief follow-up visits). The final intervention outcomes were modest, with patients in the POWeR+F group achieving an additional weight reduction of 1·5 kg (95% CI 0·6–2·4) over 12 months, and patients in the POWeR+R group achieving an additional 1·3 kg (0·34–2·2), compared with patients in the control group. However, when considered in combination with the incremental development of the initial POWeR program,2Yardley L Ware L Smith ER et al.Randomised controlled feasibility trial of a web-based weight management intervention with nurse support for obese patients in primary care.Int J Behav Nutr Phys Act. 2014; 11: 67Crossref PubMed Scopus (57) Google Scholar, 3Yardley L Morrison L Bradbury K Muller I The person-based approach to intervention development: application to digital health-related behavior change interventions.J Med Internet Res. 2015; 17: e30Crossref Scopus (566) Google Scholar use of primary care facilities and behavioural theory to improve translation, and a promising economic evaluation, the intervention could prove beneficial for real-world weight management in the UK. Unfortunately, as the investigators acknowledge, the 12 month study duration does not allow for sufficient understanding of the long-term efficacy of the program. Indeed, there seemed to be early signs of weight regain based on the crude mean values presented (a 1 kg increase between months 6 and 12 in the POWeR+F group and a 1·5 kg increase in the POWeR+R group). A similar study4Appel LJ Clark JM Yeh HC et al.Comparative effectiveness of weight-loss interventions in clinical practice.N Engl J Med. 2011; 365: 1959-1968Crossref PubMed Scopus (550) Google Scholar reported similar weight regains over a longer duration (24 months) and after higher initial losses, suggesting that there might be avenues to strengthen the POWeR+ intervention. As have many others in the eHealth field, Little and colleagues present what is now commonly referred to as a black box—an intervention with little knowledge of its internal workings. For example, to improve its longevity, the POWeR+ intervention targets psychological factors (eg, autonomous motivation), but no evaluation of changes in these constructs is presented. Furthermore, data for receipt of the intervention offer little insight into levels of user engagement and interaction with the website—a substantial component of the intervention. The average number of interactions reported represents more than half of the intended weekly logins, and some participants failed to access core components of the website. With no description of use over time, it is impossible to ascertain whether participants accessed the site ten times over a single week, rather than receiving the intended single weekly dose of, or exposure to, content. Little and colleagues' first prerogative was to evaluate weight loss, which is a crucial, objective outcome; however, successful receipt of the intervention should also be a crucial consideration of intervention efficacy. The data presented reinforce the possible importance of this consideration. The investigators report a 2–2·5 kg difference between patients who completed more than the core website components and those who did not—a value greater than the total intervention effects. This finding adds support to previous assertions about the importance of website engagement5Neve M Morgan PJ Jones PR Collins CE Effectiveness of web-based interventions in achieving weight loss and weight loss maintenance in overweight and obese adults: a systematic review with meta-analysis.Obesity Rev. 2010; 11: 306-321Crossref PubMed Scopus (322) Google Scholar and suggests the potential potency of website engagement for improving outcomes. Admittedly, quantification of a person's receipt of intervention content might be difficult in real-world scenarios (eg, group education sessions). However, electronic tools and programs possess impressive ability to understand exposure, engagement, and retention, and, consequently, the intervention doses that participants receive—eg, what features were used (eg, webpages accessed), when, and for what duration? Presentation of eHealth interventions should aspire to the same rigorous standards as for other trials, and guidelines have been developed to assist researchers working in this field.6Eysenbach G CONSORT-EHEALTH GroupCONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions.J Med Internet Res. 2011; 13: e125Crossref Scopus (164) Google Scholar Existing analysis of usage data has led to concepts such as non-use attrition7Eysenbach G The law of attrition.J Med Internet Res. 2005; 7: e11Crossref PubMed Scopus (1640) Google Scholar and intervention adherence8Kelders SM Kok RN Ossebaard HC Van Gemert-Pijnen JEWC persuasive system design does matter: a systematic review of adherence to web-based interventions.J Med Internet Res. 2012; 14: 17-40Crossref Scopus (792) Google Scholar that have allowed much better understanding of intervention efficacy and avenues to improve potency of web-based interventions. For example, in a large web-based trial,9Brindal E Freyne J Saunders I Berkovsky S Smith G Noakes M Features predicting weight loss in overweight or obese participants in a web-based intervention: randomized trial.J Med Internet Res. 2012; 14: e173Crossref Scopus (72) Google Scholar we witnessed weight loss results simultaneously with poor initial retention and poor uptake of specific website features. With analysis of intervention data, we were able to understand key website features that predict weight loss in order to improve future interventions. During piloting of the initial POWeR intervention, usage metrics were also analysed in great depth,10Arden-Close EJ Smith E Bradbury K et al.A visualization tool to analyse usage of web-based interventions: the example of Positive Online Weight Reduction (POWeR).JMIR Hum Factors. 2015; 2: e8Crossref Scopus (25) Google Scholar yet no such data are reported in the present study. Not all usage data need to be explored this deeply, and simple data can still offer insight into intervention adherence, uptake, and exposure without diluting evaluation of primary outcomes or alienating specific readerships, when reported appropriately.4Appel LJ Clark JM Yeh HC et al.Comparative effectiveness of weight-loss interventions in clinical practice.N Engl J Med. 2011; 365: 1959-1968Crossref PubMed Scopus (550) Google Scholar Future studies should aspire to incorporate excellent translational strategies and rigorous research methods such as those used by Little and colleagues, but they also need to unlock the black box so interventions can rise to the challenge of long-term weight maintenance.11Wing RR Phelan S Long-term weight loss maintenance.Am J Clin Nutr. 2005; 82: 222S-225SCrossref PubMed Google Scholar Some findings have suggested that technology can provide only a transient effect12Svetkey LP Stevens VJ Brantley PJ et al.Comparison of strategies for sustaining weight loss: the weight loss maintenance randomized controlled trial.JAMA. 2008; 299: 1139-1148Crossref PubMed Scopus (576) Google Scholar and that face-to-face contact might be the answer to longer term maintenance of weight loss.13Wing RR Tate DF Gorin AA Raynor HA Fava JL A self-regulation program for maintenance of weight loss.N Engl J Med. 2006; 355: 1563-1571Crossref PubMed Scopus (623) Google Scholar Carefully developed, integrated interventions such as POWeR+ might be able to overcome these limitations. However, an intervention's true efficacy will only be fully understood when we can ensure that participants are receiving the intended intervention dose. This is particularly the case when eHealth components are integral to an intervention because user engagement could be a crucial moderator of success. I declare no competing interests An internet-based intervention with brief nurse support to manage obesity in primary care (POWeR+): a pragmatic, parallel-group, randomised controlled trialWeight loss can be maintained in some individuals by use of novel written material with occasional brief nurse follow-up. However, more people can maintain clinically important weight reductions with a web-based behavioural program and brief remote follow-up, with no increase in health service costs. Future research should assess the extent to which clinically important weight loss can be maintained beyond 1 year. Full-Text PDF Open AccessLooking deeper into POWeR+We welcomed Brindal's comment1 on our Article reporting the results of our Positive Online Weight Management Plus (POWeR+) trial, which demonstrated the efficacy of an internet-based weight loss intervention with brief remote or face-to-face support in promoting weight loss in obese adults in a primary care setting.2 Brindal questioned whether participants received a sufficient dose (website usage) of POWeR+ to achieve optimal efficacy of the intervention. We should have mentioned that we are undertaking a process analyses of (POWeR+), exploring associations between psychological factors, website usage, nurse support, and weight loss. Full-Text PDF