Rapid Evaluation of Innovative Intervention Components to Maximize the Health Benefits of Behavioral Obesity Treatment Delivered Online: An Application of Multiphase Optimization Strategy
Phase: PHASE2
Conditions: Overweight and Obesity
Sponsor: The Miriam Hospital

 
 
 
 
 
 
 
 
Study Protocol and Statistical Analysis Plan 
 
 
Rapid Evaluation of Innovative Intervention Components to Maximize the 
Health Benefits of Behavioral Obesity Treatment Delivered Online: 
An Application of Multiphase Optimization Strategy 
 
NCT04520256 
 
March 27, 2023 
 

 
1. Study Protocol 
1.1 Study aims and design 
This study, sponsored by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), 
aims to optimize online behavioral lifestyle intervention (BLI) by testing whether any of five theory- 
driven experimental intervention components increase the proportion of individuals achieving a minimum 
clinically significant weight loss of ≥ 5% of initial body weight and/or mean weight loss after 12 months 
of treatment. The rationale and design of the study methods are informed by the 3-Phase Multiphase 
Optimization Strategy (MOST) framework. Per the MOST Optimization Phase, a factorial experiment will 
be conducted to determine which, if any, of the experimental intervention components contributes to 
improved treatment outcomes. Adults with a body mass index (BMI) in the overweight or moderately 
obese range (25-50 kg/m2) will receive the preexisting 12-month Rx Weight Loss (RxWL) and a random 
combination of 0-5 of the experimental intervention components. A priori optimization criteria will be 
used to evaluate which, if any, of the experimental intervention components should be included in an 
optimized RxWL treatment package, which can then be tested in a future randomized controlled trial per 
the MOST Evaluation Phase. Mediation analysis will also be conducted to determine whether the 
experimental intervention components exerted their intended effects on their putative behavioral and 
psychosocial mechanisms, and whether those effects were associated with improved weight loss 
outcomes. Moderators analysis will explore for whom and under what circumstances the experimental 
intervention components might have been more or less effective. Assessments of weight and hypothesized 
mediators will occur at baseline and 3, 6, and 12 months after randomization. The study aims are: 
 
1.1.1 Aim 1: Optimize RxWL via a factorial experiment to determine which combination of 
intervention components maximizes the proportion of participants achieving a ≥5% weight loss and mean 
weight loss at 12-months. 
 
1.1.2 Aim 2: Conduct mediation analysis using proximal outcomes to test hypothesized mechanisms of 
action and moderator analysis to understand for whom and under what circumstances the interventions are 
effective. 
 
1.2 The RxWL intervention. 
RxWL is an automated online BLI that was initially developed to support weight loss in adult primary 
care patients with overweight/obesity.  
 
All participants begin the program with an automated online “kickoff” session that explains BLI basics 
including goal-setting and self-monitoring. Participants are encouraged to lose 0.5 kg to 1 kg per week, by 
limiting energy intake to 1200 kcal to 1500 kcal per week depending on starting body weight, to achieve a 
total weight loss of ≥ 10% of starting body weight. Other goals include consumption of a low-fat or 
Mediterranean diet (decided by the participant) and regular moderate-to-vigorous physical activity 
(MVPA) with weekly MVPA goals that gradually increase to 150 minutes per week by 3-months and 250 
minutes by 12 months. 
After the kickoff session, participants are provided with weekly interactive multimedia lessons on key BLI 
topics for 3 months, followed by 9 months of monthly lessons that are designed to support further weight 
loss and weight loss maintenance. Lessons highlight strategies from the Diabetes Prevention Program and 
Look AHEAD trial BLIs, with topics that include consuming a healthy diet, meal planning, building an 

exercise habit, restaurant eating, social support, and weight regain prevention, among others. The lessons 
involve a combination of animated graphical content with voiceover and live action video. The lessons are 
self-paced and allow participants to explore additional information on key topics of interest (e.g., 
additional examples of how to calculate the calorie content of various prepackaged foods in a lesson 
explaining label reading and portion size). Navigation through each lesson is self-paced and they are 
designed to require about 15 minutes to complete. 
In addition to the lessons, participants are asked to self-monitor their total energy intake (kcals), daily 
MVPA minutes, and body weight for the first 3 months of the program and to enter these data on the 
RxWL platform at least weekly. They are asked to weigh immediately upon waking in the morning, in 
light or no clothing, with the scale placed on a hard surface. Participants are given instructions and 
resources for using paper diaries and online/mobile tracking apps for this purpose. Flexibility in the 
method of self-monitoring improves the potential for reach and dissemination of the program given that 
that some individuals may not own a smartphone for use with tracking apps and/or may have limited 
confidence in using tracking apps, a more complex technology. Participants are not encouraged to use one 
method or app over the others, but if the FitBit app is used for self- monitoring, the data can be shared 
automatically with RxWL via an application programming interface (API). During the final 9 months of 
the program, participants are asked to self-monitor for one week per month, coinciding with the provision 
of the monthly lesson. At the end of each week of self-monitoring, a feedback message is algorithmically 
generated based on participants’ self-reported weight, daily reported energy intake (kcals), and daily 
reported exercise minutes. The content of the feedback changes weekly and provides encouragement and 
constructive feedback to support adherence to BLI strategies taught in the lessons. The RxWL platform 
also contains supplementary printable guides on topics such as eating healthfully on a budget, exercising 
safely, meal planning, and problem- solving. Finally, reminders to engage with the program are sent 
weekly to participants who have not logged their self-monitoring data. 
 
1.3 Description of the experimental intervention components. 
1.3.1 Online VR intervention for training in basic behavioral weight loss skills. Experience 
Success is a fully automated online based VR intervention for experiential training in basic behavioral 
weight loss skills. Skills training occurs in virtual environments where challenges typically experienced 
by individuals losing weight (e.g., selecting from a variety of foods ranging in calories and healthfulness, 
peer pressure to overeat and be inactive, environmental cues for overeating, common barriers to physical 
activity) are presented. Participants see VR actors implementing skills, practice skills themselves, and 
receive praise for successful use of skills. Participants end each session by making explicit plans for real-
world skills use. The intervention consists of four modules corresponding to four virtual environments: 
home, the workplace, the gym, and a party (i.e., a social eating situation). Each module contains about 
two hours of content, but based on a participant’s choices, a single scene playthrough takes about 20 
minutes. An example of the challenges faced by users in the home module involves how to prepare and 
pack a healthy low-calorie lunch before the bus to work arrives. The user is presented with a selection of 
ingredients, cooking methods, and packaged foods. They then experiment with different combinations 
that balance caloric content, dietary quality, and preparation time until they finalize a meal, which they 
then learn to prepare. Experience Success was recently found to enhance 12- week weight loss outcomes 
of the commercial online Weight Watchers program. 
 
 
 

1.3.2 Tailored interactive video feedback targeting personalized training in dietary skills. 
Like the existing text feedback that this component replaces, the video feedback is automated and uses 
the exact same algorithm to assemble an aggregate message of 3-4 pre- recorded video clips 
(approximately 30 seconds each) that varied each week. Video feedback messages are delivered on-
camera by an obesity treatment expert for a more interactive experience and to enhance a sense of human 
connectedness. Feedback content is nearly identical to the text version but modified for a more 
conversational tone. At the end of the feedback message, participants are given the opportunity to 
“interact” by choosing from a predetermined set of situations and behavioral challenges to receive a 
further refined feedback video. For example, participants may select that they are traveling or have 
experienced sickness, injury, or a significant life stressor. The corresponding feedback teaches skills such 
as healthy restaurant eating and stress management. If a participant indicates that they are traveling, they 
might be encouraged to order smaller portions (e.g., appetizers instead of entrees) at restaurants and look 
for opportunities to increase physical activity to offset increased intake. Additional feedback given in 
response to selecting a behavioral challenge generally falls into one of the following categories: 
behavioral skills to support healthy eating (e.g., meal planning), selecting and preparing healthy foods 
(e.g., increasing fruits and vegetables, reducing sugary beverage intake), maximizing accuracy of self-
monitoring (e.g., portion size estimation), and physical activity adherence (e.g., enhancing motivation, 
introducing variety). Training in dietary skills is particularly emphasized as it provides a method of 
improving two key elements that drive the feedback, namely dietary intake and weight change. The 
options for interactivity are titrated each week to reflect the progressive nature of weight management 
skills acquisition and complement skills taught in the RxWL video lessons (i.e., more basic skills are 
emphasized earlier in the program, and more advanced skills later in the program). 
 
1.3.3 Tailored interactive intervention to promote structured physical activity. This 
component consists of three parts, including evaluation of physical activity, a tailored physical activity 
plan, and instructional and educational videos for home-based physical activity. The core RxWL program 
focuses on eating and dietary behavior in the firsts two weeks and then targets physical activity behavior 
beginning in the third week, which involves gradually increasing PA to 150 minutes during the first 3 
months, then increasing to 250 minutes by the end of the 12-month program. Core program content 
focuses on gradually increasing PA to meet overall goals, and involves general suggestions for safely 
building a PA habit that emphasizes brisk walking (e.g., planning ahead for PA, stretching before and 
after PA). 
Consistent with the core RxWL program structure, in week 3 participants receiving the experimental PA 
intervention component complete an online evaluation of physical activity, with a focus on time spent in 
structured MVPA. These data inform a tailored physical activity plan that promotes adding 50min/wk 
(+10 min/day on 5d/wk) to participants’ current level of physical activity each month until participants 
achieve the national guideline of MVPA for weight loss (250 structured minutes weekly). Participants 
select their preferred plan (10, 20, 30, 40, or 50 min/day on ≥5 days weekly) for the upcoming four 
weeks. If participants feel strongly that their suggested goal is not appropriate for them, they can select a 
different goal (e.g., to push themselves harder or select a lower activity level). The physical activity 
evaluation, generation of the tailored activity plan, and delivery of tailored video content reoccurs after 
two weeks, and monthly thereafter for the duration of their participation. 
Participants are given long-form exercise videos to complete at home. To ensure that participants have 
appropriate support for each of the physical activity goals they could select during the evaluation and 

planning step, videos are available with durations of 10, 20, 30, 40, and 50 minutes. The complexity and 
intensity of the videos increases slightly with duration, but all videos are designed to produce MVPA. 
Videos primarily utilize body weight exercises targeting all muscle groups and increasing heart rate. 
Each exercise is presented with modifications for adaptation to the participant’s functional capacity, 
preexisting injuries, and movement preferences. All the exercises can be done within a 3’× 2’ space. No 
equipment is needed for most exercises, but a mat and light hand weights are utilized in some 
adaptations. Participants are given guidance on safety, avoiding injury, and alternatives to hand weights 
that they may use if desired. Most of the exercises are conducted over a time interval (e.g., “Do this 
exercise for 45 seconds”) so that participants do not need to count repetitions and sets of weight-bearing 
exercises. Each video includes background music, narration, and instructors synchronously 
demonstrating the same exercise from different views and/or at different ability levels. When the PA goal 
is selected, all videos at that duration and lower are made available to allow flexibility in meeting the 
goal (e.g., at participant with a 40 min/day goal may choose to do the 20 minute video twice in a day if 
they find it difficult to set aside 40 uninterrupted minutes for physical activity). 
Participants are provided with supplementary videos each month in addition to the long- form videos. 
The topics are primarily instructional (e.g., upper body exercises, lower body exercises, warming up and 
cooling down, core mat work) or educational (e.g., overcoming exercise barriers, converting intentions 
into actions). Participants setting a higher MVPA goal (based on recent MVPA engagement) receive 
additional videos to facilitate additional optional strength training and exercise of higher intensities. 
Participants are instructed to count sets and repetitions (or use time goals when appropriate) and 
determine the appropriate supplementary weight needed. They are also given injury prevention 
information, tips for proper form, and suggestions for modifying the 10-50 minute videos to increase 
intensity. These videos are not made available to participants selecting a lower minute goal in order to 
avoid overwhelming them and to focus them on building an initial physical activity habit. Participants 
may access the supplementary videos available to them as often as they like. 
1.3.4 Tailored interactive skills training to address dysregulated eating. This component 
consists of interactive skills training videos (i.e., “modules”) that utilize ACT approaches to target the 
impact of dysregulated eating on weight loss. The ACT strategies are presented as compatible with the 
“control” and “change” strategies that are taught much more briefly in the core RxWL program; 
participants are encouraged to use ACT strategies when they find that the strategies taught in the core 
program are insufficient for addressing dysregulated eating. Starting in week 4, participants complete an 
evaluation to identify negative internal experiences (i.e., thoughts, feelings, and bodily sensations) that 
can trigger dysregulated eating by rating statements on a 5-point Likert scale (e.g., “When I’m upset, I 
use food to make myself feel better”, “I need to have fewer food cravings in order to succeed with my 
weight management”, “Excuses often get in the way of me meeting my health goals”). An algorithm then 
generates suggestions for modules to be completed based on participant responses and provides a brief 
summary of the content of each module. The participant selects a module, which consists of four 
consecutive weeks of 20-minute video lessons with assignments to practice skills in between sessions. 
There are four modules: 1) the “Cravings” module uses guided-meditation exercises to teach craving 
awareness and tolerance (i.e., the ability to non-judgmentally observe cravings without eating to make 
them go away); 2) the “Thoughts” module uses metaphors and experiential exercises to help participants 
recognize patterns of difficult thoughts and learn alternative strategies for coping with them (e.g., non-
judgmental acceptance); 3) the “Acceptance” module uses metaphors and guided-mediation exercises to 
illustrate the consequences of trying to control emotional experiences (e.g., using eating to change 
negative emotions) and presents an alternative: being willing to feel undesirable emotions in the service 

of staying on track with one’s health goals; and 4) the “Motivation” module uses experiential exercises to 
clarify participant values and explicitly link these values to RxWL program goals (i.e., reducing calorie 
intake, improving diet quality), and then teaches strategies to strengthen commitment to one’s chosen 
values. The dysregulated eating evaluation and tailored suggestion process repeats monthly. 
 
1.3.5 Online platform for social support and friendly competition. This component provides 
participants with the opportunity to post messages to each other in a shared public form. Conversation 
threads allow for in-depth conversation and discussion, while a Twitter-like feed allows participants to 
post short messages and questions. Participants may also create a profile, including a photo, their first 
name (or a preferred nickname to maintain anonymity), and a brief message about interests, preferred 
weight loss strategies, and challenges. Participants will be discouraged from discussing their experiences 
with the other experimental intervention components on the public form in order to limit risk of 
contamination between conditions. Communication in the online forum will be monitored and 
participants will be asked to remove any mention of other experimental intervention components, should 
they be posted. 
Friendly competition, accountability, and recognition of success are facilitated by a leaderboard for 
weekly/total weight loss, weekly/total MVPA minutes, and longest streak of days for self-monitoring. 
The top 10 participants’ first name or nickname is displayed. Participants are also awarded badges that 
are displayed on their profile when they accomplish important milestones. Some can be reached quickly, 
such as losing the first 5 lbs., self-monitoring daily for a full week, and exercising for ≥50 mins within a 
single week. Conversely, some milestones require weeks or months to achieve, such as reaching a 5% and 
10% weight loss, self-monitoring daily for a month, and exercising for a total of 5,000/10,000 minutes. 
When a new badge is earned, the participant receives an online notification. 
 
1.4 The factorial experiment. 
All participants in the proposed randomized experiment will receive access to 12 months of the RxWL 
intervention and in addition will be randomized to receive 0-5 of the experimental intervention 
components, using a full factorial design (Table 1).16 The majority of participants (63%) will receive two 
or three intervention components, 16% will receive one or four components, 3% will receive zero or five 
components. Importantly, each intervention component has been designed explicitly such that receiving 
any of the other intervention components is neither required nor precluded to fully experience the 
intervention component and obtain its full benefit. The full factorial design will allow tests of whether 
there are interactions between components in their effect on weight loss such that certain combinations 
produce an effect that is stronger or weaker than would be expected given the strength of the main effects. 
1.4.1 Participant eligibility criteria. Eligibility criteria are kept broad in order to select a sample 
representative of the target audience for dissemination of RxWL. Inclusion criteria include age 18 to 70 
years old; BMI 25-50 kg/m2; access to the Internet; an ability to walk two city blocks without stopping; 
and English fluency and literacy at the 6th grade level. Exclusion criteria include current participation in 
another weight loss program; current use of weight loss medication; weight loss of ≥ 5% of body weight in 
the previous 6 months; history of bariatric surgery; pregnancy within the previous 6 months; a plan to 
become pregnant within 12 months; report of a heart condition; chest pain during periods of activity or 
rest in the previous 6 months; loss of consciousness in the previous 6 months; report of a medical 
condition that could make participation in unsupervised physical activity unsafe; or report of a condition 
that would result in an inability to follow the study protocol including terminal illness, substance abuse, an 
eating disorder not including Binge Eating Disorder, and untreated major psychiatric illness. 

 
1.4.2 Participant recruitment and enrollment. Participants will be recruited via advertisements in 
local media (e.g. newspapers, radio); targeted online advertising (e.g., Facebook, Google AdWords); 
flyers and advertisements posted in waiting rooms and exam rooms in primary care offices used in 
previous studies of RxWL; informational materials made available as part of the health and wellness 
program for employees in the Lifespan Heath System and hospital network (an approach used in a 
previous RxWL trial); and direct mailings. A screener completed online or by phone will be used to 
determine initial eligibility. Individuals who appear eligible will be invited to attend an orientation session 
at the research center. The study will be described in detail and participants will have the opportunity to 
complete informed consent, begin the baseline assessment, and schedule their randomization visit. 
Participants are required to complete the baseline assessment, including wearing the physical activity 
monitor and completing dietary recalls (described below), to be eligible for randomization. 
 
1.4.3 Randomization. Prior to enrollment, a randomization schedule will be prepared based on 12 
blocks of 32 patterns of assignment to each of the five intervention components. A sequentially-numbered 
sealed envelope containing the assignment will be opened by research staff at the time of randomization. 
The participant will receive brief written materials explaining their assignment and how to begin treatment 
via the online system. Further orientation to the treatment system is conducted online via interactive video 
lesson. Study staff will be available by phone to trouble-shoot technical difficulties, but no human- 
delivered weight loss counseling will be provided in-person or by phone. Participants will be enrolled until 
the total N=384 target enrollment has been satisfied. 
 
1.5 Measures 
Assessments will be conducted at baseline, 3, 6, and 12 months. The measures below will be administered 
by an assessor blinded to group assignment. Participants will receive $25 for completing the 3-month 
assessment and $50 for completing the 6 and 12-month assessments. 
 
1.5.1 Primary Outcomes.—Weight will be measured to the nearest 0.1 kg using a digital scale; 
height will be measured to the nearest millimeter with a stadiometer using standard procedures. 
Measurements will be made in light indoor clothing without shoes. Percent weight loss will be calculated 
as: [(baseline weight – follow-up weight) ÷ baseline weight] × 100. 
 
1.6 Statistical Analysis Plan 
1.6.1 Analytic plan. Statistical analysis will follow good practices for the evaluation of randomized 
controlled trials as embodied in the CONSORT statement. Missing data will be imputed using a multiple 
imputation approach and outcome models averaged across imputations to adhere to the intent-to-treat 
principle. We will compare the sensitivity of the findings to alternative methods for handling missing data 
(see Missing Data Section below). 
As a preliminary step, demographics, baseline weight and health history will be summarized across the 
aggregate sample and compared between cells (intervention components) using Analysis of Variance 
(ANOVA) for continuous variables, chi-squared analyses for categorical variables and non-parametric 
tests as appropriate. Variables will be considered potential confounders in the subsequent models if these 
variables are correlated with the outcome under consideration (at a modest p<.30 level). The distribution 
of each of study outcome will be assessed using both parametric and graphical methods and transformed 
as necessary (e.g. log transform towards normality). Potential distributions for the outcome variables 

include normal and binomial, and zero-inflated distributions. 
Evaluation of the intervention components will involve generalized linear models that regress percent 
weight change from baseline (e.g., normal distribution & identity link function) and achievement of a ≥5% 
weight loss (binomial distribution & logit link function) on effect-coded indicators for each of the five 
intervention components and unique 2 through 5-way interactions of components. Effect coding (−1,1) is 
essential in analyzing factorial designs and results in regression weights that correspond to main effects 
and interactions. 
Effect-coded main effects and interactions are orthogonal to one another, yielding unbiased estimation of 
standard errors, and equal power for main effects and interactions for given effect sizes. Using this 
approach, individual cell sizes (for each combination of components) does not limit statistical power. 
Models will adjust for confounders identified during preliminary analyses, including demographics, health 
and weight history that are associated with weight loss outcomes. 
 
1.6.2 Missing data. Analyses will be conducted on the intent-to-treat sample (everyone randomized 
will be included in the final analysis) under various assumptions about the missing data mechanism. 
Sensitivity to these assumptions will be tested. Specifically, we will gather follow-up information and 
reasons for dropout regardless of protocol completion and censor at the point of loss. We will compare the 
robustness of our findings using three statistical approaches for handling missing data. First, we will use a 
multiple imputation approach to impute missing outcomes. Next, we will use inverse probability 
weighting with propensity scores. This is a two-step method: 1) using logistic regression, the probability 
of missingness is modeled as a function of baseline covariates and baseline values of the outcome and 2) 
the inverse of the propensity scores (predicted probabilities of dropout from the first step) serve as weights 
in our regression model of the outcomes. Provided the data are missing at random (MAR) or that the 
probability of missingness can be fully explained by observable data, this approach produces 
asymptotically unbiased estimates. To allow for the possibility that the MAR assumption may not hold, 
we will also use a third approach, pattern mixture models, in which the distribution of the outcome is 
assumed to follow a mixture of two distributions: one for those who complete follow up and another for 
those who do not. 
 
1.6.3 Sample size and power estimates. Following the full factorial design, each of the planned 
384 participants is randomly assigned to each of the five intervention components. There are 32 possible 
combinations of intervention components, but each component will be administered to 192 persons and 
withheld from 192 persons. Using the generic approach described by Lehr,64 a sample size of n = 192 per 
group permits the detection of standardized effect sizes (SEs) of Δ = (4/n^.5) (where 4 encompasses 
standard assumptions of a type-I and type-II error rate of α = .05 and β = 20%, respectively where 4 
≈ √2 × (|zα/2 + zβ |), and solves to SE = 0.29). According to estimates based on our preliminary data, RxWL 
without additional intervention components produces ≥5% weight loss for 50% of the sample, and the 
standard deviation on percent weight change will be 5.8% at 12-months. This generic approach estimates 
power to detect intervention component effects of 9% on proportion achieving ≥5% weight loss and 
effects of 1.7% on mean weight loss. Statistical power was further evaluated using Monte Carlo 
estimation, the model specification (with effect coding) for the proposed study, and parameter estimates 
based on previous RxWL trials. This tailored approach yielded estimates of 91% power to detect a 
component marginal effect size of ES = 0.16 or an additional 3% achieving ≥5% weight loss and just 
under 1% additional weight loss due to component effects. The study is adequately powered because a 
lesser effect would not be clinically significant.