Encouraging Healthy and Sustainable Dietary Substitutions: A Randomized Experiment With Emerging Adults
Phase: NA
Conditions: Food Selection
Sponsor: Stanford University

Complete Title: Encouraging Healthy and Sustainable Dietary Substitutions 
Short Title: Encouraging Healthy and Sustainable Dietary Substitutions 
Protocol Date: September 22, 2023 
NCT #: NCT06175026 
 
Study Principal Investigator:  
Anna H. Grummon  
Department of Pediatrics 
3145 Porter Drive, A103 
Palo Alto, CA 94304 
Phone 650-644-7366  
email: agrummon@stanford.edu  
 
 
 
 

Study Title 
Encouraging Healthy and Sustainable Dietary Substitutions 
Funder 
National Heart, Lung, and Blood Institute 
Clinical Phase 
NA 
Study Rationale 
• The foods people eat have major implications for both human 
health and environmental sustainability. Approximately 20% 
deaths in the US are attributed to unhealthy diet and more than 
30% of all anthropogenic greenhouse gas emissions are caused by 
food production. 
• Recently, researchers have identified four simple dietary 
substitutions that are healthier (i.e., they would improve diet 
quality and thus reduce risk of diet-related disease) and more 
environmentally sustainable (i.e., they would reduce carbon 
emissions associated with food production): replacing beef or 
pork entrees with chicken or vegetarian entrees; replacing juice 
with whole fruit; replacing dairy milk with non-dairy alternatives 
like soy or almond milk; and replacing sugar-sweetened beverages 
with water. 
• However, it remains unknown how to encourage people to adopt 
these healthy and sustainable dietary substitutions. Messages 
about health, the environment, or both health and the environment 
could encourage these substitutions, but limited research has 
tested these messages against one another.  
• Emerging adults merit special attention in research on healthy and 
sustainable dietary substitutions because they have worse dietary 
quality than older adults and because their dietary behaviors track 
into later adulthood. 
Study Objective(s) 
Primary  
• To evaluate whether health, environment, or health + environment 
messages elicit higher perceived message effectiveness than 
control messages.  
 
Secondary 
• To evaluate whether health, environment, or health + environment 
messages elicit higher perceived message effectiveness than one 
another.   
Study Design 
 
Randomized experiment.  
Subject Population Inclusion Criteria 
1. Age 18-25 

key criteria for 
Inclusion and 
Exclusion: 
2. Reside in the United States 
3. Able to complete a survey in English 
Exclusion Criteria 
1. Under the age of 18 or over the age of 26 
2. Reside outside of the United States 
3. Unable to complete a survey in English 
Number of 
Subjects  
1,600 
Study Duration 
Each subject’s participation will last approximately 10 minutes. The 
enrollment period is expected to last ~1-2 weeks. 
Study Phases 
  
There are two phases: 
(1) Screening: screening for eligibility and obtaining consent and  
(2) Intervention: study intervention/experimental treatment. 
Efficacy 
Evaluations 
The primary outcome is perceived message effectiveness for encouraging 
healthy and sustainable dietary substitutions. It will be measured with 4 
items adapted from prior studies.  
Statistical and 
Analytic Plan 
Primary outcome 
• We will use linear mixed models to examine the effect of each 
intervention topic (health, environment, or health + environment) 
on perceived message effectiveness compared to the control.  
 
Secondary outcomes 
• We will use linear mixed models to examine the effect of each 
intervention topic on perceived message effectiveness compared 
to the other intervention topics.  
 
Data and Safety 
Monitoring Plan 
• The principal investigator is responsible for data quality 
management and ongoing assessment of safety.   
 
 

Introduction 
The goal of the analyses described here is to use data we collected through an online 
randomized experiment to examine consumer responses to different messages about healthy and 
sustainable dietary substitutions. This analysis examines the effect of different message topics 
(control vs. health vs. environment vs. health + environment) on perceived message effectiveness 
for encouraging the healthy and sustainable dietary substitutions. This analysis plan pre-specifies 
the analyses before collecting data and therefore serves as our ex-ante planned analysis.  
 
Study Protocol 
Participants will complete an online survey programmed in Qualtrics. After providing 
informed consent, participants will complete 4 experiments in random order: 1 for each of 4 
target dietary substitutions (beef and pork, juice, milk, and sugary drinks). Participants will be 
randomized to 1 of 4 between-subjects experimental arms representing different message topics: 
a control arm or 1 of 3 intervention arms: health, environment, or health + environment.  
Participants will be in the same experimental arm for all 4 experiments. In each of the 4 
experiments, participants will view 2 messages from their assigned arm in random order and rate 
each message. (There will be 3 possible health, environment, and health + environment messages 
for each of the 4 target dietary substitutions. Participants in these experimental arms will view 2 
of these 3 messages, selected at random. There are only 2 possible control messages; participants 
in the control arm will view these 2 control messages. In all experimental arms, the 2 messages 
will be shown in random order). In total, participants will rate 8 messages: 2 messages for each 
of the 4 experiments.  
 
Objective 
The objective of this study is to examine the effect of message topic on perceived 
message effectiveness for encouraging healthy and sustainable dietary substitutions. We will 
compare dietary substitution messages focused on health, the environment, both health + 
environment, or a control topic. We hypothesize that messages about health, the environment, or 
both health + environment will receive higher perceived message effectiveness ratings than 
messages about the control topics.  
 

Statistical Considerations 
General Principles  
We will use a two-sided critical alpha of 0.05 to conduct all statistical tests. All 
confidence intervals presented will be 95% and two-sided. Analyses will include all randomized 
participants according to experimental conditions they were randomized to (i.e., intent-to-treat). 
Participants will be prompted to respond to all survey questions before they can continue, so we 
expect there will be minimal missing data and therefore plan to use complete case analysis.  
Overview 
Participant characteristics will be descriptively summarized by experimental arm prior to 
analysis. Means and standard deviations will be used to characterize continuous variables, and 
frequencies and percentages will be used to describe categorical variables.  
Primary Outcome 
The primary outcome is perceived message effectiveness (PME). We will assess PME 
with 4 items: 3 items adapted from Baig1 and 1 item adapted from Jongenelis.2 The items will be 
tailored to each of the 4 target dietary substitutions (beef/pork, juice, milk, and sugary drinks), as 
shown below:  
1. How much does this message discourage you from wanting to [eat beef and pork / 
drink juice / drink cow's milk / drink sugary drinks]? 
2. How much does this message make [eating beef and pork / drinking juice / 
drinking cow’s milk / drinking sugary drinks] seem unpleasant to you? 
3. How much does this message make you concerned about the effects of [eating 
beef and pork / drinking juice / drinking cow’s milk / drinking sugary drinks]? 
4. How much does this message make you want to [eat chicken or vegetarian 
options instead of beef and pork, eat whole fruit instead of drinking juice / drink 
non-dairy milk instead of cow’s milk / drink water instead of sugary drinks]?  
Responses options for each item are presented as a Likert-type scale: Not at all (coded as 
1); Very little (2); Somewhat (3); Quite a bit (4); A great deal (5). We will average responses to 
these 4 items to create a PME score for each message, assuming sufficient internal consistency 
(Cronbach’s alpha≥.70).  
Statistical Analyses 

We will run separate analyses for each of the 4 experiments (i.e., for each of the 4 dietary 
substitutions). For each experiment, we will conduct the following analyses: 
1. First, primary analyses will use mixed effects linear regression models to examine 
whether the intervention topics receive higher PME ratings than the control topic. 
We will regress PME on indicator variables for the experimental arms (health, 
environment, or health + environment, treating the control as the referent group). The 
coefficients on the indicator variables for the intervention topics are average differential 
effects (ADEs) indicating the difference in mean PME between the intervention topic 
messages and the control topic messages. Our hypothesis will be supported if the 
coefficients on the indicator variables for the intervention topics are positive and 
statistically significant. For these planned primary analyses, we will not correct p-values 
for multiple comparisons. 
2. Second, secondary analyses will used mixed effects linear regression models to examine 
whether the intervention topics receive different PME ratings than one another. We 
will use the same model as in (1) to compare the ADEs for each intervention topic to one 
another using Chi-Squared tests. These analyses will correct p-values for multiple 
comparisons using the Bonferroni-Holm procedure, considering 3 tests to compare each 
intervention topic to the others.  
3. Third, we will descriptively rank the messages. We will estimate mean PME for each 
message statement and rank those means. These analyses will be descriptive (i.e., we do 
not plan to perform hypothesis tests).  
Exclusions 
Participants will be excluded if they are younger than 18 years old or older than 25 years 
old, do not complete the survey, or complete the survey implausibly quickly, defined as 
completing in <1/3 median survey completion time in a soft launch of the survey. 
Sample Size and Power 
We used G*Power 4 to estimate sample size needs to detect an effect of the health, 
environment, or health + environment messages vs. the control messages using a repeated 
measures ANOVA examining between-subjects factors. We estimated power assuming an 
alpha=0.05, correlation among repeated measures of 0.5 (a conservative estimate based on prior 
studies3,4), and 2 repeated measures (reflecting that participants will rate 2 messages for their 

assigned topic in each of the 4 experiments). Under these assumptions, a sample size of 400 
participants per experimental arm (1,600 participants total) will yield 90% power to detect a 
small standardized effect of Cohen’s f=.10 (or Cohen’s d=.20) or larger. To our knowledge, there 
are not similar studies of dietary substitution messages, so we powered to detect a small 
standardized effect size.5 The planned sample size will also yield 90% power to detect effects of 
Cohen’s f=.11 (or d=.22) or larger for comparing each intervention topic vs. the other 
intervention topics, assuming an alpha=0.0167 (i.e., 0.05 divided by 3, to correct for multiple 
comparisons).  
 

References 
1. Baig SA, Noar SM, Gottfredson NC, Boynton MH, Ribisl KM, Brewer NT. UNC Perceived 
Message Effectiveness: Validation of a brief scale. Ann Behav Med. 2019;53(8):732-742. 
doi:10.1093/abm/kay080 
2. Jongenelis MI, Drane C, Hasking P, et al. Development and validation of the Alcohol Message 
Perceived Effectiveness Scale. Sci Rep. 2023;13(1):997. doi:10.1038/s41598-023-28141-x 
3. Grummon AH, Ruggles PR, Greenfield TK, Hall MG. Designing effective alcohol warnings: 
Consumer reactions to icons and health topics. Am J Prev Med. Published online 2022. 
doi:https://doi.org/10.1016/j.amepre.2022.09.006 
4. Grummon AH, Hall M, Taillie L, Brewer N. How should sugar-sweetened beverage health 
warnings be designed? A randomized experiment. Prev Med. 2019;121:158-166. 
5. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Academic press; 2013.