Assessing the Impact of myHealth Rewards Program-related Communications on Enrollment: Replication
Conditions: Health Promotion, Wellness Program
Sponsor: Geisinger Clinic

 
1 
Assessing the Impact of myHealth Rewards Program-related Communications on Enrollment: 
Replication 
(NCT04289623) 
 
Study Protocol with Statistical Analysis Plan 
 
June 17, 2021 
 

 
2 
 
Study Protocol 
 
Purpose 
 
The purpose of the study was to evaluate, prospectively, the potential impact of 
different email message conditions (standard promotional email, loss frame, endorsement from 
a medical expert, endorsement from a rank-and-file employee) on enrollment in a wellness 
program (myHealth Rewards) by Geisinger Health Plan (GHP) members. The current study was 
developed to test whether a loss frame email was more effective than a standard reminder 
email (replicating an earlier finding) in increasing myHealth Rewards login and enrollment rates.  
It was hypothesized that the loss frame email would increase enrollment compared with the 
standard email. The study also compared testimonials from a medical expert and a rank-and-file 
employee. This particular comparison was exploratory, and we did not have an a priori 
hypothesis on the performance of these emails. 
 
Methods 
 
Sample 
 
Our population consisted of benefits-eligible Geisinger employees, excluding anyone 
who registered for the 2020 mHR program prior to the Wave 1 send date (February 25, 2020). 
 
From the gathered data, we dropped 702 employees did not have registration data (i.e., 
no confirmation if a registration did or did not occur), 506 employees who were not sent any 
email, and 68 employees who registered before the study period began. 
 
After exclusions, there were 13,546 employees eligible for the study, with 8,143 who 
previously participated in the program and 5,403 who never participated (see Table 1).  In each 
of these subgroups, employees were randomly assigned to one of two conditions in Wave 1 
(standard email vs. loss frame) Two weeks after the Wave 1 emails were sent, there were 
12,216 eligible employees who did not yet register, with 6,963 having been enrolled previously 
and 5,253 never having been enrolled. In each of these subgroups, employees were randomly 
assigned to one of two conditions to receive reminder emails in Wave 2 (peer endorsement vs 
expert endorsement). 
 
Control and experimental conditions 
 
All employees received an email, which began by addressing recipients by their first names, 
with subject lines and messages varying as shown below. In Wave 1, employees received either 
the standard or the loss frame email. In Wave 2, employees who did not enroll after Wave 1 
received either the expert endorsement or peer endorsement email. 
 

 
3 
Wave 1: Standard Email. The standard email had the subject line “myHealth Rewards 
registration is now open. Sign up to improve your health and save money.” As with the Study 1 
email, it mentioned the average premium savings, the speed and ease of starting the 
enrollment process, and the deadline for registering and having health measures on file.  The 
standard email also provided two text links for registering and finding free health screenings 
where health measures could be collected and registered at one convenient time and location. 
It also mentioned an additional financial reward – early-bird enrollees get entered to a weekly 
drawing for a $20 gift card and will be entered into a final grand prize drawing for one of three 
prizes worth $500. This group served as the active control group. 
 
The other emails in this and subsequent waves contained and built upon all of the same 
information included in the standard email. 
 
Wave 1: Loss frame email. The loss frame email had the subject line “Why are you 
throwing away $2,227?” It suggested that employees were currently "throwing away" a precise 
dollar amount by not participating and that they could therefore avoid missing out on 
substantial gains (i.e., savings) by taking action. 
 
Wave 2: Expert endorsement email. The expert endorsement email (see Figure 11) had 
the subject line “How myHealth Rewards changed my life.” It provided a testimonial from a 
Geisinger doctor and emphasized the simplicity and ease of taking the first step toward 
enrollment. The personal endorsement from a doctor presented a medically-trusted exemplar 
that indicated that the behavior was desirable and feasible. 
 
Wave 2: Peer endorsement email. The peer endorsement email (see Figure 12) had the 
subject line “How myHealth Rewards changed my life.” It provided a testimonial from a 
Geisinger customer care specialist and emphasized the simplicity and ease of taking the first 
step toward enrollment. The personal endorsement from a rank-and-file presented a relatable 
exemplar – relative to many Geisinger staff who are not doctors – that indicated that the 
behavior was desirable and feasible. 
 
Wave 3. A planned third wave that compared social norms presented in numeric or 
percentage formats was not conducted due to COVID-19-related changes in the mHR program. 
 
Outcome measures 
 
The outcome measures included (1) enrollment in the mHR program, (2) clicking on the 
mHR link in the email, and (3) opening the mHR email within each wave’s two-week duration, 
starting from the date the emails were sent. Wave 1 started on February 25 and ended on 
March 9, 2020. Wave 2 started on March 10 and ended on March 23, 2020. All measures were 
binary. 
 
Enrollment was the primary outcome of interest, since it reflected the terminal behavior 
that could lead to savings and health outcomes improvements.  However, the clicks to the mHR 

 
4 
link were also of interest, as a proximal outcome and potential mediator. For Wave 1, opening 
the mHR email was also a proximal indicator of how effective the message was, given that the 
subject line also varied by condition; in Wave 2, the subject lines were the same regardless of 
condition. 
 
Statistical Analysis Plan 
 
We conducted separate, parallel analyses for those who previously participated and 
never participated in the program. We also conducted separate analyses for waves 1 and 2. 
Binary logistic generalized linear models (GLMs) were used to analyze (1) enrollment and (2) 
click rates as a function of experimental condition. Odds ratios (ORs) were calculated, along 
with 95% confidence intervals (CIs); two-tailed p-values < 0.05 were used to determine 
statistical significance.  To represent effect size, the standardized statistic, Cohen’s d, was 
estimated using the formula 𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿 × √3 𝜋𝜋
⁄  (Hasselblad & Hedges, 1995). Raw 
percentages with 95% CIs were also presented in graphs. All analyses were conducted in R.