A Large-Scale Observational Study of the Causal Effects of a Behavioral Health Nudge
Abstract
The Apple Watch encourages users to stand throughout the day by delivering a notification onto the users’ wrist if they have been sitting for the first 50 minutes of an hour.
This simple behavioral intervention exemplifies the classical definition of nudge as a choice architecture that alters behavior without forbidding options or significantly changing economic incentives.
In order to estimate from observational data the causal effect of the notification on the user’s standing probability throughout the day, we introduce a novel regression discontinuity design for time series data with time-varying treatment.
Using over 76 billions minutes of private and anonymous observational standing data from more than 160,000 subjects enrolled in the public Apple Heart and Movement Study from 2019 to 2022, we show that the nudge increases the probability of standing by up to 49.5% across all the studied population.
The nudge is similarly effective for participants self-identified as male or female, and it is more effective in older people, increasing the standing probability in people over 75 years old by more than 60% .
We also demonstrate that closing Apple Watch Activity Rings, another simple choice architecture that visualizes the participant’s daily progress in Move, Exercise, and Stand, correlates with user’s response to the intervention; for users who close their activity rings regularly, the standing nudge almost triples their probability of standing.
This observational study, which is one of the largest of its kind exploring the causal effects of nudges in the general population, demonstrates the effectiveness of simple behavioral health interventions and introduces a novel application of regression discontinuity design extended here to time-varying treatments.
Introduction
For people who are able to stand, prolonged sitting is known to negatively affect their health, e.g., increasing premature mortality risk [17] as well as diabetes and cardiovascular events [20].
Conversely, interrupting prolonged sitting by standing for a few minutes has been shown to limit these risks [8, 10] and even reduce postprandial blood glucose [7].
A key question is therefore how to encourage people to stand more.
Here is where the concept of nudges enters the equation.
Following the seminal work of Thaler and Sunstein [19], a nudge is any aspect of choice architecture that alters behavior in a predictable way, without forbidding options or significantly changing economic incentives.
To count as a mere nudge, the intervention must be easily avoidable.
Nudges are not mandates, e.g., putting the fruit at eye level is a nudge but banning junk food is not.
Not all nudges are successful, see [6, 16] for positive examples and [2, 14] for negative ones.
A major challenge is the design of successful nudges to improve health, and another challenge is to evaluate them with observational studies in the general targeted population (most have been evaluated with experimental design such as A/B testing). [Table]
An example of a health nudge is the Apple Watch Standing one, see Figure 1.
The Stand reminder encourages the user to stand after a long period of inactivity.
As described in the Apple Watch settings, the user will “Receive a reminder to stand if [they]’ve been sitting for the first 50 minutes of an hour.”
The notification will stay on for 10 minutes until the next hour.
With the popularity of the Apple Watch, this nudge presents a unique opportunity for studying the causal effects of such behavioral interventions in the general population.
In this work, we exploit the Apple Heart and
Movement Study [4] to address this.
In collaboration with the American Heart Association and
Brigham and Women’s Hospital, the Apple Heart and Movement Study explores, in a fully private and anonymous fashion, the link between physical activity and heart health.
The study is designed to identify factors affecting heart health, mobility, and heart function over time.
To illustrate the massive amount of data in this study, the total number of days pooling all users together reaches over 53 million.
Using sensors on the Watch, the study records standing data that we bucketed as a binary standing/non-standing variable in 5 minutes intervals, see Figure 2.
For our particular study, this data contains over 76 billions minutes of observational standing data from more than 160,000 users collected from 2019 to 2022.
We stress that this is an observational study and not a randomized control trial.
As such, it becomes one of the largest study investigating the causal effects of a behavioral health nudge.
Being an observational study, we resort to the classical econometrics tool of regression discontinuity 2 [3, 5, 13, 15], that we adapt into a novel method for time-series outcomes.
A regression discontinuity aims to determine the causal effects of interventions by identifying a threshold above or below which an intervention is assigned.
By comparing observations lying closely on either side of the threshold, it is possible to estimate the average treatment effect in environments where randomization is not available.
The Apple Watch Standing data is a time series with a time-varying treatment (the nudge) with a natural threshold (being idle for at least 50 minutes); in this paper we design a regression discontinuity strategy for time-varying treatment and exploit it to study the causal effects of the
Standing nudge.
It extends the method of regression discontinuity in time [9] which does not consider time-varying treatments (e.g. nudge, no nudge, no nudge, nudge ...)
but only units assigned to a treatment that begins at a particular threshold in time.
A regression discontinuity experimental design for time-varying treatment
Standard regression discontinuity design.
In a standard regression discontinuity design with binary treatment N and outcome Y [15], the treatment is assigned according to a function g of an additional exogenous variable Z , by N = g ( Z ) .
The function g is of the form z
(cid:55)→ g ( z ) :=( z ≥ β ) for a fixed threshold β .
The variable Z is called the forcing variable (or assignment variable) since it determines the treatment assignment.
The discontinuity of the function g at z = β allows to estimate the causal effect of the treatment N on the outcome Y around Z = β using the formula lim ϵ
→ 0 + ( E [ Y | Z
= β + ϵ ] − E
[ Y | Z = β − ϵ ]) .
A common estimation method is to further assume a parametric model
[Equation]
[Figure]
where a, b are the regression coefficients, τ is the causal effect, and ϵ is the random error.
Fitting this regression for Z around β then yields an estimation of τ .
This design only considers a fixed single treatment N instead of time-varying treatments as in our scenario.
It is therefore appealing to extend it for time-varying data, as we do next.
Dynamic treatment strategy for time-varying outcomes.
We consider a time-varying outcome variable ( Y t ) with a time-varying treatment ( N t ) ∈ { 0 , 1 } .
We assume that ( N t ) obeys to a dynamic treatment strategy f t [11, Chap.
That is, there exists a function f t of the previous outcomes that determines each treatment N t at time t as N t = f t
( Y 0 , Y 1 , ..., Y t − 1 ) .
This way, the full history of outcomes becomes a forcing variable for the next treatment.
For simplicity, and to draw a parallel with the more standard regression discontinuity, we assume that f t is of the form f t ( Y 0 , ..., Y t − 1 ) =( ζ t ( Y 0 , ..., Y t − 1 ) ≥ α t ) for some function ζ t and threshold α t .
The function ζ t projects the multivariate vector of past outcomes into a one-dimensional quantity ∆ t := ζ t ( Y 0 , ..., Y t − 1 ) , which is then compared to a threshold α t to determine the treatment N t .
We are interested in the causal effect of the treatment N t on the outcome Y t .
In the Apple Standing data in particular, Y t indicates if the user was standing at time t and N t if the user was nudged to stand at t .
We are therefore interested in the causal effect of the nudge N t on the standing status Y t .
Regression discontinuity design for time-varying outcomes.
In the time-varying outcome model with dynamic treatment strategy, the treatment is not assigned depending on a single exogenous variable Z , but it is instead assigned depending on the full outcomes history.
Nevertheless, if the assignment functions f t exhibit discontinuities just as the standard function g did, we can estimate the causal effect around the discontinuities with a formula similar to the standard regression discontinuity,
[Equation]
In practice, we further assume a parametric linear model adapted from Equation (1) and we fit
[Equation]
around the discontinuity ∆ t = α t .
The coefficients τ t for each time t estimate the causal effect of N t on Y t around ∆ t = α t from only observational data.
Finally, we define the relative causal effect as ρ t = τ t E
[ Y t | ∆ t = α t ] − τ t , which represents the relative increase in Y t due to the treatment N t .
Causal effect estimation of the standing nudge
The Stand Minutes data.
Users who choose to enroll in the Apple Heart and Movement Study,[4], privately and anonymously share their Stand Minutes data from the Health application.
This data indicates if the user is standing or sitting (idling) throughout each day with a resolution of 5 minutes.
More precisely, each day is divided into 284 intervals of 5 minutes, referred to as buckets , and for
[Figure]
i
[Figure]
i
[Figure]
r a t
[Figure]
increase in standing rate happens exactly once the idling δ becomes long enough to trigger the nudge.
each user and bucket, the Stand Minutes data indicates if the user was detected to be standing during the bucket time period.
The first bucket of the day is from 00:00:00 to 00:04:59, also written [00:00, 00:05[, and the other buckets follow successively.
We refer to buckets by their starting time, such that bucket [ t , t + 5 [ is called bucket t .
Figure 2 shows the Stand Minutes profile for a weekday stored in the Health app of one of the authors.
To form the dataset for this large-scale observational study, we pooled the data of 166,034 study participants enrolled between November 14th, 2019, and May 31st, 2022.
Days with no standing data were removed.
A total of 53,410,838 days of standing data have been accumulated across all participants.
It represents a total of 76,911,606,720 minutes and of 15,382,321,344 buckets that are individually labeled stood or idle .
The massive size of this dataset is key for applying a regression discontinuity design, which requires a large quantity of data to eventually form estimators using only data around specific discontinuities.
The time-varying model.
For each pair of user u and day d , the binary variable Y ( u,d ) t indicates if user u was standing on day d during bucket t .
The treatment variable N ( u,d ) t is not readily available in the data.
Instead, we compute it by replaying the history of standing minutes and apply the rules that are used on the Apple Watch to trigger the nudge:A notification is delivered at 50 minutes of an hour if the subject was idle during the first 50 minutes of the hour.
The notification stays on the screen until the end of the hour unless the subject eventually stands or manually dismisses it.
This mechanism corresponds to
[Equation]
where ζ t computes the number of continuous minutes spent idling (not standing) before time t , ∆ ( u,d ) t = ζ t [Equation] Y [-( u,d ) 00:00 , ..., Y ( u,d ) t − 00:05 (cid:17) = max (cid:110) k ≥ 
-][Equation]
[Equation] [Equation] [Equation]
[Equation]
[Equation] = max [Equation] ..., Y
[Equation] [Equation]
[-A b s o l u t e c a u s a l e ff e c t t Minutes
 [Figure] [Figure]-]= (
[Equation]
, 0 , ..., 0) (cid:111) , and α t = (cid:26) + ∞ if t.minutes ∈ { 0 , 5 , ..., 45 } , t.minutes if t.minutes ∈ { 50 , 55 } .
[Figure]
A b s
[Figure]
l u t e c a u s a l e ff e c t t
[Figure]
The relative increase in standing probability at 50 minutes surpasses 40% from 6:00 to 17:00.
The notification, which can stay on until 59 minutes, still has an effect at 55 minutes, meanwhile no significant causal effect is detected at 45 minutes.
Figure 3 shows the functions t (
cid:55)→ E [ Y t | ∆ t = 45] and t (cid:55)→ E [ Y t | ∆ t = 50] with t varying throughout the day.
In Figure 3a, the standing notifications are never triggered because ∆ t < 50 .
On the contrary, in Figure 3b, the notifications are delivered at 50 minutes of each hour since ∆ t ≥ 50 .
We observe a discontinuous jump/increase in standing probability at each hour and 50 minutes between (a) and (b).
The discontinuity at ∆ t = 50 can be observed in Figure 3c, which shows δ (cid:55)→ E
[ Y t | ∆ t = δ ] for different values of t .
For a t that is not at 50 or 55 minutes of an hour (remember that the data is in buckets of 5 minutes), there is no discontinuity, which is expected since no notification nudge are ever triggered before 50 minutes.
Instead, for a time t at 50 minutes of an hour (resp. 55), a discontinuity is visible at ∆ t = 50 (resp.
∆ t = 55 ), which are the thresholds above which N t = 1 .
Estimation of the causal effect.
We estimate the causal effect τ t of the notification at each hour and 50 minutes and each hour and 55 minutes, using respectively the discontinuities ∆ t =and ∆ t = 55 .
We also estimate the causal effect at 45 minutes as a control.
We know that there is no notification at 45 minutes and that no causal effect should be discovered.
The model from Equation (3) is fitted in Python as a linear model with the library statsmodels [18], using the iteratively reweighted least squares (IRLS) method [12].
We restrict the data around the discontinuities ∆ t = 50 or ∆ t = 55 by considering each outcome where ∆ t ∈ [30 , 90] (Figure 3 suggests that the linear approximation is legitimate on this interval).
We also compute the relative effects ρ t = τ t E
[ Y t | ∆ t = α t ] − τ t .
These estimations are reported in Figure 4 with 99% confidence intervals.
We find a statistically significant positive causal effect of the nudge on the standing probability.
The relative increase in standing probability reaches over 40% every hour from 6:00 to 17:00.
In Table 1 we report the relative causal effects of the nudge at 50 minutes, averaged over three periods of the day: the morning, the afternoon, and the evening.
We find that the nudge increases the standing rate by 49.5% in the morning and by 43.8% in the afternoon.
The relative increase in the evening is around 15.7%.
The nudge has a significant impact on the standing rate of the Apple Watch users, especially in the morning and afternoon.
These times of day correspond to work hours, which are, for example, moments of prolonged sitting for jobs of the service sector.
Controlling the causal effect estimation for population covariates
The Apple Heart and Movement Study contains private and anonymous information about the participants’ demographics, which can be used to measure the impact of the nudge on different sub-populations.
When the users choose to enroll in the study, they submit their age as well as their biological sex.
The age is mandatory to determine eligibility, while the biological sex is optional and can be one of Female, Male or Other.
In addition, the Apple Heart and Movement Study contains information about the users’ activity level, which we leverage to understand how behavioral differences emerge between people with different engagement with their fitness levels.
Tacking into account these sub-groups, we report the relative causal effect between 6:00 and 23:59 in Figure 5 and the relative causal effect grouped by period of day in Table 1.
[Figure]
Controlling for age.
We form seven age groups using the users reported age
Group n ∈ { 2 , 3 , 4 , 5 , 6 , 7 , 8 } contains individuals between 10 n − 5 and 10 n + 4 years old (for n = 2 we start at 18 instead of 15).
We estimate the causal effect of the nudge on each group separately and report the results in Figure 5a and Table 1.
Older individuals are more compliant with the nudge.
Controlling for biological sex.
Figure 5b shows that the nudge is similarly effective for participants self-identified as male , female , or other .
No statistically significant differences exists between the groups.
The group other exhibits large confidence intervals because it contains fewer individuals.
Moderating by the engagement with the Apple Rings.
We group users by how often they close their Exercise ring (set to a goal of 30 minutes of exercise per day for the majority of users).
Individuals in group n achieve their exercise goal between 10 n % (included) and 10( n + 1)% (excluded) of the days (group 10 completes the goal every single day).
Figure 5c shows that participants who close the Exercise ring very often are much more compliant to the Standing nudge.
We observe the same behavior with the two other rings.
Discussion and concluding remarks
In this paper we extended classical regression discontinuity to time-varying data and applied it to study the causal effects of the Standing nudge in Apple Watch.
This works constitutes one of the largest studies on causal effects of behavioral health nudges from observational data [Table] and helped to demonstrate that this simple nudge significantly and positively affects behavior.
[Table]
Acknowledgments
The authors thank Eitan Sapiro-Gheiler from MIT Economics for introducing us to regression discontinuities and discussions on its applications to this work.
We also thank Andy Miller, Nick Foti, and Shirley Ren from Apple for valuable discussion and feedback, and the members of the AHMS for their support and constructive comments on this work