Evaluating Impact of Outreach Nudges Among Individuals Losing Medicaid Coverage in California
Phase: NA
Conditions: Behavioral Economics, Health Insurance
Sponsor: Covered California

1 
 
Evaluating impact of outreach nudges among individuals losing Medicaid coverage in 
California   
 
Principal Investigators 
 
Emory Wolf, MPP, Covered California 1601 Exposition Blvd Sacramento CA 95815 
Emory.wolf@covered.ca.gov 
 
Marina Lovchikova, PhD, Covered California, 1601 Exposition Blvd Sacramento CA 95815 
Marina.lovchikova@covered.ca.gov 
 
 
 
Funding 
None 
 
Conflict of Interest 
None  
 
Date of the document creation: March 19, 2024 
 
Date of the document update: June 5, 2024 
 
 
 

2 
 
Design Document and Analysis Plan 
 
Project Objective 
In May 2023, Covered California, California’s ACA marketplace, implemented its automatic 
enrollment program for individuals losing Medicaid coverage, as defined by Senate Bill 260. If 
individuals are found eligible for subsidized marketplace coverage, individuals will be 
automatically enrolled in the lowest cost silver plan available to them, but individuals must still 
take action to confirm their plan or pay their first month’s premium. To identify whether the 
phone outreach is the effective outreach strategy to increase take-up in this population, Covered 
California implemented this intervention. The results of this evaluation will inform operational 
and budgeting decisions. 
 
 
Target population 
The experimental population includes an estimated 60,045 households among consumers who 
have been discontinued from Medi-Cal, California’s Medicaid program, and found eligible for 
subsidized marketplace coverage. 
 
Interventions and Randomization 
 
We evaluate the effectiveness of phone outreach on health insurance take-up among auto-
enrolled households given that this is a new population for Covered California.  
 
The project's research design is a block-randomized intervention among automatically enrolled 
households.  We separate households by preferred spoken language and net premium of the 
assigned health plan and randomly assign households within each on continuous basis to (1) a 
control group with a standard eligibility notice for auto-enrolled households, or to (2) a treatment 
group to receive additional phone outreach from Covered California’s service center.  
 
 
Table 1. Approximate sizes of control and treatment groups for the evaluation 
 
Cohort month 
Control 
Treatment 
August 2023 
750 
500 
September 2023 
1,698 
5,365 
October 2023 
7,466 
3,132 
November 2023 
1,838 
4,935 
December 2023 
487 
5457 
May 2024 
6,696 
2,021 
June 2024 
8,000 
2,100 
July 2024 
8,000 
2,100 
Total 
30,101 
29,944 
 
Power: 
To arrive at an estimate for the minimum detectable effect (MDE) for our primary outcome, we 
assume a baseline health insurance take up of 6 percent. The baseline rate was determined 
with the historical data on the plan effectuation of individuals disenrolled from Medi-Cal in 2018-

3 
 
2022 years. The intervention was powered at the 80% level to detect a 0.6 percentage point 
increase in effectuation rate1. 
 
 
Likely Effect Size: 
Based on prior phone outreach campaigns carried out by Covered California, we would expect 
to observe an ITT effect between 0 and 4 percentage points.   
 
Data and Data Structure 
Outcomes:  
We use Covered California administrative data to examine health insurance take-up behavior 
and health care utilization among households included in the study.  
 
The primary outcome of interest will be a binary indicator for whether a consumer is effectuated 
into Covered California plan, either the plan assigned through the auto-enrollment process, or 
into another plan chosen by the consumer by the end of the month at which outreach was 
conducted.  
 
Our secondary outcomes will be measured 1-2 years after the intervention when data becomes 
available will include: 
1. Tenure with Covered California (measured in months); 
2. A binary indicator for whether someone had an office visit during 1-2 years since the 
intervention; 
3. A binary indicator for whether someone had a prescription drug fill during 1-2 years since 
the intervention; 
4. A binary indicator for whether someone had an emergency room visit during 1-2 years 
since the intervention; 
5. A binary indicator for whether someone had a hospital admission 1-2 years before and 
1-2 years after the intervention. 
6. Total out-of-pocket spending for the 1-2 years since the intervention  
7. A binary indicator for whether someone self-reported being uninsured while filing their 
taxes; 
 
 
Data: 
We will use Covered California administrative data to obtain enrollment outcomes and baseline 
demographics for our sample and health care claims database to obtain health care utilization 
and financial outcomes.  
 
We will use data from the Department of Health Care Access and Information for hospital 
admissions data for all individuals in the intervention. We will use data from the California 
Franchise Tax Board to measure which individuals reported not having Minimum Essential 
Coverage (MEC) while filing their state taxes.  
 
 
Quality Control Checks: 
After carrying out the randomization, we will check for balance across several observable 
covariates (e.g., gender of head of household, presence of email address on the application, 
 
1 Authors’ calculations based on a two-sample proportion Pearson’s chi-squared test using Stata 18 SE 
software 

4 
 
age, and income as a percent of the federal poverty level), which could indicate that there would 
be no significant dissimilarities across treatment arms.  
 
 
Statistical Models 
Intention-to-treat estimation: To estimate treatment effects, our primary analysis will be an 
intent-to-treat (ITT) specification, examining the effect of treatment assignment. We will estimate 
the effect of the treatment using ordinary least squares (OLS) regression. That is, we will 
regress the outcome of interest (e.g., take-up) for household i on the treatment indicator:  
outcomei = ∝+ β1Treatmenti + εi 
 
The coefficient β1 will be the intent-to-treat estimate of the causal effect of the outreach.  
 
Complier average causal effect estimation: We will examine the effects of health insurance 
(among households induced to enroll in a Covered California as a result of random assignment) 
on our four utilization outcomes. We anticipate a significant share of noncompliers among those 
households assigned to receive outreach (intervention) as they may have opted out of assigned 
plan or not enroll into another Covered California plan. Thus, to augment our analysis of 
insurance take-up, we will also estimate treatment effects based on treatment receipt 
(enrollment into Covered California Plan), using two-stage least squares regression (2SLS).  
 
Pre-Specified Subgroup Analyses 
We will examine treatment heterogeneity by race and ethnicity of the head of household, written 
or spoken language, gender of head of household, income group measured by the percentage 
of the federal poverty level, net premium of assigned plan, age bracket (30 years and younger, 
31-49 years old, older than 50 years old), health status as measured by HCAI discharge data, 
mixed eligibility household status. Same subgroup analyses will be performed for our complier 
average causal effect analysis.  
 
Inference Criteria, Including Any Adjustments for Multiple Comparisons: 
We will not perform any corrections for multiple hypothesis testing, and we will use two-tailed 
tests with p-values <= 0.05 to denote statistically significant effects.