Evaluating the Impact of Educational Flyers Promoting $0 Deductibles and Lower Costs With Silver Plans on California Marketplace Plan Enrollment and Care Utilization
Phase: NA
Conditions: Outreach Intervention
Sponsor: Covered California

Evaluating the Impact of Benefit Information Inserts on Health Plan Selection and 
Utilization among Renewing Members of Covered California 
 
Sulochana Paudyal, PhD, Covered California, 1601 Exposition Blvd Sacramento CA 95815  
Sulochana.Paudyal@covered.ca.gov  
  
Emily Kohn, MS MPP, Covered California 1601 Exposition Blvd Sacramento CA 95815, 
Emily.Kohn@covered.ca.gov  
  
Emory Wolf, MPP, Covered California 1601 Exposition Blvd Sacramento CA 95815  
Emory.wolf@covered.ca.gov  
  
Isaac Menashe, Covered California 1601 Exposition Blvd Sacramento CA 95815  
Isaac.Menashe@covered.ca.gov  
 
Funding 
None 
 
Conflict of Interest 
None  
 
IRB Approval Number: 2025-073 
 
Date of this document created: 05/06/2025  
Date of latest update: 08/01/2025 
 
NCT number: N/A 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Design Document and Analysis Plan 
 
Background:  
The Affordable Care Act (ACA) established health insurance marketplaces (commonly referred 
to as ACA marketplaces) to provide affordable insurance coverage. In addition to premium 
subsidies, administered through the form of advanced premium tax credits (APTCs), low-income 
consumers are eligible for reduced out-of-pocket costs when they enroll in a Silver plan through 
the ACA marketplaces. These ACA benefits, known as Cost-Sharing Reductions (CSRs), 
improve the plan generosity (through the form of lower copays and deductibles) while keeping 
premium costs at the level of a Silver plan. Traditionally, lower out-of-pocket costs are 
associated with higher premium payments, such as in the Platinum tier, whereas high out-of-
pocket costs are paired with lower premiums, as in the Bronze tier. These CSR benefits are 
available for consumers who enroll in the Silver metal tier and have an income below 250 
percent of the Federal Poverty Level (FPL).   
 
California launched its own state-funded Cost-Sharing Reduction (CSR) program in 2024, 
increasing the benefit levels for those eligible for federally funded CSRs. These plans are 
referred to as “enhanced CSR plans.” In the 2025 plan year, California expanded eligibility for 
enhanced CSR plans to all enrollees, regardless of income. This policy change improves 
affordability and accessibility by offering coverage comparable to Gold plans but with lower 
monthly premiums for all enrollees. Members already enrolled in Silver plans were automatically 
upgraded to the enhanced CSR plans, while those in other metal tiers (Bronze, Gold, or 
Platinum) had to take action to switch during the annual renewal period to benefit from these 
new plan designs. This study explores how additional communication in the form of an 
informational flyer insert about the new enhanced CSR plan benefits and their expanded 
eligibility may have helped overcome inertia, likely due to informational and awareness barriers, 
for renewing consumers.  
 
Project Objective: 
The primary objective of the study is to evaluate whether including an informational flyer insert 
with renewal notices affects consumer behavior in two key areas: 
1. Enrollment in Enhanced CSR Plans: Assess whether members currently enrolled in 
other metal tiers are more likely to switch to the enhanced CSR plan when informed 
about the new benefits. 
2. Health Care Utilization for Current Silver Plan Enrollees: Determine whether members 
already enrolled in Silver plans, who were newly eligible for enhanced silver benefits, 
increased their utilization of health care services after receiving the flyer insert which has 
information such as ‘$0 deductible’, ‘Lower monthly premiums’, ‘$0 for preventative 
visits’, ‘lower costs for primary and emergency care visits’, ‘lower medication costs’.  
This evaluation seeks to understand if targeted communication can effectively inform 
consumers about the availability of more affordable and generous health coverage options, 
thereby encouraging a shift towards plans that offer better value and accessibility. 
 
 
 
Target Population/Data:  

This evaluation will employ Covered California administrative data from around 1.8 
million Covered California members (representing about 1.2 million households) who 
renewed coverage for the 2025 plan year. The study population will exclude those 
individuals who terminated their plan in 2024, after renewal notices and the flyer were 
mailed, and were no longer active members in 2025.    
The administrative data will include demographic details, eligibility details, and 
enrollment information. We also intend to use data from Covered California’s health 
care claims database to study utilization of services such as primary care visits, 
emergency room visits, prescription fills, and out-of-pocket spending.  
 
Intervention and Randomization:  
In November 2025, in advance of the Open Enrollment period, Covered California sent 
notices to households with effectuated coverage at the end of 2024, informing them of 
their upcoming coverage renewal and encourages consumers to shop for coverage 
ahead of the next coverage year. The notice was sent to households with one or more 
individuals who were currently enrolled in a Covered California health plan.   
For this study, in addition to the standard notice, we randomly assigned households to 
receive an informational flyer to provide additional details about plan benefit changes 
through the enhanced CSR program. This insert, designed as a single-page flyer, was 
included with the notices sent to renewing households highlighting the general benefits 
of renewing membership, with an added emphasis on the specific benefits received 
when visiting a doctor.  
The project employs a randomized control trial (RCT) with a block design, segmented 
into two phases to facilitate random assignment of households to either the control or 
treatment groups.   
Study Phases  
• Phase 1 (control phase): Day 1-6, where about 100,000 households are 
randomly selected daily to receive the notice without the insert.  
• Phase 2 (treatment phase): Day 7-12 immediately following phase 1, where a 
different set of about 100,000 households were randomly selected daily to 
receive the notice with an insert.  
 
Limitation of study design and quality checks:  
Due to the inability of the notice system to administer control and treatment conditions 
on the same day, we used the existing notice sending system run by California 
Healthcare Eligibility, Enrollment and retention System (CalHEERS), which sends 
notices to a random set of cases each day stratified by region. This deviation could 
introduce some biases or systematic differences between treatment and control groups, 
as the control groups were sent notices few days before than treatment group. To 
mitigate potential biases, we will analyze the data considering the time factor. This could 
involve examining whether the outcomes vary significantly from day to day and if certain 
external factors related to timing (e.g., day of the week, holidays) might have influenced 
the results. We will also do quality checks to verify that there are no systematic 

differences between the treatment and control groups by assessing balance across 
various observable covariates (e.g., gender, age, race/ethnicity, language, and income 
as a percentage of the federal poverty level). This ensures that there are no significant 
disparities between the treatment arms. 
 
Statistical Analysis Plan: 
To analyze the impact of receiving a notice insert on two key outcomes, we will employ 
specific statistical models tailored to the nature of our hypotheses and the data 
collected.  
 
A.  The uptake of the Enhanced CSR plans by Bronze/Gold/Platinum plan members  
 
We will calculate the rate (proportion of individuals) switching to Silver plans and 
compare the rate between those who received the notice insert (treatment group) and 
those who did not receive the notice insert (control group).  
 
Hypothesis testing:  
• Null Hypothesis (H0): There will be no significant difference in the rate of 
Bronze/Gold/Platinum plan members switching to Silver plans between those 
who receive the insert and those in the control group who receive no insert.  
• Alternative Hypothesis (H1): The rate of Bronze/Gold/Platinum plan members 
switching to Silver plans will be significantly higher for those who receive the 
insert and those in the control group who receive no insert.  
 
We will perform two-tailed t-test comparing the mean rate of switching p-values <= 0.05 
to denote statistically significant effects. We will also examine whether the outcomes 
vary significantly from day to day and if certain external factors related to timing (e.g., 
day of the week, holidays) might have influenced the results. 
Subgroup Analysis: In our study, we plan to explore how plan choice behavior differs 
among different enrollee characteristics and demographics. Key subgroups for analysis 
include age groups, gender, written language preference, race/ethnicity, income level, 
net premium amount, health status, chronic condition, prior utilization, health risk score 
calculated using prior year utilization data. 
Interaction effect: To understand the combined effects of the notice insert, 
demographics subgroups, and other outreach efforts, we will use a logistic regression 
model. Given that our outcome of interest is binary (1=switching plan, 0=not-switching 
plan), a logistic regression model will be used to estimate the probability of a member 
switching from Bronze/Gold/Platinum plans to Silver plans, as influenced by the receipt 
of the insert.  
 
In (
 ) = 
 +  
X + 
 + 
 + 
Dk 
 
Where, p is the probability of switching to a Silver plan, X is a binary independent variable 
indicating whether the member received the insert (1) or did not receive the insert (0). D1 
through Dk represent all the demographic subgroups such as age, gender, race/ethnicity, 

income, premium. 𝛽0 is the intercept, 𝛽1is the coefficient for the effect of receiving the insert, 
𝛽2 − 𝛽𝑘 are the coefficient for the effect of demographics/subgroup/other outreach effort.  
 
 
B.    Increase in health care utilization among those transitioning from standard Silver to 
enhanced CSR plans, and among those who previously had an enhanced CSR plan 
coverage.  
 
We will examine data on various utilization metrics, including primary care visits, 
behavioral health visits, specialist visits, emergency room visits, hospital admissions, 
and pharmacy claims. We anticipate an increase in utilization following exposure to the 
insert that mentions benefits such as $0 deductible, $0 for preventative visits, lower 
costs for primary and emergency care visits, affordable medications as low as $5.  
• Hypothesis testing: Null Hypothesis (H0): There will be no significant difference in 
health care utilization among Silver plan members transitioning to enhanced CSR 
plans  and those remaining in standard Silver plans, irrespective of receiving the 
informational insert. 
• Alternative Hypothesis (H1): Health care utilization will significantly increase 
among Silver plan members who transition to enhanced CSR plans, compared to 
those in standard Silver plans, particularly following the exposure to the 
informational insert about enhanced benefit. 
 
We will perform two-tailed t-test comparing healthcare utilization per 1000 individuals 
with p-values <= 0.05 to denote statistically significant effects. We also plan to explore 
how different groups of people react to the health plan information we provide on plan 
benefits. Key subgroups for analysis include age groups, gender, written language 
preference, race/ethnicity, income level, average net premium, and health risk score 
calculated using prior year utilization data. 
 
Additionally, we will evaluate the metrics previously discussed to determine whether 
there is an uptick in healthcare utilization among individuals already enrolled in standard 
silver plans in 2024, aiming to assess whether increased awareness of their benefits 
influences their healthcare usage.