Effect of Cash Benefits on Health Care Spending
Phase: NA
Conditions: Health Care Costs, Health Care Utilization
Sponsor: Harvard Medical School (HMS and HSDM)

1 
 
Cash Benefits and Health Care Spending 
Protocol and Analysis Plan 
 
November 15, 2024 
 
ClinicalTrials.gov Identifier: NCT07381049 
 
 

2 
 
Table of Contents 
 
Section A. Introduction …………………………………………………………………… 
3  
 
 
Section B. Treatment ……………………………………………………………………… 
3  
 
 
Section C. Randomization and Participant Eligibility …………………………………… 
3  
 
 
Section D. Data, Study Sample, and Outcomes …………………………...……………… 
5  
 
 
Section E. Statistical Analysis ……………………………………………….…………… 
6  
 
 
Section F. References ……………………………………………………………………… 
8 
 
 
 
 
 
 
 

3 
 
A. Introduction 
During the first two years of the COVID-19 pandemic, the City of Chelsea, Massachusetts held a 
lottery to allocate monthly cash benefits to its low-income residents. Prior work demonstrated 
that, compared to the control group, individuals who received the cash benefit had significantly 
fewer visits to the emergency department, including those resulting in admission to the hospital, 
and higher use of outpatient subspecialty care.1 By decreasing the demand for more expensive 
acute care relative to outpatient care, cash benefits have the potential to be cost saving to the 
health care system. Using health care claims data and the random assignment embedded in the 
lottery, we propose to study the impact of cash benefits on health care spending. This proposal 
and analysis plan detail the intervention, randomization, and data source. Importantly, while this 
analysis plan is specified after the conduct of the trial, it pre-specifies the outcomes and 
statistical analysis prior to analysis of the data.  
 
B. Treatment 
The City of Chelsea – a low-income, majority Latino and immigrant community near Boston, 
Massachusetts – was disproportionately affected by the health and economic consequences of the 
COVID-19 pandemic. By the end of 2020, the city had the highest COVID-19 cumulative 
incidence rate in Massachusetts, and the population of Chelsea was heavily concentrated in 
sectors of the economy that shut down. In April 2020, local community organizations and the 
City responded to the economic crisis facing jobless Chelsea residents by mounting an 
unprecedented food distribution effort. In September 2020, after five months of running its food 
distribution sites, the City redirected its efforts toward distributing financial support so that 
residents could purchase their own food through a program called Chelsea Eats. By combining 
city general revenues, state funds, and philanthropic contributions, the City assembled enough 
resources to distribute and replenish the debit cards monthly for a total of six months, that was 
later extended to nine months, with the first disbursement on November 24, 2020. The card 
amounts varied with household size, with most applicants receiving $400 per month. Spending 
from the debit cards was not restricted to food and could be spent on anything and anywhere 
Visa was accepted.  
 
C. Eligibility and Randomization 
To be eligible, one had to be a resident of Chelsea and have a family income below 30% of the 
U.S. Department of Housing and Urban Development’s median income for the Boston 
metropolitan statistical area (i.e., $26,850 for one-person households, $30,700 for two-person 
households, $34,550 for three-person households, $38,350 for four-person households, $41,450 
for five-person households, $44,500 for six-person households, $47,600 for seven-person 
households, and $50,650 for eight-person households).  
Between July 27, 2020 and August 17, 2020, the City accepted applications for the debit cards. 
Only one individual was permitted to apply on behalf of his or her family. Multilingual 
information about the cards was distributed to individuals using the city-run food distribution 

4 
 
sites. Additionally, information and applications were disseminated to community-based 
organizations, food pantries, faith groups, health care organizations, and low- and moderate-
income housing complexes. Direct outreach to residents by city staff occurred in multiple 
locations, including food pantry lines, COVID-19 testing lines, and social service agencies. 
Applications could be submitted online via the city’s website or on paper by dropping them off 
or mailing them to City Hall. In practice, most applications were submitted by residents 
attending one of the various city or community partner food pantries, where city staff, equipped 
with tablets, assisted residents with the application.  
The City announced that, “Families with children, disabled residents, veterans, seniors over 65, 
households that have been economically impacted by COVID-19, and households that are 
ineligible for other forms of federal assistance, such as SNAP and WIC, will have preference” in 
the lottery. These preferences were implemented by giving applicants a different number of 
lottery “tickets” depending on how many criteria they met. The eight criteria are: (1) no one in 
the family was currently working; (2) no one in the family was receiving income support through 
unemployment insurance; (3) the applicant was not receiving food assistance (e.g., from the 
Supplemental Nutrition Assistance Program); (4) the applicant was disabled; (5) the applicant 
was a veteran; (6) there was a family member who was under 6 years of age living at the same 
address; (7) there was a family member between the ages of 6 and 17 living at the same address; 
and (8) there was a family member over 65 years of age living at the same address. Every 
applicant received an initial ticket and then applicants received additional tickets for meeting 
each of up to eight additional criteria.  
Because the City anticipated demand among eligible residents would exceed funding, the City 
decided to allocate the debit cards via lottery from a list of applications. Households who had 
multiple applications in the lottery were excluded from the study population, and we account for 
the differential probability of winning the lottery across applicants in our statistical analysis. The 
lottery occurred at 5:25pm on September 17, 2020, and was broadcast live on the City’s 
Facebook page.  
At the time of the lottery, the City created a randomized waitlist that was not initially publicized 
but could be used to allocate unused cards in case any lottery winners were determined to be 
ineligible or did not claim their cards. Additional applicants from the waitlist who were offered 
debit cards are included in the treatment group. While these individuals did not receive their 
cards until December, their cards were credited with both the November and December 
payments, so the total payments received by waitlist households are the same as initial lottery 
winners, and their outcomes are assessed over the same intervention period as the other 
applicants in the treatment and control groups.  
In the event that individuals reported lost debit cards, their cards were replaced so that they could 
continue to receive funds going forward but funds for prior months were not restored. The 
payments were initially intended to run for six months. Given the ongoing economic and public 
health difficulties in Chelsea, the City and its philanthropic partners extended the program for an 
additional three months. The total cost of the program was approximately $9 million. 

5 
 
D. Data, Study Sample, Outcomes, and Other Variables 
D.1. Data and Study Samples 
In Massachusetts, Medicaid and Children’s Health Insurance Program (CHIP) are combined into 
one program called MassHealth, which provides health benefits to qualifying children and adults. 
Participants in the study will be linked to their MassHealth claims using a probabilistic algorithm 
based on name, date of birth, gender, address, and phone number. The health care claims data 
will include that of the participants in the study and other MassHealth members in the 
participants’ families. Our primary study sample consists of the study participants. There are two 
additional/secondary study samples. The first will be children of the study participants, and the 
second if the data allows will be other adults in the household. 
 
D.2. Planned Outcomes 
Outcomes will be assessed on a per person basis and measured using the health care claims data 
over the nine months of the trial in the primary analysis, and exploratorily after discontinuation 
of the intervention. 
1. Primary Outcome 
a. Health care spending for medical services 
i. 
For this outcome, we will use the total payments to providers for 
medical services, including the portions paid by the enrollee and the 
insurer. We expect that out-of-pocket payments to be low in Medicaid 
and, using the claims data, will confirm that this outcome thus represents 
spending primarily by the insurer. 
ii. 
The primary outcome will first be assessed among the participants 
themselves. On an exploratory basis and as the data allows, we will 
assess this outcome among the secondary study samples – children and 
other adults in the household – to assess for spillover effects within the 
household. 
 
2. Secondary Outcomes 
a. Health care spending for medical services disaggregated by setting/specialty 
i. 
Specifically, we will assess spending in the acute care setting, including 
the emergency department and hospitalizations, and outpatient setting, 
including primary care, urgent care, and subspecialty care. 
ii. 
As an exploratory analysis to inform interpretation of the results, we will 
also assess utilization of these services as count measures.  
b. Health care spending for prescription drugs 
i. 
For this outcome, we will use the total payments for prescription 
services, including the portions paid by the enrollee and the insurer. We 
expect that out-of-pocket payments to be low in Medicaid and, using the 
claims data, will confirm that this outcome thus represents spending 
primarily by the insurer.  

6 
 
ii. 
As an exploratory analysis to inform interpretation of the results, we will 
explore total health care spending that includes both medical and drug 
spending, as well as utilization and adherence (i.e., proportion of days 
covered) of specific classes of medications for acute and chronic care. 
 
D.3. Patient Characteristics 
The data includes information on participants’ household size, disability status, veteran status, 
work status, income, receipt of other benefits or assistance, and utilization/spending prior to 
randomization. These characteristics come from the lottery application form or health care 
claims data and are assessed prior to randomization. In the event that there are high levels of 
missingness for any key covariate (i.e., >2%), multiple imputation methods will be used. 
 
E. Statistical Analysis 
E.1. Evaluation of randomization, balance, and attrition 
We will test for balance between treatment and control based upon observable baseline 
characteristics for the overall study population.  
Because differential attrition correlated with treatment, for example due to death, could introduce 
bias into our results, we will also evaluate the attrition rate and assess for balance between 
treatment and control based upon both baseline characteristics for the final analytic sample (and 
attritors) and potential causes of attrition. We will also assess linkage rates and enrollment 
changes to help with interpretation of the results. 
 
E.2. Statistical specification 
Our primary analytic approach is an analysis based on the intent-to-treat principle that compares 
outcomes for those randomized into the treatment group to those who were randomized into the 
control group. Specifically, we will estimate the following linear regression model: 
𝑦! = 𝛽" + 𝛽#CASH! + 𝛽$𝐗! + 𝜖!, 
where 𝑦! is the utilization or clinical outcome for individual or household 𝑖. See section D.2. for 
a list of our primary and secondary outcomes. “𝐶𝐴𝑆𝐻!” is an indicator for whether individual 𝑖 
won the lottery and was thus randomized into the treatment group. 𝑿𝒊 is a vector of covariates, 
specifically patient characteristics, including baseline (pre-randomization) values of the outcome 
variable, which are not explicitly necessary since they should be unrelated to treatment status, 
but they may increase the precision of our estimates to the extent that they explain some of the 
variance in the outcome. 
The coefficient on CASH!, 𝛽#, is our main coefficient of interest; it provides the difference in 
means between the treatment groups and the control group. In addition to our primary 

7 
 
specification above that corresponds to testing for a level shift, we will also include an 
interaction term between an indicator for time and treatment status to test for changes in slope. 
As described in section C, a household could receive additional lottery tickets by meeting 
specific criteria. Observations will thus be weighted by the inverse probability of winning the 
lottery so that 𝛽# is an unbiased estimate of the relationship between winning the lottery and the 
outcome. An alternative specification that includes indicators for number of lottery tickets is 
discussed below. Standard errors will be adjusted for heteroskedasticity.  
 
E.3. Subgroup analyses 
To examine heterogenous effects of the cash benefits, we will repeat our analyses for pre-
specified subgroups, each defined based on data from the pre-randomization period. The 
subgroups will be based on: (1) parental survey participation (given prior work demonstrating 
important heterogeneity by participation in the baseline survey)1, and (2) health care spending at 
baseline. 
 
E.4. Alternative specifications and sensitivity analyses 
Our primary specification includes patient characteristics and baseline values of the outcome in 
the model to improve power as well as any chance imbalance between the study arms after 
randomization. As described in section E.1., we will compare covariates between the treatment 
and control groups, and we will explore whether our results are sensitive to inclusion of these 
covariates in the model. Importantly, our primary specification accounts for differential 
probability of winning the lottery using inverse probability weights. As an alternative approach, 
we will include the set of covariates that are correlated with treatment probability into the model, 
specifically indicators for number of lottery tickets.  
To ensure our estimates are robust to method of estimation, we will also estimate generalized 
linear models with a gamma distribution and log link for the continuous outcomes. As additional 
robustness checks, we will log transform and Winsorize the measures to ensure that our 
estimates are not sensitive to outliers. 
Finally, the health care spending represented in claims may not represent the true resource costs 
of the services delivered. Therefore, as a supplemental analysis, we will adjust the amounts using 
cost-to-charge ratios and Medicare reimbursement levels. 
 
E.5. Statistical significance and adjustments for multiple comparisons 
Statistical significance was defined as two-sided P<0.05 for the primary outcome. Because we 
have one prespecified primary outcome, we will not make any adjustments for multiple 
inference. For our secondary outcomes, we will the Benjamini-Hochberg procedure or similar to 
calculate adjusted p-values that account for testing of multiple outcomes. 

8 
 
 
F. References  
1. Agarwal SD, Cook BL, Liebman JB. Effect of Cash Benefits on Health Care Utilization 
and Health: A Randomized Study. JAMA. 2024;332(17):1455-1463. 
doi:10.1001/jama.2024.13004