Evaluation of the Healthy Incentives Pilot (HIP)

FINAL REPORT

Food and Nutrition Service

September 2014

U.S. Department of Agriculture
Mention or display of companies or commercial products does not imply recommendation or 

endorsement by the U.S. Department of Agriculture over others not mentioned or shown. 

Non-Discrimination Policy 

The U.S. Department of Agriculture (USDA) prohibits discrimination against its customers, 

employees, and applicants for employment on the bases of race, color, national origin, age, disability, 

sex, gender identity, religion, reprisal, and where applicable, political beliefs, marital status, familial 

or parental status, sexual orientation, or if all or part of an individual's income is derived from any 

public assistance program, or protected genetic information in employment or in any program or 

activity conducted or funded by the Department. (Not all prohibited bases will apply to all programs 

and/or employment activities.) 

If you wish to file a Civil Rights program complaint of discrimination, complete the USDA Program 

Discrimination Complaint Form, found online at http://www.ascr.usda.gov/complaint_filing_ 

cust.html, or at any USDA office, or call (866) 632-9992 to request the form. You may also write a 

letter containing all of the information requested in the form. 

Send your completed complaint form or letter to us by mail at U.S. Department of Agriculture, 

Director, Office of Adjudication, 1400 Independence Avenue, S.W., Washington, D.C. 20250-9410, 

by fax (202) 690-7442 or email at program.intake@usda.gov. Individuals who are deaf, hard of 

hearing or have speech disabilities and wish to file either an EEO or program complaint please 

contact USDA through the Federal Relay Service at (800) 877-8339 or (800) 845-6136 (in Spanish). 

Persons with disabilities who wish to file a program complaint, please see information above on how 

to contact us by mail directly or by email. If you require alternative means of communication for 

program information (e.g., Braille, large print, audiotape, etc.) please contact USDA's TARGET 

Center at (202) 720-2600 (voice and TDD). 
Evaluation of the 

Healthy Incentives Pilot (HIP): 

Final Report 

Authors: 

Susan Bartlett 

Jacob Klerman 

Lauren Olsho 

Christopher Logan 

Michelle Blocklin 

Marianne Beauregard 

Ayesha Enver 

Abt Associates 

Parke Wilde 

Friedman School of Nutrition Science and 

Policy, Tufts University

Cheryl Owens 

Margaret Melhem 

MAXIMUS 

Submitted by: 

Abt Associates Inc. 

55 Wheeler Street 

Cambridge MA 02138 

Project Director: 

Susan Bartlett 

Submitted to: 

Office of Policy Support 

Food and Nutrition Service

3101 Park Center Drive 

Alexandria, VA 22302-1500 

Project Officer: 

Danielle Berman 

This study was conducted under Contract number AG-3198-D-10-0044 with the Food and 

Nutrition Service, United States Department of Agriculture. 

This report is available on the Food and Nutrition Service website: http://www.fns.usda.gov

Suggested Citation: 

Bartlett, Susan, Jacob Klerman, Lauren Olsho, et al. Evaluation of the Healthy Incentives 

Pilot (HIP): Final Report. Prepared by Abt Associates for the U.S. Department of 

Agriculture, Food and Nutrition Service, September 2014. 

Food and Nutrition Service, Office of Policy Support September 2014 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Acknowledgements ▌pg. i

Acknowledgements 

The Healthy Incentives Pilot (HIP) evaluation was conducted under the direction of the Office of 

Policy Support in the Food and Nutrition Service, US Department of Agriculture. We thank our 

project officer, Danielle Berman, for her guidance and support. We also thank Kathryn Law, Anita 

Singh, Kelly Kinnison, Usha Kalro, and Frank Martinez Nocito for thoughtfully reviewing the report. 

The evaluation would not have been possible without the support of the Massachusetts Department of 

Transitional Assistance (DTA) as well as the households that agreed to participate in the study. DTA 

staff participated in interviews and answered questions throughout the pilot. Householders responded 

to several rounds of interviews, including 24-hour dietary recall interviews. We are grateful for the 

information and insights provided by all those who participated in the evaluation. 

The evaluation’s Technical Working Group (TWG) provided valuable guidance throughout the study, 

advising on design and analysis issues and reviewing reports. The TWG members, experts in the 

fields of evaluation design, nutrition, and public health, included Tom Baranowski (Department of 

Pediatrics, Baylor College of Medicine), Simone French (Division of Epidemiology and Community 

Health, School of Public Health, University of Minnesota), Joel Gittelsohn (Department of 

International Health, Johns Hopkins Bloomberg School of Public Health), David Just (Department of 

Applied Economics and Management, Cornell University), and Diane Whitmore Schanzenbach 

(School of Education and Social Policy, Northwestern University and National Bureau of Economic 

Research). 

At Abt Associates, Patty Connor played a key role in developing the survey instruments and 

overseeing data collection activities. Our evaluation partner, Westat, collected and processed all the 

household data. Susie McNutt led the Westat team along with Crystal MacAllum. Peter Relich at 

MAXIMUS made valuable contributions to the cost analysis. Karen Webb and Lorrene Ritchie of the 

Atkins Center for Weight and Health, University of California, Berkeley, helped develop the 

participant survey instruments. Jan Nicholson produced the report. Finally, William Hamilton and 

Patricia McKinney, our Project Quality Advisors, provided guidance and insight. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. iii

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Table of Contents 

Executive Summary ............................................................................................................................. 1

Evaluation Objectives .................................................................................................................. 1

Design and Data Collection ......................................................................................................... 2

Evaluation Design ............................................................................................................. 2

Data Collection .................................................................................................................. 2

Findings........................................................................................................................................ 3

HIP Impacts on Participants .............................................................................................. 3

Effects of HIP on Retailers ................................................................................................ 6

HIP Implementation, Costs, and Feasibility of Expansion ................................................ 8

Conclusions .................................................................................................................................. 8

1. Introduction .............................................................................................................................. 11

1.2 Pilot Background ............................................................................................................. 12

The Healthy Incentives Pilot Site .................................................................................... 13

1.3 Research Objectives ........................................................................................................ 13

1.4 Theory and Conceptual Model ........................................................................................ 15

Factors That Could Influence the Magnitude of HIP Impacts ......................................... 16

1.5 Previous Research ........................................................................................................... 17

Price and Income Effects on Food Spending ................................................................... 17

Other Interventions to Promote Fruit and Vegetable Consumption ................................ 20

Improving the Availability of Fruits and Vegetables in the Food Retail Environment ... 21

1.6 Organization of the Report .............................................................................................. 22

2. Evaluation Design, Data, and Methods .................................................................................. 23

2.1 Random Assignment Research Design ............................................................................ 25

2.2 Participant Survey ........................................................................................................... 26

Participant Survey Sampling ........................................................................................... 26

Participant Survey Instruments ........................................................................................ 28

Participant Survey Data Collection ................................................................................. 29

2.3 EBT Transaction Data ..................................................................................................... 30

2.4 Retailer Survey and Observations ................................................................................... 31

Participating Retailer Surveys ........................................................................................... 32

Later Implementing Retailers .......................................................................................... 33

Non-Participating Retailers ............................................................................................. 34

Store Observations ........................................................................................................... 34

2.5 Stakeholder Interviews .................................................................................................... 35

2.6 Participant Focus Groups................................................................................................. 36

2.7 SNAP Caseload Data ....................................................................................................... 37

2.8 Cost Data ......................................................................................................................... 37

2.9 Analytic Approach ........................................................................................................... 38

Primary Evaluation Outcome .......................................................................................... 38

Regression-Adjusted Differences Between HIP and non-HIP Groups ........................... 39
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. iv ▌Contents Abt Associates

Analysis of Binary and Categorical Outcomes ............................................................... 40

Survey Weights ............................................................................................................... 40

2.10 Sample Description ......................................................................................................... 41

Hampden County SNAP Participants ............................................................................. 41

Participant Survey Sample .............................................................................................. 43

2.11 HIP/Non-HIP Balance and Attrition Analysis ................................................................ 48

3. HIP Implementation and Operations .................................................................................... 51

3.1 Overview of Implementation and Operations ................................................................. 51

HIP Project Team ............................................................................................................ 52

HIP Systems Design and Modifications ......................................................................... 52

Retailer Recruitment ....................................................................................................... 53

HIP Steering Committee ................................................................................................. 56

Notification and Training of HIP Participants ................................................................ 57

3.2 Challenges Potentially Affecting Participant Outcomes ................................................. 60

Technical Systems Issues ................................................................................................ 60

Retailer Participation ....................................................................................................... 61

Participant Understanding of HIP ................................................................................... 61

3.3 Discussion ....................................................................................................................... 62

4. Retailer Experiences ................................................................................................................ 65

4.1 Changes in Food Retail Environment ............................................................................. 66

Activities to Promote and Sell Fruits and Vegetables ..................................................... 66

Self-Reported “Effects” of HIP on Check-Out Time, Sales, and Profits ........................ 68

Changes in Fruit and Vegetable Stocking, Availability, and Prices ............................... 70

4.2 Retailer Satisfaction ........................................................................................................ 73

Understanding of HIP Objectives, Problems, and Questions Encountered .................... 73

Satisfaction With Ability To Implement and Participate in the Pilot.............................. 77

4.3 Impact of HIP on Retailer Redemptions ......................................................................... 80

4.4 Discussion ....................................................................................................................... 82

5. HIP Participant Experiences, TFV Purchases, and Incentives Earned .............................. 85

5.1 Awareness and Understanding of HIP ............................................................................ 85

5.2 Experiences and Satisfaction........................................................................................... 92

5.3 HIP Purchases and Earning HIP Incentives .................................................................... 97

SNAP and HIP Purchases by Retailer Type.................................................................... 99

HIP Incentives Earned .................................................................................................. 104

SNAP and HIP Purchases by Participant Subgroup ...................................................... 105

5.4 Discussion ..................................................................................................................... 107

6. Effects on Expenditures and Shopping Behaviors .............................................................. 109

6.1 Food and Targeted Fruit and Vegetable Expenditures .................................................. 110

HIP Impact on Total SNAP Spending .......................................................................... 110

HIP Impact on TFV Purchases ...................................................................................... 110

Self-Reported Expenditures on All Food and on Fruits and Vegetables ...................... 112

Comparing EBT Transaction Data and Survey Evidence on TFV Spending ............... 113

6.2 Shopping Patterns for HIP and non-HIP Participants ................................................... 116
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. v

HIP Impacts on Shopping Patterns in Different Retailer Types .................................... 116

Self-Reported Shopping Patterns ................................................................................... 117

Relationship Between Pre-Implementation Shopping Patterns and HIP Impacts ......... 123

6.3 HIP Spending in Farmers Markets ................................................................................ 126

6.4 Discussion ...................................................................................................................... 128

7. Impacts on Attitudes Toward Fruits and Vegetables and Family Food Environment .... 131

7.1 Exposure to Nutrition Education and Promotion ........................................................... 132

7.2 Food Preferences and Beliefs ........................................................................................ 132

7.3 Perceived Barriers to Fruit and Vegetable Consumption .............................................. 133

7.4 Barriers to Grocery Shopping ........................................................................................ 135

7.5 Family Food Environment ............................................................................................. 135

7.6 Discussion ...................................................................................................................... 137

8. Impact on Fruit and Vegetable Consumption ..................................................................... 139

8.1 Fruits and Vegetables .................................................................................................... 140

Targeted Fruits and Vegetables ..................................................................................... 140

Other Fruit and Vegetable Aggregates .......................................................................... 144

8.2 Comparing HIP Impacts on Fruit and Vegetable Spending and Intake ......................... 146

8.3 Targeted Fruit and Vegetable Groups and Subgroups ................................................... 148

USDA Food Pattern Fruit and Vegetable Groups and Subgroups ................................. 148

Usual Daily Intake from Fruit and Vegetable Screener ................................................. 150

8.4 Fruit and Vegetable Cutoff Measures ............................................................................ 151

8.5 Differences in Impacts by Demographic Subgroup ....................................................... 154

8.6 Differences in Impacts by Baseline Fruit and Vegetable-Related Behaviors and 

Preferences .................................................................................................................... 157

8.7 Differences between Round 2 and Round 3 .................................................................. 159

8.8 Secondary Dietary Outcomes ........................................................................................ 160

Other Intake ................................................................................................................... 160

Total Food Energy ......................................................................................................... 162

Fiber and Micronutrients ............................................................................................... 163

Other Ingredients in Foods with Fruits and Vegetables ................................................ 164

8.9 Dietary Quality .............................................................................................................. 164

Dietary Guidelines for Americans ................................................................................. 164

Healthy Eating Index ..................................................................................................... 166

8.10 Discussion ...................................................................................................................... 167

9. Costs of Pilot and Feasibility of Nationwide Expansion ..................................................... 169

9.1 Data Description ............................................................................................................ 169

Stakeholders Included in the Cost Analysis .................................................................. 169

Cost Data Sources .......................................................................................................... 171

Exclusions from the HIP Pilot Cost Analysis ................................................................ 172

9.2 Pilot Costs ...................................................................................................................... 173

Overview of Pilot Costs ................................................................................................. 173

DTA Contractor Costs ................................................................................................... 177

Retailer Infrastructure Costs .......................................................................................... 177

Cost for Farmers Markets .............................................................................................. 178
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. vi ▌Contents Abt Associates

9.3 Costs of Nationwide Expansion .................................................................................... 179

Cost of the HIP Incentive .............................................................................................. 179

Implementation Costs Overview ................................................................................... 181

State Agency Expansion Costs...................................................................................... 184

Retailer Infrastructure Expansion Costs ........................................................................ 186

FNS Oversight of Acceptance Testing .......................................................................... 189

Five-Year Expansion ..................................................................................................... 189

Undetermined Costs ...................................................................................................... 192

9.4 Issues to Consider for HIP Expansion .......................................................................... 194

Legislation, Regulations, and Industry Standards ......................................................... 194

System Development and Testing ................................................................................. 195

Retailer Participation and Readiness ............................................................................. 196

Participant Notification, Outreach, and Support ........................................................... 197

Monitoring Operations and Compliance ....................................................................... 197

9.5 Discussion ..................................................................................................................... 198

10. Conclusions ............................................................................................................................ 201

10.1 Review of Key Results .................................................................................................. 202

Objective 1: Assess the causal impact of HIP on fruit and vegetable consumption by 

SNAP participants, and on other key measures of dietary intake. ................................ 202

Objective 2: Identify and assess factors that influence how HIP impacts 

participants. ................................................................................................................... 203

Objective 3: Describe the processes involved in implementing and operating HIP. .... 205

Objective 4: Assess the impact on the HIP grantee (the State SNAP agency), the 

local SNAP agency, and their team of partners (including retailers, State EBT 

processor, and community organizations). .................................................................... 206

Objective 5: Quantify, to the extent possible, the Federal, State, and local 

administrative and benefit costs of the pilot.................................................................. 207

10.2 Comparing HIP Impacts on Spending and Intake ......................................................... 207

10.3 Limitations .................................................................................................................... 209

Incomplete Follow-Up .................................................................................................. 209

Dietary Recall Instrument ............................................................................................. 210

Statistical Power Considerations ................................................................................... 211

Generalizability ............................................................................................................. 211

10.4 Conclusions ................................................................................................................... 211

References ......................................................................................................................................... 213

Appendix A: Economic Theory ...................................................................................................... 221

A.1 A Basic Elasticity Analysis ........................................................................................... 221

A.2 Incorporating Income Effects ........................................................................................ 222

A.3 Cash vs. SNAP .............................................................................................................. 223

A.4 Incomplete Coverage of Fruits and Vegetables ............................................................ 223

A.5 Incomplete Retailer Participation .................................................................................. 223

A.6 Measured TFV Purchases ............................................................................................. 224
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. vii

Appendix B: Random Assignment and Sampling ......................................................................... 227

B.1 Random Assignment to HIP/non-HIP Status ................................................................ 227

Step 1: Review Data ...................................................................................................... 227

Step 2: Create Blocking Variables and Per Block Sample Size Determination ............ 228

Step 3: Random Assignment and Proportion Verification ............................................ 229

Step 4: Balance Test and Sample Rejection .................................................................. 230

Step 5: Divide Samples into 3 Waves ........................................................................... 233

B.2 Random Sampling for Participant Survey ..................................................................... 233

Create Sampling Frame ................................................................................................. 233

Select Round 1 Participant Survey Sample ................................................................... 235

Select Round 2 Participant Survey Sample ................................................................... 237

Select Round 3 Participant Survey Sample ................................................................... 238

Summary of Sample Sizes ............................................................................................. 239

Appendix C: Weighting Methodology ............................................................................................ 241

C.1 Round 1 ......................................................................................................................... 241

Wave-Specific Base Weights ........................................................................................ 241

Pooled Base Weights ..................................................................................................... 242

Ratio Adjustment of the Pooled Weights ...................................................................... 242

Non-Response Adjustment ............................................................................................ 242

C.2 Round 2 ......................................................................................................................... 243

Initial Weights ............................................................................................................... 243

Non-Response Adjustment ............................................................................................ 243

Second-Day Intake Weights .......................................................................................... 244

C.3 Round 3 ......................................................................................................................... 244

Initial Weights ............................................................................................................... 244

Non-Response Adjustment ............................................................................................ 245

Second-Day Intake Weights .......................................................................................... 245

C4. Non-Response Bias Analysis ......................................................................................... 245

Analysis of Non-Response in Round 1 .......................................................................... 246

Non-Response Bias Analysis for Round 2 .................................................................... 248

Non-Response Bias Analysis for Round 3 .................................................................... 249

Summary and Conclusions ............................................................................................ 250

Appendix D: Participant Data Preparation ................................................................................... 255

D.1 Coding of Dietary Recall Data ...................................................................................... 255

AMPM Interview Data Entry/Standard SurveyNet Processing ..................................... 255

USDA Food Pattern Food Groups, MyPyramid Food Groups, and Healthy Eating 

Index .............................................................................................................................. 259

Special Outcomes for This Study .................................................................................. 260

Dietary Supplement Data ............................................................................................... 262

D.2 Coding of Other Participant Survey Data ...................................................................... 262

Initial Processing of Participant Survey Data ................................................................ 262

Construction of Analytic Outcomes and Covariates ..................................................... 263

D.3 EBT Transaction Data Processing Methods .................................................................. 265
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. viii ▌Contents Abt Associates

Initial Construction of Raw Monthly Transaction-Level Files from HIP Daily 

Activity Files and Monthly Missing Data Files ............................................................ 265

Cleaning of Raw Monthly Transaction-Level Files ...................................................... 266

Construction of the Monthly Household-Level Analysis Files ..................................... 266

Appendix E: Analytic Methods ....................................................................................................... 271

E.1 Multivariate Models ...................................................................................................... 271

E.2 Limited Dependent Variables ....................................................................................... 273

E.3 Subgroup Analyses ....................................................................................................... 274

E.4 Survey Weights, Standard Errors, and Confidence Intervals ........................................ 275

E.5 Testing for Inter-Round Change ................................................................................... 275

E.6 Multiple Comparisons ................................................................................................... 276

E.7 Sensitivity to Outliers .................................................................................................... 276

E.8 Usual Intake Estimation ................................................................................................ 276

Advantages of the NCI Method .................................................................................... 277

Regularly and Episodically Consumed Dietary Components ....................................... 278

Estimation Procedures ................................................................................................... 278

Assessing Overall Dietary Quality ................................................................................ 279

Appendix F: Baseline Outcomes, Alternative Models, and Other Supplemental Tables .......... 281

F.1 Chapter 1 Supplemental Table: Definition of TFVs ..................................................... 281

F.2 Chapter 2 Supplemental Tables: Baseline Characteristics and Balance Tests .............. 282

F.3 Chapter 4 Supplemental Tables: Early and Late HIP Implementation Retailer 

Experiences ................................................................................................................... 290

F.4 Chapter 5 Supplemental Table: TFV Purchases by Store Type .................................... 305

F.5 Chapter 6 Supplemental Tables: Baseline Outcomes, Alternative Models, Farmers 

Market Purchases .......................................................................................................... 306

F.6 Chapter 7 Supplemental Tables: Baseline Outcomes and Alternative Models ............. 314

F.7 Chapter 8 Supplemental Tables: Full Regression Results, Additional Outcomes, Baseline 

Fruit and Vegetable Screener ........................................................................................ 326

Appendix G: Analyses by Follow-Up Survey Round .................................................................... 335

Appendix H: Analyses by Subgroup .............................................................................................. 377

Appendix I: Supplementary Non-Experimental Analyses ........................................................... 527

I.1 Mediation Analyses ....................................................................................................... 527

Single Mediator Models ................................................................................................ 527

Mediation Models Estimated ........................................................................................ 529

Mediation Results ......................................................................................................... 532

Implications, Limitations, and Assumptions ................................................................. 539

Discussion ..................................................................................................................... 541

I.2 HIP Awareness and Understanding .............................................................................. 541

Expenditures .................................................................................................................. 543

Consumption ................................................................................................................. 548

Discussion ..................................................................................................................... 548
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. ix

List of Exhibits 

Executive Summary 

Exhibit ES.1: HIP Participants Consumed 0.24 Cup-Equivalent More Fruits and 

Vegetables per Day ................................................................................................. 4

Exhibit ES.2: HIP Households Purchased More Fruits and Vegetables per Month ...................... 5

Exhibit ES.3: Percent of Retailers Somewhat or Very Satisfied with How HIP Worked in 

Their Stores ............................................................................................................. 7

Chapter 1 

Exhibit 1.1: Conceptual Model Describing the Potential Effect of HIP on Food 

Consumption ......................................................................................................... 15

Chapter 2 

Exhibit 2.1: Overview of Sampling Design and Data Collection Activities ............................. 24

Exhibit 2.2: Households and Persons in the HIP and non-HIP Sampling Frames by 

Wave ..................................................................................................................... 25

Exhibit 2.3: Sample Sizes, Eligible Respondents, Completed Surveys, and Response 

Rates: HIP and non-HIP Households by Survey Round ....................................... 27

Exhibit 2.4: Participant Survey Topics, by Round .................................................................... 28

Exhibit 2.5: Retailer Data Collection Activities ....................................................................... 32

Exhibit 2.6: Retailer Survey: Samples and Response Rates ..................................................... 33

Exhibit 2.7: Characteristics of SNAP Households in Hampden County, July 2011 ................. 42

Exhibit 2.8: Baseline Characteristics of Households Completing Follow-Up Participant 

Surveys (Round 2 or 3), by Treatment and Control Status ................................... 43

Exhibit 2.9: Baseline Characteristics of Household Heads for Respondents Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control 

Status ..................................................................................................................... 46

Exhibit 2.10: HIP/Non-HIP Balance from the SNAP Caseload Files ........................................ 49

Chapter 3 

Exhibit 3.1: Hampden County Retailers Eligible for HIP and Participating in HIP ................. 54

Exhibit 3.2: HIP Card Sleeve .................................................................................................... 58

Exhibit 3.3: Brochure from Second Interim Mailing (2 of 6 Panels) ........................................ 59

Exhibit 3.4: Sample Retailer Signage ....................................................................................... 62

Chapter 4 

Exhibit 4.1: Retailer Promotion of Fruits and Vegetables, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period ................................................ 67

Exhibit 4.2: Retailer Activities to Sell More Fruits and Vegetables, Overall and by Store 

Type: Retailer Survey, Late HIP Implementation Period ..................................... 68

Exhibit 4.3: Self-Reported Effects of HIP on Time and Effort, Overall and by Store 

Type: Retailer Survey, Late HIP Implementation Period ..................................... 69

Exhibit 4.4: Self-Reported Effects of HIP on Sales and Profits, Overall and by Store 

Type: Retailer Survey, Late HIP Implementation Period ..................................... 70

Exhibit 4.5: Self-Reported Effect of HIP on Product Stocking, Overall and by Store 

Type: Retailer Survey, Late HIP Implementation Period ..................................... 72
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. x ▌Contents Abt Associates

Exhibit 4.6: Retailer Reported Reasons for Stocking More Fruits or Vegetables, Overall 

and by Store Type: Retailer Survey, Late HIP Implementation Period ................ 73

Exhibit 4.7: Retailer Understanding and Perceptions of HIP, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period ............................................... 75

Exhibit 4.8: Retailer Problems with HIP in the Previous Three Months, by Grocery and 

Convenience Stores: Retailer Survey, Late HIP Implementation Period ............. 76

Exhibit 4.9: Retailer Perceptions of Customer Questions about HIP, Overall and by 

Store Type: Retailer Survey, Late HIP Implementation Period ........................... 77

Exhibit 4.10: Retailer Satisfaction with HIP, Overall and by Store Type: Retailer Survey, 

Late HIP Implementation Period .......................................................................... 78

Exhibit 4.11: Reasons Later Implementing Stores Did Not Join HIP at the Start, Overall 

and by Store Type: Retailer Survey, Late HIP Implementation Period ................ 79

Exhibit 4.12: Reasons Later Implementing Stores Joined HIP, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period ............................................... 80

Exhibit 4.13: Monthly SNAP Purchases per Household and Projected in Aggregate for 

Hampden County .................................................................................................. 81

Chapter 5 

Exhibit 5.1: Primary Shopper Awareness of HIP, by Treatment and Control Status, by 

Round: Round 2 & 3 Participant Surveys ............................................................. 86

Exhibit 5.2: Where Primary Shopper Heard of HIP, by Treatment and Control Status: 

Round 2 & 3 Participant Surveys ......................................................................... 87

Exhibit 5.3: Primary Shopper Attendance at Training and Use of Hotline, by Round: 

Round 2 & 3 Participant Surveys, HIP Participants ............................................. 89

Exhibit 5.4: Primary Shopper Understanding of HIP, by Round: Round 2 & 3 

Participant Surveys, HIP Participants ................................................................... 91

Exhibit 5.5: Primary Shopper HIP Experiences, by Round: Round 2 & 3 Participant 

Surveys, HIP Participants ..................................................................................... 94

Exhibit 5.6: Primary Shopper Problems with EBT Card, by Treatment and Control 

Status, by Round: Round 2 & 3 Participant Surveys ............................................ 95

Exhibit 5.7: Type of Primary Shopper Problems with EBT Card, by Treatment and 

Control Status: Round 2 & 3 Participant Surveys ................................................ 96

Exhibit 5.8: Primary Shopper Satisfaction with HIP, by Round: Round 2 & 3 Participant 

Surveys, HIP Participants ..................................................................................... 96

Exhibit 5.9: Mean Monthly SNAP Redemptions of All HIP Participant Households at 

Participating and Non-Participating Stores .......................................................... 98

Exhibit 5.10: Mean Monthly SNAP Redemptions of All HIP Participant Households at 

Participating and Non-Participating Stores .......................................................... 98

Exhibit 5.11: Mean Monthly HIP Purchases of all HIP Participant Households, by Store 

Type ...................................................................................................................... 99

Exhibit 5.12: Mean Monthly SNAP Purchases and HIP Purchases of all HIP Participant 

Households, by Round ........................................................................................ 101

Exhibit 5.13: Mean Monthly SNAP Purchases for HIP Participant Households at HIP 

Participating Retailers, by Round ....................................................................... 103

Exhibit 5.14: Distribution of HIP Households by TFV Purchases ........................................... 105

Exhibit 5.15: HIP Purchases, SNAP Purchases, and HIP Incentives Earned, by Subgroup .... 106
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. xi

Chapter 6 

Exhibit 6.1: Impact of HIP on Mean Monthly SNAP Issuance, SNAP Purchases and 

TFV Purchases .................................................................................................... 111

Exhibit 6.2: Impact of HIP on Self-Reported Monthly Expenditures, Linear Regression 

Model .................................................................................................................. 112

Exhibit 6.3: Impact of HIP on Monthly Expenditures, Survey-Based vs. EBT 

Transaction-Based Measures, Linear Regression Model .................................... 115

Exhibit 6.4: Impact of HIP on Mean Monthly SNAP Purchases by Retailer Type ................ 117

Exhibit 6.5: Impact of HIP on Self-reported Usual Grocery Store Type, Linear 

Probability Model ............................................................................................... 118

Exhibit 6.6: Impact of HIP on Reasons for Choice of Usual Grocery Shopping Place, 

Linear Probability Model .................................................................................... 119

Exhibit 6.7: Impact of HIP on Grocery Shopping Behaviors, Linear Regression Model ....... 121

Exhibit 6.8: Primary Shopper Self-Reported Changes in Fruit and Vegetable Purchasing 

Due to HIP: Round 2 & 3 Participant Surveys, HIP Participants ....................... 122

Exhibit 6.9: Primary Shopper Self-Reported Changes in Shopping Location Due to HIP: 

Round 2 & Round 3 Participant Surveys, HIP Participants ................................ 123

Exhibit 6.10: Impact of HIP on Mean Monthly SNAP and TFV Purchases at Participating 

Supermarkets and Superstores, by November 2011 (Pre-Implementation) 

Shopping Behavior .............................................................................................. 125

Exhibit 6.11: Percent of Households That Shopped at Farmers Markets ................................. 127

Exhibit 6.12: Total SNAP Sales at Farmers Markets ................................................................ 128

Chapter 7 

Exhibit 7.1: Impact of HIP on Self-Reported Exposure to Nutrition Education and 

Promotion in Past Three Months, Linear Probability Model .............................. 132

Exhibit 7.2: Impact of HIP on Food Preferences and Beliefs, Linear Regression Model....... 133

Exhibit 7.3: Impact of HIP on Perceived Barriers to Fruit and Vegetable (FV) 

Consumption, Linear Regression Model ............................................................. 134

Exhibit 7.4: Impact of HIP on Grocery Shopping Barriers, Linear Regression Model .......... 135

Exhibit 7.5: Impact of HIP on Family Food Environment, Linear Regression Model ........... 136

Chapter 8 

Exhibit 8.1: Impact of HIP on Consumption of Targeted Fruits and Vegetables, Mean 

Cup-Equivalents Consumed ................................................................................ 141

Exhibit 8.2: Targeted Fruits and Vegetables (TFV) as a Proportion of All Fruits and 

Vegetables ........................................................................................................... 142

Exhibit 8.3: Impact of HIP on Consumption of Fruits and Vegetables and Disaggregated 

Components, Cup-Equivalents ............................................................................ 144

Exhibit 8.4: Comparison of HIP Impacts on Intake and Spending Measures, Targeted 

Fruits and Vegetables and All Fruits and Vegetables ......................................... 147

Exhibit 8.5: Impact of HIP on Consumption of Targeted Fruits and Vegetables (TFV), 

Cup-Equivalents, by USDA Food Pattern Food Group ...................................... 149

Exhibit 8.6: Impact of HIP on Estimated Usual Daily Intake from Fruit and Vegetable 

Screener, Cup-Equivalents .................................................................................. 151
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. xii ▌Contents Abt Associates

Exhibit 8.7: Impact of HIP on Cutoff Measures of Fruit and Vegetable Intake in the Past 

24 Hours ............................................................................................................. 152

Exhibit 8.8: Impact of HIP on Probability of Any Fruit and Vegetable Intake, by USDA 

Food Pattern Food Group ................................................................................... 153

Exhibit 8.9 Impact of HIP on Total Number of USDA Food Pattern Fruit and 

Vegetable Groups Consumed in Past 24 Hours .................................................. 154

Exhibit 8.10: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables, Cup-Equivalents, by Demographic Subgroup ................................ 155

Exhibit 8.11: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables, Cup-Equivalents, by Baseline Fruit and Vegetable-Related 

Behaviors and Preferences and Shopping Patterns Subgroup ............................ 158

Exhibit 8.12 Differences in Impact of HIP on Consumption of Targeted Fruits and 

Vegetables, Cup-Equivalents, by Follow-Up Round .......................................... 160

Exhibit 8.13: Impact of HIP on Consumption of Other Foods, by USDA Food Pattern 

Food Group ......................................................................................................... 162

Exhibit 8.14: Impact of HIP on Total Energy Intake ............................................................... 163

Exhibit 8.15: Impact of HIP on Nutrient Intake ....................................................................... 163

Exhibit 8.16: Impact of HIP on Other Ingredients in Foods Containing Fruits and 

Vegetables .......................................................................................................... 164

Exhibit 8.17: Impact of HIP on Proportion of Respondents Meeting 2010 Dietary 

Guidelines for Americans ................................................................................... 166

Exhibit 8.18: Impact of HIP on Healthy Eating Index-2010, Components and Total Score .... 167

Chapter 9 

Exhibit 9.1: Stakeholder Groups and Role in HIP .................................................................. 171

Exhibit 9.2: Calculation of HIP Incentives for Pilot .............................................................. 174

Exhibit 9.3: Summary of Implementation Cost of Healthy Incentives Pilot (Including 

DTA and FNS) .................................................................................................... 176

Exhibit 9.4: Detail of Pilot Contractor Costs .......................................................................... 177

Exhibit 9.5: Costs of Implementing HIP in Farmers Markets ................................................ 179

Exhibit 9.6: Projected Incentives Earned During Nationwide Expansion of HIP .................. 181

Exhibit 9.7: Estimated Implementation Cost of Statewide and Nationwide Expansion ........ 183

Exhibit 9.8: Estimated Cost of Integrated Electronic Cash Register System 

Modifications for Nationwide Expansion ........................................................... 188

Exhibit 9.9: Assumptions and Projections of Annual Costs for HIP Expansion .................... 191

Appendix B 

Exhibit B.1: Description of the Blocking Cells ....................................................................... 229

Exhibit B.2: Blocking Cell Shares of HIP and Non-HIP Households .................................... 230

Exhibit B.3: Balance Test ....................................................................................................... 232

Exhibit B.4: Frequencies of Households and Persons in the HIP and Non-HIP Sampling 

Frames by Size of Household ............................................................................. 234

Exhibit B.5: Frequencies of Persons in the HIP and Non-HIP Sampling Frames by Block 

and Wave ............................................................................................................ 234

Exhibit B.6: Intended Frequencies of Households and Persons in the HIP and Non-HIP 

Samples by Size of Household ........................................................................... 236
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. xiii

Exhibit B.7: Counts of Evaluation Sample by Treatment Status, Wave, and Block ............... 236

Exhibit B.8: Number of Round 1 Respondents Sampled for Round 2 after Deleting 

Households no Longer on SNAP ........................................................................ 238

Exhibit B.9: Number of Round 2 Respondents Sampled for Round 3 after Deleting 

Households no Longer on SNAP ........................................................................ 239

Appendix C 

Exhibit C.1: Response Rates by Round and Treatment Status ................................................ 246

Exhibit C.2: Weighted Response Rates by Round, Treatment Status, and Selected 

Characteristics ..................................................................................................... 251

Appendix D 

Exhibit D.1: Source of Foods Reported in AMPM .................................................................. 261

Exhibit D.2: Number of Observations in Monthly Household-Level Analysis Files .............. 267

Exhibit D.3: SNAP Purchases and HIP-Eligible TFV Purchases Variables by Store Type .... 268

Appendix E 

Exhibit E.1: Included Analytic Covariates, Final Analytic Sample ........................................ 272

Appendix F 

Exhibit F1.1: Minimum Requirements and Specifications for WIC Fruits and Vegetables ..... 281

Exhibit F2.1: Baseline Characteristics of Respondents Completing Follow-Up Participant 

Surveys (Round 2 or 3), by Treatment and Control Status ................................. 282

Exhibit F2.2: Self-Reported Baseline Characteristics of Respondents Completing FollowUp Participant Surveys (Round 2 or 3), by Treatment and Control Status ......... 283

Exhibit F2.3: Self-Reported Baseline Characteristics of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control 

Status ................................................................................................................... 284

Exhibit F2.4: HIP/Non-HIP Balance from the SNAP Caseload Files, Disaggregated by 

Survey and Non-Survey Samples ........................................................................ 285

Exhibit F2.5: HIP/Non-HIP Balance: Baseline Characteristics (July 2011) for Study 

Participants on SNAP in May 2012 .................................................................... 286

Exhibit F2.6: HIP/Non-HIP Balance: Baseline Characteristics (July 2011) for Study 

Participants on SNAP in October 2012 ............................................................... 287

Exhibit F2.7: HIP/Non-HIP Balance: May 2012 Characteristics for Study Participants on 

SNAP in May 2012 ............................................................................................. 288

Exhibit F2.8 HIP/Non-HIP Balance: October 2012 Characteristics for Study Participants 

on SNAP in October 2012 ................................................................................... 289

Exhibit F4.1: Retailer Promotion of Fruits and Vegetables, Overall and by Store Type, 

Early and Late HIP Implementation Periods ....................................................... 291

Exhibit F4.2: Retailer Fresh Fruit and Vegetable Inventory and Average Price/Lb, 

Overall and by Store Type, Early and Late HIP Implementation Periods .......... 293

Exhibit F4.3: Retailer Canned Fruit and Vegetable Inventory and Average Price/Lb, 

Overall and by Store Type, Early and Late HIP Implementation Periods .......... 296

Exhibit F4.4: Retailer Frozen Fruit and Vegetable Inventory and Average Price/Lb, 

Overall and by Store Type, Early and Late HIP Implementation Periods .......... 298
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. xiv ▌Contents Abt Associates

Exhibit F4.5: Retailer Problems with HIP in the Prior 3 Months, by Grocery and 

Convenience Stores, Late HIP Implementation Period ...................................... 299

Exhibit F4.6: Retailer Requests for Information from Others, Overall and by Store Type, 

Late HIP Implementation Period ........................................................................ 302

Exhibit F4.7: Questions to Retailers from Other Employees, Overall and by Store Type, 

Late HIP Implementation Period ........................................................................ 303

Exhibit F5.1: Distribution of HIP Households by TFV Purchases ........................................... 305

Exhibit F6.1: Baseline Self-Reported Monthly Expenditures of Primary Shoppers 

Completing Follow-Up Participant Surveys (Round 2 or 3), by Treatment 

and Control Status .............................................................................................. 306

Exhibit F6.2: Baseline Usual Grocery Shopping Store Type of Primary Shoppers 

Completing Follow-Up Participant Surveys (Round 2 or 3) by Treatment 

and Control Status .............................................................................................. 306

Exhibit F6.3: Impact of HIP on Usual Grocery Store Type, Logistic Regression Model ........ 307

Exhibit F6.4: Baseline Reasons for Choice of Usual Grocery Shopping Place of Primary 

Shoppers Completing Follow-Up Participant Surveys (Round 2 or 3), by 

Treatment and Control Status ............................................................................. 308

Exhibit F6.5: Impact of HIP on Reasons for Choice of Usual Grocery Store Type, 

Logistic Regression Model ................................................................................. 309

Exhibit F6.6: Baseline Grocery Shopping Frequency of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control 

Status .................................................................................................................. 310

Exhibit F6.7: Baseline Probability of Going Out of Way/Making Special Effort to Shop 

at Particular Store for Fruits and Vegetables of Primary Shoppers 

Completing Follow-Up Participant Surveys (Round 2 or 3), by Treatment 

and Control Status .............................................................................................. 310

Exhibit F6.8: Impact of HIP on Grocery Shopping Frequency, Logistic Regression .............. 311

Exhibit F6.9: Impact of HIP on Grocery Shopping Frequency, Ordered Logistic 

Regression .......................................................................................................... 311

Exhibit F6.10: Impact of HIP on Going Out of Way/Making Special Effort to Shop at 

Particular Store for Fruits and Vegetables, Logistic Regression Model ............. 312

Exhibit F6.11: Descriptive Results for EBT Analysis After Implementation, Disaggregated 

According to November 2011 (Pre-Implementation) Shopping Behavior ......... 312

Exhibit F6.12: Impact of HIP on Mean Monthly SNAP Purchases at Farmers Markets ........... 313

Exhibit F7.1: Baseline Food Preferences and Beliefs of Respondents Completing FollowUp Participant Surveys (Round 2 or 3), by Treatment and Control Status ......... 314

Exhibit F7.2: Baseline Perceived Barriers to Fruit & Vegetable Consumption of 

Respondents Completing Follow-Up Participant Surveys (Round 2 or 3), 

by Treatment and Control Status ........................................................................ 316

Exhibit F7.3: Baseline Barriers to Grocery Shopping of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control 

Status .................................................................................................................. 317

Exhibit F7.4: Baseline Family Food Environment of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control 

Status .................................................................................................................. 317
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. xv

Exhibit F7.5: Impact of HIP on Self-Reported Exposure to Nutrition Education and 

Promotion in Past 3 Months, Logistic Regression Model ................................... 319

Exhibit F7.6: Impact of HIP on Food Preferences & Beliefs, Ordered Logistic Regression 

Model .................................................................................................................. 319

Exhibit F7.7: Impact of HIP on Food Preferences & Beliefs, Logistic Regression Model....... 320

Exhibit F7.8: Impact of HIP on Perceived Barriers to Fruit & Vegetable Consumption, 

Ordered Logistic Regression Model ................................................................... 321

Exhibit F7.9: Impact of HIP on Perceived Barriers to Fruit & Vegetable Consumption, 

Logistic Regression Model ................................................................................. 322

Exhibit F7.10: Impact of HIP on Barriers to Grocery Shopping, Ordered Logistic 

Regression Model ............................................................................................... 323

Exhibit F7.11: Impact of HIP on Barriers to Grocery Shopping, Logistic Regression Model ... 323

Exhibit F7.12: Impact of HIP on Family Food Environment, Ordered Logistic Regression 

Model .................................................................................................................. 324

Exhibit F7.13: Impact of HIP on Family Food Environment, Logistic Regression Model ........ 325

Exhibit F8.1: Impact of HIP on Consumption of Fruits & Vegetables and Disaggregated 

Components, Restrictive and Inclusive Targeted Fruits and Vegetables 

(TFV) and All Fruits and Vegetables, Cup-Equivalents, Full Regression 

Results Including Coefficients for All Covariates .............................................. 327

Exhibit F8.2: Impact of HIP on Consumption of Targeted Fruits & Vegetables (TFV), 

Inclusive Measure, by USDA Food Pattern Food Group .................................... 332

Exhibit F8.3: Impact of HIP on Consumption of Total Fruits & Vegetables, by USDA 

Food Pattern Food Group .................................................................................... 333

Exhibit F8.4: Baseline Estimated Usual Daily Intake in Cup-Equivalents from Fruit & 

Vegetable Screener of Respondents Completing Follow-Up Participant 

Surveys (Round 2 or 3), by Treatment and Control Status ................................. 334

Appendix G 

Exhibit G6.1: Differences in Impact of HIP on Mean Monthly SNAP and TFV Purchases, 

by Follow-Up Round........................................................................................... 335

Exhibit G6.2: Differences in Impacts of HIP on Self-Reported Monthly Expenditures, by 

Follow-Up Round ............................................................................................... 336

Exhibit G6.3: Differences in Impact of HIP on Mean Monthly SNAP Purchases by 

Retailer Type, by Follow-Up Round ................................................................... 337

Exhibit G6.4: Differences in Impacts of HIP on Usual Grocery Store Type, by Follow-Up 

Round .................................................................................................................. 339

Exhibit G6.5: Differences in Impacts of HIP on Reasons for Choice of Usual Grocery 

Shopping Place, by Follow-Up Round ................................................................ 341

Exhibit G6.6: Differences in Impacts of HIP on Grocery Shopping Behavior, by FollowUp Round ............................................................................................................ 344

Exhibit G6.7: Primary Shopper Self-Reported Changes in Shopping Location Due to HIP: 

Round 2 and 3 Participant Surveys, HIP-Eligible Households ........................... 345

Exhibit G6.8: Differences in Impact of HIP on Mean Monthly SNAP and TFV Purchases 

at Participating Supermarkets and Superstores by Pre-Implementation 

Shopping Behavior, by Follow-Up Round .......................................................... 346
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. xvi ▌Contents Abt Associates

Exhibit G6.9: Differences in Impact of HIP on Mean Monthly SNAP Purchases at 

Farmers Markets, by Follow-Up Round ............................................................. 348

Exhibit G7.1: Differences in Impacts of HIP on Self-Reported Exposure to Nutrition 

Education and Promotion in Past 3 Months, by Follow-Up Round ................... 349

Exhibit G7.2: Differences in Impacts of HIP on Food Preferences & Beliefs, by FollowUp Round ............................................................................................................ 350

Exhibit G7.3: Differences in Impacts of HIP on Perceived Barriers to Fruit & Vegetable 

Consumption, by Follow-Up Round ................................................................... 351

Exhibit G7.4: Differences in Impacts of HIP on Grocery Shopping Barriers, by FollowUp Round ............................................................................................................ 352

Exhibit G7.5: Differences in Impacts of HIP on Family Food Environment, by Follow-Up 

Round ................................................................................................................. 353

Exhibit G8.1: Differences in Impacts of HIP on Consumption of Fruits and Vegetables, 

by Follow-Up Round .......................................................................................... 355

Exhibit G8.2a: Differences in Impacts of HIP on Consumption of Targeted Fruits & 

Vegetables (TFV), Preferred Restrictive Proxy Measure, Cup-Equivalents, 

by Food Pattern Equivalent Group, by Follow-Up Round ................................. 357

Exhibit G8.2b: Differences in Impacts of HIP on Consumption of Targeted Fruits & 

Vegetables (TFV), Alternative Inclusive Proxy Measure, Cup-Equivalents, 

by Food Pattern Equivalent Group, by Follow-Up Round ................................. 359

Exhibit G8.2c: Differences in Impacts of HIP on Consumption of All Fruits & Vegetables, 

Cup-Equivalents, by Food Pattern Equivalent Group, by Follow-Up Round .... 361

Exhibit G8.3: Differences in Impacts of HIP on Estimated Usual Daily Intake from Fruit 

& Vegetable Screener, Cups, by Follow-Up Round ........................................... 363

Exhibit G8.4: Differences in Impacts of HIP on Threshold Measures of Fruit & Vegetable 

Intake in the Past 24 Hours, by Follow-Up Round ............................................. 365

Exhibit G8.5: Differences in Impacts of HIP on Probability of Any Fruit and Vegetable 

Intake in the Past 24 Hours, by Food Equivalent Pattern Group, by FollowUp Round ............................................................................................................ 366

Exhibit G8.6: Differences in Impacts of HIP on Total Number of USDA Food Pattern 

Fruit & Vegetable Groups Consumed in Past 24 Hours, by Follow-Up 

Round ................................................................................................................. 368

Exhibit G8.7: Differences in Impacts of HIP on Consumption of Other Foods, by Food 

Pattern Equivalent Group, by Follow-Up Round ............................................... 369

Exhibit G8.8: Differences in Impacts of HIP on Total Energy Intake, by Follow-Up 

Round ................................................................................................................. 370

Exhibit G8.9: Differences in Impacts of HIP on Total Nutrient Intake, by Follow-Up 

Round ................................................................................................................. 371

Exhibit G8.10: Differences in Impacts of HIP on Intake of Other Ingredients in Foods with 

Fruits & Vegetables, by Follow-Up Round ........................................................ 372

Exhibit G8.11: Differences in Impacts of HIP on 2010 Dietary Guidelines for Americans, 

by Follow-Up Round .......................................................................................... 373

Exhibit G8.12: Differences in Impacts of HIP on Healthy Eating Index-2010 and 

Component Scores, by Follow-Up Round .......................................................... 375
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. xvii

Appendix H 

Exhibit H5.1: Proportion of Primary Shoppers Who Had Ever Heard of HIP, by 

Treatment and Control Status, by Subgroup: Round 2 and 3 Participant 

Surveys ................................................................................................................ 377

Exhibit H5.2a: Proportion of Primary Shoppers Who Attended Meeting to Learn about 

HIP, by Subgroup: Round 2 and 3 Participant Surveys, HIP Participants .......... 379

Exhibit H5.2b: Proportion of Primary Shoppers Who Called EBT or HIP Hotline with 

Questions or Problems in Past Month, by Subgroup: Round 2 and 3 

Participant Surveys, HIP Participants ................................................................. 381

Exhibit H5.3a: Primary Shopper Report: “How easy or hard has it been to understand how 

HIP works?”, by Subgroup: Round 2 and 3 Participant Surveys, HIP 

Participants .......................................................................................................... 383

Exhibit H5.3b: Primary Shopper Report: “How easy or hard is it remembering which fruits 

and vegetables earn the HIP rebate?” by Subgroup: Round 2 and 3 

Participant Surveys, HIP Participants ................................................................. 385

Exhibit H5.3c: Primary Shopper Report: “How easy or hard is it keeping track of the HIP 

rebates you earn?” by Subgroup: Round 2 and 3 Participant Surveys, HIP 

Participants .......................................................................................................... 387

Exhibit H5.4: Proportion of Primary Shoppers Who Keep Track of HIP Rebate Earned, 

by Subgroup: Round 2 and 3 Participant Surveys, HIP Participants .................. 389

Exhibit H5.5a: Proportion of Primary Shoppers Who Are Primary Users of EBT Card, by 

Treatment and Control Status, by Subgroup: Round 2 & 3 Participant 

Surveys ................................................................................................................ 391

Exhibit H5.5b: Proportion of Primary Shoppers Who Had Problems with EBT Card or 

Account in Past Month, by Treatment and Control Status, by Subgroup: 

Round 2 and 3 Participant Surveys ..................................................................... 393

Exhibit H5.6: Proportion of Primary Shoppers Who Would Like to Keep Participating in 

HIP, by Subgroup: Round & 3 Participant Surveys, HIP Participants ................ 395

Exhibit H6.1: Descriptive Results for EBT Analysis of Subgroups: Unweighted 

Percentage of All Households ............................................................................. 397

Exhibit H6.2: Differences in Impacts of SNAP Purchases and Eligible TFV Purchases at 

Supermarkets and Superstores by HIP Status and Subgroup, March to 

October 2012 ....................................................................................................... 398

Exhibit H6.3a: Differences in Impacts of HIP on Self-Reported Monthly SNAP 

Expenditures, by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ............................................................................................................ 401

Exhibit H6.3b: Differences in Impacts of HIP on Self-Reported Monthly Expenditures on 

Groceries, by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ............................................................................................................ 402

Exhibit H6.3c: Differences in Impacts of HIP on Self-Reported Monthly Expenditures on 

Groceries, Food Items, by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 403

Exhibit H6.3d: Differences in Impacts of HIP on Self-Reported Monthly Expenditures on 

Groceries, Non-Food Items, by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 404
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. xviii ▌Contents Abt Associates

Exhibit H6.3e: Differences in Impacts of HIP on Self-Reported Monthly Restaurant 

Expenditures, by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ........................................................................................................... 405

Exhibit H6.3f: Differences in Impacts of HIP on Self-Reported Monthly Fruit & Vegetable 

Expenditures, by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ........................................................................................................... 406

Exhibit H6.4: Differences in Impacts of SNAP Purchases by HIP Status, Retailer Type, 

and Subgroup, March to October 2012 ............................................................... 407

Exhibit H6.5a: Differences in Impacts of HIP on Usual Grocery Store Type (Large Chain 

Grocery Store or Supermarket), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 410

Exhibit H6.5b: Differences in Impacts of HIP on Usual Grocery Store Type (Natural or 

Organic Supermarket (such as Whole Foods Market), by Baseline Intake, 

Attitudes, and Pre-HIP Shopping Patterns Subgroups ........................................ 411

Exhibit H6.5c: Differences in Impacts of HIP on Usual Grocery Store Type (Small Local 

Store or Corner Store), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 412

Exhibit H6.5d: Differences in Impacts of HIP on Usual Grocery Store Type (Convenience 

Store (such as 7-11 or mini market), by Baseline Intake, Attitudes, and PreHIP Shopping Patterns Subgroups ...................................................................... 413

Exhibit H6.5e: Differences in Impacts of HIP on Usual Grocery Store Type (Warehouse 

Club Store (such as Sam's Club or Costco), by Baseline Intake, Attitudes, 

and Pre-HIP Shopping Patterns Subgroups ........................................................ 414

Exhibit H6.5f: Differences in Impacts of HIP on Usual Grocery Store Type (Discount 

Superstore (such as Walmart), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 415

Exhibit H6.5g: Differences in Impacts of HIP on Usual Grocery Store Type (Ethnic 

Market), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ........................................................................................................... 416

Exhibit H6.5h: Differences in Impacts of HIP on Usual Grocery Store Type (Farmers 

Market/Co-Op), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups ............................................................................................. 417

Exhibit H6.5i: Differences in Impacts of HIP on Usual Grocery Store Type (Some Other 

Location), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ........................................................................................................... 418

Exhibit H6.6a: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Prices/Affordability), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups ............................................................................................. 419

Exhibit H6.6b: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Close to Home), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups ............................................................................................. 420

Exhibit H6.6c: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Variety of Products), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups ............................................................................................. 421
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. xix

Exhibit H6.6d: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Produce Better or Fresher), by Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups .............................................................................................. 422

Exhibit H6.6e: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Sales/Promotions in Store), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 423

Exhibit H6.6f: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Preferred Products Are Available), by Baseline Intake, Attitudes, and PreHIP Shopping Patterns Subgroups ...................................................................... 424

Exhibit H6.6g: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Familiarity with Store), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 425

Exhibit H6.6h: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(One Stop Shopping), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups .............................................................................................. 426

Exhibit H6.6i: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Easy to Get There), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups .............................................................................................. 427

Exhibit H6.6j: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Quality), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ............................................................................................................ 428

Exhibit H6.6k: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Close to Work), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups .............................................................................................. 429

Exhibit H6.6l: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Close to Some Other Location), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 430

Exhibit H6.6m: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Disability Accessible), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 431

Exhibit H6.6n: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Hours of Operation Convenient), by Baseline Intake, Attitudes, and PreHIP Shopping Patterns Subgroups ...................................................................... 432

Exhibit H6.6o: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(EBT Card Accepted), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 433

Exhibit H6.6p: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Ethnic Foods Are Available), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups ............................................................................. 434

Exhibit H6.6q: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Good Service), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups .............................................................................................. 435

Exhibit H6.6r: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Bulk Purchases), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups .............................................................................................. 436
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. xx ▌Contents Abt Associates

Exhibit H6.6s: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Clean), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups ........................................................................................................... 437

Exhibit H6.6t: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Some Other Reason), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups ............................................................................................. 438

Exhibit H6.7a: Differences in Impacts of HIP on Grocery Shopping Frequency, by 

Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns Subgroups ............. 439

Exhibit H6.7b: Differences in Impacts of HIP on Probability of Going Out of Way to Shop 

for Fruits & Vegetables at a Particular Store, by Baseline Intake, Attitudes, 

and Pre-HIP Shopping Patterns Subgroups ........................................................ 440

Exhibit H6.8: Proportion of Primary Shoppers Who Reported Changing Store for 

Purchasing Fruits & Vegetables Because of HIP, by Subgroup: Round 2 & 

3 Participant Surveys, HIP Participants .............................................................. 441

Exhibit H6.9a: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Other Store Has Greater Variety of Fruits and 

Vegetables, by Subgroup: Round 2 & 3 Participant Surveys, HIP 

Participants ......................................................................................................... 442

Exhibit H6.9b: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Price of Fruits and Vegetables More Affordable at 

Other Store, by Subgroup: Round 2 & 3 Participant Surveys, HIP 

Participants ......................................................................................................... 443

Exhibit H6.9c: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Other Store Has Fresh Fruits and Vegetables, by 

Subgroup: Round 2 & 3 Participant Surveys, HIP Participants ......................... 444

Exhibit H6.9d: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Other Store Participates in HIP, by Subgroup: 

Round 2 & 3 Participant Surveys, HIP Participants ........................................... 445

Exhibit H6.9e: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables for Some Other Reason, by Subgroup: Round 2 & 3 

Participant Surveys, HIP Participants ................................................................. 446

Exhibit H7.1a: Differences in Impacts of HIP on Self-Reported Receipt of Messages about 

Fruits and Vegetables in Past 3 Months, by Baseline Intake and Attitude 

Subgroups ........................................................................................................... 447

Exhibit H7.1b: Differences in Impacts of HIP on Self-Reported Attendance at Nutrition 

Education Class or Program in Past 3 Months, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 448

Exhibit H7.2a: Differences in Impacts of HIP on Level of Agreement that “I enjoy trying 

new foods,” by Baseline Intake and Attitude Subgroups ................................... 449

Exhibit H7.2b: Differences in Impacts of HIP on Level of Agreement that “I enjoy trying 

new fruits,” by Baseline Intake and Attitude Subgroups .................................... 450

Exhibit H7.2c: Differences in Impacts of HIP on Level of Agreement that “I enjoy trying 

new vegetables,” by Baseline Intake and Attitude Subgroups ........................... 451

Exhibit H7.2d: Differences in Impacts of HIP on Level of Agreement that “I eat enough 

fruits to keep me healthy,” by Baseline Intake and Attitude Subgroups ............ 452
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. xxi

Exhibit H7.2e: Differences in Impacts of HIP on Level of Agreement that “I eat enough 

vegetables to keep me healthy,” by Baseline Intake and Attitude Subgroups .... 453

Exhibit H7.2f: Differences in Impacts of HIP on Level of Agreement that “I often 

encourage family/friends to eat fruits & vegetables,” by Baseline Intake and 

Attitude Subgroups ............................................................................................. 454

Exhibit H7.2g: Differences in Impacts of HIP on Positive Attitudes Toward Food, Fruits, 

& Vegetables Scale, by Baseline Intake and Attitude Subgroups ....................... 455

Exhibit H7.3a: Differences in Impacts of HIP on Level of Agreement that “Hard to eat 

vegetables because don't know how to prepare,” by Baseline Intake and 

Attitude Subgroups ............................................................................................. 456

Exhibit H7.3b: Differences in Impacts of HIP on Level of Agreement that “Hard to eat 

vegetables because hard to find where I shop,” by Baseline Intake and 

Attitude Subgroups ............................................................................................. 457

Exhibit H7.3c: Differences in Impacts of HIP on Level of Agreement that “Hard to eat 

fruits because hard to find where I shop,” by Baseline Intake and Attitude 

Subgroups ............................................................................................................ 458

Exhibit H7.3d: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits 

& vegetables as much as would like because cost too much,” by Baseline 

Intake and Attitude Subgroups ............................................................................ 459

Exhibit H7.3e: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits 

& vegetables as much as would like because they spoil,” by Baseline Intake 

and Attitude Subgroups ....................................................................................... 460

Exhibit H7.3f: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits 

& vegetables as much as would like because family dislikes,” by Baseline 

Intake and Attitude Subgroups ............................................................................ 461

Exhibit H7.3g: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits 

& vegetables as much as would like because I dislike,” by Baseline Intake 

and Attitude Subgroups ....................................................................................... 462

Exhibit H7.3h: Differences in Impacts of HIP on Barriers to Eating Fruits & Vegetables 

Scale, by Baseline Intake and Attitude Subgroups ............................................. 463

Exhibit H7.4a: Differences in Impacts of HIP on How Often Kept from Grocery Shopping, 

by Limited Transportation, by Baseline Intake and Attitude Subgroups ............ 464

Exhibit H7.4b: Differences in Impacts of HIP on How Often Kept from Grocery Shopping, 

by Distance to Grocery Store, by Baseline Intake and Attitude Subgroups ........ 465

Exhibit H7.4c: Differences in Impacts of HIP on Barriers to Grocery Shopping (Composite 

Scale), by Baseline Intake and Attitude Subgroups ............................................ 466

Exhibit H7.5a: Differences in Impacts of HIP on How Often Fruits are Available at Home, 

by Baseline Intake and Attitude Subgroups ........................................................ 467

Exhibit H7.5b: Differences in Impacts of HIP on How Often Fruits are in Refrigerator or 

on Counter, by Baseline Intake and Attitude Subgroups .................................... 468

Exhibit H7.5c: Differences in Impacts of HIP on How Often Vegetables are Available at 

Home, by Baseline Intake and Attitude Subgroups ............................................ 469

Exhibit H7.5d: Differences in Impacts of HIP on How Often Ready-to-Eat Vegetables are 

Available on Counter or in Refrigerator, by Baseline Intake and Attitude 

Subgroups ............................................................................................................ 470
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. xxii ▌Contents Abt Associates

Exhibit H7.5e: Differences in Impacts of HIP on Fruits and Vegetables Available at Home 

Scale, by Baseline Intake and Attitude Subgroups ............................................. 471

Exhibit H7.5f: Differences in Impacts of HIP on How Often Salty Snacks are Available at 

Home, by Baseline Intake and Attitude Subgroups ............................................ 472

Exhibit H7.5g: Differences in Impacts of HIP on How Often 1% Fat, Skim or Fat-Free 

Milk is Available at Home, by Baseline Intake and Attitude Subgroups ........... 473

Exhibit H7.5h: Differences in Impacts of HIP on How Often Soft Drinks are Available at 

Home, by Baseline Intake and Attitude Subgroups ............................................ 474

Exhibit H7.5i: Differences in Impacts of HIP on How Often All or Most of Family Sits 

Down and Eats Evening Meals Together at Home, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 475

Exhibit H7.5j: Differences in Impacts of HIP on How Often Evening Meals are Cooked at 

Home, by Baseline Intake and Attitude Subgroups ............................................ 476

Exhibit H8.1: Differences in Impacts of HIP on Consumption of Targeted Fruits (TF), 

Cup-Equivalents, by Demographic Subgroup, Preferred Restrictive Proxy 

Measure .............................................................................................................. 477

Exhibit H8.2: Differences in Impacts of HIP on Consumption of Targeted Fruits (TF), 

Cup-Equivalents, by Baseline Fruit and Vegetable Behaviors and 

Preferences Subgroup, Preferred Restrictive Proxy Measure ............................. 479

Exhibit H8.3: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

(TV), Cup-Equivalents, by Demographic Subgroup, Preferred Restrictive 

Proxy Measure .................................................................................................... 480

Exhibit H8.4: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

(TV), Cup-Equivalents, by Baseline Fruit and Vegetable Behaviors and 

Preferences Subgroup, Preferred Restrictive Proxy Measure ............................. 482

Exhibit H8.5: Differences in Impacts of HIP on Consumption of Total Fruits and 

Vegetables, Cup-Equivalents, by Demographic Subgroup ................................ 483

Exhibit H8.6: Differences in Impacts of HIP on Consumption of Total Fruits and 

Vegetables, Cup-Equivalents, by Baseline Fruit and Vegetable Behaviors 

and Preferences Subgroup .................................................................................. 485

Exhibit H8.7a: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables (TFV) from Mixed Foods, Cup-Equivalents, by Baseline Intake 

and Attitude Subgroups ...................................................................................... 486

Exhibit H8.7b: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables (TFV), Alternative Inclusive Proxy Measure, Cup-Equivalents, 

by Baseline Intake and Attitude Subgroups ........................................................ 487

Exhibit H8.7c: Differences in Impacts of HIP on Consumption of 100% Juice, CupEquivalents, by Baseline Intake and Attitude Subgroups ................................... 488

Exhibit H8.7d: Differences in Impacts of HIP on Consumption of White Potatoes, CupEquivalents, by Baseline Intake and Attitude Subgroups ................................... 489

Exhibit H8.7e: Differences in Impacts of HIP on Consumption of Legumes, CupEquivalents, by Baseline Intake and Attitude Subgroups ................................... 490

Exhibit H8.7f: Differences in Impacts of HIP on Consumption of Other Fruits and 

Vegetables Acquired Outside Stores, Cup-Equivalents, by Baseline Intake 

and Attitude Subgroups ...................................................................................... 491
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Contents ▌pg. xxiii

Exhibit H8.8a: Differences in Impacts of HIP on Consumption of Targeted Fruit from 

Citrus Fruits, Melons, and Berries, Preferred Restrictive Proxy Measure, 

Cup-Equivalents, by Baseline Intake and Attitude Subgroups ........................... 492

Exhibit H8.8b: Differences in Impacts of HIP on Consumption of Targeted Fruit from 

Other Fruits, Preferred Restrictive Proxy Measure, Cup-Equivalents, by 

Baseline Intake and Attitude Subgroups ............................................................. 493

Exhibit H8.8c: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

from Dark Green Vegetables, Preferred Restrictive Proxy Measure, CupEquivalents, by Baseline Intake and Attitude Subgroups ................................... 494

Exhibit H8.8d: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

from Red and Orange Vegetables, Preferred Restrictive Proxy Measure, 

Cup-Equivalents, by Baseline Intake and Attitude Subgroups ........................... 495

Exhibit H8.8e: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

from Tomatoes, Preferred Restrictive Proxy Measure, Cup-Equivalents, by 

Baseline Intake and Attitude Subgroups ............................................................. 496

Exhibit H8.8f: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

from Other Red and Orange Vegetables, Preferred Restrictive Proxy 

Measure, Cup-Equivalents, by Baseline Intake and Attitude Subgroups ............ 497

Exhibit H8.8g: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

from Starchy Vegetables, Preferred Restrictive Proxy Measure, CupEquivalents, by Baseline Intake and Attitude Subgroups ................................... 498

Exhibit H8.8h: Differences in Impacts of HIP on Consumption of Targeted Vegetables 

from Other Vegetables, Preferred Restrictive Proxy Measure, CupEquivalents, by Baseline Intake and Attitude Subgroups ................................... 499

Exhibit H8.9a: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, 100% Juice, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 500

Exhibit H8.9b: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Fruit, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 501

Exhibit H8.9c: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Salad, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 502

Exhibit H8.9d: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Fried Potatoes, Cup-Equivalents, by Baseline Intake 

and Attitude Subgroups ....................................................................................... 503

Exhibit H8.9e: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Other Potatoes, Cup-Equivalents, by Baseline Intake 

and Attitude Subgroups ....................................................................................... 504

Exhibit H8.9f: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Beans, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 505

Exhibit H8.9g: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Other Vegetables, Cup-Equivalents, by Baseline Intake 

and Attitude Subgroups ....................................................................................... 506
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. xxiv ▌Contents Abt Associates

Exhibit H8.9h: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Tomatoes, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 507

Exhibit H8.9i: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Salsa, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups ............................................................................................. 508

Appendix I 

Exhibit I.1: Example of a Single Mediator Model ................................................................. 527

Exhibit I.2: Logic Model ....................................................................................................... 531

Exhibit I.3: Concurrent Mediation Models Linking HIP to Fruit and Vegetable 

Consumption (TFV, Cup-Equivalents) ............................................................... 535

Exhibit I.4: Temporal Mediation Models Linking HIP to Fruit and Vegetable 

Consumption (TFV, Cup-Equivalents) ............................................................... 535

Exhibit I.5: Concurrent Mediation Models of Additional Pathways ..................................... 536

Exhibit I.6: Temporal Mediation Models of Additional Pathways ........................................ 537

Exhibit I.7: Differences in Survey- and EBT-Transactions-Based Spending Measures, 

by Awareness of HIP, Treatment Group ............................................................ 544

Exhibit I.8: Differences in Survey- and EBT-Transactions-Based Spending Measures, 

by Self-Reported Understanding of How HIP Works, Treatment Group ........... 545

Exhibit I.9: Differences in Survey- and EBT-Transactions-Based Spending Measures, 

by Self-Reported Understanding of Which Fruits & Vegetables Qualify to 

Earn the HIP Rebate, Treatment Group .............................................................. 546

Exhibit I.10: Differences in Survey- and EBT-Transactions-Based Spending Measures, 

by Whether Primary Shopper Keeps Track of HIP Rebate, Treatment 

Group .................................................................................................................. 547

Exhibit I.11: Differences in Consumption of Fruits & Vegetables and Disaggregated 

Components, Cup-Equivalents, by Awareness of HIP, Treatment Group ......... 549

Exhibit I.12: Differences in Consumption of Fruits & Vegetables and Disaggregated 

Components, Cup-Equivalents, by Self-Reported Understanding of How 

HIP Works, Treatment Group............................................................................. 550

Exhibit I.13: Differences in Consumption of Fruits & Vegetables and Disaggregated 

Components, Cup-Equivalents, by Self-Reported Understanding of Which 

Fruits & Vegetables Qualify to Earn the HIP Rebate, Treatment Group ........... 551

Exhibit I.14: Differences in Consumption of Fruits & Vegetables and Disaggregated 

Components, Cup-Equivalents, by Whether Primary Shopper Keeps Track 

of HIP Rebate, Treatment Group ........................................................................ 552
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Glossary ▌pg. xxv

Glossary of Acronyms 

BEACON DTA’s client-server based eligibility application system 

CBOs Community based organizations 

CPs Community partners 

DTA Massachusetts Department of Transitional Assistance 

EBT Electronic Benefit Transfer 

EOHHS Massachusetts Executive Office of Health and Human Services 

EPPIC Electronic Payment Processing and Information Control (Xerox’s EBT system) 

FNS Food and Nutrition Service 

HIP Healthy Incentives Pilot 

HoH Head of household 

IECR Integrated Electronic Cash Register 

JAD Joint Application Design 

NDG Novo Dia Group 

PIN Personal identification number 

POS Point of sale 

RFP Request for Proposals 

SNAP Supplemental Nutrition Assistance Program 

TFVs Targeted fruits and vegetables 

TPP Third-party processor 

USDA United States Department of Agriculture 

WIC Special Supplemental Nutrition Program for Women Infants and Children 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Executive Summary ▌pg. 1

Executive Summary 

The Food, Conservation, and Energy Act of 2008 authorized funds for pilot projects to determine if 

financial incentives provided at the point of sale to Supplemental Nutrition Assistance Program 

(SNAP) participants would increase their consumption of fruits, vegetables, and other healthful foods. 

On the basis of this legislative authority, USDA’s Food and Nutrition Service (FNS) designed the 

Healthy Incentives Pilot (HIP) to investigate the impact of making fruits and vegetables more 

affordable for SNAP participants. 

Under HIP, SNAP participants received on their SNAP Electronic Benefit Transfer (EBT) card an 

incentive of 30 cents for every dollar of SNAP benefits that they spent on targeted fruits and 

vegetables (TFVs) in participating retailers. TFVs included fresh, canned, frozen, and dried fruits and 

vegetables without added sugars, fats, oils or salt (with some exceptions), but excluded white 

potatoes, mature legumes, and 100% fruit juice (the same set of fruits and vegetables eligible for the 

WIC Fruit and Vegetable Cash Value Voucher). The incentive was capped at $60 per household per 

month to prevent misuse and ensure that total incentive payments would not exceed $2 million. 

The pilot was implemented by the Massachusetts Department of Transitional Assistance (DTA) in 

Hampden County beginning in late 2011 and continuing through the end of 2012. Located in western 

Massachusetts, the county is a mix of urban, rural, and suburban areas with 55,095 SNAP households 

in July 2011. Hampden County has the lowest median household income in the State. Massachusetts, 

like the rest of the country, is in the midst of an obesity epidemic, and residents in the western region 

have the highest rates of obesity and related chronic illness. 

HIP was evaluated using a rigorous research design where SNAP households in Hampden County 

were randomly assigned to either the HIP group or the non-HIP group. Comparisons between the two 

groups thus provide a reliable estimate of the impact of HIP. 

Abt Associates Inc. and its partners, Westat and MAXIMUS, conducted the evaluation for the USDA 

Food and Nutrition Service. This Executive Summary provides a high-level overview of the 

evaluation design and key findings from the HIP evaluation Final Report. 

Evaluation Objectives 

The goal of the HIP evaluation was to assess the impact of the financial incentive on participants’ 

intake of fruits and vegetables. Within this broad goal, FNS identified five specific objectives: 

1. Assess the causal impact of HIP on fruit and vegetable consumption by SNAP participants 

and on other key measures of dietary intake 

2. Identify and assess factors that influence the impact of HIP 

3. Describe the processes involved in implementing and operating HIP 

4. Assess HIP’s impact on the grantee (the State SNAP agency), the local SNAP agency, and 

their partners (including retailers, State EBT processor, and community organizations) 

5. Quantify, to the extent possible, the Federal, State, and local administrative and benefit costs 

of the pilot 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 2 ▌Executive Summary Abt Associates

This Final Report addresses these five research objectives, analyzing all the data collected during the 

evaluation period. 

Design and Data Collection 

Evaluation Design 

To compare food intake and other outcomes for HIP participants relative to outcomes for otherwise 

similar non-participants, the HIP evaluation used a random assignment research design, which is 

widely viewed as providing the strongest evidence of causal impact. Of the 55,095 SNAP households 

in Hampden County, 7,500 households (the “HIP group”) were randomly assigned to participate in 

HIP, while the remaining 47,595 SNAP participating households (the “non-HIP group”) continued to 

receive SNAP benefits as usual. 

The HIP households were divided into three waves of 2,500 households each, to begin earning 

incentives during the pilot’s first three months of operation. The first wave began earning the HIP 

incentive on November 1, 2011, the second wave on December 1, 2011, and the third wave on 

January 1, 2012. HIP participants were eligible to earn incentives for 12 months, ending in December 

2012. 

A random subsample of approximately 5,000 households, equally divided between the HIP and nonHIP groups was selected to participate in survey data collection. 

Data Collection 

We conducted a wide array of qualitative and quantitative data collection activities as part of the HIP 

evaluation. To collect the necessary data on dietary intake, trained telephone interviewers conducted 

24-hour dietary recall interviews, a widely used, reliable methodology. Respondents were also asked 

about their attitudes and preferences for fruits and vegetables, shopping patterns, food expenditures, 

and household characteristics. 

The data collection for the HIP evaluation included three rounds of participant surveys. Round 1 was 

conducted before HIP implementation. Rounds 2 and 3 were conducted during HIP, one fielded 4 to 6 

months after implementation, and the other fielded 9 to11 months after implementation. Both Round 

2 and Round 3 surveys collected information on dietary intake using 24-hour dietary recall 

interviews. Two rounds of focus groups were also conducted with HIP participants, corresponding to 

the Round 2 and Round 3 surveys. 

Several other types of data were collected as part of the evaluation. EBT transaction data provided 

detailed information on households’ SNAP EBT purchases, including HIP-eligible purchases and the 

incentive amounts earned. Two rounds of retailer surveys, three rounds of store observations, and 

three rounds of interviews with key stakeholders provided information on HIP operations and 

implementation, the effect of the pilot on stakeholders, and estimated costs of the pilot and of 

nationwide expansion of HIP. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Executive Summary ▌pg. 3

Findings 

The evaluation findings are discussed below, including: 

 Impacts of HIP on participants 

 Effect of HIP on Hampden County retailers 

 HIP implementation process, costs, and feasibility of nationwide expansion 

HIP Impacts on Participants 

In this section, we discuss our evaluation results regarding the main goal of HIP: increasing 

participants’ consumption of targeted fruits and vegetables (the single confirmatory outcome we 

specified prior to analyzing the data). We then discuss how HIP influenced households’ expenditures 

on targeted fruits and vegetables. Finally, we discuss HIP impacts on participant knowledge, attitudes, 

and behaviors toward fruits and vegetables. Unless otherwise noted, we only discuss HIP group/nonHIP group differences that are statistically significant at conventional significance levels (p<0.05). 

Consumption of Targeted Fruits and Vegetables (TFVs) 

The primary impact measure for the HIP evaluation is the difference in targeted fruit and vegetable 

consumption for HIP and non-HIP participants, combining reports from all dietary interviews 

conducted in both Rounds 2 and 3. Our results show that HIP participants (adults aged 16 and older1

) 

consumed significantly more targeted fruits and vegetables per day—almost a quarter of a cupequivalent—than did non-participants (ES.1). 

This represents an increase in consumption of 26 percent over non-HIP group members. 

Approximately 60 percent of the observed difference was due to a difference in consumption of 

targeted vegetables and 40 percent due to a difference in consumption of targeted fruit. 

There is no evidence that the impact of HIP was affected by the presence of children in the household, 

employment status, age, or amount of the household’s SNAP benefit. However, there is some 

evidence that impacts were larger for those who, prior to HIP, had more positive attitudes about fruits 

and vegetables. 

 

1

 The sample was intended to be representative of all types of SNAP households; respondents aged 16 and 

17, who accounted for approximately 6 percent of sampled respondents, were therefore included in the 

sample as they can be SNAP heads of households. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 4 ▌Executive Summary Abt Associates

Exhibit ES.1: HIP Participants Consumed 0.24 Cup-Equivalent More Fruits and Vegetables 

per Day 

Expenditures on Fruits and Vegetables 

HIP impacts on consumption were expected to be larger if participants responded strongly to the price 

incentive and purchased more fruits and vegetables, thus earning more incentives. Therefore, 

spending measures are important intermediate variables in the evaluation. 

SNAP EBT data show that, in an average month, two-thirds of HIP households that received SNAP 

benefits earned some HIP incentive and the other one-third earned no incentives. Households with 

higher SNAP benefits, with children in the household, and with Hispanic or Asian heads were more 

likely to earn incentives. On average, from March through October 2012, HIP households spent just 

over $12 per month on targeted fruits and vegetables in participating stores (representing 5 percent of 

their SNAP benefits), earning an average incentive of $3.65 each month. Average monthly purchases 

of targeted fruits and vegetables were similar over all the months the pilot operated. 

HIP caused households to spend more than non-HIP households on targeted fruits and vegetables in 

participating stores. (We do not have direct measures of fruit and vegetable expenditures in nonparticipating stores, which accounted for about half of all SNAP purchases by sample members.) The 

EBT data allow us to compare the TFV purchases of HIP and non-HIP households in 

supermarkets/superstores that participated in HIP. In these supermarkets/superstores, non-HIP 

households spent $10.86 each month using their EBT card on TFVs, and HIP households spent 

$12.05. This represents an increase of $1.19 or 11 percent (ES.2). 

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Mean Cup-Equivalents (Regression-adjusted)

HIP group (N=1,954 recalls from 1,010 respondents)

Non-HIP group (N=1,959 recalls from 999 respondents)

} impact = 0.24 cup-equivalent

1.15

cup-equivalents

0.91 

cup-equivalent
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Executive Summary ▌pg. 5

Exhibit ES.2: HIP Households Purchased More Fruits and Vegetables per Month 

A separate measure of fruit and vegetable spending comes from survey respondents, who reported 

their usual expenditures on all groceries, and on all fruits and vegetables. HIP positively affected selfreported spending on total fruits and vegetables. HIP participants reported spending $78.17 each 

month on fruits and vegetables (21 percent of their total grocery spending) while non-HIP households 

reported spending $72.02 (19 percent of their grocery spending). This difference of $6.15 represents 

an 8.5 percent increase in spending (Exhibit ES.2). This measure differs from the EBT-based measure 

of TFV spending because it includes fruit and vegetable purchases in non-participating stores, 

purchases made with cash or other forms of payment, and purchases of fruits and vegetables that did 

not qualify to earn incentives (e.g., white potatoes, legumes, fruit juice). 

HIP did not lead households to change their shopping patterns—where they purchased groceries or 

how frequently they shopped—to any great extent. HIP gave households some impetus to make their 

purchases of TFVs in participating stores, so they could earn the incentive. However, it does not 

appear that HIP participants responded significantly to this motivation. While participants spent about 

half their SNAP benefits in participating stores, they spent the other half in non-participating stores. 

(A similar pattern was observed for non-HIP households.) 

HIP survey respondents reported changes in fruit and vegetable purchasing, saying that they bought 

larger amounts and a greater variety of fruits and vegetables because of HIP. These households felt 

that fruits and vegetables had become more affordable due to HIP. Just over one-quarter of HIP 

households reported that they changed where they purchased fruits and vegetables, generally to have 

access to fresh produce, a greater variety of fruits and vegetables, and more affordable prices. 

Because we see no corresponding pattern in the EBT transaction data, it seems likely that these 

responses reflect only small behavioral changes, such as occasionally switching fruit and vegetable 

purchases between two stores that the participant already patronized. 

$12.05

$78.17

$10.86

$72.02

$0.00

$10.00

$20.00

$30.00

$40.00

$50.00

$60.00

$70.00

$80.00

$90.00

TFV spending with SNAP benefits in

participating supermarkets/superstores

(EBT data)

All fruit and vegetable purchases (survey

data)

HIP Group Non-HIP Group

Impact = $1.19 

Impact = $6.15 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 6 ▌Executive Summary Abt Associates

Knowledge, Attitudes, and Behaviors 

For the HIP incentive to affect purchasing behavior, HIP participants needed to know about the 

program and understand how it worked. Findings from the participant survey suggest that a sizeable 

proportion of HIP participants did not fully understand the pilot program, though understanding 

increased over the course of the pilot. 

While 62 percent of HIP participants reported that they had heard about HIP when asked in the Round 

2 survey (4 to 6 months after implementation), 38 percent reported that they had not heard about HIP. 

At the time of the Round 3 survey, which occurred 9 to11 months after HIP implementation, 24 

percent of HIP participants reported that they had not heard about the pilot; the other 76 percent had 

heard of the pilot. This increase in awareness was statistically significant. 

Thirty-eight percent of Round 2 respondents and 25 percent of Round 3 respondents reported that it 

was hard or somewhat hard (or they didn’t know how hard it was) to understand how HIP worked. 

Similar percentages said it was hard or somewhat hard (or they didn’t know how hard it was) to 

remember which fruits and vegetables qualified for the HIP incentive. The improved understanding 

reported by participants between Round 2 and Round 3 is statistically significant. Focus group 

participants also described difficulties understanding the pilot. Exploratory analyses suggest that 

limited understanding of HIP was associated with lower spending on TFVs using EBT benefits. 

Attitudes toward and preferences for fruits and vegetables may affect both the level of fruit and 

vegetable intake and how responsive participants were to the incentive. Respondents were asked 

several questions about their food preferences and about perceived barriers to consuming fruits and 

vegetables. Survey respondents generally had positive attitudes toward fruits and vegetables and did 

not report overwhelming barriers to their consumption. Generally, HIP and non-HIP households had 

similar preferences and beliefs. 

Past research has shown that having fruits and vegetables in the family food environment is 

associated with increased consumption. As reported in both surveys conducted after HIP began, HIP 

households had fruits and vegetables available at home more often than did households not 

participating in HIP. 

Effects of HIP on Retailers 

The participation of retailers was critical to the success of HIP. If the HIP incentive was to have an 

impact on food intake, SNAP participants needed to have easy and adequate access to retailers 

participating in the pilot. DTA succeeded in recruiting approximately 130 stores to participate in HIP, 

including supermarkets, superstores, grocery stores, convenience stores, and farmers markets. 

However, not all households had similar access to participating stores that sold substantial quantities 

of fruits and vegetables, particularly supermarkets, superstores, and grocery stores. All but one major 

supermarket/superstore chain in Hampden County participated in HIP, but the non-participating chain 

had a significant presence in the county. As a result, the stores participating in HIP accounted for just 

59 percent of total Hampden County SNAP redemptions. Pilot impacts, particularly the relatively low 

level of incentives earned by households, were likely affected by the limited retailer participation. 

Because HIP required the cooperation of retailers, understanding HIP’s effect on retailers’ businesses 

is critical to the potential long-run feasibility of the program. The evaluation found that HIP had 

relatively little impact on store operations. At the outset of the pilot, some retailers, particularly 

smaller stores, were concerned that HIP might increase the time and effort required to process SNAP 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Executive Summary ▌pg. 7

purchases. However, according to store managers, this does not appear to have been the case. In 

addition, few retailers reported problems during the pilot. The most common problems and questions, 

which were reported by stores without integrated electronic cash registers, concerned identifying HIPeligible items and identifying customers participating in the pilot who could earn incentives. 

HIP potentially could have induced retailers to increase their stocks of fruits and vegetables in order 

to attract HIP households, but this does not appear to have happened to any great extent during the 

pilot. Some of the smaller grocery stores, however, reported that they made additional efforts to sell 

more fruits and vegetables, including increasing their offerings. 

Overall, a majority of retailers reported that they were somewhat or very satisfied with how HIP 

worked in their stores, though satisfaction varied depending on the type of store (Exhibit ES.3). The 

vast majority of supermarkets/superstores and grocery stores reported that they were satisfied with 

HIP. In contrast, only one-third of convenience store managers were satisfied; the rest were neither 

satisfied nor dissatisfied. These findings are likely explained by the fact that convenience stores 

carried relatively few HIP-eligible items, which limited the potential benefits for them.2

 

Exhibit ES.3: Percent of Retailers Somewhat or Very Satisfied with How HIP Worked in Their 

Stores 

The incentives earned by HIP households increased SNAP redemptions at Hampden County retailers. 

However, because the amount of incentives earned was fairly small, the impact on retailer sales was 

also fairly small. Most SNAP spending occurs in supermarkets/superstores and grocery stores, and 

these stores benefitted from the increased spending. 

 

2

 The Agricultural Act of 2014 (Section 4002) established requirements for SNAP retailers to stock more 

perishable foods. These requirements may increase the proportion of SNAP retailers carrying HIP-eligible 

fruits and vegetables, thus increasing the benefits of participating in HIP. 

93%

83%

32%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Supermarkets/Superstores Grocery stores Convenience stores
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 8 ▌Executive Summary Abt Associates

HIP Implementation, Costs, and Feasibility of Expansion 

HIP was an innovative and complex project. Planning and implementation was difficult, requiring 

DTA to coordinate the work of several different entities to ensure the pilot was up and running in 15 

months. While the implementation process posed many challenges, DTA succeeded in implementing 

the pilot on schedule. 

Key planning and implementation activities included: 

 Designing and implementing EBT system changes that included developing software to 

identify when incentives were earned, to calculate the incentive amount to credit to HIP 

clients, and to draw down bank funds to pay retailers for food purchases 

 Recruiting retailers to participate in HIP was critical to the success of the pilot as HIP 

participants needed to be able to locate and access stores in which they could earn incentives 

 Developing training materials and notifications for HIP participants so that they 

understood the purpose of the pilot, were able to locate retailers, and were able to identify and 

purchase qualifying fruits and vegetables 

Total costs for implementing HIP, including the incentives earned by HIP participants, were $4.4 

million. The majority of the costs (55 percent) were incurred for system design, development, and 

testing for both EBT and retailer system changes. Retailer recruitment and participant notification and 

training accounted for an additional 14 percent of implementation costs. General administrative 

expenses for management and oversight of HIP accounted for 16 percent. Most of the remaining 10 

percent of costs were incurred in support of the evaluation. Incentive payments to HIP participants 

over the course of the pilot represented the smallest proportion of total costs—just 6 percent. 

The experience in Hampden County demonstrated that implementing and operating HIP was both 

technically and operationally feasible. Most stakeholders interviewed in the course of the evaluation, 

including DTA, EBT systems developers/operators, and large chain retailers, indicated strong support 

for expanding HIP nationwide. The one exception was convenience store chain retailers, who as 

noted above carried a limited selection of HIP-eligible items. Nationwide expansion of HIP would 

need to consider whether retailer participation would be voluntary or mandatory for some or all 

SNAP-authorized retailers. 

Based on the HIP experience and input from industry experts, the initial start-up costs to expand HIP 

nationwide to all 50 States, DC, Guam, and the Virgin Islands is projected to be $89.9 million. This 

includes the costs for State agency activities, all EBT and retailer systems modifications, and retailer 

and participant training materials. While these would be mainly one-time expenditures, payment of 

the HIP incentive to households would be ongoing in a nationwide expansion. The annual value of 

incentives earned would depend on participant behaviors that cannot be predicted fully from the pilot. 

Estimates of incentives based on plausible scenarios range from $0.8 billion to $4.5 billion annually. 

Conclusions 

Findings from the HIP evaluation indicate that HIP had positive impacts on targeted fruit and 

vegetable consumption of pilot participants. HIP participants consumed almost one-quarter cup (26 

percent) more targeted fruits and vegetables each day than did non-HIP respondents. This HIP impact 

is both statistically significant and large enough to be nutritionally relevant. Diets with increased fruit 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Executive Summary ▌pg. 9

and vegetable consumption have strong associations with reduced risk of heart disease, stroke, and 

several cancers. The HIP impact was sufficiently large to narrow the gap between current 

consumption and the Healthy People 2020 objectives for total fruit and vegetable intake. The onequarter cup increase associated with HIP reduces this “total fruit and vegetable intake gap” by 

approximately 18 percent. 

Further research to understand the underlying mechanisms by which HIP impacted fruit and vegetable 

consumption would be beneficial. The evaluation clearly shows that households responded to the 

price incentive and increased their consumption of fruits and vegetables. However, we cannot explain 

all of HIP’s impact on consumption through its impact on TFV purchases in participating stores. HIP 

may also have worked through other mechanisms, such as informational and attitudinal pathways. 

Additional studies to help disentangle possible explanations for the observed effects could provide 

valuable information for policymakers. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 1. Introduction ▌pg. 11

1. Introduction 

The Food, Conservation, and Energy Act of 2008 authorized funds for pilot projects to determine if 

financial incentives provided at the point of sale to Supplemental Nutrition Assistance Program 

(SNAP) participants would increase their consumption of fruits, vegetables, and other healthful foods. 

On the basis of this legislative authority, USDA’s Food and Nutrition Service (FNS) designed the 

Healthy Incentives Pilot (HIP) to investigate the impact of making fruits and vegetables more 

affordable for SNAP participants. 

Serving 46.6 million low-income Americans at an annual cost of $78.4 billion in 2012 (the year HIP 

operated), SNAP is the nation’s largest nutrition assistance program and a cornerstone of the social 

safety net. The primary goals of SNAP are to prevent food insecurity and hunger and to promote 

dietary quality. In addition to directly increasing the food purchasing power of program participants, 

SNAP also provides nutrition education programs to help families improve their diets. USDA’s 

economic models suggest that SNAP benefits lead to substantial increases in economic activity and 

employment in the food and agriculture sector, with follow-on effects throughout the macro-economy 

(Hanson, 2010). SNAP provides benefits to qualifying households on electronic benefit cards (EBT) 

cards, similar to bank debit cards, which allow participants to purchase foods and nonalcoholic 

beverages through authorized retailers. 

Increasing fruit and vegetable intake is one of several leading strategies recommended by U.S. public 

health authorities for promoting dietary quality (USDHHS, 2010; USDA and USDHHS, 2010). Most 

U.S. adults fail to meet the Dietary Guidelines for fruit and vegetable intake, and intake shortfalls are 

comparatively large for low-income Americans and SNAP participants. Improving dietary quality 

could help serve key national public health objectives for reducing rates of chronic disease and 

obesity (USDHHS, 2010). 

Under HIP, SNAP participants were offered an incentive of 30 cents for every dollar spent on 

targeted fruits and vegetables (TFVs), which included fresh, frozen, canned, and dried fruits and 

vegetables without added sugars, fats, oils or salt (with some exceptions). These are the same fruits 

and vegetables eligible for the Fruit and Vegetable Cash Value Voucher in the Special Supplemental 

Nutrition Program for Women, Infants, and Children (WIC).3

 For every SNAP dollar a household 

spent on TFVs, they immediately earned a 30 cent credit on their EBT card. The incentive could then 

be spent on any SNAP-eligible foods and beverages. The incentive was capped at $60 per household 

per month to prevent misuse and ensure that total incentive payments would not exceed $2 million. 

The cap did not appear to constrain households as very few households reached it. 

The HIP evaluation used a random assignment research design, which is viewed as providing the 

strongest evidence of causal impact. SNAP participants in the pilot site of Hampden County, 

Massachusetts, were randomly assigned to a HIP group, eligible to earn the HIP incentive, and a nonHIP group, not eligible to earn the HIP incentive but which continued to receive SNAP benefits as 

usual. Comparing differences in outcomes across these groups yields rigorous evidence on impacts 

attributable to HIP. 

 

3

 FNS guidance on minimum requirements and specifications for WIC fruits and vegetables is presented in 

Appendix F, Exhibit F1.1. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 12 ▌1. Introduction Abt Associates

This Final Report provides impact estimates on food intake and other outcomes from the HIP 

evaluation. It describes HIP’s effect on SNAP participants, food retailers, and other stakeholders. 

In this chapter, we first discuss the background of the pilot, focusing on the context in which it was 

developed and providing an overview of the pilot site. The second section discusses the research 

objectives of the evaluation, and the third section presents the theory and conceptual model 

underlying HIP. The fourth section discusses research on previous nutrition interventions to promote 

fruit and vegetable consumption. The final section provides a guide to the organization of this report. 

1.2 Pilot Background 

The Federal government’s Healthy People 2020 objectives and the Dietary Guidelines for Americans 

(DGA) emphasize the goal of promoting fruit and vegetable intake to reduce the risk of obesity and 

chronic disease (USDHHS, 2010; USDA and USDHHS, 2010). Epidemiological evidence suggests 

that fruit and vegetable intake reduces the long-term risk of obesity (He et al., 2004). After controlling 

for other explanatory factors, higher fruit and vegetable intake is associated with lower rates of heart 

disease (He et al., 2006, He et al., 2007) and several cancers (World Cancer Research Fund, 2007), 

major causes of death in the United States. 

In keeping with this evidence, public health authorities in recent years have emphasized increasing 

fruit and vegetable intake. The Healthy People 2020 objectives seek to increase adult intake of fruits 

and vegetables from a baseline of 1.3 cup-equivalents per 1,000 calories to 2.0 cup-equivalents per 

1,000 calories (where one cup-equivalent is equivalent to approximately two servings of fruit or 

vegetables). The Dietary Guidelines for Americans, 2010, describe a healthy diet as one that 

emphasizes fruits and vegetables, along with whole grains and fat-free or low-fat milk and milk 

products. The USDA MyPlate consumer education graphic encourages Americans to fill half their 

plate with fruits and vegetables. According to USDA’s Strategic Plan for Fiscal Years (FY) 2010-

2015, the department aims to double the number of Americans who eat five or more servings of fruits 

and vegetables daily. One of the plan’s strategies is to “promote the increased consumption of fruits 

and vegetables among the general population.” 

Most U.S. adults fail to meet fruit and vegetable intake goals as shown in studies conducted since the 

mid-1970s (Dong and Lin, 2009). Data from the National Health and Nutrition Examination Survey 

(NHANES) 1999-2002 showed that only 28 percent of adults met the 2005 DGA guidelines 

recommending daily consumption of greater than or equal to 2 servings of fruit. Excluding 100 

percent fruit juice, only 18 percent met these guidelines. Similarly, only 33 percent of adults met the 

DGA guidelines recommending daily consumption of greater than or equal to 3 servings of 

vegetables. Excluding fried potatoes, only 28 percent met these guidelines. Comparing these results to 

previous NHANES estimates from 1976-1980 shows no evidence of improvement over the years 

(Casagrande et al., 2007). Based on 2009 surveillance data with a different outcome variable (the 

number of times daily that each food was consumed), 32.4 percent of adults had fruit at least twice 

daily, and 26.3 percent of adults had vegetables at least three times daily (Grimm et al., 2012). 

Fruit and vegetable intake falls short particularly for low-income Americans and participants in 

SNAP (Guthrie et al., 2007). The Institute of Medicine in 2013 expressed concern that SNAP 

participants lack adequate food retail access and the ability to purchase fresh fruits and vegetables 

(IOM and NRC, 2013). Based on 2009 surveillance data, low-income adults, with household income 

below 130 percent of the Federal poverty standard, were less likely to have fruit at least 2 times daily 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 1. Introduction ▌pg. 13

or vegetables at least 3 times daily, compared with adults with household income above 400 percent 

of the Federal poverty standard (Grimm et al., 2012). An Abt Associates study estimated Healthy 

Eating Index component scores for SNAP participants and non-participants, based on their fruit and 

vegetable intake reported in NHANES (1999-2004). Each component score was reported as a 

percentage of the recommended food intake quantity. The fruit component score (whole fruit, 

excluding 100% fruit juice) was 50 percent of the recommendation for SNAP participants, 66 percent 

of the recommendation for income-eligible non-participants, and 74 percent of the recommendation 

for higher-income non-participants. The vegetable component score was 58 percent of the 

recommendation for SNAP participants, 64 percent of the recommendation for income-eligible nonparticipants, and 66 percent of the recommendation for higher-income non-participants (Cole and 

Fox, 2008). With earlier data from the Continuing Survey of Food Intake by Individuals (CSFII), 

Wilde, McNamara, and Ranney (2000) found that, compared with eligible non-participants, SNAP 

participants had higher intake of added sugars and total fats but not of fruits and vegetables. 

The Healthy Incentives Pilot Site 

Massachusetts was selected to implement HIP through a competitive application process in August 

2010. The State was selected based on its comprehensive pilot application that included strong 

design, implementation, staffing, and management plans. The State’s management plan included 

significant support from community partners and a wide variety of retailers and farmers markets that 

accepted EBT. 

Hampden County, the HIP pilot site, has the lowest median household income in the State and is 

located in western Massachusetts, where residents have the highest rates of obesity and related 

chronic illnesses in the State.4

 The site is self-contained, which means that most SNAP participants 

live, work, and shop within Hampden County. Thus site boundary issues are not significant, 

providing households sufficient opportunities to earn incentives and allowing for a strong test of the 

intervention. Although studying HIP in a single site limits the ability to generalize the findings to the 

national context, it has the compelling offsetting advantage of permitting the evaluation to have a 

sound random assignment research design. 

HIP was rolled out in three waves: one-third of households began earning HIP incentives on 

November 1, 2011, the second and third waves began participating on December 1, 2011, and January 

1, 2012, respectively. HIP participants were able to earn incentives for 12 months, ending in 

December 2012. 

1.3 Research Objectives 

The HIP evaluation used a rigorous research design in which SNAP participants in Hampden County 

were randomly assigned to a HIP group or a non-HIP group. The overall goal of the evaluation was to 

assess the impact of HIP on participants’ intake of fruits and vegetables. Within this broad goal, FNS 

identified five research objectives: 

1. Assess the causal impact of HIP on fruit and vegetable consumption by SNAP participants, 

and on other key measures of dietary intake 

 

4

 Chapter 2, Exhibit 2.7 provides descriptive characteristics of the SNAP population in Hampden County and 

compares the population to the national SNAP population. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 14 ▌1. Introduction Abt Associates

2. Identify and assess factors that influence the impact of HIP 

3. Describe the processes involved in implementing and operating HIP 

4. Assess HIP’s impact on the grantee (the State SNAP agency), the local SNAP agency, and 

their partners (including retailers, State EBT processor, and community organizations) 

5. Quantify, to the extent possible, the Federal, State, and local administrative and benefit costs 

of the pilot 

The primary impact measure for the evaluation is the difference in targeted fruit and vegetable intake 

between the HIP and non-HIP groups. 

The data collection for the HIP evaluation included one participant survey before HIP implementation 

(Round 1), a second participant survey 4 to 6 months after implementation (Round 2), and a third 

participant survey 9 to11 months after implementation (Round 3). These surveys included questions 

for a primary shopper (knowledgeable about food spending and general household characteristics) 

and a randomly sampled person (who reported food intake for a 24-hour recall period). In addition, 

the HIP evaluation included in-depth focus groups with HIP participants, two rounds of retailer 

surveys, on-site visits with retailers, interviews with key informants and stakeholders, EBT 

transactions data, and data on labor and other costs for stakeholders.5

The HIP evaluation was conducted by Abt Associates. The research team included Westat for 

participant data collection, MAXIMUS for stakeholder data collection and cost analysis, and the 

Friedman School of Nutrition Science and Policy at Tufts University for contributions to analysis. 

Two previous reports from the HIP evaluation described the pilot’s early implementation and 

preliminary impact estimates: 

 Early Implementation Report (Bartlett, Beauregard et al., 2013) focused on documenting 

the process of implementing HIP. It describes development activities from pilot inception to 

March 2012 when HIP was fully operational. Chapter 3 of this Final Report is based on the 

Early Implementation Report. 

 Interim Report (Bartlett, Klerman et al., 2013) provided initial exploratory analyses based 

on a limited number of outcome variables for Round 2 (descriptive statistics and some control 

variables in the analysis used data from the baseline period). This report focused on providing 

estimates of fruit and vegetable consumption for HIP and non-HIP groups and other early 

pilot impacts 4 to 6 months after HIP implementation. Based on the Round 2 data, the Interim 

Report showed that HIP had a positive and statistically significant impact on targeted fruit 

and vegetable intake. 

This third HIP evaluation report addresses all five research objectives and analyzes all the data 

collected during the evaluation period. It provides confirmatory estimates of the difference in targeted 

fruit and vegetable intake between the HIP and non-HIP groups, based on combined data from 

Rounds 2 and 3. It examines additional outcomes and includes analyses to better understand the 

process by which HIP affected participants. 

 

5

 Chapter 2 provides a detailed description of data collection activities and the analytic methods used in the 

evaluation. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 1. Introduction ▌pg. 15

The fourth and final HIP evaluation report, HIP Spatial Analysis (Grindal, Schwartz, et al., 2014), 

further investigates how the food retailer environment influenced Hampden County SNAP 

participants’ fruit and vegetable purchases in general and the HIP impact estimates in particular. 

1.4 Theory and Conceptual Model 

Exhibit 1.1 shows a conceptual model of HIP mechanisms developed to guide the data collection and 

analysis for the evaluation. It illustrates the following: 

 The primary and secondary mechanisms that may explain HIP’s impact on participant food 

consumption 

 Internal and external factors that may influence how HIP affects participants 

 Implementation activities that may affect HIP outcomes 

 The role of stakeholders who may be affected by HIP and who account for significant pilot 

costs 

The evaluation measured all constructs shown in this model and estimated the effects along all 

specified pathways. 

Exhibit 1.1: Conceptual Model Describing the Potential Effect of HIP on Food Consumption 

As the leftmost box indicates, HIP had two design elements ultimately intended to increase the 

consumption of TFVs. The primary element was HIP’s price incentive: the pilot lowered the effective 

price of targeted fruits and vegetables purchased with SNAP benefits at participating retailers and 

increased household resources by adding the monetary incentive to the household’s EBT card. In 

addition, HIP provided information and promotional materials to support the pilot. While the pilot did 

not create materials solely for fruit and vegetable promotion, HIP did involve multiple opportunities 

for communication with participants. Letters and brochures featured the HIP logo and images of 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 16 ▌1. Introduction Abt Associates

healthy fruits and vegetables. In addition, the mere fact that fruits and vegetables were selected for a 

special subsidy carried an implicit promotional message. 

HIP was hypothesized to work primarily through the middle tier of the conceptual model. By 

affecting food expenditures and shopping patterns HIP could lead participants to shift more of their 

grocery spending to retailers that participated in HIP, and at these retailers HIP participants could 

increase their expenditures on TFVs (with consequent changes in the remaining budget for other 

foods). These shopping and spending patterns could then lead to changes in the fruits, vegetables, and 

other foods available in the home food environment and to changes in preparing and sharing family 

meals. Increased availability of fruits and vegetables at home could lead to increased fruit and 

vegetable consumption, the ultimate desired outcome. The primary outcome, increased targeted fruit 

and vegetable consumption, could affect decisions about other foods that are complements and 

substitutes, both at home and away from home. 

In addition to this central pathway, we hypothesized that HIP could operate through a secondary set 

of pathways related to attitudes about fruits and vegetables, as depicted in the bottom tier of the 

conceptual model. In addition to affecting food spending directly, HIP could influence the attitudes or 

preferences of the individual responsible for food shopping and of other household members. The 

attitudes of the primary shopper could be influenced by promotional materials or other 

communications about HIP, and they also could be influenced by feedback from other household 

members. Attitudes of these household members, in turn, could be influenced by the foods available 

in refrigerators, on counters, or in storage in the home food environment. Improved attitudes toward 

and preferences for fruits and vegetables among household members would then be expected to 

directly increase fruit and vegetable intake. 

Finally, the top tier of the conceptual model depicts the role of retailers. To achieve its desired effects 

on food expenditures, HIP had to succeed in recruiting participating retailers, who then had to 

implement substantial changes to their integrated electronic cash register (IECR) systems or manual 

check-out processes. HIP could prompt participating retailers to increase their fruit and vegetable 

inventory and increase promotion of these items. Retailers might also change (increase or decrease) 

the prices of fruits and vegetables. These changes to the retail food environment as a result of HIP 

would then be expected to exert further influence on participant shopping and spending. Additionally, 

if HIP participants increased their expenditures on fruits and vegetables, retailers might experience an 

overall increase in sales of these items and also an increase in total SNAP spending. 

The primary focus of the evaluation is the impact of HIP on consumption of targeted fruits and 

vegetables, the foods qualifying for the HIP incentive. We specified this as the one confirmatory 

outcome at the beginning of the study, prior to collecting and analyzing the data. Analyses of all other 

impacts are considered exploratory. 

Factors That Could Influence the Magnitude of HIP Impacts 

Given this theory and conceptual model, several important factors could influence the magnitude of 

HIP impacts. 

 Retailer enrollment. The pilot’s impact is expected to be larger if a higher percentage of 

retailers in the pilot area participated. The more participating retailers, particularly retailers 

that carry a good selection of fruits and vegetables at reasonable prices, the easier it is for 

participants to shop in places where they can earn the incentive. For retailers, participation in 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 1. Introduction ▌pg. 17

HIP was voluntary, but extensive outreach efforts were undertaken to encourage 

participation. 

 Participant experiences. The pilot’s impact is expected to be larger if participants received 

sufficient information and training to understand the incentive, and if the procedures for 

recognizing eligible purchases and thus earning and crediting incentives functioned smoothly. 

If participants do not understand key features of the incentive, such as the potential savings 

from making fruit and vegetable purchases with SNAP benefits instead of cash income, or are 

unaware of the pilot’s existence, then the impact is expected to be smaller. 

 Expenditures and shopping behaviors. Food expenditures are central to the way HIP is 

expected to influence food intake. The overall effect of HIP on fruit and vegetable intake is 

likely to be larger if participants responded to the incentive, purchased more fruits and 

vegetables, and earned more incentives. Hence, HIP purchase amounts and HIP/non-HIP 

differences in food expenditures are important intermediate variables. 

 Attitudes toward fruits and vegetables and the family food environment. Participants’ 

attitudes toward fruits and vegetables, and their usual exposure to fruits and vegetables in the 

family food environment, may affect both the level of fruit and vegetable intake and the 

responsiveness of fruit and vegetable intake to incentives. Even if participant attitudes do not 

change, HIP could have a greater impact on intake for participants with more favorable initial 

attitudes toward fruits and vegetables. 

1.5 Previous Research 

A large body of previously published research informed the design of the pilot and offered some 

hypotheses about possible outcomes. In particular, the following areas of previous research are 

relevant to HIP: 

 Price and income effects on food spending 

 Non-price interventions to promote fruit and vegetable consumption 

 Availability of fruits and vegetables in the food retail environment 

Although informative, the literature consulted generated no firm conclusions about the likely effect of 

HIP. No previous study involved a large-scale financial incentive operating directly through the EBT 

card using normal food retail channels. All five research questions posed at the outset of the HIP 

evaluation remained open empirical questions. 

Price and Income Effects on Food Spending 

This section reviews the existing literature on the impact of changes in prices and income on fruit and 

vegetable spending. We begin with a brief overview of the theoretical issues followed by a survey of 

the general literature on the impact of prices and resources. Finally, we survey evaluations of 

programs intended to shift prices. 

Theoretical Issues 

The HIP incentive is similar to a price reduction by means of a rebate. A HIP participant using SNAP 

benefits at a participating food retailer’s cash register pays the (unchanged) posted price for targeted 

fruits and vegetables, but the incentive essentially reduces the cost of targeted fruits and vegetables by 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 18 ▌1. Introduction Abt Associates

30 percent. At check-out, the participant’s EBT card is immediately credited with additional SNAP 

benefits based on the amount of targeted fruits and vegetables purchased, thus increasing the amount 

of SNAP benefits available for the month. 

The purpose of the price incentive is simple: lower prices are intended to lead participants to consume 

more fruits and vegetables. Some additional insight comes from more formally describing the 

economic theory behind this price effect (see Appendix A). One particular insight is relevant for 

SNAP participants who purchase part of their food with SNAP benefits and part with ordinary cash 

income. If these SNAP participants purchase any fruits and vegetables, they can save money by 

making their purchases at participating HIP retailers and with SNAP benefits instead of cash income. 

For households that have access to a participating retailer, and that have not already exhausted their 

SNAP benefits, TFV purchases with cash income or in non-participating retailers represent a missed 

opportunity to claim the 30 percent incentive. 

In advance of the study, to help with research design and to plan sample size requirements, we 

considered existing information suggesting what effect size a price incentive might generate. The 

economic measure called price elasticity gives the percent change in spending that occurs in response 

to a one percent change in price. Based on price elasticity estimates from Andreyeva, Long, and 

Brownell (2010), a 30 percent decline in the price of fruits and vegetables would be expected to 

generate an increase in fruit and vegetable spending of roughly 19 percent.6

 Assuming that consumers 

change their food intake in proportion to changes in their food spending, the expected change in 

spending provides an estimate of the expected change in food intake from a simple price subsidy. 

Assessing current per-adult intake at 1.3 cup-equivalents per 1,000 kilocalories (USDHHS, 2010), or 

roughly 2.6 cup-equivalents in a 2,000-kilocalorie daily diet, this effect would generate approximately 

half of a cup-equivalent in increased intake of all fruits and vegetables.7

 The HIP evaluation was 

designed with sufficient statistical power to detect a HIP/non-HIP difference of 0.25 cup-equivalent 

or larger; thus we expected to be able to detect impacts of the size predicted by the elasticity 

estimates. 

Of course, the actual impact could not be known until the pilot was conducted. The pilot differed in 

several respects from a simple price discount. These differences turned out to be important in 

understanding the main food spending results and food intake results. As suggested in the conceptual 

model above, HIP may influence fruit and vegetable intake through important mechanisms other than 

changing the household’s food resources. The informational or marketing mechanism would suggest 

a larger impact. Non-universal participation of retailers and participants having an incomplete 

awareness of the program would suggest a smaller impact. 

HIP also may affect fruit and vegetable intake by increasing the household’s resources available for 

food spending. Previous research suggests that this “income effect” on fruit and vegetable spending is 

fairly small. Frazão et al. (2007) provided two types of evidence for this view. First, they cited 

Consumer Expenditure Survey evidence that fruit and vegetable spending remains fairly constant 

 

6

 Based on price elasticity for fruits of 0.70 (90 percent confidence interval: 0.41-0.98) and for vegetables of 

0.58 (90 percent confidence interval: 0.44-0.71); using an elasticity of 0.64 (half way between the 

respective mean elasticities for fruits and vegetables). 

7

 Calculation: 2.6 cups x 19 percent increase = 0.49 cups. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 1. Introduction ▌pg. 19

across income categories. Second, they used older econometric estimates of the effect of increases in 

SNAP benefits. These estimates indicated that food spending increases by between 17 and 47 cents 

for each dollar of increased SNAP benefits (Fraker, 1990). Given that fruit and vegetable spending 

was approximately 12 percent of all food spending, these estimates suggested that fruit and vegetable 

spending could be expected to increase by between 2 and 5.6 cents for each dollar of HIP incentive 

earned. Thus, the price effect was expected to be more important than the effect of increasing 

household resources.8

Research on Price Elasticities and Income Effects 

Some research has investigated the effects of prices on fruit and vegetable spending. There is clear 

evidence that price matters in other areas of health promotion (e.g., on smoking see Jha et al., 2006; 

more broadly, see Horgen and Brownell, 2002). With respect to food purchases, Andreyeva, Long, 

and Brownell (2010) reviewed evidence on the impact of food prices on broad food categories. Their 

estimates are in terms of elasticities. For fruits they estimate a price elasticity of 0.70 (90 percent 

confidence interval: 0.41-0.98), which indicates that, all else equal, a 10 percent decrease in the price 

of fruit leads to approximately a 7 percent increase in the amount purchased. For vegetables, the price 

elasticity was 0.58 (90 percent confidence interval: 0.44-0.71). These results are broadly consistent 

with other recent work. 

USDA research suggests that increasing total resources or resources targeted to food in general may 

have only a modest effect on fruit and vegetable spending for low-income Americans. Stewart and 

Blisard (2008) estimated income elasticities showing the percentage change in spending for several 

food products in response to a change in income. For example, for middle-income households a 10 

percent change in income was associated with a 1.3 percent change in beef spending, a 1.3 percent 

change in fruit spending, and a 1.0 percent change in vegetable spending. For low-income 

households, a 10 percent change in income was associated with a 2.5 percent change in beef 

spending, but no statistically significant change in fruit or vegetable spending. The study did not 

directly measure the effect of increasing SNAP benefits, but the authors anticipated a similar pattern. 

Stewart and Blisard concluded that there may be a hierarchy of demand, in which fruits and 

vegetables start out as a comparatively low priority for low-income Americans but become a higher 

priority as the budget constraint is relaxed at somewhat higher income levels. Based on this evidence, 

the research suggests that price interventions could have larger effects than would increasing 

resources alone (Guthrie et al., 2007). 

Interventions to Change Food Prices 

In addition to broad research on price effects that arise for economic reasons, there is a smaller 

literature on programmatic interventions providing a price incentive with the purpose of affecting 

food choices. This research generally showed that consumption responds to price changes, but these 

studies did not allow estimation of elasticities. Earlier studies included research on WIC fruit and 

vegetable vouchers and some interventions in farmers market and worksite settings. As part of a 6-

month program offering fruit and vegetable vouchers valued at $10 per week, Herman et al. (2008) 

 

8

 This conclusion applies to participants who do not reach the incentive cap. Economic theory suggests that 

the price effect on marginal additional fruit and vegetable purchases would be zero for participants who 

reach the incentive cap, but that possibility is purely hypothetical in this pilot. As noted in Chapter 5, fewer 

than 10 participant households reached the incentive cap. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 20 ▌1. Introduction Abt Associates

randomly assigned WIC mothers to one of two intervention groups (vouchers for grocery stores or 

vouchers for farmers markets) or to a control group. Fruit and vegetable consumption increased 

substantially and significantly in both voucher groups but not in the control group, with changes 

sustained 6 months after the voucher program ceased. Average increases in fruits and vegetable 

consumption were 2.4 servings per person per day for the farmers market voucher group and 0.9 

servings for the supermarket voucher group, where one serving is equivalent to approximately 1⁄2 cupequivalent of fruit or vegetables. A post hoc analysis of correlates of fruit and vegetable intake 

showed that changes in consumption differed across ethnic groups and for fruits versus vegetables. 

Some previous interventions combined pricing and promotion strategies and hence are most 

comparable to HIP. An evaluation of a farmers market nutrition program in Michigan investigated the 

effects among 564 low-income women of a multi-component program, including nutrition education 

combined with farmers market coupons (worth $20) on fruit and vegetable consumption (Anderson et 

al., 2001). The research design included random assignment to two treatments (education only or 

farmers market coupons plus nutrition education); the coupon only group and the comparison group 

were assigned based on clinic appointment time. Among the 455 participants who completed a 

posttest, the coupon-only group showed a self-reported statistically significant increase in fruit and 

vegetable consumption, and the impact was larger for nutrition education and coupons combined. 

French and colleagues have published several reviews (French, 2003, 2005; French, Story, and 

Jeffery, 2001; French and Wechsler, 2004) and conducted studies of interventions (Jeffery et al., 

1994; French et al., 1997) to increase fruit and vegetable consumption. They have identified lower 

pricing as a successful strategy, singly or in combination with education and cafeteria improvements 

in school and worksite settings. In studies French and colleagues reviewed, significant increases in 

fruit consumption (0.2 to 0.6 servings per day) have been observed as a result of reduced prices for 

fruits and vegetables at schools and in worksite cafeterias, while reduced prices have been less 

successful in changing vegetable consumption. 

The extent to which these findings are applicable to the HIP demonstration project will depend on 

several factors: the similarity of the intervention (dollar equivalent amounts for vouchers, the ability 

to use them in the same types of outlets); differences between the populations (e.g., inclusion in WIC 

of higher-income households than are found in SNAP, as well as WIC categorical eligibility factors); 

and the sample demographics. The actual economic value of the vouchers in much of this research 

has been fairly small. 

Other Interventions to Promote Fruit and Vegetable Consumption 

Food choices also depend on more proximal variables, such as attitudes, preferences, behavioral 

intentions, and self-efficacy (Larson and Story, 2009). Analysis of the National Cancer Institute’s 

Food Attitudes and Behaviors (FAB) survey found that respondents who were aware of the U.S. fruit 

and vegetable promotion campaign (“5 a day”) were more likely to consume the recommended 

number of servings (Erinosho et al., 2012). Previous research has also found some positive effects on 

fruit and vegetable intake from interventions that encouraged healthy behavioral choices in worksites 

and health centers (Sorensen et al., 2007). HIP is designed primarily as a financial incentive, but the 

pilot necessarily included significant outreach to the randomly selected HIP participant group. 

Outreach materials provided information about participating retailers and described the qualifying 

targeted fruits and vegetables (see Chapter 3). Thus the overall impact of HIP includes any effects of 

HIP on attitudes toward or preferences for fruits and vegetables as well as the main effects of the 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 1. Introduction ▌pg. 21

financial incentive. The HIP evaluation collected multiple measures of participants’ attitudes toward 

and exposure to fruits and vegetables. 

Much previous research has been done on non-price interventions and policies to increase fruit and 

vegetable intake (Government Accountability Office, 2008). Some interventions discussed in this 

review focused on nutrition education or awareness campaigns designed to influence attitudes toward 

fruits and vegetables. Other research explored the effect of prices on fruit and vegetable spending, and 

a few previous interventions included a financial component. The HIP evaluation is distinct from all 

of this previous research, because it provides the first random assignment impact estimates for a 

financial incentive that is administered directly through SNAP using the EBT card. The previous 

research does, however, provide some context for understanding the new results in this Final Report. 

An international literature review of 44 studies found that results differed based on characteristics of 

both the intervention and the target population (Pomerleau et al., 2005). Only two of these studies 

included a monetary incentive or coupon, but they give some indication of the effect sizes that have 

been found with other approaches. For example, in general population interventions, positive effects 

ranged from 0.2 to 0.6 servings per person per day (one serving is equivalent to approximately 1⁄2 cupequivalent of fruit or vegetables). In interventions focused on smaller communities, such as African 

American churches, the positive effects were larger, ranging from 0.7 to 1.4 servings per day. 

Similarly, interventions for low-income adults tended to have larger effects, ranging from 0.42 to 1.1 

servings per day. A review of 22 studies of interventions to influence fruit and vegetable intake 

revealed positive effects in 17 studies but no effect in another 5 studies (Government Accountability 

Office, 2008). Interventions with greater intensity, such as face-to-face education and counseling, are 

more effective but have substantially higher costs. 

Improving the Availability of Fruits and Vegetables in the Food Retail Environment 

A substantial amount of research has investigated the lack of access to healthy foods in low-income 

neighborhoods (USDA, 2009). Neighborhoods that are predominantly lower income and minority 

may have fewer supermarkets or longer distances to travel to supermarkets (Chung and Myers, 1999; 

Morland et al., 2002). However, it is unclear whether these differences hinder most SNAP 

participants in acquiring fruits and vegetables. Most low-income people in the United States shop at 

supermarkets and are reasonably satisfied with their level of food retail access (USDA, 2009). In a 

sample of SNAP participants, Rose and Richards (2004) studied the association between fruit and 

vegetable consumption and distance from the retailer where a respondent purchased most food. 

Greater distance was associated with significantly lower fruit consumption (and with lower vegetable 

consumption in the sample, but the vegetable results were not statistically significant). 

HIP effects could be influenced by the surrounding food retail environment. The converse is also 

possible. There has been recent interest in whether changes to major nutrition assistance programs 

could encourage food retailers serving low-income areas to stock more fruits and vegetables. With 

data from Connecticut, before and after the implementation of new WIC packages that included a 

voucher for fruit and vegetable purchase, Andreyeva, Long, and Brownell (2012) found increases in a 

basket of healthy foods (heavily weighted toward fruits and vegetables) supplied by food retailers in 

low-income neighborhoods. 

The HIP evaluation surveyed retailers that participated in HIP and collected information on the 

availability of fruits and vegetables, both prior to HIP implementation and during the pilot. The 

evaluation also collected survey data with the respondent’s own assessment of access problems in 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 22 ▌1. Introduction Abt Associates

purchasing fruits and vegetables and administrative data identifying the retailers where SNAP 

benefits were spent. We use these data to examine changes in the food retail environment, 

households’ perceptions of issues purchasing fruits and vegetables, and the types of stores where HIP 

and non-HIP households shopped. 

1.6 Organization of the Report 

The remainder of this Final Report is organized as follows: 

 Evaluation design, data, and methods (Chapter 2) 

 HIP implementation and operations (Chapter 3) 

 Retailer experiences (Chapter 4) 

 HIP participant experiences (Chapter 5) 

 Effects on expenditures and shopping behaviors (Chapter 6) 

 Impacts on attitudes toward fruits and vegetables and family food environment (Chapter 7) 

 Impacts on consumption (Chapter 8) 

 Costs of the pilot and feasibility of nationwide expansion (Chapter 9) 

 Conclusions (Chapter 10) 

Chapters 4 through 7 focus on the leading factors that could influence the magnitude of HIP 

impacts—the food retail environment, participant experiences with the pilot, fruit and vegetable 

expenditures, attitudes, and the home food environment—as discussed above and presented in the 

conceptual model (Exhibit 1.1). The main fruit and vegetable consumption impacts themselves are 

then described in Chapter 8. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 23

2. Evaluation Design, Data, and Methods 

A rigorous research design was critical to answering the HIP evaluation’s primary research 

objective—assessing the impact of HIP on participants’ intake of targeted fruit and vegetables. 

Random assignment, the “gold standard” for assessing causal impacts, was used to select households 

to participate in HIP. The random assignment design assured that any measured differences between 

the HIP and non-HIP groups can be attributed to the program (or chance) rather than to factors like 

seasonal patterns or household preferences. To measure dietary intake we used 24-hour dietary recall 

interviews, the approach used in most major national nutrition studies. Both of these approaches—

random assignment and 24-hour dietary recall interviews—are complicated and resource-intensive, 

but they give the best possible assurance that the evaluation results will be meaningful and unbiased. 

These methods serve to address the numerous challenges to measure accurately the impact of HIP. 

The amount of fruit and vegetables a household consumes depends on a wide array of factors, ranging 

from seasonal patterns of food availability to cultural preferences to the household’s daily work and 

school schedule. The research design must therefore find a way to isolate the effects of HIP from 

these other factors. Adding to the challenge is the difficulty of knowing exactly what the household 

eats, since household members may eat in a variety of locations throughout the day and do not eat the 

same foods every day. The evaluation used the most reliable known strategies to deal with these 

challenges. 

Exhibit 2.1 provides an overview of the sampling design, showing the sampling process as it occurred 

at the different stages of the study. Participant data collection activities associated with each sample 

are noted to the left of the figure. As described in Section 2.2, dietary intake data, using 24-hour 

dietary recall interviews, were collected at two points during HIP implementation. Several other types 

of data collection were additionally conducted as part of the evaluation. EBT transaction data 

provided detailed information on households’ SNAP EBT purchases. Retailer surveys and interviews 

with key stakeholders provided information to document the process of implementing and operating 

HIP, to examine the effect of the pilot on stakeholders, and to estimate the costs of the pilot and of 

nationwide expansion of HIP. 

This chapter describes the evaluation design and the different types of data collected. The first section 

describes the random assignment process in which households were sampled to participate in HIP and 

given the opportunity to earn incentives for the purchase of targeted fruits and vegetables. The second 

section discusses random sampling for participant survey data collection, including sample sizes and 

response rates. It also discusses the types of data collected from participants that are used for the 

descriptive and impact analysis included in this report. The third section discusses the EBT 

transaction data that provided information on households’ SNAP spending. The retailer survey and 

stakeholder interviews are detailed in sections four and five, respectively. The sixth section discusses 

the participant focus groups that were conducted to provide qualitative detail to assist in interpreting 

the findings. Administrative data from SNAP case files are described in the seventh section and the 

eighth section discusses data used in the analysis of pilot costs and costs of expansion. Sections nine 

and ten provide an overview of the analytic methodology used in the report and a description of the 
Evaluation of the Healthy Incentives 

pg. 24 ▌2. Evaluation Design, Data, and Methods Exhibit 2.1: Overview of Sampling Design and Data Collection Activities aExcludes child-only cases; households that signed over benefits to treatment facility. bSurvey response rates: 63 percent-84 percent; see Exhibit 2.3 for detailed calculations. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 25

baseline characteristics of HIP and non-HIP households and of the survey sample, respectively. The 

final section examines attrition from SNAP over the evaluation period.9

2.1 Random Assignment Research Design 

The random assignment of eligible SNAP participant households to HIP and non-HIP status was 

central to the evaluation design and HIP operations. The Massachusetts Department of Transitional 

Assistance (DTA) provided administrative case file records containing all households and persons on 

SNAP in Hampden County as of mid-July, 2011.10 Abt randomly selected 7,500 SNAP households 

(containing 9,286 adults) to participate in HIP. DTA notified these households, provided specially 

marked EBT card sleeves to identify them as HIP participants, and offered training on HIP 

procedures. The remaining 47,595 eligible SNAP households in Hampden (containing 59,652 adults) 

were not selected to earn the HIP incentive. 

To ensure that the HIP (treatment) and non-HIP (control) groups were balanced (similar to each 

other), we used a blocked random assignment design.11 Tests on the samples after random assignment 

confirmed that they were similar with respect to key participant characteristics. Additionally, HIP 

households were randomly divided into three groups, corresponding to the three waves DTA 

established to enroll participants in HIP.12 Non-HIP households were also divided into three waves to 

facilitate participant survey sampling. The final sizes by wave for the HIP and non-HIP groups are 

presented in Exhibit 2.2. 

Exhibit 2.2: Households and Persons in the HIP and non-HIP Sampling Frames by Wave 

HIP group Non-HIP group 

Wave 1 Wave 2 Wave 3 Total Wave 1 Wave 2 Wave 3 Total 

Number of 

households 2,500 2,500 2,500 7,500 15,865 15,865 15,865 47,595 

Number of adults 3,091 3,090 3,105 9,286 19,952 19,792 19,908 59,652 

 

9

 Data collection instruments can be found in a separate volume on the FNS website, along with other project 

reports. 10 The evaluation sought to estimate the impact of HIP on adult food intake, so child-only cases were not 

eligible for HIP. Furthermore, only households that did their own shopping were eligible for the evaluation. 

SNAP participants who signed over their benefits to a residential or treatment facility were not eligible. 

DTA excluded these households prior to providing the file. Homeless participants who retained the use of 

their own benefits remained eligible for the study. 

11 The evaluation created 12 household-level blocking cells defined by three levels of geography (Springfield; 

Chicopee/Holyoke; and remainder of Hampden County), two levels of household size (one-person and twoor-more-persons), and two genders for head of household (male-headed and female-headed). See Appendix 

B for additional details about sampling. 

12 DTA staggered enrollment in HIP over three months for ease of implementation. The first wave began 

receiving the HIP incentive on November 1, 2011, the second wave on December 1, 2011, and the third 

wave on January 1, 2012. HIP participants were eligible to earn incentives for 12 months, ending in 

December 2012. Prior to each wave’s start date, HIP households (that were active SNAP participants in the 

start month) received several mailings describing HIP. See Chapter 3 for additional details. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 26 ▌2. Evaluation Design, Data, and Methods Abt Associates

Once DTA generated the Hampden County SNAP case extract file of HIP-eligible households, these 

households remained in the HIP evaluation sample for the duration of the pilot as long as they 

continued to participate in SNAP; no additional households were added to the pilot.13 

2.2 Participant Survey 

The evaluation collected three rounds of survey data on sampled participants: 

 Round 1: baseline or pre-implementation data were collected prior to HIP implementation 

and prior to notification of HIP/non-HIP status. Data collection occurred between August and 

December 2011. 

 Round 2: early implementation data were collected when households had the opportunity to 

earn HIP incentives for 4-6 months. Data collection occurred between March and July 2012. 

 Round 3: late implementation data collection occurred when households had the opportunity 

to earn HIP incentives for 9-11 months. The data collection period occurred between August 

and November 2012. 

In this section, we discuss participant sampling and data collection activities for all three rounds of 

participant survey data collection. 

Participant Survey Sampling 

The survey samples, equally distributed between the HIP and non-HIP groups, were selected using a 

stratified random sampling procedure. The stratification variables were the same ones used in the 

random assignment of households to the HIP and non-HIP groups (geographic region of residence, 

household size, gender of household head). Among the HIP and non-HIP groups, an equal number of 

respondents were selected from each of the three waves. Sampled respondents were aged 16 and older 

and only one respondent was selected per household.14

After completion of participant sampling, each household record in the database received a HIP flag 

or indicator identifying it as one of the following four groups: 

 HIP household, non-survey group 

 HIP household, survey group 

 Non-HIP household, survey group 

 Non-HIP household, non-survey group 

These flags were applied to all members of the original households in the HIP evaluation sample. 

Changes in household composition and program participation over time had implications both for 

 

13 Households exited SNAP over the course of the pilot. By the beginning of the pilot, 9.5 percent were no 

longer receiving benefits. One year later, 21.5 percent of the original sample was not receiving SNAP 

benefits. 

14 Approximately 6 percent of sampled respondents were aged 16-17 at the Round 2 survey. This group was 

included in the sample as they can be SNAP heads of households and the sample was intended to represent 

all types of households. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 27

HIP operations and for survey sampling. The SNAP case was tied to the head of household and 

therefore, the HIP flag and the HIP incentives were also tied to this individual.15 

Exhibit 2.3 shows the number of participants sampled and the number who remained on SNAP at the 

beginning of the survey round and were otherwise eligible to participate in the survey. 

We randomly sampled 2,538 SNAP participants each from the HIP and non-HIP households to 

participate in the Round 1 survey. This sample size was chosen so that a large enough sample would 

remain for Rounds 2 and 3 to achieve the desired level of precision after accounting for participants 

who left SNAP, and thus were ineligible for the survey, or who became non-respondents in the later 

rounds.16 The target sample was designed to be able to detect a post-implementation HIP/non-HIP 

difference in targeted fruit and vegetable intake of 0.25 cup-equivalent of fruits and vegetables per 

day. This amount is somewhat less than the likely difference that we estimated based on economic 

theory and empirical results, so the target sample sizes were expected to be adequate. 

The Round 1 survey was fielded in three waves, corresponding to the three waves DTA established to 

implement HIP (November 1, 2011, December 1, 2011, and January 1, 2012, respectively). Prior to 

fielding each round, we used DTA’s SNAP case file data to exclude respondents who were no longer 

participating in SNAP (see Section 2.11 for a discussion of this decision). 

Exhibit 2.3: Sample Sizes, Eligible Respondents, Completed Surveys, and Response Rates: 

HIP and non-HIP Households by Survey Round 

Number sampled 

Number eligible 

for survey 

Number of 

surveys 

completed 

Response rate 

(%) 

HIP group 

Round 1 2,538 2,210 1,388 63 

Round 2 1,388 1,198 1,004 84 

Round 3 1,004 937 769 82 

Non-HIP group 

Round 1 2,538 2,193 1,396 64 

Round 2 1,396 1,207 994 82 

Round 3 994 931 751 80 

All respondents who completed the Round 1 survey were eligible for the Round 2 survey. Similar to 

procedures for the baseline survey, the Round 2 survey was fielded in three waves. While each wave 

had been eligible to earn incentives for 4-6 months at the time of the survey, given some EBT systems 

issues that occurred near the start of HIP implementation, most households had about three months of 

 

15 If the original head of household (HoH) left the SNAP household, DTA closed that SNAP case. Other 

household members could form a new case, but that new case did not get the HIP flag and thus did not earn 

HIP incentives even if its prior flag was a HIP case. Similarly, if a member of a HIP household other than 

the original HoH left the household, that person was not given a HIP flag and was not eligible to earn HIP 

incentives. In such cases, the household with the original HoH retained the HIP flag and HIP incentives. 

The SNAP case also could close without any changes in household composition. Regardless of how a 

SNAP case closed, if the SNAP case reopened with the original HoH, the household once again received 

the HIP flag and resumed earning HIP incentives. 

16 See Appendix B for details on sampling procedures and assumptions. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 28 ▌2. Evaluation Design, Data, and Methods Abt Associates

involvement with a fully operational pilot and thus their experiences reflected the early HIP 

implementation period.17 As in the case of the baseline survey, prior to the start of Round 2, 

respondents who were no longer receiving SNAP benefits were dropped from the sampling frame.

Similar procedures were followed for the Round 3 survey that occurred 9-11 months after HIP 

implementation. 

Because a single 24-hour dietary recall measures consumption at one point in time, which may not 

accurately represent longer-term average intake, the preferred method for estimating “usual intake” is 

to conduct a second interview for a subset of the study participants and then to use statistical models 

to adjust the estimates (see discussion in Appendix E). To estimate usual intake distributions for fruit 

and vegetable consumption, in both Rounds 2 and 3 we drew a 10 percent subsample of respondents 

who completed the survey and conducted a second dietary intake interview. 

Participant Survey Instruments 

The participant survey included two modules, one to be completed by the sampled participant and the 

other by the household’s primary shopper. In most cases (79 percent of the final analytic sample) the 

sampled participant and the primary shopper were the same person. Survey topics, by module, are 

presented in Exhibit 2.4. Where possible, we used validated questions from other surveys. 

Exhibit 2.4: Participant Survey Topics, by Round 

Survey topics 

Round 1 survey 

(baseline) 

Round 2 survey 

(4-6 months after 

implementation) 

Round 3 survey 

(9-11 months after 

implementation) 

Sampled participant module

Respondent 

characteristics 

 

Attitudes, perceptions, 

and barriers to 

consuming fruits and 

vegetables 

  

Fruit and vegetable 

consumption screener 

(frequency and quantity) 

  

Exposure to nutrition 

education 

 

24-hour dietary recall  

Primary shopper module

Household 

characteristics 

  

Participation in nutrition 

assistance programs 

 

Family food environment   

General shopping 

patterns 

  

Food expenditures   

Experiences participating 

in HIP 

 

 

17 See Chapter 3, Section 3.2 for details of the EBT systems issues. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 29

The Round 1 survey established a baseline for all measures except the outcome measures based on 

the 24-hour dietary recall, which was not included in Round 1 due to its burden and cost. The random 

assignment design uses differences between the HIP and non-HIP groups to measure the impact of the 

program and thus does not require baseline outcome measures. 

The Round 2 and Round 3 instruments collected data on most of the same measures as Round 1, 

except that neither collected information on respondent characteristics nor on participation in 

nutrition assistance programs. Interview time was limited and only baseline measures were used in 

the models.18 

Both the Round 2 and Round 3 surveys included key outcomes for the study. The 24-hour dietary 

recall was conducted using USDA’s Automated Multiple Pass Method (AMPM). The AMPM 

instrument, which is designed to enhance respondents’ ability to recall food consumed during the 

previous day, is a well-established and frequently used measurement approach. The five steps in the 

process collect different kinds of information about foods consumed, including detailed description, 

time and occasion, and amount. Use of two-dimensional drawings in a food model booklet help 

respondents estimate the amounts of foods consumed. 

Despite this, some measurement error undoubtedly remains and one can never completely rule out the 

possibility that estimated impacts are influenced by measurement error. The AMPM approach yields 

less underreporting of food and nutrient intake than other assessment methods (Subar et al., 2003), 

minimizes respondent literacy and memory issues, and minimizes respondent burden. 

To provide a consistent measure of fruit and vegetable consumption at baseline and allow for a 

comparison with the Round 2 and Round 3 surveys, respondents in all three survey rounds completed 

a modified version of the Eating at America's Table Study (EATS) Fruit and Vegetable Screener 

(Thompson et al., 2000). This screener asked about usual intake by the respondent over the previous 

month of nine common foods containing raw and cooked fruits and vegetables (including those eaten 

as snacks and at meals, eaten at home and away, and eaten alone and mixed with other foods). 

Participant Survey Data Collection 

Before the beginning of Round 1 data collection, sampled respondents were sent an advance letter 

informing them of their selection for the evaluation sample. The letters sent to participants before the 

Round 1 survey did not mention HIP, but instead referred to the study as examining ‘how SNAP is 

working for families in Hampden County.’ Round 1 data collection was completed before DTA 

notified HIP households about the pilot. Advance letters were also sent prior to the start of Round 2 

and Round 3 data collection. Within each round of the survey, the sample was released in three 

waves, corresponding to the three waves of implementation. 

Participant survey data were collected through a telephone interview. Spanish-speaking interviewers 

were available to complete surveys with respondents whose primary language was Spanish. The 

relatively few respondents whose primary language was neither English nor Spanish had family 

members or friends provide survey translations. For respondents who could not be reached by phone, 

 

18 In addition, some respondent characteristics and participation in other nutrition assistance programs could 

themselves be influenced by HIP; thus these variables cannot be included in models predicting HIP 

outcomes. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 30 ▌2. Evaluation Design, Data, and Methods Abt Associates

we sent field locaters to respondents’ homes.19 Once they located respondents and gained their 

cooperation for the interview, field locaters provided a cell phone that respondents used to complete 

the interview. 

Respondents were offered incentives for completing the surveys: $20 at Round 1, $30 at Round 2, 

and $40 at Round 3. In addition, respondents who used their own cell phones to call into the 

telephone center received an additional $10 to compensate them for minutes used. The intensive field 

methods were designed to address the concern that non-respondents might differ from respondents, 

such that estimates for respondents would not project to the entire Hampden County SNAP 

population.20

Response rates are presented in Exhibit 2.3. As is standard in panel surveys, response rates were 

lowest in Round 1; they were substantially higher in Rounds 2 and 3. Several natural disasters 

contributed to the lower than planned response rates in Round 1, including a hurricane, tornado, and 

early season snow storm. Higher than expected incidences of missing or bad phone numbers, 

Spanish-speaking-only cases, and cases that were completed in the field created additional challenges 

to the first round of data collection. 

As discussed above, in order to estimate usual intake distributions for fruit and vegetable 

consumption, we drew a 10 percent subsample of respondents who completed the surveys and 

conducted a second dietary intake interview. In Round 2, we completed a total of 230 second day 

interviews, 107 with respondents in the HIP (treatment group) and 123 with non-HIP (control group) 

respondents. In Round 3, completed second day interviews totaled 206 (94 treatment group 

respondents and 112 control group respondents). 

2.3 EBT Transaction Data 

This report uses EBT transaction data for two types of analysis. The first analysis examines HIP incentive 

earnings by pilot participants, focusing on HIP-eligible purchases, the amount of incentives earned, and 

the percent of SNAP benefits spent on HIP-eligible purchases. 

The second analysis examines the impact of HIP on SNAP spending patterns, including data for both HIP 

and non-HIP households. The analysis examines SNAP purchases overall and purchases in different types 

of stores (e.g., supermarkets, convenience stores). Not all Hampden County retailers participated in HIP 

and we therefore examine total SNAP purchases in participating and non-participating stores. Finally, we 

examine purchases by both HIP and non-HIP households of targeted fruits and vegetables in supermarkets 

and superstores equipped with electronic cash registers that participated in HIP. (No data are available on 

TFV purchases in stores that did not participate in HIP.) 

Xerox, the EBT processor for Massachusetts, collects and maintains data pertaining to SNAP EBT 

transactions. These data show the amount of SNAP benefits received and the date they were credited, and 

they show the date, time, amount, and location for each shopping transaction using SNAP benefits. In 

 

19 The percentage of interviews completed in the field varied by survey round. Field completes were: 58 

percent in Round 1, 37 percent in Round 2, and 26 percent in Round 3. 

20 We conducted a non-response analysis to assess the extent of non-response bias in the Round 1 survey. 

Results are included in Appendix C. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 31

addition, transaction data for the evaluation period provide information (date, time, amount, store) on total 

HIP-eligible purchases and HIP incentives earned. 

We obtained daily EBT transaction data for the full pool of SNAP participants in Hampden County, 

including both HIP and non-HIP households. Daily transmission of data helped identify issues in the EBT 

transaction data in time for the EBT vendor to address them. We cleaned the transaction files to remove 

duplicate transactions and account for any product returns and corrected transactions. Data were 

transmitted without social security numbers, but case file identification numbers were included to allow 

linking of EBT transaction data to administrative case file data and survey data. We compiled the EBT 

transaction data into one observation per household for each calendar month. 

EBT transaction data have a number of restrictions that affect the analyses that can be performed: 

 Only purchases made with SNAP benefits are included; purchases made with other forms of 

payment, such as cash or WIC vouchers are not captured. 

 Only HIP purchases at stores that participated in HIP earned the incentive; purchases of HIPeligible items at non-participating retailers did not earn the incentive and are thus not captured. 

 At HIP participating stores without integrated electronic cash registers (IECRs), HIP households 

needed to identify themselves as HIP participants and HIP-eligible items needed to be separated 

from other items. HIP incentives were only earned for properly identified purchases. 

In this report, we use transaction data beginning in November 2011, when HIP began.21 For analyses of 

key outcomes we focus on transactions beginning in January 2012 when all participants were active. We 

analyzed EBT data through the end of the pilot, focusing on expenditures in the two periods roughly 

coinciding with the participant surveys (March-July 2012 and August-October 2012). 

2.4 Retailer Survey and Observations 

Surveys and on-site observations of food retailers offered insight into the experiences and satisfaction 

of an important HIP stakeholder group, and they provided useful information about the pilot’s 

implementation process and costs. Exhibit 2.5 shows the different types of retailer data collected over 

the course of the pilot. The rest of the section discusses each of these activities.22 

 

21 The evaluation research design intended that EBT transaction data would be available beginning two 

months prior to the HIP implementation. However, it proved difficult to compile EBT data files during the 

period that preparations for implementation were being completed; transaction files for the preimplementation period could not therefore be reconstructed. 

22 Our evaluation plans originally called for a survey of retailers who withdrew from HIP after initially 

participating. While some stores closed or changed ownership over the evaluation period, there was only 

one small store that carried limited HIP-eligible items that decided to withdraw, so we did not pursue this 

data collection activity. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 32 ▌2. Evaluation Design, Data, and Methods Abt Associates

Exhibit 2.5: Retailer Data Collection Activities 

Type of data collection 

Early HIP implementation period 

(Oct-Dec 2011) 

Late HIP implementation period 

(Nov 2012-Jan 2013) 

Retailer survey 

HIP participating retailers  

Declined to participate in 

HIPa 

Later implementing 

retailers 



Observations in 

participating storesb  

a

 These stores received a $40 incentive to encourage completion of the survey. b

 Abt also conducted observations in 7 participating stores and 3 participating farmers markets in July-August 

2012. 

Participating Retailer Surveys 

We conducted surveys of retailers who were participating in HIP at two periods during the pilot. The first 

survey, conducted between October and December 2011, included retailers who were participating at the 

start of the pilot in November 2011. The second survey, conducted November 2012-January 2013, 

included all retailers who were participating near the end of the pilot (as of October 2012). 

The first round of the retailer survey collected information on HIP implementation activities, including: 

 How retailers learned about HIP, their understanding of HIP’s objectives, and why they chose to 

participate 

 Activities undertaken to prepare for HIP implementation 

 Training store personnel 

The second retailer survey focused on retailers’ experiences with HIP, including: 

 How HIP affected store activities, including operational problems, perceived effect on check-out 

time, and perceived effect on sales and profit 

 HIP training for new employees and refresher training 

 Opinions about HIP, including satisfaction 

In addition, both rounds of the survey collected information on fruit and vegetable promotion activities. 

The surveys also collected data on a list of specific fruits and vegetables, capturing whether they were 

available in the store and the prices of those items sold in the store. 

For sampling, the goal was to have the retailer sample reflect all participating retailers, so we wanted the 

sample to have the same proportion of retailers, by type, as among participating retailers. To achieve this 

goal, we used a stratified random sample design. Participating retailers were first grouped by store type, 

combining the 12 official FNS store types eligible for HIP into four categories (superstores, supermarkets, 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 33

grocery and food specialty stores, and convenience stores).23 The supermarkets, superstores, and 

convenience store categories included chain stores and all were included in the sample. We then selected a 

sample of stores within chains, up to a maximum of six stores per chain. For the independent stores, our 

intent was to randomly select retailers for the survey within retailer type. However, given the number of 

participating independent retailers, little sampling was required; almost all participating stores were 

selected for the first round of the survey and all stores were selected for the second round. Exhibit 2.6 

presents information on the sample sizes and response rates for the different retailer survey activities. 

Surveys of chain stores and independent retailers were conducted slightly differently. Chain store surveys 

were conducted in two parts. The first part of the chain store survey was conducted with a corporate 

representative who responded to questions concerning activities for which headquarters was responsible 

(thus reporting for multiple stores in the sample). The second part of the chain store survey was conducted 

with managers of the individual chain retail stores. Independent retailers completed one survey. Many 

questions were similar across the different types of surveys, but some questions were only appropriate for 

either independent stores or chain stores. 

Exhibit 2.6: Retailer Survey: Samples and Response Rates 

Retailer survey Number sampled 

Number eligible 

for survey 

Number of 

surveys 

completed 

Response rate 

(%) 

Participating 

retailers—early 

implementation 

75 52a

 39 75 

Participating 

retailers—late 

implementationb

61 58c

 49 84 

Non-participating 

retailers 20 16 13 81 

a

 23 of the sampled independent stores were not eligible for the survey: 14 were not committed to HIP, 3 were 

closing/selling their stores, and 6 were planning to begin participating in HIP in February 2012. b

 Sample includes 12 later implementing retailers who joined HIP after January 2012; 11 completed the survey. c

 3 of the sampled independent stores were not eligible for the survey: 2 stores had closed and 1 store changed 

ownership and the new owner chose not to participate in HIP. 

The participating retailer surveys were conducted by mail with telephone and field follow-up. Our 

experience with the first retailer survey was that in-person follow-up was most effective. Thus in the 

second fielding of the survey, most follow-up occurred in person. 

Later Implementing Retailers 

The participating retailer survey that was conducted near the end of the pilot included a group of 

“later implementing” independent retailers, defined as retailers who were not participating in HIP at 

the beginning of the pilot. DTA continued retailer recruitment efforts after HIP began and permitted 

 

23 We collapsed the 12 official FNS stores types eligible for HIP into four categories as follows: 

supermarkets; superstores; grocery stores (small, medium, and large grocery stores and fruit/vegetables, 

seafood, and meat specialty stores); and convenience stores (convenience store and combination 

grocery/other). Farmers markets (including direct marketing farmers) were not operating during the periods 

the retailer survey was conducted and thus are not included in the survey samples. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 34 ▌2. Evaluation Design, Data, and Methods Abt Associates

retailers to join HIP at several points during the pilot (February 2012, June 2012,24 and October 

2012). 

The version of the participating retailer survey completed by the later implementing retailers included 

an additional module that collected information on their decision to join HIP. Topics included: how 

they learned about HIP; why they hadn’t joined HIP when the pilot began; and their reasons for 

eventually joining. 

The participating retailer sample included 12 later implementing retailers. Of these, 11 completed the 

survey. 

Non-Participating Retailers 

The focus of the non-participating retailer survey was to understand the factors affecting a store’s 

decision not to participate in HIP. In addition, the survey collected information on how retailers 

learned about HIP, their understanding of HIP’s objectives, and fruit and vegetable promotions in 

stores. 

We selected a small sample of 20 non-participating retailers to complete the survey. Similar to the 

participating sample, we stratified stores by four store types and randomly sampled within store type. 

Many non-participating stores were part of corporate chains and we randomly selected one store per 

chain. Four of the sampled stores were ineligible for the survey: two indicated interest in joining HIP; 

and DTA was still in discussion with two other stores about participating. 

Surveys with non-participating retailers were conducted on the telephone during November and 

December 2011. We were able to complete 13 interviews or 81 percent of those eligible for the 

survey (Exhibit 2.6). 

Store Observations 

Using an observation form developed for the evaluation, we conducted three rounds of observations 

in HIP participating stores to obtain first-hand information on their fruit and vegetable inventory, the 

HIP transaction process, and other environmental factors. Specifically, we used the form to conduct 

an inventory of fresh, frozen, dried, and canned fruits and vegetables. The inventory focused on the 

availability, variety, visual appeal, and price of a selection of fruits and vegetables available in the 

store. We also collected information on store signage promoting the consumption of fruits and 

vegetables. Finally, in stores without integrated electronic cash registers (referred to as EBT-only 

systems), we observed a simulated HIP transaction. 

Observations were conducted in a purposefully selected sample of 10 participating retailers of 

different store types that agreed to participate in the first retailer survey. Observations were conducted 

in October 2011, July-August 2012, and December 2012. To the extent possible, we conducted the 

first and third rounds of observations in the same stores.25 For the second round of observations, we 

 

24 Participating farmers markets joined HIP as they opened for the season in the late spring/early summer 

2012. All had closed by the time the time the second retailer survey was conducted (November 2012-

January 2013) and thus they were not included in the retailer survey sample. 

25 One store selected for the first round observations had closed by the third round. We replaced this store 

with another store of the same type. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 35

decided that it would be useful to conduct observations in farmers markets, and we selected three 

markets to visit. We also visited 7 of the original stores selected for observations. In all three rounds, 

we were able to complete all 10 observations. 

2.5 Stakeholder Interviews 

To understand the implementation and operations of the pilot we conducted three rounds of in-depth 

interviews with a wide variety of HIP stakeholders.26 The first round of interviews, which occurred 

between mid-October and mid-December 2011, focused on tasks involved in HIP implementation, 

including design and development of the EBT system modifications for HIP, retailer recruitment, and 

preparations for notifying and training participants. We conducted a second round of interviews 

during June and early July 2012. These interviews, which occurred during the middle of pilot 

operations, addressed issues that arose during the early months, including technical systems problems, 

continuing efforts to recruit retailers to participate in HIP, and attempts to understand the relatively 

low HIP take-up rate and efforts that might increase household incentive earnings. The final round of 

interviews took place in the last months of pilot operations. We conducted most interviews between 

late November and early December 2012; some interviews with EBT system stakeholders took place 

in January 2013, once the pilot had concluded. Topics for the final round of interviews included 

operational issues in the second half of the pilot and efforts to increase HIP households’ purchases of 

targeted fruits and vegetables. During the interviews, we also solicited information on the feasibility 

of expanding HIP, both statewide and nationwide and the factors and costs that should be considered 

in such efforts. Stakeholders did not receive any incentives to participate in the interviews. 

Information obtained from the stakeholder interviews was used to describe HIP implementation and 

operations (Chapter 3) and stakeholder perspectives on the feasibility of a potential nationwide 

expansion of HIP (Chapter 9).27 Stakeholders involved in EBT and retailer systems also provided 

information to help estimate the costs of expanding HIP (Chapter 9). 

We conducted over 30 stakeholder interviews in each round; generally interviewing the same 

individuals in all three rounds. Stakeholders interviewed included the following: 

 Massachusetts Executive Office of Health and Human Services (EOHHS) and DTA staff—

including DTA executives who oversaw the entire process and were directly involved in 

various aspects of implementation and local DTA office staff in Hampden County 

 Massachusetts DTA HIP staff—dedicated staff hired specifically for HIP to manage the pilot, 

recruit retailers, and train HIP participants 

 

26 Several other activities were also important in obtaining information about HIP implementation and 

operations. We attended two key technical meetings: the orientation meeting for DTA and Xerox 

(November 2010) and the application design sessions conducted by DTA and Xerox (December 2010). 

Team members participated regularly in status calls with FNS, DTA, and Xerox to keep abreast of project 

progress and issues and to make sure that evaluation plans were well synchronized with implementation 

activities. 

27 A more complete discussion of the process involved in implementing HIP, based to a large degree on the 

stakeholder interviews, can be found in the HIP Early Implementation Report (Bartlett, Beauregard et al., 

2013). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 36 ▌2. Evaluation Design, Data, and Methods Abt Associates

 EBT processor, Xerox State and Local Solutions, Inc.28—responsible for modifying the 

Massachusetts EBT system for HIP and working with retailers to modify their systems 

 Retailers and third-party processors (TPPs)—who had to modify their systems to accept and 

process HIP 

 Novo Dia Group—technical consultants to DTA responsible for coordinating system design 

and testing activities for retailers 

 Community-based organizations (CBOs) involved in HIP 

To ensure that the interviews were conducted systematically, we developed detailed interview guides. 

Interviewers prepared for each interview by reviewing applicable documents, such as the 

Massachusetts DTA grant application, project status and progress reports, and internal 

communications. Two-person teams conducted the interviews. The teams generally consisted of a 

senior researcher and an analyst who recorded the interview. Interviews were generally one to two 

hours long. After the interviews, teams conducted telephone follow-up as necessary to clarify 

responses. Interviews were coded into the categories defined by the interview guides. 

2.6 Participant Focus Groups 

The evaluation conducted two rounds of focus groups with HIP participants to obtain qualitative 

details on their experiences with HIP. The information from the groups was used to provide 

additional context for interpreting the participant impact analysis. We conducted the first round of 

focus groups in April 2012, coinciding with the Round 2 participant survey, and the second round of 

focus groups in October 2012, coinciding with Round 3 of the survey. 

Focus group participants shared their perspectives on several broad topics outlined below: 

 How they learned about HIP, including the notifications and training received 

 Expectations for the program 

 Experiences using HIP 

 Financial impact on the household 

 Impact on consumption of fruits and vegetables 

We conducted three focus group sessions in each round, two in English and one in Spanish. Each 

group included eight to ten participants. The sessions lasted 90 minutes and participants were invited 

to come 30 minutes early for light refreshments. All focus group members received $75 for their 

participation. 

Focus group participants were recruited using the case file data used for participant sampling. Eligible 

participants for the first round of focus groups were selected from the sample of survey participants; 

participants in the second round of focus groups were not from the survey sample. In addition, focus 

group participants had to be active SNAP participants and to have heard of the HIP program. The 

 

28 Xerox State & Local Solutions, Inc. operates as a subsidiary of Xerox Corporation. Throughout the report, 

we refer to the EBT processor as Xerox. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 37

groups included a roughly equal number of women and men and included participants of different 

ages and educational levels, with no more than two individuals per group having a college degree or 

higher. 

Focus group data were analyzed by using notes taken during the groups, reviewing audio files, and 

using verbatim transcription as needed. The analysis of the focus groups focused on several issues 

and data were coded into the following analytic categories: 

 Overall participant understanding of HIP purpose and operation 

 How participants learned about HIP 

 Expectations of families and the extent to which HIP met expectations 

 Changes in willingness to purchase fruits and vegetables 

 Unexpected outcomes 

2.7 SNAP Caseload Data 

DTA maintains information on SNAP households and participants (including demographics, income, 

contact information, and benefits) in its BEACON eligibility system. This system was used for the 

initial random assignment of eligible households to HIP and for several additional evaluation 

purposes, including: 

 Survey sampling frame, blocking groups, sorting variables, and demographic analysis 

variables for the participant survey 

 Updates to contact information and demographic data for participants sampled for Rounds 2 

and 3 of the participant survey 

 Characteristics of HIP and non-HIP households that were matched with monthly EBT 

transaction data for analysis 

Important demographic variables from the SNAP eligibility system included age and race/ethnicity of 

household head, household size, relationships of SNAP household members, number of adults in the 

household, number and ages of children in the household, presence of an elderly member, 

employment status and earnings, and presence of unearned income (including for example, 

supplemental security income (SSI) and unemployment compensation). 

The SNAP caseload data were obtained for all HIP-eligible households in Hampden County receiving 

SNAP benefits in July 2011 and monthly throughout the evaluation period. 

2.8 Cost Data 

The cost data collection captured the expenses of the stakeholders participating in the implementation 

and operation of HIP in Massachusetts. These data were used to estimate the cost of the pilot and also 

the costs of a potential nationwide expansion of HIP. The primary stakeholders whose costs were 

captured include: DTA; Xerox (Massachusetts EBT processor), Novo Dia Group (technical 

consultant), retailers, and third-party processors (TPPs). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 38 ▌2. Evaluation Design, Data, and Methods Abt Associates

All of the identified costs of implementing HIP were paid for through the HIP grant awarded by FNS 

to DTA. DTA provided quarterly expenditure reports that detailed all expenditures of HIP grant 

funds, including staff costs (salaries and wages, fringe benefits, and payroll taxes), payments to 

contractors and retailers, supplies and other direct costs, and indirect/overhead costs (as charged on 

the basis of personnel costs). 

Staff time records provided the other major source of data used in the cost analysis. All DTA staff 

working exclusively on HIP reported their time on a weekly basis, with a breakdown by seven key 

functions: (1) design, development, and testing; (2) household recruiting and customer service; (3) 

retailer recruiting and relations; (4) community relations; (5) training; (6) general administration; and 

(7) evaluation. The time records were used to allocate total DTA staff costs and indirect costs among 

the functions needed to implement HIP (and to exclude evaluation costs from implementation costs). 

Time for other DTA staff involved in HIP was provided as a single number of hours by week. 

Contractors reported their HIP staff time on a monthly basis, using the functional breakdown that was 

used in the DTA time records. The evaluation team used these time records to allocate the actual 

payments to contractors (from DTA expenditure reports) to the separate functions.29 

2.9 Analytic Approach 

In this section, we describe our analytic approach to estimating the impact of HIP—outcomes with 

HIP relative to outcomes without HIP.30 We begin by describing the primary outcome of interest—

intake of targeted fruits and vegetables (TFVs), the foods qualifying for the HIP incentive. We then 

describe the regression models used to estimate impacts on continuous outcomes (such as daily intake 

of targeted fruits and vegetables). We also discuss the analysis strategy for binary outcomes (such as 

having fruit available at home) and other categorical outcomes (such as the degree of agreement with 

a statement about attitudes toward fruits and vegetables). Finally, we discuss the use of survey 

weights to make sample estimates representative of the SNAP population in Hampden County. 

Primary Evaluation Outcome 

The primary focus of the impact analysis is the impact of HIP on TFVs. However, it is not possible to 

precisely measure TFV intake using standard 24-hour dietary recall interview methods and food 

codes. The issue is that standard dietary coding schemes identify the form in which a food was 

consumed, while whether a food qualifies for the HIP incentive depends on the form in which the 

food was purchased. Our approach to dealing with this issue and constructing the TFV measure is 

discussed in detail in Chapter 8. 

 

29 Additional detail on cost data sources is presented in Chapter 9. 

30 The approach estimates the impact of being assigned to the HIP group and therefore eligible to receive the 

HIP incentive on purchases of targeted fruits and vegetables relative to being assigned to the non-HIP 

group and not being eligible to receive the HIP incentive on purchases of targeted fruits and vegetables. For 

many purposes, this is the policy relevant estimate as we cannot force people to use the benefit; all we can 

do is offer them the benefit. Thus, the analysis estimates the impact of being offered the HIP incentive. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 39

The research questions require analyses of impacts on a large number of outcomes in addition to 

TFV. When we estimate a large number of impacts, some of them are likely to appear to be 

significant even if there is no true treatment effect (Schochet, 2009). Our approach to this “issue of 

multiple comparisons” was to specify, prior to analyzing the data, one confirmatory outcome—

targeted fruit and vegetable (TFV) intake, pooled31 across all interviews from follow-up survey 

Round 2 and Round 3—so that no further multiple comparison adjustment would be needed. We 

consider all other analyses exploratory. 

Regression-Adjusted Differences Between HIP and non-HIP Groups 

The impact analyses presented in this report are based on regression-adjusted differences between the 

HIP and non-HIP groups. While a simple comparison of mean outcomes for respondents in the HIP 

and non-HIP groups would yield unbiased estimates of the impact of HIP, we use a regressionadjustment approach to improve the precision of impact estimates by controlling for some portion of 

the variation in observed outcomes. In other words, we compare the average outcomes (e.g., TFV 

intake) for HIP participants and non-HIP participants after accounting for a variety of other 

characteristics. 

The characteristics we accounted for (i.e., covariates included in all the regressions) are: 

 Stratification/ blocking variables used in the sampling, which include indicators of household 

residential location, size and composition of household, and gender and age of household 

head 

 Demographic characteristics of respondents, including gender, age, and race/ethnicity 

 Baseline fruit and vegetable consumption derived from questions on frequency and quantity 

of specific types of fruits and vegetables consumed in the week prior to the survey 

 Baseline composite scales derived from questions about the home food environment, barriers 

to grocery shopping, and attitudes about and barriers to consumption of fruits and vegetables 

For outcomes in which the same survey question was asked in Round 1, such as attitudes toward 

fruits and vegetables, the analysis included the Round 1 response as an additional control in the 

regression. Analysis of dietary intake data also included covariates indicating whether the recall was 

the first or second recall and respondents’ assessment of whether their reported intake was usual.32

We calculated regression-adjusted means for the HIP and non-HIP groups that accounted for other 

characteristics (e.g., demographic characteristics) by using the estimated regression coefficients and 

the values of the included characteristics. 

 

31 By pooling, we mean that we included all of the observations—both the main interviews for the entire 

sample and the 10 percent subsample interviews (that will be used for the usual intake computations) from 

the Round 2 and 3 follow-up surveys. Pooling or stacking the interviews is the most efficient use of the 

data and provides a natural way to combine responses for individuals with varying numbers of interviews. 

To account for multiple observations for individuals, we “cluster” the data (i.e., we use the cluster option in 

Stata). Appendix G presents analyses by follow-up survey round.

32 Appendix E, Exhibit E.1 presents the complete list of covariates used in the regressions and how they were 

defined; it also presents means and standard errors for all covariates. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 40 ▌2. Evaluation Design, Data, and Methods Abt Associates

Our strategy for reporting statistical significance in the exhibits and in the text discussion is as 

follows: 

 In the exhibits, we use asterisks to indicate statistical significance: *p < .10; **p < .05; and 

***p < .01. 

 In the text discussion we consider p-values lower than 0.05 as statistically significant and 

discuss those results. We consider p-values of greater than 0.05 and lower than 0.10 as 

“borderline” significant, and discuss those results in the text only when broadly consistent 

with other findings and/or otherwise placing those findings into context. 

Analysis of Binary and Categorical Outcomes 

Many of the secondary outcomes analyzed in this report are either binary (e.g., yes/no) or categorical 

(e.g., strongly disagree/disagree/neither agree nor disagree/agree/strongly agree) variables. As with 

continuous variables, we present results that are adjusted to account for sample members’ 

characteristics at baseline. In the body of the report, we present analyses based on linear probability 

and linear regression models as they are more easily interpretable. We also analyzed the outcomes 

using logistic and ordered logistic regression models and present the models in Appendix F. The 

results from both types of models are broadly similar. 

In all analyses presented in this report, binary outcome variables are coded so that 1= “yes” and 0= 

“no.” Each outcome is regressed on a HIP participation indicator and other explanatory variables 

(described above). The results show the effect of HIP participation on the probability of saying “yes” 

to the question. 

Categorical outcomes analyzed in this report include items such as the degree of agreement with a 

particular statement about, for example, nutrition attitudes. These outcomes were coded into fivepoint Likert scales, with higher values indicating greater levels of agreement. Just as in the binary 

case above, the outcome was regressed on the HIP participation indicator and the other explanatory 

variables. The resulting coefficient estimates provide a straightforward description of the direction of 

HIP’s impact on these outcomes (e.g., whether HIP participants more strongly agree or more strongly 

disagree with a particular statement than do non-HIP participants).33 

Survey Weights 

All analyses using participant survey data were weighted to provide estimates of the SNAP 

population in Hampden County. Weights were constructed to account for the sampling design and 

survey non-response. See Appendix C for a detailed discussion of how the weights were constructed. 

The analysis uses all available Round 2 and Round 3 interviews. As noted in Section 2.2, for 

approximately 10 percent of both the Round 2 and Round 3 samples, we conducted a second 24-hour 

dietary recall interview in order to allow estimation of usual intake. We include these interviews in 

our analysis. 

 

33 The coefficients cannot, however, be interpreted numerically as a scale score because one cannot assume 

that each categorical response is one unit different from the next. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 41

2.10 Sample Description 

The analyses in this report use two samples, one a subset of the other. As discussed above, all SNAP 

households in Hampden County were randomly assigned to either the HIP or non-HIP evaluation 

group. Much of the analysis using EBT data includes all of the approximately 55,000 SNAP 

households in the county. The participant survey samples were then drawn from among the HIP 

(treatment) and non-HIP (control) groups. The data collected on this smaller sample are used for the 

descriptive and impact participant analyses. In this section, we present descriptive statistics for both 

samples and compare the characteristics of the treatment and control groups. 

Hampden County SNAP Participants 

In this section, we use data from SNAP caseload files to describe the universe of SNAP participants 

in July 2011, when random assignment to HIP occurred prior to HIP implementation. SNAP case file 

data show (Exhibit 2.7) that just under half of SNAP heads of household (HoH) were Hispanic and 

that Spanish was spoken in nearly one-quarter of all households. SNAP household heads were 43 

years old on average and half were disabled. Most participants (80 percent) lived in private residences 

and 7 percent were homeless. One-third of households included children (and no elderly members) 

and just over 10 percent of households included elderly members. 

Although the SNAP population in Hampden County differs from the national SNAP population on 

some of these factors,34 the location was selected by Massachusetts due to its mix of urban, rural, and 

suburban areas that contain two of the lowest income cities in the State. Hampden County is racially 

diverse, has the lowest median household income and the highest poverty rate in the State. Residents 

in the western region of Massachusetts have the highest rates of obesity and related chronic illnesses 

in the State. 

In July 2011, average income for this population was $806, and a majority of SNAP household heads 

had some form of unearned income: Supplemental Security Income (32 percent); Social Security (27 

percent); Temporary Assistance to Needy Families (TANF) (13 percent); unemployment 

compensation (5 percent); and other types of unearned income (60 percent). The average SNAP 

benefit was $258 in July 2011. 

As expected, due to the random assignment design, no significant differences emerged between the 

treatment and control groups in these reported characteristics. An overall F-test indicated that there 

was no difference (p = 0.556) between treatment and control groups overall at baseline. 

 

34 Most notably, Hampden County had higher proportions of disabled and Hispanic household heads than the 

national SNAP caseload. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 42 ▌2. Evaluation Design, Data, and Methods Abt Associates

Exhibit 2.7: Characteristics of SNAP Households in Hampden County, July 2011 

Total Treatment Control 

Variable (proportion) (proportion) (proportion) P-value 

Race/ethnicity of head of household 

Hispanic 0.44 0.43 0.44 [0.722] 

Non-Hispanic white 0.37 0.37 0.37 [0.714] 

Non-Hispanic black 0.13 0.13 0.13 [0.325] 

Non-Hispanic other 0.07 0.07 0.07 [0.930] 

Spanish spoken in household 0.22 0.22 0.22 [0.925] 

Age of head of household 

16-30 0.28 0.28 0.28 [0.388] 

31-40 0.21 0.21 0.21 [0.700] 

41-54 0.26 0.26 0.27 [0.243] 

Over 54 0.25 0.25 0.25 [0.946] 

Mean age (years) 43 43 43 [0.601] 

Disabled head of household 0.50 0.50 0.50 [0.974] 

U.S. citizen head of household 0.96 0.96 0.96 [0.370] 

Household composition 

Elderly (with or without children) in 

household 0.12 0.12 0.12 [0.348] 

Children (no elderly) in household 0.36 0.37 0.36 [0.636] 

No elderly or children in household 0.51 0.51 0.51 [0.870] 

Household is homeless 0.07 0.07 0.07 [0.990] 

Housing type 

Private 0.80 0.80 0.80 [0.369] 

Public 0.14 0.14 0.14 [0.358] 

Other 0.06 0.06 0.06 [0.860] 

Monthly household gross incomea 

$0 0.24 0.23 0.24 [0.321] 

$1-$787 0.26 0.27 0.26 [0.104] 

$788-$1,082 0.25 0.25 0.25 [0.941] 

$1,083 or higher 0.25 0.25 0.25 [0.459] 

Mean income ($) $806 $804 $807 [0.714] 

Types of income received by head 

Supplemental Security Income 0.32 0.32 0.33 [0.823] 

Social Security 0.27 0.27 0.27 [0.774] 

Temporary Assistance for Needy 

Families 0.13 0.13 0.13 [0.905] 

Unemployment compensation 0.05 0.05 0.05 [0.167] 

Other unearned income 0.60 0.60 0.60 [0.241] 

Monthly SNAP benefit 

$161 or less 0.25 0.26 0.25 [0.230] 

$162-$200 0.39 0.39 0.39 [0.679] 

$201-$349 0.10 0.11 0.10 [0.254] 

$350 or higher 0.25 0.24 0.25 [0.123] 

Mean benefit ($) $258 $255 $259 [0.076]* 

Sample size 55,095 7,500 47,595 

F-valueb .93 

P-value 0.573 

Two-sided t-test: *p<0.1, **p<0.05, ***p<0.01. 

a

 Includes earned and unearned income. 

b

 Variables included in F-test, but not shown in table: Baystate combined application project (CAP) status for SSI 

recipients; recertification type (semiannual reporting, recertification, other). 

Source: DTA SNAP Caseload Data. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 43

Participant Survey Sample 

The participant survey sample included equal numbers of HIP and non-HIP respondents. The sample 

used for the analyses presented in this report included respondents who completed both the Round 1 

and either or both of the Round 2 and Round 3 surveys. In this section, we present the (weighted) 

baseline characteristics (pre-HIP implementation) of the analysis sample. Data come from both the 

DTA caseload files and the participant survey. Some baseline characteristics are measured at the 

household level and others pertain to the individual survey respondent or to the head of the sampled 

household. 

Households in the analysis sample contained two or three people, on average, and just under half 

were single-member households at baseline (Exhibit 2.8). There were children (and no elderly) in 

approximately 42 percent of households and elderly in just over 10 percent of households. Full-time 

employment was not common in SNAP households. Less than 20 percent of households had at least 

one member that was employed full-time. 

Examining participation in other assistance programs, we found that about one-fifth of households 

received WIC in the month prior to the survey, and most households with qualifying children 

received free school lunch in the week prior to the survey.35 About one-third of households received a 

monthly SNAP benefit between $162 and $200, and almost another third received over $350 per 

month. 

Comparing the household characteristics of treatment (HIP) and control (non-HIP) groups in our 

survey sample, the average number of adults was somewhat larger in the control group. No other 

statistically significant differences emerged at baseline at the 5 percent level (a few differences were 

significant at the 10 percent level, as would be expected by chance). 

Exhibit 2.8: Baseline Characteristics of Households Completing Follow-Up Participant 

Surveys (Round 2 or 3), by Treatment and Control Status 

Total Treatment Control 

Proportion

(N) 

Proportion 

(N) 

Proportion 

(N) P-value 

Sample 

size 

DTA SNAP caseload data 1954 

Household residence 

Springfield 0.53 

(1030) 

0.52 

(505) 

0.53 

(525) [0.647] 

Chicopee or Holyoke 0.25 

(505) 

0.27 

(255) 

0.25 

(250) 

Hampden County balance 0.22 

(419) 

0.21 

(220) 

0.22 

(199) 

Persons in household 

Mean (SE) 2.34 

(0.04) 

2.26 

(0.05) 

2.35 

(0.05) [0.208] 

 

35 If the interview was conducted during the summer, the respondent was asked about receipt of free school 

lunch in the prior school year. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 44 ▌2. Evaluation Design, Data, and Methods Abt Associates

Total Treatment Control 

Proportion

(N) 

Proportion 

(N) 

Proportion 

(N) P-value 

Sample 

size 

Single-member household 

One person in household 0.45 

(872) 

0.45 

(450) 

0.45 

(422) [0.916] 

Multiple persons in household 0.55 

(1082) 

0.55 

(530) 

0.55 

(552) 

Adults in household 

Mean (SE)a 1.50 

(0.02) 

1.42 

(0.02) 

1.51 

(0.03) [0.021]** 

Number of adults 

3 or fewer adults in household 0.97 

(1918) 

0.98 

(965) 

0.97 

(953) [0.067]* 

4 or more adults in household 0.03 

(36) 

0.02 

(15) 

0.03 

(21) 

Household composition 

Elderly (with or without children) in 

household 

0.11 

(238) 

0.11 

(129) 

0.11 

(109) [0.975] 

Children (no elderly) in household 

0.42 

(827) 

0.43 

(398) 

0.42 

(429) 

No elderly or children in household 

0.46 

(889) 

0.46 

(453) 

0.46 

(436) 

Monthly SNAP benefit 

$161 or less 0.24 

(498) 

0.25 

(257) 

0.24 

(241) [0.937] 

$162–$200 0.31 

(592) 

0.31 

(308) 

0.31 

(284) 

$201–$349 0.14 

(272) 

0.14 

(134) 

0.14 

(138) 

$350 0.31 

(592) 

0.30 

(281) 

0.31 

(311) 

Participant survey data (primary shopper module) b

Household composition 1849 

Persons in household [mean (SE)] 2.87 

(0.05) 

2.83 

(0.05) 

2.88 

(0.06) [0.490] 

Children under age 5 [mean (SE)] 0.28 

(0.02) 

0.32 

(0.02) 

0.28 

(0.02) [0.156] 

Children age 5-17 [mean (SE)] 0.83 

(0.03) 

0.80 

(0.04) 

0.84 

(0.04) [0.479] 

Adults age 18-64 [mean (SE)] 1.58 

(0.03) 

1.52 

(0.03) 

1.59 

(0.03) [0.154] 

Adults age 65 and up [mean (SE)] 0.18 

(0.01) 

0.18 

(0.01) 

0.18 

(0.01) [0.836] 

Household employment status 

(prior week) 1829 

Any members full-time employed 0.19 

(320) 

0.17 

(150) 

0.19 

(170) [0.327] 

Any members part-time employed 0.13 

(255) 

0.15 

(135) 

0.13 

(120) [0.239] 

Any members not employed 0.67 

(1262) 

0.69 

(646) 

0.67 

(616) [0.536] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 45

Total Treatment Control 

Proportion

(N) 

Proportion 

(N) 

Proportion 

(N) P-value 

Sample 

size 

Program participation 

Received WIC (prior month) 0.19 

(335) 

0.20 

(169) 

0.18 

(166) [0.541] 1842 

Used food pantry/soup kitchen 

(prior month) 

0.11 

(209) 

0.10 

(101) 

0.12 

(108) [0.425] 1841 

Received Sr. Nutrition/Meals on 

Wheels (prior month)c

0.02 

(32) 

0.02 

(17) 

0.02 

(15) [0.894] 1847 

Child received free/reduced price 

lunch (prior week)d

0.87 

(702) 

0.90 

(349) 

0.86 

(353) [0.098]* 789 

Participant survey data (respondent module) b

Education level 1945 

8th grade or less 0.14 

(282) 

0.15 

(148) 

0.14 

(134) [0.378] 

9th-12th grade (no diploma) 0.30 

(583) 

0.31 

(296) 

0.30 

(287) 

High school diploma/GED 0.29 

(539) 

0.26 

(252) 

0.29 

(287) 

Some college or higher 0.27 

(541) 

0.28 

(278) 

0.27 

(263) 

Weighted proportions (unweighted Ns) for categorical variables; weighted means (standard errors) for 

continuous variables. 

Chi-square test for categorical variables, t-test for continuous variables; two-sided test: *p<0.1, **p<0.05, 

***p<0.01. 

Due to rounding, reported proportions may not sum to one. 

a

 Top-coded at 7. 

b Don't know,” “refused,” “inapplicable,” and “not ascertained” responses in Participant Survey data coded as 

missing. 

c

 All households responded to this question, including those with no qualifying seniors present. 

d

 Households with qualifying child only. 

Sources: DTA SNAP Caseload Data (July 2011), Participant Survey Round 1. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 46 ▌2. Evaluation Design, Data, and Methods Abt Associates

Among household heads in our survey sample (Exhibit 2.9), we found that almost three-quarters were 

female, just over 40 percent were Hispanic and almost one-quarter spoke Spanish in the home. Age 

varied fairly evenly from between 16 and 30 years old up to 55 years and older. Very few household 

heads were homeless and a majority lived in private housing. Turning to income and benefits, almost 

one-fifth of the household heads had no monthly income at baseline, but one-third had a monthly 

income of over $1,089. Most household heads were not receiving TANF, but about one-third were 

receiving SSI; 60 percent were receiving some other form of unearned income.36

Exhibit 2.9: Baseline Characteristics of Household Heads for Respondents Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control Status 

Total Treatment Control 

Proportion 

(N) 

Proportion 

(N) 

Proportion 

(N) P-value 

Gender 

Male 0.27 

(503) 

0.27 

(257) 

0.28 

(246) [0.861] 

Female 0.73 

(1451) 

0.73 

(723) 

0.72 

(728) 

Race/ethnicity 

Hispanic 0.43 

(816) 

0.42 

(401) 

0.43 

(415) [0.882] 

Non-Hispanic white 0.37 

(733) 

0.38 

(384) 

0.37 

(349) 

Non-Hispanic black 0.14 

(278) 

0.14 

(134) 

0.14 

(144) 

Non-Hispanic other 0.07 

(127) 

0.06 

(61) 

0.07 

(66) 

Primary language spoken in home 

Spanish language 0.22 

(426) 

0.22 

(213) 

0.22 

(213) [0.869] 

Other language 0.78 

(1528) 

0.78 

(767) 

0.78 

(761) 

Disability status 

Disabled 0.51 

(1036) 

0.51 

(534) 

0.51 

(502) [0.943] 

Not disabled 0.49 

(918) 

0.49 

(446) 

0.49 

(472) 

Citizenship 

US citizen 0.95 

(1876) 

0.96 

(946) 

0.95 

(930) [0.292] 

Not a US citizen 0.05 

(78) 

0.04 

(34) 

0.05 

(44) 

 

36 For baseline characteristics of sampled respondents and primary shoppers, see Appendix F, Exhibits F2.1–

F2.3. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 47

Total Treatment Control 

Proportion 

(N) 

Proportion 

(N) 

Proportion 

(N) P-value 

Age 

16-30 years 0.23 

(407) 

0.22 

(197) 

0.23 

(210) [0.773] 

31-40 years 0.23 

(459) 

0.24 

(224) 

0.22 

(235) 

41-54 years 0.30 

(587) 

0.29 

(292) 

0.30 

(295) 

55+ years 0.24 

(501) 

0.25 

(267) 

0.24 

(234) 

Homelessness 

Homeless 0.05 

(57) 

0.04 

(23) 

0.05 

(34) [0.347] 

Not homeless 0.95 

(1897) 

0.96 

(957) 

0.95 

(940) 

Housing type 

Private 0.80 

(1583) 

0.81 

(807) 

0.80 

(776) [0.368] 

Public 0.16 

(318) 

0.16 

(152) 

0.16 

(166) 

Other 0.04 

(53) 

0.03 

(21) 

0.04 

(32) 

Monthly household gross income 

$0 0.18 

(345) 

0.20 

(180) 

0.18 

(165) [0.902] 

$1–$787 0.24 

(474) 

0.24 

(242) 

0.24 

(232) 

$788–$1,088 0.25 

(496) 

0.24 

(246) 

0.25 

(250) 

$1,089a 0.33 

(639) 

0.32 

(312) 

0.33 

(327) 

Temporary Assistance for Needy Families (TANF)

Receiving TANF 0.18 

(347) 

0.17 

(164) 

0.18 

(183) [0.302] 

Not receiving TANF 0.82 

(1607) 

0.83 

(816) 

0.82 

(791) 

Supplemental Security Income (SSI)

Receiving SSI 0.32 

(654) 

0.32 

(335) 

0.32 

(319) [0.729] 

Not receiving SSI 0.68 

(1300) 

0.68 

(645) 

0.68 

(655) 

Retirement, Survivors, and Disability Insurance (RSDI)

Receiving Social Security 0.29 

(588) 

0.28 

(306) 

0.29 

(282) [0.564] 

Not receiving Social Security 0.71 

(1366) 

0.72 

(674) 

0.71 

(692) 

Unemployment compensation 

Receiving unemployment 

compensation 

0.04 

(94) 

0.06 

(54) 

0.04 

(40) [0.047]** 

Not receiving unemployment 

compensation 

0.96 

(1860) 

0.94 

(926) 

0.96 

(934) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 48 ▌2. Evaluation Design, Data, and Methods Abt Associates

Total Treatment Control 

Proportion 

(N) 

Proportion 

(N) 

Proportion 

(N) P-value 

Other unearned income 

Receiving other unearned income 0.60 

(1234) 

0.63 

(639) 

0.60 

(595) [0.211] 

No other unearned income 0.40 

(720) 

0.37 

(341) 

0.40 

(379) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported proportions may not sum to one. 

Source: DTA SNAP Caseload Data, July 2011 (unweighted N=1,954). 

Put together, descriptive statistics suggest that characteristics of our survey sample are largely similar 

to those of the universe of SNAP participants in Hampden County. Furthermore, the random 

assignment process appears to have been successful in selecting treatment and control groups who are 

not significantly different from each other. 

2.11 HIP/Non-HIP Balance and Attrition Analysis 

The previous section shows that the samples of HIP and non-HIP households were similar at the time 

of random assignment. Our design requires the HIP and non-HIP groups also to be similar at 

subsequent interview waves. In a standard random assignment design everyone selected would be 

interviewed at each wave regardless of any subsequent behavior. However, HIP only applies to those 

receiving SNAP, and SNAP exit is common. Approximately 15 percent of the sample had exited by 

the beginning of the first follow-up interview. Thus, following the standard approach would lead to 

expensive interviews with many people who were no longer on SNAP—and therefore no longer 

eligible for the HIP incentive.37 

We conducted follow-up interviews only with those still on SNAP. This design yields valid random 

assignment estimates if either (1) attrition is not related to HIP status or (2) any differential attrition 

can be controlled for with sampling weights. Both of these assumptions seemed plausible. With 

respect to the first condition, since the HIP benefit is relatively small compared to the average 

monthly SNAP benefit it seemed unlikely to have a large impact on SNAP exit.38 With respect to the 

second condition, the SNAP case files contain rich information on the households at baseline 

(demographics, income, benefits received) which can be used to construct sampling weights, which 

could plausibly control for moderate levels of differential attrition. 

Some of these assumptions are testable. The assumption of differential attrition from SNAP is directly 

testable from the SNAP caseload data. Exhibit 2.10 reports SNAP caseloads for the HIP and non-HIP 

 

37 The net result of following the standard randomization design would have been a requirement for a sample 

more than twice the size of the current sample to include a large enough subsample still on SNAP. That 

study design was not cost feasible. 

38 Chapter 5 reports that on average households purchased just over $12 of HIP-eligible items each month, 

which earned them incentives of approximately $3.60 per month, a relatively small amount compared to an 

average monthly SNAP benefit of $258. In addition, the value to households of additional SNAP dollars is 

smaller than their cash value. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 2. Evaluation Design, Data, and Methods ▌pg. 49

groups from the time they were selected in July 2011. Our concern is whether there was differential 

attrition in the two groups. In particular, if HIP households were less likely to leave SNAP, we might 

be concerned that the value of the incentives affected SNAP participation decisions. Exhibit 2.10 

provides no evidence for such a conjecture. Retention rates for the HIP and non-HIP groups were 

quite similar. Over all months, the ratio of the retention rates ranged between 98.4 percent and 100.1 

percent. Furthermore, if anything, exit rates were higher for the HIP group despite the benefit 

(though, as just noted, the differences were trivial). 

We also examined retention rates separately for the survey and non-survey samples (see Appendix F, 

Exhibit F2.4). Again, the differences were trivial. In the survey group, across the survey months, the 

ratio of the retention rates varied between 97.9 percent and 100.5 percent; in the non-survey group, 

the ratio varied between 99.0 percent and 100.5 percent. 

We conclude that there is little evidence of more than non-trivial differences in attrition rates. It 

follows that the design decision to only survey those who remained on SNAP is unlikely to affect the 

impact estimates. 

Exhibit 2.10: HIP/Non-HIP Balance from the SNAP Caseload Files 

Number of households Retention rate (%) Ratio (%) 

Month 

HIP 

(1) 

Non-HIP 

(2) 

HIP 

(3) 

Non-HIP 

(4) 

HIP/Non-HIP 

(3): (4) 

July 2011 7,383 46,871 100.0 100.0 100.0 

August 2011 7,137 45,374 96.7 96.8 99.9 

September 2011 6,952 44,106 94.2 94.1 100.1 

October 2011 6,732 43,121 91.2 92.0 99.1 

November 2011 6,603 42,321 89.4 90.3 99.1 

December 2011 6,494 41,521 88.0 88.6 99.3 

January 2012 6,454 41,482 87.4 88.5 98.8 

February 2012 6,395 41,052 86.6 87.6 98.9 

March 2012 6,287 40,547 85.2 86.5 98.4 

April 2012 6,249 40,076 84.6 85.5 99.0 

May 2012 6,211 39,788 84.1 84.9 99.1 

June 2012 6,153 39,486 83.3 84.2 98.9 

July 2012 6,112 39,123 82.8 83.5 99.2 

August 2012 6,033 38,361 81.7 81.8 99.8 

September 2012 5,990 38,119 81.1 81.3 99.8 

October 2012 5,851 37,409 79.2 79.8 99.3 

Source: DTA SNAP Caseload Data. 

In addition, we tested for HIP/non-HIP balance in the baseline characteristics (July 2011) of the 

SNAP caseload during the month corresponding to the midpoint of Round 2 (May 2012) and the 

midpoint of the Round 3 survey (October 2012). The results of those tests are reported in Appendix F, 

Exhibits F2.5 and F2.6. There were no significant differences between the treatment and control 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 50 ▌2. Evaluation Design, Data, and Methods Abt Associates

groups on any of the variables tested, and the joint F-test further shows no evidence of difference 

between the two groups.39 

In summary, this study adopted a non-standard strategy of only following households who remained 

on SNAP. We examined the implications of that decision by analyzing attrition from both the HIP 

and non-HIP groups over time and examining HIP/non-HIP balance on a number of characteristics at 

the sampling stage and at both 4-6 months and 9-11 months after HIP implementation. All of these 

tests indicated that the HIP and non-HIP groups have remained balanced over time. Thus, we 

conclude that it is reasonable to interpret the evaluation results as one would interpret a study that 

followed everyone who had been randomized. 

 

39 Appendix F, Exhibit F2.7 shows the balance test for HIP and non-HIP participants’ characteristics in May 

2012 for sampled households on SNAP that month. Exhibit F2.8 shows the balance test for HIP and nonHIP participants’ characteristics in October 2012 for sampled households on SNAP that month. Results 

similarly suggest no overall difference between the groups. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 3. HIP Implementation and Operations ▌pg. 51

3. HIP Implementation and Operations 

Understanding how HIP was implemented and the challenges that arose is important for two reasons. 

First, implementation may influence the pilot’s outcomes. As discussed in Chapter 1, HIP impacts 

depended on the project’s success in recruiting and training retailers, implementing new EBT 

processing methods, and informing participants. During the pilot, many difficult aspects of 

implementation went smoothly, but some did not. Second, HIP provides valuable information for any 

future pilots or a broad-scale roll-out of a HIP-like program. The discussion in this chapter, which is 

based primarily on the in-depth stakeholder interviews, draws heavily from the Healthy Incentives 

Pilot (HIP) Early Implementation Report (Bartlett, Beauregard, et al., 2013), which provides 

additional details on all aspects of HIP implementation. 

This chapter proceeds as follows. The first section provides an overview of HIP implementation, 

focusing on activities that were central to the development of the pilot, particularly as it affected 

participants. The second section examines several implementation issues or challenges that arose 

during the pilot that could potentially have affected participants’ understanding of HIP or could have 

created confusion about how to earn incentives. The final section discusses how the implementation 

might have affected participant experiences and the pilot’s impact on outcomes examined in Chapters 

5-8. 

3.1 Overview of Implementation and Operations 

HIP was an innovative and complex project. Planning and implementation was difficult, requiring 

DTA to coordinate the work of several different entities to ensure the pilot was up and running in 15 

months. While the implementation process posed many challenges, DTA succeeded in implementing 

the pilot on schedule. 

Implementation of HIP required extensive preparations that began with FNS’s design of the pilot 

concept and continued with DTA’s submission of a grant application in December 2009. Pilot 

implementation activities accelerated in August 2010 when FNS selected Massachusetts to operate 

HIP. The planning and implementation phase extended until November 1, 2011 when HIP operations 

began and the first SNAP participants began earning incentives. The pilot was rolled out to 

participants in three waves, beginning November 1, 2011, December 1, 2011, and January 1, 2012. 

Key planning and implementation activities included the following: 

 Assembling the team, including hiring HIP-specific DTA personnel 

 Designing and implementing EBT system changes 

 Recruiting retailers to participate in HIP 

 Establishing a HIP Steering Committee made up of community partners 

 Developing training materials and notifications for HIP participants 

Effectively executing and managing these activities was crucial to the pilot’s success. The rest of this 

section discusses these key activities, describing the organizations that were involved and 

highlighting successes and challenges. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 52 ▌3. HIP Implementation and Operations Abt Associates

HIP Project Team 

FNS understood from the beginning that a successful pilot project would involve multiple entities, 

including Massachusetts’ SNAP EBT system provider, national retail chains, and local SNAP offices. 

Managing the number and different types of stakeholders involved in HIP was a substantial 

undertaking. 

The following seven organizations or groups played key roles in developing and implementing HIP: 

 USDA’s Food and Nutrition Service (FNS), the Federal agency responsible for SNAP 

 Massachusetts Executive Office of Health and Human Services’ (EOHHS) Department of 

Transitional Assistance (DTA), the State agency responsible for SNAP and therefore HIP 

 Xerox (formerly Affiliated Computer Systems), the EBT processor in Massachusetts that 

operated HIP as part of the EBT system 

 Third-party processors (TPPs), contractors hired by retailers (with integrated electronic cash 

registers) to provide EBT data processing services 

 Novo Dia Group (NDG), an EBT technology services and consulting company hired by DTA 

to coordinate system design and testing activities for retailers and TPPs 

 Hampden County retailers, recruited by DTA to participate in HIP, ranged from large 

grocery chains to small stores and farmers markets. Retailers with integrated electronic cash 

register (IECR) systems contracted with their technology partners to modify their store 

systems for HIP. (As discussed below, retailers without IECRs used manual processes; no 

contractor was involved.) 

 Community partners (CPs), local and regional non-profit organizations or community-based 

organizations (CBOs), State and city agencies, medical centers, religious organizations, 

libraries, and higher education institutions that assisted with implementation 

DTA had ultimate responsibility for managing the implementation and operation of HIP. To meet its 

HIP responsibilities, DTA hired seven full-time staff members, including a director, assistant director, 

retailer liaison, two trainers, and two information coordinators. An additional six existing DTA staff 

worked part-time on HIP.40 

HIP Systems Design and Modifications 

In order for HIP to operate, Massachusetts DTA and its contractors needed to adapt state-level 

information and financial systems to accommodate HIP-specific tasks that went beyond standard 

operating procedures for SNAP. Software was developed, pre-tested, and rolled out on a tightly 

coordinated schedule. EBT system modifications were necessary to identify when an incentive was 

earned, calculate the incentive amount to credit HIP clients, and draw down HIP funds from the 

Federal Reserve Bank to pay retailers for food purchases. 

 

40 By design, the three local offices (two in Springfield and one in Holyoke) had only a minimal role in the 

HIP implementation. Consistent with that design, local DTA office directors reported that HIP had little or 

no impact on their staff’s workload. Clerks and caseworkers were trained to answer basic HIP questions 

and to refer all other questions to the HIP 800 call line. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 3. HIP Implementation and Operations ▌pg. 53

FNS began the system design process in October 2008 (prior to conducting the HIP grant application 

process) and prepared high-level HIP design requirements. More detailed development began after 

FNS awarded the HIP demonstration grant to Massachusetts in August 2010. As Massachusetts’ EBT 

processor, Xerox had primary responsibility for managing the HIP EBT system design changes and 

processing HIP transactions. Xerox reviewed and discussed the HIP implementation requirements 

with the DTA HIP team at an initial start-up meeting in September 2010. In December 2010, Xerox 

led the Joint Application Design (JAD) sessions, which included DTA, FNS, Novo Dia Group, and 

the Abt evaluation team. These sessions identified the detailed requirements and rules for HIP, and 

the necessary modifications to the different systems. Based on the requirements, Xerox and DTA 

produced the design documents to guide these modifications and the changes to be made by retailers 

and TPPs. 

HIP implementation required each of the major partners in EBT operations to make substantial 

system modifications. The following modifications were made during the spring and summer of 

2011: 

 DTA modified its client eligibility system, BEACON, to support the random assignment of 

HIP participants, their identification in the system, the transmission of participant status to the 

EBT system, and the generation of notices to HIP households. 

 Xerox modified its EBT processing system, the Electronic Payment Processing Information 

Control (EPPIC), as well as its system for automated and staffed customer service, and the 

software for EBT-only point-of-sale (POS) terminals used by smaller independent retailers. 

 Retailers used specifications provided by Xerox to modify their integrated electronic cash 

register (IECR) systems to comply with HIP transaction processing requirements. The three 

TPPs used by Hampden County retailers modified their systems to pass HIP messages 

between the retailer IECR system and the EBT processing system. 

As modifications were completed for each system affected by HIP, team members and technical staff 

conducted comprehensive testing. The key tests were the user acceptance test (UAT) for the changes 

to EPPIC and the retailer acceptance tests, which involved both retailer and TPP systems. With the 

exception of one convenience store chain, the TPPs and IECR retailers were ready for the November 

1, 2011 HIP “go live” date. 

Retailer Recruitment 

DTA recognized early that retailer participation would be critical to the success of the pilot. If HIP 

was to have any influence over food intake, SNAP participants had to be able to locate and access 

participating authorized retailers. 

One of the pilot’s goals was to test this point-of-sale incentives approach in each of the environments 

in which SNAP operated. Therefore, all 474 SNAP-authorized retailers selling HIP targeted fruits and 

vegetables were eligible and invited to participate in HIP (see Exhibit 3.1). 
Evaluation of the Healthy In

pg. 54 ▌3. HIP Implementation and Operations Exhibit 3.1: Hampden County Retailers Eligible for HIP and Participating in HIP 

Store type 

(% of total Hampden 

County SNAP 

redemptions) a

Number eligible for 

HIP 

(% eligible for HIP) 

Retailers participating as of 

November 1, 2011 

Retailers pa

Octob

Number 

(%) 

Percent of 

eligible 

retailers 

Percent of total 

SNAP 

redemptions 

Number 

(%) 

P

elig

Supermarkets (23%) 

16 

(3.4) 

10 

(14.1) 62.5 57.5 12 (11.5) 

Superstores (56%) 

29 

(6.1) 

8 

(11.3) 27.6 55.6 8 (7.7) 

Grocery stores and food specialty storesb

(12%)

93 

(19.6) 

19 

(26.8) 20.4 29.0 28 (26.9) 

Convenience storesc

(9%) 

318 

(67.1) 

34 

(47.9) 10.7 5.9 40 (38.5) 

Farmers marketsd

(<1%) 

18 

(3.8) N/A N/A N/A 16 (15.4) 

Total 474 (100.0) 71 (100.0) 15.0 48.7 104 (100.0) 

a July 2011. b Includes small, medium, and large grocery; fruit/vegetable specialty; meat specialty; seafood specialty. c Includes convenience store and combination grocery/other. d Includes farmers markets and direct marketing farmers that began operating in summer 2012. 

Source: Retailer list received from DTA. Exhibit includes only stores located in Hampden County. Several chain retailers implemented HIP in store

County, notably in neighboring counties in Massachusetts and Connecticut (12 stores as of November 2011; 26 stores as of October 2012). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 3. HIP Implementation and Operations ▌pg. 55

The types of retailers that were eligible to participate are listed below: 

 Supermarket and superstore chain retailers—large retailers that serve the highest percentage 

of SNAP households and account for a substantial majority of SNAP redemptions 

 Grocery stores and specialty stores—local stores that have a smaller market share, but may 

provide ethnically diverse foods and serve households without easy access to large 

supermarkets 

 Convenience stores and combination grocery/other stores41—used frequently by SNAP 

households for small purchases of both food and non-food items. Although many of them do 

not carry a wide selection of fruits and vegetables, those that carried HIP-eligible fruits and 

vegetables were eligible to participate in HIP 

 Farmers markets—provide locally-grown fresh fruits and vegetables in season, typically 

operating between May and November 

DTA began to identify and recruit a targeted group of retailers while preparing its application. Once 

Massachusetts was selected to operate HIP, the agency used direct outreach to retailers and indirect 

outreach through other State agencies and food retailer coalitions. Community partners also played a 

substantial role in recruiting both large and small retailers. 

Large retail chains and smaller independent stores required different recruitment strategies, primarily 

because the approach and access to the individuals who could make the decision about participating 

in HIP differed. DTA worked directly with chain retailers that initially expressed interest in HIP, 

helping them to make the necessary system modifications. DTA also continued efforts to recruit other 

chain retailers, working through corporate headquarters. 

As the pilot moved forward, DTA sent letters to all SNAP-authorized retailers who had not yet been 

recruited asking them to participate in HIP. The letters invited the retailers to information sessions 

about HIP. The response from smaller independent retailers to this outreach was low and DTA shifted 

to a strategy that relied on in-person contact. To implement that revised strategy, DTA hired a retailer 

liaison to both recruit and train retailers. After this hire, the main recruitment method for independent 

retailers became in-person store visits. Store visits (generally made without an appointment) focused 

on explaining HIP and what was required of participating retailers. The retailer liaison provided 

handouts summarizing the basics of HIP and discussed HIP participation with store owners. Retailers’ 

questions and concerns generally focused on the impact the pilot might have on store operations. The 

retailer liaison reported that it required approximately five visits for a retailer to commit to HIP, 

significantly more effort than originally anticipated. The recruitment process involved developing 

relationships with store owners, which required time and multiple visits. 

As of November 1, 2011, when implementation began, 71 stores were participating in HIP. Exhibit 

3.1 shows the distribution of HIP-eligible retailers and HIP participating retailers in Hampden County 

by store type. While overall only 15 percent of eligible retailers were participating in HIP, 63 percent 

of supermarkets and 28 percent of superstores were participating. These two types of stores accounted 

for the vast majority (79 percent) of Hampden County SNAP redemptions. 

 

41 Combination grocery/other stores stock grocery items as well as other items pertinent to their business. 

These stores include, for example, gas stations with mini-marts and pharmacies that carry grocery items. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 56 ▌3. HIP Implementation and Operations Abt Associates

Convenience stores, while numerous, tend to carry very few eligible items and accounted for only 9 

percent of SNAP redemptions in the county. An additional 20 percent of eligible retailers were 

grocery stores and specialty stores; approximately 20 percent of them were participating in HIP. In 

Hampden County, 12 percent of SNAP redemptions occurred in these types of stores. 

DTA continued recruiting efforts after HIP implementation and, as a result, an additional 8 stores 

began accepting HIP as of February 1, 2012. Additionally, 16 farmers markets/farm stands/mobile 

markets joined HIP in spring and summer 2012, and 9 new stores began participating in HIP on 

October 1, 2012.42 Finally, several chain retailers implemented HIP in a total of 26 stores located 

outside of Hampden County, in neighboring Massachusetts and Connecticut counties. However, no 

new superstores joined HIP despite these intensive efforts. 

As of the beginning of October 2012 less than 22 percent of eligible stores were participating in HIP, 

but administrative data showed that SNAP redemptions in these stores represented 59 percent of total 

Hampden County SNAP redemptions. These percentages indicate that while the overall number of 

stores participating in HIP was relatively low, the stores that did participate were used by SNAP 

households for a majority of their SNAP purchases. This shows that households’ access to HIP 

participating stores was greater than is suggested by simply looking at the number of HIP 

participating stores. However, the relatively lower participation of superstores likely limited 

opportunities for HIP households to earn incentives when using SNAP benefits. 

HIP Steering Committee 

Hampden County has a strong network of community organizations, including non-profit communitybased organizations (CBOs), health centers, libraries, religious organizations, and educational 

institutions, as well as State and local agencies. These community partners proved to be an integral 

factor in the implementation of HIP and an important component of the smooth rollout of the pilot. 

Approximately 75 community partner organizations contributed services in support of HIP. 

DTA established a HIP Steering Committee (HSC) during the application process to discuss policy, 

hold DTA accountable to the community, and help identify areas where community partners could 

contribute. The Steering Committee was made up of a diverse and committed group of individuals 

and organizations that actively supported HIP implementation. It included about a dozen CBOs, plus 

representatives from WIC, the DTA Central Office, the three local DTA offices, and the DTA 

Regional Director for western Massachusetts. 

The community partners, in general, and the HIP Steering Committee, in particular, were active in the 

following ways: 

 Helping to recruit retailers, both large retailers with IECRs and small local stores without 

IECRs 

 Reviewing and providing feedback on outreach and training materials 

 Providing translation and interpretation support (Russian and Vietnamese) for participant 

training sessions 

 

42 One of the 9 stores changed ownership in November 2012. The new owner agreed to continue participating 

in HIP. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 3. HIP Implementation and Operations ▌pg. 57

 Providing facilities for HSC meetings and for participant, retailer, and community partner 

training sessions 

 Serving as an information and referral resource to HIP clients and other community 

organizations 

Notification and Training of HIP Participants 

For HIP to influence food purchases and diet quality, DTA recognized that HIP participants had to 

understand the purpose of the pilot, be able to locate retailers, and identify and purchase targeted HIP 

fruits and vegetables. 

DTA and its partners put considerable effort into the notification and training of HIP participants. 

Activities included the development of user-friendly materials as well as a schedule and process to 

disseminate those materials in a series of mailings so as not to overwhelm participants. HIP retailers 

received decals with the HIP logo to install on their store doors to identify them as participating in 

HIP. DTA scheduled over 140 training sessions and provided other resources such as a toll-free call 

line to help participants understand HIP. Each activity is described below. 

Mailings 

Prior to each wave’s HIP start date, DTA sent three mailings to participants.43 Each mailing contained 

a notification letter and a subset of training materials. 

 The first notification letter, sent about three weeks prior to each wave’s HIP start date, 

informed participants that they had been selected to participate in HIP and relayed 

information about the purpose of HIP, the selection process, the option of declining to 

participate, and the start date. An initial calendar of training sessions was included with the 

letter. 

 The second notification letter, sent about two days after the first, explained the incentive. A 

more extensive subset of training materials was included with this letter: a HIP brochure, 

guidelines for HIP-eligible fruits and vegetables, a list of participating retailers, and a list of 

frequently asked questions. 

 The third notification letter, sent several days prior to the HIP start date, explained that when 

shopping at retailers where they had to separate their SNAP items from other purchases, 

participants would need to identify themselves as HIP participants in order to earn the 

incentive. A HIP EBT card sleeve (shown in Exhibit 3.2), which contained information on 

eligible foods and could be used as proof of HIP participation, was included with this letter. 

To distinguish the HIP mailings from regular SNAP mailings, materials included the HIP logo, which 

shows pictures of fruits and vegetables and uses the tag line “It’s HIP to be healthy.” Materials were 

translated into Spanish, Russian, and Vietnamese. The mailings also included a “Babel Card,” a note 

alerting participants in 21 languages that the materials in the envelope were important and related to 

 

43 Mailings were sent to HIP households that were active SNAP participant households at the time of the 

scheduled mailing. HIP households that were inactive at the time of the scheduled mailing and who later 

became active SNAP households received the mailing when they again became SNAP participants. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 58 ▌3. HIP Implementation and Operations Abt Associates

benefits. The card encouraged them to bring the materials to a bilingual friend or relative to be 

translated. 

Exhibit 3.2: HIP Card Sleeve 

During the pilot, DTA prepared four additional mailings, designed to keep participants updated on 

pilot activities. They also prepared a final mailing to remind participants that their opportunity to earn 

the incentive would be ending. Some mailings were designed to improve participant understanding of 

the program and to promote HIP. All notifications were translated into Spanish, Russian and 

Vietnamese. 

 In early February 2012, HIP participants received an updated list of retailers that included 

those that had joined HIP as of February 1. 

 A mailing was sent in June that contained the following: 

 a colorful brochure describing HIP and its benefits in simple terms (Exhibit 3.3). This 

brochure was specifically designed to be “user-friendly” in an effort to improve 

participant awareness and understanding of HIP 

 an updated list of retailers, including participating chain retail stores outside of Hampden 

County 

 a list of participating farmers markets 

 A mailing, customized for each HIP participant, was sent in mid-August. It was designed to 

boost HIP participation by informing participants of the amount of the incentives that they 

had earned through late July and providing an estimate of how much they could earn in the 

coming month. It also included three recipe cards and information on some of the more 

visible community partners. 

 The mailing sent at the end of September included an updated retailer list and an updated 

farmers market list with confirmed closing dates. For Wave 1 HIP participants, it also served 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 3. HIP Implementation and Operations ▌pg. 59

as their final mailing, reminding them that their ability to earn the HIP incentive was ending 

at the end of October. 

 A final mailing was sent to Wave 2 and 3 participants, reminding them that their opportunity 

to earn incentives would be ending at the end of November and December, respectively. 

Exhibit 3.3: Brochure from Second Interim Mailing (2 of 6 Panels) 

Training Sessions 

DTA offered over 140 voluntary training sessions for HIP participants beginning shortly before the 

system went live (October 2011) until about four months afterward (February 2012). These sessions 

were intended to help participants understand HIP and how it could benefit their households. The 

main goals of the trainings were to explain how the financial incentive worked and which foods were 

eligible for the HIP incentive. Despite the significant efforts that went into developing HIP training, 

approximately 100 participants attended training sessions, representing less than 1.3 percent of all 

eligible HIP participants. DTA had hoped to provide meals, childcare, or transportation to help 

encourage attendance at trainings, but FNS policies did not permit the provision of these services with 

Federal grant funding. Many at DTA were not surprised by the low turnout, having had similar 

experiences with the rollout of other program changes. Most trainings were held during the work day, 

which likely limited the ability of some participants to attend. However, the trainings were well 

received by those who attended. 

Other Resources 

DTA developed a number of resources to support HIP participants including a dedicated HIP 800 call 

line, email address, and website. The HIP call line was the most used resource. Approximately 800 

calls were received over the life of the pilot from the 7,500 HIP participant households. The greatest 

proportion of questions related to how the HIP incentive operated. Calls increased in June after the 

second interim mailing; almost a quarter of calls were received from August to October 2012, around 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 60 ▌3. HIP Implementation and Operations Abt Associates

the time of the third and fourth interim mailings. In addition, community partner organizations served 

as an informational and referral resource for HIP clients, providing translation and interpretation 

support, as needed. 

3.2 Challenges Potentially Affecting Participant Outcomes 

This section describes challenges to implementation that occurred during the operation of the pilot, 

beginning November 1, 2011, and ending December 31, 2012. Some of these challenges were 

technical, resulting from the EBT and retailer systems-related changes that were made to implement 

HIP. Others arose while recruiting retailers and educating SNAP participants selected to participate in 

HIP. In this section, we focus on the challenges that could have affected HIP participants’ 

understanding of the program and their ability to earn incentives, thereby potentially affecting the 

impact of HIP on desired participant outcomes. 

Technical Systems Issues 

The first technical issue arose at the beginning of December 2011 just as the second wave of HIP 

participants began earning incentives. A third-party processor (TPP) used by two of the four chain 

retailers operational at the time inadvertently removed the HIP transaction processing code when 

making an unrelated software change. As a result, HIP participants did not earn incentives for about 

45 days during the period from December 2011 to January 2012. This issue affected retailers with 11 

participating stores, four in Hampden County and seven outside of it. The TPP corrected the problem 

within a month of its detection, and HIP functionality was restored by mid-January. DTA 

subsequently credited the HIP incentives earned by the 1,140 households affected by the outage. 

The second technical challenge arose early in 2012 and involved two chain retailers’ failure to 

maintain accurate and updated computerized lists of HIP-eligible items. An ongoing task for IECRequipped retailers was to update their databases to maintain complete lists of HIP-eligible items. 

When new HIP-eligible food items were added to the retailers’ inventory, they had to be flagged in 

the database for the IECR system. HIP participants reported that they were not earning incentives 

when purchasing certain targeted fruits and vegetables. The issue was remedied fairly quickly; the last 

reported problem occurred in mid-February. DTA and Novo Dia Group worked with retailers during 

the remainder of the pilot to ensure that their product databases were routinely updated as new food 

items were added. 

A third technical issue arose when a chain retailer installed a new software release and inadvertently 

erased the HIP eligible product list. This occurred in early August 2012 and was not corrected until 

mid-October. While about 450 SNAP clients shopped in the stores during the outage, it is likely that 

only a few HIP participants were affected since this retailer carried a very limited number of HIPeligible products. 

A fourth technical issue was discovered at the end of September 2012 as HIP was being implemented 

in the four IECR-equipped retailers that joined HIP on October 1, 2012. All of the new IECR retailers 

used the same TPP, one that was also being used by two of the existing HIP chain retailers. It was the 

same TPP that had been involved in the December 2011 issue. This time, the HIP month-to-date 

(MTD) total did not appear on HIP clients’ receipts. However, this issue involved only the receipts 

and did not affect the actual HIP incentives earned. The receipts were accurate at the start of the pilot, 

but since no HIP participants called the toll-free line to report the incorrect receipts, it is not known 

exactly when the problem emerged. The problem was not corrected by the end of the pilot. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 3. HIP Implementation and Operations ▌pg. 61

Retailer Participation 

DTA recognized the importance of ensuring that HIP participants had easy and adequate access to 

retail stores in which they could earn incentives. DTA staff therefore spent considerable effort 

recruiting retailers, both chain stores and independent stores. As is true across the country, most 

SNAP benefits in Hampden County are redeemed in supermarket/superstore chain stores and DTA 

was able to recruit all but one of the major chains in Hampden County. However, the chain that 

declined to participate has a significant presence in Hampden County. DTA, with support from FNS 

and several community partners, made substantial efforts to recruit this retailer. In the end, the retailer 

and its TPP determined that they were unable to commit the resources necessary to meet pilot 

deadlines. Thus, participants’ access to supermarkets/superstores was somewhat limited. 

Participant Understanding of HIP 

As was discussed in the previous section, DTA put considerable effort into developing participant 

notification and training materials, working to design brochures and other information that were easy 

to understand and that fit within the evaluation design parameters. Because HIP was designed to test 

the effect of a financial incentive, materials needed to inform participants about the program but did 

not include nutrition education information. In addition, it was not possible to use explicit signage 

about HIP (e.g., signs on buses) that would be visible and possibly confusing to control group 

members. 

In an effort to improve participants’ understanding of which fruits and vegetables were HIP-eligible 

and to improve participation, DTA worked with retailer partners to develop signage that was installed 

in HIP retailers during August 2012 (10 months into the pilot). The signage was designed to highlight 

targeted fruits and vegetables. In developing the signage, DTA took care to strike a balance between 

signage that would be meaningful to HIP participants, while not confusing or contaminating the 

control group. Signage included shelf strips, produce stickers, freezer decals, and standees (standalone posters). Exhibit 3.4 shows examples of the signage. All of the participating chain retailers 

(except the convenience store chain) and most of the smaller retailers used some type of signage. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 62 ▌3. HIP Implementation and Operations Abt Associates

Exhibit 3.4: Sample Retailer Signage 

3.3 Discussion 

FNS and DTA recognized that recruiting retailers to participate in HIP was crucial to the success of 

the pilot. While DTA was successful in recruiting a range of stores to participate, not all households 

had similar access to stores that sold substantial quantities of fruits and vegetables, particularly 

supermarkets, superstores, and grocery stores. In particular, one chain supermarket/superstore with 

many stores in Hampden County did not participate despite substantial recruiting efforts. 

Additionally, several supermarkets and grocery stores did not begin participating until late in the 

pilot. This suggests that pilot impacts could be affected by retailer participation levels. 

DTA expended considerable resources developing training materials, providing training sessions, and 

mailing participant notifications. Those efforts appeared to have had mixed success. As noted above, 

few HIP participants availed themselves of training sessions. Many at DTA were not surprised by the 

low turnout, having had similar experiences with the rollout of other program changes. Few training 

sessions were offered in the evening or on weekends. Some stakeholders interviewed suggested that 

more participants might have attended if some trainings were held after work and school hours. 

In retrospect, some DTA staff interviewed suggested that simpler notification materials might have 

been more effective and that alternate means of communicating with participants, such as through 

social media, should be considered. Focus groups also revealed some literacy issues among HIP 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 3. HIP Implementation and Operations ▌pg. 63

participants, suggesting that more visual media may also be useful in increasing understanding and 

awareness of HIP. 

DTA also recognized that the participant notifications needed to be more “user-friendly.” Starting 

with the second interim mailing, DTA put considerable effort into ensuring that the materials were 

simple, colorful and easy to read. They were specifically designed to improve participants’ awareness 

and understanding of HIP. Calls to the HIP call line increased in response to these mailing as some 

participants wanted to clearly understand how the HIP incentive operated. 

Finally, the technical systems issues that occurred during the pilot may have led to confusion and 

misunderstanding about what stores were participating in HIP and what items earned incentives. They 

also may have affected the ability of participants to keep track of their HIP incentive earnings which 

might have led participants not to fully understand the value of the incentives. 

We return to participant understanding of HIP later in this report. Specifically, Chapter 5 uses the 

survey data to provide a more comprehensive picture of HIP participants’ understanding of the pilot 

and the analysis of HIP incentive earnings is discussed in Chapter 6. To maximize incentive earning, 

participants had to make sure they purchased targeted fruits and vegetables with their SNAP benefits, 

that they shopped in stores participating in HIP, and that they identified themselves as HIP 

participants when shopping at stores without integrated electronic cash registers so that their HIPeligible purchases could be totaled separately. It seems likely that incomplete understanding of HIP 

limited the impact of the pilot—better understanding might have increased purchases, incentives 

earned, and possibly the estimated impact of the program on targeted fruit and vegetable intake. 

These issues are discussed in later chapters. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 65

4. Retailer Experiences 

Because HIP can only operate with the cooperation of retailers, HIP’s effects on retailers’ business is 

critical to the long-run feasibility of the program. Implementing HIP required retailers to modify 

systems or establish check-out procedures to process HIP transactions. While costs to modify IECR 

systems were covered by the HIP grant, retailers expended time to learn about HIP and to train 

cashiers and other personnel. In addition, HIP could have affected store operations, including checkout and reconciliation procedures. HIP also had some potential to increase store sales of fruits and 

vegetables and SNAP redemptions in general. 

The analysis in this chapter shows that HIP had very little effect on retailers. Few retailers 

experienced problems relating to HIP implementation and the pilot had little effect on store 

operations. HIP generated a modest increase in total SNAP spending, principally in supermarkets and 

superstores. Stores that participated in HIP captured most of the additional spending, while HIP 

slightly reduced spending in non-participating stores. 

This chapter describes the experiences of retailers who participated in HIP, using data from two 

rounds of the retailer survey. A total of 83 retailer stores participated in HIP as of November 1, 2011 

and 130 stores participated as of October 1, 2012 (see Exhibit 3.1). The first survey (early 

implementation retailer survey) was conducted in the early period of HIP implementation, from 

October to December 2011, and the second (late implementation retailer survey) occurred near the 

end of the pilot, from November 2012 to January 2013. Thirty-nine retailers participated in the early 

implementation survey and 49 participated in the late implementation survey.44 Our results are 

primarily descriptive and weighted to represent the population of participating retailers in Hampden 

County by store type. Given that we surveyed participating retailers only, we cannot make any causal 

claims about the impacts of HIP on retailers. Results are presented overall and by store type.45 When 

examining changes, we tested for significant differences between the early and late implementation 

surveys using t-tests for continuous variables and chi-square tests for categorical variables.46 Survey 

findings were supplemented with findings from observations. Observations were conducted at 10 

retailers each during early and late HIP implementation.47 

 

44 Somewhat different survey procedures were used for independent stores and chain stores. Some questions 

that were not relevant to chain stores were only asked of independent stores. Information that was 

presumed to be the same across all retailers of the same chain was asked of the corporate office and not the 

individual retailers. 

45 We collapsed the 12 FNS official stores types eligible for HIP into four categories: 

supermarkets/superstores; grocery stores (small, medium, and large grocery stores and fruit/vegetables, 

seafood, and meat specialty stores); convenience stores (convenience store and combination grocery/other); 

and farmers markets (including direct marketing farmers). Farmers markets were not operating during the 

periods the retailer survey was conducted and thus are not included in the survey samples. 

46 Only 26 retailers completed both surveys, thus unpaired tests were conducted to take advantage of the full 

sample of retailers when making comparisons between rounds. Significant changes are mentioned in the 

text and tables are shown in Appendix F. 

47 Observations were also conducted at seven of these retailers at the midpoint between early and late HIP 

implementation. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 66 ▌4. Retailer Experiences Abt Associates

The first section of this chapter examines the food retail environment and any changes that occurred 

during the course of the pilot. The second section presents results related to retailer satisfaction with 

HIP and the third section uses EBT data to examine the effect of HIP on overall retailer redemptions. 

4.1 Changes in Food Retail Environment 

This section examines changes in the food retail environment of participating retailers, including the 

promotion of fruits and vegetables, self-reported “effects” of HIP on check-out time, sales, profit, 

product stocking, and changes in fruit and vegetable availability and prices. 

Activities to Promote and Sell Fruits and Vegetables 

To assess the types of fruit and vegetable promotion used by retailers and any changes in promotion 

activities, we asked retailers whether and how they promoted fruits and vegetables at both survey 

rounds. Exhibit 4.1 shows promotion activities overall and by store type as reported in the late 

implementation survey.48 Findings differed by store type, with supermarkets/superstores reporting 

more frequent and more varied promotion activities than the other two store types. 

Supermarkets/Superstores reported using a variety of activities to promote fruits and vegetables. 

Most stores (50-70 percent) reported using most of the different methods, from posters and signs to 

recipes and fliers, advertisements, and price or volume discounts, at least once a month at late 

implementation. Just under half of the supermarkets/superstores reported using coupons. There is no 

suggestion that supermarkets/superstores’ use of these types of promotion changed over the course of 

the pilot. This is not surprising given that the scale of the HIP intervention was modest relative to 

their overall operations. 

Fruit and vegetable promotion activities were less widely used in grocery stores. However, 30-40 

percent of grocery stores reported using signs inside the store and shelf tags to promote fruits and 

vegetables at late implementation. These are the types of signage developed by DTA for use by HIP 

participating retailers. The survey showed some evidence that these activities changed over the course 

of the pilot. Most grocery stores did not use the remaining promotion activities other than price or 

volume promotions for fruits and vegetables. 

Fruit and vegetable promotion activities were also not very common among convenience stores. This 

is not surprising given that most of these stores carry only limited offerings of fruits and vegetables. 

Similar to the findings about promotion activities in grocery stores, 45-50 percent of convenience 

stores reported using posters, signs, and shelf tags at late implementation. There is some suggestion 

that convenience stores changed the frequency with which they engaged in these promotion activities, 

with some increases over the pilot period. A minority of convenience stores engaged in some price or 

volume promotions and almost none used any other types of promotions. 

 

48 Appendix F, Exhibit F4.1 presents results for both the early and late implementation retailer surveys and 

shows changes between the two survey rounds. In this section, we only discuss changes between the two 

rounds that were statistically significant. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 67

Exhibit 4.1: Retailer Promotion of Fruits and Vegetables, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period 

Activities used to promote fruits and 

vegetables once a month or more Overall 

(%) 

Supermarket

/Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

Posters or signs inside the store 54.4 72.0 30.0 51.2 

Shelf tags 53.1 72.0 37.5 45.0 

Posters or signs in store window or 

outside 45.1 57.6 12.5 46.0 

Fliers/ads in newspaper or direct mail 25.5 69.7 0.0 0.0 

Price or volume promotions 24.4 53.8 0.0 9.9 

Food samples 21.3 62.2 0.0 0.0 

Recipes or fliers in store 19.3 56.3 0.0 0.0 

Coupons 17.7 43.7 0.0 4.3 

Unweighted number of storesa 49 14 12 23 

Weighted percents. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

Observation data collected in 10 stores during December 2012 were consistent with survey data 

presented in Exhibit 4.1 and revealed that about half or more of the observed retailers had shelf tags 

or signs promoting fresh fruits and vegetables. Signage for canned and frozen fruits and vegetables 

was less common, but by late HIP implementation, 5 of the 10 observed stores had shelf tags 

promoting canned fruits and vegetables and 4 had small signs promoting frozen fruits and vegetables. 

While farmers markets were closed by the time the late implementation retailer survey was 

conducted, we conducted observations in three farmers markets during the summer. Across the three 

markets, about half of all vendors displayed HIP signs. 

In addition to the activities to promote fruits and vegetables described in Exhibit 4.1, retailers were 

also asked about any activities they had engaged in to sell more fruits and vegetables (Exhibit 4.2). 

Findings differed by store type. Supermarkets/superstores largely did not engage in activities to 

increase their sales of fruits and vegetables (or the survey respondents did not know about these 

activities). In contrast, more than half of the grocery stores received more shipments from a supplier, 

increased the frequency of restocking the display floor,49 increased shelf space, and changed where 

food items were located in the store in order to sell more fruits and vegetables. Although just over 

one-quarter of convenience stores increased the frequency of restocking the display floor, increased 

shelf space, and changed where food items were located, most did not engage in activities to increase 

their sales of fruits and vegetables (or respondents did not know about these activities). 

 

49 The question did not specify restocking of HIP-eligible items. Respondents reported efforts to increase 

sales of fruits and vegetables generally. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 68 ▌4. Retailer Experiences Abt Associates

Exhibit 4.2: Retailer Activities to Sell More Fruits and Vegetables, Overall and by Store 

Type: Retailer Survey, Late HIP Implementation Period 

Have you done any of the following since 

November 2011 so that you can sell more 

fruits and vegetables? 

Overall 

(%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

Increased frequency of restocking display floor 

Yes 31.6 15.0 58.3 27.2 

No 33.7 68.7 41.7 17.0 

Don't know 34.7 16.3 0.0 55.9 

Changed where food items are located in store or on shelves 

Yes 30.7 7.5 66.7 25.4 

No 31.5 46.3 33.3 25.4 

Don't know 37.8 46.3 0.0 49.1 

Increased shelf space 

Yes 30.3 0.0 63.6 29.6 

No 36.9 83.7 36.4 19.7 

Don't know 32.8 16.3 0.0 50.7 

Received more shipments from a supplier 

Yes 19.6 0.0 54.5 14.1 

No 35.6 53.7 36.4 28.1 

Don't know 44.7 46.3 9.1 57.8 

Installed new refrigeration or freezer units for 

storage or display 

Yes 15.4 0.0 36.4 13.6 

No 51.1 83.7 63.6 34.0 

Don't know 33.5 16.3 0.0 52.5 

Started working with a new supplier 

Yes 10.0 0.0 40.0 3.5 

No 53.2 83.7 60.0 38.7 

Don't know 36.8 16.3 0.0 57.8 

Unweighted number of stores a 49 14 12 23 

Weighted percents. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

Self-Reported “Effects” of HIP on Check-Out Time, Sales, and Profits 

Next, we assessed the perceived impact of HIP on check-out time, sales, and profits. We expected 

minimal change among supermarkets/superstores, since processes were automated with IECRs and 

sales of fruits and vegetables are common. However, most grocery stores and convenience stores 

were smaller, did not have IECRs, and were more limited in their sales of fruits and vegetables. Thus 

potential effects of HIP on check-out time, sales, and profits were more plausible in these types of 

stores. 

Exhibit 4.3 shows participating retailers’ assessment of the effects of HIP on their time and effort 

associated with HIP. The majority of respondents in all retailer types reported that HIP resulted in no 

change in any of these elements of operational cost. (While one-third of supermarkets/superstores 

indicated a large increase in reconciliation time and effort due to HIP, this represents just one retailer 

that joined HIP in October 2012.) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 69

Exhibit 4.3: Self-Reported Effects of HIP on Time and Effort, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period 

Overall 

(%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

HIP's effect on average check-out time 

Increase 8.9 0.0 16.7 5.9 

Decrease 0.0 0.0 0.0 0.0 

No change 91.1 100.0 83.3 94.1 

HIP's effect on settlement time and effort 

Increase 11.5 18.7 16.7 5.9 

Decrease 0.0 0.0 0.0 0.0 

No change 88.5 81.3 83.3 94.1 

HIP's effect on reconciliation time and effort 

Increase 9.0 31.6 8.3 5.9 

Decrease 0.0 0.0 0.0 0.0 

No change 91.0 68.4 91.7 94.1 

HIP's effect on return time and effort 

Increase 8.9 0.0 8.3 11.8 

Decrease 0.0 0.0 0.0 0.0 

No change 91.1 100.0 91.7 88.2 

Unweighted number of stores a 32 3 12 17 

Weighted percents. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey (Independent stores only). 

Exhibit 4.4 shows retailers’ self-reports of how HIP affected their sales of fruits and vegetables and 

their store profits. Overall, almost half of all participating stores reported increased sales of fruits and 

vegetables. All supermarkets/superstores and 67 percent of grocery stores reported increased sales. In 

contrast, few convenience stores reported any change in their fruit and vegetable sales, which is not 

surprising given the limited number of such items they stocked. 

Most retailers reported that HIP did not affect their stores’ profits (or they did not know). This result 

is to be expected since only a small proportion of Hampden County SNAP households participated in 

HIP. However, just over 40 percent of grocery stores (five stores), reported increased profits as a 

result of HIP. It is difficult to know what to make of this finding since EBT data (presented in Section 

4.3) show small effects of HIP on retailer redemptions. Recall, however, that a substantial proportion 

of grocery stores reported activities to increase the sale of fruits and vegetables. Perhaps this resulted 

in increased profits for some grocery stores. Alternatively, it may be coincidental that these retailers 

experienced increased profits during the pilot. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 70 ▌4. Retailer Experiences Abt Associates

Exhibit 4.4: Self-Reported Effects of HIP on Sales and Profits, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period 

Overall 

(%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

How has HIP affected your store's sales of fruits and vegetables 

Large increase in sales 7.8 0.0 16.7 6.4 

Small increase in sales 36.8 100.0 50.0 15.9 

No change 53.5 0.0 25.0 77.7 

Small decrease in sales 1.9 0.0 8.3 0.0 

Large decrease in sales 0.0 0.0 0.0 0.0 

How has HIP affected your store's profits

HIP increased profits 15.8 0.0 41.7 9.5 

No difference 74.9 62.0 50.0 87.3 

HIP decreased profits 0.0 0.0 0.0 0.0 

Don't know 9.3 38.0 8.3 3.2 

Unweighted number of stores a 49 14 12 23 

Weighted percents. 

a

 N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

Overall, it is encouraging to see that most retailers reported no increase in time and effort associated 

with HIP. Supermarkets/superstores and convenience stores reported relatively stable sales. However, 

grocery stores appeared to have increased their activities to sell more fruits and vegetables, and in 

turn reported selling more fruits and vegetables. 

Changes in Fruit and Vegetable Stocking, Availability, and Prices 

To address the research question of whether HIP resulted in changes in fruit and vegetable stocking, 

availability, and prices, retailers were asked about their stocking of fruits and vegetables late in the 

HIP implementation period compared to before HIP. They also completed a fruit and vegetable 

inventory based on a pre-defined list of selected fresh, canned, and frozen fruits and vegetables in 

which they indicated what was available in their store and the corresponding prices.50 As with the 

increased availability of healthy foods that accompanied the revised WIC food packages (Andreyeva, 

Long, and Brownell et al., 2012), it was plausible that we might see an increase in retailers’ 

availability of fruits and vegetables with the implementation of HIP. We did not expect to see price 

impacts on supermarkets/superstores where prices are driven by the broad market. Grocery and 

convenience stores, which tended to be smaller retailers, however, might have increased prices in 

response to the incentive, or decreased prices if there was an increased demand allowing them to 

purchase larger quantities at better prices. 

 

50 In order to calculate a price per pound for each fruit and vegetable, so that data were comparable across 

retailers, it was necessary to estimate the weight of different units of fruits and vegetables. Missing and 

illogical units were coded as the modal unit if the price that corresponded to the missing unit was in the 

range of prices covered by the modal unit. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 71

In general, the majority of respondents in all retailer types reported that they stocked the same amount 

of fruits and vegetables since the start of HIP in November 2011 (Exhibit 4.5).51 However, grocery 

stores often reported that they stocked more fruits and vegetables since the start of HIP; 60-70 percent 

of grocery stores reported stocking more canned fruits and fresh vegetables, and just under half 

reported stocking more fresh fruits and canned vegetables (Exhibit 4.5). Earlier in the HIP 

implementation process, observations revealed one grocery store and three convenience stores that 

had poor stocking of displays of fresh, canned, dried, or frozen fruits and vegetables, but during the 

later HIP implementation period, these stores were all observed to have adequate stocking of displays. 

Grocery stores that stocked more fruits and vegetables said they did so for a variety of reasons—most 

commonly because the store had more customers and they wanted to promote fruits and vegetables 

(Exhibit 4.6). 

The fruit and vegetable inventory revealed, not surprisingly, that almost all supermarkets/superstores 

carried a wide variety of fresh, canned, and frozen fruits and vegetables.52 Grocery store inventories 

were less extensive, though most of the fresh and canned fruits and vegetables in the inventory were 

carried by over half of the grocery stores. Frozen fruits and vegetables were much less common, 

however; one-third or fewer grocery stores carried each of the frozen fruits and vegetables on the 

inventory. Also, as expected, convenience stores carried relatively few fruits and vegetables; when 

they did carry fruits and vegetables, they were primarily fresh fruits. 

Observations similarly revealed a range in the variety of fruits and vegetables available across the 

observed retailers, from 0 to 11 or more types each of fresh, canned, dried, or frozen fruits and 

vegetables in a given store. As in the survey, observed supermarkets/superstores typically carried 

more fruits and vegetables than did grocery and convenience stores. During the late implementation 

period, all supermarkets/superstores were observed to have fresh, canned, dried, or frozen fruits and 

vegetables. All grocery stores carried canned fruits and vegetables, but only one of the three carried 

fresh, and none carried frozen fruits and vegetables. All observed convenience stores carried fresh 

and canned fruits and vegetables, but only one out of three carried frozen fruits and vegetables. Also 

similar to the surveys, observations indicated that supermarkets/superstores had more types of fresh, 

canned, dried, and frozen fruits and vegetables than did the observed grocery and convenience stores. 

While the survey analysis revealed a few statistically significant changes in the availability and prices 

of fruits and vegetables over the course of the pilot, given the number of tests conducted, this was no 

more than would be expected by chance (see Appendix F, Exhibit F4.3). 

 

51 One-third of supermarkets/superstores indicated that they stocked more fresh fruits and vegetables and 

more canned and frozen fruits since the start of HIP. However, due to missing data, this percentage refers to 

only one store. This supermarket/superstore reported increasing their stocks of these foods because their 

store had different customers (Exhibit 4.6). We do not have any evidence as to whether the new customers 

were HIP participants or whether the reported increase was a routine change in their customer base. 

52 See Appendix F, Exhibits F4.2-F4.4. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 72 ▌4. Retailer Experiences Abt Associates

Exhibit 4.5: Self-Reported Effect of HIP on Product Stocking, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period 

Since November 2011, does your store 

stock more, the same, or less of...? 

Overall 

(%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

Fruits 

Fresh 

Stock more 26.4 34.2 45.5 17.6 

Stock same 73.6 65.8 54.5 82.4 

Stock less 0.0 0.0 0.0 0.0 

Canned 

Stock more 36.6 40.6 70.0 25.4 

Stock same 63.4 59.4 30.0 74.6 

Stock less 0.0 0.0 0.0 0.0 

Frozen 

Stock more 11.8 34.2 25.0 3.6 

Stock same 85.7 65.8 75.0 92.7 

Stock less 2.5 0.0 0.0 3.6 

Dried 

Stock more 5.0 0.0 0.0 7.6 

Stock same 92.5 100.0 100.0 88.7 

Stock less 2.5 0.0 0.0 3.8 

Vegetables 

Fresh 

Stock more 29.8 40.6 60.0 17.6 

Stock same 70.2 59.4 40.0 82.4 

Stock less 0.0 0.0 0.0 0.0 

Canned 

Stock more 21.7 0.0 40.0 19.1 

Stock same 76.1 100.0 60.0 77.7 

Stock less 2.2 0.0 0.0 3.2 

Frozen 

Stock more 12.2 0.0 37.5 7.3 

Stock same 85.3 100.0 62.5 89.1 

Stock less 2.5 0.0 0.0 3.6 

Unweighted number of stores a 49 14 12 23 

Weighted percents. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 73

Exhibit 4.6: Retailer Reported Reasons for Stocking More Fruits or Vegetables, Overall and 

by Store Type: Retailer Survey, Late HIP Implementation Period 

For the foods that your store stocks more 

of, why did this happen?a Overall (%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

I/We want to promote fruits and vegetables 50.4 0.0 44.4 66.7 

My store has more customers 35.2 0.0 44.4 33.3 

My customers want more fruits and vegetables 20.2 0.0 22.2 22.2 

My store has different customers 19.4 100.0 11.1 11.1 

Other reason 5.1 0.0 0.0 11.1 

Missing 5.0 0.0 11.1 0.0 

Unweighted number of stores b 19 1 9 9 

Weighted percents. 

a Respondents could report multiple reasons so percents total to more than 100. 

b

 N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

Overall, the retailer survey suggested some self-reported increased stocking of fruits and vegetables 

and a few increases and decreases in the availability and prices of canned fruits and vegetables among 

grocery and convenience stores over time. However, unlike the increased availability of healthy foods 

associated with the revised WIC food packages, it does not appear that retailers systematically altered 

their fruit and vegetable inventory or prices over the course of the pilot. 

4.2 Retailer Satisfaction 

This section broadly discusses retailer satisfaction by examining issues with implementing and 

participating in the pilot. Specifically, the section examines retailer understanding and support for the 

objectives of HIP, problems and issues encountered during the pilot, and overall satisfaction. 

Understanding of HIP Objectives, Problems, and Questions Encountered 

Most retailers, across all store types, reported that they understood the purpose of HIP and how it was 

supposed to work (Exhibit 4.7). They also agreed that it was important to improve the choices people 

make when buying foods using SNAP, suggesting retailer buy-in of HIP. 

The survey asked retailers about their perspectives on several specific HIP implementation issues, 

such as training workers, processing HIP purchases, and store payments. Across all retailer types, 

only a minority reported that HIP had presented difficulties or resulted in additional burden (Exhibit 

4.7). 

Less than 20 percent of retailers reported that training workers had been a burden or that HIP 

purchases had been hard to process. Six of the eight retailers that indicated HIP purchases had been 

hard to process were later implementing stores, suggesting processing HIP purchases may become 

easier with experience. Retailers also had different perceptions on these issues depending on the type 

of store they managed. Half of all grocery store managers reported that it was a burden to train store 

employees and about one-third found HIP purchases difficult to process. Few of the sampled grocery 

stores used IECRs. Thus, compared with processing other SNAP purchases, processing HIP 

purchases required an additional step—separating and entering a separate subtotal for HIP-eligible 

fruits and vegetables. Cashiers had to be trained in this new process. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 74 ▌4. Retailer Experiences Abt Associates

Not surprisingly, supermarket/superstore and convenience store managers did not report these types 

of difficulties. Since all supermarkets/superstores participating in HIP used IECRs, processing HIP 

purchases was no different than processing other SNAP purchases and did not require cashiers to be 

trained in new procedures. Some convenience stores used IECRs, which minimized the burden of 

these activities. In general, convenience stores carried few HIP-eligible items, which likely minimized 

the burden of the additional HIP processing requirements. 

Mock transactions conducted as a part of the observations in both IECR and non-IECR stores 

revealed that most check-out supervisors followed the correct steps, although there were a few 

questions and some cases of confusion about separating HIP items. 

Overall, 10 percent of retailers reported problems receiving accurate and timely payments for HIP 

purchases. Supermarkets/superstores were more likely to report this problem than other retailer types. 

However, these supermarkets/superstores represented only one chain retailer with several retail 

locations. (We do not have additional information about the problems they experienced, and none of 

the interview respondents or project documents identified any issues with timely settlement related to 

HIP.) Aside from this supermarket/superstore chain, only one grocery store and one later 

implementing convenience store reported these problems. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 75

Exhibit 4.7: Retailer Understanding and Perceptions of HIP, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period 

How much do you agree or disagree...? 

Overall 

(%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

I understand the purpose of HIP 

Agree 92.7 92.5 91.7 93.2 

Neither agree nor disagree 3.8 0.0 8.3 3.4 

Disagree 3.5 7.5 0.0 3.4 

Don't know 0.0 0.0 0.0 0.0 

I understand how HIP is supposed to work 

Agree 94.5 92.5 91.7 96.5 

Neither agree nor disagree 0.0 0.0 0.0 0.0 

Disagree 5.5 7.5 8.3 3.5 

Don't know 0.0 0.0 0.0 0.0 

It is important to improve choices people make when buying foods with SNAP 

Agree 92.6 100.0 91.7 90.1 

Neither agree nor disagree 3.7 0.0 8.3 3.3 

Disagree 3.7 0.0 0.0 6.6 

Don't know 0.0 0.0 0.0 0.0 

Training workers has been a burden 

Agree 17.0 0.0 54.5 9.9 

Neither agree nor disagree 36.3 23.8 9.1 50.7 

Disagree 44.8 76.2 36.4 36.1 

Don't know 1.9 0.0 0.0 3.3 

HIP purchases have been hard to process 

Agree 15.4 7.5 30.0 13.6 

Neither agree nor disagree 34.1 16.3 10.0 49.1 

Disagree 50.5 76.2 60.0 37.4 

Don't know 0.0 0.0 0.0 0.0 

My store is paid on time for HIP purchases 

Agree 53.3 62.5 90.9 35.1 

Neither agree nor disagree 3.6 7.5 0.0 3.5 

Disagree 10.6 30.0 9.1 3.5 

Don't know 32.5 0.0 0.0 57.8 

Payments to my store are accurate 

Agree 48.3 62.5 72.7 32.8 

Neither agree nor disagree 3.7 7.5 0.0 3.6 

Disagree 10.9 30.0 9.1 3.6 

Don't know 37.1 0.0 18.2 59.9 

Unweighted number of stores a 49 14 12 23 

Weighted percents. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

Grocery store and convenience store managers (lacking IECRs) were also asked about specific 

problems encountered processing HIP purchases. (Supermarkets/superstores and other stores with 

IECRs were not asked about these problems as they likely would not have arisen with automated 

check-out procedures.) Overall, most grocery and convenience store managers reported experiencing 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 76 ▌4. Retailer Experiences Abt Associates

few problems with HIP in the previous three months (Exhibit 4.8), though several problems were 

noted by up to 30 percent of stores as having occurred at least a few times. These problems included: 

 Identifying HIP customers (25 percent of grocery stores; 29 percent of convenience stores) 

 Knowing what foods were HIP-eligible (8 percent of grocery stores;53 29 percent of 

convenience stores) 

 Separating HIP-eligible items from other items (8 percent of grocery stores;54 19 percent of 

convenience stores). 

All other problems were reported infrequently. Almost all retailers reported that the problems they 

experienced were resolved (see Appendix F, Exhibit F4.5 for detailed responses). 

Exhibit 4.8: Retailer Problems with HIP in the Previous Three Months, by Grocery and 

Convenience Stores: Retailer Survey, Late HIP Implementation Period 

Experienced problem at least a few times in the previous 3 months 

Grocery 

(%) 

Convenience

(%) 

Problems identifying HIP customers 25.0 29.4 

Problems knowing what foods are HIP-eligible 8.3 29.4 

Problems separating HIP-eligible from non-HIP-eligible foods 8.3 18.8 

Problems processing returns with HIP items 8.3 0.0 

Problems having current list of HIP-eligible items in cash registers 0.0 6.3 

Problems processing sales of HIP items 0.0 5.9 

Problems computing the purchase amount for HIP items 0.0 0.0 

Problems responding to customer questions about HIP 0.0 0.0 

Problems processing manual vouchers with HIP items 0.0 0.0 

Problems getting information about SNAP/EBT sales and settlement 0.0 0.0 

Unweighted number of stores a 12 17 

Weighted percents. 

a N varies by question due to item non-response and skip patterns. 

Source: Late Implementation Retailer Survey (Independent stores only). 

As another indicator of how well retailers understood how HIP operated, the survey asked retailers 

about questions they or their employees asked about HIP. Across all retailer types, a relatively small 

minority (about 15-30 percent) reported asking or being asked questions about HIP (see Appendix F, 

Exhibits F4.6 and F4.7). Fewer supermarket/superstore managers reported questions than did 

managers of grocery stores and convenience stores, likely because HIP did not require 

supermarkets/superstores to alter check-out procedures. Most questions focused on knowing what 

foods were HIP-eligible, being able to separate the items, and knowing how to identify HIP 

customers. These are the same areas that retailers reported as problems with HIP. 

The survey also asked retailers about questions they had received from customers in the previous 

three months (Exhibit 4.9). About one-third of all retailers reported that customers had asked them 

questions, though this varied by store type. Store managers in the majority (55–60 percent) of 

 

53 Only one grocery store indicated problems knowing what foods were HIP-eligible a few times or more. 

54 Only one grocery store indicated problems separating HIP-eligible items a few times or more. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 77

supermarkets/superstores and grocery stores reported that they had received questions about HIP from 

program participants at least once during the prior three months; most (80 percent) convenience store 

managers responded that they had not been asked any questions. 

The most common questions concerned identifying HIP-eligible foods; some customers also asked 

questions about the EBT credit and how to read the grocery receipt. Some retailers also mentioned 

customer questions about how to use HIP and whether all stores participated. Most stores did not 

receive questions about HIP from SNAP households not participating in the pilot. 

Exhibit 4.9: Retailer Perceptions of Customer Questions about HIP, Overall and by Store 

Type: Retailer Survey, Late HIP Implementation Period 

Customer questions about HIP in prior 3 

months 

Overall 

(%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

How often did HIP customers ask questions about HIP? 

Never 62.0 45.5 41.7 79.6 

Less than once a week 29.9 54.5 25.0 17.2 

Once a week 3.2 0.0 8.3 3.2 

More than once a week 4.8 0.0 25.0 0.0 

What are the most common questions HIP customers asked about HIP? a

Qs about eligible food 63.3 34.7 85.7 84.4 

Qs about EBT credit 33.8 38.9 28.6 31.2 

Qs about reading receipt 31.9 44.4 28.6 15.6 

Other Qs 8.5 0.0 14.3 15.6 

How often did SNAP, non-HIP, customers ask questions about HIP? 

Never 81.7 59.5 91.7 90.5 

Less than once a week 15.1 40.5 0.0 6.4 

Once a week 0.0 0.0 0.0 0.0 

More than once a week 3.3 0.0 8.3 3.2 

Have there been questions about HIP that you didn't know how to respond to? 

Yes 7.3 13.6 0.0 6.4 

No 92.7 86.4 100.0 93.6 

(If “yes”) Who did you refer them to? 

Did not refer them to anyone 50.0 50.0 0.0 50.0 

Local DTA 0.0 0.0 0.0 0.0 

DTA hotline 23.0 50.0 0.0 0.0 

Xerox 0.0 0.0 0.0 0.0 

SNAP program 27.0 0.0 0.0 50.0 

Unweighted number of stores b 49 14 12 23 

Weighted percents. 

a Respondents selected all options that applied; percents therefore sum to more than 100. b N varies by question due to item non-response and skip patterns. 

Source: Late Implementation Retailer Survey. 

Satisfaction With Ability To Implement and Participate in the Pilot 

Retailers were asked about their overall satisfaction with HIP. This section also reports on the stores 

that joined HIP after the pilot began and the one store that withdrew after initially participating. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 78 ▌4. Retailer Experiences Abt Associates

Overall Satisfaction 

To assess retailer satisfaction with HIP, retailers were asked about their level of satisfaction with HIP 

and if they would still join HIP if they had it to do over again (Exhibit 4.10). Overall, a majority of 

retailers reported that they were somewhat or very satisfied with HIP, though satisfaction varied 

depending on the store type. Over 90 percent of supermarkets/superstores and 83 percent of grocery 

stores were somewhat or very satisfied. In contrast, only 32 percent of convenience stores were 

somewhat or very satisfied and 10 percent (3 stores) were somewhat or very dissatisfied. This finding 

is likely related to the fact that convenience stores carried relatively few HIP-eligible items and thus 

the stores had little to gain from the pilot. When asked how HIP might be improved, one of the 

dissatisfied convenience stores did not think HIP was necessary, and another suggested more 

advertisement might improve the program. 

When asked whether they would still join HIP if they had it to do again, all supermarket/superstore 

and grocery stores and 91 percent of convenience stores responded that they would. 

Exhibit 4.10: Retailer Satisfaction with HIP, Overall and by Store Type: Retailer Survey, Late 

HIP Implementation Period 

Overall 

(%) 

Supermarket/

Superstore 

(%) 

Grocery 

(%) 

Convenience

(%) 

Overall satisfaction with HIP 

Very satisfied 34.8 62.5 58.3 15.9 

Somewhat satisfied 20.8 30.0 25.0 15.9 

Neither satisfied or dissatisfied 38.9 7.5 16.7 58.7 

Somewhat dissatisfied 1.8 0.0 0.0 3.2 

Very dissatisfied 3.7 0.0 0.0 6.4 

If you had it to do again, would you still join HIP? 

Yes 94.5 100.0 100.0 90.5 

No 5.5 0.0 0.0 9.5 

Unweighted number of stores a 49 14 12 23 

Weighted percents. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

Retailers with Limited Length of Participation 

After the launch of the pilot, an additional 12 retail stores joined HIP in February 2012 or later. As 

discussed in Chapter 3, several smaller grocery and convenience stores began participating in 

February 2012 and some larger grocery stores joined HIP in October 2012. The most common reason 

stores did not join HIP originally was that they did not know their store could be a part of HIP 

(Exhibit 4.11). Half of the stores indicated this was the case, which appears to be due to 

miscommunication, and not dissatisfaction with HIP. Some reasons given for not originally joining 

that may reflect initial dissatisfaction included perceptions that joining HIP would have been too 

difficult (indicated by one grocery store) and that not enough support was given (indicated by one 

supermarket/superstore and one grocery store). One supermarket/superstore and one convenience 

store also felt that there was not enough time to get ready and one convenience store also indicated 

that their store would need to stock more fruits and vegetables, suggesting some anticipated problems 

and difficulties associated with HIP. The initial dissatisfaction and anticipated problems seem to have 

been mitigated as these retailers decided to join HIP for a variety of reasons (see Exhibit 4.12). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 79

As noted above, most retailers were satisfied with HIP (or neither satisfied nor dissatisfied). As a 

result, there was almost no retailer attrition. Only one small convenience store withdrew from HIP 

after participating for about eight months. This store carried a limited number of HIP-eligible items, 

and decided to concentrate on the hot prepared food side of their business; they therefore determined 

that participating in HIP was not worth it for them, although they were not necessarily dissatisfied 

with HIP. 

Exhibit 4.11: Reasons Later Implementing Stores Did Not Join HIP at the Start, Overall and 

by Store Type: Retailer Survey, Late HIP Implementation Period 

Why did your store not join HIP when it 

started in Fall 2011? a Overall (N) 

Supermarket/

Superstore 

(N) 

Grocery 

(N) 

Convenience

(N) 

I didn't know that the store could be part of HIP 6 1 2 3 

Previous owner made the decision 3 0 0 3 

There was not enough time to get ready 2 1 0 1 

I did not get enough support 2 1 1 

The store would need to stock more fruits and 

vegetables 1 0 0 1 

Joining HIP would have been too difficult 1 0 1 0 

I thought that HIP would not increase sales 0 0 0 0 

I thought that HIP would increase costs 0 0 0 0 

I did not want to be part of a demonstration 0 0 0 0 

I knew other retailers who decided not to join 

HIP 0 0 0 0 

Unweighted number of stores b 12 3 2 7 

Unweighted counts. 

a Respondents selected all options that applied; counts therefore may sum to more than the total number of 

retailers.

b N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 80 ▌4. Retailer Experiences Abt Associates

Exhibit 4.12: Reasons Later Implementing Stores Joined HIP, Overall and by Store Type: 

Retailer Survey, Late HIP Implementation Period 

Why did your store join HIP this year a Overall 

(N) 

Supermarket/

Superstore 

(N) 

Grocery 

(N) 

Convenience

(N) 

New manager or owner who decided to join 5 0 2 3 

The State, DTA or another organization asked 

us 

4 2 0 2 

HIP could increase sales of fruits and 

vegetables 4 0 0 4 

HIP could increase sales of other items 3 0 0 3 

This year we had time to sign up and get ready 2 1 0 1 

New equipment made HIP easier/possible 2 1 0 1 

I talked to other retailers who are participating 1 1 0 0 

My store was losing sales to stores that are 

participating 0 0 0 0 

It was easier to do than before 0 0 0 0 

Unweighted number of stores b 12 3 2 7 

Unweighted counts. 

a Respondents selected all options that applied; counts therefore may sum to more than the total number of 

retailers.

b N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

4.3 Impact of HIP on Retailer Redemptions 

EBT transaction data provide some evidence about HIP impacts on retailers. According to our 

conceptual model, HIP may impact households’ decisions about where to shop and what to 

purchase.55 These decisions could have implications for retailer sales in Hampden County. 

As discussed in this section, we found that the effects of HIP on retailer sales during the pilot period 

likely were quite small. There are two ways that HIP could affect revenue for retailers: 

 HIP could increase total SNAP redemptions, due to the value of the incentives earned by 

SNAP participants; this increase is essentially a positive-sum game, increasing the potential 

pool of SNAP spending for all retailers in the county. 

 HIP could generate shifts in spending from one retailer type to another, perhaps due to the 

incentive that HIP provides to shop especially in participating retailers; these shifts are 

essentially a zero-sum game, because any gains for one retailer type must necessarily come at 

the expense of revenue for another retailer type. 

While HIP did increase SNAP redemptions, HIP purchases of eligible TFVs were modest in 

magnitude, which meant that these additional incentives were also fairly small (Exhibit 4.13). NonHIP participant households had mean monthly SNAP purchases of $259.81. HIP households had 

mean monthly SNAP purchases of $264.11, which included the incentives they earned on their TFV 

purchases. The difference in SNAP redemptions between HIP and non-HIP households was thus 

$4.30. 

 

55 Chapter 6 examines this issue in detail. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. RetaExhibit 4.13: Monthly SNAP Purchases per Household and Projected in Aggregate for Hampden County 

Per householda Projected for 55,095 households i

HIP ($) Non-HIP ($) Difference ($) With HIP ($) Without HIP ($) 

All retailers 264.11 259.81 4.30 14,551,140 14,314,232 

Supermarket/superstores 210.98 206.29 4.69 11,623,943 11,365,548 Convenience 26.18 26.13 0.05 1,442,249 1,439,506 Grocery 20.23 19.86 0.37 1,114,324 1,093,928 Other 2.38 2.48 -0.10 131,231 136,592 Out of State 4.35 5.06 -0.71 239,757 278,852 

Participating retailer 136.80 132.38 4.41 7,536,996 7,293,476 Non-participating retailer 127.32 127.43 -0.11 7,014,695 7,020,756 

a Source: EBT Transaction Data, pooled across March-October 2012. See Chapter 6, Exhibit 6.4 for the underlying EBT analysis. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 82 ▌4. Retailer Experiences Abt Associates

HIP also generated some shifts in spending across retailers. Relative to non-HIP participants, HIP 

participants spent more of their benefits in stores that participated in HIP, but the change was again 

small. Most of the change in shopping behavior induced by HIP arose from a modest increase in 

spending at participating supermarkets/superstores. 

We analyzed how much impact these changes in shopping behavior at the household level could have 

for the retail sector in Hampden County more generally. Taking the mean shopping amounts for nonHIP households as representative of county-wide shopping patterns in the absence of the incentive, 

and taking the mean shopping amounts for HIP households as representative of the new shopping 

behavior with the incentive, we estimated total dollar spending levels for the 55,095 households in 

Hampden County (the size of the SNAP caseload at the start of the pilot). With the new incentive 

program, projected monthly SNAP spending in all retailers would have a net increase of $236,909, or 

about 1.7 percent (Exhibit 4.13). If retailer participation patterns were the same as in the pilot, there 

would be a decrease in monthly SNAP spending of $6,060 (0.1 percent) in non-participating stores, 

which would be more than offset by an increase in monthly SNAP spending of $243,520 (3.3 percent) 

in participating stores. The increased SNAP spending would be felt mainly by 

supermarkets/superstores and grocery stores. Combining participating and non-participating stores, 

supermarkets/superstores would see a monthly increase of $258,396 (2.3 percent), and redemptions at 

grocery stores would increase by $20,396 (1.9 percent). This analysis shows that the main impact of 

HIP on retailers was a modest overall increase in SNAP redemptions, with little shifting of household 

spending from one retailer type to another. The pilot thus represented a positive-sum game for 

retailers in the county. 

4.4 Discussion 

HIP appears to have had limited effects on the food retail environment in stores participating in HIP, 

according to retailer reports. Reported promotion of fruits and vegetables changed over the course of 

the pilot in ways consistent with DTA’s activities—HIP participating stores received identifying 

decals at the start of the pilot and additional signage several months prior to the second retailer 

survey. At the outset of the pilot, there was concern, particularly among smaller stores, that HIP 

might increase the time and effort required to process purchases, but this does not appear to be the 

case. 

Grocery stores (that did not have IECR technology) reported more difficulties implementing HIP than 

did supermarkets/superstores. Nonetheless, relatively few retailers reported problems during the pilot. 

The most common problems and questions concerned identifying HIP-eligible items and identifying 

households participating in the pilot. These issues were reported more frequently by managers of 

grocery and convenience stores, who had to manually separate eligible foods and credit households; 

the process was completely automated in supermarkets/superstores eliminating most issues. 

Most supermarkets/superstores and grocery stores reported that they were satisfied with how HIP 

operated in their stores. Most convenience stores reported neither satisfaction nor dissatisfaction, 

likely reflecting the fact that they carried relatively few HIP-eligible items which limited the potential 

effect of the intervention on their store operations. 

Grocery stores reported that they made some additional efforts to sell more fruits and vegetables, 

including increasing their offerings, and they reported increased sales of fruits and vegetables and 

also increased profits. Few convenience stores reported efforts to increase sales of fruits and 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 4. Retailer Experiences ▌pg. 83

vegetables, though a small minority made some changes to stocking procedures. 

Supermarkets/superstores reported no additional efforts to promote and sell fruits and vegetables, 

which is not surprising given that the HIP intervention was modest relative to the overall scale of their 

operations. 

Analysis of EBT transaction data showed that HIP led to only a modest increase in SNAP spending in 

stores that participated in HIP, and very slightly reduced SNAP spending in stores that did not 

participate. Most of the additional dollars of SNAP spending went to supermarkets and superstores, as 

they account for the bulk of SNAP purchases. On a percentage basis, however, grocery stores 

experienced an increase in SNAP spending roughly comparable to that in supermarkets/superstores. 

The analysis of the effects of HIP on retailers presented in this chapter suggests that overall HIP had 

relatively little impact on store operations. Referring to the conceptual model describing the potential 

effect of HIP (Exhibit 1.1), the analysis suggests that HIP affected the overall retail food environment 

by directly affecting participation decisions, but had limited effects on the environment within 

participating stores. In addition, households only modestly changed their shopping patterns resulting 

in a relatively small impact on retailer redemptions. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 85

5. HIP Participant Experiences, TFV Purchases, and Incentives 

Earned 

This chapter uses survey and focus group data to describe participants’ awareness and understanding 

of HIP and their experiences and satisfaction with the pilot. The chapter also uses EBT data to 

describe HIP participants’ purchases of HIP-eligible targeted fruits and vegetables (TFVs) from 

participating retailers, which in turn determined the amount of incentives earned. As noted in Chapter 

1, each of these factors might have affected the magnitude of HIP’s impact on intake of targeted fruits 

and vegetables if HIP’s effect operated through the hypothesized causal mechanisms shown in our 

conceptual model (see Exhibit 1.1). 

Most of the participant survey questions concerning understanding of and experiences with HIP 

provided five- or six-point Likert scale response categories (e.g., very easy, easy, somewhat easy, 

somewhat difficult, difficult, very difficult). In this chapter, we are particularly interested in 

examining changes over time, and the statistical tests for changes across rounds use all items in the 

scale.56 However, we often discuss aggregated results (e.g., at least somewhat easy, at least somewhat 

difficult) in the chapter. 

The first section of the chapter describes awareness and understanding of HIP, while the second 

section describes the experiences of HIP participants and their satisfaction with the incentive 

program. Finally, the third section describes HIP purchases and HIP incentives earned. 

5.1 Awareness and Understanding of HIP 

In order for the HIP incentive to affect purchasing behavior, HIP participants needed to know about 

the program and understand how it worked. As discussed in Chapter 3, DTA sent multiple notices to 

HIP households with information on how HIP worked, including frequently-asked questions, 

guidelines for HIP-eligible fruits and vegetables, and lists of participating retailers. In this section, we 

examine survey participants’ awareness and knowledge of HIP, including knowing how they could 

earn the incentive and what fruits and vegetables were eligible for the incentive. This section draws 

on data from the first and second follow-up primary shopper interviews (Round 2 and Round 3), 

supplemented with relevant findings from focus groups. We present overall results combining reports 

from both post-implementation survey rounds. In addition, we present separate results for each survey 

round to examine how awareness and understanding changed over the course of the pilot. 

A majority (69 percent) of surveyed HIP participants reported having heard of the program (Exhibit 

5.1). Awareness of HIP increased over time. At the time of the Round 2 survey, which occurred 4-6 

months after households began participating in HIP, 62 percent had heard of HIP. By the time of the 

Round 3 survey, 9-11 months after HIP implementation, 76 percent reported having heard about HIP. 

 

56 Note that Round 2 descriptive results as reported in this Final Report may differ slightly from Round 2 

results as reported in the prior Interim Report. This is due to (1) use of pooled Round 2/3 weights across 

rounds when generating the Final Report tables, as compared to use of Round 2 weights alone in the 

Interim Report, and (2) inclusion of only those respondents in this Final Report that answered questions in 

both Rounds 2 and 3, to facilitate appropriate paired testing for changes across rounds, whereas all Round 2 

responses were included in the Interim Report. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 86 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

This increase was statistically significant. The second round of focus groups also revealed better 

awareness of HIP than did the first round of focus groups. 

Exhibit 5.1: Primary Shopper Awareness of HIP, by Treatment and Control Status, by 

Round: Round 2 & 3 Participant Surveys 

Proportion (N) 

Total Treatment Control P-value 

Ever heard of HIP? Proportion reporting “yes”

Pooled (Rounds 2 & 3) (N=2,890) 0.23 

(1222) 

0.69 

(1000) 

0.16 

(222) [<0.001]*** 

Round 2 (N=1,431) 0.19 

(533) 

0.62 

(444) 

0.13 

( 89) [<0.001]*** 

Round 3 (N=1,459) 0.26 

(689) 

0.76 

(556) 

0.19 

(133) [<0.001]*** 

Change: Round 3 – Round 2 

[P-value] 

0.07 

[<0.001]*** 

0.13 

[<0.001]*** 

0.06 

[0.001]*** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” responses coded as missing. 

Due to rounding, reported changes may differ from differences between Round 2 and Round 3 proportions. 

Pooled test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module). 

Awareness of HIP also increased slightly in the control group over time. However, as expected, given 

that no formal attempts were made to inform the control group about HIP, a significantly greater 

proportion of HIP participants had heard of the program compared to non-HIP participants. 

Awareness among HIP participants varied by some demographic characteristics and by some baseline 

fruit- and vegetable-related behaviors or preferences.57 HIP participants who were working, had 

children in the household, received higher SNAP benefits (over $200), spent more on targeted fruits 

and vegetables at baseline, or had fewer barriers to eating fruits and vegetables or to grocery shopping 

were more likely to say they had heard of HIP (see Appendix H, Exhibit H5.1).58 Perhaps these 

groups were more motivated to purchase and consume fruits and vegetables and therefore may have 

been more tuned into programs, such as HIP, that made fruits and vegetables more affordable. 

Of those who had heard of HIP, the most common source for learning about HIP was letters (Exhibit 

5.2). Over 60 percent of the HIP group reported receiving a letter about the pilot. Similarly, in the 

HIP focus groups, most respondents remembered receiving a letter notifying them of their selection 

for the HIP program. As one participant indicated, “I got a letter in the mail; they said they picked me 

randomly.” 

Curiously, in the survey both HIP and non-HIP participants reported having heard about HIP through 

letters (62 percent and 28 percent, respectively). Non-HIP participants were not sent any letters or 

other materials concerning HIP and the advance letters sent prior to the start of the telephone survey 

did not mention HIP. Perhaps non-HIP participants were confusing HIP communications with 

 

57 Appendix D, Section D.2 discusses creation of the behavior, attitudes, and barriers scales used in this subgroup analysis. 

58 See discussion in Chapter 8, Section 8.6 for details on how the subgroups were created. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 87

notifications from another program. As expected, however, a greater proportion of HIP participants 

heard about HIP through letters compared to non-HIP participants (the difference is 34 percentage 

points and clearly statistically significant). 

Exhibit 5.2: Where Primary Shopper Heard of HIP, by Treatment and Control Status: Round 

2 & 3 Participant Surveys

Proportion (N) 

Total Treatment Control P-value 

Letter 0.42 

(676) 

0.62 

(621) 

0.28 

( 55) [<0.001]*** 

Informational pamphlet 0.11 

(149) 

0.13 

(130) 

0.10 

( 19) [0.163] 

DTA case manager 0.07 

( 89) 

0.08 

( 76) 

0.06 

( 13) [0.442] 

Word of mouth 0.14 

(112) 

0.06 

( 67) 

0.20 

( 45) [<0.001]*** 

Westat (firm conducting the 

survey) 

0.07 

( 68) 

0.05 

( 50) 

0.08 

( 18) [0.131] 

Handout 0.05 

( 62) 

0.05 

( 51) 

0.05 

( 11) [0.987] 

Media (television, newspaper, 

internet) 

0.11 

( 61) 

0.02 

( 26) 

0.17 

( 35) [<0.001]*** 

Community service provider 0.02 

( 21) 

0.02 

( 15) 

0.03 

( 6) [0.174] 

Other source 0.09 

( 82) 

0.06 

( 57) 

0.11 

( 25) [0.017]** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” responses coded as missing. 

Test statistics are adjusted for clustering at the individual respondent level. 

Includes responses only from respondents who reported they had ever heard of HIP. Respondents selected all 

options that applied; proportions therefore sum to more than one. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (unweighted 

N=1,198). 

Of HIP and non-HIP participants who had heard of HIP, non-HIP participants were more likely than 

HIP participants to have heard of HIP through word of mouth, most likely through friends and family 

participating in the pilot. Non-HIP participants also reported learning about HIP through the media 

and other sources more than did HIP participants. However, there was no media discussion of HIP 

after the initial announcement of the pilot award; perhaps these respondents confused HIP with 

another program. 

HIP participants did relatively little to seek information about HIP (Exhibit 5.3). While DTA offered 

more than 140 training sessions between October 2011 and February 2012, only 5 percent of 

surveyed HIP participants reported that they attended a meeting about HIP. There was a slight 

(though statistically significant) increase, however, between Rounds 2 and 3 in the percentage of HIP 

participants who reported attending a meeting (4 percent in Round 2 to 6 percent in Round 3). No 

additional trainings occurred between Rounds 2 and 3, so participants may have been referring to 

another type of meeting. Although attendance at meetings was low across the board, participants who 

consumed three or more servings of fruits and vegetables per day at baseline were more likely to 

attend than those who consumed less than three servings a day. Similarly, those who had more fruits 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 88 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

and vegetables at home at baseline were more likely to attend the meetings than were those who had 

fewer fruits and vegetables at home (see Appendix H, Exhibit H5.2a). Focus groups indicated that 

participants knew about the trainings. As one participant indicated, “They told you about programs, 

little workshops you could go to and actually sit down and they’d explain what the program was 

about.” However, participants indicated that they were too busy to attend and some felt the trainings 

were not necessary, “Buying fruits and vegetables isn’t rocket science. They couldn’t tell you more 

than you already know [from the letter].” As mentioned in Chapter 3, attendance data similarly 

showed that overall attendance at training sessions was very low, at around 1 percent of HIP 

participants. However, those who said they attended gave positive feedback; over 90 percent felt that 

the meeting explained HIP well or very well (Exhibit 5.3). This feedback is consistent with 

participant evaluations completed after the training sessions. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 89

Exhibit 5.3: Primary Shopper Attendance at Training and Use of Hotline, by Round: Round 

2 & 3 Participant Surveys, HIP Participants 

Proportion (N) 

Pooled Round 2 Round 3 P-value 

Did you go to a meeting to learn how HIP works? (N=1,455)

Yes 0.05 

(83) 

0.04 

(34) 

0.06 

(49) [0.015]** 

No 0.95 

(1372) 

0.96 

(684) 

0.94 

(688) 

If yes, how well did the meeting explain HIP? (N=81)

Very well 0.71 

(58) 

0.73 

(25) 

0.69 

(33) [.]a

Well 0.20 

(16) 

0.21 

(7) 

0.19 

(9) 

Somewhat well 0.10 

(7) 

0.06 

(2) 

0.13 

(5) 

Not too well 0.00 

(0) 

0.00 

(0) 

0.00 

(0) 

Not at all well 0.00 

(0) 

0.00 

(0) 

0.00 

(0) 

Did you call the EBT or HIP hotline with questions or problems in the past month? (N=1,434) 

Yes 0.06 

(91) 

0.08 

(56) 

0.05 

(35) [0.006]*** 

No 0.94 

(1343) 

0.92 

(639) 

0.95 

(704) 

If yes, how helpful were the hotline staff in resolving your issue? (N=89)

Very helpful 0.50 

(43) 

0.44 

(24) 

0.60 

(19) [.]a

Helpful 0.17 

(16) 

0.17 

(10) 

0.17 

(6) 

Somewhat helpful 0.09 

(9) 

0.07 

(4) 

0.12 

(5) 

Not helpful 0.10 

(10) 

0.15 

(9) 

0.03 

(1) 

Very unhelpful 0.13 

(11) 

0.16 

(7) 

0.08 

(4) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Includes participants who reported they had not heard of HIP. a Test statistics cannot be computed because of stratum with single sampling unit. 

Source: Participant Survey (primary shopper module). 

The HIP 800 call line was used by a small percentage of participants (Exhibit 5.3). Less than 10 

percent of HIP participants said they called this hotline with questions (again, DTA data report calls 

from a much smaller proportion). Use of the hotline appeared to decline over time as fewer HIPparticipants reported calling the hotline in the month prior to Round 3 compared to the month prior to 

Round 2. Feedback on the hotline, for those who called, was generally positive. About half of HIP 

participants who said they called the hotline felt that staff were very helpful in resolving their issues, 

and three-quarters felt that hotline staff were at least somewhat helpful. Similarly, the second round of 

focus group participants noted general satisfaction with the hotline, “When I called I got a lot of 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 90 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

information,” and “I was satisfied with the phone call.” However, about one-quarter of survey 

participants felt that hotline staff were not helpful or very unhelpful. 

Despite some lack of awareness of HIP, which decreased over time, and very little information 

seeking, survey participants reported having a fairly easy time understanding how HIP worked 

(Exhibit 5.4). More than half of HIP participants who responded to survey questions (69 percent) felt 

that it was somewhat to very easy to understand how HIP worked and to remember which fruits and 

vegetables earned the HIP incentive. 

HIP survey participants who had children in the household, those who were WIC participants, those 

who received higher SNAP benefits, and those who reported fewer barriers to eating fruits and 

vegetables reported an easier time understanding how HIP worked and remembering which fruits and 

vegetables earned the HIP incentive. Households also participating in WIC likely had an easier time 

understanding HIP given that the TFVs were the same as those that could be purchased with WIC 

fruit and vegetable vouchers. Those with higher SNAP benefits and children in the home to feed and 

those with fewer barriers also may have been more motivated to understand these details. Focus 

group participants noted that they were now encouraging healthy eating behaviors in their children, 

perhaps indicating a particular motivation among participants with children in the household. 

In addition, HIP survey participants who reported more positive attitudes toward food, fruits, and 

vegetables were more likely to report an easier time remembering which fruits and vegetables earned 

the incentive compared to those who reported less positive attitudes toward food, fruits, and 

vegetables. These positive attitudes may have been associated with increased familiarity with fruits 

and vegetables, which could have helped participants to remember which fruits and vegetables 

qualified for HIP, or these positive attitudes similarly may have given participants the motivation to 

remember (see Appendix H, Exhibits 5.3a and 5.3b). 

Focus groups underscored the difficulty experienced by some HIP participants. During the first round 

of focus groups in which participants were asked directly about what they understood about HIP, 

many participants expressed a general lack of knowledge of how HIP worked, which fruits and 

vegetables qualified for the incentive, and how incentives were earned. For example, one focus group 

participant said, “It doesn’t really explain if you have to actually buy the fruit itself or if you can buy 

it in a canned product or a liquid product. And it doesn’t explain if you get the incentive or not when 

you buy those.” The different line of questioning in focus groups revealed a greater lack of 

understanding than did survey questions. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 91

Exhibit 5.4: Primary Shopper Understanding of HIP, by Round: Round 2 & 3 Participant 

Surveys, HIP Participants 

Proportion (N)

Pooled Round 2 Round 3 P-value 

How easy or hard has it been to understand how HIP works? (N=1,460)

Very easy 0.15 

(224) 

0.13 

(94) 

0.17 

(130) [<0.001]*** 

Easy 0.37 

(529) 

0.34 

(240) 

0.40 

(289) 

Somewhat easy 0.17 

(258) 

0.16 

(117) 

0.19 

(141) 

Somewhat hard 0.07 

(101) 

0.07 

(48) 

0.08 

(53) 

Hard 0.04 

(54) 

0.04 

(26) 

0.04 

(28) 

Very hard 0.03 

(47) 

0.04 

(26) 

0.03 

(21) 

Don't know 0.17 

(247) 

0.23 

(170) 

0.10 

(77) 

How easy or hard is it remembering which fruits and vegetables earn the HIP rebate? (N=1,458)

Very easy 0.13 

(199) 

0.12 

(89) 

0.14 

(110) [<0.001]*** 

Easy 0.35 

(502) 

0.33 

(233) 

0.37 

(269) 

Somewhat easy 0.21 

(304) 

0.19 

(131) 

0.24 

(173) 

Somewhat hard 0.12 

(177) 

0.14 

(95) 

0.11 

(82) 

Hard 0.07 

(96) 

0.07 

(52) 

0.06 

(44) 

Very hard 0.03 

(47) 

0.03 

(24) 

0.03 

(23) 

Don't know 0.09 

(133) 

0.13 

(95) 

0.05 

(38) 

How easy or hard is it keeping track of the HIP rebates you earn?a (N=550)

Very easy 0.25 

(139) 

0.26 

(66) 

0.23 

(73) [.]a

Easy 0.43 

(235) 

0.40 

(98) 

0.46 

(137) 

Somewhat easy 0.21 

(118) 

0.22 

(52) 

0.21 

(66) 

Somewhat hard 0.05 

(29) 

0.05 

(13) 

0.06 

(16) 

Hard 0.02 

(13) 

0.03 

(8) 

0.02 

(5) 

Very hard 0.01 

(4) 

0.01 

(3) 

<0.01 

(1) 

Don't know 0.02 

(12) 

0.02 

(6) 

0.02 

(6) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Includes participants who reported they had not heard of HIP. a

Question asked only of respondents who reported they kept track of HIP rebates earned. 

Source: Participant Survey (primary shopper module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 92 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

Both focus groups and surveys, however, revealed that understanding of how HIP worked and which 

fruits and vegetables qualified increased over time. At the time of the Round 2 survey, 63-64 percent 

of HIP participants reported that it was somewhat to very easy to understand how HIP worked and to 

remember which fruits and vegetables earned the incentive. By the time of the Round 3 survey, the 

percentage had increased to 75-76 percent (an increase of 11-13 percentage points). Some participants 

did report that they did not know how easy or hard it was to understand how HIP worked, indicating 

some confusion or lack of understanding. However, this confusion decreased over time. Focus groups 

confirmed increased understanding over time of which fruits and vegetables were eligible. For 

example, in contrast to the confusion noted above in the first round of focus groups, by the second 

round of focus groups, participants understood that all fresh produce (except white potatoes) would 

earn an incentive, provided the store was participating in HIP. 

While only about 40 percent of HIP participants said they kept track of HIP incentives earned 

(Exhibit 5.5), about 90 percent of those who did felt that keeping track of these incentives was 

somewhat to very easy. Among those who kept track of incentives, those who reported more barriers 

to grocery shopping reported a more difficult time tracking incentives than did those who reported 

fewer barriers. However, almost 90 percent of those who reported more barriers to grocery shopping 

(and kept track of incentives) still felt it was somewhat to very easy to track incentives (see Appendix 

H, Exhibit 5.3c). Participants who found it difficult to keep track of HIP incentives may not, in fact, 

have attempted to keep track of them, but those who did it consistently found it somewhat to very 

easy. 

Some confusion about the financial details and logistics of how participants received incentives on 

their EBT cards persisted through both rounds of focus groups. As one participant indicated, “I 

haven’t received anything...I was expecting to receive a little extra cash that I have not received,” and 

another incorrectly explained, “When you go shopping, it’s taken off immediately. Instead of 

spending 80 bucks, you’ll spend 76 ‘cause you got 4 dollars’ worth in HIP—you saved it right then at 

that moment, it’s like a coupon.” This confusion on financial details of the incentive may reside 

primarily among the 60 percent of HIP participants who did not track their HIP incentives. 

5.2 Experiences and Satisfaction 

This section explores different aspects of primary shoppers’ experiences with HIP and their overall 

satisfaction with the program. 

As mentioned above, survey results indicated that only 41 percent of participants kept track of the 

HIP incentives they earned, which was consistent with our focus group finding that many participants 

were unclear on the HIP incentives they earned. As one participant claimed, “I found out accidentally 

I got two dollars left.” The second round of focus groups similarly suggested that most participants 

did not know how much they earned in HIP incentives. However, the percentage of survey 

participants who reported keeping track of their incentives increased slightly between the Round 2 

and Round 3 surveys from 39 percent to 43 percent (Exhibit 5.5). 

In addition, those who reported higher spending on targeted fruits and vegetables at baseline, higher 

intake of fruits and vegetables, and those who reported fewer barriers to eating fruits and vegetables 

were more likely to keep track of incentives earned than were those who reported lower spending and 

intake and those who reported more barriers to eating fruits and vegetables, respectively (see 

Appendix H, Exhibit H5.4). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 93

Of the survey participants who did keep track of incentives, the majority did so through printed 

receipts (Exhibit 5.5). Over half of the participants felt it was very useful to have receipts showing 

fruit and vegetable expenditures and incentives, and almost 90 percent felt the receipts were at least 

somewhat useful. It is interesting that the percentage who found the receipts useful is considerably 

greater than the percentage who tracked their rebates; perhaps they liked having the information or 

thought it might be useful. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 94 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

Exhibit 5.5: Primary Shopper HIP Experiences, by Round: Round 2 & 3 Participant Surveys, 

HIP Participants 

Proportion (N) 

Pooled Round 2 Round 3 P-value 

Do you keep track of the HIP rebate you have earned? (N=1,347) 

Yes 0.41 

(550) 

0.39 

(246) 

0.43 

(304) [0.027]** 

No 0.59 

(797) 

0.61 

(395) 

0.57 

(402) 

If yes, how do you most often keep track of the HIP rebate? (N=519) 

Printed receipt 0.58 

(300) 

0.60 

(141) 

0.56 

(159) [.]a

By telephone using computerized 

system 

0.22 

(112) 

0.21 

(49) 

0.22 

(63) 

By telephone using live operator 0.04 

(18) 

0.05 

(9) 

0.03 

(9) 

Website 0.01 

(5) 

0.00 

(0) 

0.01 

(5) 

Other 0.16 

(84) 

0.13 

(35) 

0.18 

(49) 

How useful is it to have a receipt showing how much you spend on fruits and vegetables and the 

rebate you earn each time you shop? (N=1,327) 

Very useful 0.55 

(731) 

0.54 

(340) 

0.56 

(391) [0.054]* 

Somewhat useful 0.32 

(424) 

0.32 

(206) 

0.31 

(218) 

Not too useful 0.07 

(94) 

0.07 

(42) 

0.07 

(52) 

Not at all useful 0.06 

(78) 

0.06 

(40) 

0.05 

(38) 

How well do the cashiers and other workers in the store where you go grocery shopping understand 

HIP? (N=1,230) 

Very well 0.18 

(219) 

0.16 

(91) 

0.19 

(128) [0.003]*** 

Well 0.23 

(270) 

0.22 

(118) 

0.23 

(152) 

Somewhat well 0.22 

(280) 

0.22 

(133) 

0.23 

(147) 

Not too well 0.19 

(230) 

0.19 

(108) 

0.19 

(122) 

Not at all well 0.18 

(231) 

0.21 

(129) 

0.15 

(102) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Includes participants who reported they had not heard of HIP. 

Due to rounding, reported proportions may not sum to one. 

a Test statistics cannot be computed because of stratum with single sampling unit. 

Source: Participant Survey (primary shopper module). 

Participants gave mixed responses concerning how well they felt workers in their grocery stores 

understood HIP, but their perceptions of store workers’ understanding of HIP improved slightly over 

time. Just under 20 percent of participants felt grocery store workers understood HIP very well (this 

percentage increased from 16 percent at Round 2 to 19 percent at Round 3), but the same percentage 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 95

felt that grocery store workers did not understand HIP well at all (this percentage decreased from 21 

percent at Round 2 to 15 percent at Round 3). According to focus group participants, cashiers did not 

seem to take special notice of HIP-eligible items or ask customers to separate HIP items from nonHIP items. This finding does not necessarily signal a problem. Most SNAP spending occurred in 

stores with integrated electronic cash registers (IECRs); this technology did not require cashiers to 

make any special accommodations in order for HIP participants to earn incentives for HIP purchases. 

SNAP participants automatically earned the HIP incentive as items were scanned. In contrast, in 

stores without IECRs, cashiers did have to enter HIP purchases separately and they may have 

required shoppers to separate HIP and non-HIP eligible items. In these types of stores, cashiers 

needed to be aware of which items qualified for the HIP incentive. 

Exhibit 5.6: Primary Shopper Problems with EBT Card, by Treatment and Control Status, by 

Round: Round 2 & 3 Participant Surveys 

Total Treatment Control P-Value 

Primary shopper uses household EBT card most often (N=2,916)

Pooled (Rounds 2 & 3) (N=3421) 0.90 

(2646) 

0.91 

(1335) 

0.90 

(1311) [0.466] 

Round 2 (N=1,445) 0.91 

(1309) 

0.90 

(656) 

0.91 

(653) [0.763] 

Round 3 (N=1,471) 0.90 

(1337) 

0.92 

(679) 

0.90 

(658) [0.124] 

Change: Round 3 – Round 2 [P-value] -0.01 

[0.537] 

0.01 

[0.368] 

-0.01 

[0.450] 

Any problems with your EBT card or account in past month? (N=2,918)

Pooled (Rounds 2 & 3) (N=3,423) 0.03 

(88) 

0.03 

(39) 

0.03 

(49) [0.287] 

Round 2 (1,445) 0.04 

(50) 

0.03 

(21) 

0.04 

(29) [0.159] 

Round 3 (N=1,473) 0.03 

(38) 

0.03 

(18) 

0.03 

(20) [0.875] 

Change: Round 3 – Round 2 [P-value] -0.01 

[0.113] 

>-0.01 

[0.858] 

-0.01 

[0.115] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Pooled test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module). 

All SNAP participants—HIP and non-HIP—received their SNAP benefits on EBT cards and the 

surveys asked both groups about their experiences with the cards (Exhibits 5.6 and 5.7). As expected, 

the primary shopper, not other household members, used the EBT card most often (90 percent of all 

households). Few respondents (3 percent) experienced any problems with their EBT card or account 

in the month prior to the follow-up surveys. Among those who reported problems, the more common 

problems (reported by 15-30 percent), included a lost, stolen, or damaged card; trouble making a 

purchase; confusion about balance; problems with benefits; and, for HIP participants only, not getting 

the HIP incentive that was expected. No significant differences emerged between HIP and non-HIP 

participants in terms of problems experienced with EBT cards. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 96 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

Exhibit 5.7: Type of Primary Shopper Problems with EBT Card, by Treatment and Control 

Status: Round 2 & 3 Participant Surveys 

Total Treatment Control P-Value 

Problem with benefits 0.29 

(27) 

0.27 

(11) 

0.29 

(16) [0.869] 

Lost, stolen, or damaged card 0.25 

(20) 

0.24 

(9) 

0.25 

(11) [0.879] 

Did not get HIP rebate/incentive 

expected (N=39)a

0.21 

(9) 

0.21 

(9) N/A 

Trouble making purchase 0.21 

(16) 

0.12 

(5) 

0.22 

(11) [0.198] 

Confusion about balance 0.16 

(15) 

0.25 

(8) 

0.15 

(7) [0.295] 

Forgot PIN 0.06 

(4) 

0.02 

(1) 

0.07 

(3) [0.244] 

Negative attitude from cashier 0.02 

(2) 

0.02 

(1) 

0.02 

(1) [0.953] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” responses coded as missing. 

Respondents selected all options that applied; proportions therefore sum to more than one. 

Includes responses only from respondents who reported they had problems with their EBT card. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Answer option available for HIP participants only. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (unweighted N=88). 

Finally, to examine HIP participants’ satisfaction with the program, we asked primary shoppers if 

they would like to keep participating in HIP (Exhibit 5.8). Ninety-five percent of respondents 

indicated that they would like to keep participating in HIP, suggesting overall satisfaction with the 

program, which remained high over time. Satisfaction was also high across subgroups, but those who 

reported more fruit and vegetable intake and those who shopped primarily at HIP participating stores 

at baseline were even more likely to indicate that they would like to continue participating in HIP 

compared to those who reported less fruit and vegetable intake and shopped primarily at non-HIP 

participating stores at baseline (see Appendix H, Exhibit H5.6). Consistent with the satisfaction 

reflected in survey responses, focus groups indicated that, overall, participants were happy with the 

HIP program. One focus group member noted, “I would like to have the incentive program extended. 

It’s a very good program and would like to have it offered to others,” and another, “I was happy to get 

rewarded for eating healthier or making better decisions with my food stamps.” 

Exhibit 5.8: Primary Shopper Satisfaction with HIP, by Round: Round 2 & 3 Participant 

Surveys, HIP Participants

Would you like to keep participating 

in HIP? 

Proportion (N) 

Pooled Round 2 Round 3 P-value 

Yes 0.95 (1311) 0.96 (628) 0.95 (683) [0.365] 

No 0.05 (59) 0.04 (24) 0.05 (35) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Includes participants who reported they had not heard of HIP. 

Due to rounding, reported proportions may not sum to one. 

Source: Participant Survey (primary shopper module); (unweighted N=1,370). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 97

5.3 HIP Purchases and Earning HIP Incentives 

This section uses EBT data to measure the HIP purchases and incentives earned by HIP participant 

households. EBT data, described more fully in Chapter 2, report the date, location, and amount of all 

SNAP transactions. The EBT data include all SNAP households in Hampden County, not only the 

survey sample. This section uses EBT data for HIP participants only.59 

Our terminology makes the following distinction between targeted fruit and vegetable purchases 

(TFVs) in general and HIP purchases more specifically: 

 TFV purchases are the value of any household’s purchases of targeted fruits and vegetables, 

in any retailer, with any tender (such as credit card, cash, or EBT). 

 HIP purchases are the value of a HIP participant household’s TFV purchases recorded by 

the EBT system in a participating retailer. 

In participating retailers with IECRs, TFV purchases were recorded automatically for both HIP 

participants and non-HIP participants. In retailers lacking an IECR, cashiers had to identify TFV 

purchases that qualified as HIP purchases. In these retailers, TFV purchases were therefore recorded 

only for HIP participants and not for non-HIP participants. HIP participants had to identify 

themselves and the retailer had to successfully implement the manual procedures. The HIP incentive 

equaled 30 percent of HIP-eligible purchases up to the cap of $60 per household per month. 

Our analysis classified retailers by retail category (collapsing the 12 FNS official store types eligible 

for HIP into 4 categories) and by whether they participated in HIP: 

 Retail categories: supermarkets/superstores, convenience stores, grocery and specialty, and 

other (farmers markets/direct marketing farms). All supermarkets/superstores and some 

convenience stores were equipped with IECRs. Virtually all grocery, specialty, and other 

retailers, and some convenience stores did not use IECRs. 

 Retailer participation: some retailers joined HIP at the start of the pilot in November 2011, 

some joined HIP in later months (February, June, and October,60 2012), and some did not join 

HIP at all. 

For most of the pilot period, SNAP spending by HIP households averaged around $260 per household 

per month and was nearly equally distributed between participating and non-participating retailers 

(Exhibits 5.9 and 5.10).61 Near the end of the pilot nine additional retailers (including three 

supermarkets and four grocery stores) began participating in HIP. As a result, the fraction of SNAP 

spending in participating retailers increased somewhat beginning in October 2012. 

 

59 Chapter 6 uses EBT data to estimate pilot impacts on food purchase outcomes, comparing HIP participants 

to non-HIP participants. 

60 One retailer in the October 2012 cohort ended up joining on November 1, 2012. 

61 SNAP redemptions were relatively high in November 2011 as many households received additional SNAP 

disaster benefits due to the severe late October snow storm that struck western Massachusetts, including 

Hampden County. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 98 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

Exhibit 5.9: Mean Monthly SNAP Redemptions of All HIP Participant Households at 

Participating and Non-Participating Stores 

Participating retailers Non-participating retailers All retailers 

Mean SNAP 

redemptions 

($) 

Percent of all 

SNAP 

redemptions 

(%) 

Mean SNAP 

redemptions 

($) 

Percent of all 

SNAP 

redemptions 

(%) 

Mean SNAP 

redemptions 

($) 

November 2011a 164.95 47.8 180.45 52.2 345.39 

December 2011 134.09 48.6 141.64 51.4 275.73 

January 2012 132.93 50.2 131.64 49.8 264.58 

February 2012 128.32 50.6 125.40 49.4 253.72 

March 2012 131.92 50.9 127.20 49.1 259.12 

April 2012 134.34 51.8 125.23 48.2 259.57 

May 2012 132.71 50.0 132.71 50.0 265.42 

June 2012 131.53 50.3 129.95 49.7 261.48 

July 2012 132.64 50.4 130.51 49.6 263.15 

August 2012 131.33 50.2 130.08 49.8 261.41 

September 2012 139.48 53.9 119.28 46.1 258.76 

October 2012 152.54 57.8 111.39 42.2 263.93 

November 2012 147.76 56.9 111.74 43.1 259.50 

December 2012 148.43 57.2 111.14 42.8 259.58 

HIP store status determined separately for each month. 

Due to rounding, reported percentages may not sum to 100.a November 2011 SNAP redemptions elevated due 

to receipt of SNAP disaster benefits. 

Source: EBT Transaction Data (average of 46,440 households per month). 

Exhibit 5.10: Mean Monthly SNAP Redemptions of All HIP Participant Households at 

Participating and Non-Participating Stores 

November 2011 SNAP redemptions elevated due to receipt of SNAP disaster benefits 

Source: EBT Transaction Data (average of 46,440 households per month). 

There was some variation in HIP purchases of targeted fruits and vegetables from month to month, 

but, in general, monthly HIP purchases remained fairly low, relative to initial expectations, 

$0

$20

$40

$60

$80

$100

$120

$140

$160

$180

$200

Mean Household SNAP 

Redemptions

Month

Participating retailer Nonparticipating retailer
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 99

throughout the pilot.62 Within the participating retailers, mean monthly HIP purchases rose from 

slightly under $10 in January 2012 to more than $12 in June 2012, and then subsided slightly through 

September 2012 (Exhibit 5.11). Mean monthly HIP purchases increased slightly again in October 

2012 as additional retailers began participating in HIP. 

Exhibit 5.11: Mean Monthly HIP Purchases of all HIP Participant Households, by Store Type 

Supermarkets/ 

Superstores ($) 

Convenience 

 ($) 

Grocery 

 ($) 

Other 

($) 

November 2011a 11.18 0.04 0.10 0.00 

December 2011 a 7.74 0.04 0.04 0.00 

January 2012 a 9.68 0.03 0.07 0.00 

February 2012 10.60 0.07 0.04 0.00 

March 2012 10.91 0.05 0.08 0.00 

April 2012 11.68 0.06 0.10 0.00 

May 2012 11.84 0.04 0.08 0.00 

June 2012 12.80 0.03 0.12 0.02 

July 2012 12.68 0.03 0.11 0.05 

August 2012 12.14 0.06 0.15 0.07 

September 2012 11.13 0.03 0.30 0.10 

October 2012 12.32 0.03 0.38 0.09 

November 2012 11.61 0.01 0.41 0.01 

December 2012 10.71 0.01 0.39 0.00 

Includes all HIP households, including those who made no HIP purchases of TFVs during the month or no HIP 

purchases in a given store type. 

a

 HIP rollout occurred November 2011-January 2012 and HIP purchases were lower until all HIP households 

were participating; November 2011 elevated due to receipt of SNAP disaster benefits. 

Source: EBT Transaction Data (average of 46,440 households per month). 

SNAP and HIP Purchases by Retailer Type 

We examined SNAP purchases and HIP purchases separately by retailer type to understand where 

HIP participants spent their SNAP benefits and earned HIP incentives. We aggregated the monthly 

transaction data across the period from March to October 2012 and also divided transactions into two 

time periods roughly corresponding to the Round 2 and Round 3 participant surveys. This allowed us 

to examine SNAP expenditures in the same time period when food consumption was measured by the 

24-hour dietary recall data. 

For the time period corresponding to the Round 2 participant survey (March through July 2012), 

mean monthly HIP purchases were $12.12 (Exhibit 5.12). For the time period corresponding 

approximately to the Round 3 participant survey (August through October 2012), mean monthly HIP 

purchases were almost unchanged at $12.21, even though total SNAP purchases in participating 

retailers had increased moderately (from $132.99 to $140.36 per month). 

 

62 Initial expectations were based on USDA’s Thrifty Food Plan (TFP) recommendation for fruit and 

vegetable spending and Blisard and Steward’s (2006) estimate of average fruit and vegetable spending as a 

fraction of the TFP. Early calculations (designed to estimate a likely upper bound for HIP purchases, so that 

the implications of the incentive cap could be anticipated) suggested that on average SNAP households in 

Hampden County might spend about $70 on fruits and vegetables per month and earn incentives of about 

$20 per month. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 100 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

To provide a sense of the size of HIP purchases relative to households’ overall grocery spending, we 

calculated the percent of all SNAP purchases and the percent of SNAP purchases in participating 

retailers that HIP purchases represented. In both rounds, these HIP purchases represented 

approximately 5.1 percent of SNAP purchases in all retailers (including both retailers that did and did 

not participate in HIP). HIP purchases represented 10.3 percent of SNAP purchases in participating 

retailers in Round 2 and 9.7 percent of SNAP purchases in participating retailers in Round 3.
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and InceExhibit 5.12: Mean Monthly SNAP Purchases and HIP Purchases of all HIP Participant Households, by Round 

Pooled 

(March-October 2012) 

Round 2 

(March–July 2012) 

Round 3 

(August–October 2012) 

Ro

SNAP purchases at all retailers ($) 261.82 262.54 261.09 -1.4SNAP purchases at participating retailers ($) 136.68 132.99 140.36 7.3

Supermarkets/superstores ($) 126.36 122.83 129.88 7.0Convenience—IECR and non-IECR ($) 3.95 4.03 3.88 -0.1Grocery ($) 6.21 6.11 6.30 0.1Other ($) 0.16 0.03 0.30 0.2

HIP purchases ($) 12.16 12.12 12.21 0.0HIP purchases as a percent of total SNAP purchases at all retailersa (%) 5.10 5.10 5.10 -0.0HIP purchases as a percent of total SNAP purchases at participating retailersa (%) 

10.00 10.30 9.70 -0.5

Incentives earned ($) 3.65 3.64 3.66 0.0Percent of SNAP households that earned an incentive (%) 65.60 65.60 65.70 -0.1

F-test: *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported changes may differ from the differences between the means for Round 2 and Round 3. Test statistics are adjusted for clustering at the household level for the separate Round 2 and Round 3 results. a Calculated as average of household-level percentages. 

Source: EBT Transaction Data (6,214 HIP participant households). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 102 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

Almost all HIP purchases (97.7 percent), and therefore almost all incentives earned, were in 

supermarkets and superstores (Exhibit 5.13). There are three plausible and not mutually exclusive 

reasons why HIP purchases were predominantly in just this single retail format. First, most SNAP 

spending (80 percent overall and 92.5 percent of spending in HIP participating stores) is in 

supermarkets and superstores. It is common for SNAP participants to make a major shopping trip 

shortly after benefits are credited, usually in a supermarket or superstore (Wilde and Ranney, 2000; 

Wilde and Andrews, 2000; Shapiro, 2005; Castner and Henke, 2011). Second, supermarkets and 

superstores sell comparatively more fruits and vegetables than smaller stores. Third, it is possible that 

through lack of knowledge or because of the transaction time burden, some fruit and vegetable 

purchases in smaller stores lacking IECRs may not have been separated and registered so that the 

incentive could be earned (though we have no direct evidence on this point). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and InceExhibit 5.13: Mean Monthly SNAP Purchases for HIP Participant Households at HIP Participating Retailers, by Ro

Retailer type 

Pooled 

(March-October 2012) 

Round 2 

(March–July 2012) 

Round 3 

(August–October 2012) 

Change

SNAP purchases All retailers ($) 136.68 132.99 140.36 7.37 Supermarkets/superstores ($) 126.36 122.83 129.88 7.06 

Convenience—IECR and non-IECR ($) 

3.95 4.03 3.88 -0.14 

Grocery ($) 6.21 6.11 6.30 0.19 Other ($) 0.16 0.03 0.30 0.27 

HIP purchases All retailers ($) 12.16 12.12 12.21 0.09 Supermarkets/superstores ($) 11.88 11.96 11.81 -0.15 

Convenience—IECR and non-IECR ($) 0.04 0.05 0.04 >-0.01 Grocery ($) 0.19 0.10 0.27 0.17 Other ($) 0.05 0.02 0.09 0.07 

HIP purchases as percentage of SNAP purchasesaAll retailers (%) 10.00 10.30 9.70 -0.59 Supermarkets/superstores (%) 9.70 10.10 9.30 -0.82 

Convenience—IECR and non-IECR (%) 0.10 0.10 0.10 >-0.01 Grocery (%) 0.20 0.10 0.30 0.18 Other (%) 0.10 <0.01 0.10 0.05 

F-test: *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported changes may differ from the differences between the means for Round 2 and Round 3. Test statistics are adjusted for clustering at the household level for the separate Round 2 and Round 3 results. a Calculated as average of household-level percentages. 

Source: EBT Transaction Data (6,214 HIP participant households). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 104 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

HIP Incentives Earned 

More than a third (34 percent) of HIP households had no HIP purchases at all in a given month 

(Exhibit 5.14). However, few HIP households (4.5 percent) did not earn any incentives at all during 

the pilot.63 

Experiences in Rounds 2 and 3 were similar. There are two categories of households that did not earn 

incentives in a given month: 

 Half of this group had no SNAP purchases in participating HIP retailers. There are a number 

of possible explanations for this finding. Retail access issues could be one important reason 

for not making use of the pilot. Alternatively, some households may have chosen not to 

change their usual shopping location (maybe because the incentive was not large enough) or 

they may not have understood that since not all retailers participated, they might have to 

change their usual shopping location to earn the incentive. 

 The other half of the zero incentive earners did patronize participating retailers but did not 

make any HIP purchases at those retailers, suggesting that some consumers did not seek to 

purchase targeted fruits and vegetables at the participating retailers, or at least did not use 

their SNAP benefits for this purpose. 

At the other end of the spectrum, slightly more than one third (36 percent) of HIP households had 

monthly HIP purchases greater than $12. Only a few of these households either spent all their benefits 

on TVFs or reached the $60 incentive cap in any month. The remaining 30 percent of HIP households 

had monthly HIP purchases greater than zero but less than $12.64

 

63 Only 126 Wave 1 households, 122 Wave 2 households, and 77 Wave 3 households earned no incentives 

during their eligibility period. The total number of HIP households that received a SNAP benefit in any 

month during the November 2011 to December 2012 period was 7,301. Thus, the percent of HIP 

households that never earned any incentives is [(126+122+77)/7301] 4.45%. 

64 Appendix F, Exhibit F5.1 presents these data on incentive earnings broken down by store type and 

separately for Round 2 and Round 3. Most SNAP purchases, HIP purchases, and incentives earned 

occurred in supermarkets and superstores. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 105

Exhibit 5.14: Distribution of HIP Households by TFV Purchases 

Pooled 

(MarchOctober 2012) 

Round 2 

(March–July 

2012) 

Round 3 

(August–

October 2012) 

Change: Round 3 – 

Round 2 [P-value] 

Never shopped at 

participating stores (%) 17.00 17.45 16.55 -0.90 [0.004]*** 

HIP households that shopped at participating stores (%)

TFV = $0 17.36 16.97 17.75 0.78 [0.021]** 

$0<TFV<=$6 15.00 14.99 15.02 0.03 [0.923] 

$6<TFV<=$12 14.63 14.60 14.66 0.06 [0.860] 

TFV > $12 36.01 36.01 36.01 0.01 [0.987] 

Total 100.00 100.00 100.00 

F-test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported percentages may not sum to 100 and reported changes may differ from the 

differences between the means for Round 2 and Round 3. 

Test statistics are adjusted for clustering at the household level for the separate Round 2 and Round 3 results. 

Source: EBT Transaction Data (6,214 HIP participant households). 

SNAP and HIP Purchases by Participant Subgroup 

HIP purchases, and hence HIP incentives earned, differed for subgroups defined by the demographic, 

financial, and location characteristics of SNAP participant households (Exhibit 5.15). We conducted 

these subgroups analyses separately; the separate associations do not necessarily imply that all of 

these characteristics independently affect HIP purchases. 

In any given month over the period from March to October 2012, 65.7 percent of households had 

some positive amount of HIP purchases and hence earned some incentive. As noted earlier, the mean 

monthly HIP purchase was $12.16, so the mean incentive earned was $3.65. For all households 

combined (including those who had no HIP purchases), mean HIP purchases represented 5.1 percent 

of all SNAP purchases. 

As one would expect, the take-up of the HIP incentive increased with higher SNAP benefits. Of 

households with more than $350 in monthly SNAP benefits, 80 percent had some positive amount of 

HIP purchases. Compared to households with lower benefit levels, the mean HIP purchase amount in 

dollars ($18.57) was elevated for households in this highest category of SNAP benefits. However, 

HIP purchases as a percentage of all SNAP purchases was largest for households who received the 

lowest SNAP benefits. 

Male one-person households had lower take-up of HIP benefits, compared with other household types 

including single females and households with children. Only 52.7 percent of single males had any 

HIP purchases at all. The mean HIP purchase was $7.52, which represented only 4.4 percent of all 

SNAP purchases. 

Compared to other households, the take-up of the HIP incentive was higher for households whose 

primary language was Spanish. Take-up also was higher for households who identified as Hispanic or 

“other” race/ethnicity (principally Asian). The take-up of the HIP incentive was lower, and the mean 

HIP purchase amount was lower, for households that included a disabled person, compared with 

households that did not. 

For households whose head was 55 or older, the mean SNAP benefit was lower than for other 

households. SNAP households with an older head are frequently one-person households and are 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 106 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

likely to have some cash income (both of which are characteristics that lead to lower SNAP benefits). 

Despite the low mean SNAP benefits, the mean HIP purchase amount for these households was 

almost as high as for households with the youngest heads of household. For households having an 

older household head, HIP purchases were fully 7.6 percent of all SNAP purchases, a higher 

percentage than was observed for other households. 

Exhibit 5.15: HIP Purchases, SNAP Purchases, and HIP Incentives Earned, by Subgroup 

Baseline household 

characteristics 

HIP 

household 

with HIP 

purchases 

in a given 

month 

 (%) 

Mean 

monthly 

SNAP 

purchases 

($) 

Mean monthly HIP 

purchasesa

Mean HIP 

incentive 

earnedc 

($) 

 Dollars 

($) 

Percent of 

SNAP 

purchasesb 

(%) 

Race/ethnicity of HoH 

Hispanic 69.74 285.67 12.83 4.91 3.85 

White 62.37 239.53 11.64 5.32 3.49 

Black 61.31 256.36 10.07 4.42 3.02 

Other 64.52 229.32 14.79 6.81 4.43 

Across subgroups p-value <0.001*** <0.001*** <0.001*** <0.001*** <0.001*** 

Primary language 

Other primary language 64.08 265.34 11.14 4.63 3.34 

Spanish primary language 70.01 251.28 15.05 6.46 4.51 

Across subgroups p-value <0.001*** 0.002*** <0.001*** <0.001*** <0.001*** 

Monthly household income 

$0 64.76 308.03 11.97 3.97 3.59 

$1 - $787 64.73 262.83 11.88 4.73 3.56 

$788 - $1,083 63.83 203.11 10.64 5.74 3.19 

$1,084 + 69.67 285.17 14.50 5.91 4.35 

Across subgroups p-value <0.001*** <0.001*** <0.001*** <0.001*** <0.001*** 

Age of HoH 

16 - 30 69.15 337.30 12.47 3.74 3.74 

31 - 40 69.10 339.44 14.12 4.24 4.23 

41 - 54 61.04 222.69 10.45 4.71 3.14 

55 + 64.04 162.38 12.06 7.64 3.62 

Across subgroups p-value <0.001*** <0.001*** <0.001*** <0.001*** <0.001*** 

Disabled HoH 

Not disabled 71.06 330.79 14.41 4.75 4.32 

Disabled 61.04 202.36 10.28 5.43 3.08 

Across subgroups p-value <0.001*** <0.001*** <0.001*** <0.001*** <0.001*** 

Household composition 

Household with children (no 

elderly) 77.63 403.12 16.72 4.41 5.01 

Household with elderly (with or 

without children) 62.66 152.43 12.49 8.46 3.75 

Other household 57.78 187.43 8.83 4.78 2.65 

Across subgroups p-value <0.001*** <0.001*** <0.001*** <0.001*** <0.001*** 

Household size and headship 

HH Size 2+ male head 73.09 320.40 20.16 6.75 6.03 

HH Size 2+ female head 76.37 382.63 15.77 4.48 4.73 

HH Size 1 male head 52.69 170.35 7.52 4.44 2.26 

HH Size 1 female head 61.39 165.89 10.10 6.39 3.03 

Across subgroups p-value <0.001*** <0.001*** <0.001*** <0.001*** <0.001*** 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 5. HIP Participant Experiences, TFV Purchases, and Incentives Earned ▌pg. 107

Baseline household 

characteristics 

HIP 

household 

with HIP 

purchases 

in a given 

month 

 (%) 

Mean 

monthly 

SNAP 

purchases 

($) 

Mean monthly HIP 

purchasesa

Mean HIP 

incentive 

earnedc 

($) 

 Dollars 

($) 

Percent of 

SNAP 

purchasesb 

(%) 

Monthly SNAP benefit 

$1 - $160 57.20 146.28 8.86 6.53 2.66 

$161 - $200 59.08 193.92 9.47 4.85 2.84 

$201 - $349 75.93 304.36 14.89 5.12 4.47 

$350 + 79.95 466.49 18.57 4.05 5.57 

Across subgroups p-value <0.001*** <0.001*** <0.001*** <0.001*** <0.001*** 

Location 

Springfield 65.45 265.77 11.78 4.90 3.53 

Chicopee & Holyoke 68.35 264.54 12.23 5.16 3.67 

Other 63.13 248.18 13.10 5.60 3.93 

Across subgroups p-value <0.001*** <0.001*** 0.016** 0.001*** 0.016*** 

All 65.66 261.59 12.18 5.12 3.65 

F-test: *p<0.1, **p<0.05, ***p<0.01. 

Test statistics are adjusted for clustering at the household level 

a

HIP purchases are SNAP expenditures recognized in EBT records as having been spent on target fruits and 

vegetables, net of returns; mean includes households that made $0 purchases. b

HIP purchases as a percent of SNAP purchases calculated for each household; column shows the mean 

percentage across households, including those with $0 HIP purchases. 

c

Equal to 30 percent of HIP purchases up to the statutory cap; includes households with $0 HIP incentives 

earned. 

Source: EBT Transaction Data, pooled across March-October 2012 and DTA SNAP Caseload Data, July 

2011(average of 6,212 households per month). 

5.4 Discussion 

Taken together, the findings from the participant survey suggest that some HIP participants did not 

fully understand the pilot, especially near the beginning of their period of participation. Just over 60 

percent of HIP participants reported that they had heard about HIP when asked in the Round 2 survey, 

which occurred 4–6 months after HIP implementation. Awareness increased over time; by the Round 

3 survey, 9–11 months after implementation, three-quarters of HIP participants reported having heard 

about HIP. Understanding how HIP worked and which fruits and vegetables qualified for the 

incentive also increased over time. About 60 percent of households in Round 2 reported that it was at 

least “somewhat easy” to understand. Understanding increased by about 15 percentage points in 

Round 3 with approximately 75 percent of households reporting it was at least “somewhat easy” to 

understand HIP. 

As discussed in Chapter 3, DTA expended considerable resources developing participant notifications 

and materials that provided information on how HIP operated. The department also developed 

training materials and provided many training sessions. Despite these efforts, few HIP participants 

availed themselves of training sessions. Few trainings were offered in the evening or on weekends 

and some stakeholders interviewed suggested that more participants might have attended if some 

trainings were held after work and school hours. In retrospect, some DTA staff interviewed suggested 

that simpler notification materials may be more effective and that alternate means of communicating 

with participants, such as through social media, should be considered if HIP were to be expanded 

nationwide. Focus groups also revealed some literacy issues among HIP participants, suggesting that 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 108 ▌5. HIP Participant Experiences, TFV Purchases, and Incentives Earned Abt Associates

more visual media may also be useful in increasing understanding and awareness of HIP. Also, 

widespread communication about HIP was not possible during the pilot due to the design of the 

evaluation and the fact that not everyone was selected to participate in HIP. 

Although HIP participants valued the program (95 percent wanted to keep participating), HIP 

purchases remained fairly low throughout the pilot relative to initial expectations. Based on analyses 

conducted during the evaluation’s design phase (and consistent with self-reported total fruit and 

vegetable expenditures reported in Chapter 6), it seemed plausible that households might spend $70 

per month on TFVs, earning them $20 in incentives. On average, HIP households spent $11-$13 on 

targeted fruits and vegetables using their EBT card in participating retailers each month, which earned 

them an average of $3-$4 in HIP incentives. Overall, two-thirds of HIP households purchased some 

HIP-eligible foods in an average month, thus earning incentives. The other one-third of households 

did not earn any incentives in a specific month. Of these households, about half of them did not make 

any SNAP purchases in stores participating in HIP. The other half did some shopping in HIP 

participating stores, but did not purchase any targeted fruits and vegetables. 

Several barriers may have prevented HIP participants from achieving higher levels of HIP purchases 

and earning higher levels of incentives. As reported in both surveys and focus groups, some 

participants did not fully understand how the pilot worked and how incentives could be earned, 

including the types of fruits and vegetables that were eligible to earn incentives and which retail 

stores were participating in HIP. Others may have understood the basic mechanics of the pilot, but 

without fully appreciating the potential value of altering shopping patterns so that TFV spending was 

conducted specifically in participating retailers (and not in other authorized SNAP retailers) and 

specifically with EBT benefits (not cash). Still other participants may have faced transportation 

barriers that prevented their access to participating retailers, or they may have patronized participating 

retailers without IECRs that did not reliably separate out qualifying HIP purchases. Finally, some 

participants may simply not have wanted to consume fruits and vegetables in the first place. Taken 

together, HIP purchases and incentives earned were only about 20 percent of what was anticipated at 

the start of the pilot. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 109

6. Effects on Expenditures and Shopping Behaviors 

HIP was expected to affect fruit and vegetable consumption primarily by affecting food expenditures 

and shopping patterns. Over the course of the pilot, SNAP participants made decisions every month 

about where to shop and what to purchase. HIP’s price incentive may have encouraged households to 

increase total spending on targeted fruits and vegetables. Also, even when households would have 

made particular TFV purchases anyway, HIP gave the households an economic motivation to conduct 

such purchases with SNAP benefits at participating retailers, so that the purchases would earn the 

incentive. Thus, HIP potentially could have led households to shift more of their grocery spending to 

retailers that participated in HIP and to have increased their purchases of TFVs. 

As we discuss in detail, we use both EBT data and household survey data to explore the presence and 

magnitude of the expected changes in where households shopped and how much TFVs they bought. 

Specifically, this chapter examines HIP effects on expenditures and shopping behaviors and is 

organized as follows: 

 HIP impacts on food expenditures, including total SNAP spending and fruit and vegetable 

spending (Section 6.1) 

 HIP impacts on shopping patterns, including decisions about which retailers to patronize in 

order to understand the mechanisms by which HIP affected expenditures (Section 6.2) 

 Spending in farmers markets (Section 6.3) 

As discussed in Chapter 2, the analyses of both EBT transaction data and participant surveys are 

based on (regression-adjusted) comparisons of the HIP and non-HIP groups.65 Participant survey 

analyses combine reports from both post-implementation survey rounds (Round 2 and Round 3). 

Similarly, EBT analyses combine transaction data from March-October 2012, the six-month period 

spanning the last two participant surveys.66 

 

65 SNAP spending measures calculated from EBT transaction data presented in this chapter differ slightly 

from the estimates presented in Chapter 5 because Chapter 6 uses regression-adjusted estimation 

procedures. Covariates included: household residential location; size, composition, and primary language 

of household; household income and SNAP benefit; age, race/ethnicity, gender, and disability status of 

household head; and sampling wave. 

66 Appendix G presents analyses of changes between Round 2 and Round 3; results are similar to those 

presented here. Appendix H presents subgroup analyses for the outcomes presented in this chapter. 

Specifically, we considered differential impacts for those subgroups for which we found significant 

differential impact of HIP on consumption variables (current or preliminary analyses), as well as for 

subgroups by baseline shopping occurring predominantly in HIP-participating stores, which might 

substantively be expected to influence impacts on expenditures even in cases where differential impacts on 

consumption were not observed. See discussion in Chapter 8, Section 8.6 for details on how the subgroups 

were created. These differences are discussed in the main text only when consistent patterns emerged that 

provided context for the pooled findings. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 110 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

Each section uses data from both EBT transaction records and participant surveys. The two data 

sources appear to give different answers for some research questions; in these cases, we consider 

possible explanations for the observed discrepancies as appropriate. 

6.1 Food and Targeted Fruit and Vegetable Expenditures 

In this section, our analysis of food expenditures has four parts: 

 First, we report how much HIP affected total SNAP spending. HIP participants earned 

incentives and therefore had some additional resources in their SNAP account. 

 Second, we use EBT transaction data to measure HIP impacts on TFV purchases made with 

EBT cards in participating supermarkets and superstores. 

 Third, we use the participant surveys to measure HIP impacts on total food spending and on 

fruit and vegetable spending. These self-reported food spending measures encompass both 

cash and SNAP resources, in both participating and non-participating retailers. 

 Fourth, we discuss the similarities and differences between the EBT spending results and the 

participant survey spending results. 

HIP Impact on Total SNAP Spending 

Total SNAP spending recorded in EBT transaction data was slightly higher for HIP participant 

households than for non-HIP participant households (Exhibit 6.1). The original amount of SNAP 

benefits issued was not significantly different for the randomly assigned HIP and non-HIP groups. 

Total SNAP spending was higher for the HIP group because of the value of the incentives that were 

added to the SNAP account for HIP households only. Mean monthly SNAP spending was $264.11 for 

HIP households and $259.81 for non-HIP households. This small but statistically significant 

difference of $4.30 represents an increase of 1.7 percent over SNAP expenditures in the absence of 

HIP. 

This small HIP impact on total SNAP spending likely increased total food spending by HIP 

participants, although one cannot say for certain. Previous research on SNAP spending suggests that 

households that receive additional SNAP benefits offset some of their value by spending less cash on 

food (Southworth, 1945; Fox, Hamilton, and Lin, 2004; Wilde, Troy, and Rogers, 2009; Klerman, 

2013). 

HIP Impact on TFV Purchases 

In order to examine how HIP affected expenditures for TFVs, we would like to be able to compare 

TFV purchases by HIP participants to similar purchases by the non-HIP group. The following 

analysis of EBT transaction data provides some insight into that comparison, but with a limitation. 

The limitation is that such comparisons can only be done for purchases in IECR-equipped retailers, 

because these retailers automatically recorded TFV purchases for both HIP and non-HIP participants. 

In Hampden County, participating retailers included several chains of IECR-equipped 

supermarkets/superstores and only one chain of IECR-equipped convenience stores. To avoid 

disclosing company-specific information, we conducted this analysis using only data for the 

supermarkets/superstores. The IECR-equipped supermarkets/superstores also implemented HIP in 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 111

some areas outside Hampden County, notably in adjoining Massachusetts and Connecticut counties. 

SNAP household spending in these stores is captured in the EBT data. 

This analysis excludes TFVs purchased in non-supermarkets/superstores and in non-participating 

retailers. Moreover, by definition, our EBT data do not include any information about cash purchases 

of fruits and vegetables. As shown in Chapter 5 (Exhibit 5.13), 92.5 percent of all SNAP purchases 

made in HIP participating retailers and 97.7 percent of all HIP purchases in participating retailers 

were made in supermarkets/superstores, so the HIP/non-HIP comparison in supermarkets/superstores 

captures the overwhelming share of TFV purchases using SNAP benefits in participating retailers. 

In these supermarkets/superstores, non-HIP participant households had mean monthly TFV purchases 

of $10.86, and HIP participants had significantly higher mean monthly TFV purchases of $12.05, an 

increase of $1.19, or 11 percent (Exhibit 6.1). These HIP/non-HIP differences are statistically 

significant. For non-HIP households, eligible TFV purchases represented 9.3 percent of total SNAP 

purchases in participating supermarkets/superstores. For HIP households, TFV purchases were 10 

percent of SNAP purchases in these stores. 

Exhibit 6.1: Impact of HIP on Mean Monthly SNAP Issuance, SNAP Purchases and TFV 

Purchases 

Regression-adjusted 

mean (SE) Impact 

Monthly 

issuance/purchases 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

SNAP issuance ($) 260.94 

(1.07) 

260.49 

(0.42) 0.45 {1.15] {0.390} (0.696) 

SNAP purchases in all 

retailers ($) 

264.11 

(1.09) 

259.81 

(0.43) 4.30 [1.17] {3.687} (<0.001)*** 

Purchases in HIP participating supermarkets/superstores 

SNAP purchases ($) 126.51 

(1.06) 

122.17 

(0.41) 4.34 [1.13] {3.836} (<0.001)*** 

Eligible TFV purchases 

($) 

12.05 

(0.15) 

10.86 

(0.05) 1.19 [0.16] {7.600} (<0.001)*** 

 

Eligible mean household 

TFV purchases as 

percent of SNAP 

purchases (%)a

10.03 

(0.11) 

9.32 

(0.04) 0.71 [0.12] {6.197} (<0.001)*** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the household level. a

 Calculated as average of household-level percentages. 

Source: EBT Transaction Data, pooled across March-October 2012 (average of 45,912 households per month). 

In addition to these analyses of HIP impacts on TFV spending in participating 

supermarkets/superstores, Appendix H presents results by subgroups (Exhibit H6.2).67 There were 

 

67 Appendix H, Exhibit H6.2 also presents subgroup analyses of HIP impacts on SNAP purchases in 

participating supermarkets/superstores. Overall, there is little evidence of differential changes in SNAP 

purchases; the only significant difference was for residential location. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 112 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

few statistically significant differences in HIP impact across subgroups, though we did find 

differences by age of the household head, by household head race/ethnicity, household composition, 

and residential location. Given multiple comparisons issues (discussed in Chapter 2), these results 

should be viewed with caution. In particular, note that similar subgroup tests for differences in impact 

were conducted for key food consumption measures and there were no corresponding differences in 

HIP impacts (see Chapter 8). 

Self-Reported Expenditures on All Food and on Fruits and Vegetables 

Survey participants reported their usual expenditures on all groceries and more specifically on fruits 

and vegetables (Exhibit 6.2).68 In contrast with the EBT analysis in the previous section, the survey 

spending data encompass both participating and non-participating retailers, supermarkets/superstores 

and other retail formats, non-SNAP purchases (e.g., with cash), and fruits and vegetables of all kinds, 

including some that do not qualify as TFVs. No significant impacts of HIP were found on usual 

grocery spending using SNAP benefits, on usual grocery spending using cash, or on spending in 

restaurants. 

Exhibit 6.2: Impact of HIP on Self-Reported Monthly Expenditures, Linear Regression 

Model 

Regression-adjusted 

mean (SE) Impact 

Monthly expenditures ($) 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Groceries using only SNAP 

(N=31,31) 

275.16 

(4.49) 

276.84 

(3.83) -1.67 [5.90] {-0.284} (0.776) 

Groceries not using SNAP 

(N=3,051) 

149.06 

(3.83) 

148.29 

(3.91) 0.77 [5.31] {0.145} (0.885) 

Food items a (N=2,949) 105.52 

(3.34) 

106.91 

(3.33) -1.39 [4.56] {-0.304} (0.761) 

Nonfood items 

(N=2,949) 

43.30 

(1.67) 

41.26 

(1.54) 2.04 [2.30] {0.888} (0.374) 

Restaurants (N=3,088) 35.20 

(1.31) 

36.63 

(1.53) -1.43 [1.98] {-0.720} (0.472) 

All fruits and vegetables b

(N=2,708) 

78.17 

(1.95) 

72.02 

(2.06) 6.15 [2.69] {2.285} (0.022)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded 

as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Calculated as grocery expenditures not using SNAP minus expenditures on nonfood items. b

 Purchased with SNAP and with other forms of payment. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

 

68 Unless otherwise noted, the survey analysis reported in this chapter includes both Round 2 and Round 3 

interviews, clustering standard errors at the respondent level to account for non-independence of interview 

responses within respondents. Most outcomes analyzed in this chapter are either binary or categorical 

outcomes. In the chapter, we present the results of analyses based on linear probability and linear 

regression models of pooled Round 2 and Round 3 data as these models are more easily interpretable. 

Appendix F presents analyses based on logistic and ordered logistic regression models. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 113

The survey-based self-reported estimates of total grocery spending with SNAP benefits were not 

sufficiently precise to pick up the small increase in SNAP spending that EBT data showed for HIP 

participants due to the incentives earned. EBT data for all SNAP participant households in Exhibit 6.1 

showed that HIP participants had mean monthly SNAP spending $4.30 higher than non-HIP 

participants had. For the smaller survey sample in Exhibit 6.2 there was no significant HIP/non-HIP 

difference in mean self-reported grocery spending with SNAP. 

HIP positively affected self-reported spending on fruits and vegetables. HIP participants reported that 

they spent $78.17 per household per month on fruits and vegetables (across Round 2 and Round 3) 

and non-HIP households reported that they spent $72.02 per month. This difference of $6.15 per 

month is statistically significant, and represents an 8.5 percent increase in fruit and vegetable 

spending.69 Note, however, that the self-reported difference between HIP and non-HIP participants 

refers to the HIP impact on all fruit and vegetable spending—with the EBT card or with cash, at 

participating retailers and at non-participating retailers, for TFVs and for non-TFVs (e.g., white 

potatoes, fruit juice, candied yams). 

Comparing EBT Transaction Data and Survey Evidence on TFV Spending 

This section compares fruit and vegetable spending as calculated from EBT transaction data and as 

reported by survey participants. The spending measures from the two sources are defined quite 

differently, and we provide evidence on how the differences can be reconciled. We examine 

differences in the level of fruit and vegetable spending and in the HIP impact on fruit and vegetable 

spending. In order to compare EBT transaction data and survey data, we estimated models for the 

EBT food spending measures on the survey sample only (Exhibit 6.3). 

Differences in the Level of Fruit and Vegetable Spending 

The differences in the level of fruit and vegetable spending are roughly as large as one would expect, 

given the distinct food spending concepts that are captured in the two data sources. The survey-based 

measure of fruit and vegetable spending includes three types of spending that are excluded from the 

narrower EBT-based measure: 

 SNAP spending in non-participating retailers. Somewhat less than 50 percent of SNAP 

spending took place in non-participating retailers, which were excluded from the EBT-based 

estimates of fruit and vegetable spending.70 

 Grocery spending using cash income. SNAP households purchased approximately 28 percent of 

their food each month using cash income.71 

 

69 For baseline monthly expenditures of treatment and control groups, see Appendix F, Exhibit F6.1. 

70 See Chapter 5, Exhibit 5.9, for the specific estimates of SNAP redemptions in participating and nonparticipating retailers. 

71 See Exhibit 6.2 for the specific estimates of self-reported grocery spending with SNAP benefits and cash 

income. Calculation: groceries using only SNAP/total food expenditures. For HIP households: 

$275.16/($275.16+$105.52) = 72.3 percent. For non-HIP households: $276.64/($276.64+$106.91) = 72.1 

percent. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 114 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

 Fruits and vegetables that do not qualify as TFVs. A large fraction, approximately 60 percent of 

fruit and vegetable intake by volume (as measured in cup-equivalents), did not qualify as TFVs. 

The leading fruit and vegetable foods and beverages that did not qualify as TFVs were fruit juices 

and white potatoes.72 

Considering these three distinctions together, one might expect TFV spending with EBT benefits in 

participating retailers to be very roughly 14 percent as large as total fruit and vegetable spending in all 

retailers.73 In the actual spending estimates, the HIP group had $13.83 in mean monthly EBT 

spending on TFVs in participating supermarkets/superstores and $78.17 in self-reported spending on 

all fruits and vegetables in all retailers. Similarly, the non-HIP group had $12.76 in mean monthly 

EBT spending on TFVs in participating supermarkets/superstores and $72.02 in self-reported 

spending on all fruits and vegetables in all retailers. Hence, for both groups, TFV spending with EBT 

benefits in participating supermarkets/superstores was approximately 18 percent as large as total fruit 

and vegetable spending in all retailers, which is only a little higher than our rough expectation. The 

main conclusion from comparing these multiple sources of information about food spending is that 

the EBT measure is narrowly defined, so roughly 80 percent of all self-reported fruit and vegetable 

spending is not captured in the EBT estimates. 

 

72 See later Chapter 8, Exhibit 8.3, for the specific estimates of TFV intake and other fruit and vegetable 

intake. 

73 Fifty percent of spending in participating retailers * 72 percent of food spending using SNAP benefits * 40 

percent of all fruit and vegetables qualified as TFVs = 14.4. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 115

Exhibit 6.3: Impact of HIP on Monthly Expenditures, Survey-Based vs. EBT TransactionBased Measures, Linear Regression Model 

Regression-adjusted 

mean (SE) Impact 

Monthly expenditures ($) 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Survey-reported usual monthly expenditures

Groceries using only SNAP 

(N=3,131) 

275.16 

(4.49) 

276.84 

(3.83) -1.67 [5.90] {-0.284} (0.776) 

All fruits and vegetables a

(N=2,708) 

78.17 

(1.95) 

72.02 

(2.06) 6.15 [2.69] {2.285} (0.022)** 

EBT-recorded expenditures per household per month

Total EBT purchases 

(N=3,434) 

287.00 

(4.85) 

282.93 

(4.64) 4.07 [6.68] {0.610} (0.542) 

EBT purchases at HIP 

participating supermarkets/ 

superstores (N=3,434) 

146.71 

(3.79) 

141.32 

(3.44) 5.39 [5.01] {1.075} (0.282) 

EBT IECR TFV purchases 

at HIP participating IECR 

supermarkets/superstores 

(N=3,434) 

13.83 

(0.50) 

12.76 

(0.51) 1.07 [0.66] {1.614} (0.107) 

HIP incentives earned 4.15 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded 

as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Purchased with SNAP and with other forms of payment. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample and EBT 

Transaction Data, pooled across March-October 2012, for survey respondents only. 

Differences in HIP Impact on Fruit and Vegetable Spending 

A more difficult challenge is to explain the contrast between HIP impacts measured in the EBT 

transaction records and survey data. Recall that the HIP impact—that is, the HIP/non-HIP 

difference—for monthly household TFV purchases with EBT benefits in participating 

supermarkets/superstores was $1.07 (Exhibit 6.3). If HIP functioned in household budgets purely as a 

price incentive, we would expect essentially all of the increased fruit and vegetable spending to take 

place with EBT benefits in participating retailers, so we might expect this amount to represent the 

entire impact of HIP on fruit and vegetable spending. Yet, the HIP impact on self-reported total food 

spending was more than five times as large, estimated at $6.15. 

It is surprising that the HIP impact on self-reported total fruit and vegetable spending is so much 

larger than the HIP impact on TFV spending with EBT benefits in participating 

supermarkets/superstores. The conceptual framework in Chapter 1 assigned a central role to HIP’s 

financial incentive, which is only earned for TFV purchases with EBT benefits in participating 

retailers. It is challenging to explain why HIP should have had a large impact on types of fruit and 

vegetable spending that did not earn the incentive. Chapter 10 will discuss several possible 

explanations, including (1) the possibility that HIP had an educational or persuasive effect, leading 

participants to increase all types of fruit and vegetable spending and (2) the possibility that HIP 

participants misunderstood the incentive and thought they could receive the incentive for some 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 116 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

purchases that were in fact ineligible. It is likely that a combination of these explanations is 

responsible for the comparatively high estimated HIP impact on self-reported total fruit and vegetable 

spending. 

Food Spending Patterns at Different Levels of Understanding HIP 

To further address the possibility that misunderstanding the incentive could have affected the HIP 

impact estimates, we conducted a post hoc exploratory analysis of food spending and shopping 

patterns by subgroup (Appendix I). The subgroups were defined by participant self-assessment of 

how well they understood the pilot. Among HIP survey respondents, mean monthly EBT purchases of 

TFVs in participating supermarkets/superstores and superstores was $14.19 for those who found HIP 

“easy to understand,” $14.90 for those who found it “easy to understand” which fruits and vegetables 

earned incentives, and $14.95 for those who kept track of their incentive earnings. In contrast, TFV 

purchases were significantly smaller, only $12.33 among those who found HIP “not easy to 

understand,” $12.36 for those who found it “not easy to understand” which fruits and vegetables 

qualified for incentives, and $12.59 for those who did not keep track of their incentive earnings. 

Further details are in Appendix I. In contrast with the random assignment research design used for 

HIP impacts throughout this report, these comparisons are merely cross-sectional and do not show the 

“effect” or “impact” of participant understanding on HIP spending outcomes. Nonetheless, the 

analysis suggests that incomplete understanding of the pilot was associated with lower HIP spending 

on qualifying TFVs using EBT benefits.

6.2 Shopping Patterns for HIP and non-HIP Participants 

In this section, our examination of shopping patterns includes three different types of analyses: 

 First, we use EBT transaction data to study the impact of HIP on SNAP purchases in retailers 

of different types (for example, supermarkets/superstores versus smaller retailers and 

participating versus non-participating retailers). HIP turned out to have only small effects on 

these shopping patterns in the EBT data. 

 Second, we describe the changes in shopping behavior that were reported in the participant 

surveys. HIP participants reported that they changed their shopping behaviors in some 

respects, but in general HIP/non-HIP differences in self-reported shopping patterns were 

small. 

 Third, we explore whether HIP impacts on fruit and vegetable spending was influenced in 

part by variation in access to participating retailers in the baseline time period. 

HIP Impacts on Shopping Patterns in Different Retailer Types 

This section compares HIP and non-HIP participants’ SNAP purchases, including their choice of 

different types of retailers. HIP gave households an economic motivation to switch TFV purchases 

from non-participating to participating retailers because TFV purchases in non-participating retailers 

did not earn any incentives. The EBT transaction data provided information on the amount of SNAP 

purchases that took place in participating and non-participating retailers (Exhibit 6.4). 

HIP had a small but statistically significant impact on the use of SNAP benefits in retailers that 

participated in HIP and no impact on the use of SNAP benefits in non-participating retailers. For the 

non-HIP group, mean monthly purchases in participating retailers were $132.38 (representing 51 

percent of all SNAP purchases). For the HIP group, mean monthly purchases in participating retailers 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 117

were $136.80 (representing 51.8 percent of all SNAP purchases), which is $4.41 (or 3.3 percent) 

higher than those of non-HIP participants. This difference is roughly equal to our estimate of mean 

incentive earned. Thus, these estimates are consistent with the heuristic that incentives were spent 

where they were earned—in participating retailers. 

Supermarkets/superstores were the retail format in which most SNAP purchases were made. 

Approximately 80 percent of the value of all SNAP purchases made by both HIP and non-HIP 

households occurred in supermarkets/superstores (Exhibit 6.4). HIP had a significant impact on 

purchases in supermarkets/superstores. HIP participants spent $4.69 (or 2.3 percent) more in 

supermarkets/superstores than did non-HIP participants. Again, this increase is roughly equal to our 

estimate of mean incentives earned. 

Exhibit 6.4: Impact of HIP on Mean Monthly SNAP Purchases by Retailer Type 

Regression-adjusted 

mean (SE) Impact 

Monthly SNAP 

purchases ($) 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

All retailers 264.11 

(1.09) 

259.81 

(0.43) 4.30 1.17 (3.687) (<0.001)*** 

Supermarkets/ 

superstores 

210.98 

(1.11) 

206.29 

(0.43) 4.69 1.19 (3.951) (<0.001)*** 

Convenience 26.18 

(0.45) 

26.13 

(0.18) 0.05 0.48 (0.103) (0.918) 

Grocery 20.23 

0.45 

19.86 

0.18 0.37 0.48 (0.767) (0.441) 

Other 2.38 

(0.19) 

2.48 

(0.08) -0.10 0.20 (-0.481) (0.630) 

Out of State 4.35 

(0.28) 

5.06 

(0.11) -0.71 0.30 (-2.362) (0.018)** 

Participating retailers 136.80 

(1.06) 

132.38 

(0.41) 4.41 1.14 (3.879) (<0.001)*** 

Non-participating retailers 127.32 

(1.04) 

127.43 

(0.42) -0.11 1.12 (-0.101) (0.920) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the household level. 

Source: EBT Transaction Data, pooled across March-October 2012 (average of 45,912 households per month). 

Self-Reported Shopping Patterns 

Self-reported shopping patterns from the participant survey were consistent with the findings from 

EBT data indicating little change in shopping patterns. One notable exception was that a little more 

than a quarter of HIP participants reported shopping at “a different store to buy fruits and vegetables.” 

While this response is striking, whatever changes the respondents had in mind were not perceptible in 

other measures of shopping patterns, as discussed below. 

To examine the impact of HIP on self-reported general shopping patterns from survey data, we first 

compared the responses of HIP participants and non-participants concerning their usual grocery store 

type (Exhibit 6.5). Across the nine store types examined, ranging from chain supermarkets to 

convenience stores to farmers markets, no significant differences emerged. HIP participants and nonHIP participants did not differ in their usual grocery shopping location; similar to findings from the 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 118 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

EBT data, for both groups, the most common store type was a large chain grocery store or 

supermarket.74

Exhibit 6.5: Impact of HIP on Self-reported Usual Grocery Store Type, Linear Probability 

Model

Regression-adjusted 

mean (SE) Impact 

Usual place to shop 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Large chain grocery store 

or supermarket 

0.815 

(0.011) 

0.816 

(0.010) -0.001 [0.015] {-0.052} (0.958) 

Discount superstore (such 

as Walmart) 

0.125 

(0.009) 

0.118 

(0.008) 0.007 [0.012] {0.573} (0.567) 

Small local store or corner 

store 

0.022 

(0.004) 

0.026 

(0.004) -0.004 [0.006] {-0.574} (0.566) 

Warehouse club store 

(such as Sam’s Club or 

Costco) 

0.018 

(0.004) 

0.023 

(0.004) -0.006 [0.006] {-0.976} (0.329) 

Natural or organic 

supermarket (such as 

Whole Foods Market) 

0.006 

(0.002) 

0.005 

(0.002) 0.001 [0.003] {0.559} (0.577) 

Ethnic market 0.004 

(0.002) 

0.006 

(0.002) 0.002 [0.002] {-0.944} (0.345) 

Farmers market/co-op 0.003 

(0.002) 

0.001 

(0.001) 0.002 [0.002] {0.851} (0.395) 

Convenience store (such 

as 7-11 or mini market) 

0.002 

(0.001) 

0.001 

(0.001) 0.001 [0.001] {0.414} (0.679) 

Some other locationa 0.003 

(0.002) 

0.003 

(0.001) >-0.001 [0.002] {0.048} (0.962) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibit F6.3 presents the logistic regression model. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample; (unweighted 

N=3,298). 

We then examined the impact of HIP on the reasons given by respondents for choosing their usual 

grocery store type (Exhibit 6.6). For both HIP and non-HIP households, the most common reasons 

were: prices/affordability (about 50 percent); store was close to home (about one-third); and variety 

of products available (about 20-25 percent). All other reasons were reported by fewer than 20 percent 

of all households. 

 

74 For baseline proportions of treatment and control groups who shopped at these locations, see Appendix F, 

Exhibit F6.2. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 119

Exhibit 6.6: Impact of HIP on Reasons for Choice of Usual Grocery Shopping Place, Linear 

Probability Model

Regression-adjusted 

mean (SE) Impact 

Usually shop there 

because... 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Prices/affordability 0.483 

(0.015) 

0.512 

(0.014) -0.030 [0.020] {-1.507} (0.132) 

Close to home 0.309 

(0.013) 

0.324 

(0.012) -0.015 [0.018] {-0.805} (0.421) 

Variety of products 0.242 

(0.012) 

0.200 

(0.010) 0.042 [0.016] {2.711} (0.007)*** 

Produce better or fresher 0.161 

(0.010) 

0.134 

(0.009) 0.028 [0.013] {2.145} (0.032)** 

Sales/promotions in store 0.100 

(0.008) 

0.132 

(0.009) -0.032 [0.012] {-2.570} (0.010)** 

Preferred products are 

available 

0.095 

(0.008) 

0.128 

(0.008) -0.033 [0.011] {-2.941} (0.003)*** 

Familiarity with store 0.069 

(0.007) 

0.056 

(0.007) 0.013 [0.010] {1.242} (0.214) 

One-stop shopping 0.058 

(0.006) 

0.056 

(0.006) 0.002 [0.008] {0.194} (0.846) 

Quality 0.041 

(0.005) 

0.049 

(0.006) -0.009 [0.007] {-1.169} (0.243) 

EBT card accepted 0.029 

(0.005) 

0.015 

(0.003) 0.014 [0.006] {2.412} (0.016)** 

Good service 0.027 

(0.004) 

0.018 

(0.003) 0.010 [0.006] {1.714} (0.087)* 

Easy to get there 0.023 

(0.004) 

0.018 

(0.003) 0.005 [0.005] {0.977} (0.329) 

Bulk purchases 0.021 

(0.004) 

0.025 

(0.004) -0.004 [0.006] {-0.592} (0.554) 

Clean 0.020 

(0.004) 

0.017 

(0.004) 0.003 [0.005] {0.568} (0.570) 

Hours of operation 

convenient 

0.006 

(0.002) 

0.008 

(0.002) -0.001 [0.003] {-0.442} (0.659) 

Ethnic foods are available 0.006 

(0.002) 

0.008 

(0.002) -0.001 [0.003] {-0.383} (0.701) 

Close to work 0.006 

(0.002) 

0.004 

(0.001) 0.002 [0.003] {0.836} (0.403) 

Close to some other 

location 

0.004 

(0.002) 

0.004 

(0.001) <0.001 [0.002] {0.054} (0.957) 

Disability accessible 0.003 

(0.001) 

0.001 

(0.001) 0.002 [0.002] {1.147} (0.251) 

Some other reason 0.009 

(0.003) 

0.007 

(0.002) 0.002 [0.004] {0.483} (0.629) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=Yes, 0=No; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibit F6.5 presents the logistic regression model. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample; (unweighted 

N=3,281). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 120 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

Despite the lack of any significant differences in their usual grocery store type, HIP participants were 

more likely than non-participants to say they chose their usual store type because of the variety of 

products, better or fresher produce, and EBT card acceptance. They were less likely to say they chose 

their usual store type because of sales or promotions in the store,75 because their preferred products 

were available, and, among those who consumed three or more servings of fruits and vegetables a day 

at baseline, because of quality (see Appendix H Exhibit H6.6j).76 This pattern contains some hints, 

though hardly conclusive evidence, that HIP might have been influencing the choice of stores. If HIP 

participants were purchasing more fruits and vegetables and aiming to earn incentives, they may have 

chosen stores because of more variety, better produce, and acceptance of EBT cards.77 However, 

given the large number of tests, these responses can be considered no more than suggestive of an 

impact. 

HIP did not have a significant impact on reported grocery shopping frequency or the likelihood that 

participants would go out of their way or make a special effort to shop at a particular store for fruits 

and vegetables (Exhibit 6.7). Both HIP and non-HIP participants went grocery shopping about every 

other week, on average, and HIP and non-HIP participants said they were equally likely to go out of 

their way or make a special effort to shop at a particular store for fruits and vegetables.78

Aside from a few significant impacts on reasons for the choice of a usual grocery store type, survey 

results suggest that HIP did not have an impact on participants’ general shopping patterns as 

described above. This lack of impact on general shopping patterns was consistent with comments in 

HIP participant focus groups, which indicated that their shopping habits had not changed, and their 

shopping location was primarily determined by lower prices overall, rather than a store’s participation 

in HIP. As one participant noted, “Pretty much, I just have noticed I’ve stuck to my old habits.” 

 

75 This was the case primarily among those who frequently had fruits and vegetables in the home at baseline 

(see Appendix H, Exhibit H6.6e). 

76 See Appendix G, Exhibit G6.5 for some significant differences between Round 2 and Round 3 in impacts 

of HIP on reasons for choice of usual shopping location. 

77 For baseline proportions of treatment and control groups who chose their usual shopping place for different 

reasons, see Appendix F, Exhibit F6.4. 

78 For baseline statistics on reported shopping frequency and proportion who went out of their way to shop at 

a particular store, for both treatment and control groups, see Appendix F, Exhibits F6.6 and F6.7. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 121

Exhibit 6.7: Impact of HIP on Grocery Shopping Behaviors, Linear Regression Model 

Regression-adjusted 

mean (SE) Impact 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Grocery shopping 

frequencya (N=3,311) 

5.239 

(0.026) 

5.218 

(0.024) 0.020 [0.035] {0.588} (0.557) 

Go out of way to shop for 

fruits and vegetables at 

particular storeb

 (N=3,283) 

0.387 

(0.014) 

0.402 

(0.013) -0.015 [0.019] {-0.818} (0.413) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibits F6.8 and F6.10 present the logistic regression model; Exhibit F6.9 presents the ordered 

logistic regression model. 

a

Categorical outcome, 1=yearly or not at all, 2=2 to 3 times a year, 3=every other month, 4=once a month, 

5=every other week, 6=once a week, 7=more than once a week; “don’t know” and “refused” responses coded as 

missing. 

b

Binary outcome, 1=yes, 0=no; “don’t know” and “refused” responses coded as missing. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

We also examined HIP participants’ reports of HIP-induced changes in their shopping habits. We 

specifically focused on changes related to purchasing fruits and vegetables and changes in shopping 

location. While HIP participants were asked about their perception of changes in shopping habits due 

to HIP, the control group was not asked about their changes in shopping habits. Thus we cannot 

conclude that the changes reported in this section are true impacts of HIP.

More than 70 percent of HIP participant households reported that fruits and vegetables had become 

more affordable to the family due to HIP (Exhibit 6.8). Consistent with this survey finding, one focus 

group participant noted that, “It helps a little bit, you know.” Another echoed that HIP helped with 

day-to-day purchasing, “Yeah, it helps. I can’t say it doesn’t.” A majority (over 60 percent) of HIP 

participants also felt that their family was buying a larger amount and a greater variety of both fruits 

and vegetables due to HIP, and those reporting buying and eating a greater variety of fruits also 

significantly increased over time (from Round 2 to Round 3). Consistent with this survey finding, 

some focus group participants noted that they had started eating more and a greater variety of fruits 

and vegetables, and that the program has encouraged these healthier eating behaviors in their 

households. As one focus group respondent commented, “It’s causing better eating habits...for my 

health I have to start eating right. Now everyone else is getting to eat more vegetables and more fruit 

because I have to purchase more of them,” and another, “It’s not the typical apples and oranges 

anymore—you got the apples, the oranges, the grapes, the cantaloupe, a variety.” 

Less than half of HIP participants reported that HIP resulted in their family buying new fruits and 

vegetables that they had not tried before. However, this proportion of participants who reported 

buying new vegetables they had not tried before, because of HIP, significantly increased over time, 

and a focus group participant did note, “I’m trying lots of different vegetables and fruits that I never 

tried before. Before this program, I wasn’t even thinking about buying fruits and vegetables like I am 

now, I’m more conscious about it now.” In summary, participants reported that HIP was having the 

desired effects on increasing the amount and variety of fruits and vegetables purchased, as well as 

their affordability. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 122 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

Exhibit 6.8: Primary Shopper Self-Reported Changes in Fruit and Vegetable Purchasing 

Due to HIP: Round 2 & 3 Participant Surveys, HIP Participants 

Because of the Healthy Incentive 

Pilot rebates... 

Proportion (N) Change 

Pooled Round 2 Round 3 (P-value) 

Fruits and vegetables have become 

more affordable to family (N=1,571) 

0.73 

(974) 

0.71 

(447) 

0.75 

(527) 

0.03 

(0.093)* 

Fruits 

Family is buying larger amounts of 

fruits (N=1,590) 

0.64 

(866) 

0.62 

(397) 

0.67 

(469) 

0.04 

(0.068)* 

Family has bought and eaten greater 

variety of fruits (N=1598) 

0.65 

(888) 

0.63 

(404) 

0.68 

(484) 

0.04 

(0.036)** 

Family is buying new fruits not tried 

before (N=1,603) 

0.48 

(660) 

0.46 

(293) 

0.50 

(367) 

0.04 

(0.080)* 

Vegetables 

Family is buying larger amounts of 

vegetables (N=1,599) 

0.63 

(864) 

0.63 

(403) 

0.64 

(461) 

0.02 

(0.311) 

Family has bought and eaten greater 

variety of vegetables (N=1604) 

0.63 

(870) 

0.62 

(404) 

0.64 

(466) 

0.02 

(0.328) 

Family is buying new vegetables not 

tried before (N=1,600) 

0.44 

(609) 

0.42 

(267) 

0.47 

(342) 

0.06 

(0.023)** 

Weighted portions (unweighted Ns). 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported Round 2/3 change may differ from differences between reported proportions for 

Rounds 2 and 3. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

In terms of shopping location, 72 percent of HIP participants (pooled across Rounds 2 and 3), 

reported that they had not changed their shopping location for fruits and vegetables due to HIP 

(Exhibit 6.9). The other 28 percent reported some change, though from the survey responses we 

cannot determine whether they began shopping in a new store or just shifted some of their fruit and 

vegetable purchases to a store they were already patronizing. The percentage reporting a change was 

slightly higher for HIP participants that had favorable baseline attitudes toward fruits and vegetables 

(31 percent) than for those who did not have favorable baseline attitudes (23 percent) (Appendix H, 

Exhibit H6.8). Because the EBT analysis in the preceding section found comparatively small 

HIP/non-HIP differences in food spending by retailer type, and because this section found 

comparatively small HIP/non-HIP differences in usual shopping locations, the most plausible 

interpretation is that HIP participants generally made only small changes to their choice of shopping 

location. Those who changed stores reported that they did so primarily because of the price of fruits 

and vegetables, the availability of fresh fruits and vegetables, and the greater variety of fruits and 

vegetables. Those who shopped primarily at HIP participating retailers at baseline were more likely to 

endorse these reasons for changing stores than were those who shopped primarily at non-HIP 

participating retailers (see Appendix H, Exhibits H6.9a-H6.9c). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 123

Exhibit 6.9: Primary Shopper Self-Reported Changes in Shopping Location Due to HIP: 

Round 2 & Round 3 Participant Surveys, HIP Participants 

Proportion (N) 

Because of HIP, have you changed which stores you go to, to buy fruits and vegetables? (N=1,611) 

Yes 0.28 

(445) 

No 0.72 

(1166) 

If yes, why have you changed which stores you go to? a 

Price of fruits & vegetables more affordable at other store (N=442) 

0.77 

(342) 

Other store has fresh fruits & vegetables (N=442) 

0.75 

(333) 

Other store has greater variety of fruits & vegetables (N=444) 

0.65 

(287) 

Other store participates in HIP (N=443) 

0.22 

(100) 

Other reason (N=443) 

0.05 

(23) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

 “Don’t know” and “refused” responses coded as missing. 

a

Respondents could choose multiple reasons, so proportions do not add to one. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

Relationship Between Pre-Implementation Shopping Patterns and HIP Impacts 

Previous results in this chapter suggested that HIP had only small impacts on participants’ choice of 

retailers, while hinting that HIP may have had impacts on food choices within particular retailers. 

Taken together, these results raise the possibility that lack of access to participating retailers could 

have limited HIP impacts on food spending.79 If HIP participants did not change where they shopped 

during the pilot, then those who usually patronized non-participating retailers could have had fewer 

opportunities to earn incentives resulting in lower HIP impacts. 

To explore this possibility, this section uses EBT data to compare HIP impacts on TFV spending for 

two groups of participants, distinguished by where they spent most of their SNAP benefits during the 

pre-implementation period. The hypothesis is that HIP could have had a stronger impact on TFV 

spending for SNAP participants who were patronizing participating retailers even before the pilot 

began. 

This analysis used data from November 2011, the first month of the pilot, to study the preimplementation shopping patterns for a subset of households. HIP was implemented in three waves 

in, respectively, November 2011, December 2011, and January 2012. For HIP participants in Waves 2 

and 3, the EBT data for November 2011 describe shopping patterns prior to their participation in HIP. 

 

79 The evaluation explored issues of geographic access more directly in a separate report: Healthy Incentives 

Pilot (HIP) Spatial Analysis (2014). The analysis studied how HIP impacts varied with the distance to the 

nearest retailers and the distance to the retailers where SNAP participants spent most of their benefits. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 124 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

We divided the Wave 2 and Wave 3 SNAP households into two groups, based on their preimplementation shopping patterns. The two shopping groups, which were approximately equal in 

size, were defined as: 

 Pre-HIP participating store shoppers: participant households who made at least 50 percent 

of their November 2011 (pre-implementation) SNAP purchases in participating retailers.

 Pre-HIP non-participating store shoppers: participant households who made less than 50 

percent of their November 2011 (pre-implementation) SNAP purchases in participating 

retailers. 

Wave 1 households and those that had no SNAP spending in November 2011 were omitted from this 

additional analysis. A retailer counted as “participating” if it was participating in HIP sometime 

during the March-October 2012 period.80 

For both shopping groups, we used post-implementation EBT data pooled across March to October, 

2012 to repeat our analysis of HIP impacts on shopping patterns and TFV purchases in participating 

supermarkets/superstores. Note that the two shopping groups were defined based on households’ preimplementation shopping patterns, while the analysis of HIP impacts was based on their postimplementation outcomes. 

Not surprisingly, monthly TFV purchases in participating supermarkets and superstores during the 

pilot were much higher for households that primarily shopped in participating stores during the preimplementation period. Among HIP households, pre-HIP participating store shoppers purchased an 

average of $14.44 of TFVs each month, compared with $9.50 for pre-HIP non-participating store 

shoppers (Exhibit 6.10). This pattern is consistent with the notion that HIP households did not make 

major changes in where they shopped during the pilot. 

However, even though HIP purchases were greater for households that primarily shopped in 

participating stores prior to HIP, the HIP impact—the difference between purchases of treatment and 

control households—was not larger for these households than for households that primarily shopped 

in non-participating stores prior to HIP. Looking at the households who already shopped in 

participating supermarkets/superstores, the HIP impact on TFV purchases in these stores was $1.23 

(Exhibit 6.10). In contrast, for households that did not already shop at participating stores prior to 

HIP, the impact was $1.78. We had hypothesized that the HIP impact would be greater for households 

that were already shopping at retailers that later ended up participating in the pilot. However, this 

hypothesis did not turn out to be correct. This may suggest that some households who had previously 

shopped mainly in non-participating stores shifted some of their TFV purchases to participating 

stores. 

 

80 Appendix F, Exhibit F6.11 presents descriptive statistics of location, SNAP benefits, SNAP purchases for 

the two samples. Pre-HIP participating store shoppers were more likely to be located in the 

Chicopee/Holyoke region, while pre-HIP non-participating store shoppers were more likely to be located in 

Springfield or in the “other” (more rural) parts of the county. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 125

Exhibit 6.10: Impact of HIP on Mean Monthly SNAP and TFV Purchases at Participating 

Supermarkets and Superstores, by November 2011 (Pre-Implementation) 

Shopping Behavior

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

SNAP purchases ($) 

Pre-HIP participating store 

shoppers 

153.92 

 (1.67) 

148.53 

 (0.64) 

5.40 

 (0.003)*** 

Pre-HIP non-participating 

store shoppers 

90.41 

 (1.80) 

85.91 

 (0.68) 

4.50 

 (0.019)** 

Impact: participating – nonparticipating store shoppers 0.90 

 (0.732) 

Eligible TFV purchases ($) 

Pre-HIP participating store 

shoppers 

14.44 

 (0.25) 

13.21 

 (0.09) 

1.23 

 (<0.001)*** 

Pre-HIP non-participating 

store shoppers 

9.50 

 (0.28) 

7.71 

 (0.09) 

1.78 

 (<0.001)*** 

Impact: participating – non-participating store 

shoppers -0.56  (0.155) 

Eligible TFV purchase as % of SNAP Purchases a

Pre-HIP participating store 

shoppers 

9.94 

 (0.16) 

9.38 

 (0.06) 

0.56 

 (0.002)*** 

Pre-HIP non-participating 

store shoppers 

10.82 

 (0.27) 

9.46 

 (0.09) 

1.37 

 (<0.001)*** 

Impact: participating – nonparticipating store shoppers -0.80 

 (0.014)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the household level. 

Pre-HIP participating store shoppers are Wave 2 and Wave 3 households who spent at least 50 percent of their 

November 2011 (pre-implementation) SNAP benefits in retailers that later participated in HIP during MarchOctober 2012 (post-implementation). Pre-HIP non-participating store shoppers are Wave 2 and Wave 3 

households who spent less than 50 percent of their November 2011 (pre-implementation) SNAP benefits in 

retailers that later participated in HIP during March-October 2012 (post-implementation). a

Calculated as average of household-level percentages. 

Source: EBT Transaction Data, pooled across March-October 2012 (average of 29,846 Wave 2 and 3 

households per month that were identified as pre-HIP participating and non-participating store shoppers). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 126 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

6.3 HIP Spending in Farmers Markets 

A distinctive feature of HIP was the provision of a financial incentive directly through SNAP clients’ 

EBT cards, which could be used in participating retailers of all types, including farmers markets. 

There have been previous efforts, through “Bounty Bucks” or “Double Up Bucks” programs,81 to 

provide SNAP participants with incentives for fruit and vegetable purchases specifically through 

farmers markets. This section explores SNAP spending and HIP purchases in farmers markets in 

Hampden County. 

Although the farmers market SNAP purchases were quite small, this topic was salient because DTA 

and community partners had made considerable effort to ensure that almost all farmers markets were 

included in the pilot. We therefore wanted to assess whether HIP affected farmers market sales. 

Only a small fraction of SNAP households—less than half of 1 percent—shopped in farmers markets 

during any given month (Exhibit 6.11). As one would expect, the highest spending months for 

farmers markets was when farmers markets were most active, from July through October 2012.82 For 

HIP participants, use of farmers markets peaked at about 0.35 percent of all households. For non-HIP 

participants, use of farmers markets peaked just slightly lower at 0.32 percent of all households. One 

farmer who sold crops in seven different markets commented that “some people still feel stigma 

behind EBT [card purchases]” and that “people will sometimes go to markets outside the community 

to use their EBT card.” Perhaps in farmers markets where the majority of transactions were cash, 

households felt more stigma using EBT cards. In addition, the electronic modes of implementation (as 

opposed to the use of tokens) used in two of the farmers markets we observed were relatively new 

and farmers commented that HIP transactions took considerable time. This may have discouraged 

some HIP participants from using their benefits in farmers markets. 

 

81 For additional information see: http://wholesomewave.org/wholesomewaveresearch/.

82 The spike in October in the percent of HIP participants who shopped at farmers markets represents a very 

small number of households, about 20 more than would be expected from the control group percentage. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 127

Exhibit 6.11: Percent of Households That Shopped at Farmers Markets 

Source: EBT Transaction Data. HIP shoppers: average of 5,800 households per month; non-HIP shoppers: 

average of 40,640 households per month. 

Averaged across all SNAP participant households, including the large number who did not patronize 

farmers markets, the mean SNAP spending in farmers markets amounted to 1 to 4 cents per month. 

We did not find significant HIP/non-HIP differences in SNAP purchases in farmers markets (see 

Appendix F, Exhibit F6.12). 

From the perspective of the farmers markets, SNAP participants may represent a promising, and so 

far little-tapped, market. In the peak month (August 2012), the SNAP participant cohort (both HIP 

and non-HIP) represented in our EBT analysis spent almost $3,000 at Hampden County farmers 

markets (Exhibit 6.12). Because the non-HIP households were more numerous than HIP households 

overall, most of these farmers market sales came from non-HIP households. 

0.00%

0.05%

0.10%

0.15%

0.20%

0.25%

0.30%

0.35%

0.40%

Percent of HIP Shoppers that Shopped at Farmers Markets

Percent of Non-HIP Shoppers that Shopped at Farmers Markets
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 128 ▌6. Effects on Expenditures and Shopping Behaviors Abt Associates

Exhibit 6.12: Total SNAP Sales at Farmers Markets 

Source: EBT Transaction Data. HIP shoppers: average of 5,800 households per month; non-HIP shoppers: 

average of 40,640 households per month. 

Because they provide an array of fresh TFVs, farmers markets remain a promising retail format, so it 

is valuable that great effort was made to include farmers markets in HIP. Yet, the large majority of 

SNAP purchases of fruits and vegetables took place in supermarkets/superstores, which remains the 

central venue for reaching SNAP participants with financial incentives. At least in Hampden County, 

a fruit and vegetable incentive program that focused only on farmers markets would reach only a tiny 

fraction of SNAP households. 

6.4 Discussion 

HIP had significant effects on households’ expenditures on fruits and vegetables. EBT data showed 

that HIP households spent $1.19 more per month than did non-HIP households using EBT benefits on 

targeted fruits and vegetables in participating supermarkets/superstores. Survey respondents were 

asked about their total monthly expenditures for all fruits and vegetables, including spending that 

would not have qualified for HIP incentives. HIP households reported spending $6.15 more per 

month on all fruits and vegetables than did non-HIP households. 

One interpretation of the difference between the EBT estimates and the self-reported spending 

estimates is that HIP participants increased fruit and vegetable spending even when not earning the 

incentives, because the purchases were in stores that were not participating in HIP, were conducted 

using cash, or were for fruits and vegetables that did not qualify for the incentive. HIP’s impact on 

total self-reported fruit and vegetable spending is more than five times as large as HIP’s impact on the 

particular TFV purchases that earned the incentive. We return to a discussion of this divergence in 

Chapter 10. 

$0

$500

$1,000

$1,500

$2,000

$2,500

$3,000

Total SNAP Sales at Farmers Markets from HIP Households

Total SNAP Sales at Farmers Markets from Non-HIP Households
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 6. Effects on Expenditures and Shopping Behaviors ▌pg. 129

This chapter also examined the shopping patterns of HIP and non-HIP participants during the pilot. 

EBT data showed a small but significant increase in SNAP spending in retailers participating in HIP, 

and no significant impact on SNAP spending in non-participating retailers. Survey analyses generally 

corroborated the finding that HIP impacts on shopping patterns were small. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 7. Impacts on Attitudes and Family Food Environment ▌pg. 131

7. Impacts on Attitudes Toward Fruits and Vegetables and Family 

Food Environment 

This chapter analyzes the effects of HIP on participants’ personal attitudes toward fruits and 

vegetables and on the family food environment. The participant survey collected multiple measures, 

including: 

 Exposure to nutrition education and promotion 

 Food preferences and beliefs 

 Perceived barriers to the consumption of fruits and vegetables 

 Barriers to grocery shopping 

 Family food environment 

HIP’s financial incentive was the main mechanism by which the pilot was expected to influence fruit 

and vegetable intake. Yet, substantial literature supports the effects of attitudes, behaviors, and the 

family food environment, and the conceptual framework for this study recognized the role of these 

factors. 

The analysis in the chapter is based primarily on participant survey data. Focus group findings are 

incorporated when relevant. The next five sections of the chapter discuss the impact of HIP on the 

five attitudinal outcomes measured in the participant surveys. The surveys asked multiple questions to 

capture participants’ attitudes, and we estimate the impact of HIP on all the individual measures. In 

addition, to improve measurement precision and reliability, we created and analyzed composite scales 

based on multiple survey items. Specifically, we created scales to reflect: (1) positive attitudes toward 

food, fruits, and vegetables; (2) perceived barriers to fruit and vegetable consumption; (3) perceived 

barriers to grocery shopping; and (4) availability of fruits and vegetables in the home.83 

Within each section, the analysis centers on regression-adjusted comparisons of HIP and non-HIP 

responses, combining reports from both post-implementation survey rounds (Round 2 and Round 3). 

Where relevant, we discuss changes in impacts between Rounds 2 and 3 of the study and also any 

differences in impacts across subgroups of the population. 84 

The outcomes analyzed in this chapter are either binary or categorical. For ease of interpretation, the 

estimates presented here are based on linear probability and linear regression models. Appendix F 

 

83 Appendix E, Section E.4 discusses scale creation. 

84 Appendix F, Exhibits F7.1–F7.4, presents baseline responses to all questions analyzed in this chapter. As 

would be expected if randomization was properly implemented, there are few statistically significant 

differences between the treatment and control groups in these baseline measures. Appendix G presents 

separate results for Round 2 and Round 3 and tests for significant changes in the impact estimates. 

Appendix H presents subgroup analyses for the outcomes presented in this chapter. Specifically, we 

considered differential impacts for those subgroups for which we found significant differential impact of 

HIP on consumption variables (current or preliminary analyses). These differences are discussed in the 

main text only when consistent patterns emerged that provided context for the pooled findings. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 132 ▌7. Impacts on Attitudes and Family Food Environment Abt Associates

presents supplementary analyses based on logistic and ordered logistic regression models; results are 

similar to those presented here. 

7.1 Exposure to Nutrition Education and Promotion 

HIP participants were significantly more likely to say they had heard or seen messages about the 

importance of eating fruits and vegetables as part of a healthy diet. As shown in Exhibit 7.1, 76 

percent of HIP participants compared to 68 percent of non-HIP participants said they had received 

this type of message in the prior three months. By contrast, few participants in either group attended 

formal nutrition education classes or programs, and there was no significant difference across the 

groups. These findings—impacts on messages, but not on nutrition education—are not surprising 

given that the pilot included multiple occasions in which DTA communicated with HIP participants 

about targeted fruits and vegetables, but the pilot design included no formal nutrition education 

component. 

Exhibit 7.1: Impact of HIP on Self-Reported Exposure to Nutrition Education and Promotion 

in Past Three Months, Linear Probability Model 

Regression-adjusted mean 

(S.E.) Impact 

Treatment (T)

Control 

(C) T-C [S.E.] 

{tstatistic} (P-value) 

Heard or seen messages about 

fruits & vegetables (N=3,392) 

0.755 

(0.013) 

0.679 

(0.012) 0.077 [0.018] {4.331} (<0.001)*** 

Attended nutrition education 

class or program (N=3,406) 

0.101 

(0.009) 

0.108 

(0.008) -0.007 [0.012] {-0.574} (0.566) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibit F7.5 presents the logistic regression model. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 

7.2 Food Preferences and Beliefs 

HIP appears to have had little, if any, impact on the food preferences and beliefs of SNAP 

participants (Exhibit 7.2). While HIP participants appeared to agree more strongly that they enjoy 

trying new foods, 85 the difference was only borderline significant. HIP participants were no more 

likely than non-HIP participants to report that they like trying new fruits and vegetables, that they eat 

enough fruits and vegetables to keep healthy or that they often encourage family and friends to eat 

fruits and vegetables. Similarly, the composite scale reflecting positive attitudes toward food, fruits, 

and vegetables was not significantly different for HIP participants and non-participants. 

Subgroup analyses, however (see Appendix H), indicated that HIP may have had an impact on some 

food preferences and beliefs for those who consumed three or more servings a day of fruits and 

 

85 Items were measured on a scale from 1 to 5, thus mean scores closer to 5 indicate that a higher percentage 

strongly agree and mean scores closer to 1 indicate that a higher percentage strongly disagree (e.g., a mean 

score of 5.0 would indicate that 100 percent said “strongly agree”). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 7. Impacts on Attitudes and Family Food Environment ▌pg. 133

vegetables at baseline.86 Among these respondents, HIP participants subsequently had more positive 

attitudes than non-HIP participants toward food, fruits, and vegetables, as indicated by the composite 

scale (Exhibit H7.2g; subgroup difference significant at p < .10). Specifically, HIP participants who 

consumed three or more servings a day of fruits and vegetables at baseline more strongly agreed than 

did non-HIP participants that they enjoyed trying new vegetables (Exhibit H7.2c) and that they 

encouraged friends and family to eat fruits and vegetables (Exhibit H7.2f). By contrast, among 

respondents who consumed less than three servings a day of fruits and vegetables at baseline, HIP 

participants were no different from non-HIP participants with respect to these attitudes. 

Exhibit 7.2: Impact of HIP on Food Preferences and Beliefs, Linear Regression Model

How much do you agree or 

disagree that...? 

Regression-adjusted mean 

(S.E.) Impact 

Treatment 

(T) 

Control 

(C) T-C [S.E.] 

{tstatistic} (P-value) 

I enjoy trying new foods 

(N=3,381) 

3.900 

(0.022) 

3.834 

(0.024) 0.066 [0.034] {1.953} (0.051)* 

I enjoy trying new fruits 

(N=3,388) 

3.903 

(0.023) 

3.934 

(0.023) -0.031 [0.032] {-0.961} (0.337) 

I enjoy trying new vegetables 

(N=3,392) 

3.680 

(0.025) 

3.651 

(0.025) 0.029 [0.036] {0.818} (0.414) 

I eat enough fruits to keep me 

healthy (N=3,383) 

3.729 

(0.025) 

3.688 

(0.025) 0.042 [0.035] {1.182} (0.237) 

I eat enough vegetables to 

keep me healthy (N=3,393) 

3.735 

(0.026) 

3.693 

(0.023) 0.042 [0.035] {1.215} (0.225) 

I often encourage family/friends 

to eat fruits & veg. (N=3,346) 

3.866 

(0.024) 

3.819 

(0.024) 0.047 [0.034] {1.388} (0.165) 

Composite scale – Positive 

attitudes about food, fruits, & 

vegetables (N=3,392) 

3.802 

(0.015) 

3.768 

(0.015) 0.034 [0.021] {1.638} (0.102) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly 

agree; “don't know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibit F7.6 presents the ordered logistic regression model; Exhibit F7.7 presents the logistic 

regression model. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 

7.3 Perceived Barriers to Fruit and Vegetable Consumption 

The survey examined numerous perceived barriers to consuming fruits and vegetables, including 

difficulty in finding and preparing these foods, cost and spoilage, and household members disliking 

fruits and vegetables. Overall, cost and problems eating fruits and vegetables before they spoiled were 

the biggest barriers to fruit and vegetable consumption seen by both the HIP and non-HIP groups 

(Exhibit 7.3). 

 

86 See discussion in Chapter 8, Section 8.6 for details on how the subgroups were created. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 134 ▌7. Impacts on Attitudes and Family Food Environment Abt Associates

In addition to the participant survey, focus group discussions also raised concerns about the cost of 

fruits and vegetables. Focus group respondents noted that the high cost of fruits and vegetables 

limited their purchases. Representative comments included: “Fresh fruits and vegetables are 

wonderful, but they can be very expensive. You can get a quart of good quality apple juice for the 

price of two apples. It’s less expensive to buy apple juice.” 

Exhibit 7.3: Impact of HIP on Perceived Barriers to Fruit and Vegetable (FV) Consumption, 

Linear Regression Model

How much do you agree or 

disagree that...? 

Regression-adjusted mean 

(S.E.) Impact 

Treatment 

(T) 

Control 

(C) T-C [S.E.] 

{tstatistic} (P-value) 

Hard to eat vegetables 

because don't know how to 

prepare (N=3,366) 

2.388 

(0.029) 

2.398 

(0.026) -0.010 [0.039] {-0.255} (0.799) 

Hard to eat vegetables 

because hard to find where I 

shop (N=3,369) 

2.186 

(0.023) 

2.208 

(0.022) -0.022 [0.032] {-0.672} (0.502) 

Hard to eat fruits because 

hard to find where I shop 

(N=3,372) 

2.164 

(0.023) 

2.152 

(0.021) 0.011 [0.031] {0.361} (0.718) 

Don't eat FV as much as 

would like because cost too 

much (N=3,379) 

2.921 

(0.031) 

2.941 

(0.030) -0.020 [0.043] {-0.470} (0.639) 

Don't eat FV as much as 

would like because they spoil 

(N=3,372) 

2.804 

(0.029) 

2.899 

(0.029) -0.094 [0.041] {-2.281} (0.023)** 

Don't eat FV as much as 

would like because family 

dislikes (N=3,230) 

2.139 

(0.023) 

2.106 

(0.021) 0.034 [0.031] {1.081} (0.280) 

Don't eat FV as much because 

I don't like (N=3379) 

2.028 

(0.020) 

2.059 

(0.021) -0.030 [0.029] {-1.060} (0.289) 

Composite scale–barriers to 

eating fruits & vegetables 

(N=3,358) 

2.372 

(0.015) 

2.396 

(0.014) -0.024 [0.021] {-1.178} (0.239) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly 

agree; “don't know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibit F7.8 presents the ordered logistic regression model; Exhibit F7.9 presents the logistic 

regression model. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 

HIP had no statistically significant impact on the composite scale measure of barriers to fruit and 

vegetable consumption. A statistically significant impact was found for only one of the seven 

individual perceived barriers to consumption: HIP participants were significantly less likely than nonHIP participants to agree to a statement that they ate less fruits and vegetables because they spoil. 

Given the large number of comparisons and the general pattern of non-significant results, we are not 

inclined to give much weight to this one significant finding. It is noteworthy that we see no 

significant difference in agreement to the statement, “I don’t eat fruits and vegetables as much as I 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 7. Impacts on Attitudes and Family Food Environment ▌pg. 135

like to because they cost too much,” as this suggests that the HIP price incentive was not strongly 

affecting participants’ perceptions of costs. Taken as a whole, these findings provide little evidence 

that HIP affected perceived barriers to eating fruits and vegetables. 

7.4 Barriers to Grocery Shopping 

The participant survey investigated whether HIP affected barriers to local food retail access. The 

presence of supermarket “food deserts” or areas with limited supermarket access could in principle 

have reduced the effectiveness of the pilot. However, primary shoppers reported fairly good access to 

grocery retail (Exhibit 7.4). 

Primary shoppers reported that they seldom were kept from grocery shopping by limited 

transportation or distance to the grocery store. HIP respondents were somewhat more likely than nonHIP respondents to report difficulties shopping due to limited transportation, but the difference was 

only borderline significant. The composite scale summarizing the two barriers to grocery shopping 

found no statistically significant difference between HIP and non-HIP respondents. 

Exhibit 7.4: Impact of HIP on Grocery Shopping Barriers, Linear Regression Model

How often kept from 

grocery shopping by... 

Regression-adjusted 

mean (S.E.) Impact 

Treatment Control T-C [S.E.] {t-statistic} (P-value) 

Limited transportation 

(N=3,275) 

2.173 

(0.034) 

2.090 

(0.032) 0.082 [0.047] {1.748} (0.081)* 

Distance to grocery store 

(N=3,253) 

1.965 

(0.033) 

1.951 

(0.033) 0.014 [0.046] {0.304} (0.761) 

Composite scale—Barriers to 

grocery shopping (N=3,230) 

2.063 

(0.030) 

2.013 

(3.030) 0.050 [0.043] {1.161} (0.246) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always; “don’t know” and 

“refused” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibit F7.10 presents the ordered logistic regression model; Exhibit F7.11 presents the logistic 

regression model. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

7.5 Family Food Environment 

Past research suggests that having fruits and vegetables in the family food environment is associated 

with increased consumption. This could occur for several reasons: availability and visibility of fruits 

and vegetables at home might facilitate increased consumption; or people with stronger preferences 

for fruits and vegetables might be more likely to have them available at home (Jago, T. Baranowski, 

and J. C. Baranowski, 2007). To assess the family food environment, the household survey asked 

respondents a series of questions about how often fruits and vegetables were available in the home 

and how often they engaged in various healthy eating practices. 

Based on survey responses, both the HIP and non-HIP groups commonly have fruits and vegetables at 

home (Exhibit 7.5). It was slightly less common to have these fruits and ready-to-eat vegetables in the 

refrigerator or on the counter, where they are particularly accessible and visible. Most HIP and non-
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 136 ▌7. Impacts on Attitudes and Family Food Environment Abt Associates

HIP respondents reported cooking evening meals at home most of the time or always, although they 

reported sitting down for evening meals somewhat less frequently. 

Exhibit 7.5: Impact of HIP on Family Food Environment, Linear Regression Model

Regression-adjusted mean 

(S.E.) Impact 

How often do you...? 

Treatment 

(T) 

Control 

(C) T-C [S.E.] 

{tstatistic} (P-value) 

Have fruit available at home 

(N=3,314) 

4.303 

(0.023) 

4.192 

(0.024) 0.111 [0.033] {3.347} (0.001)*** 

Have fruit in refrigerator or on 

counter (N=3,311) 

4.010 

(0.026) 

3.955 

(0.025) 0.055 [0.036] {1.525} (0.130) 

Have vegetables available at 

home (N=3,315) a

4.501 

(0.023) 

4.418 

(0.023) 0.083 [0.032] {2.572} (0.010)** 

Have ready-to-eat vegetables 

in fridge or on counter 

(N=3,284) 

3.898 

(0.031) 

3.828 

(0.030) 0.070 [0.044] {1.594} (0.110) 

Composite scale—Fruits & 

vegetables available at home 

(N=3,318) 

4.175 

(0.019) 

4.094 

(0.019) 0.081 [0.027] {2.996} (0.003)*** 

Have salty snacks at home 

(chips, crackers) (N=3,312) 

3.202 

(0.033) 

3.215 

(0.031) -0.014 [0.045] {-0.302} (0.760) 

Have lowfat/nonfat milk at 

home (N=3,292) 

3.347 

(0.045) 

3.280 

(0.043) 0.067 [0.063] {1.068} (0.290) 

Have soft drinks/fruit drinks 

(not juice) at home (N=3312) 

3.149 

(0.038) 

3.238 

(0.036) -0.089 [0.053] {-1.678} (0.093)* 

Sit down with family at home 

for evening meals (N=2,306) b

3.913 

(0.035) 

3.920 

(0.031) -0.007 [0.047] {-0.147} (0.880) 

Cook evening meals at home 

(N=3,311) 

4.432 

(0.022) 

4.415 

(0.022) 0.017 [0.032] {0.538} (0.590) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never/no refrigerator or freezer, 2=rarely, 3=sometimes, 4=most of the time, 5=always; 

“'don't know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Appendix F, Exhibit F7.12 presents the ordered logistic regression model; Exhibit F7.13 presents the logistic 

regression model. 

a “No refrigerator or freezer” responses coded as missing.

b Asked only in households with more than one member. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 7. Impacts on Attitudes and Family Food Environment ▌pg. 137

HIP participation appears to have shifted those aspects of the food environment directly related to 

fruits and vegetables. HIP participants more frequently had fruits and vegetables available at home, as 

measured by the composite scale, than did households not participating in HIP. In addition, the 

individual measures of fruit availability and vegetable availability were significantly higher for HIP 

participants than non-HIP participants.87 

The perception that fruits and vegetables were available more frequently at home was also mentioned 

in the focus group discussions. One HIP participant commented, “Yeah, I definitely started eating 

more vegetables and fruits, definitely.” Another focus group participant noted that, “My daughter is 

eating more fruit now. Now she’ll have an apple, orange, grapes. Now she asks for fruit.” 

There were no significant differences in other measures of the family food environment, such as 

having fruit and ready-to eat vegetables in the refrigerator or on the counter, on the availability of 

other food categories, including salty snacks, lowfat/nonfat milk, and soft/fruit drinks, or on preparing 

and sitting down for evening meals at home.88

7.6 Discussion 

In two respects, HIP influenced informational and home environment measures in a direction that is 

consistent with our conceptual model depicting the potential effect of HIP on fruit and vegetable 

consumption and the mechanisms or intermediate outcomes through which HIP may work. 

First, HIP increased the probability of having heard or seen messages about fruits and vegetables, but 

not the probability of participating in a nutrition education class. As discussed in Chapter 3, the HIP 

group received multiple mailings from DTA about the operation of HIP. All mailings included the 

HIP logo, which showed pictures of fruits and vegetables and the tag line, “It’s HIP to be healthy.” 

Additionally, several of these mailings had an appealing graphical design and messaging that 

highlighted fruits and vegetables. The findings suggest that at least some members of the HIP group 

remembered these messages. By contrast, SNAP nutrition education (SNAP-Ed) is voluntary, and 

there was no difference in nutrition education programs offered to the HIP and non-HIP groups. Thus, 

the observed outcomes are consistent with HIP outreach activities to participants and suggest that HIP 

participants were aware of the messaging. 

Second, HIP increased the probability of having fruits and vegetables available at home. HIP had no 

such effect for non-targeted foods, whether these foods are commonly identified as more healthful 

(for example, low-fat/nonfat milk) or less healthful (for example, salty snacks and soft/fruit drinks). 

Again, the observed outcomes appear consistent with the HIP focus on fruits and vegetables rather 

than food choices more broadly. 

 

87 The composite home food environment scale did not show a statistically significant difference between 

Round 2 and Round 3 responses. However, for fruits only, the Round 2 and Round 3 results were 

statistically different; the frequency of having fruit at home was higher for the HIP than for the non-HIP 

groups in Round 2, but the HIP impact was not statistically significant in Round 3. 

88 Subgroup analyses revealed a marginally significant positive impact of HIP on how often evening meals 

are cooked at home among those who consumed less than three servings a day of fruits and vegetables at 

baseline, but not among those who consumed three or more servings a day of fruits and vegetables at 

baseline (Appendix H, Exhibit H7.5j). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 138 ▌7. Impacts on Attitudes and Family Food Environment Abt Associates

Given HIP’s financial incentive, one might have expected to find that HIP reduced the problem of 

cost as a reported barrier to fruit and vegetable consumption. However, there was no difference 

between HIP and non-HIP participants in the probability of agreeing with a statement about not eating 

fruits and vegetables as much as they would like due to the cost. Perhaps HIP participants did not 

reflect upon the price adjustment provided by HIP in answering this question. Alternatively, perhaps 

even a 30 percent incentive was not sufficient to eliminate cost as a perceived barrier. We note that 

when asked directly whether HIP made fruits and vegetables more affordable, 72 percent of HIP 

respondents replied in the affirmative (see Exhibit 6.8). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 139

8. Impact on Fruit and Vegetable Consumption 

The main goal of the HIP evaluation was to assess the impact of HIP on participants’ consumption of 

targeted fruits and vegetables, which are the fruits and vegetables that earned the incentive. This is the 

single confirmatory outcome we specified before analyzing the data. In addition to this main 

outcome, the evaluation also examined the impact of HIP on the consumption of total fruits and 

vegetables, on the consumption of different types of fruits and vegetables, and on overall dietary 

quality. 

Detailed information on the definition and construction of the targeted fruit and vegetable intake 

measure from the AMPM 24-hour dietary recall data is provided below. Like the impact analyses 

reported in earlier chapters, this analysis is based on (regression-adjusted) comparisons of the HIP 

and non-HIP responses.89 The analyses combine responses from all 24-hour dietary recalls (Round 2 

and Round 3).90

We additionally present analyses of secondary intake outcomes from both the dietary recalls and from 

the modified Eating at America’s Table Study (EATS) fruit and vegetable screener completed by 

each sampled respondent in each round. Like the analyses of intermediate outcomes 

(spending/shopping patterns, attitudes, and home food environment) presented in prior chapters, 

analyses of these secondary outcomes should be considered exploratory, since these analyses include 

multiple hypothesis tests. Because we test at the 5 percent level, it is likely that 5 percent of the tests 

would appear statistically significant using a conventional two-tailed hypothesis test just by chance. 

The best practice to address this issue is to identify a single confirmatory outcome in advance, as we 

have done here. 

The chapter begins by presenting impacts on various aggregate measures of fruit and vegetable 

intake, including our confirmatory outcome measure, targeted fruit and vegetable intake (Section 8.1). 

We then compare these intake estimates to spending estimates as reported in Chapter 6 to assess the 

overall consistency of the study findings (Section 8.2). The following two sections of the chapter, 

respectively, report impacts on consumption of specific fruit and vegetable categories (Section 8.3) 

and impacts on threshold measures of fruit and vegetable consumption (Section 8.4). We then 

examine whether the impact of HIP varies for particular demographic subgroups (Section 8.5) or for 

households with different baseline preferences and behaviors toward fruits and vegetables (Section 

8.6), and whether the Round 2 and Round 3 impacts differ (Section 8.7). The chapter then presents 

 

89 To estimate the impact of the pilot, these estimates exploit the evaluation’s random assignment design, 

comparing mean intake for those assigned to the treatment group and thus eligible to earn the HIP incentive 

(“HIP participants”) with mean intake among those assigned to the control group and thus ineligible to earn 

the HIP incentive (“HIP non-participants”). We report regression-adjusted estimates, which offer somewhat 

greater precision than direct comparison of unadjusted means (see Appendix E for additional detail on the 

methodology). 

90 The analysis sample includes both Round 2 and Round 3 recall interviews, pooling the first interviews with 

the 10 percent subsample of second interviews for each round. Our analyses cluster standard errors at the 

respondent level to account for non-independence of interview responses within respondents. After deleting 

records that lacked a complete 24-hour dietary recall, our analysis file includes 3,913 dietary recall 

interviews with 2,009 respondents. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 140 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

analyses of selected secondary food and nutrient outcomes, showing, for example, whether HIP 

increased total food energy intake or caused a substitution between one food group and another 

(Section 8.8), and impacts of HIP on overall dietary quality (Section 8.9). We conclude the chapter 

with a brief discussion in Section 8.10. 

8.1 Fruits and Vegetables 

This section reports HIP impacts on fruit and vegetable consumption. We begin by reporting the 

impact on targeted fruit and vegetable (TFV) intake. As will be described in greater detail below, our 

preferred TFV intake measure91 is a conservative proxy; it includes only fruit and vegetable intake 

from foods we can be reasonably sure would have qualified to earn the HIP incentive if purchased 

from a participating retailer. We then consider progressively broader fruit and vegetable intake 

aggregates and components. In particular, we consider impacts on an alternative, more inclusive 

proxy measure for TFV intake, and on intake of all fruits and vegetables, including a breakdown of 

impacts on individual components of these aggregates such as mixed foods; white potatoes, legumes, 

and 100% fruit juice; and other fruits and vegetables acquired outside of retail stores. 

Targeted Fruits and Vegetables 

Exhibit 8.1 shows HIP impacts on daily adult (aged 16 and older) consumption of TFV, based on our 

preferred proxy measure. TFV intake after the pilot was introduced was almost a quarter cup (0.24 

cup-equivalent)92 higher among HIP participants as compared to those not selected to participate in 

HIP. Because we used a random assignment design, this difference can be interpreted as the causal 

impact of HIP. This difference represents an increase of 26 percent compared to consumption in the 

absence of HIP (0.91 cup-equivalent for non-HIP households). Furthermore, the result is highly 

statistically significant. 93

 

91 Note that construction of this TFV proxy measure differs slightly from that of the preliminary “lowerbound modified targeted fruits and vegetables” measure described in the Interim Report. Based on a 

recoding of some of the available 24-hour dietary recall data, this proxy excludes intake from mature 

legumes, which do not qualify for the HIP incentive (which were included in the Interim Report measure), 

and includes intake from a greater proportion of mixed foods (relative to the Interim Report measure) 

where the respondent was able to explicitly identify the source of included fruit and vegetable ingredients. 

92 One cup-equivalent is equal to one cup of cut-up raw or cooked fruits or vegetables. 

93 The appropriate statistical test decisively rejects the null hypothesis of no impact. Specifically, we test the 

treatment dummy in the regression model for all 24-hour dietary recall interviews, using a 

heteroscedasticity and cluster robust version of the t-test. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 141

Exhibit 8.1: Impact of HIP on Consumption of Targeted Fruits and Vegetables, Mean CupEquivalents Consumed 

Targeted fruit and vegetable intake includes intake of fruits and vegetables acquired from the store. It excludes 

white potatoes, legumes, and 100% juice, as well as mixed foods where the source of individual ingredients was 

not identified by the respondent. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents) 

To understand these results, it is important to understand the definition of and motivation for this 

preferred TFV intake measure. 

As shown in greater detail in Appendix Exhibit F1.1, the targeted fruits and vegetables that qualified 

for the HIP incentive include fresh, canned, frozen, and dried fruits and vegetables without added 

sugars, fats, oils, or salt94 as purchased from HIP participating retailers. TFVs exclude white potatoes, 

mature legumes,95 and 100% juice.96 Exhibit 8.2 visually depicts TFVs as a subset of all fruits and 

vegetables. 

When constructing a proxy TFV intake measure based on standard data elements recorded from the 

24-hour dietary recall, the first four exclusions (100% juice, legumes, white potatoes, and fruits and 

vegetables purchased outside stores, the four “slices” at the top of the oval) are relatively 

straightforward to implement. Standard nutritional coding schemes allow for identification of fruit 

 

94 Vegetables with added salt do qualify for the HIP incentive, but fruits with added salt do not. 

95 Mature legumes include dried peas, beans, and lentils. 

96 Beverages containing less than 100% juice would also not qualify for the incentive because they generally 

include non-fruit, non-vegetable content; the exclusion of “100% juice” is thus intended to explicitly 

disqualify all-fruit or all-vegetable juices with no whole fruit or vegetable content. 

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Cup-Equivalents

Treatment group (N=1,954 recalls from 1,010 respondents)

Control group (N=1,959 recalls from 999 respondents)

} impact = 0.24 cup‐equivalent

(p<0.001)

1.15

cup‐equivalents 0.91

cup‐equivalent
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 142 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

and vegetable intake from 100% juice, legumes, or white potatoes, which can then directly be 

excluded from a constructed TFV intake proxy measure. 

Similarly, for each food consumed, the respondent reports whether that food was purchased from a 

retail store or acquired from some other location (e.g., a food pantry, as part of a school meal, at a 

restaurant). If the respondent is able to separately report individual ingredients and quantities for a 

food prepared from multiple ingredients (“mixed food”), the source of acquisition for each ingredient 

will additionally be reported. This coding allows us to appropriately restrict the measure to intake 

from foods and/or individual ingredients purchased at stores only, although note that we cannot 

further distinguish fruits and vegetables acquired from HIP participating versus non-participating 

retailers. 

Exhibit 8.2: Targeted Fruits and Vegetables (TFV) as a Proportion of All Fruits and 

Vegetables 

However, when a respondent cannot enumerate individual ingredients, intake of TFVs included in 

“mixed foods” purchased from stores presents a more formidable coding challenge for our purposes. 

For example, suppose a respondent tells us he ate beef and vegetable stew, and that the stew was 

purchased from the store. If the respondent is able to separately list the stew ingredients (e.g., beef, 

celery, carrots, potatoes, salt, spices) and tell the interviewer whether the potentially qualifying fruit 

and vegetable ingredients (celery and carrots) were purchased from the store, we can readily conclude 

whether they should be included in our TFV measure. On the other hand, if the respondent is unable 

to describe the individual ingredients and where they were acquired, as might happen, for example, if 

the meal was prepared by another household member, it is not possible to distinguish between the 

following two scenarios: 

 The stew was prepared from individual ingredients after purchase, in which case the TFV 

ingredients would have been eligible for the HIP incentive, as in the lower part of the circle 

depicting intake of fruits and vegetables from mixed foods. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 143

 The stew was purchased in prepared form, with the ingredients combined prior to purchase, 

in which case the TFV ingredients would not have been eligible for the incentive, as in the 

upper part of the circle depicting intake of fruits and vegetables from mixed foods. 

These constraints suggest two possible approaches to constructing our TFV proxy: we can either 

include all intake of fruits and vegetables from store-acquired mixed foods (the entire right-hand 

circle in Exhibit 8.2) or exclude all such intake. The former “inclusive” definition would include some 

non-TFVs, while the latter “restrictive” definition would exclude some TFVs. 

We accordingly constructed two alternative proxy measures for TFV intake reflecting these inclusive 

and restrictive definitions. For store-purchased foods only, both measures include fruit and vegetable 

intake from foods containing only fruit and vegetable ingredients. Both measures exclude white 

potatoes, legumes, 100% juice, and all fruits and vegetables purchased or acquired from restaurants, 

cafeterias, food pantries, or other non-store locations, since only store-purchased items were eligible 

for the HIP incentive. 

 The restrictive TFV intake proxy measure, which we take as our preferred measure, excludes 

all food items that include both TFV and other ingredients in cases when the respondent did 

not identify where individual ingredients were acquired. That is, it explicitly excludes any 

fruit and vegetable intake from mixed foods reported as consumed in their prepared form 

(rather than as a list of individual ingredients) that contain non-fruit, non-vegetable 

ingredients (e.g. apple pie, vegetable noodle soup, or chili). It also excludes any intake from 

fruit and vegetable preparations that include added sugars, fats, or oils (e.g. stir-fried string 

beans, dried sweetened cranberries, canned peas with added sugar), unless, once again, the 

respondent was able to identify individual ingredients and indicate where those ingredients 

were acquired. This strict definition ensures that all included fruit and vegetable intake in the 

restrictive measure comes from foods that would have qualified to earn the HIP incentive if 

purchased in a participating store. We take this as our confirmatory outcome measure because 

its conservative definition ensures that it does not include any fruit and vegetable intake from 

non-TFVs. 

 Our alternative, inclusive TFV intake proxy measure is defined somewhat less 

conservatively. It includes all those foods included in our preferred TFV proxy as described 

above, plus some additional mixed foods that may or may not have qualified for the HIP 

incentive at purchase. Like our preferred restrictive TFV proxy measure, the inclusive 

measure excludes all fruits and vegetables not purchased from a store. Unlike our preferred 

measure, however, it also includes fruit and vegetable intake from mixed foods (again 

excluding white potatoes, legumes, and 100% fruit juice), even in cases where respondents 

were unable to identify individual ingredients and sources. 

As Exhibit 8.3 shows, the difference between the two proxy measures is substantial. In the control 

group, regression-adjusted TFV intake as measured using the inclusive proxy is 34 percent higher 

than TFV intake as measured using the restrictive proxy (1.22 cup-equivalents vs. 0.91 cupequivalent). Because of this large difference between alternative definitions, we report results for our 

alternative, inclusive measure in Exhibit 8.3 alongside results for our preferred, restrictive measure. 

Intake of fruits and vegetables from mixed foods did not significantly differ between HIP participants 

and non-participants. As a result, impacts on the alternative inclusive measure were similar in 

magnitude and significance to impacts on our preferred restrictive TFV intake proxy. (However, the 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 144 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

change as a percent of control group consumption is smaller.) This result suggests that HIP primarily 

influenced intake of TFVs consumed alone, without added sugars, fats, oils, or other ingredients. 

Other Fruit and Vegetable Aggregates 

HIP may affect fruit and vegetable intake beyond targeted items for several reasons. First, a HIP 

effect on TFV intake might cause substitution between eligible and ineligible foods. Second, HIP may 

affect non-targeted fruits and vegetables by influencing participant attitudes and preferences toward 

fruits and vegetables in general, including those that do not qualify for the incentive (see Chapter 7). 

This may be a result of increasing a household’s total resources available for food purchases or due to 

confusion about what foods were eligible for the incentive (see the first section of Chapter 5 for 

evidence of such confusion). 

Exhibit 8.3: Impact of HIP on Consumption of Fruits and Vegetables and Disaggregated 

Components, Cup-Equivalents 

Regression-adjusted 

mean (S.E.) Impact 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Targeted fruits and 

vegetables (preferred 

restrictive proxy measure) 

1.149 

(0.043) 

0.910 

(0.035) 0.238 [0.054] {4.382} (<0.001)*** 

Plus TFV from mixed foods 0.307 

(0.013) 

0.304 

(0.013) 0.002 [0.019] {0.114} (0.909) 

Targeted fruits and 

vegetables (alternative 

inclusive proxy measure) 

1.455 

(0.045) 

1.215 

(0.039) 0.241 [0.059] {4.082} (<0.001)*** 

Plus additional components: 

100% fruit juice 0.549 

(0.029) 

0.453 

(0.023) 0.095 [0.036] {2.617} (0.009)*** 

White potatoes 0.336 

(0.017) 

0.359 

(0.017) -0.023 [0.024] {-0.963} (0.336) 

Legumes 0.106 

(0.007) 

0.114 

(0.007) -0.008 [0.010] {-0.858} (0.391) 

Other fruits and 

vegetables acquired 

outside stores 

0.171 

(0.016) 

0.152 

(0.015) 0.018 [0.023] {0.806} (0.420) 

All fruits and vegetables 2.616 

(0.060) 

2.294 

(0.055) 0.323 [0.080] {4.016} (<0.001)*** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

TFV intake proxy measures include intake of fruits and vegetables acquired from the store, excluding white 

potatoes, legumes, and 100% juice. The preferred restrictive proxy measure additionally excludes fruit and 

vegetable intake from mixed foods where the source of individual ingredients was not identified by the 

respondent, while the alternative inclusive proxy measure includes fruit and vegetable intake from all mixed 

foods. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 145

Exhibit 8.3 also reports impact estimates for various fruit and vegetable aggregates and included 

components to assess potential impacts on non-TFV intake of fruits and vegetables.97 Considering 

progressively more inclusive aggregates in turn provides suggestive evidence on which components 

contribute to HIP impacts on overall fruit and vegetable intake, beyond TFVs considered alone. As 

already noted, HIP did not have an impact on fruit and vegetable intake from mixed foods. 

White Potatoes, Legumes, and 100% Fruit Juice 

Because 100% fruit juice, legumes, and white potatoes do not qualify for the HIP incentive, both the 

restrictive and inclusive TFV proxy measures exclude these items. However, as seen in Exhibit 8.3, 

100% fruit juice, legumes, and white potatoes contribute substantially to total fruit and vegetable 

intake. Individuals in our control group consumed 0.45 cup-equivalent of 100% fruit juice 

(comprising about 20 percent of all fruit and vegetable intake); 0.36 cup-equivalent of white potatoes 

(about 15 percent of all fruit and vegetable intake); and 0.11 cup-equivalents of legumes (about 5 

percent of all fruit and vegetable intake). 

There was no statistically significant impact of HIP on white potato or legume consumption. In 

contrast, however, HIP participants consumed significantly more 100% fruit juice than nonparticipants (0.10 cup-equivalent), even though fruit juice does not qualify for the HIP incentive. 

Perhaps some HIP participants thought that 100% fruit juice purchases earned the incentive. 

Consistent with this hypothesis, exploratory evidence as presented in Appendix I implies that impacts 

on 100% fruit juice were concentrated among those respondents who reported that they found it 

difficult to understand how HIP worked. Alternatively, perhaps HIP influences on participants’ 

beliefs about the benefits of fruit consumption more generally induced greater intake of 100% fruit 

juice due to those perceived benefits. Or perhaps increased consumption of fruit gave participants a 

taste for 100% fruit juice. 

Fruits and Vegetables Acquired Outside Stores 

The two TFV intake proxy measures additionally exclude all intake of fruits and vegetables not 

purchased from a retail store (e.g., restaurants, cafeterias, and food pantries). Together these foods 

comprise the difference in intake between our alternative, inclusive TFV intake proxy measure and all 

fruit and vegetable consumption not otherwise attributable to intake from white potatoes, legumes, or 

100% fruit juice. This residual comprises about 0.15 cup-equivalent in the control group, or a little 

less than7 percent of all fruit and vegetable intake. We find no evidence that HIP influenced intake of 

fruits and vegetables acquired from non-store sources. 

All Fruit and Vegetable Intake 

HIP participants’ total fruit and vegetable consumption is almost one third of a cup (0.32 cupequivalent) greater than that of non-participants. This impact is somewhat larger than the 0.24 cupequivalent impact on our preferred TFV proxy measure described above.98 The 0.10 cup-equivalent 

impact on 100% fruit juice intake explains the bulk of the difference in impacts between these two 

 

97 Full regression results including coefficients for all covariates appear in Appendix F, Exhibit F8.1. 

98 However, the difference between impacts on our preferred TFV proxy and impacts on total fruits and 

vegetables was not statistically significant (test not shown); in other words, we cannot reject the hypothesis 

that impacts on TFV and impacts on total fruits and vegetables were identical in magnitude at the 

conventional 95 percent confidence level. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 146 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

measures. These findings suggest that HIP may have an impact on total fruit and vegetable 

consumption beyond its direct effects on TFV consumption, though it is unclear why these indirect 

impacts would be concentrated on 100% fruit juice. 

8.2 Comparing HIP Impacts on Fruit and Vegetable Spending and Intake 

A major goal of this evaluation is to investigate the hypothesis that the HIP price incentive 

encourages increased purchase of TFVs, and that these increased purchases may in turn lead to 

increased TFV intake. We showed in Exhibit 8.3 that, in fact, HIP increased mean daily TFV intake 

by 0.24 cup-equivalent per adult. 

In this section, we explore whether the magnitude of the estimated HIP impact on fruit and vegetable 

intake corresponds, as would be expected, to the earlier estimates of HIP impact on fruit and 

vegetable spending reported in Chapter 6, Section 6.1. To make this comparison, we rely on two key 

assumptions that allow us to translate our estimates of impacts on per-adult fruit and vegetable intake, 

reported in cup-equivalents, into estimated per-household impacts, reported in dollars. Those 

assumptions are: 

 an estimated number of adults per household 

 an estimated price per fruit and vegetable cup-equivalent 

Our estimates assume 1.5 adults per household, based on estimated counts in our sample (Exhibit 

2.8), and a price of $0.50 per cup-equivalent based on estimates from existing research by USDA’s 

Economic Research Service (Stewart et al., 2011).99 Converting estimates from a daily to a monthly 

basis is then a straightforward task. Together these conversions allow the direct comparison of intake 

estimates to spending estimates in the same units. 

Note that using the estimated number of adults per household, rather than the total number of 

household members (including children), is necessary because we did not collect dietary recall data 

from children and cannot therefore project intake levels for this group. Using the number of adults per 

household as our multiplier implicitly assumes that children consume no fruits and vegetables at all. 

If we attributed some positive amount of consumption to children, per-household intake implied by 

the survey would be higher. Our converted estimates thus represent a lower bound on true perhousehold intake.100

 

99 This price estimate from the literature is moderately smaller than a very rough price estimate derived using 

data from our own study. HIP participants reported total monthly household fruit and vegetable spending of 

$78.17, which is equivalent to $1.72 per adult per day (Exhibit 6.3). Dividing this total by estimated daily 

per-adult fruit and vegetable intake of 2.616 cup equivalents (Exhibit 8.3) suggests an implicit price of 

$0.66 per fruit and vegetable cup-equivalent. A parallel computation for the control group suggests a 

similar implicit price of $0.69 per cup-equivalent. This implicit price would be an upper bound, since we 

do not include children in the estimate of household consumption. 

100 An alternative approach would have been to multiply by the total number of persons in the household, 

implicitly assuming that child intake was the same as adult intake. Since in reality child intake is generally 

lower than adult intake, that approach would produce a likely upper bound on true per-household 

consumption. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 147

Exhibit 8.4 shows the results of this reconciliation exercise for TFVs and for all fruits and vegetables, 

respectively. For TFVs, our estimated daily per-adult impact of 0.238 cup-equivalent translates to a 

lower-bound impact of $5.43 per household per month. This survey-based measure compares to an 

impact of $1.07 on TFV spending based on EBT transactions data for our survey sample, as 

previously reported in Chapter 6; this is more than a four-fold difference. In fact, since we have 

argued that our converted intake estimates represent a lower bound on true per-household monthly 

intake in dollar terms, the true difference must be even larger. 

Exhibit 8.4: Comparison of HIP Impacts on Intake and Spending Measures, Targeted Fruits 

and Vegetables and All Fruits and Vegetables 

Measure Data Source Impact (T-C) 

Targeted fruits and vegetables (TFVs)  

TFV intake, daily cup-equivalents per adult AMPM dietary recall 

interview 0.238 

TFV intake, lower-bound, monthly dollars per 

household ($) 

AMPM dietary recall 

interview 5.43 

TFV expenditures at participating 

supermarkets/superstores, monthly dollars per 

household ($) 

EBT transactions data 1.07 

All fruits and vegetables

Fruit and vegetable intake, daily cup-equivalents per 

adult 

AMPM dietary recall 

interview 0.323 

Fruit and vegetable intake, lower-bound, monthly 

dollars per household ($) 

AMPM dietary recall 

interview 7.37 

Fruit and vegetable expenditures, self-reported, 

monthly dollars per household ($) 

Participant survey (primary 

shopper module) 6.15 

TFV intake proxy measures include intake of fruits and vegetables acquired from the store, excluding white 

potatoes, legumes, and 100% juice. The preferred restrictive proxy measure additionally excludes fruit and 

vegetable intake from mixed foods where the source of individual ingredients was not identified by the 

respondent, while the alternative inclusive proxy measure includes fruit and vegetable intake from all mixed 

foods. 

Intake measures were converted from a daily per-adult cup-equivalent basis to a monthly per-household dollar 

basis assuming 30.4375 days per month, a price of $0.50 per cup-equivalent, and 1.5 adults per household; 

since this conversion implicitly assumes zero intake for children in the household, it represents a lower bound on 

true monthly per-household intake. 

Sources: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents); Participant 

Survey (primary shopper module) (N=2,708); EBT Transaction Data, pooled across March-October 2012 

(average of 45,912 households per month). 

Unlike our TFV measures (in which intake is derived from survey data and spending is derived from 

EBT data), our intake and spending measures for all fruits and vegetables are both derived from 

survey data. Perhaps for this reason, impacts on intake and spending measures for all fruits and 

vegetables appear to be more closely aligned than those for TFVs. Specifically, the 0.323 daily cupequivalent impact on all fruit and vegetable intake translates to a $7.37 impact in monthly perhousehold terms, which is similar to the estimated impact of $6.15 on self-reported monthly 

expenditures. 

In Chapter 10, we further discuss possible explanations for the contrast between impacts on selfreported TFV intake and spending measures versus impact on TFV expenditures in participating 

retailers as derived from EBT transactions data. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 148 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

8.3 Targeted Fruit and Vegetable Groups and Subgroups 

Although fruits and vegetables from all major fruit and vegetable subgroups qualified to earn the HIP 

incentive, it is plausible that impacts could be concentrated within a particular fruit or vegetable 

group or subgroup.101 For example, HIP participants might have differentially increased spending on 

a handful of familiar fruits or vegetables within a subgroup already comprising a relatively large 

proportion of their prior fruit and vegetable consumption. Alternatively, they might have spread 

additional spending proportionally across specific subgroups. 

To understand how impacts on total intake were distributed across fruit and vegetable categories, we 

separately report HIP impacts on TFV intake disaggregated by USDA Food Pattern food groups and 

subgroups (Exhibit 8.5) and on usual intake of fruits and vegetables as reported on a fruit and 

vegetable consumption screener (Exhibit 8.6). 

USDA Food Pattern Fruit and Vegetable Groups and Subgroups 

For these analyses of fruit and vegetable groups and subgroups, we use USDA Food Pattern food 

group definitions, defined in accordance with the 2010 Dietary Guidelines for Americans (USDA and 

USDHHS, 2010). The three major fruit subgroups are (1) citrus, melon, and berries; (2) other fruits; 

and (3) 100% fruit juice. The five major vegetable subgroups are (1) dark green (broccoli, spinach, 

most greens); (2) red and orange (tomatoes, carrots, sweet potatoes, winter squash, pumpkin); (3) 

starchy (corn, white potatoes, green peas); (4) legumes (dry beans and peas)102; and (5) other 

vegetables (cabbage, celery, cucumber, lettuce, onions, peppers, green beans, cauliflower, 

mushrooms, summer squash). 

For this analysis, we use measures based on our preferred restrictive TFV intake proxy measure, 

which includes intake only from store-purchased fruits and vegetables and excludes intake from 

mixed foods. Targeted fruit intake therefore includes intake from whole fruit only, excluding intake 

from 100% fruit juice. Similarly, targeted vegetable intake excludes intake from white potatoes and 

legumes. Impacts on TFV intake disaggregated by USDA Food Pattern food group and subgroup are 

reported in Exhibit 8.5.103 

 

101 In this section and in the rest of the chapter, we use our preferred TFV intake measure, which Section 8.1 

referred to as the restrictive TFV intake proxy measure. This is a conservative proxy measure; it includes 

only fruit and vegetable intake from foods we can be reasonably sure would have qualified to earn the HIP 

incentive if purchased from a participating retailer. 

102 This legumes measure includes only beans and peas consumed in excess of levels needed to meet protein 

group recommendations in combination with intake from meat, poultry, fish, eggs, nuts, and seeds. 

103 Appendix F, Exhibits F8.2 and F8.3, provides supplemental exhibits showing impacts on our alternative 

TFV proxy and on total fruit and vegetable intake by USDA Food Pattern food group and subgroup; results 

are qualitatively similar to those for our preferred restrictive TFV proxy measure, and so have been omitted 

from the main text for the sake of brevity. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 149

Exhibit 8.5: Impact of HIP on Consumption of Targeted Fruits and Vegetables (TFV), CupEquivalents, by USDA Food Pattern Food Group

Regression-adjusted 

mean (SE) Impact 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Total fruits and vegetables 1.149 

(0.043) 

0.910 

(0.035) 0.238 [0.054] {4.382} (<0.001)*** 

Total fruits 0.571 

(0.030) 

0.465 

(0.024) 0.106 [0.038] {2.813} (0.005)*** 

Citrus fruits, melons, 

berries 

0.131 

(0.010) 

0.119 

(0.012) 0.012 [0.016] {0.774} (0.439) 

Other fruits (e.g., 

apples, pears, 

bananas, grapes, 

peaches) 

0.439 

(0.025) 

0.346 

(0.019) 0.093 [0.031] {3.005} (0.003)*** 

Total vegetables 0.578 

(0.025) 

0.445 

(0.021) 0.133 [0.032] {4.106} (<0.001)*** 

Dark green vegetables 0.076 

(0.007) 

0.044 

(0.005) 0.032 [0.009] {3.690} (<0.001)*** 

Red and orange 

vegetables 

0.136 

(0.009) 

0.105 

(0.007) 0.031 [0.012] {2.629} (0.009)*** 

Tomatoes 0.087 

(0.006) 

0.069 

(0.005) 0.017 [0.008] {2.093} (0.036)** 

Other red and 

orange vegetablesa

0.049 

(0.007) 

0.035 

(0.004) 0.014 [0.008] {1.847} (0.065)* 

Other starchy 

vegetables 

0.092 

(0.008) 

0.084 

(0.007) 0.008 [0.011] {0.720} (0.472) 

Other vegetables (e.g., 

celery, cucumbers, 

mushrooms, green 

beans, onions, 

asparagus) 

0.275 

(0.015) 

0.213 

(0.014) 0.062 [0.020] {3.090} (0.002)*** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Targeted fruit and vegetable intake proxy measure includes intake of fruits and vegetables acquired from the 

store. It excludes white potatoes, legumes, and 100% juice, as well as mixed foods where the source of 

individual ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Note that this result appears to be driven by the presence of two extreme outliers during Round 2 who reported 

intake of 5 or more cups of other read and orange vegetables in the prior 24 hours. Excluding those individuals 

from the analysis, differences between treatment and control groups are no longer statistically significant 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

We find higher TFV fruit intake of 0.11 cup-equivalent in HIP participants relative to the nonparticipants. Most of this impact was due to a statistically significant 0.09-cup equivalent difference 

in intake of TFV fruits from the “other fruit” subgroup. We did not detect a statistically significant 

difference in consumption of TFV citrus fruits, melons, and berries. 

The treatment-control difference in TFV vegetable intake was 0.13 cup-equivalent. This includes a 

0.03-cup equivalent difference in intake of dark green vegetables; a 0.06-cup equivalent difference in 

intake of the “other vegetables” subgroup; and a 0.03-cup equivalent difference in intake of red and 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 150 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

orange vegetables, driven primarily by a 0.02 cup-equivalent difference in intake of tomatoes, along 

with a borderline-significant 0.01 cup-equivalent difference in intake of other red and orange 

vegetables. We find no evidence of an increase in consumption of starchy vegetables (which, as noted 

above, excludes white potatoes, the most commonly consumed starchy vegetable). 

Usual Daily Intake from Fruit and Vegetable Screener 

In addition to completing 24-hour intake dietary recall interviews, sampled respondents also 

completed a modified version of the Eating at America's Table Study (EATS) Fruit and Vegetable 

Screener (Thompson et al., 2000) that asked about usual intake of nine common foods containing raw 

and cooked fruits and vegetables (including those eaten as snacks and at meals, eaten at home and 

away, and eaten alone and mixed with other foods) by the respondent over the prior month.104

Respondents were asked how often during the past month they had consumed 100% juice, fruit, leafy 

green salads, fried potatoes, other potatoes, beans, other vegetables, tomato sauce, and salsa. In 

addition, they reported how much they usually consumed when they ate these items. Using standard 

EATS scoring procedures these reported frequencies and amounts were used to calculate an estimate 

of usual intake per day in standardized cup-equivalents for each respondent. 

Impacts on intake from the EATS screener were broadly consistent with impacts on comparable 

measures constructed from 24-hour recall interviews. Exhibit 8.6 shows a statistically significant 

impact on self-reported usual daily intake of fruit, and a borderline-significant impact on leafy green 

salads, broadly consistent with the impacts on fruit, dark green vegetables, and other vegetables 

(including lettuce) found in the 24-hour dietary recall data. We additionally found a borderlinesignificant impact on fried potatoes, which as noted above are not included in TFV intake measures. 

We did not, however, find impacts on the screener measure of 100% juice, or on the two common 

tomato-based foods, tomato sauce and salsa, in contrast to the significant impact on 100% fruit juice 

and tomato intake from the recall data. In addition, we find no impact on intake of other potatoes or 

beans. 

 

104 Baseline responses to the fruit and vegetable screener are presented in Appendix F, Exhibit F8.4. There 

were no statistically significant differences between the treatment and control groups in reported screener 

measures at baseline. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 151

Exhibit 8.6: Impact of HIP on Estimated Usual Daily Intake from Fruit and Vegetable 

Screener, Cup-Equivalents 

Regression-adjusted 

mean (SE) Impact 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} 

(Pvalue) 

100% juice (N=3,350) 1.011 

(0.041) 

0.929 

(0.040) 

0.082 

[0.055] 

{1.478} 

(0.140) {1.478} (0.140) 

Fruit (N=3,355)a 0.676 

(0.026) 

0.608 

(0.022) 

0.068 

[0.033] 

{2.059} 

(0.040)** {2.059} (0.040)** 

Leafy green salad 

(N=3,371) 

0.306 

(0.010) 

0.281 

(0.010) 

0.025 

[0.014] 

{1.750} 

(0.080)* {1.750} (0.080)* 

Fried potatoes (N=3,389) 0.055 

(0.004) 

0.072 

(0.007) 

-0.017 

[0.009] 

{-1.929} 

(0.054)* {-1.929} (0.054)* 

Other potatoes (N=3,379) 0.215 

(0.010) 

0.205 

(0.008) 

0.009 

[0.013] 

{0.727} 

(0.467) {0.727} (0.467) 

Beans (N=3,376) 0.192 

(0.010) 

0.187 

(0.008) 

0.004 

[0.013] 

{0.352} 

(0.725) {0.352} (0.725) 

Other vegetables 

(N=3,350) 

0.506 

(0.018) 

0.476 

(0.016) 

0.030 

[0.024] 

{1.231} 

(0.219) {1.231} (0.219) 

Tomato sauce (N=3,331) 0.102 

(0.005) 

0.107 

(0.005) 

-0.005 

[0.007] 

{-0.687} 

(0.492) {-0.687} (0.492) 

Salsa (N=3,383) 0.009 

(0.001) 

0.009 

(0.001) 

<0.001 

[0.001] 

{0.035} 

(0.972) {0.035} (0.972) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

“Refused” and “don't know” responses on frequency or amount items coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Note that this result appears to be driven by the presence of one extreme outlier during Round 3 who reported 

estimated usual intake of 15 cup-equivalents of fruit per day. Excluding this individuals from the analysis, 

differences between treatment and control groups are only borderline statistically significant. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses. 

8.4 Fruit and Vegetable Cutoff Measures 

The results in the previous subsection clearly show higher fruit and vegetable intake levels among 

HIP participants. Also interesting is understanding how that difference is distributed across 

participants. For example, are individuals more likely to eat at least some fruits and vegetables on any 

given day as a result of HIP? Or, are they more likely to consume what might be considered a 

nutritionally adequate amount? 

To investigate these questions, we examined HIP impacts on several cutoff measures of fruit and 

vegetable intake: the probability that individuals consumed any fruits and vegetables in the past 24 

hours; the probability that they consumed at least one cup-equivalent of fruits and vegetables; and the 

probability that they consumed two-and-a-half or more cup-equivalents of fruits and vegetables 

(Exhibit 8.7). Note that 2.5 cup-equivalents of fruits and vegetables corresponds to 5 servings, the 

minimum recommended threshold under the older “5 A Day” program guidelines corresponding to 

the 2000 Dietary Guidelines for Americans; newer USDA food patterns under the 2010 Dietary 

Guidelines for Americans recommend between 2 and 6.5 cup-equivalents per day depending on total 

energy intake levels. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 152 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

Exhibit 8.7: Impact of HIP on Cutoff Measures of Fruit and Vegetable Intake in the Past 24 

Hours 

Regression-adjusted 

mean (SE) Impact 

Treatment 

(T) 

Control 

(C) T-C [S.E.] 

{tstatistic} (P-value) 

Consumed any fruits or 

vegetables 

0.965 

(0.005) 

0.955 

(0.005) 0.009 [0.007] {1.363} (0.173) 

Consumed 1 or more cupequivalent of fruits & 

vegetables 

0.743 

(0.011) 

0.714 

(0.011) 0.029 [0.016] {1.872} (0.061)* 

Consumed 2.5 or more 

cup-equivalents of fruits & 

vegetables 

0.416 

(0.013) 

0.354 

(0.012) 0.063 [0.017] {3.653} (<0.001)*** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

HIP participants were about 6 percentage points more likely to have consumed at least two and a half 

cup-equivalents of fruits and vegetables in the past 24 hours (42 percent vs. 35 percent).In addition, 

they were about 3 percentage points more likely to have consumed one or more cup-equivalent of 

fruits and vegetables in the past 24 hours than were non-participants (74 percent vs. 71 percent), 

though this difference was only borderline statistically significant. There was no statistically 

significant difference in proportions of HIP participants and non-participants consuming any fruits or 

vegetables, indicating that the pilot did not necessarily induce those not already consuming fruits and 

vegetables to consume them on a day they otherwise would not have. Note, however, that rates of 

consumption of any fruits and vegetables are quite high in the control group (96 percent), so there is 

not much scope for HIP to improve this measure. 

There is some evidence, however, that HIP may have caused individuals to consume some types of 

fruits and vegetables on days when they otherwise would not have consumed any (Exhibit 8.8). The 

probability of having eaten any fruits in the past 24 hours was about three percentage points higher 

among participants than among non-participants (74 percent vs. 71 percent) with increases in two of 

the three USDA Food Pattern fruit subgroups. The probability of 100% fruit juice consumption 

increased as did the consumption of other fruits. For vegetables, the probability of consuming dark 

green vegetables was higher among HIP participants than non-participants (16 percent vs. 12 

percent), as was the probability of consuming other vegetables (68 percent vs. 65 percent). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 153

Exhibit 8.8: Impact of HIP on Probability of Any Fruit and Vegetable Intake, by USDA Food 

Pattern Food Group 

Regression-adjusted 

mean (SE) Impact 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Total fruit and vegetables 0.965 

(0.005) 

0.955 

(0.005) 0.009 [0.007] {1.363} (0.173) 

Total fruit 0.742 

(0.012) 

0.710 

(0.011) 0.032 [0.017] {1.916} (0.056)* 

Citrus fruits, melons, 

berries 

0.257 

(0.011) 

0.232 

(0.010) 0.024 [0.015] {1.601} (0.109) 

Other fruits (e.g., 

apples, pears, 

bananas, grapes, 

peaches) 

0.469 

(0.013) 

0.423 

(0.012) 0.046 [0.018] {2.557} (0.011)** 

Fruit juice 0.499 

(0.014) 

0.454 

(0.013) 0.045 [0.019] {2.415} (0.016)** 

Total vegetables 0.907 

(0.007) 

0.895 

(0.008) 0.013 [0.010] {1.209} (0.227) 

Dark green vegetables 0.162 

(0.010) 

0.115 

(0.008) 0.047 [0.012] {3.903} (<0.001)*** 

Red and orange 

vegetables 

0.706 

(0.011) 

0.687 

(0.011) 0.018 [0.016] {1.127} (0.260) 

Tomatoes 0.617 

(0.012) 

0.608 

(0.012) 0.009 [0.017] {0.560} (0.575) 

Other red and 

orange vegetables 

0.240 

(0.011) 

0.217 

(0.010) 0.023 [0.015] {1.553} (0.121) 

Starchy vegetables 0.505 

(0.012) 

0.503 

(0.012) 0.002 [0.017] {0.128} (0.899) 

White potatoes 0.400 

(0.012) 

0.406 

(0.012) -0.006 [0.017] {-0.370} (0.712) 

Other starchy 

vegetables 

0.218 

(0.010) 

0.195 

(0.009) 0.024 [0.014] {1.725} (0.085)* 

Legumes (beans and 

peas computed as 

vegetables) 

0.207 

(0.010) 

0.212 

(0.009) -0.004 [0.013] {-0.319} (0.750) 

Other vegetables (e.g., 

celery, cucumbers, 

mushrooms, green 

beans, onions, 

asparagus) 

0.684 

(0.012) 

0.649 

(0.011) 0.035 [0.016] {2.233} (0.026)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

Consistent with the overall increase in fruit and vegetable intake and the increased probability of 

intake of specific fruit and vegetable subgroups, there was a corresponding increase in variety of 

fruits and vegetables consumed overall, as indicated by higher numbers of fruit and vegetable 

subgroups consumed in the past day (Exhibit 8.9). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 154 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

Exhibit 8.9 Impact of HIP on Total Number of USDA Food Pattern Fruit and Vegetable 

Groups Consumed in Past 24 Hours 

Regression-adjusted 

mean (SE) Impact 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Number of fruit & vegetable 

groups (range: 0-8) 

3.489 

(0.042) 

3.275 

(0.040) 0.213 [0.058] {3.712} (<0.001)*** 

Number of fruit groups 

(range: 0-3) 

1.224 

(0.025) 

1.109 

(0.023) 0.115 [0.034] {3.356} (0.001)*** 

Number of vegetable 

groups (range: 0-5) 

2.265 

(0.030) 

2.166 

(0.027) 0.098 [0.040] {2.459} (0.014)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

8.5 Differences in Impacts by Demographic Subgroup 

Findings above demonstrated statistically significant impacts of HIP on the treatment group as a 

whole. We were also interested in examining whether some types of individuals experienced larger or 

smaller impacts of HIP, so we assessed whether impacts differed across subgroups defined by 

household and individual demographic characteristics. To maximize power for these comparisons, we 

defined binary subgroup designations such that as close as possible to half of respondents fell into 

each group. However, for some demographic characteristics with categorizations not lending 

themselves to binary classification (e.g., race/ethnicity, education status), we considered more 

disaggregated subgroups. In particular, we tested for differences in impacts on TFV intake by the 

following:105

 Respondent gender (males vs. females) 

 Respondent age group (age 16-40 years vs. 41+ years) 

 Respondent education level (less than high school, including GED vs. high school diploma 

vs. more than high school) 

 Respondent race/ethnicity (Hispanic vs. non-Hispanic white vs. non-Hispanic black vs. nonHispanic other race) 

 Respondent disability status (disabled vs. non-disabled) 

 Primary shopper employment status (employed full- or part-time vs. not employed) 

 Household composition (households with children and no elderly vs. other households) 

 Household WIC participation (WIC participants vs. non-participants) 

 SNAP benefit size ($200 or less vs. over $200) 

 

105 Appendix F, Exhibits F2.1–F2.3 presents baseline characteristics for these measures. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 155

Exhibit 8.10 displays impact estimates on TFV intake by demographic subgroup.106 We find no 

evidence of differential impacts by any of the characteristics examined. We note, however, that our 

study was not powered to detect subgroup differences. 

Exhibit 8.10: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables, Cup-Equivalents, by Demographic Subgroup 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Respondent gender (N=3,913 recalls from 2,009 respondents)

Females 1.170 

(0.060) 

0.903 

(0.050) 

0.267 

(<0.001)*** 

Males 1.103 

(0.096) 

0.926 

(0.080) 

0.178 

(0.100)* 

Impact: females – males 0.090 

(0.477) 

Respondent age group (N=3,913 recalls from 2,009 respondents)

16-40 years 0.366 

(0.188) 

0.173 

(0.198) 

0.193 

(0.006)*** 

41+ years 1.462 

(0.086) 

1.179 

(0.081) 

0.283 

(0.001)*** 

Impact: 16-40 years – 41+ years -0.090 

(0.408) 

Respondent educational attainment (N=3,892 recalls from 2,000 respondents)a

Less than high school 

(including GED) 

1.186 

(0.064) 

0.873 

(0.048) 

0.314 

(<0.001)*** 

High school diploma 1.034 

(0.088) 

0.928 

(0.074) 

0.106 

(0.359) 

More than high school 1.155 

(0.077) 

0.976 

(0.074) 

0.179 

(0.084)* 

P-value for difference c (0.292) 

Respondent race/ethnicity (N=3,913 recalls from 2,009 respondents)

Hispanic 1.169 

(0.068) 

0.955 

(0.051) 

0.215 

(0.005)*** 

Non-Hispanic white 1.013 

(0.074) 

0.782 

(0.058) 

0.231 

(0.010)** 

Non-Hispanic black 1.168 

(0.121) 

0.949 

(0.108) 

0.220 

(0.151) 

Non-Hispanic other 1.724 

(0.249) 

1.241 

(0.225) 

0.483 

(0.153) 

P-value for difference c (0.895) 

Respondent disability status (N=3,913 recalls from 2,009 respondents)

Disabled 1.141 

(0.063) 

0.936 

(0.052) 

0.205 

(0.009)*** 

Non-disabled 1.157 

(0.063) 

0.884 

(0.050) 

0.273 

(<0.001)*** 

 

106 Corresponding estimates for targeted fruits, targeted vegetables, and total fruits and vegetables appear in 

Appendix H (Exhibits H8.1, H8.3, H8.5). No consistent patterns in subgroup differences were detected for 

these additional outcomes, although the impact of HIP on intake of total fruits and vegetables was found to 

be statistically significantly higher for those of lower educational attainment. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 156 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Impact: disabled – non-disabled -0.068 

(0.536) 

Primary shopper employment status (N=3,751 recalls from 1,916 respondents)b

Working full or part-time 1.079 

(0.091) 

0.925 

(0.071) 

0.155 

(0.171) 

Not working 1.166 

(0.050) 

0.894 

(0.038) 

0.272 

(<0.001)*** 

Impact: working – not working -0.118 

(0.364) 

Household composition (N=3,913 recalls from 2,009 respondents)

Children (and no elderly) in 

household 

1.312 

(0.077) 

1.030 

(0.069) 

0.282 

(<0.001)*** 

Other household 1.029 

(0.064) 

0.823 

(0.050) 

0.206 

(0.006)*** 

Impact: Children in HH– Other HH 0.076 

(0.492) 

Household WIC status (N=3,744 recalls from 1,915 respondents)b

Participant 1.040 

(0.093) 

0.859 

(0.071) 

0.181 

(0.082)* 

Non-participant 1.171 

(0.051) 

0.911 

(0.038) 

0.261 

(<0.001)*** 

Impact: participant – non-participant -0.079 

(0.513) 

Household SNAP benefit amount (N=3,913 recalls from 2,009 respondents)

$200 or less 1.135 

(0.069) 

0.909 

(0.053) 

0.227 

(0.004)*** 

Over $200 1.166 

(0.072) 

0.913 

(0.061) 

0.253 

(0.001)*** 

Impact: $200 or less – over $200 -0.027 

(0.805) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted fruit and vegetable intake proxy measure includes intake of fruits and vegetables acquired from the 

store. It excludes white potatoes, legumes, and 100% juice, as well as mixed foods where the source of 

individual ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Sample size is smaller for this subgroup analysis due to missing data on educational attainment for some 

sampled respondents. 

b

 Sample size is smaller for this subgroup analysis because the primary shopper employment status and 

household WIC status items are included in the primary shopper survey, which was not completed in all 

households with a sampled respondent completing a dietary recall interview. 

c

 For demographic characteristics with more than two subgroup categories (respondent educational attainment 

and respondent race/ethnicity), p-value in parentheses represents significance level for joint test across all 

categories. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 157

8.6 Differences in Impacts by Baseline Fruit and Vegetable-Related 

Behaviors and Preferences 

In addition to differences in impacts by demographic subgroup, we are interested in understanding 

how HIP impacts differed by baseline fruit and vegetable-related behaviors and preferences. For 

example, larger impacts might be expected among those individuals who already eat and enjoy fruits 

and vegetables because they may be more responsive to opportunities to increase their intake. 

Alternatively, if HIP succeeds in shifting attitudes and preferences toward fruits and vegetables, 

larger impacts might be expected among those who started off with lower levels of intake or weaker 

preferences for fruits and vegetables, since those individuals have more room to improve. 

To provide exploratory evidence on this question, we assessed differences in impacts on TFV intake 

by the following baseline respondent characteristics:107

 Baseline fruit and vegetable intake from screener (three or more servings/day vs. less than 

three servings per day)108

 Predicted baseline TFV intake (above vs. below median) 

 Predicted baseline TFV spending (above vs. below median) 

 Attitudes about fruit and vegetables scale (above vs. below median) 

 Barriers to eating fruit and vegetables scale (above vs. below median) 

 Barriers to grocery shopping (above vs. below median) 

 Fruit and vegetables at home scale (above vs. below median) 

 Pre-HIP shopping patterns (primarily at HIP vs. non-HIP participating retailers) 

Exhibit 8.11 provides these subgroup estimates for TFV intake.109 We find no evidence of a larger 

impact for those with higher reported baseline intake of fruits and vegetables from the EATS 

screener; as noted above, this is a relatively noisy measure of intake as compared to the 24-hour 

dietary recall interview (which was not conducted at baseline). We also find no evidence of a larger 

impact for those with higher predicted baseline levels of TFV intake, though there is a borderline 

significant difference in impact by predicted baseline levels of TFV spending. 

However, we do find some evidence that stronger preferences toward fruits and vegetables at baseline 

predict stronger HIP impacts. In particular, impacts were higher among those with above-median 

baseline scores on the attitudes about fruit and vegetables scale. These findings suggest that HIP may 

 

107 Appendix D, Section D.2 includes a discussion of scale creation. Appendix F, Exhibits F7.1-F7.4, F8.4 

present baseline responses to these items. 

108 As discussed in Section 8.3, fruit and vegetable intake from the screener is converted to standardized cupequivalents for each respondent; 1.5 cup-equivalents of fruits and vegetables represents 3 servings, which is 

close to the median intake level. 

109 Corresponding estimates for intake of targeted fruits, targeted vegetables, and total fruits and vegetables 

appear in Appendix H (Exhibits H8.2, H8.4, H8.6); findings are similar to those presented here for TFV 

intake. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 158 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

be more successful in increasing intake among those that already enjoy and consume fruits or 

vegetables, as opposed to those who dislike or do not regularly consume these items.110

Exhibit 8.11: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables, Cup-Equivalents, by Baseline Fruit and Vegetable-Related 

Behaviors and Preferences and Shopping Patterns Subgroup 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline fruit and vegetable intake (screener) (N=3,913 recalls from 2,009 respondents)

3+ servings/day 1.199 

(0.076) 

0.909 

(0.061) 

0.290 

(0.001)*** 

<3 servings/day 1.099 

(0.063) 

0.912 

(0.054) 

0.187 

(0.003)*** 

Impact: 3+ servings – <3 servings 0.103 

(0.346) 

Baseline TFV intake (predicted) (N=3,162 recalls from 1,604 respondents)

High (above median) 1.247 

(0.089) 

0.909 

(0.057) 

0.337 

(0.001)*** 

Low (below median) 1.077 

(0.070) 

0.878 

(0.065) 

0.199 

(0.004)*** 

Impact: high – low 0.138 

(0.256) 

Baseline TFV spending (predicted) (N=3,162 recalls from 1,604 respondents)

High (above median) 1.254 

(0.080) 

0.878 

(0.055) 

0.376 

(<0.001)*** 

Low (below median) 1.078 

(0.071) 

0.910 

(0.063) 

0.168 

(0.047)** 

Impact: high – low 0.208 

(0.093)* 

Attitudes about food, fruits, and vegetables (N=3,913 recalls from 2,009 respondents)

High (above median) 1.305 

(0.067) 

0.946 

(0.054) 

0.359 

(<0.001)*** 

Low (below median) 0.950 

(0.073) 

0.862 

(0.060) 

0.088 

(0.225) 

Impact: high – low 0.272 

(0.011)*** 

Barriers to eating fruits and vegetables (N=3,913 recalls from 2,009 respondents)

High (above median) 1.155 

(0.071) 

0.855 

(0.047) 

0.300 

(<0.001)*** 

Low (below median) 1.132 

(0.054) 

0.967 

(0.056) 

0.164 

(0.035)** 

Impact: high – low 0.136 

(0.237) 

 

110 Appendix H, Exhibits H8.7-H8.15 present subgroup analyses for the outcomes presented in this rest of this 

chapter. Subgroups included: baseline fruit and vegetable intake (screener), attitudes about food, fruits, and 

vegetables scale, and fruits and vegetables at home scale. Differences are discussed in the main text only 

when consistent patterns emerged that provided context for the pooled findings. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 159

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Barriers to grocery shopping (N=3,913 recalls from 2,009 respondents)

High (above median) 1.150 

(0.098) 

0.886 

(0.074) 

0.264 

(0.004)*** 

Low (below median) 1.148 

(0.074) 

0.932 

(0.064) 

0.216 

(0.001)*** 

Impact: high – low 0.048 

(0.672) 

Fruits and vegetables at home (N=3,913 recalls from 2,009 respondents)

Frequently 1.122 

(0.085) 

0.940 

(0.067) 

0.182 

(0.042)** 

Infrequently 1.169 

(0.066) 

0.887 

(0.060) 

0.282 

(<0.001)*** 

Impact: frequently – infrequently -0.100 

(0.377) 

Pre-HIP shopping patterns (N=2521 recalls from 1,327 respondents)

Shopped primarily at HIP 

participating retailers 

1.187 

(0.065) 

0.885 

(0.058) 

0.302 

(<0.001)*** 

Shopped primarily at nonHIP participating retailers 1.258 

(0.098) 

0.887 

(0.063) 

0.371 

(0.002)*** 

Impact: HIP shoppers – non-HIP shoppers -0.070 

(0.632) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

TFV intake proxy measure includes intake of fruits and vegetables acquired from the store. It excludes white 

potatoes, legumes, and 100% juice, as well as mixed foods where the source of individual ingredients was not 

identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round. 

8.7 Differences between Round 2 and Round 3 

The earlier sections of this chapter reported results pooling all 24-hour dietary recalls (Round 2 and 

Round 3; main interview for the entire sample and the 10 percent sub-sample second interview). We 

collected two rounds of follow-up data from HIP participants, in part to support an exploratory 

analysis of changes in HIP impacts over time. If pilot implementation and participant understanding 

improve with HIP maturity, one might expect impacts to increase between Rounds 2 and 3 of the 

study. On the other hand, impacts of many nutritional interventions tend to attenuate over time, so we 

might expect to see a decrease in impacts between Rounds 2 and 3. 

To explore this issue, Exhibit 8.12 reports differences between Round 2 and Round 3 impacts on TFV 

intake. We find no statistically significant increase or decrease in TFV intake across rounds. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 160 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

Exhibit 8.12 Differences in Impact of HIP on Consumption of Targeted Fruits and 

Vegetables, Cup-Equivalents, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Pooled (Rounds 2 & 3) 1.149 

(0.043) 

0.910 

(0.035) 

0.238 

(<0.001)*** 

Round 2 1.179 

(0.058) 

1.003 

(0.051) 

0.176 

(0.020)** 

Round 3 1.097 

(0.055) 

0.862 

(0.049) 

0.235 

(0.001)*** 

Change: Round 3 – Round 2 

0.040 

(0.653) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

TFV intake proxy measures include intake of fruits and vegetables acquired from the store, excluding white 

potatoes, legumes, and 100% juice, as well as mixed foods where the source of individual ingredients was not 

identified by the respondent. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

8.8 Secondary Dietary Outcomes 

One purpose of HIP is to encourage healthful eating patterns, which may in turn improve overall 

nutritional status and lower the risk of becoming overweight or obese and of related chronic 

conditions. It is therefore of interest to examine HIP impacts on foods other than fruits and 

vegetables, which may provide suggestive evidence as to whether observed higher targeted fruit and 

vegetable intake among HIP participants is an indication that they are substituting fruits and 

vegetables for other foods in their diets. In addition, we estimate impacts on broader measures of 

dietary status including total food energy, fiber, and micronutrients commonly found in fruits and 

vegetables. 

Other Intake 

Exhibit 8.13 reports impacts of HIP on foods other than fruits and vegetables for each of the major 

USDA Food Pattern food groups.111 If participants were substituting fruit and vegetable consumption 

for other types of intake, then we might expect concurrent decreases in consumption of other foods. 

Alternatively, income effects from the HIP incentive might result in an increase in consumption of 

other foods. (We note, however, that earnings from HIP incentives do not seem large enough to 

induce a substantial income effect; see Chapter 5, Section 5.3 and Appendix A.) To address these 

issues, we report intake estimates for nine USDA Food Pattern food group measures. These results 

 

111 Exhibit 8.13 includes all major USDA Food Pattern non-fruit, non-vegetable food groups, including 

“discretionary” foods such as alcohol, added sugar, and solid fats and oils, to provide a comprehensive 

picture of dietary composition across all major categories of intake. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 161

should be interpreted with care, as multiple comparisons considerations suggest that we would expect 

some of them to appear to be statistically significant, even if there was no true impact. 

We find statistically significantly lower grain consumption among the HIP participant group of 0.44 

ounce-equivalents,112 driven by a 0.43 ounce-equivalent difference in intake of refined grains. We 

also see greater alcohol consumption among HIP participants of 0.08 drinks,113 though this result 

appears to have been driven by a handful of extreme outliers in Round 2 who reported consuming 

eight or more alcoholic drinks per day; when these seven individuals are excluded from the sample, 

the difference between treatment and control groups is no longer statistically significant. Especially 

given concerns about multiple comparisons noted above, these patterns of impacts are not sufficiently 

consistent to provide clear evidence in support of either a substitution or an income effect. 

 

112 One ounce equivalent of grains is equal to one slice of bread, one cup of ready-to-eat cereal, or 1⁄2 cup of 

cooked rice, cooked pasta, or cooked cereal. 

113 One drink of alcohol is equal to 12 ounces of regular beer, 5 ounces of wine, or 11⁄2 ounces of 80-proof 

distilled spirits. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 162 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

Exhibit 8.13: Impact of HIP on Consumption of Other Foods, by USDA Food Pattern Food 

Group 

Regression-adjusted mean 

(SE) Impact 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Total grains (oz-eq) 5.067 

(0.102) 

5.512 

(0.099) -0.444 [0.143] {-3.106} (0.002)*** 

Whole grains (oz-eq) 0.632 

(0.027) 

0.647 

(0.028) -0.016 [0.039] {-0.402} (0.688) 

Refined grains (oz-eq) 4.436 

(0.095) 

4.864 

(0.093) -0.429 [0.135] {-3.179} (0.002)*** 

Total dairy (milk, yogurt, 

cheese, whey) (cup-eq) 

1.580 

(0.045) 

1.560 

(0.040) 0.020 [0.060] {0.331} (0.740) 

Total protein foods (oz-eq) 5.035 

(0.119) 

5.063 

(0.099) -0.028 [0.155] {-0.180} (0.857) 

Oils (gm-eq) 17.40 

(0.47) 

18.36 

(0.48) -0.96 [0.68] {-1.406} (0.160) 

Solid fats (gm-eq) 29.74 

(0.73) 

31.43 

(0.68) -1.69 [1.02] {-1.660} (0.097)* 

Added sugars (tsp)a 15.16 

(0.47) 

15.79 

(0.43) -0.63 [0.64] {-0.986} (0.324) 

Alcoholic drinks (drinks) b,c 0.209 

(0.031) 

0.132 

(0.017) 0.077 [0.035] {2.211} (0.027)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a Includes, for example, the sugar added to sweetened soft drinks consumed. 

b One drink is defined as the amount of alcoholic beverage containing 0.6 fluid ounces or 14 grams of alcohol. c Note that this result appears to be driven by the presence of several extreme outliers during Round 2 who 

reported 8 or more drinks of alcohol in the prior 24 hours. Excluding those individuals (7 in all) from the analysis, 

differences between treatment and control groups are no longer statistically significant. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

Total Food Energy 

The impact of HIP on total food energy is of interest because increasing fruit and vegetable intake 

without increasing total food energy is thought to decrease risk of weight gain (USDA and USDHHS, 

2010). 

The estimated impact of HIP on total energy intake was small (49 fewer kilocalories per day in the 

HIP group) and not statistically significant (Exhibit 8.14). However, we note that our study was not 

powered to detect an impact on total energy of the magnitude that would be implied by the observed 

one-third of a cup-equivalent increase in total fruit and vegetable consumption. In the absence of a 

statistically significant impact in either direction, we cannot determine whether higher fruit and 

vegetable consumption associated with HIP participation was in addition to or in place of the 

consumption of other foods. Doing so would have required a random sample several times as large. 

In addition, we note that regression-adjusted mean energy intake in our sample was somewhat lower 

than in the U.S. population as a whole. Nationally representative estimates from the National Health 

and Nutrition Examination Survey (NHANES) for 2009-2010 suggest mean energy intake of 2,133 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 163

kilocalories per day among adults aged 20 and over, as compared to our estimated control group mean 

of 1,797 kilocalories. This difference is explained in part by the fact that approximately three-quarters 

of our sample is female; in the NHANES 2009-2010 sample, females age 20 and over consumed 

approximately 1,778 kilocalories per day, as compared to intake levels of 2,512 kilocalories per day 

among males in the same age group (USDA, 2012). Weighting the NHANES estimates to reflect the 

gender composition of our sample would imply intake levels of 1,976 kilocalories per day, making up 

a substantial portion of the gap. In addition, intake of total food energy is generally lower among 

SNAP participants (Cole and Fox, 2008), which may explain the remaining difference. 

Exhibit 8.14: Impact of HIP on Total Energy Intake 

Regression-adjusted 

mean (SE) Impact 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Total energy (kcal) 1749 

(28) 

1797 

(25) -49 [38] {-1.28} (0.201) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

Fiber and Micronutrients 

To assess impacts of HIP on the overall nutritional profile of HIP participants, we analyzed intake of 

fiber and micronutrients most commonly found in targeted fruits and vegetables (Exhibit 8.15). We 

find statistically significant impacts on vitamin C intake, consistent with the observed increase in 

consumption of fruit; many fruits are good sources of vitamin C. 

Exhibit 8.15: Impact of HIP on Nutrient Intake 

Regression-adjusted mean 

(SE) Impact 

Treatment (T) Control (C) T-C [S.E.] {t-statistic} (P-value) 

Fiber (g) 13.47 

(0.25) 

13.09 

(0.23) 0.38 [0.33] {1.13} (0.258) 

Beta carotene (mcg) 1685 

(93) 

 1492 

(80) 193 [121] {1.59} (0.112) 

Vitamin A (mcg RAE) 586 

(18) 

 571 

(17) 15 [24] {0.63} (0.528) 

Vitamin C (mg) 107 

(3) 

 93 

(3) 14 [4] {3.31} (0.001)*** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 164 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

Other Ingredients in Foods with Fruits and Vegetables 

Finally, we were interested in knowing whether HIP participants shifted consumption toward fruits 

and vegetables without added ingredients that would disqualify those foods from earning the HIP 

incentive (sodium in fruits; discretionary oils, solid fats, and added sugar in fruits or vegetables). 

Exhibit 8.16 shows impacts on intake of sodium, discretionary oils, discretionary solid fat, and added 

sugar in foods that also contain fruit or vegetable ingredients. (Note that these estimates do not reflect 

total intake from all sources, but only amounts included in foods containing fruits or vegetables.) We 

did not detect any statistically significant differences in intake between treatment and control groups. 

Exhibit 8.16: Impact of HIP on Other Ingredients in Foods Containing Fruits and Vegetables 

Regression-adjusted mean 

(SE) Impact 

Treatment (T) Control (C) T-C [S.E.] {t-statistic} (P-value) 

Sodium (mg) 929 

(26) 

981 

(28) -52 [39] {-1.33} (0.185) 

Discretionary oils (gmeq) 7.04 

(0.28) 

7.66 

(0.31) -0.62 [0.42] {-1.48} (0.140) 

Discretionary solid fats 

(gm-eq) 

6.55 

(0.30) 

6.96 

(0.33) -0.41 [0.47] {-0.86} (0.387) 

Added sugar (tsp) 2.95 

(0.15) 

2.86 

(0.18) 0.10 [0.24] {0.40} (0.688) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that totals reported here reflect only daily intake from foods that also contain fruits and vegetables, and do 

not reflect total intake from all sources. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

8.9 Dietary Quality 

Since HIP appears to have successfully increased fruit and vegetable consumption, one might expect 

that the pilot also improved overall dietary quality among participants. To assess the extent to which 

such improvement occurred, we examined HIP impacts on the proportion of participants who 

successfully complied with selected guidelines from the 2010 Dietary Guidelines for Americans 

(DGAs), and on overall scores on the 2010 Healthy Eating Index (HEI-2010). 

Unlike analyses reported in other sections of this chapter, assessment of impacts on these outcomes 

requires estimation of the distribution of usual intake in the study population, as opposed to a single 

point estimate of mean daily intake over the past 24 hours. We employed the NCI method (Tooze et 

al., 2006) to estimate the distribution of usual intake for the analyses of impacts on the 2010 DGAs. 

For the HEI-2010, we employed the population ratio method (Freedman et al., 2010) to obtain 

estimates of usual HEI-2010 scores. More details of the usual intake estimation approach appear in 

Appendix E.8. 

Dietary Guidelines for Americans 

Exhibit 8.17 shows estimated impacts of HIP on the proportion of respondents meeting the following 

2010 DGAs: 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 165

 Daily recommendations for total fruits 

 Daily recommendations for total vegetables 

 Weekly recommendations for USDA Food Pattern vegetable subgroups (dark green 

vegetables, red and orange vegetables, beans and peas (legumes), starchy vegetables, and 

other vegetables) 

 Daily recommendations for total, whole, and enriched114 grains 

 Daily recommendations for protein foods 

 Daily recommendations for total dairy 

 Daily allowance for oils 

 Daily allowance for solid fats and added sugars (SoFAS) 

There were no statistically significant differences in proportions of respondents meeting the 2010 

DGAs. In general, though point estimates for the 2010 DGA impacts were consistent in direction with 

impacts on mean intake of corresponding USDA Food Pattern food groups and subgroups as reported 

above, statistical precision of the DGA estimates was quite low, perhaps explaining the null findings. 

 

114 Note that the USDA Food Pattern equivalent subgroups refer to non-whole grains as “refined grains.” 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 166 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

Exhibit 8.17: Impact of HIP on Proportion of Respondents Meeting 2010 Dietary Guidelines 

for Americans 

Regression-adjusted 

proportion (SE) Impact 

2010 DGAs Treatment (T) Control (C) T-C [S.E.] {t_statistic} (P-value) 

Percent of respondents at or above recommendations for...

Total fruit 0.255 

(0.175) 

0.177 

(0.193) 0.078 [0.260] {0.765} (0.765) 

Total vegetables 0.062 

(0.221) 

0.043 

(0.225) 0.019 [0.315] {0.952} (0.952) 

Dark green vegetables 0.100 

(0.210) 

0.035 

(0.225) 0.065 [0.308] {0.832} (0.832) 

Red and orange 

vegetables 

0.021 

(0.229) 

0.017 

(0.230) 0.004 [0.325] {0.990} (0.990) 

Legumes 0.132 

(0.202) 

0.150 

(0.198) -0.018 [0.282] {0.950} (0.950) 

Starchy vegetables 0.126 

(0.206) 

0.119 

(0.208) 0.007 [0.293] {0.980} (0.980) 

Other vegetables (e.g. 

celery, cucumbers, 

mushrooms, green 

beans, onions, 

asparagus) 

0.195 

(0.189) 

0.144 

(0.201) 0.051 [0.276] {0.853} (0.853) 

Total grains 0.241 

(0.177) 

0.329 

(0.157) -0.088 [0.237] {0.711} (0.711) 

Whole grains 0.001 

(0.233) 

0.001 

(0.233) <0.001 [0.330] {0.999} (0.999) 

Enriched grains 0.838 

(0.040) 

0.894 

(0.026) -0.056 [0.048] {0.244} (0.244) 

Protein foods (meat, 

poultry, seafood, eggs, 

nuts, seeds, & processed 

soy products) 

0.385 

(0.144) 

0.404 

(0.139) -0.019 [0.200] {0.923} (0.923) 

Dairy 0.060 

(0.221) 

0.063 

(0.220) -0.003 [0.311] {0.991} (0.991) 

Percent of respondents below allowances for...

Oils 0.853 

(0.199) 

0.818 

(0.191) 0.035 [0.276] {0.899} (0.899) 

Calories from solid fats & 

added sugars (SoFAS) 

0.064 

(0.017) 

0.053 

(0.014) 0.011 [0.022] {0.600} (0.600) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted proportions for the treatment and control groups. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

Healthy Eating Index 

Exhibit 8.18 shows estimates of HIP impacts on the 2010 Healthy Eating Index, including both the 

total score and individual components. Consistent with impacts on total intake of fruits and vegetables 

as described above, all four fruit and vegetable components of the HEI-2010 were significantly higher 

among the treatment group following HIP implementation. These differences drove a parallel increase 

in the total HEI-2010 score, which was approximately five points higher in the treatment group. 

Interestingly, regression-adjusted mean HEI-2010 scores for both the treatment and control groups 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 8. Impact on Fruit and Vegetable Consumption ▌pg. 167

(61.9 and 57.1, respectively) were somewhat higher than the 53.5 average seen nationally among US 

adults (Guenther et al., 2013). As for total food energy intake as reported above, some of this 

difference may be explained by the fact that our sample is disproportionately female; however, 

reweighting NHANES-based estimates to reflect the gender composition of the HIP evaluation 

sample would imply a reweighted score of 53.8, so the gender difference alone does not fully explain 

the gap. 

Exhibit 8.18: Impact of HIP on Healthy Eating Index-2010, Components and Total Score 

Regression-adjusted 

mean (SE) Impact 

HEI-2010 Component 

(maximum score) 

Treatment 

(T) 

Control 

(C) T-C [S.E.] 

{tstatistic} (P-value) 

Adequacy (higher score indicates higher consumption)

Total fruit (5) 4.39 

(0.18) 

3.53 

(0.14) 0.85 [0.23] {3.769} (<0.001)*** 

Whole fruit (5) 4.91 

(0.14 

4.00 

(0.21) 0.91 [0.25] {3.645} (<0.001)*** 

Total vegetables (5) 3.70 

(0.10) 

3.35 

(0.09) 0.35 [0.13] {2.738} (0.006)*** 

Greens and beans (5) 3.02 

(0.17) 

2.47 

(0.17) 0.55 [0.24] {2.301} (0.022)** 

Whole grains (10) 2.40 

(0.11) 

2.36 

(0.11) 0.04 [0.15] {0.272} (0.786) 

Dairy (10) 6.89 

(0.18) 

6.67 

(0.16) 0.22 [0.24] {0.919} (0.358) 

Total protein foods (5)a 5.00 

(<0.01) 

5.00 

(<0.01) <0.01 [<0.01] -- (1.000) 

Seafood and plant proteins 

(5) 

3.48 

(0.33) 

2.88 

(0.21) 0.06 [0.39] {1.531} (0.126) 

Fatty acids (10) 3.36 

(0.16) 

3.48 

(0.15) -0.12 [0.22] {-0.548} (0.584) 

Moderation (higher score indicates lower consumption)

Refined grains (10) 7.08 

(0.16) 

6.38 

(0.16) 0.71 [0.22] {3.152} (0.002)*** 

Sodium (10) 4.72 

(0.16) 

4.43 

(0.17) 0.29 [0.24] {1.239} (0.215) 

Empty caloriesb (20) 12.90 

(0.30) 

12.57 

(0.27) 0.33 [0.41] {0.810} (0.418) 

Total HEI-2010 score (100) 61.85 

(0.89) 

57.12 

(0.79) 4.73 [1.19] {3.975} (<0.001)*** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. a

All respondents in both treatment and control groups achieved the maximum score of 5 on the total protein 

foods component of the HEI-2010. 

b

Includes calories from solid fat, alcohol, and added sugars (SoFAAs) 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10 

percent second-day subsamples for each round; (unweighted N=3,913 recalls from 2,009 respondents). 

8.10 Discussion 

This chapter has reported random assignment-based estimates of the impact of HIP on food intake. 

The results show statistically significant and meaningful impacts—a little less than one-quarter of a 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 168 ▌8. Impact on Fruit and Vegetable Consumption Abt Associates

cup-equivalent, or 26 percent higher than the control group intake level—on our chosen (restrictive) 

proxy for intake of fruits and vegetables qualifying for the HIP. These results are robust when using 

the alternative inclusive TFV intake proxy measure. Exploratory analyses find impacts on both fruits 

and vegetables, distributed across several USDA Food Pattern fruit and vegetable subgroups. These 

results suggest that HIP achieved the goal of increasing fruit and vegetable consumption of SNAP 

participants. 

There are no differential impacts by demographic subgroups. However, some evidence shows 

differential impacts by baseline attitudes toward fruits and vegetables. Those who began with more 

positive attitudes about fruits and vegetables and those who were more likely to have fruits and 

vegetables available in the home have larger impacts; and impacts were borderline significantly 

higher for individuals with higher predicted baseline levels of TFV intake. However, there is no 

evidence of a differential impact by baseline food intake—as measured by the EATS screener. We 

conjecture that this null result may be due to the less precise measurement properties of the EATS 

screener. 

There is no statistically significant impact on total energy intake (though our power to detect an 

impact on this outcome is low). There is a positive impact on Vitamin C, but not on fiber and the 

other micro-nutrients (beta carotene and Vitamin A) considered to be indicators of fruit and vegetable 

consumption. 

Finally, this chapter continued the effort begun in Chapter 6 to reconcile the key estimates of 

spending for and intake of fruits and vegetables. Chapter 6 noted that the HIP impact on total selfreported fruit and vegetable spending, based on survey measures, was more than five times as large as 

the HIP impact on TFV spending with EBT benefits in participating supermarkets/superstores as 

estimated from EBT transaction data. Impacts on survey-based intake measures reported in this 

chapter were roughly equivalent to survey-based spending measures from Chapter 6, and hence much 

larger than the impacts that one would expect based on impact estimates from the EBT transactions 

data. In Chapter 10, we provide a more detailed discussion of possible explanations for this 

difference. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 169

9. Costs of Pilot and Feasibility of Nationwide Expansion 

This chapter provides details on the cost of the Healthy Incentives Pilot in Massachusetts and the 

estimated costs and feasibility of a nationwide expansion of HIP. HIP affected both the systems and 

processes used for EBT transactions. Implementation of HIP entailed costs for changes to the EBT 

system and the retailer and third-party processor (TPP) systems that interact with the EBT system. 

Implementation of HIP also entailed costs for changes to EBT processes (e.g., the establishment of 

policies, procedures and standards) and for one-time processes to notify and train stakeholders. This 

chapter also considers how HIP may affect the operational costs of EBT processes, and to what extent 

the new processes may be absorbed into ongoing operations of stakeholders without requiring 

additional resources. 

The feasibility of expanding a program such as HIP is based on the ability to modify or create 

systems, change procedures and fund the project. For HIP, we provide estimated costs for expansion 

but make no judgment as to whether or how the program would be funded or which entities would 

bear the costs of expansion. 

Total costs of the pilot were $4,441,992. This included DTA’s implementation costs of $4,109,087, 

FNS’ contractor cost of $69,862 for oversight of testing, and incentive payments to HIP participants 

of $263,043. For nationwide expansion of HIP, we project that implementation costs would total 

approximately $89.8 million. The annual value of incentives earned under nationwide expansion 

would depend on participant behaviors that cannot be predicted fully from the pilot. Estimates of 

incentives based on plausible scenarios range from $0.8 billion to $4.5 billion annually. 

The first section of the chapter describes the pilot cost data collected for the evaluation and the 

framework for analyzing pilot and expansion costs. The second section presents the pilot costs 

incurred by DTA, its contractors, and FNS’ testing oversight contractor. The third section presents the 

projected costs for statewide and nationwide expansion of HIP, including annual projections of 

implementation costs and incentive earnings. The fourth section discusses stakeholder perspectives on 

the feasibility of expansion and recommendations for how best to implement HIP on a nationwide 

basis. The chapter concludes with a discussion of key results and outstanding issues. 

9.1 Data Description 

This section summarizes the roles of the stakeholders included in the cost analysis, describes the cost 

data sources, and identifies the costs excluded from the analysis. Most of the data sources described 

in this section were used to calculate the non-recurring implementation costs. The costs for incentive 

payments come directly from pilot expenditures. 

Stakeholders Included in the Cost Analysis 

The cost data collection captured the expenses of the stakeholders participating in the implementation 

and operation of HIP in Massachusetts. These data were used to estimate the cost of the pilot and also 

the projected costs of a nationwide expansion of HIP. The primary stakeholders and their roles in HIP 

implementation and operations are identified in Exhibit 9.1. (The roles of the stakeholders are 

described in more detail in Chapter 3.) Stakeholders’ costs captured in total pilot costs include: 

 Massachusetts DTA. FNS awarded a grant to DTA to implement HIP in Hampden County. 

DTA costs included time for existing management and information technology (IT) staff, 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 170 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

additional staff hired to implement HIP, payments to contractors, supplies and other direct 

costs, and indirect costs for personnel. DTA’s partners that received HIP funds included 

Xerox (the EBT processor), retailers, third-party processors, the Novo Dia Group (technical 

consultant for retailer systems), and the Massachusetts Department of Agricultural Resources 

(for farmers market solutions). 

 Xerox, the EBT processor. Xerox incurred personnel and overhead costs for its role in HIP 

design, development, testing, and implementation. These costs were paid with funds from the 

HIP grant. DTA modified its existing contract with Xerox to include HIP-related expenses. 

 Retailers. Under agreements with DTA, major retailers using integrated electronic cash 

register (IECR) systems received HIP grant funds to modify their systems in order to identify 

HIP items and process HIP transactions. The retailers used these funds for internal 

expenses,115 payments to contractors for IECR system modifications,116 and payments to 

TPPs for testing. Xerox’s costs included modifying and deploying new HIP-capable “EBTonly” point-of-sale (POS) terminals used by the rest of the participating retailers (i.e., the 

non-IECR retailers) at no cost to the retailers. 

 Third-party processors. For TPPs that route transactions from retailers’ IECRs or 

commercial point-of-sale (POS) terminals to the appropriate processor or network for 

authorization, HIP required TPPs to modify their systems so that they could capture and 

forward additional information (the HIP purchase amount) for SNAP EBT transactions, and 

receive receipt data. Some of the HIP grant funds paid to retailers were used by retailers to 

support TPP system changes. 

 Technical consultant services. DTA contracted with Nova Dia Group (NDG) to provide a 

wide range of technical consulting services to support major retailers and their third-party 

processors for system modification, testing, trouble-shooting and problem-resolution 

throughout HIP implementation and operations. The payments to NDG were part of DTA’s 

contractual costs paid with HIP funds. 

 Farmers markets. Through a contract established between Community Involved in 

Sustaining Agriculture Inc. (CISA) and the Massachusetts Department of Agricultural 

Resources (MDAR) and paid for through the HIP grant, farmers markets used one of three 

different systems for processing HIP transactions. Two of the systems were based on 

electronic point-of-sale systems specially programmed to perform HIP transactions; the third 

system was based on a variant of the token system widely used for EBT at Massachusetts 

farmers markets. 

The categories in Exhibit 9.1 refer primarily to the one-time activities to implement HIP. The 

pilot implementation costs measured by the evaluation included operational support for 

participants, retailers, and community partners provided by DTA, Xerox, and NDG personnel 

 

115 Internal retailer expenses were not broken out by cost category and may have included modifications to 

proprietary systems, training for store personnel, identification of UPC codes for HIP-eligible items, and 

project management. 

116 IECR systems included proprietary systems and Retalix, a commercial off-the-shelf (COTS) system used 

by several of the retailers. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 171

specifically tasked to work on HIP. The measured costs also included activities by HIP project 

staff to follow up with stakeholders, document the pilot, and close out operations. Routine 

operational costs incurred by other personnel were not included and, according to stakeholders, 

were immaterial. 

Exhibit 9.1: Stakeholder Groups and Role in HIP 

Stakeholder group 

Project 

management 

and oversight 

Design, 

develop, 

test 

system 

changes 

Recruit 

participants 

and/or 

retailers 

Train 

stakeholders

Support 

the 

evaluation 

of HIP 

Costs 

included in 

the cost 

assessment

USDA FNS    Yesa

MA DTA      Yes 

Local SNAP offices   No 

Xerox (EBT 

processor) 

     Yes 

IECR retailers    Yes 

Third-party 

processors 

  Yes 

Technical 

consultants 

    Yes 

Community-based 

organizations    No 

Farmers markets   Yes 

Abt Associates  No 

a

 For FNS, cost assessment includes only costs for system testing services and incentives. 

Cost Data Sources 

All of the identified one-time costs of implementing HIP were paid for through the HIP grant 

awarded by FNS to DTA, except for FNS’ testing contractor (discussed below). FNS funded the HIP 

incentives and the evaluation from the appropriation for HIP.117 Some stakeholders, including 

retailers with internal IECR systems and TPPs, indicated that additional (i.e., not provided or paid for 

directly by HIP) resources were used to design, develop, and test system changes. As these entities 

were not required to capture labor hours and because of the confidentiality of some of the cost 

information, the expenditure of private funds was not fully captured. Additionally, FNS reported costs 

for system testing services but also spent an unreported amount of administrative (i.e., not HIP) funds 

for staff labor, supplies, and other direct costs (ODCs) to oversee HIP. 

Data for the pilot cost analysis were obtained from the following sources: 

 DTA expenditure reports. These reports provided all expenditures of HIP grant funds by 

DTA, including staff costs (salaries and wages, fringe benefits, and payroll taxes), payments 

to contractors and retailers, supplies and other direct costs, and indirect/overhead costs (as 

charged on the basis of personnel costs). 

 

117 When a participant earned a HIP incentive, an obligation for the amount earned was created. Xerox’s EBT 

system, EPPIC, tracked purchases made with HIP incentive funds separately, and HIP funds were provided 

to settle these transactions. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 172 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

 DTA staff time records. All DTA staff working on HIP reported their time on a weekly basis, 

with a breakdown by the functions presented in the analysis. The time records were used to 

allocate total DTA staff costs and indirect costs among the functions needed to implement 

HIP. 

 Contractor time records. Contractors reported their HIP staff time on a monthly basis, using 

the functional breakdown that was used in the DTA time records. The evaluation team used 

these time records to allocate DTA payments made to contractors to the activities or functions 

involved in implementing HIP. 

 Retailer payment detail. DTA and its technical consultant provided supporting detail for the 

payments to retailers,118 based on the invoices submitted to Xerox by the retailers for HIP 

infrastructure development and testing. This information was used to determine the costs of 

IECR system modification and TPP interface testing for the pilot. 

 EBT system reports. These reports were used to determine the cost of incentives earned by 

participants. 

 FNS information. FNS provided information on the cost of Booz Allen Hamilton (BAH) 

support for testing of changes to the Xerox EBT system and IECR retailer systems. 

All costs captured for the HIP project were one-time costs—i.e., specific to pilot implementation—

with the exception of the actual HIP incentive, which was a recurring cost throughout the project. 

Stakeholders indicated that the operational cost of a HIP transaction during the pilot was no different 

than that of a normal SNAP transaction. In other words, once all system changes were developed and 

tested, no additional system operations costs were incurred, according to stakeholders. DTA’s staff 

who provided participant and retailer support during HIP were hired specifically for the pilot, and 

DTA staff who would provide participant support in routine program operations were not 

significantly involved. Xerox did not identify any incremental costs for retailer and participant 

support during the pilot. The potential for ongoing costs in expansion is discussed in Section 9.3. 

Exclusions from the HIP Pilot Cost Analysis 

The previous sub-section discussed the costs for which we have detailed information and which we 

therefore include in our cost analysis. We do not have detailed cost information for other costs, as 

identified below. We cannot and do not include these costs in our cost analysis, but they ideally 

would be considered in a more complete cost analysis. 

 USDA/FNS. Our analysis includes expenditures by FNS for the HIP incentives and 

contractual costs for testing services by Booz Allen Hamilton (BAH). USDA FNS staff 

resources, supplies and other internal costs were not part of the HIP grant and were not 

tracked separately by FNS and therefore are not included in our cost analysis for the 

evaluation. 

 

118 Payments to retailers were made through Xerox, which was a party to the Memoranda of Understanding 

between DTA and retailers. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 173

 Local SNAP offices. Local SNAP office costs were not captured, but interviews with office 

directors confirmed that their staff time was limited to brief training sessions for staff 

providing general information on HIP and occasional support to individual HIP clients. 

 Community-based organizations (CBOs). The CBOs that assisted with training for HIP 

participants generally did not capture their costs for this effort, and so the cost analysis does 

not include them.119 This gap does not appear to be important to the estimates of costs for 

nationwide expansion of HIP. Information from stakeholders and experts indicates that CBOs 

might augment efforts of SNAP program staff but would likely do so on a voluntary basis, 

using existing staff, focusing on promotion of HIP and nutrition education. 

9.2 Pilot Costs 

This section presents the costs identified for the pilot, beginning with an overview and proceeding 

with discussion of incentive costs and non-recurring implementation costs, including costs for 

contractors, retailer infrastructure investment, and farmers markets. 

Overview of Pilot Costs 

In August 2010, FNS awarded a grant in the amount of $6,392,690 to the Massachusetts Executive 

Office of Health and Human Services, Department of Transitional Assistance (DTA) to implement 

the HIP pilot. Through September 2013, HIP pilot expenditures from the grant to DTA totaled 

$4,109,087 (Exhibit 9.3).120 FNS spent $69,862 for its testing contractor. The total cost of the HIP 

incentive was reported to be $263,043. The total cost of the HIP pilot was therefore $4,441,992, 

including the DTA grant expenditures, FNS’ testing contractor, and incentive payments. (This total 

excludes FNS staff time and ODCs, and other stakeholder costs not identified in available data). 

Incentive Costs 

FNS bore 100 percent of the cost of the HIP incentive. Funds for the incentives redeemed by 

participants were part of the appropriation for HIP but were not part of the grant to DTA. The 

following table (Exhibit 9.2) contains the number of active participants in HIP across all months, the 

amount earned per household per month and the total amount of incentives paid to participants over 

the course of the HIP demonstration. These costs were calculated using EBT system data, which 

includes all transaction data related to SNAP and HIP transactions. As noted in the exhibit, the 

average of $3.42 per household per month is over the entire pilot, including the periods of phase-in 

and phase-out of incentives. The average of $3.65 per household per month for March through 

October 2012 is the best indicator of HIP incentive use for extrapolating to HIP expansion (see 

Section 9.3). 

 

119 The role of CBOs in retailer outreach, materials development, and participant training was substantial and 

provided at no cost to the HIP grant. A few CBOs provided partial information on their costs for this 

process, but the data were too incomplete to use for analysis. 

120 Any additional HIP-related expenditures beyond September 2013 would be considered related to the 

evaluation and therefore not relevant to projections for expansion costs. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 174 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

Exhibit 9.2: Calculation of HIP Incentives for Pilot  

FNS-supported 

incentives: ongoing 

costs 

Cost calculations 

Total $ of incentives 

Total active 

household-months 

$ per HIP household per 

month 

Pilot incentives 263,043 76,943 3.42 

Notes: Total value of incentives earned as computed by the evaluation team from EBT transaction data. Total 

number of “active household months” is the sum of the number of active HIP households over all months of the 

pilot, including all HIP households that had a SNAP transaction (either benefit issuance or purchase). Number of 

active HIP households for November and December 2012 is estimated from actual counts of all households 

labeled as HIP in data that had transactions, including those that were no longer earning incentives (Wave 1 in 

November 2012, and Waves 1 and 2 in December 2012). This table represents the actual average incentive per 

household for the entire pilot. 

Source: EBT transaction data, November 2011-December 2012. 

Implementation Costs 

Exhibit 9.3 summarizes the total pilot implementation costs paid from the DTA grant and by FNS, by 

function and cost category. The types of costs for each function and the key stakeholders are 

identified below. (The total cost is listed in parentheses for each function.) 

System design, development and testing ($2,433,340). DTA, Xerox, retailer and TPP systems were 

modified for HIP. These costs included $1,016,156 paid to contractors, including Xerox and NDG (as 

discussed below) and $1,410,498 for retailers’ infrastructure. The latter cost included $1,283,236 paid 

to retailers and $127,262 in various expenses to develop and implement farmers market solutions 

(details and cost breakdown of these components discussed below). DTA’s internal costs constituted 

the balance, $6,686. 

Household recruiting and customer service ($131,604). DTA developed, translated, printed, and 

mailed notification and recruiting materials to households selected for participation in HIP. DTA HIP 

project staff fielded participant questions and provided HIP dispute support (e.g., answering questions 

about possible errors on receipts). Most of the DTA costs for this function ($118,058) were for 

supplies and other direct costs. About three-quarters of the costs identified as supplies and other direct 

costs were for postage, EBT card sleeves with HIP information, and printing of other participant 

materials. The rest was for temporary consultants hired for HIP and other services. 

Retailer recruiting and relations ($200,668). Retailer participation in HIP was crucial to the success 

of the pilot, which was voluntary for retailers and required either changes to IECR systems or process 

changes for retailers using EBT-only POS terminals. DTA had a staff member dedicated to retailer 

recruiting and ongoing liaison, but other project staff also supported this effort. Nearly all of the cost 

for this function ($193,778) was for DTA’s internal expenses. Xerox staff spent a modest amount of 

time for recruiting retailers for HIP, and fielding retailer questions or concerns, at a cost of $6,890. 

Community relations ($56,929). DTA staff worked with CBOs to encourage and support CBO 

involvement in HIP participant training and to solicit feedback from CBOs via the HIP Steering 

Committee. All of the DTA expenses for this function were personnel-related. 

Training ($275,943). DTA HIP staff developed training materials and provided training about HIP to 

local DTA staff, retailers, and participants. Most of the costs were for personnel, including three 

trainers active during the phase-in of HIP (October 2011 through February 2012). DTA’s costs for 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 175

supplies and ODCs for training ($64,538) included translation services, in-state travel, and 

miscellaneous other items. 

General administration ($703,481). DTA and Xerox staff activities for management and oversight 

of HIP included: establishing Memoranda of Understanding and contractual relationships with 

retailers and others (as described above), hiring additional staff, managing the HIP grant, internal 

operations, and providing required reports to USDA. DTA incurred most of the costs for these 

functions ($671,047) for personnel, supplies, and overhead; Xerox incurred the balance ($32,434).121 

Evaluation support ($273,095). These activities included providing data relevant to the HIP 

evaluation, including system data and HIP reports, and supporting the stakeholder interview process. 

Most of the evaluation support cost ($191,122) was incurred by Xerox and NDG. 

Unassigned costs. As shown in Exhibit 9.3, DTA internal costs of $34,028 were not assigned to one 

of the specific functions in the time-use and ODC data.

 

121 The DTA time for general administration appears to include time spent on activities that were difficult to 

classify. Thus, some of the cost for this function may in fact be attributable to other functions. This data 

limitation may partly explain the relatively small DTA cost for system design, development and testing. 
Evaluation of the Healthy In

pg. 176 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Exhibit 9.3: Summary of Implementation Cost of Healthy Incentives Pilot (Including DTA and FNS)

Function 

Cost category 

Salaries and 

wages ($) 

Employee 

benefits ($) 

Payroll tax 

($) 

Contractors 

($) 

Retailer 

infrastructure 

($) 

Supplies and 

other direct 

costs ($) 

System design, development and testing 4,512 1,339 77 1,016,156 1,410,498 Household recruiting and customer service 9,141 2,712 157 0 118,058 Retailer recruiting 118,589 35,182 2,031 6,890 18,030 Community relations 38,414 11,396 658 0 Training 131,457 39,000 2,251 16,586 64,538 General administration 448,341 133,010 7,677 32,434 6,608 Evaluation support 47,828 14,189 819 191,122 11,092 Unassigned costs 22,650 6,720 388 0 460 Total DTA costs 820,932 243,547 14,058 1,263,187 1,410,498 218,785 FNS expense for testing 69,862 Total implementation cost 820,932 243,547 14,058 1,333,049 1,410,498 218,785 

Notes: Salaries and wages, employee benefits, payroll tax, supplies and other direct costs, and indirect/overhead are for DTA personnel. See Exhcontractor costs. Retailer infrastructure includes all payments to retailers for HIP-related expense plus farmers market system development and d

and text for details. Retailers used HIP funds to pay IECR system vendors, TPPs, and other contractors, and for internal expenses. Detail of retai

expense for testing excludes FNS staff time and their travel, for which data were not available. Sources: DTA expenditure and time-use reports; contractor time-use reports; FNS correspondence. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 177

For DTA, the functions that used the most internal resources were (in decreasing order by size) 

general administration, training, and retailer recruiting. All personnel-related costs in Exhibit 9.3 

(including salaries and wages, fringe benefits, and payroll tax) and all indirect costs were internal 

DTA expenses. The majority of funds provided by DTA to the two main contractors, Xerox and 

NDG, were expended on system design, development and testing. All funds paid to retailers are 

categorized in Exhibit 9.3 as spending on retailer infrastructure; as discussed below, retailers used 

funds for a combination of internal activities and payments to other firms, including IECR system 

integrators and TPPs. 

DTA Contractor Costs 

As discussed in the preceding section, DTA’s primary contractors were Xerox and NDG. A third 

firm, Causemedia, was contracted to provide training materials. The breakdown of pilot contractor 

costs by firm and function is provided in Exhibit 9.4. (These costs do not include the contractor 

expenses for retailer enablement, which were incurred by contractors working for retailers and for the 

Massachusetts Department of Agricultural Resources.) 

Exhibit 9.4: Detail of Pilot Contractor Costs 

Function 

Causemedia 

($) 

 NDG 

($) 

 Xerox 

($) 

 Total 

($) 

System design, development and 

testing 0 171,912 844,244 1,016,156 

Retailer recruiting 0 0 6,890 6,890 

Training 14,035 0 2,551 16,586 

General administration 0 0 32,434 32,434 

Evaluation support 0 82,020 109,101 191,122 

Total 14,035 253,932 995,220 1,263,187

Notes: NDG provided technical support for design, development and testing. Xerox is the EBT processor. 

Sources: DTA expenditure reports; contractor time-use reports. 

In total, DTA expended $1,263,187 in contractor costs.122 The majority of contractor costs 

($1,016,156) were expended on design, development and testing. These costs included modification 

of the EBT processing system by Xerox and technical support services provided by NDG during the 

development and testing of IECR retailer and TPP systems. Another $191,122 was spent on 

evaluation-related activities. The remaining Xerox costs were for retailer recruiting, training and 

general administration. The Causemedia cost shown in Exhibit 9.4 was for consulting services to 

develop HIP training materials.123 (HIP card sleeves and promotional grocery bags purchased through 

Causemedia are included in the supplies and other direct costs in Exhibit 9.3.) 

Retailer Infrastructure Costs 

DTA paid for two kinds of retailer infrastructure costs with HIP grant funds. First, DTA paid 

$1,283,236 for IECR system modifications and other implementation activities by five retail chains 

 

122 Contractor costs are as of September 30, 2013, prior to close-out of the HIP grant. All contractor payments 

would be subject to review before they are final. 

123 FNS and the evaluation contractor developed the colorful three-step training brochure mailed to HIP 

participants in June 2012. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 178 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

and four independent retailers. Second, DTA paid $127,262 for development and implementation of 

payment systems to process HIP transactions at farmers markets. 

Five retail chains and four independent retailers agreed to participate in HIP on the condition that 

DTA would pay for their costs to ready their stores for HIP transactions. These retailers used 

proprietary or COTS IECRs for SNAP transactions, so their IECR systems were modified to identify 

HIP-eligible foods, calculate the HIP sub-total, process HIP transactions, and print receipts with HIP 

information. In addition, TPPs modified their systems to process HIP transactions as the TPPs routed 

data from participating stores to the EBT processor and back to the stores. HIP funds paid for these 

IECR and TPP modifications, and for related internal retailer costs such as identifying and managing 

the Universal Product Codes (UPCs) of HIP-eligible food items. 

Some but not all retailers provided information about how they allocated HIP funds (i.e., between 

internal costs for the retailer, payments to IECR contractors, and/or payments to a TPP). In addition, 

not all retailers provided information to separate HIP design, development, and testing from other 

retailer HIP implementation activities. Thus, all payments to retailers are reported in Exhibit 9.3 as 

“retailer infrastructure” expenses. 

The costs reported by retailers do not include any operational costs for HIP. Staff time might be 

required on an ongoing basis to address client concerns, fix system issues, and ensure that products 

are properly flagged in the IECR system. This is true for any program that restricts benefit use to 

specific products, including SNAP, WIC, and flexible payments for health benefits, and is particularly 

true during the start-up of a new program. However, retailers indicated in interviews that no new 

resources were required for HIP-related operations activities. 

Cost for Farmers Markets 

HIP was implemented through three different systems in the farmers markets in Hampden County: 

the token system, the e-HIP system, and the Mobile Market Plus (MM+) system. Each of these 

systems was designed to address the unique constraints of HIP transactions in farmers markets, 

including non-permanent locations, limited or no electric and telecommunications services, and 

limited operating funds. In addition to HIP transactions, the two electronic systems (e-HIP and MM+) 

facilitated regular SNAP transactions and use of other incentive programs for SNAP participants. 

Community Involved in Sustaining Agriculture Inc. (CISA) was awarded a contract to coordinate the 

implementation of test systems at pilot area farmers markets. The actual cost paid from the HIP grant 

for the farmers market systems was $127,262. Exhibit 9.5 provides the breakdown of these expenses. 

About 60 percent of the expenses ($78,127) were for design and development of the e-HIP and MM+ 

systems. The rest of the expenses were for the token system and project management and training for 

market managers and vendors (including salaries, supplies, and travel). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 179

Exhibit 9.5: Costs of Implementing HIP in Farmers Markets  

Farmers market 

solution 

Cost Categories 

Total ($) 

Salaries 

($) 

Supplies 

and 

travel ($) 

Token 

solutiona

($) e-HIPb ($) MM+c

($) 

Activity 

Design, develop, test and 

implement solutions 0 0 7,939 42,661 35,466 86,066 

Project management and 

training 17,089 24,108 0 0 0 41,196 

Total 17,089 24,108 7,939 42,661 35,466 127,262 

Note: Actual costs include only funds paid to CISA and were not itemized by cost categories. Percentages of 

budgeted costs for each category were used to allocate actual costs. a Modification to existing SNAP token system used in farmers markets. b Developed by Mass Farmers Markets. 

c Developed by NDG; this solution was originally developed for other farmers markets and modified for HIP. 

Source: DTA expenditure reports and budget documents. 

9.3 Costs of Nationwide Expansion 

The cost of a nationwide expansion includes the annual cost of incentive payments and the (nonrecurring) implementation costs. Nationwide implementation costs are estimated based on pilot 

implementation expenses, input from industry experts and a set of basic assumptions.124 One-time 

nationwide expansion costs include costs for the following activities: 

 Manage the implementation of HIP within a State 

 Modify and test EBT processor systems for HIP transactions 

 Modify and test IECRs for HIP transactions 

 Modify and test POS terminals for HIP transactions 

 Modify and test TPP systems for HIP transactions 

 Notify and train SNAP staff, retailers, and participants 

As discussed below, stakeholders indicated that the only identifiable ongoing cost of HIP will be the 

actual cost of the HIP incentive. Stakeholders expected that, in general, expansion would not increase 

staffing or other ongoing costs of operating and maintaining SNAP EBT systems for States, EBT 

processors, and retailers. This section first discusses the projected cost of incentive payments and then 

the non-recurring implementation costs. The section concludes by discussing the expansion and 

operational costs that are not feasible to estimate. 

Cost of the HIP Incentive 

Payment of HIP incentives to households would be ongoing in a nationwide expansion. HIP 

incentives would be by far the greatest cost of nationwide expansion to the Federal government. 

 

124 The nationwide implementation costs are provided as point estimates. There is substantial uncertainty about 

how closely the actual costs will track those of the pilot. Lacking any information about the scope of this 

uncertainty, we have not attempted to estimate a range of probable values around the point estimates. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 180 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

While the costs of HIP incentives can be projected from the experience of the pilot, there are three 

important sources of uncertainty. First, there is considerable uncertainty about how the incentive 

earnings would change with much wider or universal retailer participation, at least among stores with 

substantial inventories of fruits and vegetables. The second unknown is how much wider and deeper 

understanding of HIP among SNAP participants in the context of nationwide expansion would lead to 

higher levels of TFV purchases. Finally, the HIP participants may be systematically different from the 

average SNAP household nationwide in ways that might affect incentive earnings (such as household 

size, presence of children and elderly, and race/ethnicity). The projections in this section address the 

first two sources of uncertainty. We have not attempted to address the third because the evidence 

from the pilot suggests that differences in household characteristics between the pilot participants and 

the national population of SNAP participants would have only a modest influence on HIP incentive 

earnings relative to the potential effects of wider retailer participation and better participant 

understanding. 

Exhibit 9.6 presents the projected costs of HIP incentives under three scenarios. The projections 

assume a nationwide roll-out starting in 2014, with incentive earnings starting in 2015 and the first 

year of full nationwide operation in 2019. The scenarios and assumptions are described below. 

1. The Baseline projection assumes that the average HIP household earns $3.65 per month in 

incentives. This was the average from March through October 2012 (see Exhibit 5.12). 

2. The All retailers participating scenario assumes that all SNAP authorized retailers 

participate, and that HIP incentives earned increase in proportion to the share of SNAP 

spending in participating retailers. In the average month, HIP participants spent 52.9 percent 

of SNAP benefits in participating retailers (Exhibit 5.9, March through October 2012). Thus, 

the baseline projection is multiplied by 1.89 (calculated as 1/0.529) to yield the average 

incentive earned per HIP household in this scenario ($6.90 per month). 

3. The Thrifty Food Plan (TFP) target scenario is based on the analysis conducted to guide the 

setting of the HIP incentive cap, which projected a mean incentive earned of $19.78 per 

household. This analysis assumed participants would purchase 53 percent of the TFP 

recommendation for fruits and vegetables.125 

All scenarios are based on the same projections of the HIP caseload. The percentages of SNAP 

households participating in HIP each year are assumed, but approximately align with the assumptions 

about the number of States implementing HIP each year (as used in the implementation cost 

projections discussed later in this section).126 The SNAP household totals for these projections are 

based on projections of the number of SNAP participants by the Congressional Budget Office (CBO, 

 

125 This assumption was based on Blisard and Stewart’s (2006) estimate of average fruit and vegetable 

spending for low-income households as a fraction of TFP. 

126 The HIP percentage of SNAP households in Exhibit 9.6 approximately aligns with the assumptions of the 

implementation cost projections under the additional assumption that the average State implementing each 

year is of average size. Although the implementation cost projections assume that the first three States 

implement in 2014, the incentive projections below assume that participants in these States begin earning 

incentives in 2015, and other States also have a lag between the start of implementation and the start of 

incentive outlays, so the first year when all SNAP participants are able to earn incentives is 2019. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 181

2014) and an assumption of 2.09 persons per household, as observed in FY2012 (based on data 

reported on the FNS website). 

Exhibit 9.6: Projected Incentives Earned During Nationwide Expansion of HIP 

HIP incentive earned (millions of $) 

HIP share of 

SNAP (%) 

Total HIP 

households 

(millions) 

Baselinea 

($) 

All retailers 

participatingb 

($) 

TFP targetc

($) 

Monthly HIP incentive per 

household 3.65 6.90 19.78 

Year

2015 121 228 654 12.5 2.8 

2016 234 443 1,271 25.0 5.4 

2017 448 847 2,428 50.0 10.2 

2018 687 1,298 3,721 80.0 15.7 

2019 825 1,559 4,469 100.0 18.8 

Total 2,314 4,375 12,542 

a

Baseline: pooled mean incentive, Exhibit 5.12. 

b

All retailers participating: based on mean 52.9 percent of SNAP redemptions by HIP participants in participating 

stores (March-October 2012). 

c

TFP target: Average household spends 53 percent of the fruit/vegetable target in the TFP on TFV. 

Includes the 50 States, the District of Columbia, Guam, and the Virgin Islands. Areas operating alternatives to 

SNAP are not included. Projections are in 2012 dollars and are not adjusted for inflation or for food costs outside 

the 48 contiguous States and the District of Columbia. 

Sources: EBT transaction data; CBO SNAP caseload projections. 

As Exhibit 9.6 indicates, the annual incentive earnings in the first full year of nationwide HIP 

operation (2019) are projected to range between $825 million and $4.469 billion. The cumulative 

incentive earnings for 2015 through 2019 are projected to range between $2.314 billion and $12.542 

billion. These projections are intended to illustrate the potential scale of incentive earnings and also 

the wide band of potential earnings under plausible assumptions. While the annual totals may be seen 

as large, even the highest scenario represents a cost of just 31 cents per household per day. 

Implementation Costs Overview 

The total non-recurring cost for nationwide HIP expansion is estimated at $89.9 million. This 

estimate represents costs to expand HIP to include the rest of Massachusetts, the other 49 States, the 

District of Columbia, Guam, and the Virgin Islands. (These 53 agencies are all referred to as State 

Agencies below.) Estimated costs for nationwide expansion include those to be incurred for State 

Agency activities; EBT processor, retailer, and TPP system modifications; technical assistance 

contractors; participant and retailer training materials; and other supplies and direct costs. The 

estimates are presented in Exhibit 9.7, and the assumptions for these estimates are summarized below. 

The intent of the analysis is to estimate the full costs of nationwide HIP expansion without making 

any assumptions about which stakeholders would pay for these costs (in particular, the cost-sharing 

between State Agencies and FNS, and between public and private stakeholders). 

Compared with the pilot, there will be considerable economies of scale in nationwide expansion, 

because the costs are one-time and the largest ones are shared across States and retailers. The pilot 

cost was roughly $100 per participating household. The nationwide cost of $90 million represents a 

one-time expenditure of less than $5 per SNAP household – less than the cost of providing a $1 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 182 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

brochure to every SNAP household every year for 5 years. Moreover, over three-fifths of the cost is 

spending on retailer infrastructure that may require little if any funds from FNS or the States. 

The estimated expansion costs in Exhibit 9.7 are separated between State Agency costs and retailer 

infrastructure costs. State Agency costs for expansion include State personnel, contractual costs for 

EBT processors and consultants, and supplies and other direct costs. The rows for State Agency costs 

represent the following stages of HIP expansion: 

1. Statewide expansion within Massachusetts. This includes design, development, and testing 

of modifications to the Xerox EBT system to support nationwide expansion. 

2. The second State to implement HIP. This State will go through a similar design, 

development, and testing process with its EBT processor. (This State represents the first State 

to implement HIP with the other EBT processor, FIS.)127 

3. The third, fourth and fifth States to implement HIP. These States will need only minor 

changes to their EBT systems, since the two EBT processors will have already implemented 

HIP. However, these States’ costs will be greater than for the 48 State Agencies that follow 

due to the need to oversee certification of additional major retailers to participate, as 

discussed below. 

4. The remaining 48 State Agencies. 

The rows for retailer infrastructure represent the costs for the different categories of systems that 

would be modified to expand HIP, including: IECR systems (such as those used by the vast majority 

of supermarkets and superstores, and other types of stores operated by retail chains), commercial POS 

terminals (accepting credit and debit cards as well as EBT), and TPPs. These rows represent projected 

costs for nationwide expansion (all 53 State Agencies), including all systems that must be modified to 

support HIP in all SNAP retailers. The retailer infrastructure costs are identified separately because of 

their size (62 percent of the total for nationwide expansion) and because no assumption is made about 

the portion of these costs to be paid by State Agencies or FNS. The expansion costs in Exhibit 9.7 

include FNS costs for contractor support to oversee testing, but we have not attempted to estimate 

costs for FNS personnel or other FNS administrative costs to oversee expansion. 

 

127 The projections assume that all States implement HIP using either Xerox or FIS. FIS is the current name of 

the firm formerly known as Fidelity Information Systems. JP Morgan Chase has announced that it plans to 

cease providing EBT processing services, so the projections assume that this firm does not modify its 

systems for HIP. Pilot data do not support estimates for design, development, and testing of modifications 

for other systems, such as those used by Montana and Texas, or for extra State Agency costs in areas with 

special implementation requirements and challenges (such as Alaska, Hawaii, Guam, and the U.S. Virgin 

Islands). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility oExhibit 9.7: Estimated Implementation Cost of Statewide and Nationwide Expansion 

Cost category 

Salaries and 

wagesa ($) 

Employee 

benefitsa ($) 

Payroll 

taxa ($) 

Contractorsb 

($) 

Supplies and 

other direct 

costsc ($) 

Indirect/ 

overheada

($) 

State Agency costs MA: Remainder of State 725,549 215,250 12,424 273,550 89,636 34,407 State 2 (with design, development, and testing of HIP modifications for second EBT processor and additional IECR systems) 

725,549 215,250 12,424 1,101,278 89,636 34,407 

State 3 (with design, development, and testing of HIP modifications for additional IECR systems) 

362,775 107,625 6,212 273,550 89,636 17,203 

State 4 (with design, development, and testing of HIP modifications for additional IECR systems) 

362,775 107,625 6,212 273,550 89,636 17,203 

State 5 (with design, development, and testing of HIP modifications for additional IECR systems) 

362,775 107,625 6,212 273,550 89,636 17,203 

States 6-53 (remaining 48 State Agencies, with minor modifications to EBT system or reports) 

13,059,886 2,583,003 149,090 4,878,626 4,302,525 412,879 

Total State Agency costs 15,599,308 3,336,378 192,575 7,074,102 4,750,704 533,302 Retailer infrastructure Modification of IECR systems Modification of commercial POS terminals Modification of TPP systems Total retailer infrastructure FNS testing 2,700,000 National totals 15,599,308 3,336,378 192,575 9,774,102 4,750,704 533,302 

Includes 50 states and DC, Virgin Islands and Guam. See text for assumptions. Detail may not sum to totals due to rounding.a For salaries and wages, employee benefits, payroll tax, indirect/overhead, activities include: develop, design, test; retailer recruiting; training; geb Contractor costs include: EBT processor (system modifications, client training brochures, and production and mailing of retailer training brochure

coordinate and support retailer infrastructure modifications and testing. 

c Supplies and ODCs include: retailer recruiting (travel), training (translation, travel, misc.), G&A (misc.), unassigned (misc.). d Retailer infrastructure includes: modification and testing of proprietary and commercial off-the-shelf (COTS) IECR systems, commercial stand-be

platforms, Does not include costs of improvements to all variations of farmers markets technologies (other than EBT-only and commercial stand-b

Sources: DTA time and expenditure reports; contractor time reports; stakeholder interviews. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 184 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

State Agency Expansion Costs 

Projected State Agency costs for HIP expansion include State agency personnel, contractors, and 

supplies and other direct costs, as described in this section. 

State Agency Personnel 

All expansion estimates for State personnel costs (including salaries and wages, employee benefits, 

payroll taxes, and indirect costs) are based on the DTA personnel costs for the pilot. We excluded 

several categories of pilot costs based on the assumption that they were specific to the HIP pilot and 

would likely not occur during a statewide rollout: household recruiting, community relations and 

evaluation. 

Retailer recruiting costs from the pilot are included in the base used for estimating each stage of 

nationwide expansion costs. Clearly, a mandate that all SNAP retailers participate would obviate the 

need to recruit retailers. However, even in that case, the WIC-EBT experience suggests that there 

would likely be a significant outreach effort to educate retailers and support them through system 

changes. 

For training, the assumed approach is that each State will design and distribute informational 

brochures for participants and retailers. Participant training brochures will be distributed through local 

SNAP offices or mailings for other purposes, so the projections do not include any distribution costs. 

Production of brochures and mailings for retailers are included in the contractor costs for 

expansion.128 Existing communication and training channels will be used to prepare State and local 

staff for HIP. Although training was far more intensive in the pilot than expected for nationwide 

expansion, the State Agency labor cost for the pilot (spread over an entire State) is the best available 

proxy for these costs. 

For statewide expansion in Massachusetts, the estimates are based on the assumption that DTA will 

need to replicate its labor hours from the pilot. Even though system changes have been made, there 

will still be a significant level of effort to oversee the expansion of the retailer infrastructure, train 

local office staff, and ensure the EBT system meets the guidelines, standards and operating rules that 

will likely be established by FNS for HIP expansion. 

As noted above, State 2 represents the first State implementing HIP with the other major EBT 

processor, FIS. This State is assumed to incur costs similar to the pilot costs in Massachusetts (after 

excluding costs unique to the pilot). These include costs of system design, development, and testing; 

retailer outreach (similar to the process for expansion in Massachusetts); training (discussed below), 

and general administration. The focus of the retailer outreach effort would be the oversight of retailer 

infrastructure investment for systems that did not implement HIP as part of the Massachusetts pilot 

and its expansion. (Costs of retailer infrastructure are addressed in the next section.) 

For States 3, 4 and 5, the estimates are based on the assumption that the EBT system will be ready for 

HIP but will need minor modifications for each State (such as new reports). However, it is assumed 

that the States will have a comparable effort to the Massachusetts expansion in preparing retailers, 

participants, and agency staff for HIP. In particular, these States will need to oversee the retailer 

 

128 The costs include a brochure for every SNAP retailer, but chain retailers might prefer to distribute their 

own training materials. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 185

infrastructure investment for systems that have not already implemented HIP. Therefore it is assumed 

that State Agency personnel costs for States 3, 4 and 5 will be one-half of the cost per State as for 

State 2.129

Based on the experience with WIC EBT, it is assumed that, after five states have implemented HIP, 

the majority of retailers with proprietary IECRs and the majority of commercial off-the-shelf 

(COTS)130 IECRs will have been enabled for HIP, thereby reducing the State effort related to retailer 

infrastructure investment. Therefore for the remaining 48 states (States 6-53) it is assumed that the 

State Agency personnel costs will be three-quarters of the cost per State for States 3, 4 and 5. 

Contractors 

The assumptions for contractor costs include EBT processors, technical consultants, and other 

contractors. The roles of these contractors and the assumptions for their costs are explained below. 

For EBT processors, the assumptions are as follows: 

 For Massachusetts expansion, the EBT processor (Xerox) will make minor modifications to 

system features and reports to meet the needs for non-pilot, statewide operations and national 

expansion. The modified system will be tested as it was in the pilot. 

 One more EBT processor (FIS) will need to be enabled for HIP, as part of implementation in 

State 2. The projections for EBT processor costs are based on the Xerox costs for HIP, after 

excluding evaluation-related costs. These costs include modifications to the EBT-only 

terminals currently supported by this processor.131

 For the remaining 51 States and territories, the EBT processors will make minor state-specific 

system modifications and reports at the same per-State cost as the Massachusetts expansion. 

The first five States implementing HIP, including the Massachusetts expansion, will require the 

technical support services of a contractor specializing in retailer systems, with capabilities similar to 

NDG. As in the pilot, a contractor of this type would support and coordinate the design, development, 

and testing of modifications to IECR and TPP systems. The retailer systems specialist would also 

oversee the design, development, and testing of modifications to commercial stand-beside POS 

 

129 As noted in the discussion of pilot costs, the DTA time-use data include a large amount of effort for general 

administration, part of which may be related to system design, development, and testing and therefore not 

needed in States 3, 4 and 5. Lacking more specific data to estimate the portion of DTA’s pilot costs for 

State personnel that States 3, 4 and 5 would face, the assumption that their costs would be half of DTA’s 

appears to be the most reasonable and defensible approach for this analysis. The same rationale applies to 

the projection of personnel costs for the remaining 51 State Agencies. All State Agencies would of course 

need to develop a detailed budget based on their specific requirements, using the Massachusetts data as 

appropriate. 

130 A COTS IECR system is a package of hardware and software marketed as a complete, standardized POS 

solution, rather than a custom system developed for or by the retailer (i.e., a proprietary system). 

131 The Agricultural Act of 2014 requires retailers to pay for EBT equipment and supplies, unless they are 

exempted by USDA. If exempt retailers that carry HIP-eligible foods, such as cooperatives and military 

commissaries, continue to have no-cost EBT-only terminals, then FIS will need to modify its software for 

these terminals. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 186 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

terminals used for accepting credit, debit, and EBT in stores that do not have IECRs. (As discussed 

above, the EBT processor would modify EBT-only terminals.) Using WIC EBT as an example, after 

the first five States have implemented HIP, the need for these services will be eliminated as the great 

majority of IECR and POS systems will have been enabled for HIP. 

Some states, such as California and New York, have multiple eligibility systems, but they contract 

with one EBT provider and all eligibility systems share a common EBT platform. County or regional 

participant data are provided to the EBT system in the same file formats. Thus, the HIP 

implementation cost associated with eligibility system changes for HIP in States such as California 

and New York will be similar to that of other States that maintain one eligibility system. 

For each of the 53 states, the projections assume that the State will use a contractor to produce 

participant brochures and produce and mail retailer training brochures during the initial rollout of 

HIP. As noted above, it is assumed that States will use mailings for other purposes, hand-outs during 

in-person contacts, and existing websites to inform participants about HIP, so the States will not have 

separate costs for mailing participant brochures or other materials. 

Supplies and Other Direct Costs 

State expenses for supplies and other direct costs are based on the same categories of pilot costs as the 

personnel costs and include retailer recruiting, training, general and administration costs, and 

unassigned costs. The DTA pilot costs that are used as the basis for projected costs in this category 

include in-state travel, translation services, advertising and job related expenses. The per-state cost for 

Massachusetts expansion and each subsequent State is assumed to be the same as the applicable costs 

in the pilot. Development and production of training materials is included in the contractor cost 

column in Exhibit 9.7. 

Retailer Infrastructure Expansion Costs 

Exhibit 9.7 identifies three categories of retailer infrastructure costs for HIP expansion, as follows: 

1. IECR systems, with a total cost of $53.2 million 

2. Commercial POS terminals, with a total cost of $1.4 million 

3. TPP systems, with a total cost of $1.0 million 

These cost estimates are discussed below. As previously indicated, the estimates are intended to 

represent the cost of the resources required to implement HIP nationwide, without assuming how the 

retailer IECR costs for expansion are funded. These costs could be reimbursed directly by States, paid 

by or through EBT processors or another third party, or borne by the retailers. The cost estimates are 

based on the assumption that if HIP is implemented, all systems used by SNAP retailers will be 

modified to support HIP. (The question of which SNAP retailers would participate in the nationwide 

expansion of HIP is discussed in Section 9.4.) 

IECR Systems 

As in the pilot, two types of IECR systems will need to be modified and tested in order to support 

HIP expansion. A proprietary IECR is a system that has been developed specifically for one retail 

chain. A retail chain with a proprietary IECR system will likely modify its software and then make 

the enhanced software available to all of its retail locations in all States. One chain in the pilot has 

already enabled HIP functions in the proprietary system used in all of its stores in the U.S. Other 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 187

retail chains use commercial off-the-shelf (COTS) IECR systems; for these systems, once the system 

is modified for the first State where the system is used, enhanced software can be made available to 

all stores that use that COTS system throughout the U.S. 

As shown in Exhibit 9.8, the expansion cost estimates include a total of 224 proprietary IECR 

systems and 60 COTS IECR systems. Counts of retail chains by store type are based on FNS national 

retailer data. The estimated number of proprietary systems to be modified includes all 86 

superstore/supermarket chains, 2 grocery store chains, 49 combination grocery/other store chains, and 

87 convenience store chains. All other retailers with IECR systems are assumed to have one of the 60 

COTS IECR systems identified by industry experts consulted for this report. The total cost of 

modifying and testing IECR systems is based on the counts of systems and the average cost per 

system in the pilot. The average cost per store for IECR system modifications and testing is estimated 

to be $386, with a range across store types from $181 per store for combination grocery/other stores 

to $1,855 per store for grocery stores. The variation is due to the mix of proprietary vs. COTS 

systems and the number of stores per chain. 

Commercial POS Terminals 

The second category of retailer infrastructure costs for HIP expansion is commercial POS terminals 

that are not integrated with cash registers (known as “stand-beside” terminals). As noted above, some 

retail chains and individual retailers use these terminals to accept credit, debit, and EBT transactions, 

which are processed through a TPP. Retailers with commercial stand-beside terminals were not 

included in the pilot test due to the time constraints of the pilot. 

Based on the evaluation team’s experience with WIC EBT implementation, it is assumed that 40 

models of commercial stand-beside terminals will need to be enabled for HIP. The estimated average 

cost of $34,000 for each model of terminal is based on consultation with industry sources and 

includes testing with the retailer’s TPP. (In some cases the terminal is provided to the retailer as part 

of the TPP agreement.) Once a particular model of commercial POS terminal is enabled for HIP, 

existing processes can be used to update the software for all users at no additional cost. 

TPP Systems 

TPPs are needed to route HIP transactions from retailers using IECR or commercial POS systems to 

EBT processors. Based on research into the TPPs active in EBT, an estimated 19 additional TPP 

systems will need to be modified to accept the HIP transaction for nationwide expansion. This 

estimate takes into account the number of TPPs and the number of different systems they operate. The 

estimated cost per TPP system is $53,075 based on pilot data. Whether TPPs charge their customers 

for HIP modifications would be a business decision for the TPPs. 

Other Retailers 

The total cost to enable retailers for a nationwide rollout does not include the calculated cost of 

modifying systems for specialty stores (e.g., bakers, meat markets, and fish markets). However, it is 

likely that most or all of these stores will be able to leverage the upgrades done to COTS systems, 

commercial POS terminals, and EBT-only terminals to include sales of HIP-eligible items. 
Evaluation of the Healthy In

pg. 188 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Exhibit 9.8: Estimated Cost of Integrated Electronic Cash Register System Modifications for Nationwide Expansion 

Retailers with IECR systems 

Units Cost calculation 

Estimated number of 

proprietary systems 

Estimated number of 

commercial-off-theshelf systems Estimated cost per system ($) 

Estimated tot

IECR sy

modificat

Supermarkets/superstore 86 0 187,400 16,11Grocers 2 20 187,400 4,12Combination grocery/other 49 20 187,400 12,93Convenience stores 87 20 187,400 20,05Nationwide total 224 60 187,400 53,22

Sources: DTA expenditure reports; FNS national retailer data. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 189

FNS Oversight of Acceptance Testing 

FNS normally uses a contractor with specialized expertise to oversee a user acceptance test (UAT) 

when a State implements a new EBT system. FNS indicated that a UAT will be required when each 

of the two EBT processors (FIS and Xerox) first implements HIP for nationwide roll-out, and this 

initial UAT will be more extensive and costly than in the pilot. Subsequent to the initial UATs, FNS 

will require UATs for all other States as they implement HIP. However, FNS will only incur HIPspecific contractor expenses for UATs in those States that implement HIP outside of their normal 

cycle of EBT contracts. States must conduct UATs each time they transition to a new EBT contractor. 

FNS estimated that 33 states will require a HIP-specific UAT; the first three UATs will cost 

approximately $150,000 each while the remaining UATs will cost approximately $75,000 each.132

Five-Year Expansion 

If HIP is implemented nationally, the implementation will occur over a number of years; State 

Agency, EBT processor, TPP, and retailer systems would need to be modified. This section provides 

estimated costs based on a five-year scenario to implement HIP and the cost assumptions in the 

preceding section. While the scenario assumes that the five-year implementation period begins in 

2014, in practice HIP regulations and HIP standard operating rules for HIP will need to be 

established, and the time to complete this process is uncertain (as discussed in the next section). Costs 

of nationwide expansion are presented in Exhibit 9.9. The exhibit spreads the total implementation 

costs in Exhibit 9.7 over five years, under the assumptions described below. Costs for each year 

include the State Agency, State contractors (including EBT processor), and the FNS contractor for a 

specified group of States, together with a share of the TPP, IECR, and stand-alone POS terminal 

modification costs based on the counts of these systems for each year shown in Exhibit 9.9. 

Year One 

Projected costs for the first year (2014) include State Agency, contractor, retailer infrastructure, and 

FNS contractor costs for Massachusetts and two other States (States 2 and 3 in Exhibit 9.7) to 

implement HIP, with State 2 being the first HIP State using FIS as its EBT processor and State 3 

being the second HIP State using Xerox. For State 2, the costs include modifications to the State 

Agency eligibility system and design, development, and testing of modifications to the FIS EBT 

system for HIP. As discussed in the detailed assumptions for State Agency expansion costs, the State 

Agency and EBT processor effort for State 3 would be less than for Massachusetts. All three States 

will require the services of a retailer systems specialist contractor and an extensive UAT, supported 

by an FNS contractor. As these are the initial States, costs for retailer infrastructure will be 

considerable: 15 TPPs, 60 IECRs, and 11 stand-beside POS terminals will be modified for HIP. 

Year Two 

Projected costs for the second year cover the implementation of HIP in eight States: States 4 and 5, 

and six of the 48 remaining States (in the framework of Exhibit 9.7). States 4 and 5 will require the 

services of a retailer systems specialist contractor; the others will not. Five of the eight States will 

implement and test HIP as part of their normal transition to a new EBT processor; FNS will incur 

 

132 We assume that State Agency activities for HIP implementation will require the same level of effort 

whether the system changes are part of an EBT processor transition or a separate process, so this distinction 

does not affect the State cost projections. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 190 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

contractor costs to oversee HIP-specific testing in the other three States. Four TPPs, 80 IECRs, and 11 

stand-beside terminals will be modified for HIP. 

Year Three 

In the third year, five additional States will implement HIP as part of their normal transition to a new 

EBT processor, and seven additional States will implement HIP as a stand-alone project. The retailer 

systems specialist contractor will not be needed as most retailers with IECRs and all TPPs will have 

enhanced their systems in the first and second years. FNS will incur HIP-specific UAT costs for the 

seven States implementing HIP as a stand-alone project. In the third year, 59 IECRs and eight standbeside terminals will require upgrades. (TPP implementation will have been completed in the second 

year, so no TPP costs appear in later years). 

Year Four 

In the fourth year, five additional States will implement HIP as part of their normal transition to a new 

EBT processor and 10 additional States will implement HIP as a stand-alone project. FNS will incur 

HIP-specific UAT costs for the 10 States implementing HIP as a stand-alone project. In the fourth 

year, 48 IECRs and seven stand-beside terminals will require upgrades. 

Year Five 

HIP implementation will be completed in the fifth year, when 5 additional States will implement HIP 

as part of their normal transition to a new EBT processor and 10 additional States will implement HIP 

as a stand-alone project. FNS will incur contractor costs for HIP-specific UATs for the 10 States 

implementing HIP as a stand-alone project. In the fifth year, 37 IECRs and three stand-beside 

terminals will require upgrades. 

Assumed Quantities per Year 

The top panel of Exhibit 9.9 summarizes the assumptions described above for each of the five years 

of the implementation period. Note that the number of States implementing increases during the 

second, third, and fourth years while the number of retailer system changes decreases. This aligns 

with the assumption that work performed on an EBT, TPP or IECR system or modifications to a POS 

terminal needs only be done once, and then the upgraded system or terminal is available to users in all 

States. As discussed previously, the major changes to EBT systems occur in Year 1 of HIP expansion, 

and the EBT modifications for other States would become smaller as expansion proceeds. 

Projected Cost by Year 

The bottom panel of Exhibit 9.9 shows that the projected cost of HIP expansion in each of the five 

years will be between $15.7 and $20.7 million (in 2013 dollars), with the cost peaking in Year 2 and 

declining each year thereafter. The annual costs could differ from projections if any of the 

assumptions does not hold, so the projections provide at best an approximate guide for the spread of 

costs over time. As previously noted, the projections make no assumptions as to which entity will 

bear the cost of TPP, IECR, and POS changes. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 9. Costs of Pilot and Feasibility of NationExhibit 9.9: Assumptions and Projections of Annual Costs for HIP Expansion 

Year 1 Year 2 Year 3 Year 4 Year 

Assumptions Number of States implementing HIP as part of their SNAP transition (no incremental FNS costs for UAT) 

. 5 5 5 

Number of States implementing HIP independently (FNS incurs contractor costs to oversee UAT) 

3 3 7 10 

Total number of States implementing HIP 3 8 12 15 HIP-enablement of EBT processor systems 3 IECR system changes 60 80 59 48 TPP system changes 15 4 Stand-beside terminal changes 11 11 8 7 States needing retailer systems specialist contractor 3 2 Cost projections (in 2013 dollars) State Agencya ($) 2,737,983 3,730,325 5,126,846 6,408,557 6,408,5State contractorsb ($) 1,648,377 1,156,927 1,219,657 1,524,571 1,524,5Third-party processors ($) 796,125 212,300 0 0 Proprietary and COTS IECRs ($) 11,244,000 14,992,000 11,056,600 8,995,200 6,933,8Stand-beside POS terminals ($) 374,000 374,000 272,000 238,000 102,0FNS testing contractorc ($) 450,000 225,000 525,000 750,000 750,0 Total ($) 17,250,485 20,690,552 18,200,102 17,916,328 15,718,9

a Includes salaries and wages, fringe benefits, payroll taxes, overhead, materials, and services. b Includes EBT processor and retailer systems specialist contractor fees. c Cost of contractor overseeing UAT on behalf of FNS. 

Sources: DTA time and expenditure reports; contractor time reports; stakeholder interviews.
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 192 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

Undetermined Costs 

A number of activities that are not included in the preceding cost estimates would have to occur prior 

to and during a nationwide HIP implementation. It is not feasible to estimate the costs of these 

activities, but they are described in this section. 

HIP Regulations 

If HIP is implemented, USDA FNS will need to establish new regulations that are specific to HIP. 

The regulations would establish the percentage of the cost of TFV that would be applied to calculate 

the incentive and the maximum dollar amount that could be earned within a given month. 

Standard Operating Rules 

Prior to requesting additional processors, retailers, and TPPs to change systems for HIP, USDA FNS 

will need to standardize the message formats and operating rules. Often rules and standards are 

established by committees composed of stakeholders that include State Agencies and system and 

industry experts. 

HIP Marketing and Promotion 

If HIP is implemented on a nationwide basis, costs for marketing and promotion to participants, 

retailers, and other stakeholders would be expected. The amount of these costs would depend on the 

types and intensity of marketing and promotion activities. A further unknown is the roles of FNS, 

States, retailers, and other stakeholders, as well as the feasibility of using free media (such as public 

service announcements). Therefore, it is not feasible to estimate the costs for these activities. 

Monitoring Retailer and Participant Fraud and Abuse 

Any new type of transaction will provide new avenues for fraud and abuse. FNS will need to create 

new ways to monitor data for potential evidence of fraud and abuse related to HIP, and FNS 

compliance units will need to incorporate HIP into their compliance activities. The costs for these 

efforts are not estimated for this assessment. 

Establishing the UPC Database 

The identification of the UPCs and product look-up codes (PLUs) for eligible fruits and vegetables is 

required for any corporation, chain or store that operates with an IECR. FNS currently relies on 

retailers to flag SNAP-eligible items in their systems and monitors errors, fraud and abuse through 

compliance activities; it is envisioned that this will be the same process followed for HIP. 

Nevertheless, there will be a considerable effort made for the initial identification of UPC and PLUs 

that are eligible for incentives. Retailers did not provide data on these or other “soft” costs of 

implementing the HIP pilot, so we did not attempt to estimate these costs for expansion. Retailers that 

were interviewed for the evaluation highly recommended that a national standard be set for eligible 

food items to lessen the burden of UPC and PLU identification. (Items or UPCs eligible for the WIC 

benefit purchase are identified at the State level, and each retailer therefore has a different list for 

each State.) 

Farmers Markets 

As part of HIP implementation in Hampden County, DTA made special efforts to include farmers 

markets (FM). Community Involved in Sustaining Agriculture Inc. (CISA) received funding for this 

effort, which included the programming, testing, and implementation of two electronic systems that 

facilitated regular SNAP transactions as well as HIP transactions. Thus, the funding for CISA 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 193

combined the costs of HIP FM solutions with the costs of SNAP FM solutions, and the two cannot be 

separated. DTA provided management and oversight to the FM project but again, these labor hours 

were not reported separately and cannot be separated from overall DTA costs assigned to 

management and oversight. 

In order to implement an electronic solution for HIP, a farmers market would need electrical and 

communications connectivity. We do not know what proportion of farmers markets nationwide have 

the infrastructure capability to support an electronic solution nor can we determine any basis with 

which to make an estimate of the cost of establishing connectivity in farmers markets where it is not 

already available. In addition, rapid changes in the availability and costs of such capabilities would 

likely alter any estimates we might make. 

It is anticipated that if HIP were to be implemented, then a farmers market wanting to accept SNAP 

electronically would acquire one or more HIP-capable POS terminals. This would be the case 

regardless of whether the market operated seasonally or year round. There would be no additional 

costs for HIP for FMs that began to accept SNAP after they installed HIP-capable POS terminals, or 

for FMs using EBT-only terminals for which the EBT processor would provide the upgrade. Thus, 

only FMs using multi-purpose POS systems that are not already HIP-capable might incur an 

incremental cost for a system upgrade to support HIP. Therefore, while POS conversion costs for 

FMs cannot be estimated, it appears that only a minority of FMs might face such costs.133

Ongoing Costs 

Stakeholders indicated that once HIP is operational there would be no additional fees from EBT 

processors or TPPs for processing HIP transactions. Ongoing costs for EBT processors would be 

covered by cost per case month (CPCM) fees paid by States. These fees cover participant and EBTonly retailer support (including training for new retailers) as well as transaction processing and 

settlement. Industry sources indicated that EBT processors are not likely to raise CPCM fees just to 

cover HIP-related costs. Further, it is expected that retailers’ existing fee arrangements with IECR 

system vendors and TPPs will cover ongoing costs for support of HIP functions. Retailers might incur 

ongoing costs to maintain HIP information in their UPC and PLU databases, but these costs cannot be 

estimated with the available data. It is assumed that when a new item is sold by a store, a staff 

member must enter the UPC, product information and price into the IECR. and flag the item as a 

SNAP-eligible item, so identifying HIP items will be integrated into this process. 

USDA FNS will need to establish a policy concerning HIP capability for retailers requesting SNAP 

authorization, including whether retailers are required to have HIP-capable systems and how they 

must demonstrate compliance with this requirement. Under FNS policy, the EBT processors will be 

responsible for assuring that the IECRs or POS terminals used by retailers have been certified for 

HIP, but FNS may need to confirm that new retailers use approved systems, or at least answer 

questions about HIP (including where to go to find out about certified IECR/POS systems) from 

retailers applying to participate in SNAP. 

Costs to upgrade EBT processors and IECRs to support HIP functionality are included in our onetime implementation cost projections. While EBT processors and IECRs may make future updates to 

 

133 For a more complete discussion, refer to Lessons Learned from Implementing HIP at Farmers’ Markets, 

Community Involved in Sustaining Agriculture, 2013. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 194 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

support changes to HIP, the scope of those modifications is impossible to predict. Costs of such 

updates would be difficult to estimate as the hourly rates, fringe benefits, and overhead of the 

information technology organizations modifying retailer systems are generally considered proprietary 

and are thus unknown. 

In summary, there will be ongoing activities directly related to HIP, but the majority of these 

activities will not impact fees or staffing levels. The costs of activities that would represent ongoing 

expenses cannot be estimated with reasonable accuracy but are expected to be minor relative to the 

costs of expansion and HIP incentives. 

9.4 Issues to Consider for HIP Expansion 

With one important exception, all stakeholders indicated that expansion is technically and 

operationally feasible. Both DTA and Xerox indicated strong support for expansion, as did all of the 

participating supermarket and superstore firms that completed interviews or surveys for the 

evaluation. However, the convenience store chain indicated that HIP did not have benefits for them 

because their stores do not stock targeted fruits and vegetables (TFV). DTA was unable to convince 

one major supermarket chain to participate, despite substantial effort. While the deadlines for the pilot 

appeared to be the primary obstacle, the feasibility of including this chain in statewide 

implementation might also depend on terms of participation yet to be negotiated. The results of the 

retailer survey indicate that among retailers who participated in HIP, all would participate in HIP if 

they had to make the decision again, with the exception of some convenience stores. Results from the 

EBT analysis suggest that HIP does not increase sales for stores other than supermarkets and 

superstores, although in wider implementation HIP might increase sales for some other stores with 

particular niches, such as medium or large grocery stores in areas without good access to 

supermarkets and superstores, or fruit and vegetable specialty stores. 

Stakeholders identified several areas in which there are challenges for HIP expansion and 

opportunities to facilitate success. These include the following: 

 Legislation, regulations, and industry standards 

 System development and testing 

 Retailer participation and readiness 

 Participant notification, outreach, and support 

 Operations and compliance monitoring 

The balance of this section discusses these challenges and opportunities. 

Legislation, Regulations, and Industry Standards 

Expansion of HIP would require a framework including legislation, regulations, and industry 

standards. Based on experiences with the establishment of SNAP and WIC EBT, stakeholders 

commented that establishing this framework would take several years, given the necessary 

consultation processes. Technical experts from FNS and industry emphasized that up-front effort to 

establish standards would be needed to ensure that all stakeholders understand the technical and 

functional requirements, and to provide coordination among the numerous systems involved in HIP 

(retailer, EBT processor, and TPP). This consultation process would also help synchronize 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 195

development implementation schedules. An added dimension of the framework would be the 

agreements between States and EBT processors on State-specific HIP reporting requirements. 

Perhaps the greatest issue for HIP expansion is funding. As indicated in the previous section, by far 

the largest cost of expansion would be the incentives themselves, which could amount to $1 billion or 

more per year. However, expansion would also require substantial one-time expenditures by States, 

EBT processors, retailers and TPPs totaling approximately $90 million. Thus, a key question is 

whether and how these stakeholders would be compensated for making changes for HIP. 

Besides questions of funding, key elements of the framework for HIP expansion would include the 

following: 

 Percentage of TFV purchases to be credited in incentives, and any cap on incentives 

 Definition of what foods qualify as TFVs 

 Whether incentives will be credits against regular SNAP benefits or funded, tracked, and 

settled separately 

 Whether HIP participation would be mandatory or optional for retailers 

 Rules for transaction types and messages, and for problem resolution (including crediting 

participants when HIP purchases are not identified by IECRs) 

 Requirements for certification of EBT processors, TPPs, IECR systems, and commercial POS 

terminals 

System Development and Testing 

Regarding system development, the three key considerations for expansion emerging from 

stakeholder interviews were: (1) the need for agreement on standards up front; (2) allowing adequate 

time for design, development and testing; and (3) the substantial resources required (as discussed in 

Section 9.3). The EBT processor also noted that the pilot requirements (particularly the need to 

identify HIP participants and the separate funding stream) added complications to the design and 

development, and recommended taking a simpler approach for expansion. 

An important issue for TPPs is the number of systems that would have to be modified. TPPs would be 

required to certify each of their retailers and may need to certify to one or both EBT processors. 

Those TPPs that provide terminal software would need to modify the software and deploy it to their 

retailers. One TPP indicated they support 40 terminal types. TPPs may also operate multiple 

transaction processing platforms, each requiring modification. TPPs expected that HIP expansion 

would not bring them additional revenue. 

SNAP agencies typically have not been involved with changes to retailer systems, whereas WIC 

agencies have had this role with IECRs and TPPs. The initial states that implemented WIC EBT hired 

retailer system specialists to assist retailers and IECR system vendors in understanding the changes 

that had to be made to enable their systems for WIC EBT. This same type of activity was present in 

HIP. 

All systems that are involved in HIP expansion will need to be tested to assure that they comply with 

standards and accurately process transactions. A key challenge for this process is limiting the role and 

burden for FNS, States, EBT processors, and TPPs. Stakeholders recommended an approach that 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 196 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

would focus on one-time testing of key systems including TPPs, proprietary COTS IECR systems, 

and commercial POS terminals. Once each of these systems was certified, it would not be necessary 

to certify each retailer using the system. Even under this scenario, FNS and State resources for 

participating in tests would be critical to timely and successful implementation. 

Retailer Participation and Readiness 

The most fundamental question regarding retailer participation in HIP expansion is whether it will be 

completely voluntary, mandatory for some retailers, or mandatory for all retailers. Voluntary retailer 

participation would mean that participants’ access to HIP incentives would depend on four factors: 

(1) retailers’ cost-benefit calculations on whether to participate, (2) persuasion from FNS and other 

stakeholders, (3) any cost-sharing arrangements, and (4) individual retail firms’ schedules for 

upgrading their systems to support HIP. Participants would need to know exactly which retailers were 

participating in HIP and would need to have access to those retailers in order to receive HIP 

incentives. 

On the other hand, mandatory participation for all SNAP retailers would affect a large number of 

stores—both those that carry an extensive inventory of fruits and vegetables (primarily supermarkets 

and superstores) and others (such as specialty and convenience stores) that carry more limited eligible 

fruits and vegetables.134 For supermarkets and superstores, the impact would be the one-time cost of 

modifying IECRs, which might be an issue for chains that have relatively few SNAP redemptions. 

The impact on smaller retailers, who generally do not use IECRs, would be the extra checkout time to 

calculate the amount of the HIP-eligible purchase separately from the remainder of the SNAP 

transaction. Other impacts would be time for staff training and possible sanctions for non-compliance 

with program rules (discussed below). Participating retailers did not generally identify these factors as 

major concerns, and there was no evidence that any retailer dropped out of HIP because of burden 

issues. On the other hand, DTA personnel indicated that, in their experience, the burden of 

participating in HIP was a barrier to voluntary participation for some non-IECR SNAP retailers, and 

thus would be a concern if HIP were mandatory. With mandatory participation, issues of enforcing 

compliance with HIP rules would likely increase. A particular concern is the accuracy of retailers in 

identifying HIP items and computing the HIP amount. 

An intermediate solution would be to make HIP mandatory for some but not all retailers, such as 

superstores, supermarkets and grocery stores, the locations that currently account for the highest 

number of SNAP transactions and also accounted for the vast majority of HIP transactions in the 

pilot. For this solution, FNS and others would have to address the equity of imposing system 

modifications on some but not all of the SNAP retailers. However, the additional $1 billion or more 

in SNAP revenue could present an important motivator for major retailers. 

Several secondary issues regarding retailer participation were identified by stakeholders, including 

the following: 

 

134 SNAP rules do not require retailers to carry any of HIP-eligible fruits and vegetables. The Agricultural Act 

of 2014 (Section 4002) establishes stricter stocking requirements: at last seven varieties of staple foods in 

each of the four categories and perishable foods in at least three categories. These requirements may 

increase the proportion of SNAP retailers carrying HIP-eligible fruits and vegetables. The Agricultural Act 

of 2014 also requires SNAP retailers to use scanning or product look-up systems, but FNS can exempt 

retailers in areas with limited access to food. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 197

 If retailer incentives are provided, then the State Agency or FNS must develop a mechanism 

and process for computing the incentive and for paying retailers. 

 Training would be needed for non-IECR retailers using EBT-only and commercial POS 

terminals. 

 Retailers might encounter additional time in checkout lanes spent in disputes over what is 

HIP and non-HIP. IECRs help to reduce this problem but WIC EBT experience highlights the 

problems that arise when new items are introduced but the UPC database is not updated in a 

timely manner. 

Participant Notification, Outreach, and Support 

The HIP experience suggests that if HIP were expanded nationwide, considerable efforts would need 

to be devoted to ensuring that SNAP participants were aware of HIP and understood how HIP 

worked, particularly which fruits and vegetables earned incentives. DTA expended considerable 

resources on participant notifications, training materials, and training sessions, yet a sizeable minority 

of households reported confusion about HIP. The fact that HIP operated as a pilot, with less than 15 

percent of a county’s SNAP households participating, and was being rigorously evaluated, limited 

outreach and education efforts. The experiences of HIP participants as documented in surveys and 

focus groups will provide valuable lessons for any future expansion. 

DTA staff at the State and local levels felt that HIP could readily be implemented as a regular part of 

SNAP. DTA and community-based organization (CBO) representatives felt that the constraints on 

communications during the pilot were a significant barrier to realizing the potential of HIP. 

Communications with participants would be important to the success of expansion. Staff suggested a 

variety of channels of communication and outreach, including the following: 

 Explain HIP to clients verbally at intake and re-certification and giving them a flyer. 

 Use a variety of media to promote HIP, including public service announcement (PSAs), 

public events, signage or videos in stores and local DTA offices, social media and word of 

mouth. 

 Provide answers to questions and problem-solving via the State’s hotline for SNAP 

participants and the EBT processor’s customer service line. 

 Combine HIP with other nutrition messages. 

 Conduct periodic focus groups of clients to obtain feedback and determine how to optimize 

the use of the incentive. 

DTA and CBO representatives also offered suggestions on how to make HIP easier and more 

appealing to participants. One suggestion was to simplify and expand the definition of eligible foods. 

Another was to increase the size of the incentive. 

Monitoring Operations and Compliance 

Many stakeholders identified both challenges and suggestions for monitoring HIP operations and 

assuring compliance. The most common theme was the importance of decisions about the lists of 

eligible foods. Other discussion focused on monitoring processors and retailers to assure that 

participants are getting the incentives they earn while no one is abusing the system. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 198 ▌9. Costs of Pilot and Feasibility of Nationwide Expansion Abt Associates

Industry stakeholders (including the EBT processor and retailers) encouraged investment in a national 

database of HIP-eligible foods. National standards were seen as important, particularly by retailers 

who operate in numerous states. Retailers expressed concern with the effort to keep food lists 

updated, as their inventories change frequently. Another concern is confusion about which canned 

and frozen fruits and vegetables are eligible. Thus, they preferred to have FNS create and maintain a 

national HIP food list. This would be a major change from the current approach for SNAP, in which 

FNS identifies food items that are eligible, and retailers then must apply FNS rules to flag these items 

in the IECRs or identify them at the POS for stand-beside terminals. The list of foods eligible for HIP 

would be standardized by FNS but the specific UPCs related to that list would be identified by 

retailers. A possible solution for the dilemma posed by these different perspectives would be to build 

on food industry labeling initiatives, so that manufacturers would take responsibility for identifying 

products that meet HIP requirements. 

FNS and other stakeholders suggested that retailer compliance with HIP rules would likely be 

managed in the same way that SNAP sales compliance is managed today, using a combination of 

participant complaints and observation. Data monitoring tools (an extension of the system used to 

detect trafficking in SNAP benefits) might also be used. Stakeholders noted that potential compliance 

issues for HIP include credits for ineligible foods, inflated sales amounts for TFV, and failure to 

identify eligible foods. 

Stakeholders also suggested that system design for HIP expansion should provide enhanced 

capabilities to monitor operations. The experience with processing problems involving TPP and 

retailer systems underscored the need for functionality in the EBT system that would enable the 

processor to monitor transactions and detect system failures. Because of the way HIP was designed 

for the pilot, it was difficult to detect problems. The problems were identified by many but not all 

clients when they did not receive their HIP incentive. DTA staff recommended that FNS design 

reporting procedures for HIP so that problems could be detected. Monitoring responsibilities for FNS, 

States, and EBT processors would have to be defined, and additional resources might be needed to 

assure that HIP is operating as intended. 

9.5 Discussion 

Total costs for the pilot were $4.4 million. System changes accounted for a little over half of all costs. 

Incentive payments to HIP participants represented just 6 percent of total costs. Costs in support of 

the evaluation were roughly 10 percent. Retailer recruitment, participant notification and training, and 

expenses for management and oversight of HIP accounted for the remaining costs. 

Projected start-up costs to expand HIP nationwide are $89.8 million. Three-fifths of these costs are 

for modifying retailer point-of-sale systems; the rest are for State Agency training and management 

costs, and for implementation by EBT processors and other State contractors. The projected value of 

incentives with nationwide expansion, based on plausible scenarios about SNAP households’ fruit 

and vegetable spending, ranges from 6 to 31 cents per household per day, or $0.8 billion to $4.5 

billion annually. 

The experience in Hampden County demonstrated that HIP was both technically and operationally 

feasible. Most stakeholders indicated strong support for expanding HIP nationwide. For nationwide 

expansion of HIP, FNS and other stakeholders would need to consider whether retailer participation 

would be voluntary or mandatory for some or all categories of SNAP-authorized retailers. Other key 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 9. Costs of Pilot and Feasibility of Nationwide Expansion ▌pg. 199

considerations include: funding, rules defining a uniform framework for HIP, need for expertise in 

retailer point-of-sale systems to oversee implementation, outreach to SNAP participants and other 

stakeholders, and monitoring SNAP redemption operations and compliance. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 10. Conclusions ▌pg. 201

10. Conclusions 

U.S. consumers at all income levels fall short of the fruit and vegetable intake recommended in 

Federal dietary guidelines and health promotion objectives. Policymakers are particularly concerned 

about low intake of fruits and vegetables among low-income Americans and Supplemental Nutrition 

Assistance Program (SNAP) participants, for whom costs are thought to represent a substantial 

barrier. 

To address this concern, some have suggested direct financial incentives as a promising approach to 

encourage fruit and vegetable consumption in this group (GAO, 2007; Dong and Lin, 2009). Because 

SNAP is the nation’s largest nutrition assistance program and an important part of the social safety 

net for low-income Americans, the program presents a potentially favorable environment for such 

efforts. The Healthy Incentives Pilot (HIP) rigorously evaluated the potential for increasing fruit and 

vegetable intake via a financial incentive that operated directly through SNAP participants’ electronic 

benefit transfer (EBT) cards. 

In this pilot, 7,500 SNAP participant households in Hampden County, MA, were randomly selected 

to receive a financial incentive equal to 30 percent of eligible fruit and vegetable purchases when they 

used their EBT cards in participating retailers. Fruits and vegetables eligible to earn the incentive, 

termed “targeted fruits and vegetables,” or “TFVs,” included a wide variety of fresh, frozen, canned, 

and dried fruits and vegetables, in contrast to some previous efforts focused on fresh produce only. 

The HIP incentive could be earned in many types of retailers, including supermarkets/superstores, 

grocery and convenience stores, and farmers markets. The remaining 47,595 SNAP households in 

Hampden County who were not randomly selected to receive the financial incentive served as a 

control group for the HIP evaluation, receiving standard SNAP benefits with no additional HIP 

incentive through the duration of the pilot. 

The HIP evaluation collected extensive information about HIP and non-HIP participants through 

SNAP case file records, EBT transaction data, and three rounds of participant surveys. The evaluation 

also conducted on-site visits with food retailers, focus groups with HIP participants, and interviewed 

many stakeholder groups. It gathered information about pilot costs and assessed the feasibility and 

costs of potential expansion of the pilot incentive. This Final Report is the third report from the HIP 

evaluation, addressing FNS’s five research objectives and reporting the results of analyses using all 

data collected during the evaluation period. 

The analysis in previous chapters showed that HIP participants had greater TFV intake than did nonparticipants. Secondary analyses showed that HIP also had positive impacts on spending on fruits and 

vegetables and on the household food environment. These secondary analyses provided suggestive 

evidence on mechanisms for the observed TFV increase, but they leave some unanswered questions. 

In this concluding chapter, we do the following: 

 Review key results for each of the five evaluation objectives, consolidating themes from 

across the preceding chapters (Section 10.1). 

 Compare and contrast key estimates from earlier chapters related to food spending and food 

intake, to better understand the mechanisms through which HIP affected food intake (Section 

10.2). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 202 ▌10. Conclusions Abt Associates

 Discuss limitations of the HIP evaluation (Section 10.3). 

 Consider lessons for the future, reflecting on whether potential expansion of the pilot would 

generate costs and impacts that are smaller or larger than those that were observed in this HIP 

evaluation (Section 10.4). 

10.1 Review of Key Results 

FNS identified five research objectives for the HIP evaluation. This section briefly summarizes and 

discusses key results pertaining to each of these five objectives. 

Objective 1: Assess the causal impact of HIP on fruit and vegetable consumption by SNAP 

participants, and on other key measures of dietary intake. 

The primary goal of HIP was to assess whether direct financial incentives can increase fruit and 

vegetable consumption among SNAP participants. To examine this question, the evaluation specified 

a single confirmatory outcome prior to analyzing the data. This measure was TFV intake among 

adults in HIP participating households, as measured in the survey data (Chapter 8). We found that 

TFV intake was 0.24 cup-equivalent higher among HIP participants than among non-participants, 

representing an increase of 26 percent. Thus, HIP was successful in increasing targeted fruit and 

vegetable intake. The increase is substantively meaningful and clearly statistically significant. 

To put our main finding into context, consider that the Federal government’s Healthy People 2020 

objectives recommend total daily fruit and vegetable intake of 3.59 cup-equivalents. We estimate total 

fruit and vegetable intakes of 2.29 cup-equivalents in our non-HIP sample.135 Thus, we found that the 

HIP impact on TFVs of 0.24 cup-equivalent was sufficiently large to narrow the 1.30-cup-equivalents 

“total fruit and vegetable intake gap” by about 18 percent. 

The study also found positive HIP impacts on many secondary consumption outcomes. However, 

with the large number of secondary analyses conducted, it is likely that some HIP/non-HIP 

differences would appear statistically significant by conventional measures even when no true impact 

was present, so these secondary results should be treated as exploratory. Nonetheless, when regular 

patterns emerge across findings, especially when they are broadly consistent with theoretical 

expectations, these secondary analyses can provide important corroborating evidence to assist with 

interpretation of primary results. 

The bulk of the difference in impacts between TFV and total fruit and vegetable intake was 

attributable to higher intake of 100% fruit juice among HIP participants. The promotion of TFVs 

through HIP may have encouraged consumption of juices, because of their fruit content, even though 

juices did not earn the incentive. Or, alternatively, HIP participants may have been confused about 

which items qualified to earn the incentive, mistakenly believing that fruit juice qualified. The fact 

that impacts on 100% fruit juice declined between Round 2 and Round 3 of the participant survey 

would be consistent with improved HIP participant understanding of the details of HIP over time. 

 

135 Healthy People 2020 recommends 2.0 cups of fruits and vegetables per 1,000 kilocalories of daily food 

energy. Mean daily food energy for the non-HIP group was 1,797 kilocalories. The national estimates and 

dietary guidelines for fruits and vegetables are broader than the narrower TFV measure; we use the TFV 

estimate as it is the evaluation’s confirmatory outcome. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 10. Conclusions ▌pg. 203

From a nutritional perspective, the HIP impact estimates were favorable. HIP had statistically 

significant positive impacts on intake of fruits separately, vegetables separately, and of several 

recommended subgroups of fruits (other fruits) and vegetables (dark green vegetables, red and orange 

vegetables, and other vegetables).136 Increased intake of fruits and vegetables was accompanied by 

increases in the total variety of fruits and vegetables consumed and the proportion of individuals 

meeting Federal dietary guidelines related to fruits and vegetables and overall dietary quality as 

measured by the 2010 Healthy Eating Index. 

Objective 2: Identify and assess factors that influence how HIP impacts participants.

The study’s conceptual framework identified two key program components through which HIP could 

influence fruit and vegetable consumption (Chapter 1). 

 Price incentive. The HIP incentive is equivalent to a 30 percent price discount on TFVs 

purchased with SNAP in participating retailers. The reduced price provides a financial 

incentive to increase fruit and vegetable spending. 

 Informational/promotional activities. HIP did not include dedicated funding for social 

marketing or nutrition promotion. Yet, the very existence of the program and its 

informational materials (the EBT card sleeve, regular mailings, and the value of the incentive 

reported on store receipts) provided implicit nutritional education on the importance of 

consuming fruits and vegetables. 

These two components mirror the two major hypothesized mechanisms through which HIP might 

influence fruit and vegetable intake: 

 Economic mechanism, operating explicitly through the price incentive 

 Informational/attitudinal mechanism, operating through implicit and explicit promotional 

activity 

Food manufacturers and retailers have long believed that advertising and marketing, accompanied by 

a price discount or coupon with genuine economic value, is an effective strategy for influencing 

consumer choices. If this marketing principle is correct, the total impact of the program could be 

larger than what one would expect from the price incentive alone. On the other hand, if households 

had limited understanding of HIP or could not easily shop at participating retailers, the total impact of 

the program could be less than what one would expect from a uniformly lower price. With these two 

hypothesized mechanisms in mind, the HIP evaluation used multiple methods to understand factors 

that influenced HIP impacts. 

First, the evaluation directly investigated impacts on intermediate outcomes for both retailers and 

participants, using the conceptual model presented in Chapter 1 as a guide. For retailers, we found 

few HIP impacts on the retail food environment, though there was some evidence of an increase in 

promotional activities related to fruits and vegetables consistent with HIP promotional efforts 

(Chapter 4). HIP participants also reported increased likelihood of having seen or heard messages 

 

136 The other fruits category includes all fruits except citrus fruits, melons, and berries. The other vegetables 

category includes vegetables other than dark green, red and orange, and starchy vegetables. Examples of 

other vegetables include celery, cucumbers, mushrooms, and asparagus. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 204 ▌10. Conclusions Abt Associates

about fruits and vegetables relative to non-participants. Nonetheless, there was little evidence that 

these promotional activities influenced attitudes about fruits and vegetables, which did not 

significantly differ between participants and non-participants after pilot implementation (Chapter 7). 

We did, however, find evidence of some changes in shopping behavior due to HIP (Chapter 6). EBT 

transaction data showed that HIP households purchased more TFVs in participating 

supermarkets/superstores than did non-HIP households. In addition, analyses of self-reported 

participant survey data found evidence of increased overall spending on fruits and vegetables. 

Respondents in the HIP group additionally reported greater availability of fruits and vegetables in the 

home (Chapter 7). These results are broadly consistent with the program model that HIP caused 

increases in spending on fruits and vegetables among participants, which translated into increased 

availability of fruits and vegetables in the home, ultimately leading to greater intake of fruits and 

vegetables among household members. 

To better understand the possible relationships between intermediate outcomes and intake, we 

conducted supplemental mediation analyses.137 Mediation analyses combine multiple regression 

models to test pathways through which programs or processes may operate by looking at whether and 

how intermediate variables link with other variables. While most analyses in this Final Report rely on 

the random assignment research design to provide unbiased impact estimates, the mediation analysis 

requires additional strong statistical assumptions (which are unlikely to be exactly satisfied) to be 

considered unbiased. 

With that caveat, the mediation analysis found statistically significant evidence that food spending 

served as a mediating variable between HIP participation and fruit and vegetable intake, and 

suggestive evidence that having fruits and vegetables in the home served as a mediating variable 

(consistent with the discussion above). Interestingly, although as noted above no direct HIP impacts 

on attitudes were observed, the mediation analysis did find suggestive evidence that favorable 

attitudes toward fruits and vegetables served as a mediating variable between HIP participation and 

fruit and vegetable intake. To the extent that the statistical assumptions required for these models are 

valid, the mediation analysis thus provides some additional support for both the economic and 

informational/attitudinal mechanisms for HIP impacts. 

Finally, we conducted analyses for population subgroups, to assess whether particular types of 

individuals were more or less likely to be affected by the pilot. Given the sample sizes, the study 

could detect only relatively large differences in impacts across subgroups. With that caveat in mind, 

the results of the subgroup analysis provide no evidence of differences in HIP impacts on fruit and 

vegetable intake associated with demographic variables such as gender, age, race/ethnicity, disability 

status, employment status, and household composition. However, there was a large and statistically 

significant difference based on pre-implementation attitudes about fruits and vegetables. In 

households where respondents in the baseline survey had favorable attitudes toward fruits and 

vegetables, HIP impacts were much larger. 

 

137 Appendix I presents the detailed results from the mediation analyses. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 10. Conclusions ▌pg. 205

In summary, our findings on HIP impacts on participants are as follows: 

 HIP increased participant spending on fruits and vegetables and availability of fruits and 

vegetables at home, which appears to explain, in part, the ultimate increase in fruit and 

vegetable intake among HIP participants. 

 While we found no direct evidence of HIP impacts on participant attitudes related to fruits 

and vegetables, there was some weak evidence that attitudes served as a mediator between 

HIP impacts and fruit and vegetable intake. 

 Impacts were concentrated among those participants with more favorable attitudes about 

fruits and vegetables prior to implementation. 

A puzzle remains in interpreting our findings. The observed increase in TFV spending captured in the 

EBT transaction data seems too small to explain most of the ultimate impact on fruit and vegetable 

intake purely through the price mechanism, suggesting that other mechanisms were important. Taken 

as a whole, our results provide evidence that HIP increased fruit and vegetable consumption through 

both the price mechanism and the informational/attitudinal mechanism. We revisit this issue in 

Section 10.2. 

Objective 3: Describe the processes involved in implementing and operating HIP.

Understanding the process involved in implementing HIP and the challenges that arose are important 

for two reasons. First, implementation may influence the pilot’s outcomes. As discussed in Chapter 1, 

HIP impacts depended on the project’s success in recruiting and training retailers, implementing new 

EBT processing methods, and informing participants. During the pilot, many difficult aspects of 

implementation went smoothly, but some did not. Second, HIP provides valuable information for any 

future pilots or major roll-out of a HIP-like program. 

As described in Chapter 3, HIP implementation was a complex undertaking, with many actors and 

stakeholders. Despite some technical systems issues, principally occurring early in the 

implementation period (late 2011 and early 2012), the pilot clearly demonstrated that issuing direct 

financial incentives through the SNAP EBT card is technically feasible under real-world conditions in 

all types of commercial food retail formats. 

The two biggest implementation challenges were retailer recruitment and ensuring that participants 

fully understood how the pilot worked. 

 DTA reached out to recruit as many authorized SNAP retailers in Hampden County as 

possible, at all scales of operation ranging from supermarkets/superstores to corner stores to 

farmers markets. Supermarkets/superstores accounted for approximately 80 percent of SNAP 

redemptions in Hampden County and therefore were particularly important to overall HIP 

impacts. One major supermarket/superstore chain with a large retail presence declined to 

participate despite vigorous recruitment efforts. In the end, only half of SNAP redemptions of 

HIP participant households took place in participating retailers (Chapter 5). In Section 10.2 

we discuss the potential effects on HIP impacts of not having easy access to a participating 

supermarket/ superstore based on analysis of EBT spending patterns. 

 DTA invested heavily in developing direct participant notification and training materials, 

working to design brochures and other information that were easy to understand. For 

communication through retail channels, using in-store signage and placards, DTA had to 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 206 ▌10. Conclusions Abt Associates

strike a difficult balance, designing signage that communicated clearly to HIP participants 

without confusing or contaminating the control group. While DTA offered more than 140 

training sessions, only 1.3 percent of HIP participants reported attending one of them. Just 

over 10 percent of HIP participants called DTA’s telephone assistance line. Stakeholder 

interviews (Chapter 3), as well as participant surveys and focus group interviews (Chapter 5) 

suggested that many participants had incomplete awareness and understanding of HIP. In the 

final round of the participant survey, nearly a quarter of HIP participants still reported that 

they had not heard of HIP. In addition, both survey and focus group data found that many 

participants appear to have misunderstood how HIP worked, including which retailers were 

participating, which foods were eligible, or how the incentive was earned. 

The retailer recruitment challenge does not appear fundamental to the nature of the financial 

incentive. As we discuss below, it would probably be surmounted if a HIP-like incentive were 

expanded beyond the pilot scale or on a longer-term basis (Section 10.4). The participant awareness 

and understanding challenge is more subtle. Awareness and understanding are likely to improve with 

broad roll-out, but may remain quite imperfect, with potential implications for the impact of a broadly 

rolled-out program. DTA’s attempts at in-person training were intensive and well designed. The very 

low attendance suggests that increasing awareness and understanding will require some other 

strategy. 

Objective 4: Assess the impact on the HIP grantee (the State SNAP agency), the local SNAP 

agency, and their team of partners (including retailers, State EBT processor, and community 

organizations).

HIP involved extensive implementation efforts from Massachusetts DTA staff, the State EBT 

processor (Xerox), and third-party processors. Almost all their labor and non-labor costs were 

reimbursed by FNS through the pilot budget (Chapter 9). These stakeholders reported in interviews 

that HIP expansion would be technically and operationally feasible. DTA staff considered the pilot to 

be a successful innovation, enhancing the agency’s efforts to promote health and nutrition through the 

Federal nutrition assistance programs. 

For retailers in Hampden County, impacts were mixed. One convenience store chain reported that 

HIP offered little benefit, because the chain carried few fruits and vegetables. As noted earlier, one 

supermarket/superstore chain did not participate, but this was likely due to the temporary nature of 

the pilot and the relatively short implementation period. For the many Hampden County retailers that 

did participate, HIP had only modest effects on SNAP redemptions. Just 7,500 of the 55,095 SNAP 

households in Hampden County were randomly assigned to the HIP group, and the mean HIP 

purchase amounts for participating households were quite small. Presumably the benefits for retailers 

and for product suppliers in the fruit and vegetable industries would be considerably larger if the 

financial incentive were expanded into a permanent program for the full SNAP population. 

HIP had relatively little effect on store operations. Over 90 percent of retailers reported that they 

experienced no change in check-out time due to HIP. Less than 20 percent of retailers reported that 

HIP purchases were hard to process or that training workers was a burden. Few retailers reported 

problems during the pilot and almost all supermarkets/superstores and grocery stores were satisfied 

with how HIP worked in their stores. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 10. Conclusions ▌pg. 207

Objective 5: Quantify, to the extent possible, the Federal, State, and local administrative and 

benefit costs of the pilot.

HIP costs include variable costs that are proportional to the number of participants (primarily the 

costs of the actual incentive earned per participant household) and fixed costs that must be incurred 

just once during implementation (such as changes to software that processes EBT transactions). If a 

financial incentive were extended nationwide, the fixed costs would be spread over a much larger 

number of SNAP households, so the variable costs would be the most important cost consideration 

for Federal policymakers. 

Total costs for implementing HIP, not counting the incentives earned by HIP participants, were $4.2 

million. The largest component was $2.5 million for system design, development, and testing 

(including FNS’ costs for testing consultant). General administration was $0.7 million. Other cost 

items were much smaller. Most of these costs are fixed costs. They substantially affected the total 

cost of implementing a pilot in a single county. Yet, even acknowledging that some of these costs 

would be multiplied by the number of State agencies, EBT processors, and retail chains across the 

country, our corresponding estimate of total national implementation costs is $89.8 million. 

The most important variable cost is the value of the financial incentive itself. In the pilot, the mean 

monthly incentive earned (during March-October 2012) was $3.65 per HIP household. This was 

considerably smaller than anticipated, and resulted in total incentive payments of $263,043. In 

estimating incentive costs in a nationwide expansion, we considered several scenarios, based on 

different assumptions about participant behaviors, particularly related to improved client awareness 

and understanding and increased retailer participation. At the low end, assuming household incentive 

earnings are the same as in the pilot, annual national incentive costs would be $0.8 billion. At the high 

end, assuming that households purchase an amount of TFV worth 53 percent of USDA’s Thrifty Food 

Plan (TFP) recommendation for fruits and vegetables, annual incentive costs would be $4.5 billion. 

10.2 Comparing HIP Impacts on Spending and Intake 

As noted above, the evaluation specified a single confirmatory outcome—targeted fruits and 

vegetables as measured in the 24-hour dietary recall interviews. Based on that confirmatory analysis, 

we conclude that HIP was successful in achieving its chief goal of increasing fruit and vegetable 

intake among participants. However, this evidence must be assessed in concert with secondary 

evidence from the evaluation—in particular, estimated impacts on TFV spending in participating 

stores (based on EBT data), which are lower than would be expected given the estimated impact on 

TFV intake. 

In one sense, the magnitude of the HIP impact on TFV intake was only moderately higher than one 

would expect from a 30 percent price incentive working only through the economic mechanism (see 

Appendix A). The earlier literature reviewed in Chapter 1, based on elasticity estimates compiled by 

Andreyeva, Long, and Brownell (2012), led to a preliminary hypothesis that a 30 percent price 

reduction would imply approximately a 19 percent increase in TFV spending. For comparison, the 

HIP evaluation found that HIP increased targeted fruit and vegetable intake by 26 percent, which is 

moderately higher than the prediction based on mean elasticity estimates. A discrepancy of this 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 208 ▌10. Conclusions Abt Associates

magnitude could easily be explained by either the variability in the earlier elasticity estimates or by 

the sampling variability in the evaluation’s impact estimates. 138 

In another sense, the HIP impact on self-reported fruit and vegetable spending and intake was much 

larger than can be explained under the assumption that HIP operated primarily through the economic 

mechanism ( i.e., due to the direct effects of the price incentive). If HIP participants understood that 

they would earn the incentive only by making TFV purchases at HIP participating retailers using their 

SNAP benefits, and if no other factors caused their TFV purchasing behavior to differ from non-HIP 

households, then the impact on TFV purchases in participating retailers (as recorded in the EBT data) 

would be expected to closely correspond to ultimate impacts on TFV intake (as recorded in the 

household survey data). However, Section 6.1 noted that the HIP impact on self-reported total fruit 

and vegetable spending is roughly five times as large as the HIP impact on qualifying TFV purchases 

with EBT benefits in participating retailers. Similarly, Section 8.2 noted that the HIP impact on TFV 

intake is approximately four times higher than would be implied by the corresponding impact on TFV 

spending in participating retailers as estimated from the EBT transaction data. 

Taken together, these results can be interpreted as indicating that HIP affected self-reported spending 

and intake for fruits and vegetables, including both fruits and vegetables that earned and did not earn 

the HIP incentive. The substantial estimated effect on fruits and vegetables that did not earn the 

incentive was surprising. Fruits and vegetables that did not earn incentives included those acquired in 

several ways, including: 

 TFVs purchased at non-participating retailers (the incentive was earned only in participating 

retailers). 

 TFVs purchased with cash or acquired for free (the incentive was earned only for purchases 

with the EBT card). 

 Fruits and vegetables that did not qualify as TFVs, including white potatoes and 100% fruit 

juices (the incentive was earned only for purchases of qualifying TFVs.) 

We have identified three possible reasons for HIP’s impact on self-reported spending and intake for 

fruits and vegetables that did not earn the incentive. A combination of all three reasons likely explains 

the finding. 

1. An informational/attitudinal mechanism. As noted in Section 10.1, HIP may have affected 

participant exposure to or preferences for fruits and vegetables. We found no direct evidence 

of this, but such effects are suggested by evidence from the mediator analysis and from nonexperimental subgroup analyses (Appendix I). These effects could in turn have influenced 

future intake of fruits and vegetables, whether or not they earned the incentive. Some impact 

through this informational pathway was anticipated from the earliest stages of designing the 

 

138 The original estimates by Andreyeva, Long, and Brownell (2012) included a 90 percent confidence interval 

around the mean elasticities in the literature. Our estimated HIP impact of 26 percent is within the range of 

impacts that is consistent with earlier elasticity estimates (see Chapter 1). Intake estimates in Chapter 8 

have a 95 percent confidence interval that ranges from 0.132 cups to 0.344 cup-equivalents, which suggests 

an elasticity of between 15 and 38 percent, clearly consistent with the literature on the price elasticity of 

fruit and vegetable purchase. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 10. Conclusions ▌pg. 209

HIP evaluation and is part of the conceptual framework. However, it was expected that this 

pathway would be secondary to the price incentive mechanism. 

2. Misunderstanding how the pilot worked. Some HIP participants may have thought they would 

earn the incentive in certain circumstances that did not in fact earn the incentive, such as 

purchases of non-qualifying fruits and vegetables or transactions in non-participating 

retailers. There is some evidence for this conjecture. Survey and focus group evidence 

presented in Chapter 5 is consistent with imperfect understanding. In addition, Section 8.1 

noted that the HIP impact on 100% fruit juice was concentrated among respondents who also 

reported that the incentive was difficult to understand. Section 8.7 observed that the HIP 

impact on fruit juice intake shrank from Round 2 to Round 3, as would be expected if 

participants were still learning which foods and beverages qualified. 

3. Survey Response Bias. Finally, there is some possibility of survey response bias. Social 

desirability bias could potentially have induced differential survey response error among HIP 

participants. In particular, the pilot may have caused HIP participants to become increasingly 

aware of the social desirability of fruit and vegetable intake, leading them to report higher 

spending on and intake of fruits and vegetables than non-HIP participants did. Alternatively, 

the pilot may have made HIP households more conscious of fruit and vegetable intake, 

leading to recall of more of the fruits and vegetables consumed (relative to the non-HIP 

group). As noted earlier (and discussed in detail in the next section), the procedures through 

which food intake was measured utilized a standard and well-accepted protocol, which would 

seem to limit the likelihood of substantial bias. In addition, we found that survey estimates of 

fruit and vegetable intake were consistent with survey estimates of spending on fruits and 

vegetables. However, we have no direct evidence either for or against the form of survey 

response bias that would lead to over-estimates of impact. 

Absent other plausible explanations, it seems reasonable to explain the discrepancy in impacts by 

some combination of the three hypotheses and their underlying causes: attitudinal improvements, 

confusion about the HIP incentive, and/or survey response bias. As noted above, the impact on TFV 

intake is approximately four times higher than would be expected given estimated impacts on TFV 

spending with EBT benefits in participating retailers. This implies that mechanisms other than the 

economic incentive must explain approximately three-quarters of the total impact on TFV intake. 

10.3 Limitations 

The random-assignment research design used in the HIP evaluation offers what is considered the 

strongest available method for estimating program impacts without bias. In alternative, nonexperimental regression-based approaches, bias commonly arises from unobserved confounding 

variables and reverse causation. Random assignment, when properly implemented, remedies both of 

these problems. There is no evidence that random assignment was improperly implemented in the 

HIP evaluation. Nevertheless, the research design used here does have limitations. 

Incomplete Follow-Up

Random assignment yields causal estimates when outcomes for the treatment and control groups are 

compared for everyone randomized. In practice, this condition is almost never exactly satisfied. In the 

HIP evaluation there were two issues: survey non-response and a deliberate decision not to follow 

households that left SNAP. We discuss these two issues below. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 210 ▌10. Conclusions Abt Associates

First, when there is substantial non-response, the survey sample may not represent the underlying 

population without bias. This, in turn, can lead to the treatment and control groups not being 

otherwise identical. The response rates for the pre-implementation participant survey were 63 percent 

of HIP-eligible sampled households and 64 percent of non-HIP-eligible sampled households. In 

Rounds 2 and 3, the follow-on response rates were between 80 and 84 percent (among those who 

responded to the baseline survey). These response rates are consistent with other surveys of similar 

populations based on program lists (and the locating information in those program lists). Furthermore, 

we constructed weights to adjust for differential non-response with respect to information known to 

both respondents and non-respondents , mitigating concerns about bias from this source. 

Second, while standard practice in random assignment studies is (to attempt) to interview (and then 

analyze) everyone randomized, budget considerations led the HIP evaluation to interview (and then 

analyze) only households who remained on SNAP. Households that left SNAP were not surveyed. 

Statistical tests indicated that the HIP and non-HIP groups remained balanced over time—both in 

terms of exit and retention rates and in terms of observable characteristics. It is therefore reasonable 

to interpret the evaluation results in the same way as for a study that followed everyone who had been 

randomized. 

Dietary Recall Instrument

For the key food intake outcomes, there may be underreporting in the 24-hour dietary recall 

instrument. The Automated Multiple Pass Method (AMPM) is designed to enhance respondents’ 

ability to recall food consumed during the previous day, and is thought to yield less underreporting 

than other assessment methods (Subar et al., 2003). In the participant survey, total daily food energy 

intake was 1,749 kilocalories (HIP) and 1,797 (non-HIP), which appears to be lower than typical food 

energy intake for adults. For example, in the Nutritional and Health Examination Survey (NHANES), 

mean daily food energy intake for U.S. male adults aged 20 years and older exceeds 2,600 

kilocalories, and mean daily food energy intake for female adults aged 20 years and older was 1,785 

kilocalories.139 The relatively high proportions of females, elderly, and low-income respondents in 

our sample likely explain most of the discrepancy, but we cannot entirely dismiss the possibility of 

underreporting. 

Inasmuch as there is underreporting of total intake, there is also likely to be underreporting of fruit 

and vegetable intake. The study’s primary interest is in differential HIP/non-HIP fruit and vegetable 

intake, so differential HIP/non-HIP misreporting of fruit and vegetable intake would be problematic. 

Such differential reporting is possible, but there is no direct evidence that it occurred. Instead, the HIP 

evaluation relies on the plausible and conventional assumption that any underreporting is not related 

to program participation status, and hence unable to cause bias in key impact estimates. However, 

social desirability considerations might suggest that HIP participants would be more likely than nonHIP participants to overstate (or to understate to a lesser degree) their fruit and vegetable intake. 

Differential misreporting of this form would lead to overstating the true impact. 

Although we cannot absolutely rule out the possibility of this type of bias, several features of the 

study design bolster our confidence that any such bias is likely to be small. The AMPM interview is 

 

139 National Health and Nutrition Examination Study (NHANES): 

http://www.ars.usda.gov/SP2UserFiles/Place/12355000/pdf/0506/Table_2_NIF_05.pdf 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates 10. Conclusions ▌pg. 211

considered the gold standard for assessing dietary intake, consisting of a complete itemization of 

foods consumed on the previous day. Relative to a simpler food frequency questionnaire or other 

short battery of survey questions, this data collection method should minimize respondents’ ability to 

(consciously or subconsciously) over-report fruit and vegetable consumption relative to other types of 

consumption. In addition, there was no mention of HIP in Round 1 of the participant survey. In 

Round 2 and Round 3, the AMPM interview was conducted prior to mentions of HIP and other parts 

of the interview. When completing the AMPM interview, HIP participants were not therefore 

“primed” with exposure earlier in the same interview to questions about HIP or their attitudes about 

fruits and vegetables. (By Round 3, HIP participants had already experienced all sections of the 

interview, albeit several months earlier). 

Statistical Power Considerations

The research design had sufficient statistical power to measure statistically significant HIP/non-HIP 

differences in the main confirmatory TFV intake outcome, but other outcomes should be treated as 

exploratory, as should our subgroup analyses. 

Generalizability

In addition to these research design limitations, caution is needed when extrapolating from the pilot 

setting to a much broader roll-out. Our random assignment design offers the highest level of internal 

validity (providing great assurance that we correctly measure HIP impacts in the Hampden County 

pilot) but less external validity (providing less assurance that the HIP impacts would be the same for 

the whole United States). Also, a universal and permanent financial incentive would almost certainly 

generate different retailer and participant responses than did a partial and temporary pilot incentive. 

This study partially addressed this possibility by using two rounds of post-implementation surveys, 

one early and one later, but it remains possible that incentive impacts in a permanent program would 

either increase or decrease over a longer period of time. The design provides no direct evidence on 

the differential impact of a full versus partial program. 

10.4 Conclusions 

HIP was envisioned primarily as a financial inducement to increase fruit and vegetable intake among 

SNAP participants. A 30 percent incentive lowers the net price and standard economic theory predicts 

that this decrease in price would increase intake by roughly 20 percent. In addition, HIP’s impact 

could plausibly be amplified by the implicit and explicit nutrition education and marketing provided 

by HIP—mailings, the EBT card sleeve, and register receipts summarizing incentive earnings—and 

by the very existence of a program that rewarded fruit and vegetable purchases. 

The evaluation estimated that HIP increased TFV intake by 0.24 cup-equivalents, or about 26 percent, 

consistent with the projected magnitude of impacts based on economic theory. This increase is clearly 

statistically significant and substantively large—closing 18 percent of the gap between current and 

recommended fruit and vegetable intake. Thus, by our confirmatory measure, HIP was successful. 

However, our impact estimates on TFV expenditures based on EBT transaction data are substantially 

smaller than would be expected given the observed impacts on TFV intake. Taken together, the two 

pieces of evidence suggest that economic mechanisms alone cannot fully explain the increase in 

survey-reported TFV intake induced by HIP. The principal alternative explanations for this 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 212 ▌10. Conclusions Abt Associates

divergence, which must explain a large fraction of the impact, likely include participant confusion 

about the pilot and informational and attitudinal effects. 

The experience of implementing HIP and some of the secondary analyses conducted as part of the 

evaluation provide additional insights about what impacts might be seen in a larger roll out of a 

similar program and suggest avenues for future research. There are several reasons to think of the 

pilot as a work in progress, and therefore to expect larger incentive earnings in a universal and 

permanent program. Given our findings about the relative importance of economic, informational, 

and attitudinal mechanisms in understanding HIP’s impact on TFV intake, it is unclear to what extent 

increased incentive use would lead to greater impacts on TFV intake. It is also unclear to what extent 

a larger incentive would lead to greater impacts on TFV intake or a smaller incentive would lead to 

smaller impacts on TFV. 

First, with a permanent program and more participants, a larger fraction of retailers, and especially a 

larger fraction of supermarkets and superstores, would likely participate. We earlier described 

evidence that HIP purchases were somewhat larger for participants who did not need to alter their 

choice of retailer, because they already were patronizing a participating retailer before pilot 

implementation. 

Second, with universal participation, there would be more public information about the incentives, 

more retailer signage, and more word-of-mouth about how the program worked. Together these 

mechanisms would likely lead to improved understanding of the operation of the incentive. 

Participants would thereby come to understand the savings from shifting any existing TFV purchases 

onto their EBT card rather than paying for fruits and vegetables with cash or other tenders. These 

dynamics could lead to increased costs for the financial incentive and perhaps to increased impacts on 

consumption. 

The HIP evaluation analysis suggests a number of areas in which further research could help us 

understand the underlying mechanisms by which HIP impacted fruit and vegetable consumption. 

Understanding the underlying mechanisms could allow us to disentangle the possible explanations for 

the observed effects and thus better understand how to design an effective incentive program. 

The evaluation clearly showed that households responded to the price incentive. One research 

question to pursue is how incentives earned and impacts on TFV intake would vary with the size of 

the incentive. Inasmuch as HIP worked through the non-price pathway, a smaller incentive, perhaps 

20 percent or even 10 percent, might generate similar increases in fruit and vegetable consumption. 

Conversely, inasmuch as HIP worked through the price pathway, a larger incentive might generate 

larger impacts on TFV intake. 

A second avenue to explore is how to increase information about the incentive and whether and how 

much increased information about the incentive program would lead to greater impacts. The 

evaluation showed that a substantial number of households were confused about HIP—about the 

program itself, about what fruits and vegetables earned the incentive, and in what stores incentives 

could be earned. 

Finally, while messages about the importance of consuming fruits and vegetables as part of a healthy 

diet are fairly widespread, examining the combined effect of intensive nutrition education with a 

financial incentive program could provide valuable insights. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates References ▌pg. 213

References 

Anderson, J. V., Bybee, D. I., Brown, R. M., McLean, D. F., Garcia, E. M., Breer, M. L., and Schillo, 

B. A. (2001). 5 a day fruit and vegetable intervention improves consumption in a low income 

population. Journal of the American Dietetic Association 101(2), 195–202. 

Andreyeva, T., Long, M. W., and Brownell, K. D. (2010). The impact of food prices on consumption: 

A systematic review of research on the price elasticity of demand for food. American Journal of 

Public Health 100, 216–222. 

Andreyeva, T., Luedicke, J., Middleton, A. E., Long, M. W., and Schwartz, M. B. (2012). Positive 

influence of the Revised Special Supplemental Nutrition Program for Women, Infants, and Children 

food packages on access to healthy foods. Journal of the Academy of Nutrition and Dietetics 112, 

850–858. 

Angrist, J. D. (2001). Estimations of limited dependent variable models with dummy endogenous 

regressors: Simple strategies for empirical practice. Journal of Business and Economic Statistics 

19(1), 2–16. 

Angrist, J. D., and Pischke, J.-S. (2008). Mostly harmless econometrics: An empiricist's companion.

Princeton, NJ: Princeton University Press. 

Baron, R. M., and Kenny, D. A. (1986). The moderator-mediator variable distinction in social 

psychological research: Conceptual, strategic, and statistical consideration. Journal of Personality 

and Social Psychology 5(6), 1173–1182. Available at: 

http://www.public.asu.edu/~davidpm/classes/psy536/Baron.pdf.

Bartlett, S., Beauregard, M., Logan, C., Komarovsky, M., Wommack, T., Wilde, P., Owens, C., and 

Melhem, M. (2013). Healthy Incentives Pilot (HIP) early implementation report. Alexandria, VA: 

U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support. 

Bartlett, S., Klerman, J., Wilde, P., Olsho, L., Blocklin, M., Logan, C., and Enver, A. (2013). Healthy 

Incentives Pilot (HIP) interim report. Alexandria, VA: U.S. Department of Agriculture, Food and 

Nutrition Service, Office of Policy Support. 

Blisard, W. N., and Stewart, H. (2006). How low-income households allocate their food budget 

relative to the cost of the Thrifty Food Plan. Washington, DC: U.S. Department of Agriculture, 

Economic Research Service. 

Bowman, S. A., Friday, J. E., and Moshfegh, A. (2008). MyPyramid Equivalents Database, 2.0 for 

USDA Survey Foods, 2003-2004 [online document]. Food Surveys Research Group, Beltsville 

Human Nutrition Research Center, Agricultural Research Service. Beltsville, MD: U.S. Department 

of Agriculture. Retrieved from: http://www.ars.usda.gov/ba/bhnrc/fsrg. 

Bullock, J. G., Green, D. P., and Ha, S. E. (2010). Yes, but what's the mechanism? (Don't expect an 

easy answer). Journal of Personality and Social Psychology, 98, 550–558. Available at: 

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1543925.

Carriquiry, A. L., and Camano-Garcia, G. (2006). Evaluation of dietary intake data using the tolerable 

upper intake levels. Journal of Nutrition 136, 507S–513S. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 214 ▌References Abt Associates

Casagrande, S., Wang, Y., Anderson, C., and Gary, T. (2007). Have Americans increased their fruit 

and vegetable intake? The trends between 1988 and 2002. American Journal of Preventive Medicine 

32(4), 257–263. 

Castner, L., and Henke, J. (2011). Benefit Redemption Patterns in the Supplemental Nutrition 

Assistance Program. Alexa Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition 

Service, Office of Research and Analysis. 

Chu, A. (2013). Healthy Incentives Pilot (HIP) interim report: Participant survey weighting 

methodology. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service, Office of 

Policy Support. 

Chung, C., and Myers, S. (1999). Do the poor pay more for food? An analysis of grocery store 

availability and food price disparities. Journal of Consumer Affairs 33, 276–296. 

Cody, S., Castner, L. A., Mabli, J., and Sykes, J. (2007). Dynamics of Food Stamp Program 

participation, 2001–2003. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition 

Service, Office of Research, Nutrition, and Analysis. 

Cole, N., and Fox, M.K. (2008). Diet quality of Americans by Food Stamp participation status: Data 

from the National Health and Nutrition Examination Survey, 1999–2004. Alexandria, VA: U.S. 

Department of Agriculture, Food and Nutrition Service, Office of Research, Nutrition and Analysis. 

Congressional Budget Office (2014). Supplemental Nutrition Assistance Program CBO Baseline, 

April 2014. http://www.cbo.gov/sites/default/files/cbofiles/attachments/44211-2014-04-SNAP.pdf. 

Accessed May 1, 2014. 

Cramér, H. (1946). Mathematical models of statistics. Princeton: Princeton University Press, p. 353. 

Dong, D., and Lin, B. H. (2009). Fruit and vegetable consumption by low-income Americans: Would 

a price reduction make a difference? (ERS Report 70). Washington, DC: U.S. Department of 

Agriculture, Economic Research Service. Available at: 

http://www.ers.usda.gov/publications/err70/err70.pdf. 

Erinosho, T. O., Moser, R. P., Oh, A. Y., Nebeling, L. C., and Yaroch, A. L. (2012). Awareness of 

the Fruits and Veggies—More Matters campaign, knowledge of the fruit and vegetable 

recommendation, and fruit and vegetable intake of adults in the 2007 Food Attitudes and Behaviors

(FAB) Survey. Appetite 59, 155–160. 

Fox, M. K., Hamilton, W., and Lin, B-H. (2004). Effects of food assistance and nutrition programs on 

nutrition and health: Volume 3, literature review (Food Assistance and Nutrition Research Report 

No. FANRR19-3). Washington, DC: U.S. Department of Agriculture, Economic Research Service. 

Fraker, T. (1990). The effects of food stamps on food consumption: A review of the literature. 

Washington, DC: U.S. Department of Agriculture, Food and Nutrition Service. 

Frazão, E., Andrews, M. S., Smallwood, D., and Prell, M.A. (2007). Food spending patterns of lowincome households: Will increasing purchasing power result in healthier food choices? (Economic 

Information Bulletin Number 29-4). Washington, DC: U.S. Department of Agriculture, Economic 

Research Service. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates References ▌pg. 215

Freedman, D. A., Choi, S. K., Hurley, T., Anadu, E., and Hébert, J.R. (2013). A farmers’ market at a 

federally qualified health center improves fruit and vegetable intake among low-income diabetics. 

Preventive Medicine 56(5), 288–292. doi:10.1016/j.ypmed.2013.01.018. 

Freedman, L. S., Guenther, P. M., Krebs-Smith, S. M., Dodd, K. W., and Midthune, D. (2010). A 

population's distribution of Healthy Eating Index-2005 component scores can be estimated when 

more than one 24-hour recall is available. Journal of Nutrition 140(8), 1529–1534. doi: 

10.3945/jn.110.124594. 

French, S.A. (2003). Pricing effects on food choices. Journal of Nutrition 133(3), 841S–843S. 

French, S.A. (2005). Public health strategies for dietary change: schools and workplaces. Journal of 

Nutrition 135(4), 910–912. 

French, S.A., and Wechsler, H. (2004). School-based research and initiatives: Fruit and vegetable 

environment, policy, and pricing workshop. Preventive Medicine 39(Suppl 2), S101–107. 

French, S.A., Story, M., and Jeffery, R. W. (2001). Environmental influences on eating and physical 

activity. Annual Review of Public Health 22, 309–335. 

French, S.A., Story, M., Jeffery, R. W., Snyder, P., Eisenberg, M., Sidebottom, A., and Murray, D. 

(1997). Pricing strategy to promote fruit and vegetable purchase in high school cafeterias. Journal of 

the American Dietetic Association 97(9), 1008–1010. 

George, D., and Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 

11.0 update (4th ed.). Boston: Allyn & Bacon. 

Government Accountability Office. (2008). Food Stamp Program: Options for delivering financial 

incentives to participants for purchasing targeted foods (GAO-08-415). Washington, D.C.: Author. 

Available at: http://www.gao.gov/new.items/d08415.pdf. 

Greene, W. H. (2003). Econometric analysis (5th ed.). New York: Prentice-Hall, p. 913f. 

Grimm, K. A., Foltz, J. L., Blanck, H. M., and Scanlon, K. S. (2012). Household income disparities in 

fruit and vegetable consumption by state and territory: Results of the 2009 Behavioral Risk Factor 

Surveillance System. Journal of the Academy of Nutrition and Dietetics 112, 2014–2021. 

Grindal, T., Schwartz, G., Klerman, J., Bartlett, S., and Wilde, P. (2014). Healthy Incentives Pilot 

(HIP) spatial analysis. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service, 

Office of Policy Support. 

Guenther, P.M., Casavale, K.O., Kirkpatrick, S.I., Reedy, J., Hiza, H.A.B., Kuczynski, K.J., Kahle, 

L.L., and Krebs-Smith, S.M. (2013). “Diet Quality of Americans in 2001-02 and 2007-08 as 

Measured by the Healthy Eating Index-2010.” Nutrition Insight 51. Available at: 

http://www.cnpp.usda.gov/Publications/NutritionInsights/Insight51.pdf. 

Guthrie, J., Lin, B., Ver Ploeg, M., and Frazao, E. (2007). Can food stamps do more to improve food 

choices? (Economic Information Bulletin No. 29-1). Washington, DC: U.S. Department of 

Agriculture, Economic Research Service. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 216 ▌References Abt Associates

Hanson, K. (2010). The Food Assistance National Input-Output Multiplier (FANIOM) Model and 

Stimulus Effects of SNAP (Economic Research Report No. ERR-103). Washington, DC: U.S. 

Department of Agriculture, Economic Research Service. 

He, K., Hu, F.B., Colditz, G.A., Manson, J.E., Willett, W.C., and Liu, S. (2004). Changes in intake of 

fruits and vegetables in relation to risk of obesity and weight gain among middle-aged women. 

International Journal of Obesity Related Metabolic Disorders 28(12), 1569-74. 

He, F. J., Nowson, C. A., Lucas, M., and MacGregor, G. A. (2007). Increased consumption of fruit 

and vegetables is related to a reduced risk of coronary heart disease: Meta-analysis of cohort studies. 

Journal of Human Hypertension 21, 717–728. 

He, F. J., Nowson, C. A., and MacGregor, G. A. (2006). Fruit and vegetable consumption and stroke: 

Meta-analysis of cohort studies. Lancet 367, 320–326. 

Herman, D.R., Harrison, G.G., Afifi, A.A., and Jenks, E. (2008). Effect of a targeted subsidy on 

intake of fruits and vegetables among low-income women in the Special Supplemental Nutrition 

Program for Women, Infants, and Children. American Journal of Public Health 98(1), 98-105. 

Horgen, K.B., and Brownell, K.D. (2002). Comparison of price change and health message 

interventions in promoting healthy food choices. Health Psychology 21, 505-12. 

Imai, K., Keele, L., and Yamamoto, T. (2010). Identification, inference, and sensitivity analysis for 

causal mediation effects. Statistical Science, 25(1), 51-71. 

Institute of Medicine (IOM). (2000a). Dietary Reference Intakes: Applications in dietary assessment. 

Washington, DC: National Academies Press. 

Institute of Medicine (IOM). (2000b). Dietary Reference Intakes for vitamin C, vitamin E, selenium, 

and carotenoids. Washington, DC: National Academies Press. 

Institute of Medicine and National Research Council. (2013). Supplemental Nutrition Assistance 

Program: Examining the evidence to define benefit adequacy. Washington, DC: The National 

Academies Press. 

Jago, R., Baranowski, T., and Baranowski, J. C. (2007). Fruit and vegetable availability: A micro 

environmental mediating variable? Public Health Nutrition 10(7), 681–689. 

Jeffery, R.W., French, S.A., Raether, D., and Baxter, J. E. (1994). An environmental intervention to 

increase fruit and salad purchases in a cafeteria. Preventive Medicine 23(6), 788–792. 

Jha, P., Chaloupka, F.J., Corrao, M., and Jacob, B. (2006). Reducing the burden of smoking worldwide: Effectiveness of interventions and their coverage. Drug Alcohol Review 25(6), 597–609. 

Kuchler, F., Tegene, A., and Harris, J. (2005). Taxing snack foods: manipulating diet quality or 

financing information program? Review of Agricultural Economics 27(1), 4−20. 

Klerman, J. A. (2013). Random assignment evidence on the Southworth Hypothesis. Cambridge, MA: 

Abt Associates Inc., June. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates References ▌pg. 217

Larson, N., and Story, M. (2009). A review of environmental influences on food choices. Annals of 

Behavioral Medicine 38, 56–73. 

MacKinnon, D. (1994). Analysis of mediating variables in prevention and intervention research. In A. 

Cázares and L. Beatty (Eds.), Scientific methods for prevention intervention research (NIDA 

Monograph No. 139, pp. 127–153). Rockville, MD: National Institute on Drug Abuse. 

MacKinnon, D.P. (2008). Introduction to statistical mediation analysis. Mahwah, NJ: Erlbaum. 

Maddala, G.S. (1983). Limited dependent and qualitative variables in econometrics. Cambridge 

University Press. 

Morland, K., Wing, S., Diez Roux, A., and Poole, C. (2002). Neighborhood characteristics associated 

with the location of food stores and food service places. American Journal of Preventive Medicine 22, 

23–29. 

Nunnally, J. (1978). Psychometric theory. New York: McGraw-Hill. 

Nusser, S.M., Carriquiry, A. L., Dodd, K. W., and Fuller, W. A. (1996). A semi-parametric 

transformation approach to estimating usual nutrient intake distributions. Journal of the American 

Statistical Association 91, 1440–1449. 

Oehlert, G.W. (1992). A note on the delta method. The American Statistician, 46(1), 27–29. 

Pomerleau, J., Lock, K., Knai, C., and McKee, M. (2005). Interventions designed to increase adult 

fruit and vegetable intake can be effective: A systematic review of the literature. The Journal of 

Nutrition 135(10), 2486–2495. 

Rose, D., and Richards, R. (2004). Food store access and household fruit and vegetable use among 

participants in the US Food Stamp Program. Public Health Nutrition 7(8), 1081–1088. 

Schochet, P. Z. (2008). Statistical power for random assignment evaluations of education programs. 

Journal of Educational and Behavioral Statistics 33(1), 62–87. 

Schochet, P. Z. (2009). An approach for addressing the multiple testing problem in social policy 

impact evaluations. Evaluation Review 33(6), 537–567. 

Shapiro, J. M. (2005). Is there a daily discount rate? Evidence from the food stamp nutrition cycle. 

Journal of Public Economics 89(2), 303–325. 

Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation 

models. Sociological Methodology, 13, 290–312. 

Sorensen, G., Stoddard, A. M., Dubowitz, T., Barbeau, E. M., Bigby, J., Emmons, K. M., Berkman, 

L. F., and Peterson, K. E. (2007). The influence of social context on changes in fruit and vegetable 

consumption: Results of the Healthy Directions Studies. American Journal of Public Health 97(7), 

1216–1227. 

Southworth H. M. (1945). The economics of public measures to subsidize food consumption. Journal 

of Farm Economics 27, 38–66. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 218 ▌References Abt Associates

Stewart, H., and Blisard, N. (2008). Are lower income households willing and able to budget for fruits 

and vegetables? (ERS Report No. 54). Alexandria, VA: U.S. Department of Agriculture, Economic 

Research Service. 

Stewart, H., Hyman, J., Buzby, J. C., Frazão, E., and Carlson, A. (2011). How much do fruits and 

vegetables cost? (EIB-71). Washington, DC: U.S. Department of Agriculture, Economic Research 

Service. 

Subar, A.F., Kipnis, V., Troiano, R.P., Midthune, D., Schoeller, D.A., Bingham, S., Sharbaugh, C.O., 

Trabulsi, J., Runswick, S., Ballard-Barbash, R., Sunshine, J., and Schatzkin, A. (2003). Using intake 

biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: The OPEN 

study. American Journal of Epidemiology 158, 1–13. 

Subar A. F., Dodd, K. W., Guenther, P. M., Kipnis, V., Midthune, D., McDowell, M., Tooze, J. A., 

Freedman, L. S., and Krebs-Smith, S. M. (2006). The food propensity questionnaire: concept, 

development, and validation for use as a covariate in a model to estimate usual food intake. Journal of 

the American Dietetic Association 106(10), 1556–1563. 

Thompson, F. E., Kipnis, V., Subar, A. F., Krebs-Smith, S. M., Kahle, L. L., Midthune, D., 

Potischman, N., and Schatzkin, A. (2000). Evaluation of 2 brief instruments and a food-frequency 

questionnaire to estimate daily number of servings of fruit and vegetables. American Journal of 

Clinical Nutrition 71(6), 1503–1510. 

Tooze, J. A., Midthune. D., Dodd, K. W., Freedman, L. S., Krebs-Smith, S. M., Subar, A. F., 

Guenther, P. M., Carroll, R. J., and Kipnis, V. (2006). A new statistical method for estimating the 

usual intake of episodically consumed foods with applications to their distribution. Journal of the 

American Dietetic Association 106(10), 1575–1587. 

U.S. Department of Agriculture and U.S. Department of Health and Human Services. (2010). Dietary 

guidelines for Americans, 2010 (7th ed.). Washington, DC: U.S. Government Printing Office. 

U.S. Department of Agriculture, Economic Research Service. (2009). Access to affordable and 

nutritious food: Measuring and understanding food deserts and their consequences, report to 

Congress. Washington, DC. Available at: www.ers.usda.gov/Publications/AP/AP036/. 

U.S. Department of Agriculture. (2010). Strategic Plan FY 2010-2015. Washington, DC: Author. 

Available at: http://www.ocfo.usda.gov/sp2010/sp2010.pdf.

U.S. Department of Agriculture, Agricultural Research Service. (2012). Nutrient intakes from food: 

mean amounts consumed per individual, by gender and age, what we eat in America, NHANES 2009-

2010. Washington, DC: Author. Available at: www.ars.usda.gov/ba/bhnrc/fsrg 

U.S. Department of Agriculture, Center for Nutrition Policy and Promotion. (2013a). Addendum to 

MyPyramid Equivalents Database, Version 2.0 [online document]. Alexandria, VA: Author. 

Available at: http://www.cnpp.usda.gov/OtherProjects.htm 

U.S. Department of Agriculture, Center for Nutrition Policy and Promotion. (2013b). Healthy Eating 

Index Support Files [online files]. Alexandria, VA: Author. Available at: 

http://www.cnpp.usda.gov/HealthyEatingIndexSupportFiles0102.htm 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates References ▌pg. 219

U.S. Department of Health and Human Services. (2010). Healthy people 2020 (2nd ed.). Washington, 

DC: U.S. Government Printing Office. 

Wilde, P. E., and Andrews, M. S. (2000). The food stamp program in an era of welfare reform: 

Electronic benefits and changing sources of cash income. Journal of Consumer Affairs 34(1), 31–46. 

Wilde, P.E., McNamara, P. E., and Ranney, C. K. (2000). The effect on dietary quality of 

participation in the Food Stamp and WIC Programs (Food Assistance and Nutrition Research Report 

No 9). Washington, DC: U.S. Department of Agriculture, Economic Research Service. 

Wilde, P. E., and Ranney, C. K. (2000). The monthly food stamp cycle: Shopping frequency and food 

intake decisions in an endogenous switching regression framework. American Journal of Agricultural 

Economics 82(1), 200–213. 

Wilde, P., Troy, L., and Rogers, B. (2009). Food stamps and food spending: An Engel function 

approach. American Journal of Agricultural Economics 91(2), 416. 

World Cancer Research Fund. (2007). Food, nutrition, physical activity, and the prevention of 

cancer: A global perspective. Washington DC: American Institute for Cancer Research. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix A: Economic Theory ▌pg. 221

Appendix A: Economic Theory 

Chapter 1 briefly presents a conceptual model for HIP (Exhibit 1.1). That discussion emphasized that 

HIP might work through two distinct pathways: (i) an economic pathway, i.e., the HIP incentive 

lowers the net price of TFVs; and (ii) an informational/attitudinal pathway, i.e., HIP’s educational and 

promotional materials, and/or the implicit messaging about the benefits of fruits and vegetables 

associated with provision of the incentive, might increase TFV intake, independent of the price effect. 

This appendix develops the argument for the “economic pathway” in more detail. The additional 

detail serves two purposes. First, it provides a benchmark of the likely impact of HIP through that 

pathway—assuming participants perfectly understood how the incentive worked and responded 

according to theory. Second, it helps us to understand that the observed HIP/non-HIP difference in 

TFV expenditures in participating IECR supermarkets/superstores is probably an upper bound on 

(i.e., greater than) the total increase in TFV expenditure that can be attributed purely to the economic 

pathway. 

This appendix proceeds in six sections. Section A.1 presents a basic elasticity analysis. The 

subsequent sections then consider how relaxing the assumptions of the basic elasticity analysis affects 

the conclusions. Section A.2 presents a more complete analysis incorporating potential income effects 

based on published Consumer Expenditure Survey data. Section A.3 considers the implications of the 

fact that some TFVs are paid for with cash—and that these purchases do not earn HIP incentives. 

Section A.4 considers the implications of the fact that some fruits and vegetables (prepared foods, 

white potatoes, fruit juices, mature legumes) are not covered by HIP. Section A.5 considers the 

implications of the fact that—measured by SNAP purchases—only about half of all retailers 

participated in HIP. Section A.6 considers the relation between the HIP/non-HIP difference in TFV 

expenditure in participating IECR retailers and the total increase in TFV expenditure. 

A.1 A Basic Elasticity Analysis 

Some of the motivation for HIP comes from the simple observation of economic theory that “demand 

curves slope down”; i.e., lowering the price will increase purchases (see Frazao et al., 2007; Dong and 

Lin, 2009; Guthrie et al., 2007; Kuchler, Tegene, and Harris, 2005; GAO, 2008). The magnitude of 

the decline is an empirical question which is usually represented by an “elasticity,” which describes 

the percentage change in purchases with a percentage change in price. 

Andreyeva, Long, and Brownell (2010) survey the literature on fruit and vegetable elasticities. They 

identified 20 estimates of the elasticity of fruit purchases; the mean estimate across those 20 studies 

was 0.70, with a range of 0.16 to 3.02. They also identified 20 estimates of the elasticity of vegetable 

purchases, with a mean of 0.58 with a range of 0.2 to 1.11. They also explore subgroup impacts, but 

find no consistent evidence that impacts vary by income or program participation. Dong and Lin 

(2009) provide elasticity estimates and a literature review that is broadly consistent with the 

Andreyeva et al. figures. 

A simple extrapolation to HIP might proceed as follows. The simple average of the mean estimates 

for fruits and vegetables is very close to two-thirds (i.e., 0.64 vs. 0.67). HIP provides a 30 percent 

incentive for TFV purchases, so in rounded terms we would expect HIP’s financial incentive to 

increase TFV spending by roughly 20 percent. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 222 ▌ Appendix A: Economic Theory Abt Associates

A.2 Incorporating Income Effects 

The simple elasticity analysis assumes that the increase in TFV spending is entirely attributable to a 

pure price or substitution effect, where the participant increases HIP-eligible purchases in direct 

response to the price differential. If additional income generated by incentive earnings is sufficiently 

high, however, TFV spending could also increase simply because the household has more income to 

spend. Since the 30 percent change in price could plausibly generate a fairly substantial earned 

incentive amount, in this section we extend the previous analysis to take into account additional 

increases in TFV spending that might be attributable to an income effect. 

Our data for this exercise come from the Consumer Expenditure Survey (CEX).140 Taking CEX data 

for the lowest quintile of the income distribution as a rough proxy for the SNAP population and 

converting to a monthly basis suggests total household monthly expenditures of $1,833, total food 

expenditures of $296 (16 percent of total expenditures), and total fruit and vegetable expenditures of 

$37 (2 percent of total expenditures and 13 percent of food expenditures). 

If the household understands the incentive and accordingly increases fruit and vegetable expenditures, 

elasticities from Andreyeva, Long, and Brownell (2010) as described above imply an increase of 

approximately $7 (20 percent of $37) due to the price effect alone, implying total fruit and vegetable 

spending of $44, and associated monthly incentive earnings of about $13. Note that this amount 

represents an upper bound relative to the case when some fruit and vegetable purchases do not in fact 

qualify to earn the HIP incentive. 

The additional incentive income would be deposited into the household’s SNAP EBT account. How 

the household would then spend the additional income is the subject of debate in the economics 

literature (the “Southworth Hypothesis”), but for our purposes it is sufficient to note that the fraction 

of the incentive that would be spent on fruit and vegetables is somewhere between the fraction of all 

income spent on fruit and vegetables (i.e., about 2 percent) and the fraction of food expenditures spent 

on fruit and vegetables (i.e., about 13 percent); or between well under a dollar and slightly less than 

two dollars, on top of the $7 increase attributable for the pure price effect. As an upper bound, then, 

we would expect HIP impacts of no more than $9 per month in a typical low-income household, if 

HIP operates only through economic pathways (price and income effects).141 These impacts are small, 

under half a percent of baseline income. 

 

140 The analysis here is drawn directly from published tabulations for 2011. See 

http://www.bls.gov/cex/csxstnd.htm.

141 Formally, this argument is about the “compensated elasticity.” Andreyeva, Long, and Brownell (2010) 

report “uncompensated elasticities.” The Slutsky Equation implies that the compensated elasticity equals 

the uncompensated elasticity plus the product of the share of fruit and vegetable consumption (2 percent) 

and the income elasticity of fruit and vegetable consumption. Tabulations from the CEX suggest that the 

income elasticity is small. Specifically, between the first and second quintiles, income increases from 

$22,001 to $32,092; i.e., 45 percent. Between those same quintiles, fruit and vegetable consumption 

increases from $448 to $556; i.e., 24 percent. We can roughly approximate the income elasticity (i.e., the 

percentage change in fruit and vegetable consumption with a percentage change in income) as the ratio of 

these two figures: 0.53=24 percent/45 percent. So, if the uncompensated elasticity (from Andreyeva, Long, 

and Brownell, 2010) is approximately 0.67, then the compensated elasticity is 0.68—a trivial difference. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix A: Economic Theory ▌pg. 223

A.3 Cash vs. SNAP 

HIP only applies to purchases made with SNAP. The previous analysis implicitly assumes that all 

fruit and vegetable purchases are made at the lower net price—i.e., that they are made with SNAP. 

The SNAP benefit formula, in contrast, assumes that households with income will use some cash to 

purchase food. 

Nevertheless, the previous analysis is approximately correct. HIP gives households an incentive to 

use SNAP to pay for fruits and vegetables. Thus, households that previously used cash to pay for 

some fruits and vegetables can simply shift the cash to non-fruits and vegetables, freeing up SNAP 

for fruits and vegetables.142 Assuming households understand and appropriately react to HIP, then we 

can (approximately) treat its likely impacts as if all fruit and vegetable expenditure was made with 

SNAP (rather than cash). The previous analysis therefore stands as described. 

A.4 Incomplete Coverage of Fruits and Vegetables 

The previous analysis implicitly assumes that the HIP incentive applies to everything classified as 

part of “fruit and vegetable” expenditures by the CEX. In fact, HIP does not apply to prepared foods 

or to white potatoes, 100% fruit juice, or mature legumes. In the lowest quintile, total fruit and 

vegetable expenditures ($448 per year) are 17 percent processed fruits ($76) and 20 percent processed 

vegetables ($90). From the CEX, we do not have an estimate of dollar expenditures on white potatoes 

and legumes. Nevertheless, it seems plausible that less than half of all CEX fruits and vegetables 

actually qualify for the HIP incentive. Under this assumption, all of the estimates in the previous 

section should be halved; that is, a price elasticity of two-thirds implies an increase in TFV spending 

of about $4-5, and monthly earned incentives of about $7. 

A.5 Incomplete Retailer Participation 

The previous analysis also implicitly assumes that all retailers participate in HIP. In fact, when 

measured by SNAP expenditure, only about half of all SNAP retailers participated (see the discussion 

in Chapter 3). How would we expect HIP households to behave given incomplete retailer 

participation? 

A simplifying assumption would be to treat household retailer choice as fixed. In that case, 

households that shop in participating retailers would face and react to the lower net price. Households 

that shop in non-participating retailers would not face the lower net price; we would expect their 

expenditures to be unchanged. 

However, the assumption that retailer choice is fixed is arguably incorrect. HIP gives households an 

incentive to change where they shop—at least where they shop for fruits and vegetables. The previous 

analysis suggests that a household that used to shop in a non-participating retailer could earn another 

$7 per month simply by switching to a participating retailer, but not changing what was bought. 

 

142 This discussion implicitly assumes that the SNAP benefit is greater than fruit and vegetable spending. 

Fruits and vegetables are a sufficiently small share of food expenditure that fruit and vegetable 

expenditures greater than the total SNAP benefit will be unusual. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 224 ▌ Appendix A: Economic Theory Abt Associates

Whether switching retailers was worthwhile would involve balancing the incentives that might be 

earned against the costs—including, but not limited to, out of pocket expenses and time—for 

switching retailers. It seems plausible that some households would do so, while others would not. For 

households already shopping at both participating and non-participating retailers, the costs of 

switching TFV purchases to participating retailers would be lower. This suggests that the appropriate 

estimate of the increase in TFVs and incentives earned that can be attributed to the pure financial 

pathway is somewhere between the earlier estimate and half the earlier estimate; i.e., $2 to $3 for 

increased TFV expenditure and $3 to $7 for incentive earned. 

A.6 Measured TFV Purchases 

The evaluation has available EBT transactions data from participating IECR retailers enabling us to 

measure TFV purchases for both HIP and non-HIP households. It is thus tempting to estimate the 

impact of HIP on TFV purchases as the difference between purchases by HIP and non-HIP 

households, since randomization implies that the households are otherwise identical. In practice, 

proprietary data considerations cause us to restrict our analysis to purchases in participating IECR 

supermarkets and superstores, excluding other, smaller, retailers such as groceries and convenience 

stores. 

We discuss the strategy of using the difference in TFV purchases in participating supermarkets and 

superstores as an approximation to the impact of HIP on all TFV purchases in detail in Section 6.1. 

The obvious—but not directly addressable—difficulty with this strategy is that some TFV 

expenditures are not captured; i.e., expenditures (i) with cash (ii) in non-participating retailers (iii) in 

participating non-IECR retailers; and/or (iv) in participating IECR retailers that are not supermarkets 

or superstores. The third (in participating non-IECR retailers) and fourth (non-supermarket or 

superstore IECR retailers) sets of expenditures are unlikely to be a major issue. Those expenditures 

represent less than 5 percent of all expenditures at participating retailers. 

The other two expenditure types are more problematic. As noted in Sections A.3 and A.5, HIP gives 

households a motivation to shift expenditures from cash and non-participating retailers to SNAP EBT 

at participating retailers. By doing so, households earn the incentive. Any such shift to participating 

retailers is likely to be overwhelmingly to IECR participating retailers. Thus, the HIP/non-HIP 

difference in TFV expenditure at participating IECR retailers is likely to be an over-estimate of the 

true difference in TFV expenditure that can be attributed purely to the financial pathway. 

If such a shift had occurred, we would expect to see a decline in expenditures at non-participating 

retailers. At the very least, households would shift their TFV purchases from non-participating to 

participating retailers. Fixed costs of shopping might imply that they would also move (at least) some 

of their non-TFV purchases (see Section A.5). However, the analysis in Section 6.3 finds no evidence 

of such a decrease. It therefore seems plausible to assume that this source of over-estimate is small, 

and thus the bound on the HIP impact that can be attributed just to the economic pathway is relatively 

tight. 

In practice, Chapter 6 shows that the magnitude of the estimated impact on TFV expenditures in 

participating IECR supermarkets and superstores is broadly consistent with the theoretical analysis 

described above. However, Chapter 8 shows that the HIP impact on TFV intake appeared to be much 

greater than can be explained by the pure economic pathway described and analyzed in this appendix. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix A: Economic Theory ▌pg. 225

Section 8.2 compares relevant HIP impacts on food spending and food intake, and Chapter 10 

discusses several possible explanations. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix B: Random Assignment and Sampling ▌pg. 227

Appendix B: Random Assignment and Sampling 

A rigorous research design was critical to assessing the impact of HIP on participants’ intake of fruits 

and vegetables. Random assignment was used to determine which SNAP households in Hampden 

County would participate in HIP. Then, random sampling was used to select respondents for the 

participant surveys. (Exhibit 2.1 in Chapter 2 provides a graphical overview of the evaluation design.) 

The first section of this appendix discusses random assignment to HIP/non-HIP status. The second 

section of this appendix discusses random sampling of participant survey respondents. 

B.1 Random Assignment to HIP/non-HIP Status 

The random assignment of eligible SNAP participant households to HIP and non-HIP status was 

central to the evaluation design and HIP operations. DTA provided administrative case file records 

containing all households and persons on SNAP in Hampden County as of mid-July, 2011.143 We 

randomly selected 7,500 SNAP households (containing 9,286 persons) to participate in HIP. The 

remaining 47,595 eligible SNAP households in Hampden (containing 59,652 persons) were not 

selected to earn the HIP incentive. 

To ensure that the HIP (treatment) and non-HIP (control) groups were balanced or similar, we used a 

blocked random assignment design. Tests on the samples after random assignment confirmed that 

they were similar with respect to key participant characteristics. Additionally, HIP households were 

randomly divided into three groups, corresponding to the three waves DTA established to enroll 

participants in HIP.144 Non-HIP households were also divided into three waves to facilitate participant 

survey sampling. 

The rest of this section provides a detailed description of the steps in the random assignment process. 

Step 1: Review Data 

To conduct random assignment, we used the July 2011 extract of households in Hampden County, 

Massachusetts that were eligible for HIP. We reviewed the file and the related documentation to 

ensure that all the eligible households, and only the eligible households, were included in the file. The 

evaluation sought to estimate the impact of HIP on adult food intake, so child-only cases were not 

eligible for HIP. Furthermore, only households that did their own shopping were eligible for the 

evaluation. SNAP participants who sign over their benefits to a residential or treatment facility were 

not eligible. Homeless participants who retained the use of their own benefits remained eligible for 

the study. DTA excluded ineligible households prior to providing the file. 

We verified that the exclusion criteria applied to the extract was correct by reviewing the SQL query 

DTA used to extract the data and by verifying through tabulations that the cases to be excluded were 

not included in the file. We verified that all households in the file extract had: 

 

143 Once DTA generated this Hampden County SNAP case extract file of HIP-eligible households, these 

households remained in the HIP evaluation sample for the duration of the pilot as long as they continued to 

participate in SNAP; no additional households were added to the pilot. 

144 DTA staggered enrollment in HIP over three months for ease of implementation. Prior to each wave’s start 

date, HIP households received several mailings describing HIP. See Chapter 3 for additional details. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 228 ▌Appendix B: Random Assignment and Sampling Abt Associates

 Active SNAP cases in July 2011 

 Residential or mailing addresses in Hampden County 

 An active grantee (head of household) 

 At least one active member 16 years of age or older 

 Benefit amount greater than $0 in June 2011 

We also verified (using the ‘residential facility type code’ or res_type) that none of the households in 

the file extract were in the following excluded residential facilities: 

 Approved Public Medical Institution 

 Assessment Center 

 Licensed Chronic Hospital 

 Educational Residential Facility 

 Hospital 

 Licensed Intermediate Care Facility 

 Penal Institution 

 Licensed Residential Care Facility 

 Long Term Care Facility 

 Public Nonmedical Institution 

 Private Psychiatric Residential Facility 

 Public Psychiatric Residential Facility 

 Residential Inpatient Treatment Center 

Step 2: Create Blocking Variables and Per Block Sample Size Determination 

We created 12 household-level blocking cells defined by completely cross-classifying the following 

three variables: 

 Geography (3 levels): Springfield; Chicopee/Holyoke; and remainder of Hampden County; 

 Household size (2 levels): 1-person and 2-or-more-persons; and 

 Gender for head of household (2 levels): male-headed and female-headed. 

Next, we calculated the proportion (Ph) of total households in each of the (h = 1 to 12) blocking cells, 

i.e. Ph = Yh/YT where Yh is the number of households in the blocking cell h and YT is total number of 

households. This proportion is the “cell size.”145

Then, we determined the number of households to select for HIP in each cell. First, we calculated mh

= 7,500 * Ph. The total number of households to select for HIP across all cells would be 7,500 (i.e. 

 

145 We planned to combine small blocking cells of cell size less than .01 and recalculate Ph if necessary. As 

Exhibit B.1 shows, none of the cell sizes were this small, so this step was not necessary. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix B: Random Assignment and Sampling ▌pg. 229

∑mh = 7,500). However, since mh’s would be in decimals, we used stochastic rounding146 to obtain 

integer cell sample sizes, nh. 

The source file contained 55,095 HIP-eligible households. For each of the 12 blocking cells, Exhibit 

B.1 presents the total number of households (Yh), the proportion of households in each cell (ph), the 

unrounded number of households to select from each cell (mh), and the stochastically rounded number 

of households to select (nh). The total number of households to be selected was exactly 7,500. 147

Exhibit B.1: Description of the Blocking Cells 

Blocking cells Yh ph mh nh

Springfield, HH Size 2+, Male Head 1,153 0.0209 157.0 157 

Springfield, HH Size 2+, Female Head 11,608 0.2107 1,580.2 1,580 

Springfield, HH Size 1, Male Head 8,651 0.1570 1,177.6 1,177 

Springfield, HH Size 1, Female Head 7,580 0.1376 1,031.9 1,032 

Chicopee & Holyoke, HH Size 2+, Male Head 591 0.0107 80.5 81 

Chicopee & Holyoke, HH Size 2+, Female Head 5,578 0.1012 759.3 759 

Chicopee & Holyoke, HH Size 1, Male Head 3,762 0.0683 512.1 512 

Chicopee & Holyoke, HH Size 1, Female Head 3,702 0.0672 503.9 504 

Hampden Balance, HH Size 2+, Male Head 998 0.0181 135.9 136 

Hampden Balance, HH Size 2+, Female Head 4,366 0.0792 594.3 595 

Hampden Balance, HH Size 1, Male Head 3,333 0.0605 453.7 454 

Hampden Balance, HH Size 1, Female Head 3,773 0.0685 513.6 513 

All Blocking Cells 55,095 1.0000 7,500.0 7,500 

Step 3: Random Assignment and Proportion Verification 

We randomly selected nh households from each blocking cell h (e.g. Springfield, household size 1, 

female household head) to be in HIP. The remaining households were put in the non-HIP group. 

Thus, within blocks, we have simple random sampling without replacement which we implemented in 

SAS using PROC SURVEYSELECT. 

The result of the random assignment by blocking cells is presented in Exhibit B.2. As would be 

expected if random assignment was properly conducted, the percentages of HIP and non-HIP 

households in each of the blocking cells are virtually identical to each other and the cell proportions, 

ph, from Exhibit B.1 above (also replicated as the “All” column). This indicates that proportions by 

blocking cells were computed and applied correctly during random assignment. 

 

146 We rounded the decimal sample size mh to integer sample size nh by rounding it up or down randomly. If 

the number we drew randomly from a uniform distribution on the interval [0, 1] was less than the decimal 

component of mh, then mh was rounded down to nh. But if the random number was equal or greater to the 

decimal component mh, then mh was rounded up to nh. 

147 Due to stochastic rounding of blocking cells, the actual HIP households could be slightly different than 

7,500. In a test simulation of 1,000 runs, the sample size ranged from 7,497 to 7,503. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 230 ▌Appendix B: Random Assignment and Sampling Abt Associates

Exhibit B.2: Blocking Cell Shares of HIP and Non-HIP Households

Blocking cells 

HIP Non-HIP All 

N % N % N % 

Springfield, HH Size 2+, Male Head 157 2.1 996 2.1 1,153 2.1 

Springfield, HH Size 2+, Female Head 1,580 21.1 10,028 21.1 11,608 21.1 

Springfield, HH Size 1, Male Head 1,177 15.7 7,474 15.7 8,651 15.7 

Springfield, HH Size 1, Female Head 1,032 13.8 6,548 13.8 7,580 13.8 

Chicopee & Holyoke, HH Size 2+, Male Head 81 1.1 510 1.1 591 1.1 

Chicopee & Holyoke, HH Size 2+, Female Head 759 10.1 4,819 10.1 5,578 10.1 

Chicopee & Holyoke, HH Size 1, Male Head 512 6.8 3,250 6.8 3,762 6.8 

Chicopee & Holyoke, HH Size 1, Female Head 504 6.7 3,198 6.7 3,702 6.7 

Hampden Balance, HH Size 2+, Male Head 136 1.8 862 1.8 998 1.8 

Hampden Balance, HH Size 2+, Female Head 595 7.9 3,771 7.9 4,366 7.9 

Hampden Balance, HH Size 1, Male Head 454 6.1 2,879 6.0 3,333 6.0 

Hampden Balance, HH Size 1, Female Head 513 6.8 3,260 6.8 3,773 6.8 

All Blocking Cells 7,500 100.0 47,595 100.0 55,095 100.0 

Step 4: Balance Test and Sample Rejection 

We tested balance on baseline characteristics between the HIP and non-HIP groups using variables 

provided in the case file extract. We used a robust global F-test on a linear regression148 as the 

primary test for systematic differences between the HIP and non-HIP households. We tested on the 

following variables: 

 Monthly SNAP Benefit (4 categories);

 Spanish Language Flag (2 categories);

 Recertification Type (3 categories);

 Monthly Income (4 categories);

 Bay State CAP Flag (2 categories); 

 Homeless Status Flag (2 categories); 

 Residence Type (3 categories);

 Age of Household Head (4 categories); 

 Race/Ethnicity of Household Head (4 categories); 

 Disability Flag (2 categories);

 U.S. Citizenship Flag (2 categories); 

 TANF/AFDC Flag (2 categories);

 Unearned Income Flag (2 categories); 

 SSI Flag (2 categories);

 RSDI Flag (2 categories); 

 

148 We regressed a 0/1 indictor for non-HIP/HIP status on all the selected baseline characteristics (as binary or 

categorical variables) as well as the dummy variables indicating the blocking cells. Reference groups were 

omitted for variables with multiple categories, e.g. monthly SNAP benefit $1-$161 was excluded in the 

linear regression, while the rest of the SNAP benefit categories were included. The global F-test for the null 

hypothesis that the coefficients of all the baseline characteristics (excluding the dummy variables for the 

blocking cells) are zero was assessed at the 20 percent significance level to determine if there was a less 

than 20 percent chance that the groups were produced by a process in which all baseline characteristics are 

unrelated to non-HIP/HIP status. See the body of the appendix for a discussion of the 20 percent level. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix B: Random Assignment and Sampling ▌pg. 231

 Unemployment Compensation Flag (2 categories); and 

 Household Type (3 categories). 

The purpose of this test was to determine if a chance “bad draw” was obtained from the random 

selection process, not to judge whether the process was in fact random; our knowledge of the 

randomization mechanism and the results of a previously conducted simulation of test randomizations 

lead us to infer that the procedure was random. A chance “bad draw” does not bias the random 

assignment result but it does create differences in baseline characteristics that needlessly increase the 

variance of the impact estimates to be produced by the study. 

We planned to discard the selected sample if the global F-test failed (i.e. p-value < 0.20). Then we 

would select a replacement sample until an acceptable sample was obtained. We only expected 1 in 5 

samples to fail this test and verified this using a simulation of 1,000 samples. This broader rejection 

standard (compared to a standard of p-value < .05 or p < .10) reflects the desire to more closely match 

the two groups on baseline characteristics than would be accomplished by the usual standards. More 

readily rejecting groups produced by initial tries assures that the eventual accepted groups more 

closely match on the examined characteristics. 

Exhibit B.3, presents the results of these tests. The p-value of 0.536 for the global F-statistic, shown 

at the bottom of the exhibit, leads us to conclude that there is no evidence of systematic differences 

between the HIP and non-HIP households in this sample and accept the sample as the basis for the 

experiment. 

As shown in Exhibit B.3, we also conducted individual t-tests on the 35 variables representing the 

baseline characteristics for the sample. With so many individual t-tests, a few are bound to appear 

statistically significant just by chance.149 Therefore the t-tests were not used to accept (or discard) the 

sample but only for diagnostic purposes. In the sample drawn, there were no statistically significant 

differences between HIP and non-HIP households for any of the 35 characteristics tested. In fact, the 

smallest individual p-value for the t-test is 0.104 for the $1-$787 category of monthly income. Thus, 

these individual tests also provide no evidence of lack of randomness. 

 

149 For example, if the outcomes were uncorrelated (which they are not), we would expect 1 in 20 t-tests to be 

statistically significant just by chance at the significance level of 5 percent. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 232 ▌Appendix B: Random Assignment and Sampling Abt Associates

Exhibit B.3: Balance Test

Household characteristics HIP (%) Non-HIP (%) P-value 

N 7,500 47,595 

Monthly SNAP benefit 

$1-$160 25.7 25.0 0.201 

$161–$200 39.3 39.6 0.631 

$201–$349 10.8 10.4 0.254 

$350 + 24.3 25.1 0.123 

Spanish language 22.2 22.2 0.925 

Recertification type 

Recertification 53.0 52.4 0.315 

Semiannual reporting 32.0 32.2 0.708 

Other reevaluation 15.0 15.4 0.364 

Monthly income 

$0 23.4 24.0 0.321 

$1–$787 26.5 25.7 0.104 

$788–$1,083 25.3 25.3 0.941 

$1,084 + 24.7 25.1 0.459 

Baystate CAP 7.9 8.0 0.903 

Homeless 6.8 6.8 0.990 

Housing type 

Private 80.0 80.5 0.369 

Public 13.9 13.5 0.358 

Other 6.0 6.0 0.860 

Household head age 

16–30 28.1 27.6 0.388 

31–40 21.4 21.2 0.700 

41–54 25.9 26.6 0.243 

55 + 24.6 24.6 0.946 

Household head race/ethnicity 

Hispanic 43.3 43.6 0.722 

White 36.5 36.7 0.714 

Black 13.3 12.9 0.325 

Other 6.9 6.8 0.930 

Disabled 49.6 49.6 0.974 

US citizen 95.7 95.9 0.370 

TANF/AFDC 13.4 13.4 0.905 

Unearned income 60.2 59.5 0.241 

SSI 32.4 32.5 0.823 

RSDI 27.3 27.5 0.774 

Unemployment compensation 5.0 4.6 0.167 

Household type 

Household with elderly 12.1 12.5 0.352 

Household with children (no elderly) 36.5 36.3 0.632 

Other household 51.4 51.3 0.879 

F statistic 0.95 0.536
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix B: Random Assignment and Sampling ▌pg. 233

Step 5: Divide Samples into 3 Waves 

Having drawn a final sample, the HIP and non-HIP groups were randomly divided into 3 waves for 

HIP implementation and then for the fielding of the survey. As is discussed in Chapter 3, issuing and 

activating HIP was staggered; one wave of participants began HIP each month over the period 

November 2011 through January 2012. 

B.2 Random Sampling for Participant Survey 

The survey samples of respondents, equally distributed between the HIP and non-HIP household 

groups, were selected using a stratified random sampling design. The stratification variables were the 

same ones used in the random assignment of households to the HIP and non-HIP groups. Among the 

HIP and non-HIP groups, an equal number of respondents were selected from each of the three 

waves. Sampled respondents were aged 16 and older and only one respondent was selected per 

household.150

We randomly sampled 2,538 SNAP participants from both the HIP and non-HIP households to 

participate in the Round 1 survey (i.e., a total of 5,076). This sample size was chosen so that a large 

enough sample would remain for Rounds 2 and 3 to achieve the desired level of precision after 

accounting for participants who left SNAP, and thus were ineligible for the survey, or who became 

non-respondents in the later rounds. The target sample was designed to be able to detect a postimplementation HIP/non-HIP difference in targeted fruit and vegetable intake of 0.25 cup-equivalents 

of fruits and vegetables per day.151 

The rest of the section describes the process for sampling persons for the participant surveys. 

Create Sampling Frame 

Using the DTA administrative case file data, we created two files for sampling purposes: (1) a 

household level file of eligible SNAP households in Hampden County Massachusetts that had been 

randomly assigned to HIP and non-HIP groups (referred to as the “AU file”); and (2) the 

corresponding person-level file of household members 16 years of age or older in the eligible SNAP 

households (referred to as the “AP file”). 

Three key variables were used to select the respondent sample: 

1. HIP_IND: This household-level variable contained the preassigned codes for the treatment 

status of the SNAP household (H = HIP; K = non-HIP). 

2. BLOCK: This variable designated the 12 household-level blocking cells defined by a crossclassification of three levels of geography, two levels of household size, and two levels of 

gender of head of household (as defined in the previous section). 

 

150 Approximately 6 percent of sampled respondents were aged 16-17 (as of the Round 2 survey). This group 

was included in the sample as they can be SNAP heads of households and the sample was intended to 

represent all types of households. 

151 Based on assumptions provided in FNS’s RFP and discussed in Abt’s proposal. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 234 ▌Appendix B: Random Assignment and Sampling Abt Associates

3. WAVE: This variable was created as part of the random assignment process and 

corresponded to the three waves DTA established to enroll participants in HIP (November 1, 

2011, December 1, 2011, and January 1, 2012, respectively). 

Exhibit B.4 presents the numbers of households and persons in the HIP evaluation sampling frames 

by treatment status and size of household. Exhibit B.5 summarizes the number persons in the 

sampling frames by treatment status, blocking cell, and wave. As expected, based on the random 

assignment of households discussed in the previous section, the populations of persons within 

households in the sampling frames were well balanced with respect to the blocking cells across the 

different treatment groups and waves. 

Exhibit B.4: Frequencies of Households and Persons in the HIP and Non-HIP Sampling 

Frames by Size of Household 

# persons 

at least 16 

years old in HH 

HIP Non-HIP 

Households Persons Households Persons 

No. Percent 

(%) 

No. Percent

(%) 

No. Percent

(%) 

No. Percent

(%) 

1 6,054 80.72 6,054 65.19 38,107 80.07 38,107 63.88 

2 1,167 15.56 2,334 25.13 7,479 15.71 14,958 25.08 

3 230 3.07 690 7.43 1,559 3.28 4,677 7.84 

4 39 0.52 156 1.68 361 0.76 1,444 2.42 

5 8 0.11 40 0.43 75 0.16 375 0.63 

6 2 0.03 12 0.13 9 0.02 54 0.09 

7 0 0.00 0 0.00 3 0.01 21 0.04 

8 0 0.00 0 0.00 2 <0.01 16 0.03 

Total 7,500 100.00 9,286 100.00 47,595 100.00 59,652 100.00 

Exhibit B.5: Frequencies of Persons in the HIP and Non-HIP Sampling Frames by Block and 

Wave 

Blocking Cells 

HIP Non-HIP 

Wave 1 Wave 2 Wave 3 Wave 1 Wave 2 Wave 3 

Springfield, HH Size 2+, Male Head 93 98 93 635 613 641 

Springfield, HH Size 2+, Female Head 781 764 787 5,040 5,054 5,081 

Springfield, HH Size 1, Male Head 392 393 392 2,491 2,491 2,492 

Springfield, HH Size 1, Female Head 344 344 344 2,183 2,183 2,182 

Chicopee & Holyoke, HH Size 2+, Male Head 50 47 51 338 322 322 

Chicopee & Holyoke, HH Size 2+, Female 

Head 366 370 364 2,424 2,403 2,410 

Chicopee & Holyoke, HH Size 1, Male Head 171 170 171 1,084 1,083 1,083 

Chicopee & Holyoke, HH Size 1, Female Head 168 168 168 1,066 1,066 1,066 

Hampden Balance, HH Size 2+, Male Head 88 95 89 605 567 572 

Hampden Balance, HH Size 2+, Female Head 316 319 323 2,040 1,964 2,012 

Hampden Balance, HH Size 1, Male Head 151 151 152 960 959 960 

Hampden Balance, HH Size 1, Female Head 171 171 171 1,086 1,087 1,087 

Total 3,091 3,090 3,105 19,952 19,792 19,908 

We prepared the sampling frame for sample selection in the following steps: 

1. Random number assigned to each household (RAND). We generated and assigned a 

uniform random number between 0 and 1 to each household in the AU file. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix B: Random Assignment and Sampling ▌pg. 235

2. Household-type variable created (HH_TYP). At the same time, we created a household 

level variable that classified households into three types: households with persons 65 years or 

older; households with children 5 years or younger but no one 65 or older; and all other 

households. 

3. Person level file created. We merged the two extract files to create a person-level file 

containing all of the household-level variables in the AU file. 

4. Number of adults in household variable created (NUMADLT34). We created this 

dichotomous variable, indicating whether the household had fewer than four adults in the 

household or four or more adults in the household. We used this variable to adjust the sample 

sizes in order to achieve better control of the sample sizes across the treatment groups. 

5. Person-level file split into two sampling frames (HIP_IND). We divided the person level 

file created in step 3 into two sampling frames—HIP and non-HIP. 

6. Records in each frame sorted. Prior to sampling, we sorted the records in each of the two 

sampling frames as follows: 

 by WAVE 

 by BLOCK within WAVE 

 by NUMADLT34 within BLOCK 

 by HH_TYP within NUMADLT34 

 by RAND within HH_TYPE 

 by AUID (unique household identifier) within RAND. 

We used HH_TYP and NUMADLT34 as sorting variables within the primary strata 

defined for sampling (i.e., the primary strata defined by BLOCK and WAVE) to achieve 

better balance with respect to household composition across the various waves. 

Sorting by AUID ensured that persons in the same household were listed together in the 

final sorted file. We used the variable WAVE as a stratification variable to permit the 

selection of equal numbers of sampled persons for each wave. 

Select Round 1 Participant Survey Sample 

Because of the small size of the HIP sampling frame, the desired sampling rate for the HIP sample 

was about 1 in 3.7, compared to 1 in 23 in the non-HIP sample. Therefore, it was possible to select

more than one eligible person in HIP households consisting of four or more adults. To avoid this 

possibility, we lowered the sampling rates for households with four or more adults in the HIP sample 

and to achieve approximately equal sample sizes by size of household for the two treatment groups, 

made a corresponding downward adjustment in sampling rates in non-HIP households with four or 

more persons. 

For the HIP sample we deviated slightly from equal-probability systematic sampling by (1) selecting 

households with 4 or more adults with certainty and then randomly selecting 1 person per household 

and (2) applying a small compensatory increase in the probability of selection in households with less 

than 4 adults in order to achieve the planned sample size of 2,538. For the non-HIP sample we 

selected approximately the same number of non-HIP persons in each household size class (less than 4 

adults and more than 4 adults) as was previously noted for the HIP sample. Exhibit B.6 summarizes 

the intended sample sizes resulting from these adjustments. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 236 ▌Appendix B: Random Assignment and Sampling Abt Associates

Exhibit B.6: Intended Frequencies of Households and Persons in the HIP and Non-HIP 

Samples by Size of Household 

# persons at 

least 16 years 

old in HH 

HIP Non-HIP 

Households Persons Households Persons 

No. 

Percent

(%) No. 

Percent

(%) No. 

Percent

(%) No. 

Percent

(%) 

1 1,667 65.68 1,667 65.68 1,632 64.30 1,632 64.30 

2 637 25.10 637 25.10 638 25.14 638 25.14 

3 185 7.29 185 7.29 213 8.39 213 8.39 

4 39 1.54 39 1.54 45 1.77 45 1.77 

5 8 0.32 8 0.32 9 0.35 9 0.35 

6 2 0.08 2 0.08 1 0.04 1 0.04 

7 0 0.00 0 0.00 0 0.00 0 0.00 

8 0 0.00 0 0.00 0 0.00 0 0.00 

Total 2,538 100.00 2,538 100.00 2,538 100.00 2,538 100.00 

Actual sample sizes will vary from these numbers. 

Exhibit B.7 summarizes the distribution of the evaluation sample by treatment status, wave, and 

block. Despite the adjustments made in sampling to balance sample sizes, the sample sizes shown in 

this table are specified by design and are roughly proportional to the corresponding population counts 

shown in Exhibit B.5. 

Exhibit B.7: Counts of Evaluation Sample by Treatment Status, Wave, and Block 

Blocking Cells 

HIP Non-HIP 

Wave 1 Wave 2 Wave 3 Wave 1 Wave 2 Wave 3 

Springfield, HH Size 2+, Male Head 27 27 27 27 27 27 

Springfield, HH Size 2+, Female Head 215 215 215 215 215 215 

Springfield, HH Size 1, Male Head 106 106 106 106 106 106 

Springfield, HH Size 1, Female Head 93 93 93 93 93 93 

Chicopee & Holyoke, HH Size 2+, Male Head 14 14 14 14 14 14 

Chicopee & Holyoke, HH Size 2+, Female 

Head 103 103 103 103 103 103 

Chicopee & Holyoke, HH Size 1, Male Head 46 46 46 46 46 46 

Chicopee & Holyoke, HH Size 1, Female 

Head 45 45 45 45 45 45 

Hampden Balance, HH Size 2+, Male Head 26 26 26 26 26 26 

Hampden Balance, HH Size 2+, Female 

Head 84 84 84 84 84 84 

Hampden Balance, HH Size 1, Male Head 41 41 41 41 41 41 

Hampden Balance, HH Size 1, Female Head 46 46 46 46 46 46 

Total 846 846 846 846 846 846 

After completion of participant sampling, we assigned each household a HIP flag or indicator 

identifying it as belonging to one of the following four groups: 

 HIP household, non-survey group (Group H) 

 HIP household, survey group (Group I) 

 Non-HIP household, survey group (Group J) 

 Non-HIP household, non-survey group (Group K) 

We applied these flags to all members of the original households in the HIP evaluation sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix B: Random Assignment and Sampling ▌pg. 237

Select Round 2 Participant Survey Sample 

Eligible sample cases at Round 2 were defined as those who completed a Round 1 survey and were 

SNAP participants at the time of the interview.152 To ensure the sample was on SNAP at the time of 

the interview, we compared each wave to the previous month’s DTA SNAP file immediately before 

releasing it to determine if any cases had exited SNAP. We removed such cases before releasing the 

sample. We also excluded cases if the case file indicated that the respondent was institutionalized or 

did not meet the “following rules” determined by DTA. The SNAP case was tied to the head of 

household and therefore, the HIP status and the HIP incentives were also tied to this individual..153 

Exhibit B.8 summarizes the numbers of respondents initially included in the Round 2 sampling frame 

based on the January, 2012 DTA file, by HIP treatment status, blocking group, and wave. Of the 

1,388 Round 1 respondents in the HIP group, 88 percent were still active in January 2012 and eligible 

to participate in Round 2. Of the 1,396 Round 1 respondents in the non-HIP group, 89 percent were 

still active and eligible to participate in January 2012.154 

 

152 Chapter 2, Section 2.11 discusses the reason for excluding cases no longer on SNAP. 

153 If the original head of household (HoH) left the SNAP household, DTA closed that SNAP case. Other 

household members could form a new case, but that new case did not get the HIP flag and thus did not earn 

HIP incentives even if its prior flag was a HIP case. Similarly, if a member of a HIP household other than 

the original HoH left the household, that person was not given a HIP flag and was not eligible to earn HIP 

incentives. In such cases, the household with the original HoH retained the HIP flag and HIP incentives. 

The SNAP case also could close without any changes in household composition. Regardless of how a 

SNAP case closed, if the SNAP case reopened with the original HoH, the household once again received 

the HIP flag and resumed earning HIP incentives. 

154 During data collection, we discovered that some additional respondents were “out of scope” because, for 

example, they moved out of state, were institutionalized, or were deceased. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 238 ▌Appendix B: Random Assignment and Sampling Abt Associates

Exhibit B.8: Number of Round 1 Respondents Sampled for Round 2 after Deleting 

Households no Longer on SNAP 

Blocking group 

HIP Non-HIP 

Grand 

total 

Wave 

Total 

Wave 

1 2 3 1 2 3 Total 

Springfield, HH Size 1, Female 

Head 51 52 55 158 55 57 44 156 314 

Springfield, HH Size 1, Male 

Head 41 52 34 127 40 49 42 131 258 

Springfield, HH Size 2+, 

Female Head 101 119 96 316 110 131 97 338 654 

Springfield, HH Size 2+, Male 

Head 11 13 7 31 14 14 11 39 70 

Chicopee/Holyoke HH Size 1, 

Female Head 26 19 27 72 24 31 24 79 151 

Chicopee/Holyoke HH Size 1, 

Male Head 19 21 20 60 17 21 15 53 113 

Chicopee/Holyoke HH Size 

2+, Female Head 51 66 44 161 53 66 49 168 329 

Chicopee/Holyoke HH Size 

2+, Male Head 7 8 10 25 5 4 8 17 42 

Hampden Balance, HH Size 1, 

Female Head 26 24 25 75 17 27 14 58 133 

 Hampden Balance, HH Size 

1, Male Head 15 20 14 49 20 16 15 51 100 

 Hampden Balance, HH Size 

2+, Female Head 36 43 37 116 47 43 33 123 239 

 Hampden Balance, HH Size 

2+, Male Head 13 13 10 36 12 9 9 30 66 

Total 397 450 379 1226 414 468 361 1243 2469 

Select Round 3 Participant Survey Sample 

We selected the Round 3 sample from respondents who completed a Round 2 survey. At the end of 

Round 2, a total of 2,006 respondents had completed the AMPM interview and another 44 had 

completed the Respondent interview (but not the AMPM interview), for a total of 2,050 cases. Of 

these 2,050 cases, 157 were no longer study eligible cases according to the DTA case record files as 

they were either not receiving SNAP or did not meet the “following rules” (described in the previous 

section). We deleted these cases from the sample. Thus, the total number of cases fielded in Round 3 

was 1,893. 

This approach of only including cases in the Round 3 sampling frame if they had completed a Round 

2 survey was based on two factors. First, it was unlikely that we would be able to find and interview 

many of the households who did not complete the Round 2 interview. The Round 3 interviews began 

soon after the Round 2 interviews were completed. We expended considerable effort to locate 

households in Round 2 and did not expect we would be any more successful locating them in Round 

3. Second, our approach resulted in a true longitudinal data set, since all Round 3 completes also had 

a Round 2 interivew. 

Exhibit B.9 presents the Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix B: Random Assignment and Sampling ▌pg. 239

Exhibit B.9: Number of Round 2 Respondents Sampled for Round 3 after Deleting 

Households no Longer on SNAP 

Blocking Group 

HIP Non-HIP 

Grand 

Total 

Wave 

Total 

Wave 

1 2 3 1 2 3 Total 

1. Springfield, HH Size 1, Female 

Head 42 46 43 131 46 45 33 124 255 

2. Springfield, HH Size 1, Male 

Head 34 42 24 100 32 44 28 104 204 

3. Springfield, HH Size 2+, Female 

Head 80 90 70 240 87 106 71 264 504 

4. Springfield, HH Size 2+, Male 

Head 7 11 6 24 13 11 7 31 55 

5. Chicopee/Holyoke HH Size 1, 

Female Head 25 16 24 65 18 25 19 62 127 

6. Chicopee/Holyoke HH Size 1, 

Male Head 15 15 16 46 11 13 12 36 82 

7. Chicopee/Holyoke HH Size 2+, 

Female Head 38 53 30 121 43 53 32 128 249 

8. Chicopee/Holyoke HH Size 2+, 

Male Head 5 7 8 20 2 3 5 10 30 

9. Hampden Balance, HH Size 1, 

Female Head 26 20 18 64 15 24 11 50 114 

10. Hampden Balance, HH Size 1, 

Male Head 9 18 8 35 13 13 8 34 69 

11. Hampden Balance, HH Size 2+, 

Female Head 30 33 20 83 34 30 19 83 166 

12. Hampden Balance, HH Size 2+, 

Male Head 8 7 5 20 7 4 7 18 38 

Total 319 358 272 949 322 373 255 944 1893 

Summary of Sample Sizes 

Exhibit B.10 summarizes the sample flow for the three rounds of the participant survey for the HIP 

and non-HIP groups separately. As shown, 2,538 participants in each group were initially sampled for 

Round 1. We assumed there would be 20.3 percent attrition in each group through the course of 

Round 1 data collection based on published national SNAP exit rates (Cody et al., 2007).155 However, 

as shown, the attrition rate combined with cases that we determined were out of scope during data 

collection (i.e., deceased, institutionalized, out of state) resulted in 13 percent of the sample in the 

HIP Group and 14 percent in the non-HIP Group being ineligible for survey participation. As 

described above, all respondents that completed Round 1 were included in the Round 2 frame. 

Attrition and out of scope cases in Round 2 were similar to that in Round 1. All Round 2 completed 

cases were released for Round 3. 

 

155 Massachusetts statewide exit rates provided by the Massachusetts DTA are broadly similar to the national 

rates. While analysis done by DTA indicated exit rates in Hampden County were less than national SNAP 

exit rates and also less than Massachusetts statewide exit rates, we did not change our estimated sample 

sizes because we assumed a higher sample size would allow us to achieve the desired number of completed 

interviews in Rounds 2 and 3.
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 240 ▌Appendix B: Random Assignment and Sampling Abt Associates

Exhibit B.10: Actual Sample Sizes, by HIP and Non-HIP Status 

HIP Group 

Non-HIP Group 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix C: Weighting Methodology ▌pg. 241

Appendix C: Weighting Methodology 

This appendix summarizes the procedures used to weight the person-level survey data collected in the 

Healthy Incentives Pilot (HIP) evaluation; it also provides a non-response bias analysis for Round 1 

of the participant survey. Survey weights are required to project the sample results to population 

levels. We therefore computed weights for the completed cases in the sample at the end of each data 

collection round. These weights are designed to (a) take account of the varying probabilities with 

which persons were selected for the study and (b) compensate for differential rates of survey nonresponse. Non-response adjustments were calculated to reflect the fact that non-response could occur 

either prior to or after ascertaining eligibility for the survey. Additional details of the weighting 

methodology can be found in the separate volume, Evaluation of the Healthy Incentives Pilot (HIP) 

Final Report, Technical Appendix: Participant Survey Weighting Methodology (Chu, 2014). 

For each survey round, sampled-person weights were constructed for analysis of the sampled person 

interviews. A parallel set of primary-shopper weights were constructed for the primary shopper 

interviews. For many household-level variables, the primary-shopper weights serve as household 

weights, because there is only one primary shopper per household, and the corresponding questions 

appeared on the primary shopper portion of the survey. In addition to the two sets of full-sample 

weights, a series of replicate weights using a jackknife method was constructed for variance 

estimation purposes. 

Specifically, the first three sections of this appendix discuss construction of weights for Round 1, 

Round 2, and Round 3, respectively. Because only cases interviewed at Round 1 were interviewed at 

Round 2 and only cases interviewed at Round 2 were interviewed at Round 3, there is no need for 

separate longitudinal weights. The Round 2 weights apply to longitudinal analyses of Round 1 and 

Round 2; the Round 3 weights apply to longitudinal analyses of Round 2 and Round 3 or all three 

waves. The fourth section provides a non-response bias analysis for Round 1 of the participant 

survey. 

C.1 Round 1 

This section describes the construction of Round 1 weights. The section begins with a discussion of 

wave-specific base weights and pooled base weights. It then describes ratio adjustments and nonresponse adjustments. 

Wave-Specific Base Weights 

Under the stratified sampling design employed for the HIP evaluation, the probability of selecting an 

eligible individual for the study depended on the (randomly-assigned) wave of data collection 

(corresponding to the three waves of implementation), “blocking groups” defined by location and 

selected characteristics of households, and on the size of the household expressed in terms of the 

number of eligible adults residing in the household. For brevity, we refer to each blocking group-bysize category combination as a “stratum.” The wave-specific base weight for person i in stratum s in 

wave v was computed as: 

௜௩௦ݓ

௕௔௦௘ = 1/ܲ௩௦ (1) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 242 ▌Appendix C: Weighting Methodology Abt Associates

where ܲ௩௦ = the probability of selecting persons in stratum s and wave v (v = 1, 2, 3). This probability 

generally equals the number of adults sampled in a given wave and stratum divided by the 

corresponding number of adults in the sampling frame. 

Pooled Base Weights 

The base weights defined by formula (1) are appropriate for analysis of each individual wave of data 

collection. To analyze the combined sample, the wave-specific base weights were adjusted to take 

account of differences in coverage by wave, and permit unbiased estimation based on all three waves 

of data. 

The goal of this step of the weighting process was to adjust the wave-specific base weights in a 

manner that minimized the variation of the overall combined-sample weights (also referred to as 

“pooled” or “composite” weights), while at the same time providing unbiased weights for the 

combined sample. 

These pooled base weights were created in two steps. First, the wave-specific base weights were 

scaled up or down by wave-specific scaling factors designed to align the resulting weighted sample 

counts to known population counts. That is, a rescaled base weight for the ith sample person in wave 

v and subgroup g was computed as: 

௜௚௩ݓ 

௔ௗ௝ = ܵ௩௚ ݓ௩௚௜

௕௔௦௘ , (2) 

where ܵ௩௚ is the appropriate wave-specific scaling factor. Next, approximately optimal composite 

estimation factors, ܣ௩௚, designed to minimize the variation of the resulting combined-sample weights 

were applied to the wave-specific adjusted weights to obtain the pooled weights, ݓ௩௚௜

௣௢௢௟, as follows: 

௜௚௩ݓ௜௚௩ݓ ௚௩ܣ = ௟௢௢௣

௔ௗ௝ . (3) 

Ratio Adjustment of the Pooled Weights 

Although the composite weights defined in formula (3) are theoretically unbiased, the corresponding 

weighted counts are subject to sampling variability, and consequently do not always match known 

population counts by blocking group. Therefore, a ratio adjustment was applied to the pooled weights 

so that weighted counts of the sample agreed with the corresponding population (frame) counts for 

each of 12 blocking groups. Using appropriate post-stratification factors, ܨ௦

ሺ௣௦ሻ, the final poststratified 

pooled weight was computed as: 

௜௦ݓ

௣௦ = ܨ௦

ሺ௣௦ሻ ݓ௦௜

௣௢௢௟ (4) 

Non-Response Adjustment 

The final step in the weighting process for Round 1 was to adjust the post-stratified base weights 

defined by formula (4) to compensate for varying rates of non-response in the baseline survey. Since 

non-response could have occurred either (1) prior to determining eligibility (e.g., the sampled person 

could not be contacted or located); or (2) after determining eligibility (e.g., the person was located 

and eligibility was determined), the non-response adjustment was done in two phases. 

The purpose of the first-phase adjustment was to distribute a portion of the weighted count of the 

unknown-eligibility cases to those cases for which eligibility was ascertained. A CHAID analysis 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix C: Weighting Methodology ▌pg. 243

(Chi Square Automatic Interaction Detector) was conducted separately for each treatment group to 

identify cells within which the predicted probabilities of ascertaining eligibility were similar. The 

results of the CHAID analysis were used to define the cells (labeled r = 1, 2, ..., R) for the first-phase 

non-response adjustment. The weighted first-phase response rates varied from around 50 percent to 

over 95 percent across the final adjustment cells. The first-phase non-response adjustment factor, ܣ ௥, 

was computed as the inverse of the weighted first-phase response rate in final cell r. The first-phase 

adjusted weight for the ith sampled person in cell r for whom eligibility was determined was 

computed as: 

௜௥ݓ

௜௥ݓ ௥ܣ = ேோଵ

௣௦ (5) 

The purpose of the second-phase adjustment was to distribute the weighted count of the knowneligible non-respondents to the Round 1 respondents. The results of a CHAID analysis were used to 

defined the cells (labeled s = 1, 2, ..., S) for the second-phase non-response adjustment. The secondphase non-response adjustment factor, ܤ௦ , was computed as the inverse of the weighted secondphase response rate in final cell s. The final non-response-adjusted weight for the ith responding 

person in cell s (i.e., cases in response status group 1) was then computed as: 

௜௦ݓ

ேோଶ = ܤ௦ ݓ௦௜ேோଵ (6) 

C.2 Round 2 

This section discusses the construction of Round 2 weights. It first considers the initial weights and 

then the non-response adjustment. Finally, construction of second-day 24-hour dietary recall 

interview weights is discussed. 

Initial Weights 

The Round 2 sampling weights are constructed based on the set of final non-response-adjusted person 

௜௥ݓ ,weights

ோ௢௨௡ௗ	ଵ, developed for analysis of respondents in the baseline survey (see Section C.1). 

These weights were designed to provide for substantially unbiased estimation of the characteristics of 

SNAP beneficiaries (by treatment group) in Hampden County, Massachusetts, who were active 

participants in the July 2011 case files provided by the Massachusetts DTA, and who remained 

eligible through the end of Round 1 data collection. 

Since all of the still-eligible Round 1 respondents were carried over into (i.e., “sampled” for) Round 

2, the final weights from Round 1 are the “initial” weights for Round 2. The “initial” weights are 

adjusted for non-response experienced in Round 2, as described in the following section. 

Non-Response Adjustment 

As in Round 1, non-response could have occurred either (1) prior to determining eligibility (e.g., the 

sampled person could not be contacted or located); or (2) after determining eligibility (e.g., the person 

was located and eligibility was determined). Therefore, as in Round 1, the non-response adjustment 

was done in two phases. 

In the first phase of adjustment, a portion of the weighted count of the unknown-eligibility cases was 

distributed to the known-eligible cases. As in Round 1, CHAID was used to identify cells within 

which the predicted probabilities of ascertaining eligibility were similar. In addition to the 

classification variables used previously to weight the Round 1 sample, selected variables (responses) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 244 ▌Appendix C: Weighting Methodology Abt Associates

from the Round 1 interview were used as possible independent (predictor) variables in the CHAID 

analysis. For both HIP and non-HIP samples, the weighted (conditional) first-phase response rates 

were high, varying from around 87 percent to 100 percent over the final adjustment cells. The firstphase non-response adjustment factor, ܣ ௥, was computed as the inverse of the weighted first-phase 

response rate in final cell r. The (intermediate) first-phase adjusted weight for the ith sampled person 

in cell r for whom eligibility was determined was computed as: 

௜௥ݓ

௜௥ݓ ௥ܣ = ேோଵ

ோ௢௨௡ௗ	ଵ (7) 

The purpose of the second-phase adjustment was to distribute the weighted count of the knowneligible non-respondents in Round 2 to the Round 2 respondents. For the second-phase adjustment, 

the same set of independent variables used previously for the first-phase adjustment were specified as 

possible independent variables in a CHAID analysis. The output from the CHAID analysis was used 

to define the second-phase non-response-adjustment weighting cells (denoted by the subscript s = 1, 

2, ..., S). The second-phase non-response adjustment factor, ܤ௦ , was computed as the inverse of the 

weighted second-phase response rate in final cell s. The final non-response-adjusted weight for the ith 

responding person in cell s was then computed as: 

௜௦ݓ

ோ௢௨௡ௗ	ଶ = ܤ௦ ݓ௦௜

ேோଵ (8) 

Second-Day Intake Weights 

Approximately 10 percent of the Round 2 respondents completing the first 24-hour dietary recall 

interview also completed a second 24-hour dietary recall interview. Weights for analysis of the 

second intake were constructed by applying appropriate inflation factors to the final weights 

previously created for the first intake interview. Note that the second-day intake weights apply to 

those respondents that completed both the first and second intake interviews. In Round 2, three cases 

that completed the second intake did not have corresponding Day 1 intake data. These cases were 

excluded from the weighting process. 

C.3 Round 3 

This section discusses the construction of Round 3 weights. It first describes the initial weights, then 

the non-response adjustment and the second-day 24-hour dietary recall interview weights. The 

discussion is similar to that for Round 2. No new issues are raised. 

Initial Weights 

The Round 3 sampling weights are constructed based on the set of final non-response-adjusted person 

௜௥ݓ ,weights

ோ௢௨௡ௗ	ଶ, developed for analysis of respondents in the Round 2 survey (see Section C.2). 

These weights were designed to provide for substantially unbiased estimation of the characteristics of 

SNAP beneficiaries in Hampden County, Massachusetts, who were active participants in the July 

2011 case files provided by the Massachusetts DTA, and who remained eligible through the end of 

Round 2 data collection. Since all of the still-eligible Round 2 respondents were carried over into 

(i.e., “sampled” for) Round 3, the final weights from Round 2 are the “initial” weights for Round 3 

weighting. The “initial” weights are adjusted for non-response experienced in Round 3, as described 

in the following section. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix C: Weighting Methodology ▌pg. 245

Non-Response Adjustment 

Again, we used a two-phase adjustment for non-response. In the first phase of adjustment, a portion 

of the weighted count of the unknown-eligibility cases was distributed to the known-eligible cases. As 

in prior rounds, CHAID was used to identify cells within which the predicted probabilities of 

ascertaining eligibility were similar. In addition to the variables used previously to weight the Round 

2 sample, selected variables (responses) from the Round 2 interview were used as possible 

independent (predictor) variables in the CHAID analysis. For both HIP and non-HIP samples, the 

weighted (conditional) first-phase response rates were high, varying from around 75 percent to 100 

percent over the final adjustment cells. The first-phase non-response adjustment factor, ܣ ௥, was 

computed as the inverse of the weighted first-phase response rate in final cell r. The (intermediate) 

first-phase adjusted weight for the ith sampled person in cell r for whom eligibility was determined 

was computed as: 

௜௥ݓ

௜௥ݓ ௥ܣ = ேோଵ

ோ௢௨௡ௗ	ଶ (9) 

The purpose of the second-phase adjustment was to distribute the weighted count of the knowneligible non-respondents in Round 3 to the Round 3 respondents. For the second-phase adjustment, 

the same set of independent variables used previously for the first-phase adjustment were specified as 

possible independent variables in a CHAID analysis. The output from the CHAID analysis was used 

to define the second-phase non-response-adjustment weighting cells (denoted by the subscript s = 1, 

2, ..., S). The second-phase non-response adjustment factor, ܤ௦ , was computed as the inverse of the 

weighted second-phase response rate in final cell s. The final non-response-adjusted weight for the ith 

responding person in cell s was then computed as: 

௜௦ݓ

ோ௢௨௡ௗ	ଷ = ܤ௦ ݓ௦௜

ேோଵ . (10) 

Second-Day Intake Weights 

Approximately 10 percent of the Round 3 respondents completing the first 24-hour dietary recall 

interview also completed a second 24-hour dietary recall interview. Weights for analysis of the 

second intake were constructed by applying appropriate inflation factors to the final weights 

previously created for the first intake interview. Note that the second-day intake weights apply to 

those respondents that completed both the first and second intake interviews. In Round 3, one case 

that completed the second intake did not have corresponding Day 1 intake data. This case was 

excluded from the weighting process. 

C4. Non-Response Bias Analysis 

As specified in the Standards and Guidelines for Statistical Surveys published by the Office and 

Management and Budget (September 2006)156, a non-response bias analysis is required if the overall 

unit response rate for a survey is less than 80 percent (Guideline 3.2.9). As summarized in Exhibit 

C.1, the overall unconditional (weighted) response rates for Round 1 of the SNAP participant surveys 

were 62 percent for the HIP group and 63 percent for the non-HIP group. For Round 2, the roundspecific weighted response rate (conditional on completing Round 1) was 84 percent for the HIP 

group and 82 percent for the non-HIP group. The corresponding unconditional response rates were 52 

 

156 http://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 246 ▌Appendix C: Weighting Methodology Abt Associates

percent for both HIP and non-HIP groups. For Round 3, the round-specific response rate (conditional 

on completing Round 2) was 82 percent for the HIP group and 80 percent for the non-HIP group; 

unconditional response rates were 42 percent for the HIP group and 41 percent for the non-HIP group. 

The various components of the response rates achieved in the HIP evaluation study are summarized in 

Exhibit C.1 by round and treatment status. Although both unweighted and weighted response rates are 

shown, the weighted response rates are featured prominently in this discussion because they are used 

to adjust the sampling weights to compensate for differential rates of non-response. 

Exhibit C.1: Response Rates by Round and Treatment Status 

Unweighted Weighteda

Round Component HIP 

NonHIP HIP NonHIP 

1 Phase 1 (prior to eligibility determination) 85.2 85.8 84.8 85.4 

Phase 2 (after eligibility determination) 74.0 74.5 73.1 73.5 

Round 1 response rate (unconditional) 63.0 64.0 62.0 62.8 

2 Phase 1 (prior to eligibility determination) 97.6 97.5 97.4 97.2 

Phase 2 (after eligibility determination) 86.2 84.6 86.1 84.5 

Round 2 response rate (conditional on completing 

Round 1) 84.1 82.5 83.8 82.1 

Unconditional Round 2 response rate (Round 

1*Round 2 conditional) 53.0 52.8 51.9 51.5 

3 Phase 1 (prior to eligibility determination) 96.9 96.3 96.7 96.3 

Phase 2 (after eligibility determination) 84.3 83.4 84.3 83.2 

Round 3 response rate (conditional on completing 

Rounds 1 & 2) 81.7 80.3 81.5 80.2 

Unconditional Round 3 response rate (Round 2 

unconditional*Round 3 conditional) 43.3 42.4 42.3 41.3 

a For Round 1, weights are the (post-stratified) base weights. For Round 2, weights are the non-responseadjusted Round 1 weights. For Round 3, weights are the non-response-adjusted Round 2 weights. 

Below we summarize the findings of an analysis of non-response in each of the three rounds of the 

SNAP participant surveys. The main goals of the analysis are to: (1) document the variation in 

response rates for selected subsets of the sample (2) evaluate the extent to which the final (nonresponse adjusted) sampling weights developed for analysis may be effective in countering the effects 

of the differential response rates on weighted distributions of the sample; and (3) assess the impact 

the differential response rates may have on estimates derived from the survey. 

Analysis of Non-Response in Round 1 

This section provides a non-response analysis for Round 1. The findings reported here are based on 

the detailed exhibits and analyses presented in Chu (2014). 

Response Rates by Selected Characteristics (Round 1) 

To examine the extent to which missing data resulting from non-response were “missing at random,” 

we calculated response rates for subsets of the sample based on selected characteristics available in 

the sampling frame. We performed these calculations separately for the two phases where nonresponse could occur: (1) prior to determining eligibility (e.g., the sampled person could not be 

contacted or located); or (2) after determining eligibility (e.g., the sampled person was located and 

eligibility was determined, but the person did not complete in the survey). These characteristics 

include both household-level characteristics (e.g., size of household, presence of children or elderly, 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix C: Weighting Methodology ▌pg. 247

housing type, amount of SNAP benefit, income category, and others), and selected person-level 

characteristics (e.g., age, sex, race/ethnicity, disability status, and others). 

Within the HIP sample, the first-phase response rates were found to vary significantly (using RaoScott chi-square test) by location, wave of sample release, monthly SNAP benefit, monthly income, 

homeless status, housing type, age of household head, race/ethnicity of household head, citizenship 

status of household head and of sampled respondent, household type, gender, age of sampled person, 

race/ethnicity of sampled person, and household size. Many of these same variables were also 

significantly associated with response status for the non-HIP sample with some exceptions (e.g., 

unlike the HIP sample, response rates for the non-HIP sample did not vary significantly by location or 

citizenship status of household head). 

For the HIP sample, the second phase response rates varied significantly for nine of the characteristics 

included in the analysis (Exhibit C.2), compared with 15 characteristics for the corresponding firstphase response rates. For the non-HIP sample, the second phase response rates varied significantly for 

eight of the characteristics (Exhibit C.2), compared with 12 characteristics for the corresponding firstphase response rates. 

Comparison of Respondents and Non-respondents (Round 1) 

The overall weighted response rate is the product of the first- and second-phase response rate. To 

examine the combined effect of the first- and second-phase non-response on weighted distributions of 

the sample, we compared the (unadjusted) base-weighted distributions of the respondents and nonrespondents for the selected characteristics. Overall, there were significant differences (based on tests 

of association between response status and each of the characteristics using Rao-Scott chi-square test) 

between the distributions of the respondents and non-respondents for nine of the characteristics for 

the HIP sample, and nine of the characteristics for the non-HIP sample, with wave of sample release, 

citizenship status, and gender common to both treatment groups. For both the HIP and non-HIP 

samples, relatively more persons in the respondent sample were in waves 2 and 3(corresponding to 

months 2 and 3 of survey fielding) than in the total sample and relatively fewer persons in the 

respondent sample were in wave 1 than in the total sample, reflecting the generally lower response 

rates achieved in wave 1. Similarly, the percentage of females in the respondent sample is higher than 

the percentage in the total sample for both HIP and non-HIP samples, indicating the generally higher 

response rates achieved for females. For those characteristics for which there are appreciable 

distributional differences between the respondents and the total sample, estimates for survey items 

that are correlated with these characteristics can potentially be biased unless weights are constructed 

to compensate for these differences. 

As described in section C.1, adjustments were made to the (poststratified) base weights to compensate 

for any distributional differences resulting from differential response rates. These non-responseadjusted weights are the final weights used to derive the survey-based estimates from Round 1. While 

significant differences were observed for many characteristics prior to non-response adjustment, as 

expected, after non-response adjustment, the differences for all of these characteristics have 

essentially disappeared. In other words, for both HIP and non-HIP samples, the non-response 

adjustments used to develop the final weights for analysis are effective in realigning the weighted 

distributions of the respondent sample to the corresponding distributions of the total (selected) sample 

prior to losses resulting from non-response. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 248 ▌Appendix C: Weighting Methodology Abt Associates

Comparisons Before and After Non-Response Adjustments for Selected Survey Results 

(Round 1) 

The final set of comparisons conducted in the non-response bias analysis for Round 1 involved a 

comparison of weighted estimates of a limited number of survey items using the base weights and 

non-response-adjusted weights. The items chosen from the baseline survey included a few categorical 

variables related to opinions about enjoyment and accessibility of fruits and vegetables, and a few 

numeric variables related to the number of times certain fruits or vegetables were reported to have 

been consumed. Among the 18 statistics considered in the analysis, the difference between the 

unadjusted and non-response-adjusted estimates is generally small and differed significantly (using 

Rao-Scott chi-square test for categorical variables and tests reflecting the complex sample design for 

numeric variables) for only two of the items reported by the HIP sample and for only one item 

reported by the non-HIP sample. Despite the similarity of the estimates for both HIP and non-HIP 

samples, the potential for bias exists, and use of the non-response-adjusted weights to analyze the 

survey data is expected to help reduce biases that may occur for statistics not considered in this 

analysis. 

Non-Response Bias Analysis for Round 2 

This section provides a non-response analysis for Round 2. The findings reported here are based on 

the detailed exhibits and analyses presented in Chu (2014). 

Response Rates by Selected Characteristics (Round 2) 

Within the HIP sample, the first-phase response rates were found to vary significantly by age and 

race/ethnicity of the household head, and by gender, age, and race/ethnicity of the sampled person. 

Within the non-HIP sample, the first-phase response rates were found to vary significantly by wave of 

sample release, monthly income, homeless status, housing type, age of head of household, Social 

Security status, and age of the sampled person. 

For the HIP sample, the second phase response rates varied significantly for four of the characteristics 

listed in the table (Exhibit C.2), compared with five characteristics for the corresponding first-phase 

response rates. Similarly, for the non-HIP sample, the second phase response rates varied 

significantly for only two of the characteristics (Exhibit C.2), compared with seven characteristics for 

the corresponding first-phase response rates. 

Comparison of Respondents and Non-Respondents (Round 2) 

To examine the combined effect of the first- and second phase non-response on weighted 

distributions of the sample, we compared the weighted distributions of the respondents and nonrespondents for the selected characteristics. The weights used here are the final non-response-adjusted 

weights from Round 1 which act as “base weights” in this analysis. Overall, there were significant 

differences between the distributions of the respondents and non-respondents for four of the 

characteristics (two of which relate to citizenship status and are highly correlated) for the HIP sample, 

and only one of the characteristics for the non-HIP sample. For the HIP sample, relatively fewer 

persons in the respondent sample were semiannual reporters, noncitizens, and non-Social Security 

participants compared with the total sample. For the non-HIP sample, relatively more persons in the 

respondent sample were in wave 2 than in the total sample. The small number of significant 

differences in Round 2 suggests that much of the variation in response rates in Round 2 had been 

accounted for in the weighting adjustments from Round 1. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix C: Weighting Methodology ▌pg. 249

As described in section C.2, adjustments were made to the non-response-adjusted weights from 

Round 1 to compensate for any distributional differences resulting from differential response rates in 

Round 2. These non-response-adjusted weights (referred to as the final Round 2 weights) are the 

weights used to derive the survey-based estimates from Round 2. Somewhat surprisingly, for the HIP 

sample, Social Security status and the two citizenship variables remained significant after nonresponse adjustment; however, there seems to be little practical difference between the postadjustment and unadjusted distributions despite the statistical significance. Recertification type, 

which was highly significant prior to adjustment, was no longer significant after non-response 

adjustment. For the non-HIP sample, none of the variables considered were significant after nonresponse adjustment. 

Comparisons Before and After Non-Response Adjustments for Selected Survey Results 

(Round 2) 

The final set of comparisons conducted in the non-response bias analysis for Round 2 involved a 

comparison of weighted estimates of a limited number of survey items using the final weights from 

the Round 1 and the corresponding non-response-adjusted weights developed for Round 2. The items 

chosen from the Round 2 surveys were the same as for Round 1 (i.e., a few categorical variables 

related to opinions about enjoyment and accessibility of fruits and vegetables and a few numeric 

variables related to the number of times certain fruits or vegetables were reported to have been 

consumed) as well as selected intake variables from the AMPM. Among the 28 statistics considered 

in this analysis, the difference between the unadjusted and non-response-adjusted estimates is 

generally small and differed significantly for only four of the items reported by the HIP sample and 

for none of the items reported by the non-HIP sample. The similarity of the estimates suggests that for 

many of the variables collected in the Round 2 survey, including many of the nutrient items derived 

from the AMPM, estimates may not be affected appreciably by the level of non-response experienced 

in Round 2 of the study. However, the potential for bias exists, and use of the non-response-adjusted 

weights to analyze the survey/AMPM data may help reduce biases that may occur for statistics not 

considered in this analysis. 

Non-Response Bias Analysis for Round 3 

This section provides a non-response analysis for Round 3. The findings reported here are based on 

the detailed exhibits and analyses presented in Chu (2014). 

Response Rates by Selected Characteristics (Round 3) 

Within the HIP sample in Round 3, the first-phase response rates were found to vary significantly by 

location, wave of sample release, Social Security status, and age of the sampled person. For non-HIP 

sample, the first-phase response rates were found to vary significantly only by SSI status. 

For the HIP sample, the second-phase response rates varied significantly for only four of the 

characteristics considered (Exhibit C.2). For the non-HIP sample, the second phase response rates 

varied significantly only by citizenship status (Exhibit C.2). 

Comparison of Respondents and Non-Respondents by Selected Characteristics (Round 3) 

To examine the combined effect of the first- and second phase non-response on weighted 

distributions of the sample, we compared the weighted distributions of the respondents and nonrespondents for the selected characteristics. The weights used here are the final non-response-adjusted 

weights from Round 2 which act as “base weights” in this analysis. Overall, there were significant 

differences between the distributions of the respondents and non-respondents for only four of the 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 250 ▌Appendix C: Weighting Methodology Abt Associates

characteristics for the HIP sample, and the citizenship status variable(s) for the non-HIP sample. For 

the HIP sample, relatively more persons in the respondent sample resided in Chicopee/Holyoke, were 

homeless, received Social Security, or received unemployment compensation than in the total sample. 

For the non-HIP sample, relatively more persons in the respondent sample were US citizens than in 

the total sample. The small number of significant differences in Round 3 suggests that much of the 

variation in response rates in Round 3 may have been accounted for in the weighting adjustments 

from Rounds 1 and 2. 

As described in section C.3, adjustments were made to the non-response-adjusted weights from 

Round 2 to compensate for any distributional differences resulting from differential response rates in 

Round 3. These non-response-adjusted weights (referred to as the final Round 3 weights) are the 

weights used to derive the survey-based estimates from Round 3. Although a small number of 

differences remained statistically significant after non-response adjustment, there was little practical 

difference between the unadjusted and post-adjustment estimates for the vast majority of 

characteristics included in the analysis. 

Comparisons Before and After Non-Response Adjustments for Selected Survey Results 

(Round 3) 

The final set of comparisons conducted in the non-response bias analysis for Round 3 involved a 

comparison of weighted estimates of a limited number of survey items using the final weights from 

Round 2 and the corresponding non-response-adjusted weights developed for Round 3. The items 

chosen from the Round 3 surveys were the same as those chosen for Round 2 (a few categorical 

variables related to opinions about enjoyment and accessibility of fruits and vegetables, a few 

numeric variables related to the number of times certain fruits or vegetables were reported to have 

been consumed, and selected intake variables from the AMPM). Among the 28 statistics considered 

in the analysis, the difference between the unadjusted and non-response-adjusted estimates is 

generally small and differed significantly for four of the items reported by the HIP sample and for two 

items reported by the non-HIP sample. All of the statistically significant results were for items 

pertaining to opinions and the number of times certain foods had been consumed. None of the 

differences between the unadjusted and adjusted estimates of the mean nutrient intakes from the 

AMPM were significantly different. While this could indicate that estimates may not be affected 

appreciably by the level of non-response experienced in Round 3 of the study, the potential for bias 

exists, and use of the non-response-adjusted weights to analyze the survey/AMPM data may help 

reduce biases that may occur for statistics not considered in this analysis. 

Summary and Conclusions 

The overall weighted response rate for the HIP evaluation samples in Round 1 was 62.0 percent for 

the HIP group and 62.8 percent for the non-HIP group (Exhibit C.2). For Round 2, the overall 

weighted (conditional) response rates were 83.8 percent for the HIP group and 82.1 percent for the 

non-HIP group. For Round 3, the overall weighted (conditional) response rates were 81.5 percent for 

the HIP group and 80.2 percent for the non-HIP group. 

For the HIP sample, response rates varied significantly by wave of sample release, disability status of 

household head, citizenship status of household head, unearned income status, and other 

characteristics. For the non-HIP sample, response rates varied significantly by location, wave of 

sample release, race/ethnicity of household head, citizenship status of household head, TANF/AFDC 

status, and other characteristics. To compensate for the differential survey response rates in each 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix C: Weighting Methodology ▌pg. 251

round, weight adjustments were developed and used to derive final round-specific weights using a 

CHAID analysis to identify appropriate weight adjustment classes. In general, such weight 

adjustments will reduce non-response bias if the variables used in forming the weight adjustment 

classes are correlated with response propensity (the probability that a sampled person will respond to 

the survey) and with the characteristics obtained from the survey. 

There are reasons to believe that the non-response-adjusted weights developed for the HIP evaluation 

surveys will be reasonably effective in reducing potential biases. First, the weight adjustments 

removed virtually all of the disparities between the weighted distributions of the respondents and the 

corresponding distributions of the total sample. Second, we compared unadjusted and adjusted 

estimates for a limited number of items collected in all three surveys, and found significant 

differences in only a small number of instances, suggesting a potential for bias reductions when the 

non-response-adjusted weights are used in analysis. Short of conducting a comprehensive follow-up 

study of the non-respondents, there is no direct way of assessing the potential biases arising from 

survey non-response. The types of indirect analyses conducted in this evaluation do suggest, however, 

that non-response biases can be reduced to some extent through the use of the non-response-adjusted 

weights developed for this study. 

Exhibit C.2: Weighted Response Rates by Round, Treatment Status, and Selected 

Characteristics 

 Round 1 Round 2b Round 3b

Characteristica HIP NonHIP HIP Non-HIP HIP non-HIP 

Total sample 61.95 62.76 83.83 82.08 81.53 80.15

Location *** ** 

Springfield 60.96 62.87 84.53 82.74 80.90 79.77 

Chicopee/Holyoke 63.34 66.52 84.37 82.94 83.54 80.49 

Balance of Hampden 62.50 58.41 81.49 79.63 80.35 80.76 

Wave of sample release *** *** * ** * *

Wave 1 52.07 53.77 85.87 82.49 85.67 84.04 

Wave 2 64.78 68.01 85.12 86.18 80.65 80.65 

Wave 3 73.10 69.53 79.04 76.23 76.11 74.17 

Monthly SNAP benefit *

$1-$161 68.18 67.58 84.92 83.64 85.82 81.74 

$162–$200 57.66 54.95 87.48 82.70 81.51 83.68 

$201–$349 62.90 68.26 84.55 82.64 78.00 78.83 

$350 + 61.59 65.25 79.22 80.07 79.63 76.10 

Spanish language 

Yes 62.05 59.59 84.10 80.95 83.62 81.18 

No 61.92 63.74 83.76 82.40 80.93 79.85 

Recertification type *** * 

Recertification 61.95 63.33 84.39 82.70 84.33 82.30 

Semiannual reporting 64.28 66.09 84.19 82.59 77.31 78.64 

Other reevaluation 62.37 62.37 86.19 81.94 80.67 76.02 

Monthly income 

$0 54.14 57.62 82.02 76.37 79.44 81.90 

$1–$787 62.45 57.71 84.01 83.10 80.32 83.90 

$788–$1,088 64.81 65.29 87.35 82.46 83.43 76.49 

$1,089 + 65.17 68.18 82.26 84.59 82.18 79.32 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 252 ▌Appendix C: Weighting Methodology Abt Associates

 Round 1 Round 2b Round 3b

Characteristica HIP NonHIP HIP Non-HIP HIP non-HIP 

Baystate cap 

Yes 65.54 61.38 84.46 81.48 78.79 76.88 

No 61.68 62.87 83.78 82.13 81.78 80.46 

Homeless ** 

Yes 36.71 42.10 73.28 70.73 82.83 75.64 

No 63.67 64.25 84.27 82.77 81.48 80.34 

Housing type 

Private 63.61 64.71 84.82 83.13 81.26 80.12 

Public 66.14 68.83 85.17 83.65 81.55 81.12 

Other 41.06 45.64 74.66 69.81 88.84 76.80 

Household head age *

16–30 53.00 59.16 77.99 76.73 77.68 80.83 

31–40 64.16 62.96 81.70 82.70 77.26 75.63 

41–54 64.99 65.50 86.19 84.96 83.34 79.92 

55+ 66.65 63.11 88.75 83.50 86.74 83.86 

Household head race/ethnicity *** *

Hispanic 60.04 63.15 80.99 79.91 78.90 78.25 

White 63.94 61.04 84.91 82.46 84.07 80.94 

Black 64.94 70.12 90.34 87.33 83.37 83.62 

Other 57.21 56.20 83.20 83.62 80.20 81.52 

Disabled household head *** *

Yes 65.04 62.63 85.79 84.56 83.64 80.85 

No 59.11 62.87 81.75 79.56 79.11 79.39 

US citizenship of household head ***   ** * 

Yes 62.50 62.97 84.26 82.12 81.77 81.03 

No 51.04 58.34 75.07 81.27 76.31 62.93 

TANF/AFDC ***

Yes 65.64 70.47 81.91 81.65 80.86 76.57 

No 61.28 61.19 84.23 82.17 81.67 81.01 

Unearned income ** *

Yes 64.87 62.27 85.46 83.22 82.92 81.45 

No 57.78 63.42 81.11 80.41 79.02 78.16 

SSI 

Yes 63.99 61.38 85.11 82.81 83.43 79.80 

No 61.04 63.37 83.22 81.73 80.54 80.32 

RSDI ** *** * *** 

Yes 68.71 63.56 89.08 86.77 87.62 83.55 

No 59.70 62.48 81.80 80.32 79.06 78.78 

Unemployment compensation * ** 

Yes 60.07 63.59 88.96 80.88 91.68 75.84 

No 62.04 62.71 83.55 82.12 80.92 80.33 

Household type * * *

Household with elderly 64.99 59.25 90.89 80.50 87.59 79.17 

Household with children (no elderly) 61.83 67.82 81.53 79.83 78.48 77.93 

Other household 61.27 59.21 84.27 84.62 82.85 82.45 

Female ** *** * 

Yes 65.71 67.47 84.81 81.55 83.45 79.95 

No 55.73 55.02 81.99 83.12 77.58 80.55 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix C: Weighting Methodology ▌pg. 253

 Round 1 Round 2b Round 3b

Characteristica HIP NonHIP HIP Non-HIP HIP non-HIP 

Age of person *** *

16–30 54.66 59.94 79.82 78.80 76.60 77.97 

31–40 63.42 64.66 81.29 82.30 79.79 74.80 

41–54 66.69 65.42 85.73 85.38 83.53 82.08 

55+ 67.94 63.29 89.28 83.46 86.56 84.95 

Race/ethnicity * ***

Hispanic 60.08 62.88 81.14 80.15 78.71 78.13 

White 63.61 61.29 85.11 81.94 84.34 81.03 

Black 65.81 69.83 90.38 87.39 83.49 83.71 

Other 57.36 56.69 81.02 84.79 80.06 81.81 

US citizenship of sampled person ***   ** * 

Yes 62.28 63.03 84.22 82.13 81.73 80.89 

No 55.13 57.31 76.06 81.20 77.62 65.74 

Disabled sampled person *** ** * 

Yes 65.67 62.58 85.94 84.80 84.65 81.05 

No 58.91 62.89 81.76 79.60 78.24 79.27 

Unemployment compensation 

Yes 61.39 61.53 87.00 78.54 85.71 72.79 

No 61.97 62.80 83.67 82.21 81.30 80.42 

Household size (no. adults 16+) 

1 61.06 60.79 85.86 81.30 82.83 82.42 

2 62.61 63.43 80.38 81.70 79.33 77.27 

3 63.71 72.89 80.03 89.25 78.79 77.53 

4 + 75.58 72.27 70.97 82.41 71.62 61.49 

Rao-Scott chi-square test: *p<0.1, **p<0.05, ***p<0.01. Tests not corrected for multiple comparisons. a

Household and person characteristics reported in SNAP sampling frame. 

b

Response rates are conditional response rates. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 255

Appendix D: Participant Data Preparation 

This appendix discusses several facets of data preparation activities for the participant data used in the 

HIP evaluation. The first two sections describe coding of the participant survey data collected for the 

HIP evaluation. The first section describes coding of the 24-hour dietary recall data. The second 

section discusses other aspects of participant survey coding, including initial processing and 

construction of analytic outcomes and covariates. The third section describes processing of the EBT 

transaction data and construction of the analytic files. 

D.1 Coding of Dietary Recall Data 

AMPM Interview Data Entry/Standard SurveyNet Processing 

Westat collected the 24-hour dietary recall data for the HIP study using the same system used in the 

National Health and Nutrition Examination Survey, What We Eat in America (NHANES, WWEIA) 

interview. This system consists of 3 components: the Automated Multiple Pass Method (AMPM) 

interview system, the Post Interview Processing System (PIPS), and the SurveyNet coding 

application. 

Westat processed the recall data through PIPS and then created SurveyNet batches containing 

approximately 20 intake days each. The SurveyNet batches were entered into the online Coder 

Tracking System, which was used to track each batch through the various coding and review steps. 

Each batch was assigned to one of six dietary coders, with each coder completing the coding for all 

intake days within a single assigned batch. 

Coding 

Assigning Food Codes 

The data collected in the AMPM was automatically imported into SurveyNet. SurveyNet displays a 

shorthand version of each question and the selected response for all food description and portion data 

in a text box at the top of the food coding screen. This interview data is reviewed by the dietary coder 

and used to select the appropriate food code and enter the quantity reported. During PIPS processing, 

approximately 70 percent of foods are auto-coded, meaning that a food code and/or a portion quantity 

is pre-assigned; in those cases, the dietary coder merely reviews the pre-filled fields to ensure that no 

changes need to be made. Changes to these pre-assigned data might have been required if the 

interviewer had entered a comment or a text response in any field that would cause the coder to 

change the pre-assigned code or quantity. For all foods not auto-coded during PIPS, the dietary coders 

reviewed all question responses to determine the most appropriate food code to apply. 

Recipe Modifications 

Coders had the ability to create recipe modifications to more closely match the reported food. Coders 

followed the same modification guidelines used in NHANES, which allowed modification of a recipe 

for the type of fat used in cooking, the type of milk used in preparing selected foods (e.g., beverages, 

pudding, cooked cereal), amount of liquid used to prepare condensed soup (when different from 

instructions), and the type of salad dressing used in salads such as coleslaw or chicken salad. 

New Foods 

The coders also flagged new food items that could not be linked to an acceptable food code in 

SurveyNet. Coding supervisors did additional research to determine if the food could be matched to 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 256 ▌Appendix D: Participant Data Preparation Abt Associates

an existing food code or if the food needed to be flagged for nutrient modification after analysis, 

because the nutrient profile of the foods differed too much from existing food codes. Two food items 

were handled in this way: quinoa and almond milk. Nutrient information for these products was 

obtained from USDA Database for Survey Research (quinoa) or product labels (almond milk) and 

corrected in the SurveyNet analysis files. 

Coding Guidelines 

Coding guidelines were used to resolve common coding problems and to establish consistent coding 

methods. NHANES coding guidelines were provided to the coders. These guidelines are organized by 

food category and contain rules for coding foods when not enough information is available (e.g., how 

much meat to code in a sandwich when the amount is not given, how to handle reports of nonstick 

spray, etc.). Additional guidelines were developed throughout the study as new issues were resolved. 

These guidelines were documented in a decision log maintained throughout the study. 

Entering Quantities 

Once the food code was assigned or reviewed (in the case of auto-coding), coders reviewed the autocoded quantity or entered the amount of food reported. SurveyNet allows entry of portions using the 

same food models presented in the AMPM, and also provides predetermined food weights for foods 

in commonly eaten portions (e.g., one half grapefruit, one medium chicken leg). Food amounts that 

were entered as a shape, by dimensions (length, width, and height), volume, or weight were 

automatically converted to a weight in grams. Coders could also use SurveyNet to code imprecise 

measures, such as “handful,” “medium bowl,” or “swallow.” When respondents reported “Don’t 

know” for the quantity consumed, coders were instructed to first consult the coding guidelines, which 

provide default amounts for items in a sandwich, salad, added to coffee, and other common 

combinations. Should the coding guidelines not apply, coders selected the “quantity not specified” 

portion option presented in SurveyNet. 

Combinations 

Foods added to another food (e.g., cream added to coffee) or eaten in combination (e.g., the bread, 

meat, cheese, and spread on a sandwich) were flagged in SurveyNet using combination codes. The 

combinations were usually identified during data collection by AMPM and so a combination type 

code was pre-filled in SurveyNet. If coders needed to add additional food codes to represent the 

reported food, the coder used the combination type code to link the foods. 

Review 

After the dietary coders assigned food codes, coders and supervisors conducted quality control by 

verifying, adjudicating and editing the assigned food codes and portion amounts. Verifying involved a 

detailed review of coded intakes by a second coder. Any notepad entries made by the second coder 

highlighting questions or disagreement between coders were adjudicated by one of the four coding 

supervisors. All adjudicated records were reviewed by the supervisor, and decisions were made on 

notepad questions and unfound foods. Coded intakes that were adjudicated by one supervisor were 

edited by a second supervisor; thus each intake was reviewed by two supervisors. The adjudication 

process also allowed evaluation of the accuracy of each coder’s work. Two intakes from every batch 

were used for calculation of accuracy, so that 10 percent of each coder’s work was assessed. Coders 

were required to maintain 95 percent accuracy. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 257

Analysis 

Nutrient analysis was performed using SurveyNet’s analysis system. The system automatically 

generates error reports that documented unresolved issues such as missing or invalid food codes, 

recipe modification, or portion codes. All errors were resolved and the analysis re-run. Two analysis 

data files were prepared: an “ANA” file, which contained one line of data for every food or 

supplement reported by the respondent on the intake day; and a “TOT” file, which contained one line 

of data for each respondent for a single intake day. The standard values provided included 65 

nutrients from the Food and Nutrient Database for Dietary Studies (FNDDS). Additional variables 

were appended to these files (USDA Food Pattern food groups, MyPyramid food groups, Healthy 

Eating Index, and special outcomes for the HIP study). Construction of these variables is described at 

the end of the section (after the discussion of quality control procedures). 

Quality Control Review 

Standard quality control (QC) checks were performed on the analyzed data as a means of identifying 

errors. Outlier reports identified unusually high or low portions for key food items and high or low 

amounts of key nutrients. The outliers were reviewed and any deemed to be the result of coding errors 

were corrected. These outlier checks are explained in more detail below. 

Portion Outliers 

Portion outlier reports were used to identify errors for reported amount of foods consumed. In 

addition, they served as a check for intakes where an incorrect form of the food was applied157 when 

specifying the amount. The USDA SurveyNet software used to code AMPM intakes identified 

intakes where the portion of the reported food was either below or above established portion size 

range for that food item; these portion size ranges were specific for the age and gender of the 

respondent. 

The following criteria are used to triage which records identified by SurveyNet as outside the portion 

range should be checked for accuracy 

 Beverages greater or equal to 1⁄2 gallon 

 Meat, fish, poultry greater than or equal to 12 weighted ounces (WO) (342 grams) 

 Mixed dishes greater than or equal to 6 cups 

 Greater than or equal to 8 WO snack foods (chips, nut, etc.) 

 Review coding of foods with yields from dry amounts rather than from cooked volume OR 

cooked weights that might have been from dry 

 Beans 

 Cooked cereals 

 Pasta 

 Rice 

 Other grains, such as bulgur, couscous 

 Popcorn, as un-popped rather than popped volume 

 Unreconstituted mixes, soups, drinks, etc. 

 WO entry of ice cream 

 

157 For example, the coder entered 1 cup of rice as uncooked by mistake when it was reported as cooked. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 258 ▌Appendix D: Participant Data Preparation Abt Associates

 Less than or equal to 1/8 teaspoon 

Intakes that included portions above or below identified limits were reviewed to ensure that no 

recording158 or coding errors had been made. When the portion outlier reviews were completed and 

edits made, the data were re-analyzed prior to the next QC step so that nutrients would reflect the 

edited data. 

AMPM Nutrient Outliers 

Key nutrient values were reviewed as a means to identify errors or anomalies. These nutrients were 

calories, total protein, total fat, total beta-carotene, and total vitamin C. A review at the nutrient level 

was considered indicative of any biases due to database errors, routine coding decisions, editing 

resolutions, or coding guidelines. The 5th and 95th percentile of intakes for these five nutrients were 

determined for a specified age and gender group from the NHANES data, and cut points were then 

determined based on the percentiles. 

The cut points used for the nutrient outlier review were as follows: 

1. Kcal 

Gender/Age Low High 

Adult women >/=12 years old 600 4400 

Adult males >/= 12 years old 650 5700 

2. Protein (grams) 

Gender/Age Low High 

Adult women >/=12 years old 10 180 

Adult males >/= 12 years old 25 240 

3. Fat (grams) 

Gender/Age Low High 

Adult women >/=12 years old 15 185 

Adult males >/= 12 years old 25 230 

4. Vitamin C (mg) 

Gender/Age Low High 

Adult women >/=12 years old 5 350 

Adult males >/= 12 years old 5 400 

5. Beta-carotene (mcg) 

Gender/Age Low High 

Adult women >/=12 years old 15 7100 

Adult males >/= 12 years old 15 8200 

 

158 Errors were suspected when the amount entered was the same as the number for the unit and it was an odd 

combination (e.g., 23 G3 glasses, where 23 is the code for the G3 glass). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 259

Prior to examining macro and micro-nutrient outliers, reports were generated for total calorie intake 

outliers. Records flagged as outliers for calories were examined and any interviewer or coding errors 

were corrected. The records were re-analyzed prior to generating outlier reports for the remaining 

nutrients. 

Minimum Criteria for Inclusion in Dataset 

When conducting reviews of the intakes identified in any of the outlier reports, a determination of 

whether or not the intake met minimum criteria was made. In general, an intake did not meet 

minimum criteria if any of the following situations were noted: 

1. Interview was broken off prior to completing the time and occasion pass. If the breakoff 

happened before the time and occasion was recorded for every food in the intake, the intake 

failed the minimum criteria and was deleted from the dataset; without time and occasion 

information for each food, it was not possible to determine that the reported foods spanned an 

entire day’s intake. 

2. Intake was judged as “unreliable.” Although interviewers did not provide feedback on 

whether or not a respondent was reliable, guidelines developed in previous studies were 

implemented. 

3. Meals with missing foods. This flag was implemented when a respondent reported a meal, 

but could not recall foods eaten at the meal. For example, the respondent reported eating a 

meal at a friends’ house but could not recall the foods. 

USDA Food Pattern Food Groups, MyPyramid Food Groups, and Healthy Eating Index 

After all dietary recall data files were edited and finalized, nutrient values, MyPyramid Equivalent 

(MPE) values, and Food Pattern Equivalent (FPE) values were appended to each record. Nutrient 

values were taken from the USDA Food and Nutrient Database for Dietary Studies, 4.1 (FNDDS4.1). 

FPE values were taken from the Food Pattern Equivalent Database (FPED) 09-10159; food codes that 

did not have a match in the FPED were reviewed and food group values were imputed. MPE values 

were taken from MyPyramid Equivalents Database (MPED) version 2.0 (Bowman, Friday, and 

Moshfegh, 2008) supplemented with the USDA Center for Nutrition Policy and Promotion (CNPP) 

MPED Addendum (2013a) to allow for compatibility with FNDDS4.1. The additional Whole Fruit 

equivalent for calculating Healthy Eating Index-2005 (HEI-2005) was taken from the CNPP Support 

Files (USDA CNPP, 2013b). As the CNPP MPED Addendum does not provide MPE values for all 

FNDDS4.1 food codes, additional MPE and HEI-2005 values were imputed by Westat, as needed, to 

attain complete MPE and HEI-2005 data. 

The HEI-2005 and HEI-2010160 was calculated for all intakes using the SAS code provided by 

CNPP;161 the HEI-2005 and HEI-2010 component scores as well as the total HEI-2005 and HEI-2010 

 

159 U.S. Department of Agriculture, Agricultural Research Service. (2013) Food Patterns Equivalents 

Database. http://www.ars.usda.gov/Services/docs.htm?docid=23869 

160 U.S. Department of Agriculture, Center for Nutrition Policy and Promotion. (2013). Healthy Eating Index. 

http://www.cnpp.usda.gov/HealthyEatingIndexSupportFiles0708.htm 

161 http://www.cnpp.usda.gov/HealthyEatingIndex-2005report.htm 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 260 ▌Appendix D: Participant Data Preparation Abt Associates

scores were provided for each intake day. The density components provided with the CNPP SAS 

code for HEI-2010 were also provided for each intake day. 

Special Outcomes for This Study 

Additional variables included in the analysis are described below. 

Targeted Fruit and Vegetable (TFV) intake (preferred restrictive measure and alternative inclusive 

measure) 

One of the goals of HIP was to increase consumption of targeted fruits and vegetables (TFVs), 

defined as those eligible for the financial incentive, which are the same foods that are eligible for 

WIC fruit and vegetable vouchers. These include fresh, canned, frozen, and dried fruits and 

vegetables without added sugars, fats, or oils. Fruits may not have added salt and fruit juices are 

excluded. Vegetables may be regular or lower sodium, and white potatoes are excluded. No foods 

prepared away from home or served ready to eat are included. 

The AMPM documents the source of the food or most of the ingredients of the food; for the TFV 

intake measure, only food items where the source was “store” were included. Because not all survey 

respondents were able to separately report individual ingredients and quantities for food prepared 

from multiple ingredients (“mixed foods”), coding of the AMPM interview does not always allow us 

to distinguish if a food item with added sugar was purchased already prepared (and therefore not 

eligible) or prepared at home from an eligible food item. Two versions of the TFV measure were 

developed: a preferred, restrictive measure, which excluded fruits and vegetables in mixed foods 

when the respondent was not able to identify individual ingredients and their sources, and an 

alternative, inclusive measure, which included fruits and vegetables in both mixed foods as well as 

single items. 

Food codes included in the preferred restrictive TFV intake measure included food codes for 

individual fruits and vegetables, and mixtures of just fruits and vegetables, such as broccoli cooked 

with fat added, sweetened frozen strawberries, or frozen mixed vegetables. These simple mixtures 

were included in the restrictive measure because if the source of the food was the store, it was highly 

likely that the fruit or vegetable was an eligible purchase. The goal of the restrictive TFV measure 

was to exclude fruits and vegetables contained in mixed foods like soups or lasagna, which contained 

non-fruit, non-vegetable ingredients. The alternative inclusive TFV measure summed the cup 

equivalents for whole fruit and whole vegetables (i.e., no juices were included), and then subtracted 

the cup equivalents for white potatoes. 

Other ingredients in foods with fruits & vegetables 

In addition to the TFV values, Westat calculated amounts of two nutrients (total sugar and sodium) 

and three USDA Food Pattern food groups (oils, solid fats, and added sugars) in foods containing 

fruits and vegetables. This was done by identifying all foods with any fruit or vegetable content and 

summing up nutrient and USDA Food Pattern food group equivalent amounts for each respondent 

within those foods only. 

Form of preservation (fresh/canned/frozen/dried) 

In order to assess the form of preservation for fruits and vegetables consumed in the study, Westat 

categorized FNDDS food codes containing fruits and vegetables by the form specified in the food 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 261

code description. Note that form of preservation is usually only specified when the food contains 

either a single fruit of vegetable (e.g., fresh apple, raw carrots), or a fruit or vegetable with simple 

preparation, such as those that would fall into the restrictive TFV measure described above (e.g., 

carrots cooked from frozen with fat added). Cup-equivalents were summed by form of preservation 

for the food codes able to be categorized. 

Store and other purchases 

The TFV values described above total the fruits and vegetables reported as being obtained from a 

store. In addition to these, Westat calculated similar TFV values for fruits and vegetables obtained 

from a source other than a store. The other sources include restaurants, cafeterias, community food 

programs, etc. A complete list of the sources offered in the AMPM interview is presented in 

Exhibit D.1. 

Exhibit D.1: Source of Foods Reported in AMPM 

Code Source 

1 Store 

2 Restaurant with waiter/waitress 

3 Restaurant fast food/pizza 

4 Bar/tavern/lounge 

5 Restaurant no additional info 

6 Cafeteria not at school 

7 Cafeteria at school 

8 Child care center 

9 Family/adult day care center 

10 Soup kitchen/shelter/food pantry 

11 Meals on wheels 

12 Community food program—other 

14 Vending machine 

15 Common coffee pot or snack tray 

16 From someone else/gift 

17 Mail order purchase 

18 Residential dining facility 

19 Grown or caught by you or someone you know 

24 Sport recreation, or entertainment facility 

25 Street vendor, vending truck 

26 Fundraiser sales 

91 Other, Specify 

Alternate measures: fats 

The MPE and FPE discretionary oils and solid fats are provided as grams of fat in the databases. The 

MPE and FPE components were converted to teaspoons of fat, using the conversion of 4.53 

g/teaspoon for oils and 4/27 g/teaspoons for solid fats. Additional variables of discretionary oils and 

solid fats in teaspoon measures were added to each intake day. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 262 ▌Appendix D: Participant Data Preparation Abt Associates

SoFAS 

The Dietary Guidelines for Americans (DGA)162 specifies limits of calories from solid fats and added 

sugars, called SoFAS. The MPE and FPE components for solid fats and added sugar were used to 

generate SoFAS variables, using the conversion of 9 kcal/g for solid fats and 16 kcal/teaspoon for the 

added sugars. The new SoFAS variables were added to each intake day. 

Legumes 

We computed variables to allocate legumes properly to either the protein group or the vegetable 

group, based on the recommended intake specified in the USDA Meal Patterns provided by the DGA. 

In this calculation, legumes (as ounce equivalents) are added to the protein foods total until the 

recommended amount of protein foods is met; any amount of legumes that remains is converted to 

cup equivalents and added to the total vegetable amount (1 ounce equivalent of legumes equals 1⁄4 

cup). The protein foods recommendation is based on the recommended calorie needs per day by age, 

gender and physical activity level. For this calculation, calories for the moderately active physical 

activity level were used. The new total protein foods and total vegetables variables were added to 

each intake day. 

Dietary Supplement Data 

A dietary supplement module (DSM) was programmed that utilized the questions from the NHANES 

dietary supplement interview. Dietary supplement data files were converted from the Manipula files 

to text files using the program provided by USDA. The text file was converted into Excel and coded 

within Excel by matching a supplement code from the NHANES Dietary Supplement Database 

(DSD).163 Once the coding was finished, Westat used the nutrients provided in the DSD to analyze the 

coded records for the 65 nutrients provided in FNDDS. 

D.2 Coding of Other Participant Survey Data 

Initial Processing of Participant Survey Data 

Participant survey data were collected using a computer-assisted telephone interview (CATI) system, 

developed and programmed in Blaise. With Westat’s CATI system, the initial data collection, receipt 

control, coding and editing, and data entry were performed as a single operation. Most edit checks 

were performed online as the interviewer entered responses into the computer. Range checks 

prevented interviewers from entering impossible responses for precoded and numerical items. 

Consistency checks between items were performed during the interview, so that inconsistent 

responses were clarified immediately with the respondent. 

CATI will accept open-ended data, though interviewers were trained to perform as much coding as 

possible while the telephone interview was in progress. Westat conducted post-data collection 

processing, which included review of text entries for “Other, specify” fields and interviewer 

comments recorded in the CATI interviews. Review of text entries resulted in either “upcoding” the 

 

162 U.S. Department of Agriculture and U.S. Department of Health and Human Services. Dietary Guidelines 

for Americans, 2010. 7

th 

Edition, Washington, DC: U.S. Government Printing Office, December 2010. 

Available at http://www.health.gov/dietaryguidelines/dga2010/DietaryGuidelines2010.pdf 

163 http://www.cdc.gov/nchs/nhanes/nhanes1999-2000/DSPI.htm 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 263

text responses to an existing code or the creation of a new code. All new codes were incorporated into 

the codebook. Interviewer comments were reviewed to look for possible data problems (e.g., 

difficulty in categorizing the respondent’s answer into the precoded choices on CATI). Data 

management staff conferred with project staff to determine if data changes were needed; all changes 

were documented in a data change spreadsheet. This spreadsheet was then used to perform qualitycontrol checks on the data set to verify changes. The Data Manager reviewed response frequencies 

and cross-tabulations during and at the end of data collection to check for outliers (values that were 

too large or small). Outlier checks performed during the interview allowed interviewers to confirm or 

correct the response immediately with the respondent. Outliers identified during processing were 

discussed with project staff and any changes were documented on the data change spreadsheet. 

Construction of Analytic Outcomes and Covariates 

Creation of Composite Scales 

In order to improve measurement precision and reliability, we created scales based on multiple survey 

items. Scales were created to reflect 1) positive attitudes toward food, fruits, and vegetables; 2) 

barriers to fruit and vegetable consumption; 3) barriers to grocery shopping; and 4) availability of 

fruits and vegetables in the home. 

To create scales, we first selected the survey items relevant to each scale. To ensure that all selected 

items for each scale measured the same construct, and that each scale was internally consistent, we 

examined the reliability coefficient (alpha, α) of the items related to each scale. We used an 

approximate cutoff of α = 0.70 or higher for adequate reliability and internal consistency.164 If α was 

below 0.70, we removed items with the lowest correlation with the total from the scale until we 

achieved an α of 0.70 or higher. Once the final items for each scale were determined, all items were 

coded so that higher scores indicated “more.” Scales were then created by taking the mean of all 

items when at least 75 percent of the items were non-missing. Below, we describe each of the four 

scales in detail. 

The positive attitudes toward food, fruits, and vegetables scale was created using the following 6 

survey items which asked respondents to indicate their agreement on a scale from 1 (strongly 

disagree) to 5 (strongly agree): 

1. I enjoy trying new foods. 

2. I enjoy trying new fruits. 

3. I enjoy trying new vegetables. 

4. I eat enough fruits to keep me healthy. 

5. I eat enough vegetables to keep me healthy. 

6. I often encourage my family and friends to eat fruits and vegetables. 

 

164 The following citations suggest that a reliability coefficient of 0.70 and above is adequate: Nunnally 

(1978), George and Mallery (2003). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 264 ▌Appendix D: Participant Data Preparation Abt Associates

If at least 5 of the 6 items were non-missing, items were averaged to create the scale (α = 0.68165 for 

round 1, α = .73 for round 2, and α = .74 for round 3). 

The barriers to fruit and vegetable consumption scale was created using the following 7 items which 

asked respondents to indicate their agreement on a scale from 1 (strongly disagree) to 5 (strongly 

agree): 

1. It’s hard for me to eat more vegetables because I don’t know how to prepare them. 

2. It's hard for me to eat more vegetables because they are hard to find where I shop for food. 

3. It's hard for me to eat more fruits because they are hard to find where I shop for food. 

4. I don’t eat fruits and vegetables as much as I like to because they cost too much. 

5. I don’t eat fruits and vegetables as much as I like to because they often spoil before I get a 

chance to eat them. 

6. I don’t eat fruits and vegetables as much as I like to because my family doesn’t like them. 

7. I don’t eat fruits and vegetables because I don't like them. 

If at least 7 of the 8 items were non-missing, items were averaged to create the scale (α = 0.73 for 

round 1, α =.73 for round 2, and α = .74 for round 3). 

The barriers to grocery shopping scale was created using the following 2 items which asked 

respondents to indicate frequency on a scale from 1 (always) to 5 (never): 

1. How often does limited transportation keep you from shopping for groceries? 

2. How often does distance to grocery store keep you from shopping for groceries? 

If both items were non-missing, the items were reverse-coded and then averaged to create the scale (α

= 0.80 for round 1, α = .84 for round 2, and α = .81 for round 3). 

The availability of fruits and vegetables in the home scale was created using the following 4 items166

which asked respondents to indicate frequency on a scale from 1 (always) to 5 (never): 

1. How often do you have fruits available at home? This includes fresh, dried, canned and 

frozen fruits. 

2. How often do you have fruits in the refrigerator or on the kitchen counter?167

 

165 Although the α did not reach the 0.70 cutoff for Round 1, it is very close. Removing items or splitting the 

items into two scales further decreased the α, thus indicating that all six items work best together as one 

scale. (The α is also above 0.70 for Rounds 2 and 3.) 

166 There are 5 additional items included on the family food environment section of the survey that do not 

focus specifically on fruits and vegetables, which when included in the scale lowered the reliability to an 

inadequate level. Principal components analysis confirmed that these items do not load strongly onto any 

one factor, and thus they will not be included in the scale. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 265

3. How often do you have vegetables available at home? This includes fresh, dried, canned, and 

frozen vegetables.168

4. How often do you have ready to eat vegetables such as baby carrots, cherry tomatoes, or 

vegetables that you have sliced to make them ready to eat in the refrigerator or on the kitchen 

counter? 

If at least 3 of the 4 items were non-missing, items were reverse-coded and averaged to create the 

scale (α = 0.75 for round 1, α = .75 for round 2, and α = .76 for round 3). 

D.3 EBT Transaction Data Processing Methods 

The EBT transaction data were processed for the full pool of HIP and non-HIP participants in Hampden 

County. The analysis in this report used transaction data beginning in November 2011, when HIP began. 

Key outcome analyses focused on transactions beginning in January 2012 when all participants were 

active. 

The EBT data were analyzed through the end of the pilot, focusing on expenditures in the two periods 

coinciding with the participant surveys and when all HIP households were eligible to earn incentives 

(March-July 2012 and August-October 2012). 

The EBT data were received in the form of daily HIP activity files that included information on the time, 

location, amount and type of each SNAP transaction. The data were cleaned and combined into monthly 

analysis files with variables for total household SNAP issuances, SNAP purchases, HIP-eligible TFV 

purchases and HIP incentive earnings, as well as purchases by store type. The following sections describe 

the data cleaning and variable construction process, which also required merging the EBT data with 

additional sources of data, such as the Retailer EBT Data Exchange (REDE) files and administrative case 

file data. 

Initial Construction of Raw Monthly Transaction-Level Files from HIP Daily Activity Files and 

Monthly Missing Data Files 

The EBT processor for Massachusetts, Xerox, collects and maintains data pertaining to the SNAP 

EBT transactions. These data show the date and amount that SNAP benefits were credited, and they 

show the date, time, amount, and location for each shopping transaction using SNAP benefits. In 

addition, transaction data for the evaluation period provide information (date, time, amount, store) on 

HIP-eligible purchases and HIP incentives earned. Only purchases made with SNAP benefits are 

included; purchases made with other forms of payment, such as cash or WIC vouchers are not 

captured. EBT transaction data were transmitted for the full pool of HIP and non-HIP participants in 

Hampden County without social security numbers, but case file identification numbers were included 

to allow linking of EBT transaction data to administrative case file data and survey data. 

 

167 Respondents were also given the option of “Don’t have a refrigerator” for this item. These responses were 

recoded to “Never.” 

168 Respondents were incorrectly given the option of “Don’t have a refrigerator” for this item. These responses 

were recoded to missing. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 266 ▌Appendix D: Participant Data Preparation Abt Associates

EBT data were first received in the form of daily HIP activity files. Initial checks revealed that these 

files were missing transactions for approximately four percent of SNAP participants in the HIP 

universe. Xerox ascertained that these cases were left out from the source table of HIP and non-HIP 

households used to generate the HIP daily activity files. After correcting the source table, Xerox 

provided monthly files with the missing transactions. These files were merged with the original HIP 

daily activity files to produce complete transaction files. 

Cleaning of Raw Monthly Transaction-Level Files 

The daily raw transaction data were combined to produce a transaction-level file for each month of 

the pilot. Each month’s transaction-level file included records for the issuance of benefits made 

available during that month and cancellations, client initiated transactions and adjustments initiated 

during the month. The following checks and data-cleaning steps were performed on the complete 

monthly transaction-level files: 

 Duplicate transactions, defined as extra records with identical values in each field, were 

removed from the files. 

 Denied transactions and balance inquiries were removed from the files. 

 Voucher settlement records corresponding to a previous transaction record were removed so 

that the value of the voucher transaction would not be counted twice in the analysis. 

 The HIP household flag was corrected to be current with the date of the transaction. The HIP 

flag provided in the EBT data did not match the household’s current HIP flag for some 

transactions in the first few months of the pilot. The correct HIP flag was determined from 

the household’s assigned status during random assignment and then merged to the transaction 

data. The flag was also updated for fifteen households that were originally assigned to HIP, 

but opted out of the program during the pilot. The flag for these households was switched to 

‘non-HIP’ status. 

Construction of the Monthly Household-Level Analysis Files 

Outcome Variables 

Household-level analysis files for each month of the pilot were created from the clean monthly 

transaction-level files by aggregating transaction records for each household. Transaction amounts 

were summed to compute net credits and debits for SNAP issuance, SNAP purchase, HIP-eligible 

TFV purchase and HIP incentive earnings transactions for each household that had SNAP activity 

during the month. The following household analysis variables were created by taking the difference 

between net credits and debits for the various types of transactions: 

 Net issued SNAP benefits 

 Net available SNAP benefits for the month, which included benefits carried forward from the 

previous month 

 Net SNAP purchases 

 Net HIP-eligible TFV purchases 

 Net HIP incentives earned 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 267

The computation of the SNAP benefit and SNAP purchase variables were checked by comparing the 

household’s calculated account balance (Net available SNAP benefits – SNAP purchases) to the 

account balance given in the transaction data on the last day of the month. Likewise, computations of 

household HIP-eligible TFV purchases and HIP incentives earnings were checked by comparing the 

calculated HIP incentives earnings variable (0.3*HIP-eligible TFV purchases) to the “HIP incentives 

earned to date” variable given in the data on the last day of the month. 

A total of 14 household-level analysis files were created corresponding to each month of the pilot. 

Exhibit D.2 shows the number of households in each monthly analysis file, and the number of 

households that received a SNAP benefit). Our analysis was conducted on the households that 

received a benefit during the month. 

Exhibit D.2: Number of Observations in Monthly Household-Level Analysis Files 

Month 

Total number of households in 

file 

Number of active households 

(received a SNAP benefit) 

November 2011 50,788 50,156 

December 2011 49,933 49,079 

January 2012 49,123 48,378 

February 2012 48,541 47,982 

March 2012 48,104 47,507 

April 2012 47,511 46,822 

May 2012 47,166 46,517 

June 2012 46,820 46,157 

July 2012 46,698 45,730 

August 2012 46,568 45,444 

September 2012 46,120 44,833 

October 2012 45,544 44,289 

November 2012 44,979 43,845 

December 2012 44,554 43,421 

Average across all pilot months 47,318 46,440 

Store Type and Participating Retailer Variables 

In addition to the outcome variables listed above, net SNAP purchase variables were constructed for 

each store type and HIP participating retailer classification, and for combinations of these 

classifications. HIP-eligible TFV purchase variables were constructed for participating stores only. 

HIP-eligible TFV purchases at participating integrated electronic cash register (IECR) stores were 

identified for both HIP and non-HIP households. HIP-eligible TFV purchases at participating nonIECR stores such as grocery and convenience stores and farmers markets were calculated for HIP 

households that identified themselves as HIP participants and separated HIP-eligible items during 

check-out. 

The construction of purchase variables by store type involved merging the monthly transaction-level 

files with a cumulative FNS REDE file that included information on all SNAP-authorized retailers, 

including store type. The cumulative file was constructed by combining monthly Massachusetts 

REDE files169 to identify a patronized retailer’s store type170. Retailers were grouped into the 

following store types: 

 

169 We received monthly REDE files for October 2011 to June 2012, and September 2012 to January 2013. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 268 ▌Appendix D: Participant Data Preparation Abt Associates

 Supermarkets and Superstores 

 Grocery and specialty stores 

 Convenience stores 

 Farmers Markets 

 Other stores171 

 Unknown stores if the retailer was not found in the Massachusetts REDE file 

Retailers were classified as participating in HIP during the month if any of the following criteria were 

met: 

 The retailer was on the final list of participating retailers within and outside Hampden County 

provided by FNS and started participating in HIP that month or earlier 172 

 The transaction data showed HIP-eligible TFV purchases at that retailer by a HIP or non-HIP 

household during the month. 

Exhibit D.3 lists the store types and participating retailer classification combinations for which SNAP 

and HIP-eligible TFV purchase variables were constructed. 

Exhibit D.3: SNAP Purchases and HIP-Eligible TFV Purchases Variables by Store Type 

SNAP purchases HIP-eligible TFV purchases 

All supermarkets and superstores Participating supermarkets and superstores 

All IECR and non-IECR convenience stores Participating IECR and non-IECR convenience stores 

All grocery stores Participating grocery stores 

All farmers markets Participating farmers markets 

All other stores Participating other stores 

Participating supermarkets and superstores 

Participating IECR and non-IECR convenience 

stores 

Participating grocery stores 

Participating farmers markets 

Participating other stores 

Nonparticipating supermarkets and superstores 

Nonparticipating IECR and non-IECR convenience 

stores 

Nonparticipating grocery stores 

Nonparticipating farmers markets 

Nonparticipating other stores 

Unknown store type (out of State) 

 

170 There were 20 stores that were classified differently across the months. For the analysis, the retailer was 

classified using the most recent store type. 

171 Other store types include: alcohol or drug treatment program, bakery specialty, non-profit food buying coop, shelter for battered women, communal dining facility, delivery route, group living arrangement, 

homeless meal provider, military commissary, meal delivery service, and senior citizens center/residential 

building. 

172 One chain retailer implemented HIP nationwide and thus all its stores were classified as participating. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix D: Participant Data Preparation ▌pg. 269

Demographic Subgroup Variables 

For the analysis of EBT purchase patterns by household characteristics, the monthly household-level 

analysis files were merged with July 2011 case variables from the administrative case file data. The 

following demographic information was derived from the case files: 

 SNAP benefit amount 

 Monthly household income 

 Spanish speaking 

 Race and ethnicity of household head 

 Disability status of household head 

 Age of head of household 

 Household composition (presence of children/elderly) 

 Household size and gender of household head 

 Location of residence 

Pre-HIP Shopping Pattern Indicators 

One of the analyses presented in this report is the impact of HIP on TFV purchases during the March 

to October 2012 period for households that primarily shopped at HIP participating stores before the 

pilot began. Similar analyses were conducted for households that primarily shopped at 

nonparticipating stores before the pilot. To classify pre-HIP participating and nonparticipating store 

shoppers, November 2011 transaction data were analyzed for Wave 2 and Wave 3 households (which 

were not yet eligible to earn HIP incentives). The evaluation research design intended that EBT 

transaction data would be available beginning two months prior to the HIP implementation. However, 

it proved difficult to compile EBT data files during the period that preparations for implementation 

were being completed; transaction-level files for the pre-implementation period could not be 

reconstructed in time for this report. Therefore the pre-HIP shopping pattern analysis in this report is 

restricted to Wave 2 and 3 households that had transaction activity in November 2011. 

A Wave 2 or Wave 3 household was classified as a pre-HIP participating store shopper if more than 

50 percent of their November 2011 SNAP purchases were made at stores that were participating in 

HIP during the period of the main analysis, i.e., March to October 2012. A household was classified 

as a pre-HIP nonparticipating store shopper if at least 50 percent of their November 2011 SNAP 

purchases were made at stores that were not participating during March to October 2012. Based on 

these criteria, an indicator was constructed for the classification of households as pre-HIP 

participating and nonparticipating store shoppers. This indicator was merged with the March to 

October 2012 monthly household-level analysis files to estimate the impact of HIP on HIP-eligible 

TFV purchases by pre-HIP shopper type. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix E: Analytic Methods ▌pg. 271

Appendix E: Analytic Methods 

The appendix discusses eight analytic issues: (1) regression models for estimating impact; (2) the 

treatment of limited dependent variable models; (3) regression models for subgroup analyses; (4) the 

computation of standard errors; (5) estimation of inter-round change; (6) multiple comparisons and 

the single confirmatory outcome—TFV; (7) sensitivity to outliers; and (8) usual intake estimation. 

E.1 Multivariate Models 

As noted in Chapter 2, our primary estimates of HIP impacts are regression-adjusted, as opposed to 

simple differences in treatment and control group means. Regression adjustment improves the 

comparability of the treatment and control groups and increases the precision of our estimators. For 

continuous outcomes, we use a model of the form: 

(E.1) h r i h h r i h,r,i y HIP ControlVars + ε , ,   0  1   2 , ,

Where the subscripts are h for household, r for round (Round 2 or Round 3), and i for interview (at 

Round 2 and Round 3, 10 percent of the sample was interviewed a second time to support analysis of 

usual intake); and the variables are as follows: y is an outcome of interest, HIP is a binary variable 

that identifies the treatment group, ControlVars is a vector of characteristics measured as of the 

Round 1 (baseline) survey or at baseline from administrative data, and  is a regression residual, 

representing unmeasured factors. Random assignment assures that  is uncorrelated with HIP. 

In this specification, 1 gives the impact of HIP, and H0: 1=0 is a test for any impact of HIP. 

Exhibit E.1 lists the included ControlVars and tabulates simple descriptive statistics (mean and 

standard error). As Exhibit E.1 shows, ControlVars includes all variables used in blocking and 

stratification (geography within Hampden county, household composition, survey wave, and gender 

of household head), as well as respondent demographic characteristics (age group, gender, 

race/ethnicity), measures related to baseline consumption according to the Fruit and Vegetable 

Screener, and baseline composite scales derived from questions about the home food environment, 

barriers to grocery shopping, and attitudes about and barriers to consumption of fruits and vegetables. 

For regressions based on the dietary recall interview only, we additionally included covariates about 

the AMPM interview itself, including day of interview (first or second) and the respondent’s 

assessment of the prior day’s consumption relative to usual levels (more, less, or the same as usual). 

Finally, for all variables in which baseline outcome data were collected, the baseline outcome was 

also included as a final covariate. These control variables were selected as likely to be strong 

predictors of the confirmatory outcome. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 272 ▌ Appendix E: Analytic Methods Abt Associates

Exhibit E.1: Included Analytic Covariates, Final Analytic Sample 

 Total Treatment Control P-value 

Stratification variables 

Geography 

Springfield 0.53 (1030) 0.52 (505) 0.53 (525) [0.647] 

Chicopee or Holyoke 0.25 (505) 0.27 (255) 0.25 (250) 

Hampden balance (omitted category) 0.22 (419) 0.21 (220) 0.22 (199) 

Persons in household 

One person in household 0.45 (872) 0.45 (450) 0.45 (422) [0.916] 

Multiple persons in household (omitted 

category) 0.55 (1082) 0.55 (530) 0.55 (552) 

Adults in household 

3 or fewer adults in household (omitted 

category) 0.97 (1918) 0.98 (965) 0.97 (953) [0.067]* 

4 or more adults in household 0.03 (36) 0.02 (15) 0.03 (21) 

Household composition 

Elderly (with/without children) in 

household 0.11 (238) 0.11 (129) 0.11 (109) [0.975] 

Children (no elderly) in household 0.42 (827) 0.42 (398) 0.42 (429) 

No children/elderly (omitted category) 0.46 (889) 0.46 (453) 0.46 (436) 

Household head gender 

Male (omitted category) 0.27 (503) 0.27 (257) 0.27 (246) [0.861] 

Female 0.73 (1451) 0.73 (723) 0.73 (728) 

Wave 

HIP start date Nov. 1, 2011 (omitted 

category) 0.37 (660) 0.39 (329) 0.37 (331) [0.507] 

HIP start date Dec. 1, 2011 0.36 (749) 0.36 (370) 0.36 (379) 

HIP start date Jan. 1, 2012 0.26 (545) 0.25 (281) 0.27 (264) 

AMPM interview characteristics (N=3955)a 

Recall interview 

First or only recall interview (omitted 

category) 0.89 (3521) 0.89 (1778) 0.89 (1743) [0.996] 

Second recall interview 0.11 (434) 0.11 (201) 0.11 (233) 

Day of intake 

Weekday recall interview (omitted 

category) 0.86 (3379) 0.85 (1680) 0.86 (1699) [0.449] 

Weekend recall interview 0.14 (576) 0.15 (299) 0.14 (277) 

Intake described relative to usual levels 

Same as usual (omitted category) 0.09 (368) 0.09 (175) 0.10 (193) [0.020]** 

More than usual 0.60 (2420) 0.62 (1226) 0.60 (1194) 

Less than usual 0.27 (1029) 0.26 (511) 0.27 (518) 

Missing (don't know or break-off) 0.04 (136) 0.03 (65) 0.04 (71) 

Respondent demographics 

Age group 

16-30 years 0.34 (585) 0.31 (278) 0.34 (307) [0.466] 

31-40 years 0.18 (373) 0.19 (182) 0.18 (191) 

41-54 years 0.24 (505) 0.26 (257) 0.24 (248) 

55+ years (omitted category) 0.24 (491) 0.24 (263) 0.24 (228) 

Gender 

Male (omitted category) 0.34 (618) 0.34 (307) 0.34 (311) [0.780] 

Female 0.66 (1336) 0.66 (673) 0.66 (663) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix E: Analytic Methods ▌pg. 273

 Total Treatment Control P-value 

Race/ethnicity 

Hispanic 0.43 (823) 0.42 (407) 0.43 (416) [0.781] 

Non-Hispanic white (omitted category) 0.37 (727) 0.38 (381) 0.37 (346) 

Non-Hispanic black 0.14 (279) 0.14 (134) 0.14 (145) 

Non-Hispanic other 0.07 (125) 0.06 (58) 0.07 (67) 

Baseline fruit & vegetable screener 

100% juice 1.58 (0.07) 1.45 (0.07) 1.60 (0.09) [0.180] 

Fruit 0.84 (0.04) 0.79 (0.04) 0.84 (0.04) [0.355] 

Salad 0.38 (0.02) 0.36 (0.02) 0.38 (0.02) [0.368] 

Fried potatoes 0.10 (0.01) 0.10 (0.01) 0.10 (0.01) [0.688] 

Other potatoes 0.28 (0.01) 0.27 (0.01) 0.28 (0.01) [0.688] 

Beans 0.26 (0.01) 0.26 (0.02) 0.25 (0.01) [0.676] 

Other vegetables 0.61 (0.02) 0.61 (0.03) 0.61 (0.03) [0.988] 

Tomato sauce 0.15 (0.01) 0.16 (0.01) 0.15 (0.01) [0.564] 

Salsa 0.01 (<0.01) 0.02 (<0.01) 0.01 (0.01) [0.416] 

Composite scales 

Fruits & vegetables available at home 4.01 (0.02) 3.97 (0.02) 4.02 (0.02) [0.135] 

Positive attitudes about food, fruits, & 

vegetables 3.88 (0.02) 3.87 (0.02) 3.88 (0.02) [0.744] 

Barriers to eating fruits & vegetables 2.42 (0.02) 2.44 (0.02) 2.41 (0.02) [0.318] 

Barriers to grocery shopping 2.15 (0.03) 2.17 (0.04) 2.15 (0.04) [0.698] 

Weighted proportions and unweighted Ns for categorical variables; means and standard deviations for 

continuous variables (fruit and vegetable screener and composite scales). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported proportions may not sum to one. 

a

Descriptive statistics for sample including both first and second day interviews. AMPM test statistics are 

adjusted for clustering at the individual respondent level. 

Source: DTA SNAP Caseload Data; Participant Survey (respondent module) (unweighted N=1,954). 

Ordinary least squares (OLS) regressions were performed in Stata using the standard svy suite of 

commands and respondent weights to account for the complex sampling structure. (We discuss the 

estimation of standard errors below in Section E.4.) Following the regression, the margins command 

was used to estimate regression-adjusted means and standard errors. 

E.2 Limited Dependent Variables 

Equation (E.1) presents our multivariate model for continuous outcomes—in particular, our single 

confirmatory outcome, TFV. However, some of our outcome variables are not continuous. For 

example, respondent reports of whether they had seen or heard messages about fruits and vegetables 

in the past three months yield a binary outcome (responses equal to either “yes” or “no”). In addition, 

many questions about attitudes and preferences (e.g., degree of agreement with statements like “I 

enjoy trying new foods”) are Likert scales, coded 1 to 5 (with 1 indicating “strongly disagree” and 5 

indicating “strongly agree,” for this example). Likert scales are ordinal (i.e., ordered), but not interval 

(i.e., the number itself is not meaningful). Complete distributions of our binary and ordered scale 

outcomes at baseline are reported in Appendix F. 

By analogy with our approach to the estimation of the impact of HIP for continuous outcomes, we 

also want a multivariate specification for binary and ordered outcomes. With respect to binary 

variables, there is some controversy in the literature about how to estimate impact in the random 

assignment context. Related issues are raised by ordered scales. Our preference is to use linear 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 274 ▌ Appendix E: Analytic Methods Abt Associates

regression for both binary and ordered scale outcomes (Angrist, 2001; Angrist and Pischke, 2008). 

This is our preference because we report impacts in terms of percentage points and thus prefer to do 

estimation in terms of percentage points. Failure to do so can result in a range of anomalous results 

(especially with subgroup analyses). 

Our primary specification thus simply treats binary and ordered scale outcomes as if they were 

continuous and applies linear regression. (In the case of binary outcomes, this approach is known as a 

“linear probability model” specification.) As is always true, this gives the best linear predictor—

where best is defined in a least squares sense, given the outcome (e.g., treating the ordinal outcomes 

as if they were interval). This approach gives a simple summary measure for whether HIP shifts the 

distribution. Again, we use robust standard errors to address deviations from distributional 

assumptions. 

Another approach is to estimate the corresponding canonical limited dependent variable models 

(Maddala, 1983). The argument for using the corresponding canonical limited dependent variable 

models is that the linear probability model cannot be the correct specification because it does not 

model the limited dependent variable nature of the data and that it can yield predictions out of the 

range of the dependent variable. 

For a binary outcome, the canonical model is logistic regression; i.e., 

(E.2)    h r i h h r i h,r,i ln odds y HIP ControlVars + ε , ,  0  1   2 , ,

where odds(y=1) indicates the odds that the binary outcome happened, ln is the natural logarithm 

operator, and  is assumed to have an extreme value distribution. Estimation proceeds by maximum 

likelihood. For ordered outcomes, the equivalent model is ordered logistic regression (see Maddala, 

1983, for a formal specification). In Appendix F of this report, we provided estimates using the 

canonical limited dependent variable models for non-continuous outcomes. Results using the 

corresponding canonicial limited dependent variable model were qualitatively similar to those using 

linear regression. 

E.3 Subgroup Analyses 

We performed subgroup analyses to test whether outcomes varied by baseline demographics, 

attitudes, and behaviors. All subgroup analyses are considered exploratory. Specifically, subgroup 

analyses proceed using a generalization of our earlier regression model: 

(E.3) 

h h h h h,r,i

h r i h r i

HIP Subgroup_ _Dummy HIP Subgroup_ _Dummy ε

y ControlVars

3 1 4 2

, , 0 2 , ,

  

 

 

 

Unless explicitly noted in the discussion, each of the subgroups is defined using only Round 1 

characteristics, so there is no endogenous selection (i.e., subgroup membership was not itself affected 

by HIP). In particular, Subgroup_1_Dummy is a binary variable (e.g., =1 if the primary shopper is 

employed, =0 otherwise), and Subgroup_2_Dummy is the reverse (e.g., =0 if the primary shopper is 

employed, =1 otherwise). Testing for 3=4 provides a test for differences across subgroups. 

In practice, we discuss sub-group results as follows. We begin by examining the results of the test for 

3=4. If we fail to reject (i.e., impacts do not significantly differ by subgroups), then we simply 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix E: Analytic Methods ▌pg. 275

report that there was no difference in impact across this subgroup. If we reject, we conclude that there 

was a difference in impact across the subgroups. We then discuss the estimated impact in each of the 

subgroups. 

E.4 Survey Weights, Standard Errors, and Confidence Intervals 

Unless otherwise noted, all analyses of survey data use person-level or household-level survey 

weights depending on the unit of analysis. As a result, the sample estimates provide unbiased 

estimates of the corresponding population statistics (for those who consent and complete the Round 1 

survey) in the pilot site. Construction of those survey weights was discussed in Appendix C. 

Standard errors and confidence intervals take account of the survey design, using the replicate 

sampling weights. All standard error estimates are robust to heteroscedasticity.173

The analysis uses all available interviews. For approximately 10 percent of the sample, there was a 

second interview at Round 2. For a different approximately 10 percent of the sample, there was a 

second interview at Round 3. For about 1 percent of the sample, there was a second interview at both 

Round 2 and Round 3, yielding a total of five interviews (baseline, two at Round 2 and two at Round 

3). As discussed below in Section E.8, this second interview in a given round of the survey allows the 

estimation of “usual intake.” 

Even analyses of non-usual intake use all of the available interviews. Specifically, the analyses 

reported here pool the first and (where available) second interviews from Round 2 and Round 3. 

Multiple interviews for a given household introduce non-independence into the reported outcomes. 

To address this non-independence, analysis proceeds by clustering on household, using the 

appropriate survey commands in Stata, with household specified as the clustering variable. 

E.5 Testing for Inter-Round Change 

We use a slightly different method to test for and estimate inter-round change. It is possible to test for 

inter-round change in the framework provided by Equation (E.3); i.e., to treat Round 2 and Round 3 

as subgroups. However, this approach ignores the fact that we have observations on the same 

households at Round 2 and Round 3. Given that we have observations on the same households at 

Round 2 and Round 3, a within estimator will be more efficient. By analogy to simple univariate 

statistics, Equation (E.3) is the equivalent of a simple t-test, with correct standard errors; however, it 

would be preferable to use a paired t-test. 

Specifically, we estimate equations of the form: 

(E.4) 

h h h h h h,r,i

h r i h r i

HIP Round_ _Dummy HIP Round_ _Dummy ε

y ControlVars

3 2 4 3

, , 0 2 , ,

   

 

  

  

Where the only change is that we specify the subgroup as Round 2/Round 3 and include a fixed 

effect, , that is common for all of the observations for a given household. Rather than estimating all 

of the fixed effects, estimation proceeds via a within transformation. 

 

173 Randomization does not guarantee homoscedasticity, and the linear probability model on binary outcomes 

induces heteroscedasticity. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 276 ▌ Appendix E: Analytic Methods Abt Associates

E.6 Multiple Comparisons 

Having a large number of hypothesis tests creates a danger of finding “false positives,” seemingly 

significant impacts when in fact the true impact of HIP is zero. For each hypothesis test, a 

conventional approach allows a 5 percent chance of incorrectly rejecting the null hypothesis (a Type I 

error) and concluding that an impact has occurred where none has. With more than one test, and 

especially with a large number of tests as is true in this evaluation, the risk of Type I error increases. 

Hence, it is recommended to identify a single confirmatory outcome ahead of time (Schochet, 2008). 

To address this multiple-comparisons problem, we have specified one “confirmatory” outcome for a 

study: the HIP/non-HIP difference in TFV intake, pooling on data from Rounds 2 and 3 of the 

participant survey, using regression adjustment for control variables. 

The Updated Study Plan described the following strategy for using the result of the test for 

significance of the confirmatory outcome in writing up all of the results: 

If the main confirmatory HIP/non-HIP difference is statistically significant at the 5 percent level, 

we will use the conventional approach to testing HIP/non-HIP differences for all outcomes and 

subgroups. In presenting results, we will describe analyses other than the main confirmatory 

outcome as “exploratory,” pointing out that occasional “significant” differences could appear 

simply due to sampling variation in multiple hypothesis tests. 

If the main confirmatory HIP/non-HIP difference is statistically insignificant at the 5 percent 

level, we will still use the conventional approach to testing differences for all outcomes and 

subgroups, but the accompanying discussion will warn that seemingly significant differences for 

particular outcomes and subgroups could be spurious. As before, the discussion will describe the 

analysis of these other outcomes and subgroups as exploratory. The Executive Summary and 

other summary documents will simply report that the HIP evaluation found no significant impact 

on the main outcome and not mention any of the exploratory results. 

E.7 Sensitivity to Outliers 

We conducted further analysis in order to confirm that significant findings were not due to the 

presence of outliers. We ran frequencies and univariate statistics for continuous outcomes on which 

HIP had a significant impact (29 variables total). Outliers were identified by examining the 

distributions and considering reasonable values for each outcome. Models were then re-run with 

outliers excluded. The vast majority of impacts were robust to the exclusion of outliers. However, the 

exclusion of outliers did alter the results in three cases (see Exhibits 8.5, 8.6, and 8.13). 

E.8 Usual Intake Estimation 

A single 24-hour dietary recall measures consumption at one point in time. However, intake estimates 

calculated based on a single day of recall data may not accurately represent long-term average intake 

for that individual, referred to as “usual intake.” The distribution of single-day intake has a larger 

variance than the distribution of usual intake because there is substantial variation in consumption 

patterns from day to day. 

For estimating impacts on mean intake levels, large day-to-day within-person variation does not pose 

a problem, as a simple comparison of means across subgroups is sufficient to obtain unbiased 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix E: Analytic Methods ▌pg. 277

estimates. However, estimating the proportion of the population with intake above or below some 

standard (e.g. USDA Food Pattern food group serving guidelines for fruit and vegetable intakes) 

based on a single day of recall data (or even a two-day average) will lead to biased estimates. The 

large day-to-day within-person variation will also lead to loss of statistical power in regression-based 

analyses. While multiple days of intake tend to be more representative of usual intakes of individuals, 

it is not practical to collect more than one day of intake on the entire sample proposed (without either 

dramatically increasing the cost of the study and/or reducing the sample size and power to detect 

differences in intake). 

There are, however, statistical methods for estimating usual intake for samples in which a subset of 

respondents report a second day of recall data (IOM, 2000a). Usual intake in these models is 

conceptualized as the probability of consumption on a given day times the average amount consumed 

on a “consumption day.” 

Our study incorporated collection of a second, nonconsecutive day of diary-assisted 24-hour dietary 

recall data for a 10 percent subsample of respondents. This strategy allowed us to employ standard 

statistical dietary assessment methodology to estimate the distribution of usual intake for our study 

population, yielding valid estimates of the prevalence of inadequate intake. 

According to current IOM dietary assessment guidance, for estimation of usual intake the number of 

replicate observations is more important than the proportion of replicate observations relative to the 

full sample. Nusser et al. (1996) recommend that replicate data be collected on not fewer than about 

50 or 60 subjects. IOM guidance notes that replicate subsamples consisting of fewer than 70 to 80 

individuals have been successfully used in the past to obtain usual intake estimates (IOM, 2000b). 

Our proposed representative second-day replicate 10 percent sample was intended to include at least 

200 respondents per round, 100 HIP participants and 100 non-HIP participants, far exceeding the 

recommended IOM recommendations for the number of replicate observations for obtaining usual 

intake estimates. In fact, by standard IOM guidance this replicate sample would be adequate for 

estimating usual intake for subgroups comprising ~38 percent or more of respondents completing 

second-day interviews, or approximately 75 respondents in the intervention group and 75 respondents 

in the control group. 

We estimated usual intake distributions based on the coded first- and second-day 24-hour dietary 

recall data employing methodology recently developed by the National Cancer Institute (NCI) in 

collaboration with staff at the USDA Center for Nutrition Policy and Promotion. The NCI method 

models usual intake as the product of the probability of consumption on a given day and the average 

amount consumed per consumption day. See Tooze et al. (2006) for a detailed description of the NCI 

method. 

Advantages of the NCI Method 

Like the Iowa State University (ISU) method, the previous standard for estimating usual intake 

(Nusser et al., 1996; Carriquiry, 2003), the NCI method takes into account reported zero-consumption 

days and reported consumption-day amounts that are positively skewed, and distinguishes between 

within-person and between-person variation in consumption. The NCI method has two advantages 

over the ISU method. First, it allows for correlation between amount and frequency of consumption, 

and permits the incorporation of covariates such as weekend indicators or supplementary information 

on frequency of fruit and vegetable consumption from a food frequency questionnaire (FFQ) or 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 278 ▌ Appendix E: Analytic Methods Abt Associates

similar instrument, which in some cases can improve the power to detect relationships between 

dietary intake and other variables (Subar et al., 2006). 

Second, unlike the ISU method, the NCI method allows for efficient estimation of usual intake for 

subgroups. Instead of stratifying the sample by subpopulation and estimating usual intake separately 

for each subgroup, we include covariates defining subgroups in the NCI model, such that covariate 

values differ across the subgroups, but the (harder to estimate) variance components are assumed 

common, and estimated from the full sample. For subgroups of respondents comprising a relatively 

small proportion of the full sample, the efficiency gains from this capability are likely to be 

substantial. 

Regularly and Episodically Consumed Dietary Components 

We produced usual intake estimates for two broad types of dietary components: 1) nutrients, 

including beta carotene, vitamin A, vitamin C, and fiber; and 2) foods and food groups, such as fruits, 

vegetables, and corresponding subgroups, and other food groups of interest, including those used to 

compute HEI-2010 scores. 

Nutrients are consumed by nearly every individual in the population on a daily basis. In contrast, 

some foods and food groups of interest may be consumed on only an episodic basis. For example, a 

respondent may not eat dark-green vegetables or citrus fruits every day. To estimate usual intake 

distributions for nutrients and other dietary components that are consumed regularly (non-zero 

consumption for at least 90 percent of sample respondents), we employed a version of the NCI 

method in which only the amount of consumption is estimated. To estimate usual intake distributions 

for episodically consumed dietary components such as fruits and vegetables, we employed a two-part 

model, in which both probability and amount of consumption are estimated. 

Estimation Procedures 

Our estimation procedures followed Tooze et al. (2006), using SAS macros supplied on the NCI 

website. In both the amount-only and the two-part models, the amount data were first transformed to 

approximate normality using the Box-Cox transformation. Then, using the transformed data, for each 

individual in the sample, we estimated a linear predictor of amount of consumption. Estimation 

proceeded via a generalized linear model with model covariates including respondent gender and age 

group, and measures of baseline consumption levels of fruits, leafy greens, and other vegetables from 

the EATS fruit and vegetable screener.174 Parameter estimates from this model were then used as 

starting values in a nonlinear mixed model with person-specific random effects. 

In the two-part models for episodically consumed dietary components (non-zero consumption for 10 

percent or more of the sample), probability of any consumption was additionally estimated via 

 

174 As a sensitivity check, we additionally performed test runs for one outcome with the full set of covariates 

used in the main impact analyses (respondent gender, race/ethnicity, age group; household characteristics 

including geographic location of residence, household size and composition, and gender of household head; 

AMPM interview characteristics (weekend vs. weekday; respondent assessment of intake relative to usual 

levels); measures of baseline consumption levels from the EATS fruit and vegetable screener; and baseline 

composite scales derived from questions about the home food environment, barriers to grocery shopping, 

and attitudes about and barriers to consumption of fruits and vegetables). As these runs did not result in any 

material differences in findings, we used the more limited set of covariates to improve processing speed. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix E: Analytic Methods ▌pg. 279

logistic regression, with the same set of model covariates as in the amount specification. Parameter 

estimates from the logistic regression were then used as starting values in a nonlinear mixed model 

with person-specific random effects. The probability model was then linked with the amount model 

by using the parameter estimates from the two uncorrelated specifications as starting values for a 

model in which the two person-specific random effects are permitted to be correlated. 

Next, Monte Carlo simulation was used to generate random effects for 100 pseudo-persons for each 

respondent in the original sample. The random effect was then added to the linear predictor for each 

pseudo-person, and the amount estimates were back-transformed to the original scale with Taylor 

linearization. Means and percentiles were estimated empirically from the resulting distribution. 

Assessing Overall Dietary Quality 

The 2010 Dietary Guidelines for Americans (DGA) establish recommendations for daily consumption 

by food group and subgroup, including fruits and vegetables. Standard usual intake procedures 

allowed us to estimate the proportion of respondents meeting these recommendations, in both the HIP 

and non-HIP groups. Comparing these two estimates allowed us to determine the impact of HIP on 

compliance with DGA recommendations. 

Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 281

Appendix F: Baseline Outcomes, Alternative Models, and Other 

Supplemental Tables 

This appendix contains supplemental tables, alternative models, and baseline differences on outcome 

variables. Exhibits are organized by the report chapter to which they correspond. 

F.1 Chapter 1 Supplemental Table: Definition of TFVs 

Exhibit F1.1: Minimum Requirements and Specifications for WIC Fruits and Vegetables 

Any variety of fresh whole or cut fruit without added sugars.a 

Any variety of fresh whole or cut vegetable, except white potatoes, without added sugars, fats, or oils (orange 

yams and sweet potatoes are allowed). a

Any variety of cannedb fruits (must conform to FDA standard of identity (21 CFR Part 145); including applesauce, 

juice pack or water pack without added sugars, fats, oils, or salt (i.e. sodium). Any variety of frozen fruits without 

added sugars.c

Any variety of cannedb or frozen vegetables (must conform to FDA standard of identity (21 CFR Part 155)) 

except white potatoes (orange yams and sweet potatoes are allowed); without added sugars, fats, or oils. May 

be regular or lower in sodium.c

Any type of dried fruits or dried vegetable without added sugars, fats, oils, or salt (i.e., sodium).a 

a

Herbs or spices; edible blossoms and flowers, e.g., squash blossoms (broccoli, cauliflower and artichokes are 

allowed); creamed or sauced vegetables; vegetable‐grain (pasta or rice) mixtures; fruit‐nut mixtures; breaded 

vegetables; fruits and vegetables for purchase on salad bars; peanuts; ornamental and decorative fruits and 

vegetables such as chili peppers on a string; garlic on a string; gourds; painted pumpkins; fruit baskets and party 

vegetable trays; and items such as blueberry muffins and other baked goods are not authorized. Mature legumes 

(dry beans and peas) and juices are not authorized. 

b

“Canned” refers to processed food items in cans or other shelf‐stable containers, e.g., jars, pouches. Home 

canned fruits and vegetables, such as those sold at Farmers Markets, are not allowable. c

Excludes white potatoes; catsup or other condiments; pickled vegetables, olives; soups; juices; and fruit leathers 

and fruit roll‐ups. 

Source: Reproduced from FNS Request for Application, Supplemental Nutrition Assistance Program (SNAP), 

Healthy Incentives Pilot (HIP), CFDA #:10.580, Figure 1. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 282 ▌Appendix F: Supplemental Tables Abt Associates

F.2 Chapter 2 Supplemental Tables: Baseline Characteristics and Balance 

Tests 

Chapter 2 supplemental tables include additional baseline characteristics of respondents completing 

follow-up surveys (Exhibits F2.1–F2.3) and additional HIP/non-HIP balance tests (Exhibit F2.4–

F2.8). 

Exhibit F2.1: Baseline Characteristics of Respondents Completing Follow-Up Participant 

Surveys (Round 2 or 3), by Treatment and Control Status 

Total Treatment Control P-value 

Age group 

16–30 years 0.34 (585) 0.31 (278) 0.34 (307) [0.466] 

31–40 years 0.18 (373) 0.19 (182) 0.18 (191) 

41–54 years 0.24 (505) 0.26 (257) 0.24 (248) 

55+ years 0.24 (491) 0.24 (263) 0.24 (228) 

Gender

Male 0.34 (618) 0.34 (307) 0.34 (311) [0.780] 

Female 0.66 (1336) 0.66 (673) 0.66 (663) 

Race/ethnicity 

Hispanic 0.43 (823) 0.42 (407) 0.43 (416) [0.781] 

Non-Hispanic white 0.37 (727) 0.38 (381) 0.36 (346) 

Non-Hispanic black 0.14 (279) 0.14 (134) 0.14 (145) 

Non-Hispanic other 0.07 (125) 0.06 (58) 0.07 (67) 

Disability status 

Disabled 0.49 (999) 0.50 (520) 0.49 (479) [0.721] 

Not disabled 0.51 (955) 0.50 (460) 0.51 (495) 

Citizenship 

US citizen 0.95 (1874) 0.96 (944) 0.95 (930) [0.435] 

Not a US citizen 0.05 (80) 0.04 (36) 0.05 (44) 

Unemployment compensation 

Receiving unemployment compensation 0.04 (84) 0.05 (50) 0.03 (34) [0.021]** 

Not receiving unemployment 

compensation 0.96 (1870) 0.95 (930) 0.97 (940) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported proportions may not sum to one. 

Source: DTA SNAP Caseload Data (unweighted N=1954). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 283

Exhibit F2.2: Self-Reported Baseline Characteristics of Respondents Completing Follow-Up 

Participant Surveys (Round 2 or 3), by Treatment and Control Status 

Total Treatment Control P-value 

Ethnicity (N=1947) 

Hispanic/Latino 0.51 (1023) 0.53 (523) 0.51 (500) [0.510] 

Not Hispanic/Latino 0.49 (924) 0.47 (452) 0.49 (472) 

Race (N=1904) a

White 0.47 (891) 0.47 (458) 0.47 (433) [0.998] 

Black or African American 0.16 (324) 0.17 (158) 0.16 (166) [0.514] 

Asian 0.01 (18) 0.01 (11) 0.01 (7) [0.346] 

Native Hawaiian or Pacific Islander 0.01 (17) 0.01 (12) 0.01 (5) [0.147] 

American Indian or Alaskan Native 0.06 (98) 0.05 (44) 0.06 (54) [0.358] 

Other race <0.01 (2) 0.00 (0) <0.01 (2) [0.575] 

Reports race as Hispanic only 0.34 (644) 0.34 (318) 0.34 (326) [0.848] 

Marital status (N=1953) 

Married 0.15 (285) 0.13 (134) 0.16 (151) [0.797] 

Not married but living with partner 0.08 (162) 0.08 (82) 0.08 (80) 

Widowed 0.07 (145) 0.07 (80) 0.06 (65) 

Divorced 0.16 (332) 0.17 (172) 0.16 (160) 

Separated 0.09 (189) 0.10 (97) 0.09 (92) 

Never Married 0.44 (840) 0.44 (414) 0.44 (426) 

Education level (N=1945) 

Never attended/kindergarten only 0.01 (24) 0.02 (14) 0.01 (10) [0.363] 

1st grade <0.01 (7) <0.01 (3) <0.01 (4) 

2nd grade <0.01 (10) <0.01 (5) <0.01 (5) 

3rd grade 0.01 (24) 0.01 (10) 0.02 (14) 

4th grade 0.02 (42) 0.03 (29) 0.01 (13) 

5th grade 0.01 (22) 0.01 (9) 0.01 (13) 

6th grade 0.02 (43) 0.02 (24) 0.02 (19) 

7th grade 0.02 (40) 0.02 (17) 0.02 (23) 

8th grade 0.03 (70) 0.04 (37) 0.03 (33) 

9th grade 0.07 (153) 0.09 (85) 0.07 (68) 

10th grade 0.07 (144) 0.08 (78) 0.07 (66) 

11th grade 0.11 (199) 0.09 (83) 0.11 (116) 

12th grade, no diploma 0.04 (87) 0.05 (50) 0.04 (37) 

High school graduate 0.20 (381) 0.18 (178) 0.21 (203) 

GED or equivalent 0.08 (158) 0.08 (74) 0.09 (84) 

Some college, no degree 0.14 (273) 0.14 (140) 0.13 (133) 

Associate degree: occ/tech/voc 0.06 (114) 0.06 (56) 0.06 (58) 

Associate degree: academic program 0.02 (50) 0.03 (28) 0.02 (22) 

Bachelor's degree 0.04 (71) 0.04 (36) 0.04 (35) 

Master's degree 0.01 (19) 0.01 (11) 0.01 (8) 

Professional school degree <0.01 (11) <0.01 (6) <0.01 (5) 

Doctoral degree <0.01 (3) <0.01 (1) <0.01 (2) 

Language spoken at home (N=1954) a

English 0.82 (1605) 0.83 (807) 0.82 (798) [0.679] 

Spanish 0.42 (808) 0.41 (402) 0.42 (406) [0.894] 

Other 0.06 (110) 0.05 (52) 0.06 (58) [0.270] 

Respondent interview language (N=1954)

English 0.76 (1480) 0.75 (738) 0.76 (742) [0.444] 

Spanish 0.24 (474) 0.25 (242) 0.24 (232) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 284 ▌Appendix F: Supplemental Tables Abt Associates

Total Treatment Control P-value 

Proxy needed for interview (N=1954) 

Proxy needed 0.02 (41) 0.02 (21) 0.02 (20) [0.840] 

No proxy needed 0.98 (1913) 0.98 (959) 0.98 (954) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

a

Respondents selected all options that applied; proportions therefore sum to more than one. 

Source: Participant Survey (respondent module, AMPM dietary recall module). 

Exhibit F2.3: Self-Reported Baseline Characteristics of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control Status 

Total Treatment Control P-value 

Employment status (N=1845) 

Working full-time 0.11 (190) 0.10 (87) 0.11 (103) [0.608] 

Working part-time 0.11 (189) 0.10 (89) 0.11 (100) 

Temporarily laid off 0.01 (15) 0.01 (6) 0.01 (9) 

Sick or maternity leave <0.01 (8) <0.01 (4) <0.01 (4) 

Looking for work 0.13 (233) 0.13 (118) 0.13 (115) 

Unemployed 0.06 (111) 0.06 (57) 0.06 (54) 

Retired 0.10 (199) 0.10 (101) 0.11 (98) 

Disabled, permanently or temporarily 0.30 (570) 0.31 (295) 0.29 (275) 

Keeping house 0.12 (227) 0.12 (116) 0.12 (111) 

Student 0.05 (103) 0.07 (60) 0.05 (43) 

Primary shopper interview language (N=1851)

English 0.77 (1430) 0.77 (726) 0.77 (704) [0.944] 

Spanish 0.23 (421) 0.23 (211) 0.23 (210) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know,” “refused,” and “not ascertained” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Source: Participant Survey (primary shopper module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix Exhibit F2.4: HIP/Non-HIP Balance from the SNAP Caseload Files, Disaggregated by Survey and Non-Survey Samples 

Number of households Retention rate 

HIP, nonsurvey 

(1) 

HIP, survey 

(2) 

Non-HIP, 

survey 

(3) 

Non-HIP, 

non-survey

(4) 

HIP, nonsurvey (%) 

(5) 

HIP, survey 

(%) 

(6) 

Non-HIP, 

survey (%) 

(7) 

Non-HIP

non-surve

(%) 

(8) 

September 2011 4,596 2,356 2,326 41,780 100.0 100.0 100.0 100.0 October 2011 4,462 2,270 2,266 40,855 97.1 96.3 97.4 97.8 November 2011 4,375 2,228 2,234 40,087 95.2 94.6 96.0 95.9 December 2011 4,302 2,192 2,210 39,311 93.6 93.0 95.0 94.1 January 2012 4,264 2,190 2,209 39,273 92.8 93.0 95.0 94.0 February 2012 4,220 2,175 2,187 38,865 91.8 92.3 94.0 93.0 March 2012 4,142 2,145 2,150 38,397 90.1 91.0 92.4 91.9 April 2012 4,137 2,112 2,118 37,958 90.0 89.6 91.1 90.9 May 2012 4,120 2,091 2,093 37,695 89.6 88.8 90.0 90.2 June 2012 4,078 2,075 2,082 37,404 88.7 88.1 89.5 89.5 July 2012 4,053 2,059 2,068 37,055 88.2 87.4 88.9 88.7 August 2012 3,994 2,039 2,003 36,358 86.9 86.5 86.1 87.0 September 2012 3,961 2,029 2,013 36,106 86.2 86.1 86.5 86.4 October 2012 3,889 1,962 1,964 35,445 84.6 83.3 84.4 84.8 

Source: DTA SNAP Caseload Data. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 286 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F2.5: HIP/Non-HIP Balance: Baseline Characteristics (July 2011) for Study 

Participants on SNAP in May 2012 

Variable 

Total 

(proportion) 

Treatment 

(proportion) 

Control 

(proportion) P-value 

Race/ethnicity of head 

Hispanic 0.44 0.45 0.44 [0.378] 

Non-Hispanic white 0.36 0.36 0.36 [0.176] 

Non-Hispanic black 0.13 0.13 0.13 [0.525] 

Non-Hispanic other 0.07 0.07 0.07 [0.998] 

Spanish spoken in household 0.23 0.23 0.23 [0.455] 

Age of head 

16-30 0.26 0.26 0.26 [0.372] 

31-40 0.21 0.21 0.21 [0.489] 

41-54 0.27 0.26 0.27 [0.167] 

Over 54 0.27 0.26 0.27 [0.895] 

Household head disabled 0.53 0.54 0.53 [0.635] 

Household head U.S. citizen 0.96 0.96 0.96 [0.331] 

Household composition 

Elderly (with or without children) in 

household 0.13 0.13 0.13 [0.524] 

Children (no elderly) in household 0.38 0.38 0.38 [0.892] 

No elderly or children in household 0.49 0.49 0.49 [0.763] 

Housing type 

Private 0.80 0.80 0.80 [0.521] 

Public 0.15 0.15 0.15 [0.275] 

Other 0.05 0.05 0.05 [0.546] 

Household is homeless 0.06 0.05 0.06 [0.806] 

Monthly household gross income 

$0 0.21 0.21 0.21 [0.495] 

$1-787 0.27 0.28 0.27* [0.067]* 

$788-1,082 0.27 0.27 0.27 [0.936] 

$1,083 or higher 0.25 0.24 0.25 [0.190] 

Types of income received by head 

SSI 0.36 0.36 0.36 [0.906] 

Social Security 0.30 0.30 0.30 [0.962] 

TANF 0.14 0.14 0.14 [0.929] 

Unemployment compensation 0.04 0.04 0.04 [0.357] 

Other unearned income 0.63 0.64 0.63 [0.203] 

SNAP monthly benefit amount 

$160 or less 0.25 0.26 0.25 [0.234] 

$161-$200 0.38 0.38 0.38 [0.538] 

$201-$349 0.11 0.11 0.11 [0.454] 

$350 or higher 0.26 0.25 0.26 [0.301] 

Sample size 45,955 6,204 39,751 

F-valuea 0.87 

P-valuea 0.658 

Two-sided t-test: *p<0.1, **p<0.05, ***p<0.01. 

a

Variables included in F-test, but not shown in table: Baystate combined application project (CAP) status for SSI 

recipients; recertification type (semiannual reporting, recertification, other). 

Source: DTA SNAP Caseload Data. 

 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 287

Exhibit F2.6: HIP/Non-HIP Balance: Baseline Characteristics (July 2011) for Study 

Participants on SNAP in October 2012 

Variable 

Total 

(proportion) 

Treatment 

(proportion) 

Control 

(proportion) P-value 

Race/ethnicity of head 

Hispanic 0.44 0.45 0.44 [0.827] 

Non-Hispanic white 0.36 0.36 0.36 [0.485] 

Non-Hispanic black 0.13 0.13 0.13 [0.757] 

Non-Hispanic other 0.07 0.07 0.07 [0.621] 

Spanish spoken in household 0.23 0.24 0.23 [0.730] 

Age of head 

16-30 0.25 0.26 0.25 [0.605] 

31-40 0.21 0.21 0.21 [0.434] 

41-54 0.27 0.27 0.27 [0.377] 

Over 54 0.27 0.27 0.27 [0.734] 

Household head disabled 0.54 0.55 0.54 [0.287] 

Household head U.S. citizen 0.96 0.96 0.96 [0.259] 

Household composition 

Elderly (with or without children) in 

household 0.13 0.13 0.14 [0.253] 

Children (no elderly) in household 0.39 0.38 0.39 [0.713] 

No elderly or children in household 0.48 0.49 0.48 [0.255] 

Housing type 

Private 0.80 0.80 0.80 [0.645] 

Public 0.15 0.15 0.15 [0.415] 

Other 0.05 0.05 0.05 [0.621] 

Household is homeless 0.05 0.05 0.05 [0.800] 

Monthly household gross income 

$0 0.21 0.20 0.21 [0.593] 

$1-787 0.27 0.28 0.27 [0.039]** 

$788-1,082 0.28 0.28 0.28 [0.942] 

$1,083 or higher 0.25 0.24 0.25 [0.090]* 

Types of income received by head 

SSI 0.37 0.37 0.37 [0.787] 

Social Security 0.30 0.30 0.30 [0.739] 

TANF 0.15 0.15 0.15 [0.942] 

Unemployment compensation 0.04 0.04 0.04 [0.231] 

Other unearned income 0.64 0.65 0.64 [0.156] 

SNAP monthly benefit amount 

$160 or less 0.26 0.26 0.25 [0.316] 

$161-$200 0.38 0.37 0.38 [0.547] 

$201-$349 0.11 0.11 0.11 [0.247] 

$350 or higher 0.26 0.25 0.26 [0.251] 

Sample size 43,207 5,842 37,365 

F-valuea 0.95 

P-valuea 0.542 

Two-sided t-test: *p<0.1, **p<0.05, ***p<0.01. 

a

Variables included in F-test, but not shown in table: Baystate combined application project (CAP) status for SSI 

recipients; recertification type (semiannual reporting, recertification, other). 

Source: DTA SNAP Caseload Data. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 288 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F2.7: HIP/Non-HIP Balance: May 2012 Characteristics for Study Participants on SNAP 

in May 2012 

Variable 

Total 

(proportion) 

Treatment 

(proportion) 

Control 

(proportion) P-value 

Race/ethnicity of head 

Hispanic 0.44 0.45 0.44 [0.333] 

Non-Hispanic white 0.36 0.35 0.36 [0.158] 

Non-Hispanic black 0.13 0.13 0.13 [0.408] 

Non-Hispanic other 0.07 0.07 0.07 [0.755] 

Spanish spoken in household 0.23 0.23 0.23 [0.522] 

Age of head 

16-30 0.23 0.24 0.23 [0.368] 

31-40 0.21 0.22 0.21 [0.282] 

41-54 0.27 0.26 0.27 [0.046]** 

Over 54 0.28 0.28 0.28 [0.878] 

Household head disabled 0.56 0.56 0.56 [0.749] 

Household head U.S. citizen 0.96 0.96 0.96 [0.619] 

Household composition 

Elderly (with or without children) in 

household 0.15 0.15 0.15 [0.890] 

Children (no elderly) in household 0.37 0.37 0.37 [0.899] 

No elderly or children in household 0.49 0.49 0.49 [0.980] 

Housing type 

Private 0.80 0.80 0.80 [0.340] 

Public 0.16 0.16 0.15 [0.410] 

Other 0.04 0.04 0.04 [0.683] 

Household is homeless 0.05 0.05 0.05 [0.404] 

Monthly household gross income 

$0 0.19 0.19 0.19 [0.919] 

$1-787 0.25 0.26 0.25 [0.034]** 

$788-1,082 0.29 0.29 0.29 [0.766] 

$1,083 or higher 0.27 0.26 0.27 [0.023]** 

Types of income received by head 

SSI 0.37 0.37 0.37 [0.758] 

Social Security 0.31 0.31 0.31 [0.990] 

TANF 0.13 0.13 0.13 [0.435] 

Unemployment compensation 0.03 0.03 0.03 [0.264] 

Other unearned income 0.65 0.65 0.65 [0.606] 

SNAP monthly benefit amount 

$160 or less 0.27 0.28 0.27 [0.219] 

$161-$200 0.37 0.37 0.37 [0.302] 

$201-$349 0.10 0.10 0.11 [0.366] 

$350 or higher 0.25 0.25 0.25 [0.600] 

Sample size 45,955 6,204 39,751 

F-valuea 0.95 

P-valuea 0.533 

Two-sided t-test: *p<0.1, **p<0.05, ***p<0.01. 

a

Variables included in F-test, but not shown in table: Baystate combined application project (CAP) status for SSI 

recipients; recertification type (semiannual reporting, recertification, other). 

Source: DTA SNAP Caseload Data. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 289

Exhibit F2.8 HIP/Non-HIP Balance: October 2012 Characteristics for Study Participants on 

SNAP in October 2012 

Variable 

Total 

(proportion) 

Treatment 

(proportion) 

Control 

(proportion) P-value 

Race/ethnicity of head 

Hispanic 0.44 0.45 0.44 [0.741] 

Non-Hispanic white 0.36 0.36 0.36 [0.419] 

Non-Hispanic black 0.13 0.13 0.13 [0.721] 

Non-Hispanic other 0.07 0.07 0.07 [0.678] 

Spanish spoken in household 0.23 0.23 0.23 [0.761] 

Age of head 

16-30 0.22 0.22 0.22 [0.503] 

31-40 0.21 0.22 0.21 [0.464] 

41-54 0.27 0.27 0.27 [0.385] 

Over 54 0.29 0.29 0.29 [0.675] 

Household head disabled 0.57 0.58 0.57 [0.242] 

Household head U.S. citizen 0.96 0.96 0.96 [0.210] 

Household composition 

Elderly (with or without children) in 

household 0.15 0.15 0.15 [0.751] 

Children (no elderly) in household 0.37 0.37 0.37 [0.922] 

No elderly or children in household 0.48 0.49 0.48 [0.748] 

Housing type 

Private 0.80 0.79 0.80 [0.489] 

Public 0.16 0.17 0.16 [0.336] 

Other 0.04 0.04 0.04 [0.696] 

Household is homeless 0.05 0.05 0.05 [0.840] 

Monthly household gross income 

$0 0.17 0.18 0.17 [0.555] 

$1-787 0.25 0.26 0.25 [0.395] 

$788-1,082 0.30 0.30 0.30 [0.597] 

$1,083 or higher 0.28 0.27 0.28 [0.062]* 

Types of income received by head 

SSI 0.39 0.39 0.39 [0.536] 

Social Security 0.32 0.33 0.32 [0.582] 

TANF 0.13 0.14 0.13 [0.147] 

Unemployment compensation 0.03 0.03 0.03 [0.539] 

Other unearned income 0.66 0.66 0.66 [0.334] 

SNAP monthly benefit amount 

$160 or less 0.28 0.28 0.28 [0.655] 

$161-$200 0.37 0.36 0.37 [0.707] 

$201-$349 0.10 0.10 0.10 [0.536] 

$350 or higher 0.25 0.25 0.25 [0.694] 

Sample size 43,207 5,842 37,365 

F-valuea 0.65 

P-valuea 0.921 

Two-sided t-test: *p<0.1, **p<0.05, ***p<0.01. 

a

Variables included in F-test, but not shown in table: Baystate combined application project (CAP) status for SSI 

recipients; recertification type (semiannual reporting, recertification, other). 

Source: DTA SNAP Caseload Data. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 290 ▌Appendix F: Supplemental Tables Abt Associates

F.3 Chapter 4 Supplemental Tables: Early and Late HIP Implementation 

Retailer Experiences 

Chapter 4 supplemental tables include tests for change between early and late HIP implementation in 

retailer experiences (Exhibits F4.1-F4.4) and more detailed reporting on problems and questions 

regarding HIP (Exhibits F4.5-F4.7). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix Exhibit F4.1: Retailer Promotion of Fruits and Vegetables, Overall and by Store Type, Early and Late HIP Implementation 

How often does your store 

promote fruits and/or 

vegetables using these 

activities? 

Overall Supermarket Grocery 

Early 

(%) 

Late 

(%) Change P-value 

Early 

(%) 

Late 

(%) Change P-value 

Early 

(%) 

Late 

(%) Change P-value 

Posters or signs inside the store Once a month or more 35.0 54.4 19.4 82.8 72.0 -10.9 54.5 30.0 -24.5 Less than once a month 37.1 18.0 -19.1 9.1 16.7 7.6 0.0 40.0 40.0 Never 27.9 27.7 -0.2 [0.031]** 8.1 11.4 3.3 [0.722] 45.5 30.0 -15.5 [0.026]** Shelf tags Once a month or more 39.3 53.1 13.8 91.9 72.0 -19.9 41.7 37.5 -4.2 Less than once a month 33.0 8.7 -24.3 8.1 9.8 1.8 16.7 37.5 20.8 Never 27.6 38.2 10.5 [0.003]*** 0.0 18.2 18.2 [0.174] 41.7 25.0 -16.7 [0.461] Posters or signs in store window or outside Once a month or more 22.9 45.1 22.2 55.6 57.6 2.0 33.3 12.5 -20.8 Less than once a month 11.2 17.9 6.7 8.1 19.7 11.6 16.7 37.5 20.8 Never 65.9 37.0 -28.8 [0.005]*** 36.4 22.7 -13.6 [0.476] 50.0 50.0 0.0 [0.357] Fliers/ads in newspaper or direct mail Once a month or more 19.7 25.5 5.8 63.9 69.7 5.8 8.3 0.0 -8.3 Less than once a month 16.2 16.4 0.2 36.1 23.5 -12.6 25.0 37.5 12.5 Never 64.1 58.1 -6.0 [0.723] 0.0 6.8 6.8 [0.403] 66.7 62.5 -4.2 [0.575] Recipes or fliers in store Once a month or more 14.5 19.3 4.8 45.5 56.3 10.8 0.0 0.0 0.0 Less than once a month 17.0 7.5 -9.5 38.4 10.1 -28.3 18.2 12.5 -5.7 Never 68.5 73.2 4.7 [0.262] 16.2 33.6 17.5 [0.110] 81.8 87.5 5.7 [0.710] Price or volume promotions Once a month or more 18.4 24.4 6.0 23.2 53.8 30.6 46.2 0.0 -46.2 Less than once a month 50.1 25.0 -25.2 60.6 23.5 -37.1 23.1 62.5 39.4 Never 31.5 50.6 19.2 [0.014]** 16.2 22.7 6.6 [0.059]* 30.8 37.5 6.7 [0.031]** Food samples Once a month or more 22.3 21.3 -1.0 63.9 62.2 -1.7 18.2 0.0 -18.2 Less than once a month 7.9 11.2 3.3 13.0 15.1 2.2 18.2 25.0 6.8 Never 69.8 67.5 -2.4 [0.822] 23.1 22.7 -0.5 [0.982] 63.6 75.0 11.4 [0.380] Coupons Once a month or more 18.4 17.7 -0.7 69.7 43.7 -26.0 0.0 0.0 0.0 Less than once a month 6.0 4.2 -1.8 6.1 0.0 0.0 18.2 28.6 10.4 Never 75.6 78.1 2.5 [0.900] 24.2 56.3 32.1 [0.105] 81.8 71.4 -10.4 [0.564] Unweighted number of storesa 39 49 8 14 14 12 

Weighted percents and unweighted total Ns. 
Evaluation of the Healthy In

pg. 292 ▌Appendix F: Supplemental Tables Chi-square test for categorical variables; *p<0.1, **p<0.05, ***p<0.01. Unpaired tests were conducted due to limited overlap in the early and late implementation samples. aN varies by question due to item non-response. 

Source: Early and Late Implementation Retailer Surveys.
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 293

Exhibit F4.2: Retailer Fresh Fruit and Vegetable Inventory and Average Price/Lb, Overall and 

by Store Type, Early and Late HIP Implementation Periods 

Percent available (%) and mean price per pound ($) 

Early 

Implementation 

Late 

Implementation Change P-value 

Apple 

Overall 

Available 81.0 80.6 -0.3 [0.961] 

Price per pound 1.64 1.57 -0.07 [0.603] 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 1.48 1.45 -0.03 [0.319] 

Grocery 

Available 61.5 75.0 13.5 [0.422] 

Price per pound 1.26 1.16 -0.10 [0.469] 

Convenience 

Available 81.0 73.7 -7.2 [0.478] 

Price per pound 1.90 1.82 -0.09 [0.254] 

Banana 

Overall 

Available 85.2 88.6 3.4 [0.550] 

Price per pound 1.10 1.06 -0.04 [0.618] 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 0.64 0.65 0.01 [0.726] 

Grocery 

Available 69.2 75.0 5.8 [0.720] 

Price per pound 0.83 0.89 0.06 [0.765] 

Convenience 

Available 85.7 89.8 4.1 [0.608] 

Price per pound 1.50 1.40 -0.10 [0.527] 

Orange 

Overall 

Available 76.4 74.1 -2.3 [0.750] 

Price per pound 1.33 1.35 0.02 [0.704] 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 1.46 1.32 -0.14 [0.791] 

Grocery 

Available 61.5 66.7 5.1 [0.766] 

Price per pound 0.86 1.08 0.22 [0.195] 

Convenience 

Available 71.4 63.9 -7.6 [0.506] 

Price per pound 1.45 1.47 0.02 [0.918] 

Grape 

Overall 

Available 40.9 49.3 8.4 [0.320] 

Price per pound 2.45 2.68 0.23 [0.380] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 294 ▌Appendix F: Supplemental Tables Abt Associates

Percent available (%) and mean price per pound ($) 

Early 

Implementation 

Late 

Implementation Change P-value 

Supermarket 

Available 100.0 100.0 0.0 

Price per pound 2.63 3.01 0.38 [0.249] 

Grocery 

Available 38.5 66.7 28.2 [0.115] 

Price per pound 2.39 2.05 -0.34 [0.466] 

Convenience 

Available 9.5 10.2 0.7 [0.927] 

Price per pound 1.49 2.34 0.85 [0.249] 

Carrot 

Overall 

Available 45.2 43.9 -1.3 [0.877] 

Price per pound 1.48 1.42 -0.07 [0.763] 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 1.43 1.40 -0.04 [0.978] 

Grocery 

Available 46.2 50.0 3.8 [0.830] 

Price per pound 1.69 1.28 -0.41 [0.340] 

Convenience 

Available 14.3 9.9 -4.4 [0.573] 

Price per pound 1.24 2.00 0.76 [0.145] 

Tomato 

Overall 

Available 53.7 57.9 4.2 [0.618] 

Price per pound 1.91 2.28 0.37 [0.200] 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 2.41 2.75 0.34 [0.651] 

Grocery 

Available 61.5 66.7 5.1 [0.766] 

Price per pound 1.32 1.62 0.31 [0.053]* 

Convenience 

Available 23.8 30.6 6.8 [0.535] 

Price per pound 1.57 1.90 0.32 [0.447] 

Broccoli 

Overall 

Available 38.9 40.6 1.7 [0.838] 

Price per pound 1.83 1.53 -0.31 [0.144] 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 1.85 1.54 -0.31 [0.310] 

Grocery 

Available 30.8 41.7 10.9 [0.525] 

Price per pound 1.86 1.49 -0.37 [0.402] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 295

Percent available (%) and mean price per pound ($) 

Early 

Implementation 

Late 

Implementation Change P-value 

Convenience 

Available 9.5 6.6 -3.0 [0.653] 

Price per pound 1.69 1.49 -0.20 [0.766] 

Lettuce 

Overall 

Available 51.7 57.7 5.9 [0.483] 

Price per pound 1.29 1.19 -0.10 [0.524] 

Supermarket 

Available 100.0 100.0 0.0 

Price per pound 1.31 1.14 -0.18 [0.513] 

Grocery 

Available 53.8 75.0 21.2 [0.220] 

Price per pound 1.25 1.25 0.00 [0.997] 

Convenience 

Available 23.8 23.8 0.0 [0.997] 

Price per pound 1.3 1.27 -0.03 [0.886] 

Unweighted number of storesa 

Overall 39 49 

Supermarket 8 14 

Grocery 14 12 

Convenience 17 23 

Weighted percents and means; unweighted total Ns. 

Chi-square test for categorical variables, t-test for continuous variables; *p<0.1, **p<0.05, ***p<0.01. 

Unpaired tests were conducted due to limited overlap in the early and late implementation samples. 

Early Implementation Survey conducted from October-December 2011; Late Implementation Survey conducted 

from November 2012-January 2013. 

a

N varies by question due to skip patterns and item non-response. 

Source: Early and Late Implementation Retailer Surveys. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 296 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F4.3: Retailer Canned Fruit and Vegetable Inventory and Average Price/Lb, Overall 

and by Store Type, Early and Late HIP Implementation Periods 

Percent available (%) and mean price per pound ($) 

Early 

Implementation 

Late 

Implementation Change P-value 

Tomato 

Overall 

Available 62.2 52.3 -9.9 [0.237] 

Price per pound 1.36 1.28 -0.08 [0.259] 

Supermarket 

Available 100.0 90.9 -9.1 [0.189] 

Price per pound 0.91 1.28 0.38 [0.214] 

Grocery 

Available 76.9 41.7 -35.3 [0.043]** 

Price per pound 1.82 1.00 -0.82 [0.006]*** 

Convenience 

Available 33.3 32.8 -0.5 [0.966] 

Price per pound 1.65 1.39 -0.26 [0.397] 

Corn 

Overall 

Available 62.5 74.6 12.1 [0.122] 

Price per pound 1.27 1.71 0.45 [0.044]** 

Supermarket 

Available 100.0 90.9 -9.1 [0.189] 

Price per pound 1.01 1.26 0.25 [0.080]* 

Grocery 

Available 69.2 66.7 -2.6 [0.878] 

Price per pound 1.30 1.37 0.08 [0.665] 

Convenience 

Available 38.1 67.8 29.7 [0.014]** 

Price per pound 1.61 2.18 0.57 [0.169] 

Pea 

Overall 

Available 62.5 60.5 -2.0 [0.807] 

Price per pound 1.27 1.44 0.17 [0.209] 

Supermarket 

Available 100.0 90.9 -9.1 [0.189] 

Price per pound 1.02 1.24 0.21 [0.155] 

Grocery 

Available 69.2 58.3 -10.9 [0.525] 

Price per pound 1.31 1.53 0.22 [0.184] 

Convenience 

Available 38.1 42.7 4.6 [0.699] 

Price per pound 1.59 1.66 0.07 [0.801] 

Applesauce 

Overall 

Available 45.5 63.2 17.8 [0.035]** 

Price per pound 1.21 1.54 0.33 [0.064]* 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 297

Percent available (%) and mean price per pound ($) 

Early 

Implementation 

Late 

Implementation Change P-value 

Supermarket 

Available 100.0 90.9 -9.1 [0.189] 

Price per pound 1.13 1.16 0.04 [0.945] 

Grocery 

Available 38.5 33.3 -5.1 [0.766] 

Price per pound 1.41 1.77 0.36 [0.284] 

Convenience 

Available 19.0 57.9 38.9 [0.001]*** 

Price per pound 1.24 1.92 0.68 [0.011]** 

Pineapple 

Overall 

Available 51.7 54.0 2.2 [0.791] 

Price per pound 1.34 1.49 0.14 [0.387] 

Supermarket 

Available 100.0 90.9 -9.1 [0.189] 

Price per pound 1.15 1.19 0.04 [0.973] 

Grocery 

Available 53.8 50.0 -3.8 [0.830] 

Price per pound 1.65 1.83 0.18 [0.407] 

Convenience 

Available 23.8 32.8 9.0 [0.409] 

Price per pound 1.41 1.76 0.35 [0.163] 

Raisins 

Overall 

Available 49.2 47.4 -1.8 [0.836] 

Price per pound 3.15 3.60 0.46 [0.384] 

Supermarket 

Available 100.0 90.9 -9.1 [0.189] 

Price per pound 2.63 3.53 0.90 [0.058]* 

Grocery 

Available 61.5 41.7 -19.9 [0.266] 

Price per pound 3.25 3.10 -0.15 [0.811] 

Convenience 

Available 14.3 23.0 8.7 [0.359] 

Price per pound 4.94 4.23 -0.71 [0.535] 

Unweighted number of storesa 

Overall 39 49 

Supermarket 8 14 

Grocery 14 12 

Convenience 17 23 

Weighted percents and means; unweighted total Ns. 

Chi-square test for categorical variables, t-test for continuous variables; *p<0.1, **p<0.05, ***p<0.01. 

Unpaired tests were conducted due to limited overlap in the early and late implementation samples. 

Early Implementation Survey conducted from October-December 2011; Late Implementation Survey conducted 

from November 2012-January 2013. 

a

N varies by question due to skip patterns and item non-response. 

Source: Early and Late Implementation Retailer Surveys. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 298 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F4.4: Retailer Frozen Fruit and Vegetable Inventory and Average Price/Lb, Overall 

and by Store Type, Early and Late HIP Implementation Periods 

Percent available (%) and mean price per pound ($) 

Early 

Implementation 

Late 

Implementation Change P-value 

Strawberries 

Overall 

Available 30.5 29.5 -1.0 [0.892] 

Price per pound 2.98 2.77 -0.20 [0.903] 

Supermarket 

Available 84.3 90.9 6.6 [0.521] 

Price per pound 2.78 2.76 -0.02 [0.299] 

Grocery 

Available 23.1 8.3 -14.7 [0.268] 

Price per pound 3.69 2.99 -0.70 [0.647] 

Convenience 

Available 4.8 0.0 -4.8 [0.186] 

Price per pound 2.99 NA NA NA 

Peaches 

Overall 

Available 20.3 18.6 -1.8 [0.790] 

Price per pound 3.45 3.25 -0.20 [0.795] 

Supermarket 

Available 69.4 60.6 -8.8 [0.561] 

Price per pound 3.59 3.25 -0.34 [0.762] 

Grocery 

Available 7.7 0.0 -7.7 [0.289] 

Price per pound 2.16 NA NA NA 

Convenience 

Available 0.0 0.0 0.0 NA 

Price per pound NA NA NA NA 

Beans 

Overall 

Available 37.0 35.7 -1.2 [0.883] 

Price per pound 1.78 1.48 -0.31 [0.084]* 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 1.51 1.46 -0.05 [0.876] 

Grocery 

Available 23.1 25.0 1.9 [0.900] 

Price per pound 2.59 1.87 -0.72 [0.250] 

Convenience 

Available 9.5 3.3 -6.2 [0.287] 

Price per pound 2.32 0.99 -1.33 [0.385] 

Corn 

Overall 

Available 38.7 40.6 2.0 [0.811] 

Price per pound 1.67 1.60 -0.07 [0.595] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 299

Percent available (%) and mean price per pound ($) 

Early 

Implementation 

Late 

Implementation Change P-value 

Supermarket 

Available 100.0 95.5 -4.5 [0.360] 

Price per pound 1.44 1.55 0.11 [0.356] 

Grocery 

Available 38.5 33.3 -5.1 [0.766] 

Price per pound 2.27 1.96 -0.31 [0.427] 

Convenience 

Available 4.8 9.9 5.1 [0.423] 

Price per pound 1.79 1.49 -0.30 [0.788] 

Unweighted number of storesa 

Overall 39 49 

Supermarket 8 14 

Grocery 14 12 

 Convenience 17 23 

Weighted percents and means; unweighted total Ns. 

Chi-square test for categorical variables, t-test for continuous variables; *p<0.1, **p<0.05, ***p<0.01. 

Unpaired tests were conducted due to limited overlap in the early and late implementation samples. 

NA = all data was missing due to non-response or skip patterns; p-values could not be calculated. 

Early Implementation Survey conducted from October-December 2011; Late Implementation Survey conducted 

from November 2012-January 2013. 

a

N varies by question due to skip patterns and item non-response. 

Source: Early and Late Implementation Retailer Surveys. 

Exhibit F4.5: Retailer Problems with HIP in the Prior 3 Months, by Grocery and Convenience 

Stores, Late HIP Implementation Period 

Grocery 

(%) 

Convenience

(%) 

Problems knowing what foods are HIP-eligible? 

Never 83.3 64.7 

Once 8.3 5.9 

A few times 8.3 29.4 

Frequently 0.0 0.0 

Problems knowing HIP-eligible foods resolved? 

Yes, resolved 16.7 29.4 

No, not resolved 0.0 5.9 

No problem 83.3 64.7 

Problems having current list of HIP-eligible items in cash 

registers? 

Never 100.0 93.8 

Once 0.0 0.0 

A few times 0.0 6.3 

Frequently 0.0 0.0 

Problems having current list of HIP-eligible items resolved? 

Yes, resolved 0.0 6.3 

No, not resolved 0.0 0.0 

No problem 100.0 93.8 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 300 ▌Appendix F: Supplemental Tables Abt Associates

Grocery 

(%) 

Convenience

(%) 

Problems separating HIP-eligible from non-HIP-eligible foods? 

Never 91.7 81.3 

Once 0.0 0.0 

A few times 8.3 18.8 

Frequently 0.0 0.0 

Problems separating HIP-eligible from non-HIP eligible resolved? 

Yes, resolved 8.3 12.5 

No, not resolved 0.0 0.0 

Missing resolution 0.0 6.3 

No problem 91.7 81.3 

Problems identifying HIP customers? 

Never 75.0 64.7 

Once 0.0 5.9 

A few times 16.7 11.8 

Frequently 8.3 17.6 

Problems identifying HIP customers resolved? 

Yes, resolved 8.3 17.6 

No, not resolved 8.3 11.8 

Missing resolution 8.3 5.9 

No problem 75.0 64.7 

Problems computing the purchase amount for HIP items? 

Never 100.0 100.0 

Once 0.0 0.0 

A few times 0.0 0.0 

Frequently 0.0 0.0 

Problems computing the amount for HIP items resolved? 

Yes, resolved 0.0 0.0 

No, not resolved 0.0 0.0 

No problem 100.0 100.0 

Problems processing sales of HIP items? 

Never 100.0 94.1 

Once 0.0 0.0 

A few times 0.0 5.9 

Frequently 0.0 0.0 

Problems processing sales resolved? 

Yes, resolved 0.0 0.0 

No, not resolved 0.0 0.0 

Missing resolution 0.0 5.9 

No problem 100.0 94.1 

Problems processing returns with HIP items? 

Never 91.7 100.0 

Once 0.0 0.0 

A few times 8.3 0.0 

Frequently 0.0 0.0 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 301

Grocery 

(%) 

Convenience

(%) 

Problems processing returns resolved? 

Yes, resolved 0.0 0.0 

No, not resolved 0.0 0.0 

Missing resolution 8.3 0.0 

No problem 91.7 100.0 

Problems processing manual vouchers with HIP items? 

Never 100.0 54.5 

Once 0.0 0.0 

A few times 0.0 0.0 

Frequently 0.0 0.0 

Not applicable 0.0 45.5 

Problems processing manual vouchers resolved? 

Yes, resolved 0.0 0.0 

No, not resolved 0.0 0.0 

No problem 100.0 54.5 

Not applicable 0.0 45.5 

Problems getting information about SNAP/EBT sales and settlement? 

Never 83.3 64.7 

Once 0.0 0.0 

A few times 0.0 0.0 

Frequently 0.0 0.0 

Missing frequency 8.3 0.0 

Don't know 8.3 35.3 

Problems getting information about sales and settlement resolved? 

Yes, resolved 8.3 0.0 

No, not resolved 0.0 0.0 

No problem 83.3 64.7 

Don't know 8.3 35.3 

Problems responding to customer questions about HIP? 

Never 100.0 100.0 

Once 0.0 0.0 

A few times 0.0 0.0 

Frequently 0.0 0.0 

Problems responding to customer questions resolved? 

Yes, resolved 0.0 0.0 

No, not resolved 0.0 0.0 

No problem 100.0 100.0 

Unweighted number of stores a 12 17 

Weighted percents. 

a N varies by question due to item non-response and skip patterns. 

Source: Late Implementation Retailer Survey (Independent stores only). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 302 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F4.6: Retailer Requests for Information from Others, Overall and by Store Type, Late 

HIP Implementation Period 

How often have you asked for information 

from another employee in your store or 

outside your store in the past 3 

months...? 

Overall 

(%) 

Supermarket

(%) 

Grocery 

(%) 

Convenience

(%) 

Knowing what food items are eligible for HIP 

Never in the past 3 months 62.2 75.8 50.0 58.5 

1-2 times in the past 3 months 18.0 0.0 25.0 26.5 

3 or more times in the past 3 months 9.5 14.4 16.7 3.4 

Don't know 10.4 9.8 8.3 11.6 

Having a current list of HIP eligible items in cash registers 

Never in the past 3 months 64.4 73.1 50.0 65.3 

1-2 times in the past 3 months 16.6 5.0 16.7 23.1 

3 or more times in the past 3 months 8.3 10.9 25.0 0.0 

Don't know 10.7 10.9 8.3 11.6 

Separating HIP food items from non-HIP food items 

Never in the past 3 months 58.5 75.8 50.0 50.3 

1-2 times in the past 3 months 21.8 4.5 33.3 28.5 

3 or more times in the past 3 months 10.7 9.8 16.7 8.8 

Don't know 9.0 9.8 0.0 12.4 

How to identify HIP customers 

Never in the past 3 months 59.8 75.8 58.3 50.0 

1-2 times in the past 3 months 18.3 0.0 16.7 31.0 

3 or more times in the past 3 months 9.9 9.8 25.0 3.5 

Don't know 12.0 14.4 0.0 15.5 

Computing subtotal for HIP items 

Never in the past 3 months 67.4 71.2 66.7 65.3 

1-2 times in the past 3 months 17.5 9.1 25.0 19.8 

3 or more times in the past 3 months 4.7 9.8 8.3 0.0 

Don't know 10.4 9.8 0.0 15.0 

Processing sales with HIP items 

Never in the past 3 months 64.8 75.8 66.7 57.1 

1-2 times in the past 3 months 16.1 4.5 25.0 19.8 

3 or more times in the past 3 months 10.4 9.8 8.3 11.6 

Don't know 8.7 9.8 0.0 11.6 

Processing returns of HIP items 

Never in the past 3 months 68.3 75.8 66.7 64.1 

1-2 times in the past 3 months 14.9 0.0 25.0 20.4 

3 or more times in the past 3 months 4.8 9.8 0.0 3.5 

Don't know 12.0 14.4 8.3 12.0 

Processing manual vouchers with HIP items 

Never in the past 3 months 68.3 75.8 66.7 64.1 

1-2 times in the past 3 months 13.2 0.0 25.0 16.9 

3 or more times in the past 3 months 3.1 9.8 0.0 0.0 

Don't know 15.4 14.4 8.3 19.0 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 303

How often have you asked for information 

from another employee in your store or 

outside your store in the past 3 

months...? 

Overall 

(%) 

Supermarket

(%) 

Grocery 

(%) 

Convenience

(%) 

Getting information about SNAP/EBT sales 

Never in the past 3 months 70.0 75.8 75.0 64.1 

1-2 times in the past 3 months 16.3 4.5 25.0 20.4 

3 or more times in the past 3 months 3.1 9.8 0.0 0.0 

Don't know 10.6 9.8 0.0 15.5 

Responding to customer questions about HIP 

Never in the past 3 months 50.4 40.2 66.7 50.3 

1-2 times in the past 3 months 29.8 30.3 25.0 31.3 

3 or more times in the past 3 months 11.1 19.7 8.3 6.8 

Don't know 8.7 9.8 0.0 11.6 

Unweighted number of stores a 49 14 12 23 

Weighted percents and unweighted total Ns. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 

Exhibit F4.7: Questions to Retailers from Other Employees, Overall and by Store Type, Late 

HIP Implementation Period 

How often other employees asked you 

questions in the past 3 months...? 

Overall 

(%) 

Supermarket

(%) 

Grocery 

(%) 

Convenience

(%) 

Knowing what food items are eligible for HIP 

Never in the past 3 months 60.9 90.9 41.7 50.0 

1-2 times in the past 3 months 28.6 9.1 50.0 32.2 

3 or more times in the past 3 months 3.3 0.0 8.3 3.3 

Don't know 7.2 0.0 0.0 14.5 

Having a current list of HIP eligible items in cash registers 

Never in the past 3 months 72.1 88.6 50.0 70.4 

1-2 times in the past 3 months 14.0 6.8 25.0 14.1 

3 or more times in the past 3 months 5.1 0.0 16.7 3.5 

Don't know 8.9 4.5 8.3 12.0 

Separating HIP food items from non-HIP food items 

Never in the past 3 months 75.2 95.5 58.3 69.0 

1-2 times in the past 3 months 17.3 4.5 33.3 19.0 

3 or more times in the past 3 months 1.7 0.0 8.3 0.0 

Don't know 5.8 0.0 0.0 12.0 

How to identify HIP customers 

Never in the past 3 months 75.0 100.0 58.3 65.5 

1-2 times in the past 3 months 14.2 0.0 33.3 15.5 

3 or more times in the past 3 months 3.4 0.0 8.3 3.5 

Don't know 7.5 0.0 0.0 15.5 

Computing subtotal for HIP items 

Never in the past 3 months 79.3 95.5 66.7 74.0 

1-2 times in the past 3 months 11.6 4.5 25.0 10.5 

3 or more times in the past 3 months 3.4 0.0 8.3 3.5 

Don't know 5.8 0.0 0.0 12.0 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 304 ▌Appendix F: Supplemental Tables Abt Associates

How often other employees asked you 

questions in the past 3 months...? 

Overall 

(%) 

Supermarket

(%) 

Grocery 

(%) 

Convenience

(%) 

Processing sales with HIP items 

Never in the past 3 months 71.2 95.5 58.3 60.5 

1-2 times in the past 3 months 13.9 4.5 25.0 15.5 

3 or more times in the past 3 months 9.1 0.0 16.7 12.0 

Don't know 5.8 0.0 0.0 12.0 

Processing returns of HIP items 

Never in the past 3 months 79.1 100.0 50.0 77.5 

1-2 times in the past 3 months 11.8 0.0 41.7 7.0 

3 or more times in the past 3 months 1.7 0.0 0.0 3.5 

Don't know 7.5 0.0 8.3 12.0 

Processing manual vouchers with HIP items 

Never in the past 3 months 77.6 95.5 50.0 77.5 

1-2 times in the past 3 months 11.8 0.0 41.7 7.0 

3 or more times in the past 3 months 1.7 0.0 0.0 3.5 

Don't know 8.9 4.5 8.3 12.0 

Getting information about SNAP/EBT sales 

Never in the past 3 months 81.0 90.2 66.7 81.0 

1-2 times in the past 3 months 13.2 9.8 33.3 7.0 

3 or more times in the past 3 months 0.0 0.0 0.0 0.0 

Don't know 5.8 0.0 0.0 12.0 

Responding to customer questions about HIP 

Never in the past 3 months 63.2 75.8 58.3 57.0 

1-2 times in the past 3 months 24.5 14.4 33.3 27.5 

3 or more times in the past 3 months 6.5 9.8 8.3 3.5 

Don't know 5.8 0.0 0.0 12.0 

Unweighted number of stores a 49 14 12 23 

Weighted percents and unweighted total Ns. 

a N varies by question due to item non-response. 

Source: Late Implementation Retailer Survey. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 305

F.4 Chapter 5 Supplemental Table: TFV Purchases by Store Type 

Exhibit F5.1: Distribution of HIP Households by TFV Purchases 

Retailer type 

Never 

shopped at 

participating 

stores 

(%) 

HIP households that shopped at participating stores 

Total 

TFV = $0 (%) 

(%) 

$0<TFV<=$6

(%) 

$6<TFV<=$12 

(%) 

TFV > $12

(%) 

Pooled 

(March–October 2012) 

All participating retailers 17.00 17.36 15.00 14.63 36.01 100.00 

Supermarkets and 

superstores 23.20 11.65 14.94 14.67 35.56 100.00 

Convenience 84.63 14.74 0.38 0.13 0.10 100.00 

Grocery 84.86 14.32 0.32 0.17 0.33 100.00 

Other 99.55 0.28 0.02 0.05 0.12 100.00 

Round 2 

(March–July 2012) 

All participating retailers 17.45 16.97 14.99 14.60 36.01 100.00 

Supermarkets and 

superstores 

24.05 10.80 14.85 14.63 35.69 100.00 

Convenience 84.13 15.19 0.45 0.10 0.11 100.00 

Grocery 85.06 14.43 0.18 0.12 0.21 100.00 

Other 99.84 0.08 <0.01 0.03 0.05 100.00 

Round 3 

(August–November 2012) 

All participating retailers 16.55 17.75 15.02 14.66 36.01 100.00 

Supermarkets and 

superstores 

22.36 12.50 15.03 14.70% 35.43 100.00 

Convenience 85.13 14.28 0.32 0.16 0.08 100.00 

Grocery 84.67 14.20 0.45 0.22 0.45 100.00 

Other 99.25 0.48 0.03 0.06 0.18 100.00 

Change: Round 3 – 

Round 2 [P-value] 

All retailers -0.90 

[0.004]*** 

0.78 

[0.021]** 

0.03 

 [0.923] 

0.06 

[0.860] 

0.01 

[0.987] 

Supermarkets and 

superstores 

-1.69 

[<0.001]*** 

1.70 

[<0.001]*** 

0.17 

 [0.601] 

0.07 

 [0.832] 

-0.26 

[0.499] 

Convenience—IECR and 

non-IECR 

1.00 

[0.001]*** 

-0.91 

[0.002]*** 

-0.13 

[0.026]** 

0.06 

 [0.121] 

-0.03 

[0.357] 

Grocery -0.38 

 [0.163] 

-0.23 

 [0.408] 

0.26 

[<0.001]*** 

0.10 

 [0.011]** 

0.24 

[<0.001]***

Other -0.58 

[<0.001]*** 

0.40 

[<0.001]*** 

0.03 

[0.057]* 

0.03 

[0.203] 

0.13 

[0.001]*** 

F-test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported percentages may not sum to 100 and reported changes may differ from the 

differences between the means for Round 2 and Round 3. 

Source: EBT Transaction Data (6,214 HIP participant households). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 306 ▌Appendix F: Supplemental Tables

F.5 Chapter 6 Supplemental Tables: Baseline Outcomes, Alternative 

Models, Farmers Market Purchases 

Chapter 6 supplemental tables include baseline differences in outcome variables, alternative models 

(e.g., logistic regression models), and impacts on purchases at farmers markets. Tables are ordered 

according to the order of the tables in the main body of the report to which they correspond. 

Exhibit F6.1: Baseline Self-Reported Monthly Expenditures of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control Status 

Usual monthly spending for... Total Treatment Control P-value 

Groceries using only SNAP (N=1786) 278.33 (5.38) 273.12 (5.84) 279.14 (6.26) [0.506] 

Groceries not using SNAP (N=1753) 158.72 (4.97) 155.62 (5.03) 159.21 (5.71) [0.640] 

Food items a (N=1718) 110.24 (4.58) 104.39 (4.28) 111.16 (5.26) [0.320] 

Nonfood items (N=1718) 49.53 (1.85) 50.75 (2.78) 49.34 (2.10) [0.686] 

Restaurants (N=1776) 36.60 (1.59) 36.22 (1.67) 36.66 (1.83) [0.859] 

All fruits and vegetablesb (N=1667) 71.19 (2.11) 71.20 (2.15) 71.19 (2.43) [0.997] 

Weighted means (standard errors). 

Two-sided t-test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded 

as missing. 

a

Calculated as grocery expenditures not using SNAP minus expenditures on nonfood items. Due to missing data, 

the weighted means of expenditures on food and nonfood items do not total to the weighted mean of 

expenditures on groceries not using SNAP. 

b

Purchased with SNAP and with other forms of payment. 

Source: Participant Survey (primary shopper module). 

Exhibit F6.2: Baseline Usual Grocery Shopping Store Type of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3) by Treatment and Control Status 

Total Treatment Control P-value 

Large chain grocery store or supermarket 0.80 (1473) 0.81 (749) 0.80 (724) [0.478] 

Natural or organic supermarket (such as 

Whole Foods Market) 0.01 (10) 0.01 (5) 0.01 (5) 

Small local store or corner store 0.03 (47) 0.03 (25) 0.03 (22) 

Convenience store (such as 7-11 or mini 

market) <0.01 (5) <0.01 (2) <0.01 (3) 

Warehouse or club store (such as Sam’s 

Club or Costco) 0.01 (29) 0.02 (20) 0.01 (9) 

Discount superstore (such as Walmart) 0.14 (256) 0.13 (121) 0.14 (135) 

Ethnic market 0.01 (10) <0.01 (4) 0.01 (6) 

Farmers market/co-op <0.01 (6) <0.01 (4) <0.01 (2) 

Other <0.01 (3) <0.01 (2) <0.01 (1) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Source: Participant Survey (primary shopper module); (unweighted N=1,839). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix Exhibit F6.3: Impact of HIP on Usual Grocery Store Type, Logistic Regression Model 

Usual place to shop 

Predicted Probability (S.E.) Impact 

Treatment Control Odds Ratio [S.E.] {

Large chain grocery store or supermarket 0.840 (0.012) 0.842 (0.011) 0.988 [0.111] Natural or organic supermarket (such as Whole Foods Market) 0.001 (0.001) <0.001 (<0.001) 1.259 [0.813] Small local store or corner store 0.009 (0.003) 0.012 (0.003) 0.733 [0.217] Convenience store (such as 7-11 or mini market)a N/A N/A N/A N/A Warehouse club store (such as Sam”s Club or Costco 0.010 (0.003) 0.013 (0.003) 0.773 [0.223] Discount superstore (such as Walmart) 0.095 (0.009) 0.086 (0.008) 1.104 [0.146] Ethnic market <0.001 (<0.001) <0.001 (<0.001) 0.725 [0.505] Farmers market/co-op a N/A N/A N/A N/A Some other location a N/A N/A N/A N/A 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. Standard errors and test statistics are adjusted for clustering at the individual respondent level. aInestimable (covariates perfectly predict success or failure within one or more stratum). 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (unweighted N=3,298).
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 308 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F6.4: Baseline Reasons for Choice of Usual Grocery Shopping Place of Primary 

Shoppers Completing Follow-Up Participant Surveys (Round 2 or 3), by 

Treatment and Control Status 

Total Treatment Control P-value 

Close to home 0.32 (617) 0.36 (335) 0.31 (282) [0.032]** 

Close to work 0.01 (12) 0.01 (5) 0.01 (7) [0.617] 

Close to some other location 0.01 (8) <0.01 (2) 0.01 (6) [0.174] 

Easy to get there 0.02 (29) 0.01 (14) 0.02 (15) [0.781] 

Disability accessible 0.01 (8) <0.01 (3) 0.01 (5) [0.305] 

Hours of operation convenient <0.01 (5) <0.01 (4) <0.01 (1) [0.188] 

One stop shopping 0.01 (18) 0.01 (5) 0.01 (13) [0.096]* 

Bulk purchases 0.01 (28) 0.02 (15) 0.01 (13) [0.923] 

Prices/affordability 0.59 (1061) 0.56 (519) 0.60 (542) [0.109] 

Sales/promotions in store 0.17 (322) 0.18 (170) 0.17 (152) [0.547] 

EBT card accepted 0.01 (13) 0.01 (8) 0.01 (5) [0.498] 

Variety of products 0.27 (490) 0.26 (239) 0.28 (251) [0.494] 

Ethnic foods are available 0.02 (38) 0.02 (16) 0.02 (22) [0.460] 

Preferred products are available 0.13 (234) 0.12 (114) 0.13 (120) [0.569] 

Quality 0.03 (42) 0.02 (17) 0.03 (25) [0.083]* 

Produce better or fresher 0.19 (342) 0.19 (170) 0.19 (172) [0.807] 

Good service 0.03 (64) 0.04 (35) 0.03 (29) [0.610] 

Clean 0.02 (37) 0.02 (16) 0.02 (21) [0.353] 

Familiarity with store 0.07 (130) 0.08 (70) 0.06 (60) [0.288] 

Some other reason 0.02 (32) 0.02 (17) 0.02 (15) [0.840] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know,” “refused,” and “inapplicable” responses coded as missing. 

Respondents could choose multiple reasons, so proportions may not add to one. 

Source: Participant Survey (primary shopper module); (unweighted N=1,831). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix Exhibit F6.5: Impact of HIP on Reasons for Choice of Usual Grocery Store Type, Logistic Regression Model 

Usually shop there because... 

Predicted Probability (S.E.) Impact 

Treatment Control Odds Ratio [S.E.] {

Close to home 0.288 (0.014) 0.304 (0.014) 0.928 [0.087] Close to work 0.001 (0.001) <0.001 (<0.001) 1.413 [1.086] Close to some other location <0.001 (<0.001) <0.001 (<0.001) 0.840 [0.699] Easy to get there 0.010 (0.003) 0.009 (0.002) 1.216 [0.354] Disability accessible a N/A N/A N/A N/A Hours of operation convenient 0.001 (0.001) 0.001 (0.001) 0.792 [0.357] One stop shopping 0.050 (0.006) 0.049 (0.006) 1.035 [0.171] Bulk purchases 0.013 (0.004) 0.015 (0.003) 0.888 [0.256] Prices/affordability 0.479 (0.016) 0.513 (0.015) 0.874 [0.076] Sales/promotions in store 0.086 (0.008) 0.113 (0.009) 0.737 [0.092] EBT card accepted 0.013 (0.004) 0.007 (0.002) 1.952 [0.569] Variety of products 0.233 (0.012) 0.189 (0.011) 1.308 [0.125] Ethnic foods are available <0.001 (<0.001) <0.001 (<0.001) 0.956 [0.457] Preferred products are available 0.075 (0.008) 0.108 (0.008) 0.676 [0.087] Quality 0.033 (0.005) 0.041 (0.005) 0.805 [0.146] Produce better or fresher 0.136 (0.010) 0.112 (0.009) 1.257 [0.151] Good service 0.016 (0.004) 0.010 (0.003) 1.508 [0.412] Clean 0.010 (0.003) 0.009 (0.002) 1.118 [0.386] Familiarity with store 0.054 (0.008) 0.043 (0.006) 1.267 [0.233] Some other reason 0.001 (0.001) 0.001 (0.001) 1.094 [0.648] 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. Binary outcomes, 1=Yes, 0=No; “don’t know” responses coded as missing. Standard errors and test statistics are adjusted for clustering at the individual respondent level. aInestimable (covariates perfectly predict success or failure within one or more stratum). 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (unweighted N=3281). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 310 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F6.6: Baseline Grocery Shopping Frequency of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control Status 

Total Treatment Control P-value 

Yearly or not at all <0.01 (1) <0.01 (1) 0.00 (0) [0.522] 

2-3 times a year <0.01 (2) 0.00 (0) <0.01 (2) 

Every other month <0.01 (6) <0.01 (4) <0.01 (2) 

Once a month 0.28 (534) 0.29 (273) 0.28 (261) 

Every other week 0.36 (639) 0.34 (319) 0.36 (320) 

Once a week 0.22 (413) 0.23 (219) 0.21 (194) 

More than once a week 0.14 (253) 0.13 (121) 0.14 (132) 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Source: Participant Survey (primary shopper module); (unweighted N=1,848). 

Exhibit F6.7: Baseline Probability of Going Out of Way/Making Special Effort to Shop at 

Particular Store for Fruits and Vegetables of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control Status 

Total Treatment Control P-value 

Go out of way to shop for FV at particular 

store 0.51 (945) 0.52 (485) 0.51 (460) [0.685] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Source: Participant Survey (primary shopper module); (unweighted N=1,834). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix Exhibit F6.8: Impact of HIP on Grocery Shopping Frequency, Logistic Regression 

Predicted probability (S.E.) Impact 

Treatment Control Odds Ratio [S.E.] {

Grocery shopping frequency 0.332 (0.017) 0.330 (0.016) 1.009 [0.106] 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. Binary outcome, 1=Once a week or more, 0=less than once a week; “don’t know” and “refused” responses coded as missing. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (unweighted N=3,311). Exhibit F6.9: Impact of HIP on Grocery Shopping Frequency, Ordered Logistic Regression 

Regression-adjusted mean (S.E.) Impact 

Treatment Control T-C [S.E.] {

Grocery shopping frequency 5.197 (0.705) 5.188 (0.705) 0.009 [0.078] 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. Categorical outcome, 1=yearly or not at all, 2=2 to 3 times a year, 3=every other month, 4=once a month, 5=every other week, 6=once a week, 7=

know” and “refused” responses coded as missing. Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment anStandard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (unweighted N=3,311). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 312 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F6.10: Impact of HIP on Going Out of Way/Making Special Effort to Shop at Particular 

Store for Fruits and Vegetables, Logistic Regression Model 

Predicted Probability 

(S.E.) Impact 

Treatment Control 

Odds 

Ratio [S.E.] 

{tstatistic} (P-value) 

Go out of way to shop for FV 

at particular store 

0.378 

(0.015) 

0.391 

(0.014) 0.947 [0.082] {-0.634} (0.526) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcome, 1=yes, 0=no; “don’t know” and “refused” responses coded as missing. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (unweighted 

N=3,279). 

Exhibit F6.11: Descriptive Results for EBT Analysis After Implementation, Disaggregated 

According to November 2011 (Pre-Implementation) Shopping Behavior 

Variable HIP Non-HIP Difference P-value

Average number of households 

Pre-HIP participating store shoppers 2342 15028 

Pre-HIP non-participating store shoppers 1,674 10,802 

Mean household size 

Pre-HIP participating store shoppers 2.00 2.04 -0.04 [0.199] 

Pre-HIP non-participating store shoppers 1.89 1.90 -0.01 [0.862] 

Location: Springfield 

Pre-HIP participating store shoppers (%) 51.89 52.29 -0.42 [0.713] 

Pre-HIP non-participating store shoppers (%) 53.42 53.41 0.01 [0.995] 

Location: Chicopee & Holyoke 

Pre-HIP participating store shoppers (%) 29.92 29.66 0.26 [0.792] 

Pre-HIP non-participating store shoppers (%) 18.57 18.56 0.02 [0.988] 

Location: Other

Pre-HIP participating store shoppers (%) 18.19 18.05 0.14 [0.870] 

Pre-HIP non-participating store shoppers (%) 28.01 28.03 -0.02 [0.984] 

SNAP benefits 

Pre-HIP participating store shoppers ($) 264.01 265.21 -1.20 [0.742] 

Pre-HIP non-participating store shoppers ($) 255.25 253.97 1.29 [0.768] 

SNAP purchases 

Pre-HIP participating store shoppers ($) 268.59 264.96 3.63 [0.328] 

Pre-HIP non-participating store shoppers ($) 257.55 252.80 4.74 [0.282] 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported differences may differ from differences between means for the treatment and control 

groups. 

Pre-HIP participating store shoppers are Wave 2 and Wave 3 households who spent at least 50 percent of their 

November 2011 (pre-implementation) SNAP benefits in retailers that later participated in HIP during Round 2 

(post-implementation). Pre-HIP non-participating store shoppers are Wave 2 and Wave 3 households who spent 

less than 50 percent of their November 2011 (pre-implementation) SNAP benefits in retailers that later 

participated in HIP during Round 2 (post-implementation). 

Source: EBT Transaction Data, pooled across March-October 2012 (average of 29,846 households per month). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 313

Exhibit F6.12: Impact of HIP on Mean Monthly SNAP Purchases at Farmers Markets

Regression-adjusted 

mean (SE) Impact 

Purchases ($) 

Treatment

(T) ($) 

Control 

(C) ($) T-C ($) [S.E.] ($) {t-statistic} (P-value) 

All farmers markets 0.03 

(0.01) 

0.03 

(<0.01) <0.01 0.01 (0.123) (0.902) 

Participating farmers 

markets 

0.03 

(0.01) 

0.02 

(<0.01) <0.01 0.01 (0.442) (0.659) 

Non-participating farmers 

markets 

0.01 

(<0.01) 

0.01 

(<0.01) <0.01 <0.01 (-0.619) (0.536) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the household level. 

Means include households that made no SNAP purchases at farmers markets. 

Source: EBT Transaction Data, pooled across March-October 2012 (average of 45,912 households per month). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 314 ▌Appendix F: Supplemental Tables Abt Associates

F.6 Chapter 7 Supplemental Tables: Baseline Outcomes and Alternative 

Models 

Chapter 7 supplemental tables include baseline differences between treatment and control groups on 

the outcome variables (Exhibits F7.1-F7.4), followed by alternative models (i.e., logistic regression 

and ordered logistic regression models; Exhibits F7.5-F7.13). 

Exhibit F7.1: Baseline Food Preferences and Beliefs of Respondents Completing Follow-Up 

Participant Surveys (Round 2 or 3), by Treatment and Control Status 

How much do you agree or disagree 

that... Total Treatment Control P-value 

I enjoy trying new foods (N=1946) 

Strongly disagree 0.01 (24) 0.01 (14) 0.01 (10) [0.628] 

Disagree 0.09 (177) 0.09 (90) 0.09 (87) 

Neither disagree nor agree 0.05 (110) 0.06 (62) 0.05 (48) 

Agree 0.65 (1269) 0.64 (636) 0.66 (633) 

Strongly agree 0.19 (366) 0.18 (176) 0.19 (190) 

I enjoy trying new fruits (N=1949) 

Strongly disagree 0.01 (21) 0.01 (8) 0.01 (13) [0.098]* 

Disagree 0.07 (154) 0.09 (88) 0.07 (66) 

Neither disagree nor agree 0.05 (87) 0.04 (37) 0.05 (50) 

Agree 0.64 (1245) 0.64 (627) 0.64 (618) 

Strongly agree 0.23 (442) 0.23 (219) 0.23 (223) 

I enjoy trying new vegetables (N=1950)

Strongly disagree 0.03 (41) 0.01 (15) 0.03 (26) [0.109] 

Disagree 0.16 (304) 0.19 (158) 0.15 (146) 

Neither disagree nor agree 0.04 (70) 0.03 (26) 0.05 (44) 

Agree 0.61 (1214) 0.61 (619) 0.61 (595) 

Strongly agree 0.16 (321) 0.16 (162) 0.16 (159) 

I eat enough fruits to keep my healthy (N=1951)

Strongly disagree 0.01 (21) 0.01 (7) 0.01 (14) [0.478] 

Disagree 0.13 (257) 0.13 (130) 0.13 (127) 

Neither disagree nor agree 0.04 (75) 0.04 (41) 0.03 (34) 

Agree 0.64 (1247) 0.65 (631) 0.64 (616) 

Strongly agree 0.18 (351) 0.17 (169) 0.18 (182) 

I eat enough vegetables to keep me healthy (N=1949)

Strongly disagree 0.02 (36) 0.01 (13) 0.02 (23) [0.391] 

Disagree 0.13 (265) 0.14 (136) 0.13 (129) 

Neither disagree nor agree 0.04 (75) 0.03 (31) 0.04 (44) 

Agree 0.63 (1228) 0.63 (624) 0.63 (604) 

Strongly agree 0.18 (345) 0.17 (174) 0.18 (171) 

I often encourage family/friends to eat fruits and vegetables (N=1933)

Strongly disagree 0.02 (29) 0.02 (15) 0.02 (14) [0.924] 

Disagree 0.10 (181) 0.10 (88) 0.10 (93) 

Neither disagree nor agree 0.03 (69) 0.04 (38) 0.03 (31) 

Agree 0.61 (1170) 0.60 (582) 0.61 (588) 

Strongly agree 0.24 (484) 0.25 (247) 0.24 (237) 

Composite scale—positive attitudes 

about food, fruits, & vegetables 

(N=1954) 

3.88 (0.02) 3.87 (0.02) 3.88 (0.02) [0.744] 

Weighted proportions and unweighted Ns for categorical variables; mean and standard deviation for composite 

scale. 

Two-sided chi-square test for categorical variables, t-test for composite scale: *p<0.1, **p<0.05, ***p<0.01. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 315

“Don’t know,” “refused,” and “does not apply” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Source: Participant Survey (respondent module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 316 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F7.2: Baseline Perceived Barriers to Fruit & Vegetable Consumption of Respondents 

Completing Follow-Up Participant Surveys (Round 2 or 3), by Treatment and 

Control Status 

How much do you agree or disagree 

that... Total Treatment Control P-value 

It’s hard for me to eat more vegetables because I don’t know how to prepare them (N=1939) 

Strongly disagree 0.18 (349) 0.17 (179) 0.18 (170) [0.878] 

Disagree 0.50 (969) 0.50 (487) 0.50 (482) 

Neither disagree nor agree 0.04 (87) 0.05 (46) 0.04 (41) 

Agree 0.23 (434) 0.22 (211) 0.23 (223) 

Strongly agree 0.05 (100) 0.06 (53) 0.05 (47) 

It’s hard for me to eat more vegetables because they are hard to find where I shop for food (N=1940)

Strongly disagree 0.19 (348) 0.17 (163) 0.19 (185) [0.089]* 

Disagree 0.62 (1186) 0.61 (590) 0.62 (596) 

Neither disagree nor agree 0.02 (38) 0.02 (15) 0.02 (23) 

Agree 0.15 (324) 0.18 (182) 0.14 (142) 

Strongly agree 0.02 (44) 0.03 (26) 0.02 (18) 

It’s hard for me to eat more fruits because they are hard to find where I shop for food (N=1940)

Strongly disagree 0.19 (353) 0.17 (164) 0.19 (189) [0.044]** 

Disagree 0.63 (1219) 0.63 (615) 0.63 (604) 

Neither disagree nor agree 0.02 (34) 0.01 (13) 0.02 (21) 

Agree 0.14 (286) 0.15 (152) 0.14 (134) 

Strongly agree 0.02 (48) 0.03 (31) 0.02 (17) 

I don’t eat fruits and vegetables as much as I like to because they cost too much (N=1944) 

Strongly disagree 0.09 (158) 0.08 (73) 0.09 (85) [0.704] 

Disagree 0.39 (758) 0.41 (390) 0.39 (368) 

Neither disagree nor agree 0.04 (71) 0.03 (30) 0.04 (41) 

Agree 0.38 (751) 0.38 (380) 0.38 (371) 

Strongly agree 0.10 (206) 0.11 (105) 0.10 (101) 

I don’t eat fruits and vegetables as much as I like to because they often spoil before I get a chance to

eat them (N=1941) 

Strongly disagree 0.07 (137) 0.07 (72) 0.07 (65) [0.590] 

Disagree 0.45 (877) 0.46 (445) 0.45 (432) 

Neither disagree nor agree 0.05 (85) 0.04 (35) 0.05 (50) 

Agree 0.37 (720) 0.37 (360) 0.37 (360) 

Strongly agree 0.06 (122) 0.07 (61) 0.06 (61) 

I don’t eat fruits and vegetables as much as I like to because my family doesn’t like them (N=1886)

Strongly disagree 0.15 (288) 0.15 (142) 0.16 (146) [0.969] 

Disagree 0.69 (1304) 0.70 (669) 0.68 (635) 

Neither disagree nor agree 0.02 (39) 0.02 (19) 0.02 (20) 

Agree 0.12 (219) 0.11 (106) 0.12 (113) 

Strongly agree 0.02 (36) 0.02 (18) 0.02 (18) 

I don’t eat fruits and vegetables because I don’t like them (N=1947)

Strongly disagree 0.23 (434) 0.21 (204) 0.24 (230) [0.355] 

Disagree 0.64 (1292) 0.68 (669) 0.64 (623) 

Neither disagree nor agree 0.02 (30) 0.02 (15) 0.02 (15) 

Agree 0.09 (165) 0.08 (76) 0.09 (89) 

Strongly agree 0.01 (26) 0.01 (11) 0.01 (15) 

Composite scale—barriers to eating 

fruits & vegetables (N=1954) 2.42 (0.02) 2.44 (0.02) 2.41 (0.02) [0.318] 

Weighted proportions and unweighted Ns for categorical variables; mean and standard deviation for composite 

scale. 

Two-sided chi-square test for categorical variables, t-test for composite scale: *p<0.1, **p<0.05, ***p<0.01. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 317

“Don’t know,” “refused,” and “does not apply” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Source: Participant Survey (respondent module). 

Exhibit F7.3: Baseline Barriers to Grocery Shopping of Primary Shoppers Completing 

Follow-Up Participant Surveys (Round 2 or 3), by Treatment and Control Status 

How often kept from grocery shopping 

by... Total Treatment Control P-value 

Limited transportation (N=1831) 

Never 0.42 (765) 0.42 (383) 0.42 (382) [0.415] 

Rarely 0.17 (288) 0.15 (143) 0.17 (145) 

Sometimes 0.24 (444) 0.24 (227) 0.24 (217) 

Most of the time 0.11 (193) 0.10 (95) 0.11 (98) 

Always 0.07 (141) 0.09 (80) 0.07 (61) 

Distance to grocery store (N=1819) 

Never 0.50 (905) 0.49 (452) 0.50 (453) [0.150] 

Rarely 0.13 (252) 0.15 (141) 0.12 (111) 

Sometimes 0.24 (401) 0.20 (187) 0.24 (214) 

Most of the time 0.08 (157) 0.09 (84) 0.08 (73) 

Always 0.06 (104) 0.06 (57) 0.06 (47) 

Composite scale—Barriers to grocery 

shopping (N=1933) 2.15 (0.03) 2.17 (0.04) 2.15 (0.04) [0.698] 

Weighted proportions and unweighted Ns for categorical variables; mean and standard deviation for composite 

scale. 

Two-sided chi-square test for categorical variables, t-test for composite scale: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

Source: Participant Survey (primary shopper module). 

Exhibit F7.4: Baseline Family Food Environment of Primary Shoppers Completing Follow-Up 

Participant Surveys (Round 2 or 3), by Treatment and Control Status 

How often do you have... Total Treatment Control P-value 

Fruits available at home (N=1845)

Never 0.01 (18) 0.01 (12) 0.01 (6) [0.116] 

Rarely 0.04 (68) 0.04 (34) 0.04 (34) 

Sometimes 0.20 (389) 0.23 (219) 0.19 (170) 

Most of the time 0.34 (622) 0.33 (308) 0.34 (314) 

Always 0.42 (748) 0.39 (361) 0.43 (387) 

Fruits in the refrigerator or on the kitchen counter (N=1844)

Never/No refrigerator 0.02 (44) 0.02 (24) 0.02 (20) [0.141] 

Rarely 0.05 (101) 0.06 (60) 0.04 (41) 

Sometimes 0.27 (516) 0.29 (278) 0.26 (238) 

Most of the time 0.36 (633) 0.33 (301) 0.37 (332) 

Always 0.30 (550) 0.29 (272) 0.30 (278) 

Vegetables available at home (N=1844) a

Never 0.01 (21) 0.01 (12) 0.01 (9) [0.854] 

Rarely 0.02 (38) 0.02 (16) 0.02 (22) 

Sometimes 0.12 (227) 0.13 (119) 0.12 (108) 

Most of the time 0.24 (430) 0.23 (215) 0.24 (215) 

Always 0.61 (1128) 0.61 (572) 0.61 (556) 

Ready to eat vegetables in the refrigerator or on the kitchen counter (N=1845)

Never 0.06 (113) 0.06 (57) 0.06 (56) [0.638] 

Rarely 0.08 (151) 0.09 (82) 0.07 (69) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 318 ▌Appendix F: Supplemental Tables Abt Associates

How often do you have... Total Treatment Control P-value 

Sometimes 0.29 (527) 0.27 (261) 0.29 (266) 

Most of the time 0.28 (511) 0.28 (261) 0.28 (250) 

Always 0.29 (543) 0.29 (274) 0.29 (269) 

Composite scale—fruits & vegetables 

available at home (N=1933) 4.01 (0.02) 3.97 (0.02) 4.02 (0.02) [0.135] 

Salty snacks such as chips and crackers available at home (N=1848)

Never 0.07 (150) 0.08 (81) 0.07 (69) [0.631] 

Rarely 0.17 (311) 0.17 (160) 0.17 (151) 

Sometimes 0.37 (659) 0.35 (323) 0.38 (336) 

Most of the time 0.16 (313) 0.17 (160) 0.16 (153) 

Always 0.22 (415) 0.23 (211) 0.22 (204) 

1% fat, skim or fat-free milk available at home (N=1838)

Never 0.32 (566) 0.29 (276) 0.32 (290) [0.200] 

Rarely 0.05 (95) 0.06 (55) 0.05 (40) 

Sometimes 0.11 (191) 0.10 (91) 0.11 (100) 

Most of the time 0.09 (160) 0.08 (72) 0.09 (88) 

Always 0.44 (826) 0.48 (434) 0.44 (392) 

Soft drinks, fruit-flavored drinks, or fruit punch available at home (N=1845)

Never 0.15 (289) 0.15 (152) 0.15 (137) [0.276] 

Rarely 0.15 (283) 0.17 (157) 0.14 (126) 

Sometimes 0.31 (536) 0.27 (251) 0.31 (285) 

Most of the time 0.15 (275) 0.15 (141) 0.15 (134) 

Always 0.25 (462) 0.25 (232) 0.25 (230) 

All or most of your family sit down and eat evening meals together at home during the past month

(N=1354) b

Never 0.03(49) 0.04 (26) 0.03 (23) [0.199] 

Rarely 0.08 (105) 0.07 (49) 0.09 (56) 

Sometimes 0.20 (293) 0.23 (157) 0.19 (136) 

Most of the time 0.31 (429) 0.32 (221) 0.30 (208) 

Always 0.38 (478) 0.33 (226) 0.38 (252) 

Evening meals cooked at home during the past month (N=1845)

Never 0.01 (16) 0.01 (7) 0.01 (9) [0.882] 

Rarely 0.02 (26) 0.01 (12) 0.02 (14) 

Sometimes 0.09 (170) 0.10 (91) 0.09 (79) 

Most of the time 0.30 (576) 0.31 (297) 0.30 (279) 

Always 0.58 (1057) 0.57 (526) 0.58 (531) 

Weighted proportions and unweighted Ns for categorical variables; mean and standard deviation for composite 

scale. 

Two-sided chi-square test for categorical variables, t-test for composite scale: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported proportions may not sum to one. 

a

“No refrigerator or freezer” responses coded as missing. 

b

Asked only in households with more than one member. 

Source: Participant Survey (primary shopper module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 319

Exhibit F7.5: Impact of HIP on Self-Reported Exposure to Nutrition Education and Promotion 

in Past 3 Months, Logistic Regression Model 

Predicted probability 

(S.E.) Impact 

Treatment

(T) 

Control 

(C) 

Odds 

Ratio [S.E.] 

{tstatistic} (P-value) 

Heard or seen messages 

about fruits & vegetables 

(N=3387) 

0.768 

(0.013) 

0.690 

(0.013) 1.490 [0.139] {4.282} (<0.001)***

Attended nutrition education 

class or program (N=3363) 

0.086 

(0.009) 

0.093 

(0.008) 0.921 [0.127] {-0.600} (0.549) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 

Exhibit F7.6: Impact of HIP on Food Preferences & Beliefs, Ordered Logistic Regression 

Model 

Regression-adjusted 

mean (S.E.) Impact 

How much do you agree 

or disagree that...? 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value)

I enjoy trying new foods 

(N=3381) 

5.921 

(0.610) 

5.815 

(0.601) 0.106 [0.084] {1.272} (0.204)* 

I enjoy trying new fruits 

(N=3388) 

5.907 

(0.678) 

6.013 

(0.665) -0.107 [0.086] {-1.245} (0.213) 

I enjoy trying new 

vegetables (N=3392) 

6.432 

(0.629) 

6.417 

(0.619) 0.015 [0.084] {0.178} (0.859) 

I eat enough fruits to keep 

me healthy (N=3383) 

5.567 

(0.603) 

5.515 

(0.597) 0.052 [0.085] {0.618} (0.537) 

I eat enough vegetables to 

keep me healthy (N=3393) 

5.336 

(0.631) 

5.267 

(0.619) 0.069 [0.087] {0.794} (0.427) 

I often encourage 

family/friends to eat fruits & 

veg. (N=3346) 

5.443 

(0.627) 

5.385 

(0.618) 0.058 [0.083] {0.703} (0.482) 

Composite scale—Positive 

attitudes about food, fruits, 

& vegetables (N=3480) 

8.727 

(0.592) 

8.607 

(0.583) 0.120 [0.076] {1.579} (0.115) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly 

agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 320 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F7.7: Impact of HIP on Food Preferences & Beliefs, Logistic Regression Model 

Predicted probability 

(S.E.) Impact 

How much do you agree 

or disagree that...? 

Treatment 

(T) 

Control 

(C) 

Odds 

Ratio [S.E.] 

{tstatistic} (P-value) 

I enjoy trying new foods 

(N=3381) 

0.862 

(0.011) 

0.840 

(0.012) 1.189 [0.144] {1.427} (0.154) 

I enjoy trying new fruits 

(N=3386) 

0.870 

(0.011) 

0.877 

(0.010) 0.935 [0.113] {-0.556} (0.579) 

I enjoy trying new 

vegetables (N=3388) 

0.799 

(0.013) 

0.777 

(0.013) 1.145 [0.125] {1.239} (0.216) 

I eat enough fruits to keep 

me healthy (N=3383) 

0.838 

(0.012) 

0.813 

(0.012) 1.189 [0.131] {1.571} (0.116) 

I eat enough vegetables to 

keep me healthy (N=3393) 

0.840 

(0.013) 

0.821 

(0.012) 1.141 [0.134] {1.121} (0.262) 

I often encourage 

family/friends to eat fruits & 

veg. (N=3346) 

0.863 

(0.011) 

0.852 

(0.011) 1.090 [0.132] {0.718} (0.473) 

Composite scale—Positive 

attitudes about food, fruits, 

& vegetables (N=3480) 

0.517 

(0.016) 

0.468 

(0.015) 1.218 [0.109] {2.204} (0.028)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes: 1=strongly agree or agree, 0=neither agree nor disagree, disagree, or strongly disagree; “don’t 

know” and “does not apply” responses coded as missing. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 321

Exhibit F7.8: Impact of HIP on Perceived Barriers to Fruit & Vegetable Consumption, 

Ordered Logistic Regression Model 

Regression-adjusted mean 

(S.E.) Impact 

How much do you agree 

or disagree that...? 

Treatment 

(T) 

Control 

(C) T-C [S.E.] 

{tstatistic} (P-value) 

Hard to eat vegetables 

because don't know how to 

prepare (N=3366) 

1.478 

(0.524) 

1.475 

(0.512) 0.003 [0.079] {0.040} (0.968) 

Hard to eat vegetables 

because hard to find where 

I shop (N=3369) 

3.392 

(0.573) 

3.459 

(0.569) -0.067 [0.085] {-0.792} (0.428) 

Hard to eat fruits because 

hard to find where I shop 

(N=3372) 

3.097 

(0.610) 

3.069 

(0.604) 0.028 [0.089] {0.315} (0.753) 

Don't eat FV as much as 

would like because cost too 

much (N=3379) 

0.682 

(0.643) 

0.747 

(0.633) -0.065 [0.081] {-0.803} (0.422) 

Don't eat FV as much as 

would like because they 

spoil (N=3372) 

1.354 

(0.625) 

1.527 

(0.616) -0.173 [0.082] {-2.122} (0.034)** 

Don't eat FV as much as 

would like because family 

dislikes (N=3230) 

0.996 

(0.656) 

0.927 

(0.646) 0.068 [0.094] {0.728} (0.467) 

Don't eat FV as much 

because I don't like 

(N=3379) 

0.385 

(0.653) 

0.427 

(0.643) -0.041 [0.088] {-0.472} (0.637) 

 Composite scale—barriers 

to eating fruits & vegetables 

(N=3358) 

2.472 

(0.515) 

2.585 

(0.514) -0.114 [0.075] {-1.511} (0.131) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly 

agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 322 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F7.9: Impact of HIP on Perceived Barriers to Fruit & Vegetable Consumption, 

Logistic Regression Model 

Predicted Probability 

(S.E.) Impact 

How much do you agree 

or disagree that...? 

Treatment 

(T) 

Control 

(C) 

Odds 

Ratio [S.E.] 

{tstatistic} (P-value) 

Hard to eat vegetables 

because don't know how to 

prepare (N=3366) 

0.194 

(0.013) 0.201 (0.012) 0.961 [0.100] {-0.381} (0.703) 

Hard to eat vegetables 

because hard to find where 

I shop (N=3369) 

0.104 

(0.010) 0.115 (0.009) 0.889 [0.110] {-0.944} (0.345) 

Hard to eat fruits because 

hard to find where I shop 

(N=3372) 

0.091 

(0.010) 0.085 (0.009) 1.073 [0.141] {0.534} (0.594) 

Don't eat FV as much as 

would like because cost 

too much (N=3379) 

0.427 

(0.017) 0.436 (0.016) 0.962 [0.092] {-0.399} (0.690) 

Don't eat FV as much as 

would like because they 

spoil (N=3372) 

0.374 

(0.015) 0.420 (0.016) 0.826 [0.076] {-2.076} (0.038)** 

Don't eat FV as much as 

would like because family 

dislikes (N=3225) 

0.096 

(0.010) 0.083 (0.008) 1.164 [0.155] {1.145} (0.252) 

Don't eat FV as much 

because I don't like 

(N=3379) 

0.053 

(0.007) 0.064 (0.007) 0.822 [0.122] {-1.321} (0.187) 

Composite scale – barriers 

to eating fruits & 

vegetables (N=3480) 

0.445 

(0.017) 0.487 (0.016) 0.843 [0.080] {-1.803} (0.072)* 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes: 1=strongly agree or agree, 0=neither agree nor disagree, disagree, or strongly disagree; “don’t 

know” and “does not apply” responses coded as missing. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample. 

 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 323

Exhibit F7.10: Impact of HIP on Barriers to Grocery Shopping, Ordered Logistic Regression 

Model 

How often kept from grocery 

shopping by... 

Regression-adjusted 

mean (S.E.) Impact 

Treatment Control T-C [S.E.] 

{tstatistic} (P-value) 

Limited transportation 

(N=3186) 

1.297 

(0.579) 

1.147 

(0.572) 0.150 [0.083] {1.800} (0.072)* 

Distance to grocery store 

(N=3164) 

1.882 

(0.616) 

1.864 

(0.612) 0.018 [0.086] {0.212} (0.832) 

Composite scale – barriers to 

grocery shopping (N=3141) 

1.714 

(0.569) 

1.609 

(0.564) 0.105 [0.082] {1.287} (0.198) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always; “don’t know” and 

“refused” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

Exhibit F7.11: Impact of HIP on Barriers to Grocery Shopping, Logistic Regression Model 

How often kept from grocery 

shopping by... 

Predicted probability 

(S.E.) Impact 

Treatment Control 

Odds 

Ratio [S.E.] 

{tstatistic} (P-value) 

Limited transportation 

(N=3163) 

0.147 

(0.011) 

0.119 

(0.009) 1.274 [0.149] {2.075} (0.038)** 

Distance to grocery store 

(N=3160) 

0.102 

(0.010) 

0.101 

(0.009) 1.011 [0.130] {0.085} (0.932) 

Composite scale—barriers to 

grocery shopping (N=3480) 

0.393 

(0.015) 

0.367 

(0.015) 1.116 [0.100] {1.224} (0.221) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes: 0=Never, rarely, or sometimes, 1=most of the time or always; “don’t know” and “refused” 

responses coded as missing. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 324 ▌Appendix F: Supplemental Tables Abt Associates

Exhibit F7.12: Impact of HIP on Family Food Environment, Ordered Logistic Regression 

Model 

Regression-adjusted 

mean (S.E.) Impact 

How often do you...? 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Have fruit available at home 

(N=3314) 

5.767 

(0.615) 

5.522 

(0.603) 0.245 [0.084] {2.913} (0.004)*** 

Have fruit in refrigerator or 

on counter (N=3311) 

5.316 

(0.583) 

5.213 

(0.572) 0.102 [0.079] {1.300} (0.194) 

Have vegetables available 

at home (N=3315)a

 

5.130 

(0.738) 

4.894 

(0.720) 0.236 [0.096] {2.451} (0.014)** 

Have ready-to-eat 

vegetables in fridge or on 

counter (N=3284) 

4.690 

(0.598) 

4.545 

(0.586) 0.146 [0.079] {1.835} (0.067)* 

 Composite scale – Fruits & 

veg. available at home 

(N=3318) 

6.724 

(0.625) 

6.518 

(0.613) 0.206 [0.078] {2.638} (0.008)*** 

Have salty snacks at home 

(chips, crackers) (N=3312) 

2.235 

(0.583) 

2.237 

(0.577) -0.002 [0.074] {-0.025} (0.980) 

Have lowfat/nonfat milk at 

home (N=3292) 

3.385 

(0.580) 

3.295 

(0.573) 0.091 [0.082] {1.103} (0.270) 

Have soft drinks/fruit drinks 

(not juice) at home 

(N=3312) 

1.874 

(0.557) 

1.997 

(0.546) -0.124 [0.076] {-1.625} (0.104)* 

Sit down with family at home 

for evening meals 

(N=2306)b

 

4.443 

(0.778) 

4.504 

(0.771) -0.060 [0.093] {-0.649} (0.516) 

Cook evening meals at 

home (N=3311) 

3.783 

(0.666) 

3.758 

(0.652) 0.025 [0.085] {0.292} (0.771) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never/no refrigerator or freezer, 2=rarely, 3=sometimes, 4=most of the time, 5=always; 

“don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

“No refrigerator or freezer” responses coded as missing. 

b

Asked only in households with more than one member. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix F: Supplemental Tables ▌pg. 325

Exhibit F7.13: Impact of HIP on Family Food Environment, Logistic Regression Model 

Predicted probability 

(S.E.) Impact 

How often do you...? 

Treatment

(T) 

Control 

(C) 

Odds 

Ratio [S.E.] {t-statistic} (P-value) 

Have fruit available at home 

(N=3309) 

0.881 

(0.011) 

0.829 

(0.012) 1.527 [0.183] {3.530} (<0.001)***

Have fruit in refrigerator or on 

counter (N=3310) 

0.756 

(0.014) 

0.711 

(0.015) 1.264 [0.127] {2.331} (0.020)** 

Have vegetables available at 

home (N=3309)a

 

0.917 

(0.009) 

0.891 

(0.009) 1.346 [0.190] {2.104} (0.035)** 

Have ready-to-eat vegetables in 

fridge or on counter (N=3284) 

0.675 

(0.016) 

0.654 

(0.015) 1.102 [0.104] {1.035} (0.301) 

 Composite scale – Fruits & 

veg. available at home 

(N=3456) 

0.563 

(0.017) 

0.526 

(0.016) 1.161 [0.108] {1.612} (0.107) 

Have salty snacks at home 

(chips, crackers) (N=3312) 

0.349 

(0.015) 

0.350 

(0.014) 0.996 [0.088] {-0.041} (0.967) 

Have lowfat/nonfat milk at 

home (N=3292) 

0.560 

(0.018) 

0.541 

(0.016) 1.081 [0.103] {0.816} (0.415) 

Have soft drinks/fruit drinks (not 

juice) at home (N=3312) 

0.385 

(0.016) 

0.428 

(0.016) 0.837 [0.079] {-1.895} (0.058)* 

Sit down with family at home for 

evening meals (N=2304)b 

0.753 

(0.019) 

0.726 

(0.017) 1.149 [0.143] {1.122} (0.262) 

Cook evening meals at home 

(N=3300) 

0.930 

(0.008) 

0.914 

(0.008) 1.251 [0.180] {1.559} (0.119) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes: 1=Always or most of the time, 0=Sometimes, rarely, never, or no refrigerator/freezer; “don’t 

know” responses coded as missing. 

a

“No refrigerator or freezer” responses coded as missing. 

b

Asked only in households with more than one member. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 326 ▌Appendix F: Supplemental Tables

F.7 Chapter 8 Supplemental Tables: Full Regression Results, Additional 

Outcomes, Baseline Fruit and Vegetable Screener 

Chapter 8 supplemental tables include full regression results (Exhibit F8.1), results for inclusive TFV 

measure (Exhibit F8.2) and total fruits and vegetables by USDA Food Pattern food group (Exhibit 

F8.3), and baseline usual daily intake from fruit and vegetable screener differences (Exhibit F8.4).
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix Exhibit F8.1: Impact of HIP on Consumption of Fruits & Vegetables and Disaggregated Components, Restrictive and Inc

Vegetables (TFV) and All Fruits and Vegetables, Cup-Equivalents, Full Regression Results Including Coeff

Restrictive 

TFV 

TFV from 

mixed 

foods 

Inclusive 

TFV 

100% fruit 

juice 

White 

potatoes Legum

Treatment status 0.238*** 0.002 0.241*** 0.095*** -0.023 -0.008 

(0.054) (0.019) (0.059) (0.036) (0.024) (0.010

Respondent gender indicator – female -0.011 0.007 -0.004 -0.060 -0.114* -0.002 

(0.102) (0.040) (0.111) (0.071) (0.064) (0.021

Respondent gender indicator – male -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Respondent race/ethnicity indicator—non-Hispanic black 0.164 -0.025 0.140 0.270*** -0.084* 0.025 

(0.108) (0.043) (0.123) (0.088) (0.048) (0.016

Respondent race/ethnicity indicator—Hispanic 0.170** -0.082** 0.089 0.103** -0.161*** 0.129

(0.075) (0.034) (0.085) (0.053) (0.038) (0.016

Respondent race/ethnicity indicator—non-Hispanic other race 0.492** 0.084 0.577** 0.045 -0.148** 0.072

(0.196) (0.074) (0.224) (0.089) (0.059) (0.029

Respondent race/ethnicity indicator—non-Hispanic white -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Respondent age indicator—16;ln 30 years -0.613*** -0.014 -0.627*** 0.040 -0.010 0.021 

(0.129) (0.045) (0.144) (0.084) (0.071) (0.020

Respondent age indicator—31–40 years -0.418*** 0.010 -0.408** -0.091 -0.045 0.034

(0.145) (0.058) (0.166) (0.082) (0.072) (0.023

Respondent age indicator—41–54 years -0.382*** -0.013 -0.395*** -0.109 -0.086 0.016 

(0.130) (0.036) (0.144) (0.077) (0.066) (0.019

Respondent age indicator—55+ years -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Geography indicator—Springfield -0.133 -0.017 -0.151 0.0486 -0.040 0.008 

(0.087) (0.037) (0.099) (0.055) (0.047) (0.015

Geography indicator—Chicopee/Holyoke -0.051 -0.055 -0.106 -0.018 -0.042 0.027 

(0.094) (0.034) (0.105) (0.059) (0.048) (0.017

Geography indicator—Hampden county balance -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Household size indicator—1 household member 0.300*** 0.022 0.322*** -0.114 -0.105 -0.002 

(0.090) (0.031) (0.099) (0.067) (0.072) (0.023
Evaluation of the Healthy In

pg. 328 ▌Appendix F: Supplemental Tables 

Restrictive 

TFV 

TFV from 

mixed 

foods 

Inclusive 

TFV 

100% fruit 

juice 

White 

potatoes Legum

Household size indicator—more than one household member -- -- -- -- -- (EXCLUDED category) -- -- -- -- -- Household head gender indicator—female -0.103 -0.089** -0.192 -0.167** 0.033 -0.004 

(0.135) (0.044) (0.144) (0.077) (0.064) (0.021

Household head gender indicator—male -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Elderly (with/without children) in household indicator -0.035 -0.030 -0.065 -0.135* 0.057 -0.006 

(0.140) (0.039) (0.153) (0.071) (0.073) (0.020

Children (no elderly) in household indicator 0.218** 0.036 0.254*** -0.005 -0.093 -0.028

(0.090) (0.032) (0.098) (0.075) (0.063) (0.024

No children or elderly in household indicator -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Household # of adults indicator—four or more adults in household 0.501* 0.049 0.549* -0.229** -0.087 0.078 

(0.294) (0.065) (0.284) (0.104) (0.082) (0.050

Household # of adults indicator—three or fewer adults in household -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Sampling wave indicator—Wave 2 (HIP start date Dec. 1, 2011) -0.035 -0.036 -0.071 0.013 <0.001 -0.008 

(0.074) (0.030) (0.084) (0.048) (0.034) (0.014

Sampling wave indicator—Wave 3 (HIP start date Jan. 1, 2012) -0.076 -0.057* -0.133* -0.024 -0.063* -0.001 

(0.071) (0.030) (0.079) (0.050) (0.036) (0.015

Sampling wave indicator—Wave 1 (HIP start date Nov. 1, 2011) -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Second recall interview (binary) -0.046 0.006 -0.040 -0.062 -0.048 0.026 

(0.071) (0.033) (0.077) (0.042) (0.053) (0.021

First recall interview (binary) -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Weekend intake day (binary) -0.078 0.031 -0.047 -0.033 0.030 >-0.001 

(0.079) (0.028) (0.084) (0.053) (0.047) (0.018

Weekday intake day (binary) -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix 

Restrictive 

TFV 

TFV from 

mixed 

foods 

Inclusive 

TFV 

100% fruit 

juice 

White 

potatoes Legum

Intake level more than usual (binary) Intake level less than usual (binary) Usual intake level assessment missing due to “don't know” or break-off (binary) -0.090 -0.043 -0.133 -0.065 -0.058 -0.025 

(0.092) (0.039) (0.103) (0.073) (0.052) (0.022

Intake level same as usual -- -- -- -- -- 

(EXCLUDED category) -- -- -- -- -- Estimated daily servings—juice (baseline) 0.019 0.003 0.022 0.061*** 0.008 0.003 

(0.016) (0.005) (0.016) (0.019) (0.006) (0.003

Estimated daily servings—fruit (baseline) 0.029 -0.014 0.015 -0.026 -0.019* >-0.001 

(0.029) (0.009) (0.029) (0.017) (0.010) (0.005

Estimated daily servings—leafy green salad (baseline) 0.102 -0.015 0.087 -0.026 0.008 0.008 

(0.072) (0.017) (0.074) (0.037) (0.031) (0.010

Estimated daily servings—fried potatoes (baseline) 0.082 -0.086** -0.004 0.381*** 0.090 0.032 

(0.136) (0.036) (0.140) (0.088) (0.062) (0.039

Estimated daily servings—other potatoes (baseline) -0.035 0.023 -0.012 0.003 0.054* -0.007 

(0.076) (0.026) (0.086) (0.045) (0.031) (0.012

Estimated daily servings—beans (baseline) 0.018 0.023 0.041 -0.049 -0.045* 0.012 

(0.074) (0.025) (0.077) (0.041) (0.028) (0.017

Estimated daily servings—other vegetables (baseline) 0.138** -0.006 0.132** 0.019 -0.019 -0.013

(0.054) (0.014) (0.058) (0.034) (0.023) (0.008

Estimated daily servings—tomato sauce (baseline) 0.220 0.123 0.343* -0.179*** 0.094 0.028 

(0.144) (0.103) (0.189) (0.058) (0.073) (0.032

Estimated daily servings—salsa (baseline) -0.034 0.181 0.146 -0.044 0.981 0.401

(0.582) (0.207) (0.549) (0.413) (0.877) (0.242

Imputation indicator for estimated daily servings—juice (baseline) 0.116 -0.137** -0.021 -0.258* -0.131 -0.055 

(0.344) (0.066) (0.368) (0.133) (0.107) (0.037

Imputation indicator for estimated daily servings—fruit (baseline) -0.274 -0.053 -0.327 0.210 0.057 -0.016 

(0.193) (0.085) (0.231) (0.205) (0.131) (0.045
Evaluation of the Healthy In

pg. 330 ▌Appendix F: Supplemental Tables 

Restrictive 

TFV 

TFV from 

mixed 

foods 

Inclusive 

TFV 

100% fruit 

juice 

White 

potatoes Legum

Imputation indicator for estimated daily servings—leafy green salad (baseline) -0.257 0.113 -0.144 -0.368* 0.026 -0.025 

(0.301) (0.153) (0.331) (0.196) (0.100) (0.049

Imputation indicator for estimated daily servings—fried potatoes 

(baseline) -0.031 0.191 0.159 0.588 0.373** 0.028 

(0.282) (0.200) (0.392) (0.410) (0.152) (0.042

Imputation indicator for estimated daily servings—other potatoes 

(baseline) -0.294 0.025 -0.269 -0.241 -0.122 -0.071

(0.284) (0.113) (0.337) (0.220) (0.104) (0.041

Imputation indicator for estimated daily servings—beans (baseline) -0.033 0.234* 0.201 -0.064 -0.100 0.137

(0.233) (0.136) (0.266) (0.142) (0.105) (0.078

Imputation indicator for estimated daily servings—other vegetables (baseline) -0.316 0.0525 -0.264 0.021 -0.006 -0.011 

(0.210) (0.069) (0.226) (0.153) (0.085) (0.028

Imputation indicator for estimated daily servings—tomato sauce 

(baseline) -0.198 0.060 -0.137 0.036 0.062 -0.058

(0.175) (0.117) (0.209) (0.171) (0.129) (0.028

Imputation indicator for estimated daily servings—salsa (baseline) 0.137 -0.059 0.078 0.286 -0.016 -0.014 

(0.380) (0.091) (0.385) (0.219) (0.101) (0.039

Constructed scale—attitudes about food, fruits, and vegetables (baseline) 0.215*** 0.044 0.259*** 0.017 0.073** 0.012 

(0.057) (0.031) (0.069) (0.037) (0.029) (0.011

Constructed scale—barriers to eating fruits and vegetables (baseline) 0.009 -0.004 0.005 -0.015 0.021 0.003 

(0.030) (0.013) (0.033) (0.018) (0.016) (0.005

Constructed scale—barriers to grocery shopping (baseline) -0.012 0.002 -0.009 -0.009 <0.001 0.004 

(0.054) (0.022) (0.061) (0.031) (0.025) (0.011

Constructed scale—fruits and vegetables at home (baseline) 0.142*** 0.010 0.152*** 0.034 -0.012 0.001 

(0.043) (0.018) (0.048) (0.033) (0.021) (<0.001
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix 

Restrictive 

TFV 

TFV from 

mixed 

foods 

Inclusive 

TFV 

100% fruit 

juice 

White 

potatoes Legum

Imputation indicator for constructed scale—attitudes about food, fruits, and vegetables (baseline) -0.954*** -0.389*** -1.344*** 0.124 0.022 -0.117

(0.246) (0.083) (0.283) (0.144) (0.118) (0.044

Imputation indicator for constructed scale—barriers to eating fruits and vegetables (baseline) 0.508 -0.042 0.466 0.309 0.018 -0.033 

(0.424) (0.074) (0.483) (0.291) (0.110) (0.028

Imputation indicator for constructed scale—barriers to grocery shopping (baseline) 0.212 0.103 0.315 0.227 -0.178** 0.032 

(0.272) (0.092) (0.291) (0.233) (0.085) (0.047

Imputation indicator for constructed scale—fruits and vegetables 

at home (baseline) -0.320 -0.007 -0.327 -0.315 -0.123 0.072

(0.489) (0.086) (0.550) (0.298) (0.115) (0.040

Constant -0.296 0.249* -0.048 0.480** 0.421** -0.010 

(0.319) (0.142) (0.364) (0.226) (0.164) (0.070

Observations 3,913 3,913 3,913 3,913 3,913 3,913 R-squared 0.114 0.038 0.115 0.077 0.045 0.077 

Standard errors in parentheses. Two-sided test; *p<0.1, **p<0.05, ***p<0.01. TFV=Intake of fruits and vegetables acquired from the store, excluding white potatoes, legumes, and 100% juice. 

Inclusive measure includes fruit & vegetable intake from mixed foods, restrictive measure excludes mixed foods. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 sample (unweighted N=3913 recalls from 2009 responde
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 332 ▌ Appendix F: Supplemental Tables Abt Associates

Exhibit F8.2: Impact of HIP on Consumption of Targeted Fruits & Vegetables (TFV), Inclusive 

Measure, by USDA Food Pattern Food Group 

Regression-adjusted 

mean (SE) Impact 

TFV, Inclusive Measure 

Treatment

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

Total fruits and vegetables 1.455 

(0.045) 

1.215 

(0.039) 0.241 [0.059] {4.082} <(0.001)*** 

Total fruits 0.644 

(0.030) 

0.530 

(0.025) 0.114 [0.039] {2.940} (0.003)*** 

Citrus fruits, melons, & 

berries 

0.148 

(0.011) 

0.134 

(0.012) 0.014 [0.016] {0.857} (0.392) 

Other fruits (e.g. 

apples, pears, 

bananas, grapes, 

peaches) 

0.495 

(0.026) 

0.396 

(0.019) 0.100 [0.032] {3.116} (0.002)*** 

Total vegetables 0.811 

(0.028) 

0.685 

(0.025) 0.127 [0.037] {3.404} (0.001)*** 

Dark green vegetables 0.085 

(0.007) 

0.048 

(0.006) 0.037 [0.009] {4.085} <(0.001)*** 

Red and orange 

vegetables 

0.266 

(0.012) 

0.236 

(0.010) 0.030 [0.016] {1.928} (0.054)* 

Tomatoes 0.201 

(0.009) 

0.186 

(0.009) 0.016 [0.013] {1.223} (0.221) 

Other red and 

orange vegetables 

0.065 

(0.007) 

0.051 

(0.004) 0.015 [0.008] {1.788} (0.074)* 

Other starchy 

vegetables 

0.106 

(0.009) 

0.100 

(0.007) 0.006 [0.011] {0.544} (0.586) 

Other vegetables (e.g. 

celery, cucumbers, 

mushrooms, green 

beans, onions, 

asparagus) 

0.354 

(0.016) 

0.300 

(0.016) 0.054 [0.023] {2.348} (0.019)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

TFV=Intake of fruits and vegetables acquired from the store, excluding white potatoes and 100% juice. 

Inclusive measure includes fruit and vegetable intake from mixed foods. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 sample (unweighted 

N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix F: Supplemental Tables ▌pg. 333

Exhibit F8.3: Impact of HIP on Consumption of Total Fruits & Vegetables, by USDA Food 

Pattern Food Group 

Regression-adjusted 

mean (SE) Impact 

Fruits and Vegetables 

Treatment 

(T) 

Control 

(C) T-C [S.E.] {t-statistic} (P-value) 

 

Total fruits and vegetables 2.616 

(0.060) 

2.294 

(0.055) 0.323 [0.080] {4.016} <(0.001)*** 

Total fruits 1.259 

(0.044) 

1.031 

(0.037) 0.228 [0.057] {3.977} <(0.001)*** 

Citrus fruits, melons, & 

berries 

0.171 

(0.012) 

0.145 

(0.013) 0.026 [0.017] {1.512} (0.131) 

Other fruits (e.g. 

apples, pears, 

bananas, grapes, 

peaches) 

0.539 

(0.026) 

0.433 

(0.020) 0.106 [0.033] {3.238} (0.001)*** 

100% fruit juice 0.549 

(0.029) 

0.453 

(0.023) 0.095 [0.036] {2.617} (0.009)*** 

Total vegetables 1.357 

(0.034) 

1.262 

(0.034) 0.095 [0.048] {1.965} (0.050)** 

Dark green vegetables 0.102 

(0.008) 

0.063 

(0.006) 0.039 [0.010] {3.910} <(0.001)*** 

Red and orange 

vegetables 

0.338 

(0.013) 

0.320 

(0.012) 0.018 [0.017] {1.026} (0.305) 

Tomatoes 0.264 

(0.011) 

0.257 

(0.010) 0.008 [0.015] {0.510} (0.610) 

Other red and 

orange vegetables 

0.240 

(0.011) 

0.217 

(0.010) 0.023 [0.015] {1.553} (0.121) 

Starchy vegetables 0.452 

(0.019) 

0.469 

(0.018) -0.016 [0.027] {-0.613} (0.540) 

White potatoes 0.336 

(0.017) 

0.359 

(0.017) -0.023 [0.024] {-0.963} (0.336) 

Other starchy 

vegetables 

0.116 

(0.009) 

0.109 

(0.007) 0.007 [0.012] {0.607} (0.544) 

Legumes 0.106 

(0.007) 

0.114 

(0.007) -0.008 [0.010] {-0.858} (0.391) 

Other vegetables (e.g. 

celery, cucumbers, 

mushrooms, green 

beans, onions, 

asparagus) 

0.423 

(0.017) 

0.367 

(0.017) 0.055 [0.024] {2.324} (0.020)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 sample (unweighted 

N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 334 ▌ Appendix F: Supplemental Tables Abt Associates

Exhibit F8.4: Baseline Estimated Usual Daily Intake in Cup-Equivalents from Fruit & 

Vegetable Screener of Respondents Completing Follow-Up Participant Surveys 

(Round 2 or 3), by Treatment and Control Status 

Total Treatment Control P-value 

100% juice (N=1954) 1.58 (0.07) 1.45 (0.07) 1.60 (0.09) [0.180] 

Fruit (N=1954) 0.84 (0.04) 0.79 (0.04) 0.84 (0.04) [0.355] 

Salad (N=1954) 0.38 (0.02) 0.36 (0.02) 0.38 (0.02) [0.368] 

Fried potatoes (N=1954) 0.10 (0.01) 0.10 (0.01) 0.10 (0.01) [0.688] 

Other potatoes (N=1954) 0.28 (0.01) 0.27 (0.01) 0.28 (0.01) [0.688] 

Beans (N=1954) 0.26 (0.01) 0.26 (0.02) 0.25 (0.01) [0.676] 

Other vegetables (N=1954) 0.61 (0.02) 0.61 (0.03) 0.61 (0.03) [0.988] 

Tomato sauce (N=1954) 0.15 (0.01) 0.16 (0.01) 0.15 (0.01) [0.564] 

Salsa (N=1954) 0.01 (<0.01) 0.02 (<0.01) 0.01 (<0.01) [0.416] 

Weighted means (standard error). 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know,” “refused,” and “inapplicable” responses on frequency or amount items coded as missing. 

Source: Modified EATS Fruit and Vegetable Screener, Participant Survey (respondent module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 335

Appendix G: Analyses by Follow-Up Survey Round 

Exhibit G6.1: Differences in Impact of HIP on Mean Monthly SNAP and TFV Purchases, by 

Follow-Up Round 

Usual place to shop 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

SNAP purchases at supermarkets and superstores ($)

Pooled (Rounds 2 & 3) 126.87 (1.08) 122.64 (0.41) 4.23 (<0.001)*** 

Round 2 124.08 (1.12) 119.78 (0.43) 4.31 (<0.001)*** 

Round 3 130.93 (1.22) 127.35 (0.46) 3.58 (0.006)*** 

Change: Round 3 - Round 2 -0.51 (0.583) 

Eligible TFV purchases at supermarkets and superstores ($)

Pooled (Rounds 2 & 3) 12.04 (0.15) 10.89 (0.05) 1.16 (<0.001)*** 

Round 2 12.08 (0.16) 10.98 (0.06) 1.10 (<0.001)*** 

Round 3 11.92 (0.16) 10.72 (0.06) 1.20 (<0.001)*** 

Change: Round 3 – Round 2 0.10 (0.440) 

Eligible mean household TFV purchase as % of SNAP purchases at supermarkets and superstores 

(computed) 

Pooled (Rounds 2 & 3) 9.94 (0.12) 9.27 (0.04) 0.67 (<0.001)*** 

Round 2 10.25 (0.13) 9.56 (0.05) 0.69 (<0.001)*** 

Round 3 9.45 (0.13) 8.87 (0.05) 0.58 (<0.001)*** 

Change: Round 3 – Round 2 -0.03 (0.811) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the household level for the separate Round 2 and 

Round 3 results. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. The same demographic covariates, which include a 

mix of time-variant and time-invariant variables, are included in both models. These alternative models would 

produce identical point estimates of the Round 2/3 difference if the change model included the Round 2/3 

change in demographic covariates instead of the levels. 

Source: EBT Transaction Data, 45,854 households that received SNAP benefits in every month from March to 

October 2012. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 336 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Exhibit G6.2: Differences in Impacts of HIP on Self-Reported Monthly Expenditures, by 

Follow-Up Round 

Usual monthly spending 

($) for... 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

SNAP (N=2641) 

Pooled (Rounds 2 & 3) 273.981 (4.976) 279.201 (4.130) -5.220 (0.425) 

Round 2 276.576 (5.612) 279.038 (4.995) -2.463 (0.747) 

Round 3 271.339 (5.615) 279.368 (4.857) -8.029 (0.282) 

Change: Round 3 – Round 2 -5.289 (0.489) 

Groceries (N=2570) 

Pooled (Rounds 2 & 3) 148.332 (4.126) 143.232 (4.110) 5.100 (0.358) 

Round 2 149.051 (5.151) 139.705 (4.909) 9.346 (0.176) 

Round 3 147.578 (5.023) 146.839 (5.120) 0.739 (0.916) 

Change: Round 3 – Round 2 -10.131 (0.218) 

Groceries—food itemsa (N=2481) 

Pooled (Rounds 2 & 3) 103.595 (3.601) 102.927 (3.569) 0.669 (0.889) 

Round 2 104.472 (4.522) 100.446 (4.366) 4.026 (0.507) 

Round 3 102.693 (4.541) 105.462 (4.278) -2.769 (0.647) 

Change: Round 3 – Round 2 -7.952 (0.287) 

Groceries—nonfood items (N=2841) 

Pooled (Rounds 2 & 3) 44.594 (1.799) 39.962 (1.556) 4.633 (0.049)** 

Round 2 44.291 (2.210) 40.171 (2.157) 4.120 (0.176) 

Round 3 44.903 (2.535) 39.747 (1.934) 5.156 (0.106) 

Change: Round 3 – Round 2 0.427 (0.920) 

Restaurants (N=2613) 

Pooled (Rounds 2 & 3) 33.142 (1.330) 34.057 (1.394) -0.916 (0.640) 

Round 2 32.740 (1.563) 35.140 (1.673) -2.400 (0.305) 

Round 3 33.567 (1.772) 32.933 (1.685) 0.634 (0.796) 

Change: Round 3 – Round 2 3.986 (0.167) 

Fruits and vegetablesb (N=2271) 

Pooled (Rounds 2 & 3) 77.578 (2.137) 71.201 (2.104) 6.377 (0.027)** 

Round 2 78.547 (2.714) 70.141 (2.359) 8.406 (0.016)** 

Round 3 76.586 (2.544) 72.266 (2.634) 4.319 (0.227) 

Change: Round 3 – Round 2 -4.364 (0.283) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded 

as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. a

Calculated as total grocery expenditures minus expenditures on nonfood items. b

Purchased with SNAP and with other forms of payment. 

Source: Participant Survey (primary shopper module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 337

Exhibit G6.3: Differences in Impact of HIP on Mean Monthly SNAP Purchases by Retailer 

Type, by Follow-Up Round 

Usual place to shop 

Regressionadjusted 

treatment mean 

(SE) ($) 

Regressionadjusted 

control mean 

(SE) ($) 

Treatmentcontrol impact 

(P-value) ($) 

Difference in 

impacts (Pvalue) 

All retailers 

Pooled (Rounds 2 & 3) 265.04 (1.13) 260.79 (0.44) 4.30 (<.001)*** 

Round 2 265.46 (1.18) 262.02 (0.46) 3.44 (0.007)*** 

Round 3 263.61 (1.27) 258.81 (0.50) 4.79 (<.001)*** 

Change: Round 3 – Round 2 1.66 (0.093)* 

Supermarkets and superstores 

Pooled (Rounds 2 & 3) 211.79 (1.14) 207.21 (0.44) 4.58 (<.001)*** 

Round 2 212.42 (1.19) 208.12 (0.46) 4.30 (<.001)*** 

Round 3 210.05 (1.29) 205.62 (0.50) 4.42 (0.001)*** 

Change: Round 3 – Round 2 0.36 (0.718) 

Convenience 

Pooled (Rounds 2 & 3) 26.42 (0.46) 26.32 (0.18) 0.09 (0.848) 

Round 2 26.43 (0.47) 26.68 (0.20) -0.25 (0.621) 

Round 3 26.32 (0.51) 25.68 (0.20) 0.64 (0.242) 

Change: Round 3 – Round 2 0.92 (0.017)** 

Grocery 

Pooled (Rounds 2 & 3) 20.35 (0.46) 19.90 (0.18) 0.45 (0.354) 

Round 2 20.49 (0.48) 20.24 (0.19) 0.25 (0.627) 

Round 3 20.04 (0.49) 19.31 (0.19) 0.73 (0.164) 

Change: Round 3 – Round 2 0.49 (0.168) 

Other

Pooled (Rounds 2 & 3) 2.39 (0.19) 2.48 (0.08) -0.09 (0.651) 

Round 2 2.31 (0.19) 2.43 (0.08) -0.11 (0.598) 

Round 3 2.46 (0.19) 2.59 (0.08) -0.12 (0.551) 

Change: Round 3 – Round 2 <.01 (0.968) 

Out of state 

Pooled (Rounds 2 & 3) 4.09 (0.28) 4.87 (0.12) -0.78 (0.009)*** 

Round 2 3.81 (0.28) 4.56 (0.12) -0.75 (0.014)** 

Round 3 4.74 (0.36) 5.62 (0.16) -0.87 (0.027)** 

Change: Round 3 – Round 2 -0.11 (0.715) 

HIP participating 

Pooled (Rounds 2 & 3) 137.18 (1.09) 132.92 (0.42) 4.26 (<.001)*** 

Round 2 134.29 (1.14) 130.10 (0.44) 4.19 (<.001)*** 

Round 3 141.45 (1.23) 137.57 (0.47) 3.88 (0.003)*** 

Change: Round 3 – Round 2 -0.07 (0.939) 

Non-HIP participating 

Pooled (Rounds 2 & 3) 127.86 (1.06) 127.87 (0.43) -0.01 (0.995) 

Round 2 131.17 (1.13) 131.92 (0.45) -0.75 (0.538) 

Round 3 122.16 (1.16) 121.25 (0.47) 0.91 (0.465) 

Change: Round 3 – Round 2 1.73 (0.057)* 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the household level for the separate Round 2 and 

Round 3 results. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 338 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. The same demographic covariates, which include a 

mix of time-variant and time-invariant variables, are included in both models. These alternative models would 

produce identical point estimates of the Round 2/3 change if the model that regressed the Round 2/3 change in 

outcomes included the Round 2/3 change in demographic covariates instead of the levels. 

Source: EBT Transaction Data, 45,854 households that received SNAP benefits in every month from March to 

October 2012. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 339

Exhibit G6.4: Differences in Impacts of HIP on Usual Grocery Store Type, by Follow-Up 

Round 

Usual place to shop 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Large chain grocery store or supermarket

Pooled (Rounds 2 & 3) 0.817 (0.012) 0.818 (0.011) -0.001 (0.970) 

Round 2 0.796 (0.015) 0.803 (0.015) -0.006 (0.761) 

Round 3 0.839 (0.014) 0.833 (0.013) 0.006 (0.776) 

Change: Round 3 – Round 2 0.013 (0.580) 

Natural or organic supermarket (such as Whole Foods Market)

Pooled (Rounds 2 & 3) 0.005 (0.002) 0.006 (0.002) -0.001 (0.763) 

Round 2 0.005 (0.003) 0.009 (0.004) -0.003 (0.471) 

Round 3 0.004 (0.003) 0.003 (0.002) 0.002 (0.640) 

Change: Round 3 – Round 2 0.004 (0.470) 

Small local store or corner store 

Pooled (Rounds 2 & 3) 0.021 (0.004) 0.024 (0.004) -0.002 (0.711) 

Round 2 0.022 (0.006) 0.028 (0.006) -0.006 (0.475) 

Round 3 0.021 (0.005) 0.020 (0.005) 0.001 (0.872) 

Change: Round 3 – Round 2 0.002 (0.861) 

Convenience store (such as 7-11 or mini market)

Pooled (Rounds 2 & 3) 0.001 (0.001) 0.001 (0.001) 0.001 (0.529) 

Round 2 0.002 (0.002) 0.002 (0.001) <0.001 (0.880) 

Round 3 0.001 (0.001) <0.001 (<0.001) 0.001 (0.373) 

Change: Round 3 – Round 2 <0.001 (0.830) 

Warehouse club store (such as Sam's Club or Costco)

Pooled (Rounds 2 & 3) 0.020 (0.004) 0.023 (0.005) -0.004 (0.571) 

Round 2 0.019 (0.005) 0.023 (0.006) -0.004 (0.583) 

Round 3 0.020 (0.006) 0.024 (0.006) -0.003 (0.714) 

Change: Round 3 – Round 2 0.004 (0.636) 

Discount superstore (such as Wal-Mart)

Pooled (Rounds 2 & 3) 0.124 (0.010) 0.119 (0.009) 0.005 (0.710) 

Round 2 0.143 (0.013) 0.128 (0.012) 0.015 (0.418) 

Round 3 0.105 (0.012) 0.109 (0.011) -0.005 (0.766) 

Change: Round 3 – Round 2 -0.017 (0.408) 

Ethnic market 

Pooled (Rounds 2 & 3) 0.004 (0.002) 0.005 (0.002) -0.001 (0.653) 

Round 2 0.004 (0.002) 0.007 (0.003) -0.003 (0.480) 

Round 3 0.004 (0.002) 0.003 (0.002) 0.001 (0.851) 

Change: Round 3 – Round 2 0.004 (0.308) 

Farmers market/co-op 

Pooled (Rounds 2 & 3) 0.003 (0.002) 0.001 (0.001) 0.001 (0.580) 

Round 2 0.005 (0.003) 0.001 (0.001) 0.003 (0.333) 

Round 3 <0.001 (0.001) 0.001 (0.001) -0.001 (0.306) 

Change: Round 3 – Round 2 -0.005 (0.175) 

Some other location 

Pooled (Rounds 2 & 3) 0.002 (0.001) 0.003 (0.001) -0.001 (0.673) 

Round 2 0.003 (0.002) <0.001 (0.001) 0.003 (0.215) 

Round 3 0.002 (0.001) 0.007 (0.003) -0.005 (0.129) 

Change: Round 3 – Round 2 -0.008 (0.052)* 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=Yes, 0=No; “don’t know” responses coded as missing. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 340 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module); (unweighted N=2,777). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 341

Exhibit G6.5: Differences in Impacts of HIP on Reasons for Choice of Usual Grocery 

Shopping Place, by Follow-Up Round 

Usually shop there 

because... 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Prices/affordability 

Pooled (Rounds 2 & 3) 0.489 (0.016) 0.520 (0.015) -0.031 (0.156) 

Round 2 0.502 (0.019) 0.540 (0.018) -0.039 (0.145) 

Round 3 0.476 (0.020) 0.499 (0.018) -0.023 (0.401) 

Change: Round 3 – Round 2 0.004 (0.912) 

Close to home 

Pooled (Rounds 2 & 3) 0.301 (0.015) 0.323 (0.013) -0.022 (0.266) 

Round 2 0.298 (0.018) 0.316 (0.017) -0.018 (0.471) 

Round 3 0.304 (0.018) 0.330 (0.017) -0.027 (0.285) 

Change: Round 3 – Round 2 -0.006 (0.846) 

Variety of products 

Pooled (Rounds 2 & 3) 0.243 (0.013) 0.202 (0.011) 0.041 (0.015)** 

Round 2 0.219 (0.016) 0.207 (0.015) 0.011 (0.604) 

Round 3 0.269 (0.018) 0.197 (0.015) 0.072 (0.002)*** 

Change: Round 3 – Round 2 0.064 (0.032)** 

Produce better or fresher 

Pooled (Rounds 2 & 3) 0.168 (0.011) 0.137 (0.010) 0.030 (0.035)** 

Round 2 0.179 (0.015) 0.161 (0.014) 0.019 (0.351) 

Round 3 0.156 (0.014) 0.114 (0.012) 0.042 (0.018)** 

Change: Round 3 – Round 2 0.022 (0.369) 

Sales/promotions in store 

Pooled (Rounds 2 & 3) 0.099 (0.009) 0.131 (0.010) -0.032 (0.018)** 

Round 2 0.127 (0.013) 0.123 (0.012) 0.004 (0.814) 

Round 3 0.069 (0.011) 0.139 (0.013) -0.070 (<0.001)*** 

Change: Round 3 – Round 2 -0.074 (0.002)** 

Preferred products are available 

Pooled (Rounds 2 & 3) 0.093 (0.009) 0.123 (0.009) -0.030 (0.013)** 

Round 2 0.091 (0.011) 0.122 (0.012) -0.030 (0.066)* 

Round 3 0.095 (0.011) 0.125 (0.013) -0.030 (0.071)* 

Change: Round 3 – Round 2 0.002 (0.917) 

Familiarity with store 

Pooled (Rounds 2 & 3) 0.071 (0.008) 0.058 (0.008) 0.013 (0.249) 

Round 2 0.060 (0.009) 0.066 (0.010) -0.006 (0.663) 

Round 3 0.083 (0.011) 0.051 (0.009) 0.032 (0.028)** 

Change: Round 3 – Round 2 0.042 (0.013)** 

One stop shopping 

Pooled (Rounds 2 & 3) 0.059 (0.007) 0.057 (0.006) 0.001 (0.868) 

Round 2 0.046 (0.008) 0.046 (0.008) 0.001 (0.963) 

Round 3 0.072 (0.010) 0.070 (0.010) 0.003 (0.846) 

Change: Round 3 – Round 2 0.008 (0.656) 

Easy to get there 

Pooled (Rounds 2 & 3) 0.023 (0.004) 0.015 (0.003) 0.008 (0.140) 

Round 2 0.019 (0.005) 0.015 (0.004) 0.004 (0.542) 

Round 3 0.028 (0.006) 0.015 (0.005) 0.013 (0.100)* 

Change: Round 3 – Round 2 0.007 (0.451) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 342 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Usually shop there 

because... 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Quality 

Pooled (Rounds 2 & 3) 0.042 (0.005) 0.048 (0.006) -0.005 (0.512) 

Round 2 0.040 (0.007) 0.055 (0.009) -0.015 (0.199) 

Round 3 0.045 (0.008) 0.040 (0.008) 0.005 (0.679) 

Change: Round 3 – Round 2 0.025 (0.097)* 

Close to work 

Pooled (Rounds 2 & 3) 0.005 (0.002) 0.003 (0.001) 0.003 (0.376) 

Round 2 0.009 (0.005) 0.004 (0.002) 0.005 (0.357) 

Round 3 0.002 (0.002) 0.002 (0.002) <0.001 (0.997) 

Change: Round 3 – Round 2 -0.006 (0.375) 

Close to some other location 

Pooled (Rounds 2 & 3) 0.002 (0.001) 0.004 (0.002) -0.001 (0.485) 

Round 2 0.002 (0.002) 0.007 (0.003) -0.005 (0.157) 

Round 3 0.002 (0.002) <0.001 (<0.001) 0.002 (0.207) 

Change: Round 3 – Round 2 0.008 (0.041)** 

Disability accessible 

Pooled (Rounds 2 & 3) 0.003 (0.002) 0.002 (0.001) 0.002 (0.272) 

Round 2 0.003 (0.002) <0.001 (0.001) 0.003 (0.125) 

Round 3 0.004 (0.002) 0.003 (0.002) 0.001 (0.810) 

Change: Round 3 – Round 2 -0.001 (0.684) 

Hours of operation convenient

Pooled (Rounds 2 & 3) 0.005 (0.002) 0.009 (0.003) -0.003 (0.352) 

Round 2 0.005 (0.003) 0.010 (0.004) -0.005 (0.301) 

Round 3 0.006 (0.003) 0.007 (0.004) -0.001 (0.777) 

Change: Round 3 – Round 2 0.003 (0.648) 

EBT card accepted 

Pooled (Rounds 2 & 3) 0.029 (0.005) 0.015 (0.003) 0.014 (0.031)** 

Round 2 0.035 (0.007) 0.025 (0.006) 0.010 (0.296) 

Round 3 0.023 (0.006) 0.006 (0.003) 0.017 (0.010)*** 

Change: Round 3 – Round 2 0.006 (0.552) 

Ethnic foods are available 

Pooled (Rounds 2 & 3) 0.007 (0.002) 0.007 (0.002) >-0.001 (0.990) 

Round 2 0.007 (0.003) 0.008 (0.003) -0.001 (0.782) 

Round 3 0.007 (0.003) 0.006 (0.003) 0.001 (0.815) 

Change: Round 3 – Round 2 -0.001 (0.823) 

Good service 

Pooled (Rounds 2 & 3) 0.028 (0.005) 0.019 (0.004) 0.008 (0.182) 

Round 2 0.026 (0.006) 0.024 (0.006) 0.003 (0.747) 

Round 3 0.029 (0.007) 0.015 (0.005) 0.014 (0.082)* 

Change: Round 3 – Round 2 0.012 (0.290) 

Bulk purchases 

Pooled (Rounds 2 & 3) 0.022 (0.005) 0.025 (0.005) -0.004 (0.606) 

Round 2 0.024 (0.006) 0.025 (0.006) -0.001 (0.928) 

Round 3 0.019 (0.006) 0.025 (0.006) -0.006 (0.444) 

Change: Round 3 – Round 2 -0.005 (0.614) 

Clean 

Pooled (Rounds 2 & 3) 0.022 (0.004) 0.018 (0.004) 0.004 (0.477) 

Round 2 0.026 (0.006) 0.021 (0.006) 0.005 (0.541) 

Round 3 0.018 (0.005) 0.015 (0.005) 0.003 (0.661) 

Change: Round 3 – Round 2 -0.004 (0.693) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 343

Usually shop there 

because... 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Some other reason 

Pooled (Rounds 2 & 3) 0.010 (0.003) 0.007 (0.002) 0.003 (0.517) 

Round 2 0.009 (0.004) 0.006 (0.003) 0.003 (0.499) 

Round 3 0.010 (0.004) 0.008 (0.003) 0.002 (0.689) 

Change: Round 3 – Round 2 >-0.001 (0.990) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module); (unweighted N=2,763). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 344 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Exhibit G6.6: Differences in Impacts of HIP on Grocery Shopping Behavior, by Follow-Up 

Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Grocery shopping frequency (N=2789)a 

Pooled (Rounds 2 & 3) 5.234 (0.028) 5.208 (0.026) 0.026 (0.504) 

Round 2 5.229 (0.033) 5.207 (0.032) 0.022 (0.625) 

Round 3 5.238 (0.035) 5.210 (0.031) 0.029 (0.537) 

Change: Round 3 – Round 2 0.033 (0.531) 

Go out of way to shop for FV at particular store (N=2765)b

Pooled (Rounds 2 & 3) 0.384 (0.015) 0.403 (0.014) -0.020 (0.335) 

Round 2 0.401 (0.019) 0.411 (0.018) -0.010 (0.713) 

Round 3 0.366 (0.018) 0.396 (0.018) -0.030 (0.245) 

Change: Round 3 – Round 2 -0.027 (0.395) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. a

Categorical outcome: 1=yearly or not at all, 2=2 to 3 times a year, 3=every other month, 4=once a month, 

5=every other week, 6=once a week, 7=more than once a week; “don’t know” and “refused” responses coded as 

missing. 

b

Binary outcome: 1=yes, 0=no; “don’t know” and “refused” responses coded as missing. 

Source: Participant Survey (primary shopper module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 345

Exhibit G6.7: Primary Shopper Self-Reported Changes in Shopping Location Due to HIP: 

Round 2 and 3 Participant Surveys, HIP-Eligible Households 

Proportion (N) Change 

Pooled Round 2 Round 3 (P-value) 

Because of HIP, have you changed which stores you go to, to buy fruits and vegetables? (N=1377)

Yes 0.28 (389) 0.27 (179) 0.29 (210) 0.02 (0.258) 

No 0.72 (988) 0.73 (481) 0.71 (507) 

If yes, why have you changed which stores you go to?a

Other store has greater variety of fruits 

& vegetables (N=388) 0.62 (244) 0.62 (111) 0.63 (133) 0.03 (.)b

Price of fruits & vegetables more 

affordable at other store (N=386) 0.76 (295) 0.77 (136) 0.75 (159) -0.08 (.)b

Other store has fresh fruits & 

vegetables (N=386) 0.74 (288) 0.76 (136) 0.72 (152) -0.02 (.)b

Other store participates in HIP (N=388) 0.24 (94) 0.23 (42) 0.24 (52) 0.03 (.)b

Other reason (N=388) 0.04 (16) 0.06 (11) 0.02 ( 5) -0.06 (.)b

Weighted proportions (unweighted Ns). 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

 “Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported Round 2/3 change may differ from differences between reported proportions for 

Rounds 2 and 3. 

a

Respondents could choose multiple reasons, so proportions do not add to one. b

Test statistics cannot be computed because of stratum with single sampling unit. 

Source: Participant Survey (primary shopper module).
Evaluation of the Healthy In

pg. 346 ▌Appendix G: Analyses by Follow-Up Survey Round Exhibit G6.8: Differences in Impact of HIP on Mean Monthly SNAP and TFV Purchases at Participating Supermarkets and 

Implementation Shopping Behavior, by Follow-Up Round 

Pre-HIP participating store shoppers Pre-HIP non-participat

Usual place to shop 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

SNAP purchases at supermarkets and superstores ($) Pooled (Rounds 2 & 3) 159.05 

(1.67) 

153.24 

(0.65) 

5.81 

(0.001)*** 

84.13 

(1.77) 

80.60 

(0.65) 

Round 2 156.28 

(1.74) 

150.28 

(0.69) 

5.99 

(0.001)*** 

82.34 

(1.83) 

77.78 

(0.68) 

Round 3 163.07 

(1.96) 

158.11 

(0.74) 

4.96 

(0.018)** 

87.02 

(2.05) 

85.46 

(0.76) 

Change: Round 3 – Round 2 

-0.49 

(0.762) 

Eligible TFV purchases at supermarkets and superstores ($) Pooled (Rounds 2 & 3) 14.62 

(0.25) 

13.39 

(0.09) 

1.24 

(<0.001)*** 

9.25 

(0.27) 

7.57 

(0.08) 

Round 2 14.69 

(0.27) 

13.49 

(0.10) 

1.21 

(<0.001)*** 

9.31 

(0.30) 

7.63 

(0.09) 

Round 3 14.43 

(0.28) 

13.22 

(0.10) 

1.21 

(<0.001)*** 

9.09 

(0.30) 

7.43 

(0.09) 

Change: Round 3 – Round 2 

0.03 

(0.876) 

Eligible mean household TFV purchase as % of SNAP purchases at supermarkets and superstores (computed) Pooled (Rounds 2 & 3) 9.66 

(0.16) 

9.16 

(0.06) 

0.51 

(0.003)*** 

10.90 

(0.28) 

9.64 

(0.11) 

Round 2 9.95 

(0.18) 

9.42 

(0.07) 

0.52 

(0.007)*** 

11.28 

(0.31) 

10.05 

(0.12) 

Round 3 9.16 

(0.17) 

8.78 

(0.07) 

0.38 

(0.042)*** 

10.36 

(0.31) 

9.08 

(0.12) 

Change: Round 3 – Round 2 

>-0.01 

(0.313) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by FTwo-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment anNote that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in estimated impacts for Rounds 2 and 

third column of this table. This discrepancy arises because we estimate the Round 2/3 change in impacts using a regression model with the chan

variable, while the estimated impacts for Rounds 2 and 3 are separately estimated using the level of the outcome as the dependent variable. The 

which include a mix of time-variant and time-invariant variables, are included in both models. These alternative models would produce identical podifference if the change model included the Round 2/3 change in demographic covariates instead of the levels. 

Pre-HIP participating store shoppers are Wave 2 and Wave 3 households who spent at least 50 percent of their November 2011 (pre-implementalater participated in HIP during March-October 2012 (post-implementation). Pre-HIP non-participating store shoppers are Wave 2 and Wave 3 hou

percent of their November 2011 (pre-implementation) SNAP benefits in retailers that later participated in HIP during March-October 2012 (post-im

Source: EBT Transaction Data, 29,799 households that received SNAP benefits in every month from March to October 2012. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 348 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Exhibit G6.9: Differences in Impact of HIP on Mean Monthly SNAP Purchases at Farmers 

Markets, by Follow-Up Round 

Usual place to shop 

Regressionadjusted 

treatment 

meana(SE) ($) 

Regressionadjusted 

control meana

(SE) ($) 

Treatmentcontrol impact 

(P-value) ($) 

Difference in 

impacts 

(P-value) ($) 

All farmers markets 

Pooled (Rounds 2 & 3) 0.03 

(0.01) 

0.03 

(<0.01) 

<0.01 

(0.866) 

Round 2 0.02 

(<0.01) 

0.02 

(0.01) 

-0.01 

(0.470) 

Round 3 0.06 

(0.01) 

0.04 

(<0.01) 

0.01 

(0.363) 

Change: Round 3 – Round 2 0.02 

(0.144) 

Participating farmers markets 

Pooled (Rounds 2 & 3) 0.03 

(0.01) 

0.02 

(<0.01) 

<0.01 

(0.598) 

Round 2 0.01 

(<0.01) 

0.01 

(<0.01) 

>-0.01 

(0.549) 

Round 3 0.05 

(0.01) 

0.04 

(<0.01) 

0.01 

(0.291) 

Change: Round 3 – Round 2 0.02 

(0.139) 

Non-participating farmers markets 

Pooled (Rounds 2 & 3) 0.01 

(<0.01) 

0.01 

(<0.01) 

>-0.01 

(0.498) 

Round 2 0.01 

(<0.01) 

0.01 

(<0.01) 

>-0.01 

(0.572) 

Round 3 <0.01 

(<0.01) 

0.01 

(<0.01) 

>-0.01 

(0.671) 

Change: Round 3 – Round 2 >-0.01 

(0.824) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. The same demographic covariates, which include a 

mix of time-variant and time-invariant variables, are included in both models. These alternative models would 

produce identical point estimates of the Round 2/3 difference if the change model included the Round 2/3 

change in demographic covariates instead of the levels. 

a

Means include households that made no SNAP purchases at farmers markets. 

Source: EBT Transaction Data, 45,854 households that received SNAP benefits in every month from March to 

October 2012. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 349

Exhibit G7.1: Differences in Impacts of HIP on Self-Reported Exposure to Nutrition Education 

and Promotion in Past 3 Months, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Heard or seen messages about fruits & vegetables (N=2898)

Pooled (Rounds 2 & 3) 0.767 

(0.014) 

0.686 

(0.013) 

0.081 

(<0.001) *** 

Round 2 0.780 

(0.016) 

0.707 

(0.017) 

0.073 

(0.002)*** 

Round 3 0.754 

(0.017) 

0.665 

(0.017) 

0.089 

(<0.001) *** 

Change: Round 3 – Round 2 

0.018 

(0.537) 

Attended nutrition education class or program (N=2911)

Pooled (Rounds 2 & 3) 0.101 

(0.010) 

0.108 

(0.009) 

-0.007 

(0.613) 

Round 2 0.098 

(0.012) 

0.101 

(0.011) 

-0.003 

(0.837) 

Round 3 0.104 

(0.013) 

0.115 

(0.012) 

-0.010 

(0.548) 

Change: Round 3 – Round 2 

-0.008 

(0.669) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=Yes, 0=No; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 350 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Exhibit G7.2: Differences in Impacts of HIP on Food Preferences & Beliefs, by Follow-Up 

Round 

How much do you agree or 

disagree that...? 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

I enjoy trying new foods (N=2887) 

Pooled (Rounds 2 & 3) 3.905 (0.025) 3.840 (0.027) 0.065 (0.082)* 

Round 2 3.903 (0.029) 3.833 (0.030) 0.070 (0.099)* 

Round 3 3.907 (0.032) 3.846 (0.034) 0.060 (0.197) 

Change: Round 3 – Round 2 -0.058 (0.233) 

I enjoy trying new fruits (N=2894) 

Pooled (Rounds 2 & 3) 3.896 (0.026) 3.940 (0.025) -0.044 (0.222) 

Round 2 3.942 (0.029) 3.950 (0.029) -0.008 (0.850) 

Round 3 3.849 (0.033) 3.931 (0.032) -0.082 (0.077)* 

Change: Round 3 – Round 2 -0.090 (0.077)* 

I enjoy trying new vegetables (N=2898) 

Pooled (Rounds 2 & 3) 3.687 (0.028) 3.654 (0.028) 0.033 (0.403) 

Round 2 3.709 (0.033) 3.649 (0.034) 0.059 (0.214) 

Round 3 3.664 (0.033) 3.658 (0.032) 0.006 (0.890) 

Change: Round 3 – Round 2 -0.081 (0.119) 

I eat enough fruits to keep me healthy (N=2894)

Pooled (Rounds 2 & 3) 3.724 (0.028) 3.682 (0.027) 0.041 (0.293) 

Round 2 3.746 (0.034) 3.642 (0.034) 0.104 (0.033)** 

Round 3 3.701 (0.034) 3.723 (0.032) -0.022 (0.635) 

Change: Round 3 – Round 2 -0.148 (0.008)*** 

I eat enough vegetables to keep me healthy (N=2903)

Pooled (Rounds 2 & 3) 3.738 (0.028) 3.703 (0.025) 0.035 (0.363) 

Round 2 3.737 (0.034) 3.700 (0.031) 0.037 (0.432) 

Round 3 3.739 (0.031) 3.706 (0.031) 0.032 (0.464) 

Change: Round 3 – Round 2 -0.003 (0.953) 

I often encourage family/friends to eat fruits & veg. (N=2857)

Pooled (Rounds 2 & 3) 3.876 (0.026) 3.820 (0.027) 0.056 (0.132) 

Round 2 3.873 (0.032) 3.831 (0.032) 0.043 (0.350) 

Round 3 3.879 (0.030) 3.809 (0.032) 0.070 (0.115) 

Change: Round 3 – Round 2 0.001 (0.988) 

Composite scale—Positive attitudes about food, fruits, and vegetables (N=2902)

Pooled (Rounds 2 & 3) 3.803 (0.016) 3.771 (0.016) 0.033 (0.159) 

Round 2 3.818 (0.019) 3.764 (0.019) 0.054 (0.044)** 

Round 3 3.788 (0.019) 3.777 (0.019) 0.011 (0.686) 

Change: Round 3 – Round 2 -0.062 (0.027)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly 

agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 351

Exhibit G7.3: Differences in Impacts of HIP on Perceived Barriers to Fruit & Vegetable 

Consumption, by Follow-Up Round 

How much do you agree 

or disagree that...? 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Hard to eat vegetables because don't know how to prepare (N=2876)

Pooled (Rounds 2 & 3) 2.406 (0.032) 2.400 (0.028) 0.006 (0.882) 

Round 2 2.434 (0.041) 2.392 (0.037) 0.042 (0.454) 

Round 3 2.378 (0.037) 2.408 (0.036) -0.030 (0.568) 

Change: Round 3 – Round 2 -0.086 (0.190) 

Hard to eat vegetables because hard to find where I shop (N=2878)

Pooled (Rounds 2 & 3) 2.196 (0.026) 2.222 (0.024) -0.025 (0.480) 

Round 2 2.236 (0.034) 2.204 (0.031) 0.032 (0.498) 

Round 3 2.155 (0.032) 2.239 (0.031) -0.083 (0.061)* 

Change: Round 3 – Round 2 -0.120 (0.035)** 

Hard to eat fruits because hard to find where I shop (N=2880)

Pooled (Rounds 2 & 3) 2.164 (0.025) 2.162 (0.024) 0.002 (0.948) 

Round 2 2.192 (0.031) 2.150 (0.031) 0.041 (0.349) 

Round 3 2.136 (0.031) 2.173 (0.030) -0.037 (0.378) 

Change: Round 3 – Round 2 -0.067 (0.219) 

Don't eat fruits and vegetables as much as would like because cost too much (N=2889) 

Pooled (Rounds 2 & 3) 2.927 (0.035) 2.957 (0.033) -0.030 (0.529) 

Round 2 2.913 (0.041) 2.937 (0.039) -0.024 (0.674) 

Round 3 2.940 (0.042) 2.976 (0.040) -0.036 (0.531) 

Change: Round 3 – Round 2 -0.042 (0.508) 

Don't eat fruits and vegetables as much as would like because they spoil (N=2881) 

Pooled (Rounds 2 & 3) 2.815 (0.033) 2.918 (0.032) -0.103 (0.025)** 

Round 2 2.795 (0.041) 2.932 (0.038) -0.137 (0.014)** 

Round 3 2.835 (0.040) 2.903 (0.038) -0.068 (0.217) 

Change: Round 3 – Round 2 0.045 (0.477) 

Don't eat fruits and vegetables as much as would like because family dislikes (N=2749) 

Pooled (Rounds 2 & 3) 2.149 (0.026) 2.115 (0.023) 0.034 (0.327) 

Round 2 2.168 (0.033) 2.144 (0.030) 0.024 (0.597) 

Round 3 2.131 (0.033) 2.087 (0.028) 0.044 (0.306) 

Change: Round 3 – Round 2 0.004 (0.947) 

Don't eat fruits and vegetables as much because I don't like (N=2888)

Pooled (Rounds 2 & 3) 2.026 (0.022) 2.068 (0.023) -0.041 (0.191) 

Round 2 2.033 (0.029) 2.052 (0.028) -0.018 (0.649) 

Round 3 2.019 (0.028) 2.084 (0.028) -0.065 (0.100) 

Change: Round 3 – Round 2 -0.058 (0.235) 

Composite scale—barriers to eating fruits & vegetables (N=2869)

Pooled (Rounds 2 & 3) 2.380 (0.017) 2.406 (0.016) -0.026 (0.249) 

Round 2 2.393 (0.021) 2.404 (0.018) -0.011 (0.689) 

Round 3 2.367 (0.020) 2.409 (0.018) -0.042 (0.118) 

Change: Round 3 – Round 2 -0.046 (0.132) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly 

agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 352 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module). 

Exhibit G7.4: Differences in Impacts of HIP on Grocery Shopping Barriers, by Follow-Up 

Round 

How often kept from 

grocery shopping by... 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Limited transportation (N=2745) 

Pooled (Rounds 2 & 3) 2.178 (0.038) 2.098 (0.035) 0.080 (0.127) 

Round 2 2.172 (0.046) 2.123 (0.042) 0.048 (0.442) 

Round 3 2.185 (0.046) 2.073 (0.045) 0.112 (0.085)* 

Change: Round 3 – Round 2 0.079 (0.276) 

Distance to grocery store (N=32726) 

Pooled (Rounds 2 & 3) 1.961 (0.037) 1.956 (0.035) 0.006 (0.912) 

Round 2 1.927 (0.043) 2.033 (0.044) -0.106 (0.085)* 

Round 3 1.996 (0.046) 1.878 (0.043) 0.119 (0.062)* 

Change: Round 3 – Round 2 0.231 (0.001)*** 

Composite scale—barriers to grocery shopping (N=2706)

Pooled (Rounds 2 & 3) 2.064 (0.034) 2.019 (0.032) 0.045 (0.347) 

Round 2 2.042 (0.040) 2.073 (0.039) -0.032 (0.573) 

Round 3 2.087 (0.041) 1.965 (0.039) 0.122 (0.033)** 

Change: Round 3 – Round 2 0.163 (0.008)*** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always; “don’t know” and 

“refused” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 353

Exhibit G7.5: Differences in Impacts of HIP on Family Food Environment, by Follow-Up 

Round 

How often do you...? 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Have fruit available at home (N=2794)

Pooled (Rounds 2 & 3) 4.295 (0.025) 4.188 (0.026) 0.107 (0.003)*** 

Round 2 4.333 (0.029) 4.180 (0.031) 0.154 (<0.001) *** 

Round 3 4.255 (0.032) 4.197 (0.031) 0.058 (0.199) 

Change: Round 3 – Round 2 -0.104 (0.035)** 

Have fruit in refrigerator or on counter (N=2789)a

Pooled (Rounds 2 & 3) 4.000 (0.028) 3.950 (0.027) 0.049 (0.209) 

Round 2 4.021 (0.033) 3.964 (0.032) 0.057 (0.215) 

Round 3 3.977 (0.036) 3.937 (0.034) 0.041 (0.409) 

Change: Round 3 – Round 2 -0.010 (0.857) 

Have vegetables available at home (N=2794)a

Pooled (Rounds 2 & 3) 4.499 (0.025) 4.424 (0.025) 0.076 (0.031)** 

Round 2 4.508 (0.029) 4.417 (0.031) 0.091 (0.031)** 

Round 3 4.490 (0.032) 4.430 (0.029) 0.060 (0.164) 

Change: Round 3 – Round 2 -0.053 (0.281) 

Have ready-to-eat vegetables in fridge or on counter (N=2769)

Pooled (Rounds 2 & 3) 3.901 (0.034) 3.828 (0.033) 0.073 (0.126) 

Round 2 3.862 (0.043) 3.808 (0.041) 0.054 (0.359) 

Round 3 3.942 (0.041) 3.848 (0.040) 0.094 (0.104) 

Change: Round 3 – Round 2 0.031 (0.640) 

Composite scale—barriers to eating fruits & vegetables (N=2796)

Pooled (Rounds 2 & 3) 4.171 (0.021) 4.093 (0.021) 0.079 (0.009)*** 

Round 2 4.179 (0.023) 4.085 (0.025) 0.094 (0.006)*** 

Round 3 4.163 (0.026) 4.100 (0.025) 0.063 (0.079)* 

Change: Round 3 – Round 2 -0.037 (0.305) 

Have salty snacks at home (chips, crackers) (N=2789)

Pooled (Rounds 2 & 3) 3.193 (0.035) 3.206 (0.034) -0.013 (0.791) 

Round 2 3.245 (0.043) 3.228 (0.041) 0.018 (0.769) 

Round 3 3.139 (0.044) 3.184 (0.044) -0.045 (0.470) 

Change: Round 3 – Round 2 -0.060 (0.405) 

Have lowfat/nonfat milk at home (N=2775)

Pooled (Rounds 2 & 3) 3.348 (0.051) 3.297 (0.047) 0.050 (0.462) 

Round 2 3.362 (0.060) 3.305 (0.057) 0.057 (0.488) 

Round 3 3.333 (0.063) 3.290 (0.061) 0.044 (0.616) 

Change: Round 3 – Round 2 0.040 (0.687) 

Have soft drinks/fruit drinks (not juice) at home (N=2790)

Pooled (Rounds 2 & 3) 3.151 (0.042) 3.233 (0.040) -0.082 (0.161) 

Round 2 3.135 (0.049) 3.177 (0.049) -0.042 (0.544) 

Round 3 3.167 (0.051) 3.290 (0.050) -0.122 (0.090)* 

Change: Round 3 – Round 2 -0.095 (0.228) 

Sit down with family at home for evening meals (N=1900)b

Pooled (Rounds 2 & 3) 3.911 (0.039) 3.935 (0.034) -0.024 (0.640) 

Round 2 3.940 (0.049) 3.961 (0.039) -0.021 (0.733) 

Round 3 3.882 (0.050) 3.908 (0.045) -0.026 (0.698) 

Change: Round 3 – Round 2 -0.028 (0.712) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 354 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

How often do you...? 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Cook evening meals at home (N=2790) 

Pooled (Rounds 2 & 3) 4.442 (0.024) 4.429 (0.024) 0.013 (0.710) 

Round 2 4.461 (0.028) 4.410 (0.031) 0.051 (0.229) 

Round 3 4.422 (0.030) 4.448 (0.028) -0.027 (0.518) 

Change: Round 3 – Round 2 -0.077 (0.108) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Never, 2=Rarely, 3=Sometimes, 4=Most of the time, 5=Always; “don’t know” responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted; inclusion of weights results in the observed 

difference in point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. a

”No refrigerator or freezer” responses coded as missing. 

b

Asked only in households with more than one member. 

Source: Participant Survey (primary shopper module). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 355

Exhibit G8.1: Differences in Impacts of HIP on Consumption of Fruits and Vegetables, by 

Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Targeted fruits & vegetables

Pooled (Rounds 2 & 3) 1.138 (0.045) 0.932 (0.040) 0.206 (0.001)*** 

Round 2 1.179 (0.058) 1.003 (0.051) 0.176 (0.020)** 

Round 3 1.097 (0.055) 0.862 (0.049) 0.235 (0.001)*** 

Change: Round 3 – Round 2 0.040 (0.653) 

Plus TFV from mixed foods

Pooled (Rounds 2 & 3) 0.298 (0.015) 0.301 (0.015) -0.003 (0.898) 

Round 2 0.301 (0.020) 0.316 (0.021) -0.015 (0.627) 

Round 3 0.295 (0.020) 0.286 (0.018) 0.009 (0.749) 

Change: Round 3 – Round 2 0.015 (0.684) 

Targeted fruits & vegetables (alternative inclusive proxy measure)

Pooled (Rounds 2 & 3) 1.437 (0.049) 1.233 (0.044) 0.204 (0.002)*** 

Round 2 1.481 (0.061) 1.319 (0.055) 0.162 (0.048)** 

Round 3 1.392 (0.059) 1.148 (0.053) 0.244 (0.002)*** 

Change: Round 3 – Round 2 0.055 (0.555) 

100% juice 

Pooled (Rounds 2 & 3) 0.544 (0.029) 0.477 (0.026) 0.068 (0.071)* 

Round 2 0.569 (0.039) 0.500 (0.038) 0.069 (0.196) 

Round 3 0.519 (0.034) 0.454 (0.033) 0.066 (0.160) 

Change: Round 3 – Round 2 -0.028 (0.672) 

White potatoes 

Pooled (Rounds 2 & 3) 0.332 (0.018) 0.368 (0.019) -0.036 (0.185) 

Round 2 0.358 (0.026) 0.350 (0.023) 0.008 (0.815) 

Round 3 0.307 (0.023) 0.387 (0.030) -0.080 (0.037)** 

Change: Round 3 – Round 2 -0.094 (0.063)* 

Legumes 

Pooled (Rounds 2 & 3) 0.103 (0.008) 0.113 (0.007) -0.010 (0.338) 

Round 2 0.106 (0.010) 0.110 (0.010) -0.004 (0.776) 

Round 3 0.101 (0.010) 0.117 (0.011) -0.016 (0.275) 

Change: Round 3 – Round 2 -0.013 (0.518) 

Other fruits & vegetables acquired outside stores

Pooled (Rounds 2 & 3) 0.176 (0.019) 0.147 (0.017) 0.029 (0.282) 

Round 2 0.183 (0.025) 0.154 (0.026) 0.029 (0.439) 

Round 3 0.169 (0.024) 0.140 (0.022) 0.029 (0.378) 

Change: Round 3 – Round 2 -0.002 (0.971) 

All fruits & vegetables 

Pooled (Rounds 2 & 3) 2.592 (0.066) 2.338 (0.064) 0.254 (0.005)*** 

Round 2 2.696 (0.084) 2.432 (0.083) 0.264 (0.023)** 

Round 3 2.488 (0.080) 2.246 (0.077) 0.242 (0.027)** 

Change: Round 3 – Round 2 -0.081 (0.551) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 356 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits and vegetables acquired from 

the store, excluding white potatoes, legumes, and 100% juice. The preferred restrictive proxy measure 

additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was 

not identified by the respondent, while the alternative inclusive proxy measure includes fruit and vegetable intake 

from all mixed foods. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 357

Exhibit G8.2a: Differences in Impacts of HIP on Consumption of Targeted Fruits & Vegetables 

(TFV), Preferred Restrictive Proxy Measure, Cup-Equivalents, by Food Pattern 

Equivalent Group, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Total fruits & vegetables 

Pooled (Rounds 2 & 3) 1.138 (0.045) 0.932 (0.040) 0.206 (0.001)*** 

Round 2 1.179 (0.058) 1.003 (0.051) 0.176 (0.020)** 

Round 3 1.097 (0.055) 0.862 (0.049) 0.235 (0.001)*** 

Change: Round 3 – Round 2 0.040 (0.653) 

Total fruits 

Pooled (Rounds 2 & 3) 0.554 (0.028) 0.492 (0.028) 0.062 (0.121) 

Round 2 0.593 (0.035) 0.533 (0.036) 0.060 (0.234) 

Round 3 0.514 (0.034) 0.452 (0.035) 0.063 (0.201) 

Change: Round 3 – Round 2 0.003 (0.966) 

Citrus fruits, melons, & berries

Pooled (Rounds 2 & 3) 0.125 (0.011) 0.127 (0.015) -0.003 (0.891) 

Round 2 0.164 (0.016) 0.138 (0.017) 0.026 (0.265) 

Round 3 0.085 (0.013) 0.116 (0.024) -0.031 (0.256) 

Change: Round 3 – Round 2 -0.053 (0.131) 

Other fruits (e.g., apples, pears, bananas, grapes, peaches)

Pooled (Rounds 2 & 3) 0.429 (0.024) 0.365 (0.022) 0.064 (0.050)** 

Round 2 0.429 (0.030) 0.395 (0.030) 0.034 (0.417) 

Round 3 0.429 (0.030) 0.335 (0.025) 0.094 (0.017)** 

Change: Round 3 – Round 2 0.056 (0.243) 

Total vegetables 

Pooled (Rounds 2 & 3) 0.584 (0.029) 0.440 (0.023) 0.145 (<0.001) *** 

Round 2 0.586 (0.037) 0.470 (0.030) 0.116 (0.013)** 

Round 3 0.583 (0.039) 0.410 (0.031) 0.172 (<0.001) *** 

Change: Round 3 – Round 2 0.038 (0.543) 

Dark green vegetables 

Pooled (Rounds 2 & 3) 0.072 (0.008) 0.044 (0.006) 0.028 (0.004)*** 

Round 2 0.081 (0.010) 0.049 (0.008) 0.032 (0.014)** 

Round 3 0.063 (0.010) 0.039 (0.007) 0.024 (0.061)* 

Change: Round 3 – Round 2 -0.007 (0.687) 

Red & orange vegetables 

Pooled (Rounds 2 & 3) 0.141 (0.011) 0.104 (0.008) 0.037 (0.007)*** 

Round 2 0.157 (0.018) 0.106 (0.010) 0.051 (0.014)** 

Round 3 0.124 (0.011) 0.102 (0.011) 0.023 (0.160) 

Change: Round 3 – Round 2 -0.041 (0.115) 

Tomatoes 

Pooled (Rounds 2 & 3) 0.087 (0.007) 0.070 (0.006) 0.016 (0.070)* 

Round 2 0.086 (0.010) 0.069 (0.007) 0.017 (0.150) 

Round 3 0.087 (0.009) 0.072 (0.009) 0.016 (0.219) 

Change: Round 3 – Round 2 -0.011 (0.497) 

Other red & orange vegetables

Pooled (Rounds 2 & 3) 0.054 (0.008) 0.034 (0.004) 0.020 (0.030)** 

Round 2 0.071 (0.015) 0.037 (0.007) 0.034 (0.042)** 

Round 3 0.037 (0.006) 0.030 (0.006) 0.007 (0.394) 

Change: Round 3 – Round 2 -0.030 (0.119) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 358 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Starchy vegetables 

Pooled (Rounds 2 & 3) 0.100 (0.010) 0.086 (0.008) 0.015 (0.263) 

Round 2 0.088 (0.011) 0.099 (0.012) -0.012 (0.484) 

Round 3 0.113 (0.018) 0.072 (0.010) 0.040 (0.047)** 

Change: Round 3 – Round 2 0.053 (0.043)** 

Other vegetables (e.g., celery, cucumbers, mushrooms, green beans, onions, asparagus)

Pooled (Rounds 2 & 3) 0.272 (0.017) 0.206 (0.016) 0.066 (0.003)*** 

Round 2 0.261 (0.022) 0.216 (0.021) 0.045 (0.127) 

Round 3 0.283 (0.024) 0.197 (0.021) 0.086 (0.007)*** 

Change: Round 3 – Round 2 0.033 (0.417) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits and vegetables acquired from 

the store, excluding white potatoes, legumes, and 100% juice. The preferred restrictive proxy measure 

additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was 

not identified by the respondent, while the alternative inclusive proxy measure includes fruit and vegetable intake 

from all mixed foods. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 359

Exhibit G8.2b: Differences in Impacts of HIP on Consumption of Targeted Fruits & Vegetables 

(TFV), Alternative Inclusive Proxy Measure, Cup-Equivalents, by Food Pattern 

Equivalent Group, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Total fruits & vegetables 

Pooled (Rounds 2 & 3) 1.233 (0.044) 0.204 (0.002)*** 

Round 2 1.481 (0.061) 1.319 (0.055) 0.162 (0.048)** 

Round 3 1.392 (0.059) 1.148 (0.053) 0.244 (0.002)*** 

Change: Round 3 – Round 2 0.055 (0.555) 

Total fruits 

Pooled (Rounds 2 & 3) 0.626 (0.030) 0.558 (0.029) 0.068 (0.103) 

Round 2 0.667 (0.038) 0.604 (0.037) 0.063 (0.236) 

Round 3 0.585 (0.036) 0.514 (0.036) 0.072 (0.160) 

Change: Round 3 – Round 2 0.012 (0.843) 

Citrus fruits, melons, & berries

Pooled (Rounds 2 & 3) 0.143 (0.011) 0.142 (0.015) 0.001 (0.960) 

Round 2 0.179 (0.016) 0.155 (0.017) 0.024 (0.310) 

Round 3 0.107 (0.014) 0.129 (0.024) -0.022 (0.431) 

Change: Round 3 – Round 2 -0.042 (0.248) 

Other fruits (e.g., apples, pears, bananas, grapes, peaches)

Pooled (Rounds 2 & 3) 0.483 (0.026) 0.416 (0.023) 0.067 (0.051)* 

Round 2 0.488 (0.033) 0.449 (0.031) 0.039 (0.389) 

Round 3 0.478 (0.031) 0.384 (0.026) 0.094 (0.021)** 

Change: Round 3 – Round 2 0.054 (0.292) 

Total vegetables 

Pooled (Rounds 2 & 3) 0.810 (0.032) 0.675 (0.028) 0.136 (0.001)*** 

Round 2 0.814 (0.040) 0.715 (0.035) 0.099 (0.061)* 

Round 3 0.807 (0.042) 0.635 (0.035) 0.172 (0.002)*** 

Change: Round 3 – Round 2 0.043 (0.512) 

Dark green vegetables 

Pooled (Rounds 2 & 3) 0.079 (0.008) 0.050 (0.006) 0.029 (0.004)*** 

Round 2 0.089 (0.010) 0.057 (0.009) 0.032 (0.018)** 

Round 3 0.070 (0.011) 0.044 (0.007) 0.026 (0.041)** 

Change: Round 3 – Round 2 -0.003 (0.864) 

Red & orange vegetables 

Pooled (Rounds 2 & 3) 0.268 (0.014) 0.229 (0.011) 0.039 (0.024)** 

Round 2 0.277 (0.021) 0.232 (0.015) 0.044 (0.083)* 

Round 3 0.260 (0.016) 0.226 (0.014) 0.034 (0.119) 

Change: Round 3 – Round 2 -0.028 (0.379) 

Tomatoes 

Pooled (Rounds 2 & 3) 0.200 (0.010) 0.179 (0.009) 0.021 (0.134) 

Round 2 0.191 (0.014) 0.182 (0.013) 0.009 (0.626) 

Round 3 0.209 (0.015) 0.176 (0.012) 0.033 (0.096)* 

Change: Round 3 – Round 2 0.009 (0.723) 

Other red & orange vegetables

Pooled (Rounds 2 & 3) 0.069 (0.009) 0.050 (0.005) 0.018 (0.063)* 

Round 2 0.085 (0.016) 0.050 (0.007) 0.035 (0.043)** 

Round 3 0.052 (0.006) 0.050 (0.006) 0.002 (0.844) 

Change: Round 3 – Round 2 -0.037 (0.063)* 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 360 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Starchy vegetables 

Pooled (Rounds 2 & 3) 0.115 (0.011) 0.100 (0.008) 0.015 (0.256) 

Round 2 0.104 (0.011) 0.115 (0.013) -0.012 (0.485) 

Round 3 0.126 (0.018) 0.084 (0.010) 0.042 (0.044)** 

Change: Round 3 – Round 2 0.053 (0.048)** 

Other vegetables (e.g., celery, cucumbers, mushrooms, green beans, onions, asparagus)

Pooled (Rounds 2 & 3) 0.348 (0.019) 0.296 (0.019) 0.052 (0.046)** 

Round 2 0.345 (0.023) 0.310 (0.024) 0.035 (0.299) 

Round 3 0.351 (0.026) 0.281 (0.024) 0.070 (0.044)** 

Change: Round 3 – Round 2 0.021 (0.610) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits and vegetables acquired from 

the store, excluding white potatoes, legumes, and 100% juice. The preferred restrictive proxy measure 

additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was 

not identified by the respondent, while the alternative inclusive proxy measure includes fruit and vegetable intake 

from all mixed foods. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 361

Exhibit G8.2c: Differences in Impacts of HIP on Consumption of All Fruits & Vegetables, CupEquivalents, by Food Pattern Equivalent Group, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Total fruits & vegetables 

Pooled (Rounds 2 & 3) 2.592 (0.066) 2.338 (0.064) 0.254 (0.005)*** 

Round 2 2.696 (0.084) 2.432 (0.083) 0.264 (0.023)** 

Round 3 2.488 (0.080) 2.246 (0.077) 0.242 (0.027)** 

Change: Round 3 – Round 2 -0.081 (0.551) 

Total fruits 

Pooled (Rounds 2 & 3) 1.244 (0.044) 1.080 (0.043) 0.164 (0.007)*** 

Round 2 1.320 (0.058) 1.140 (0.058) 0.180 (0.027)** 

Round 3 1.168 (0.053) 1.021 (0.052) 0.147 (0.046)** 

Change: Round 3 – Round 2 -0.057 (0.557) 

Citrus fruits, melons, & berries

Pooled (Rounds 2 & 3) 0.169 (0.013) 0.152 (0.015) 0.017 (0.417) 

Round 2 0.206 (0.018) 0.163 (0.017) 0.043 (0.086)* 

Round 3 0.131 (0.015) 0.141 (0.025) -0.010 (0.736) 

Change: Round 3 – Round 2 0.040 (0.653) 

Other fruits (e.g., apples, pears, bananas, grapes, peaches)

Pooled (Rounds 2 & 3) 0.531 (0.026) 0.451 (0.023) 0.080 (0.022)** 

Round 2 0.545 (0.034) 0.477 (0.031) 0.068 (0.140) 

Round 3 0.518 (0.032) 0.426 (0.027) 0.091 (0.029)** 

Change: Round 3 – Round 2 0.022 (0.678) 

100% fruit juice 

Pooled (Rounds 2 & 3) 0.544 (0.029) 0.477 (0.026) 0.068 (0.071)* 

Round 2 0.569 (0.039) 0.500 (0.038) 0.069 (0.196) 

Round 3 0.519 (0.034) 0.454 (0.033) 0.066 (0.160) 

Change: Round 3 – Round 2 -0.028 (0.672) 

Total vegetables 

Pooled (Rounds 2 & 3) 1.348 (0.039) 1.258 (0.039) 0.090 (0.107) 

Round 2 1.376 (0.050) 1.292 (0.050) 0.084 (0.232) 

Round 3 1.320 (0.052) 1.225 (0.051) 0.095 (0.194) 

Change: Round 3 – Round 2 -0.025 (0.783) 

Dark green vegetables 

Pooled (Rounds 2 & 3) 0.095 (0.009) 0.067 (0.007) 0.028 (0.011)** 

Round 2 0.102 (0.011) 0.077 (0.010) 0.026 (0.092)* 

Round 3 0.088 (0.012) 0.057 (0.008) 0.031 (0.030)** 

Change: Round 3 – Round 2 0.009 (0.662) 

Red & orange vegetables 

Pooled (Rounds 2 & 3) 0.337 (0.015) 0.313 (0.012) 0.025 (0.212) 

Round 2 0.344 (0.022) 0.323 (0.018) 0.022 (0.446) 

Round 3 0.330 (0.020) 0.303 (0.016) 0.028 (0.277) 

Change: Round 3 – Round 2 -0.012 (0.734) 

Tomatoes 

Pooled (Rounds 2 & 3) 0.261 (0.013) 0.250 (0.011) 0.011 (0.507) 

Round 2 0.252 (0.016) 0.258 (0.016) -0.006 (0.786) 

Round 3 0.270 (0.018) 0.241 (0.014) 0.028 (0.217) 

Change: Round 3 – Round 2 0.020 (0.491) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 362 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Other red & orange vegetables

Pooled (Rounds 2 & 3) 0.076 (0.009) 0.063 (0.005) 0.014 (0.187) 

Round 2 0.092 (0.016) 0.064 (0.008) 0.028 (0.119) 

Round 3 0.061 (0.007) 0.061 (0.007) -0.001 (0.939) 

Change: Round 3 – Round 2 -0.033 (0.118) 

Starchy vegetables 

Pooled (Rounds 2 & 3) 0.458 (0.021) 0.477 (0.021) -0.019 (0.535) 

Round 2 0.475 (0.029) 0.472 (0.027) 0.003 (0.949) 

Round 3 0.441 (0.029) 0.481 (0.031) -0.040 (0.354) 

Change: Round 3 – Round 2 -0.048 (0.414) 

White potatoes 

Pooled (Rounds 2 & 3) 0.332 (0.018) 0.368 (0.019) -0.036 (0.185) 

Round 2 0.358 (0.026) 0.350 (0.023) 0.008 (0.815) 

Round 3 0.307 (0.023) 0.387 (0.030) -0.080 (0.037)** 

Change: Round 3 – Round 2 -0.094 (0.063)* 

Other starchy vegetables 

Pooled (Rounds 2 & 3) 0.126 (0.011) 0.108 (0.008) 0.017 (0.197) 

Round 2 0.117 (0.012) 0.122 (0.013) -0.006 (0.746) 

Round 3 0.134 (0.018) 0.094 (0.011) 0.040 (0.055)* 

Change: Round 3 – Round 2 0.046 (0.088)* 

Legumes 

Pooled (Rounds 2 & 3) 0.103 (0.008) 0.113 (0.007) -0.010 (0.338) 

Round 2 0.106 (0.010) 0.110 (0.010) -0.004 (0.776) 

Round 3 0.101 (0.010) 0.117 (0.011) -0.016 (0.275) 

Change: Round 3 – Round 2 -0.013 (0.518) 

Other vegetables (e.g., celery, cucumbers, mushrooms, green beans, onions, asparagus)

Pooled (Rounds 2 & 3) 0.416 (0.019) 0.359 (0.020) 0.057 (0.037)** 

Round 2 0.411 (0.024) 0.373 (0.026) 0.038 (0.292) 

Round 3 0.422 (0.027) 0.345 (0.024) 0.077 (0.033)** 

Change: Round 3 – Round 2 0.024 (0.589) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 363

Exhibit G8.3: Differences in Impacts of HIP on Estimated Usual Daily Intake from Fruit & 

Vegetable Screener, Cups, by Follow-Up Round 

Regressionadjusted 

treatment 

mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

100% juice (N=3692) 

Pooled (Rounds 2 & 3) 1.009 (0.041) 0.930 (0.040) 0.079 (0.154) 

Round 2 1.020 (0.048) 0.937 (0.045) 0.083 (0.200) 

Round 3 0.994 (0.059) 0.921 (0.063) 0.073 (0.386) 

Change: Round 3 – Round 2 -0.045 (0.656) 

Fruit (N=3697) 

Pooled (Rounds 2 & 3) 0.675 (0.026) 0.608 (0.022) 0.067 (0.042)** 

Round 2 0.714 (0.031) 0.626 (0.027) 0.088 (0.029)** 

Round 3 0.624 (0.038) 0.584 (0.033) 0.039 (0.433) 

Change: Round 3 – Round 2 -0.100 (0.096)* 

Salad (N=3713) 

Pooled (Rounds 2 & 3) 0.306 (0.010) 0.281 (0.010) 0.025 (0.079)* 

Round 2 0.329 (0.013) 0.290 (0.014) 0.039 (0.038)** 

Round 3 0.275 (0.012) 0.269 (0.012) 0.006 (0.709) 

Change: Round 3 – Round 2 -0.046 (0.059)* 

Fried potatoes (N=3731) 

Pooled (Rounds 2 & 3) 0.055 (0.003) 0.072 (0.007) -0.017 (0.052)* 

Round 2 0.057 (0.004) 0.075 (0.012) -0.018 (0.151) 

Round 3 0.053 (0.005) 0.068 (0.005) -0.015 (0.047)** 

Change: Round 3 – Round 2 0.018 (0.368) 

Other potatoes (N=3721) 

Pooled (Rounds 2 & 3) 0.215 (0.010) 0.206 (0.008) 0.010 (0.446) 

Round 2 0.222 (0.011) 0.194 (0.009) 0.028 (0.061)* 

Round 3 0.207 (0.016) 0.221 (0.013) -0.014 (0.489) 

Change: Round 3 – Round 2 -0.034 (0.159) 

Beans (N=3718) 

Pooled (Rounds 2 & 3) 0.193 (0.010) 0.187 (0.008) 0.005 (0.672) 

Round 2 0.197 (0.014) 0.189 (0.010) 0.008 (0.654) 

Round 3 0.187 (0.012) 0.185 (0.010) 0.002 (0.881) 

Change: Round 3 – Round 2 0.007 (0.718) 

Other vegetables (N=3692) 

Pooled (Rounds 2 & 3) 0.506 (0.018) 0.476 (0.016) 0.030 (0.211) 

Round 2 0.519 (0.023) 0.480 (0.019) 0.039 (0.187) 

Round 3 0.489 (0.025) 0.471 (0.023) 0.019 (0.584) 

Change: Round 3 – Round 2 -0.037 (0.380) 

Tomato sauce (N=3673) 

Pooled (Rounds 2 & 3) 0.102 (0.005) 0.107 (0.005) -0.005 (0.467) 

Round 2 0.103 (0.006) 0.111 (0.006) -0.007 (0.427) 

Round 3 0.099 (0.007) 0.102 (0.006) -0.003 (0.761) 

Change: Round 3 – Round 2 0.004 (0.752) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 364 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Regressionadjusted 

treatment 

mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Salsa (N=3725) 

Pooled (Rounds 2 & 3) 0.009 (0.001) 0.009 (0.001) <0.001 (0.980) 

Round 2 0.008 (0.001) 0.009 (0.001) >-0.001 (0.853) 

Round 3 0.009 (0.002) 0.008 (0.001) <0.001 (0.805) 

Change: Round 3 – Round 2 0.002 (0.434) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

"Refused" and "don't know" responses on frequency or amount items coded as missing 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Pooled standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 365

Exhibit G8.4: Differences in Impacts of HIP on Threshold Measures of Fruit & Vegetable 

Intake in the Past 24 Hours, by Follow-Up Round 

Regressionadjusted 

treatment mean

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Consumed any fruits & vegetables

Pooled (Rounds 2 & 3) 0.964 (0.005) 0.958 (0.006) 0.006 (0.420) 

Round 2 0.972 (0.006) 0.956 (0.008) 0.016 (0.106) 

Round 3 0.956 (0.008) 0.960 (0.007) -0.003 (0.756) 

Change: Round 3 – Round 2 -0.021 (0.098)* 

Consumed 1 or more cup-eq of fruits & vegetables

Pooled (Rounds 2 & 3) 0.753 (0.013) 0.720 (0.012) 0.033 (0.061)* 

Round 2 0.775 (0.016) 0.722 (0.016) 0.053 (0.019)** 

Round 3 0.731 (0.017) 0.718 (0.016) 0.013 (0.586) 

Change: Round 3 – Round 2 -0.051 (0.086)* 

Consumed 2.5 or more cup-eq of fruits & vegetables

Pooled (Rounds 2 & 3) 0.411 (0.015) 0.358 (0.013) 0.053 (0.007)*** 

Round 2 0.424 (0.019) 0.377 (0.018) 0.047 (0.063)* 

Round 3 0.398 (0.019) 0.340 (0.017) 0.058 (0.024)** 

Change: Round 3 – Round 2 -0.001 (0.983) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 366 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Exhibit G8.5: Differences in Impacts of HIP on Probability of Any Fruit and Vegetable Intake 

in the Past 24 Hours, by Food Equivalent Pattern Group, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Total fruits & vegetables 

Pooled (Rounds 2 & 3) 0.964 (0.005) 0.958 (0.006) 0.006 (0.420) 

Round 2 0.972 (0.006) 0.956 (0.008) 0.016 (0.106) 

Round 3 0.956 (0.008) 0.960 (0.007) -0.003 (0.756) 

Change: Round 3 – Round 2 -0.021 (0.098)* 

Total fruits 

Pooled (Rounds 2 & 3) 0.747 (0.013) 0.728 (0.012) 0.019 (0.295) 

Round 2 0.757 (0.017) 0.734 (0.016) 0.023 (0.316) 

Round 3 0.737 (0.017) 0.722 (0.016) 0.015 (0.527) 

Change: Round 3 – Round 2 -0.002 (0.954) 

Citrus fruits, melons, & berries

Pooled (Rounds 2 & 3) 0.254 (0.013) 0.232 (0.012) 0.022 (0.199) 

Round 2 0.299 (0.018) 0.255 (0.016) 0.044 (0.067)* 

Round 3 0.209 (0.015) 0.208 (0.015) 0.001 (0.980) 

Change: Round 3 – Round 2 -0.036 (0.221) 

Other fruits (e.g., apples, pears, bananas, grapes, peaches)

Pooled (Rounds 2 & 3) 0.479 (0.015) 0.437 (0.014) 0.042 (0.040)** 

Round 2 0.483 (0.019) 0.439 (0.018) 0.043 (0.098)* 

Round 3 0.476 (0.019) 0.436 (0.018) 0.040 (0.124) 

Change: Round 3 – Round 2 <0.001 (0.994) 

100% fruit juice 

Pooled (Rounds 2 & 3) 0.507 (0.015) 0.464 (0.014) 0.044 (0.034)** 

Round 2 0.522 (0.019) 0.452 (0.018) 0.070 (0.007)*** 

Round 3 0.492 (0.019) 0.475 (0.019) 0.017 (0.516) 

Change: Round 3 – Round 2 -0.052 (0.121) 

Total vegetables 

Pooled (Rounds 2 & 3) 0.907 (0.009) 0.898 (0.008) 0.009 (0.470) 

Round 2 0.923 (0.010) 0.899 (0.011) 0.025 (0.104) 

Round 3 0.891 (0.012) 0.898 (0.011) -0.008 (0.642) 

Change: Round 3 – Round 2 -0.038 (0.061)* 

Dark green vegetables 

Pooled (Rounds 2 & 3) 0.159 (0.011) 0.124 (0.009) 0.034 (0.013)** 

Round 2 0.176 (0.014) 0.144 (0.013) 0.032 (0.102) 

Round 3 0.142 (0.014) 0.105 (0.011) 0.037 (0.038)** 

Change: Round 3 – Round 2 0.010 (0.680) 

Red & orange vegetables 

Pooled (Rounds 2 & 3) 0.702 (0.013) 0.688 (0.013) 0.014 (0.443) 

Round 2 0.730 (0.017) 0.696 (0.017) 0.034 (0.159) 

Round 3 0.674 (0.019) 0.679 (0.017) -0.005 (0.831) 

Change: Round 3 – Round 2 -0.045 (0.171) 

Tomatoes 

Pooled (Rounds 2 & 3) 0.608 (0.014) 0.606 (0.013) 0.003 (0.887) 

Round 2 0.627 (0.018) 0.607 (0.018) 0.020 (0.441) 

Round 3 0.590 (0.019) 0.604 (0.018) -0.014 (0.584) 

Change: Round 3 – Round 2 -0.037 (0.295) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 367

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Other red & orange vegetables

Pooled (Rounds 2 & 3) 0.240 (0.012) 0.223 (0.011) 0.017 (0.314) 

Round 2 0.244 (0.016) 0.226 (0.016) 0.018 (0.436) 

Round 3 0.235 (0.016) 0.219 (0.015) 0.017 (0.460) 

Change: Round 3 – Round 2 -0.009 (0.760) 

Starchy vegetables 

Pooled (Rounds 2 & 3) 0.514 (0.014) 0.513 (0.013) 0.001 (0.950) 

Round 2 0.540 (0.019) 0.521 (0.019) 0.019 (0.474) 

Round 3 0.488 (0.019) 0.505 (0.018) -0.017 (0.530) 

Change: Round 3 – Round 2 -0.041 (0.264) 

White potatoes 

Pooled (Rounds 2 & 3) 0.402 (0.014) 0.416 (0.013) -0.014 (0.462) 

Round 2 0.420 (0.019) 0.407 (0.018) 0.013 (0.620) 

Round 3 0.385 (0.019) 0.426 (0.018) -0.041 (0.121) 

Change: Round 3 – Round 2 -0.056 (0.124) 

Other starchy vegetables 

Pooled (Rounds 2 & 3) 0.227 (0.012) 0.198 (0.011) 0.028 (0.072)* 

Round 2 0.243 (0.016) 0.224 (0.015) 0.019 (0.380) 

Round 3 0.211 (0.016) 0.174 (0.014) 0.037 (0.083)* 

Change: Round 3 – Round 2 0.017 (0.579) 

Legumes 

Pooled (Rounds 2 & 3) 0.206 (0.011) 0.218 (0.010) -0.011 (0.468) 

Round 2 0.217 (0.015) 0.212 (0.015) 0.005 (0.799) 

Round 3 0.196 (0.014) 0.223 (0.015) -0.027 (0.179) 

Change: Round 3 – Round 2 -0.032 (0.262) 

Other vegetables (e.g., celery, cucumbers, mushrooms, green beans, onions, asparagus)

Pooled (Rounds 2 & 3) 0.683 (0.013) 0.653 (0.013) 0.030 (0.097)* 

Round 2 0.705 (0.017) 0.669 (0.017) 0.036 (0.136) 

Round 3 0.661 (0.018) 0.637 (0.018) 0.024 (0.339) 

Change: Round 3 – Round 2 0.040 (0.653) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 368 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Exhibit G8.6: Differences in Impacts of HIP on Total Number of USDA Food Pattern Fruit & 

Vegetable Groups Consumed in Past 24 Hours, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Number of fruit & vegetable groups (range: 0–8)

Pooled (Rounds 2 & 3) 3.505 (0.048) 3.328 (0.046) 0.177 (0.007)*** 

Round 2 3.672 (0.059) 3.389 (0.058) 0.283 (0.001)*** 

Round 3 3.337 (0.058) 3.268 (0.056) 0.069 (0.390) 

Change: Round 3 – Round 2 -0.225 (0.018)** 

Number of fruit groups (range: 0–3) 

Pooled (Rounds 2 & 3) 1.241 (0.028) 1.133 (0.026) 0.108 (0.005)*** 

Round 2 1.304 (0.036) 1.147 (0.033) 0.157 (0.001)*** 

Round 3 1.177 (0.034) 1.119 (0.033) 0.058 (0.218) 

Change: Round 3 – Round 2 -0.087 (0.125) 

Number of vegetable groups (range: 0–5)

Pooled (Rounds 2 & 3) 2.264 (0.034) 2.195 (0.031) 0.069 (0.133) 

Round 2 2.368 (0.042) 2.242 (0.042) 0.126 (0.034)** 

Round 3 2.160 (0.044) 2.149 (0.042) 0.011 (0.854) 

Change: Round 3 – Round 2 -0.137 (0.069)* 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 369

Exhibit G8.7: Differences in Impacts of HIP on Consumption of Other Foods, by Food Pattern 

Equivalent Group, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Total grains (oz-eq) 

Pooled (Rounds 2 & 3) 5.062 (0.115) 5.507 (0.113) -0.445 (0.006)*** 

Round 2 5.115 (0.149) 5.685 (0.148) -0.570 (0.007)*** 

Round 3 5.009 (0.136) 5.332 (0.133) -0.323 (0.090)* 

Change: Round 3 – Round 2 0.255 (0.284) 

Whole grains (oz-eq) 

Pooled (Rounds 2 & 3) 0.645 (0.031) 0.653 (0.031) -0.008 (0.863) 

Round 2 0.637 (0.039) 0.656 (0.040) -0.019 (0.736) 

Round 3 0.654 (0.043) 0.650 (0.042) 0.003 (0.955) 

Change: Round 3 – Round 2 0.010 (0.891) 

Other grains (oz-eq) 

Pooled (Rounds 2 & 3) 4.417 (0.107) 4.854 (0.107) -0.437 (0.004)*** 

Round 2 4.478 (0.143) 5.029 (0.145) -0.551 (0.007)*** 

Round 3 4.355 (0.125) 4.681 (0.124) -0.326 (0.065)* 

Change: Round 3 – Round 2 0.244 (0.295) 

Total dairy (milk, yogurt, cheese) (cup-eq)

Pooled (Rounds 2 & 3) 1.550 (0.051) 1.536 (0.042) 0.014 (0.829) 

Round 2 1.644 (0.068) 1.543 (0.051) 0.101 (0.236) 

Round 3 1.456 (0.056) 1.529 (0.055) -0.073 (0.353) 

Change: Round 3 – Round 2 -0.174 (0.075)* 

Meat, poultry, fish, eggs, soy, nuts, seeds, & legumes (oz-eq)

Pooled (Rounds 2 & 3) 4.966 (0.118) 5.055 (0.108) -0.089 (0.576) 

Round 2 5.090 (0.156) 5.213 (0.150) -0.123 (0.567) 

Round 3 4.841 (0.147) 4.899 (0.136) -0.058 (0.771) 

Change: Round 3 – Round 2 -0.024 (0.928) 

Discretionary oils (gm-eq) 

Pooled (Rounds 2 & 3) 17.28 (0.51) 18.22 (0.56) -0.94 (0.224) 

Round 2 16.96 (0.65) 17.85 (0.67) -0.89 (0.350) 

Round 3 17.61 (0.69) 18.59 (0.73) -0.99 (0.327) 

Change: Round 3 – Round 2 -0.34 (0.781) 

Discretionary solid fats (gm-eq) 

Pooled (Rounds 2 & 3) 29.23 (0.82) 31.34 (0.74) -2.11 (0.056)* 

Round 2 30.52 (1.05) 32.66 (0.98) -2.15 (0.134) 

Round 3 27.94 (0.94) 30.04 (0.85) -2.10 (0.099)* 

Change: Round 3 – Round 2 -0.07 (0.965) 

Added sugar (tsp)a 

Pooled (Rounds 2 & 3) 14.72 (0.51) 15.74 (0.47) -1.02 (0.147) 

Round 2 15.85 (0.62) 16.22 (0.61) -0.37 (0.678) 

Round 3 13.59 (0.56) 15.27 (0.57) -1.68 (0.037)** 

Change: Round 3 – Round 2 -1.28 (0.162) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 370 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Alcohol (drinks)b 

Pooled (Rounds 2 & 3) 0.196 (0.035) 0.133 (0.019) 0.062 (0.114) 

Round 2 0.253 (0.060) 0.119 (0.021) 0.135 (0.035)** 

Round 3 0.138 (0.028) 0.148 (0.028) -0.010 (0.800) 

Change: Round 3 – Round 2 -0.142 (0.035)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01.\ 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

Includes, for example, the sugar added to sweetened soft drinks consumed. 

b

Reflects alcohol content in beverages consumed. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 

Exhibit G8.8: Differences in Impacts of HIP on Total Energy Intake, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Total energy (kcal) 

Pooled (Rounds 2 & 3) 1725 (30) 1795 (29) -69 (0.096)* 

Round 2 1785 (36) 1834 (35) -49 (0.327) 

Round 3 1666 (34) 1756 (33) -90 (0.059)* 

Change: Round 3 – Round 2 -54 (0.288) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 371

Exhibit G8.9: Differences in Impacts of HIP on Total Nutrient Intake, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Fiber

Pooled (Rounds 2 & 3) 13.39 (0.27) 13.12 (0.26) 0.27 (0.467) 

Round 2 13.85 (0.33) 13.42 (0.31) 0.43 (0.344) 

Round 3 12.93 (0.32) 12.83 (0.33) 0.10 (0.825) 

Change: Round 3 – Round 2 -0.42 (0.418) 

Beta carotene 

Pooled (Rounds 2 & 3) 1699 (114) 1502 (93) 196 (0.174) 

Round 2 1921 (193) 1629 (153) 292 (0.226) 

Round 3 1475 (105) 1378 (98) 98 (0.494) 

Change: Round 3 – Round 2 -251 (0.357) 

Vitamin A 

Pooled (Rounds 2 & 3) 588 (22) 567 (18) 21 (0.446) 

Round 2 620 (29) 578 (24) 42 (0.255) 

Round 3 556 (27) 556 (24) -0 (0.999) 

Change: Round 3 – Round 2 -43 (0.358) 

Vitamin C 

Pooled (Rounds 2 & 3) 106 (4) 96 (3) 9 (0.046)** 

Round 2 114 (5) 104 (5) 11 (0.111) 

Round 3 97 (4) 89 (4) 8 (0.154) 

Change: Round 3 – Round 2 -6 (0.417) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 372 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Exhibit G8.10: Differences in Impacts of HIP on Intake of Other Ingredients in Foods with 

Fruits & Vegetables, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Sodium (mg) 

Pooled (Rounds 2 & 3) 876 (28) 974 (31) -98 (0.024)** 

Round 2 922 (38) 978 (43) -57 (0.332) 

Round 3 830 (35) 969 (39) -139 (0.009)*** 

Change: Round 3 – Round 2 -105 (0.131) 

Discretionary oils (gm-eq) 

Pooled (Rounds 2 & 3) 6.74 (0.29) 7.69 (0.36) -0.94 (0.047)** 

Round 2 6.80 (0.40) 6.91 (0.41) -0.11 (0.853) 

Round 3 6.69 (0.40) 8.45 (0.58) -1.76 (0.012)** 

Change: Round 3 – Round 2 -1.85 (0.040)** 

Discretionary solid fats (gm-eq) 

Pooled (Rounds 2 & 3) 6.14 (0.30) 6.76 (0.32) -0.61 (0.162) 

Round 2 6.45 (0.43) 6.99 (0.47) -0.54 (0.398) 

Round 3 5.83 (0.38) 6.53 (0.40) -0.69 (0.204) 

Change: Round 3 – Round 2 0.06 (0.935) 

Added sugar (tsp) 

Pooled (Rounds 2 & 3) 2.92 (0.17) 2.92 (0.20) >-0.01 (0.987) 

Round 2 3.14 (0.24) 2.69 (0.24) 0.45 (0.184) 

Round 3 2.69 (0.22) 3.15 (0.29) -0.46 (0.206) 

Change: Round 3 – Round 2 -0.94 (0.036)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Note that the point estimate for the Round 2/3 change in impacts is not necessarily equal to the difference in 

estimated impacts for Rounds 2 and 3 as reported separately in the third column of this table. This discrepancy 

arises because we estimate the Round 2/3 change in impacts using a regression model with the change in 

outcomes as the dependent variable, while the estimated impacts for Rounds 2 and 3 are separately estimated 

using the level of the outcome as the dependent variable. These alternative models would produce identical point 

estimates of the Round 2/3 change if they were unweighted and included identical covariates; inclusion of 

weights and slightly different covariates for interview-level characteristics results in the observed difference in 

point estimates. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=2959 recalls from 1484 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 373

Exhibit G8.11: Differences in Impacts of HIP on 2010 Dietary Guidelines for Americans, by 

Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Percent of respondents at or above recommendations for...

Total fruit 

Pooled (Rounds 2 & 3) 0.255 (0.175) 0.177 (0.193) 0.078 (0.765) 

Round 2 0.287 (0.168) 0.204 (0.187) 0.083 (0.741) 

Round 3 0.245 (0.195) 0.197 (0.214) 0.048 (0.868) 

Change: Round 3 – Round 2 -0.035 (0.927) 

Total vegetables 

Pooled (Rounds 2 & 3) 0.062 (0.221) 0.043 (0.225) 0.019 (0.952) 

Round 2 0.072 (0.218) 0.049 (0.223) 0.023 (0.741) 

Round 3 0.089 (0.238) 0.066 (0.250) 0.024 (0.945) 

Change: Round 3 – Round 2 0.001 (0.998) 

Dark green vegetables 

Pooled (Rounds 2 & 3) 0.100 (0.210) 0.035 (0.225) 0.065 (0.832) 

Round 2 0.116 (0.206) 0.034 (0.225) 0.082 (0.941) 

Round 3 0.148 (0.221) 0.142 (0.228) 0.006 (0.984) 

Change: Round 3 – Round 2 -0.076 (0.863) 

Red and orange vegetables 

Pooled (Rounds 2 & 3) 0.021 (0.229) 0.017 (0.230) 0.004 (0.990) 

Round 2 0.030 (0.228) 0.023 (0.229) 0.007 (0.789) 

Round 3 0.052 (0.247) 0.047 (0.255) 0.004 (0.990) 

Change: Round 3 – Round 2 -0.003 (0.995) 

Legumes 

Pooled (Rounds 2 & 3) 0.132 (0.202) 0.150 (0.198) -0.018 (0.950) 

Round 2 0.127 (0.203) 0.127 (0.203) <0.001 (0.999) 

Round 3 0.979 (0.007) 0.978 (0.007) 0.001 (0.910) 

Change: Round 3 – Round 2 0.001 (0.997) 

Starchy vegetables 

Pooled (Rounds 2 & 3) 0.126 (0.206) 0.119 (0.208) 0.007 (0.980) 

Round 2 0.049 (0.224) 0.045 (0.225) 0.005 (0.988) 

Round 3 0.223 (0.204) 0.212 (0.212) 0.011 (0.970) 

Change: Round 3 – Round 2 0.006 (0.989) 

Other vegetables 

Pooled (Rounds 2 & 3) 0.195 (0.189) 0.144 (0.201) 0.051 (0.853) 

Round 2 0.202 (0.187) 0.160 (0.197) 0.042 (0.876) 

Round 3 0.216 (0.205) 0.153 (0.227) 0.063 (0.837) 

Change: Round 3 – Round 2 0.021 (0.959) 

Total grains 

Pooled (Rounds 2 & 3) 0.241 (0.177) 0.329 (0.157) 0.088 (0.711) 

Round 2 0.258 (0.174) 0.360 (0.150) -0.102 (0.656) 

Round 3 0.252 (0.194) 0.314 (0.183) -0.062 (0.817) 

Change: Round 3 – Round 2 0.040 (0.910) 

Whole grains 

Pooled (Rounds 2 & 3) 0.001 (0.233) 0.001 (0.233) <0.001 (0.999) 

Round 2 0.005 (0.233) 0.006 (0.232) -0.001 (0.998) 

Round 3 0.008 (0.257) 0.008 (0.264) -0.000 (0.999) 

Change: Round 3 – Round 2 0.001 (0.998) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 374 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Enriched grains 

Pooled (Rounds 2 & 3) 0.838 (0.040) 0.894 (0.026) -0.056 (0.244) 

Round 2 0.826 (0.042) 0.886 (0.028) -0.060 (0.241) 

Round 3 0.800 (0.055) 0.863 (0.039) -0.063 (0.351) 

Change: Round 3 – Round 2 -0.003 (0.972) 

Protein foods 

Pooled (Rounds 2 & 3) 0.385 (0.144) 0.404 (0.139) -0.019 (0.923) 

Round 2 0.396 (0.142) 0.425 (0.135) -0.029 (0.881) 

Round 3 0.367 (0.163) 0.379 (0.164) -0.011 (0.961) 

Change: Round 3 – Round 2 0.018 (0.953) 

Dairy 

Pooled (Rounds 2 & 3) 0.060 (0.221) 0.063 (0.220) -0.003 (0.991) 

Round 2 0.069 (0.219) 0.070 (0.219) -0.002 (0.996) 

Round 3 0.056 (0.246) 0.067 (0.249) -0.010 (0.977) 

Change: Round 3 – Round 2 -0.008 (0.986) 

Percent of respondents below allowances for...

Oils 

Pooled (Rounds 2 & 3) 0.853 (0.199) 0.818 (0.191) 0.035 (0.899) 

Round 2 0.865 (0.203) 0.833 (0.195) 0.032 (0.908) 

Round 3 0.808 (0.209) 0.769 (0.204) 0.040 (0.892) 

Change: Round 3 – Round 2 0.008 (0.984) 

Calories from solid fats & added sugars (SoFAS) 

Pooled (Rounds 2 & 3) 0.064 (0.017) 0.053 (0.014) 0.011 (0.600) 

Round 2 0.056 (0.015) 0.046 (0.012) 0.010 (0.604) 

Round 3 0.114 (0.031) 0.086 (0.024) 0.028 (0.479) 

Change: Round 3 – Round 2 0.018 (0.678) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Note that these analyses treat observations as independent rather than paired across rounds, and therefore 

likely somewhat understate actual statistical significance. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=3,919 recalls from 2,009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix G: Analyses by Follow-Up Survey Round ▌pg. 375

Exhibit G8.12: Differences in Impacts of HIP on Healthy Eating Index-2010 and Component 

Scores, by Follow-Up Round 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Total vegetables

Pooled (Rounds 2 & 3) 3.70 (0.10) 3.35 (0.09) 0.35 (0.006)*** 

Round 2 3.62 (0.11) 3.34 (0.10) 0.28 (0.062)* 

Round 3 3.89 (0.14) 3.36 (0.13) 0.53 (0.006)*** 

Change: Round 3 – Round 2 0.25 (0.302) 

Greens and beans

Pooled (Rounds 2 & 3) 3.02 (0.17) 2.47 (0.17) 0.55 (0.022)** 

Round 2 3.02 (0.22) 2.45 (0.20) 0.57 (0.053)* 

Round 3 2.95 (0.24) 2.31 (0.30) 0.64 (0.096)* 

Change: Round 3 – Round 2 0.07 (0.884) 

Total fruit

Pooled (Rounds 2 & 3) 4.39 (0.18) 3.53 (0.14) 0.85 (<0.001) *** 

Round 2 4.45 (0.19) 3.60 (0.16) 0.85 (0.001)*** 

Round 3 4.25 (0.22) 3.58 (0.19) 0.66 (0.023)** 

Change: Round 3 – Round 2 -0.19 (0.614) 

Whole fruit 

Pooled (Rounds 2 & 3) 4.91 (0.14) 4.00 (0.21) 0.91 (<0.001) *** 

Round 2 4.90 (0.15) 4.04 (0.21) 0.86 (0.001)*** 

Round 3 4.83 (0.22) 4.08 (0.30) 0.75 (0.042)** 

Change: Round 3 – Round 2 -0.11 (0.808) 

Whole grains 

Pooled (Rounds 2 & 3) 2.40 (0.11) 2.36 (0.11) 0.04 (0.786) 

Round 2 2.36 (0.12) 2.32 (0.13) 0.04 (0.817) 

Round 3 2.54 (0.16) 2.47 (0.16) 0.07 (0.748) 

Change: Round 3 – Round 2 0.03 (0.916) 

Dairy 

Pooled (Rounds 2 & 3) 6.89 (0.18) 6.67 (0.16) 0.22 (0.358) 

Round 2 6.99 (0.20) 6.63 (0.18) 0.36 (0.181) 

Round 3 6.71 (0.22) 6.68 (0.23) 0.02 (0.945) 

Change: Round 3 – Round 2 -0.34 (0.417) 

Total protein foodsa

Pooled (Rounds 2 & 3) 5.00 (0.00) 5.00 (0.00) 0.00 (1.000) 

Round 2 5.00 (0.00) 5.00 (0.00) 0.00 (1.000) 

Round 3 5.00 (0.00) 5.00 (0.00) 0.00 (1.000) 

Change: Round 3 – Round 2 0.00 (1.000) 

Seafood & plant protein

Pooled (Rounds 2 & 3) 3.48 (0.33) 2.88 (0.21) 0.60 (0.126) 

Round 2 2.98 (0.24) 2.83 (0.25) 0.15 (0.658) 

Round 3 3.76 (0.47) 2.89 (0.34) 0.87 (0.132) 

Change: Round 3 – Round 2 0.72 (0.288) 

Fatty acid ratio

Pooled (Rounds 2 & 3) 3.36 (0.16) 3.48 (0.15) -0.12 (0.584) 

Round 2 3.08 (0.17) 3.30 (0.18) -0.22 (0.378) 

Round 3 3.79 (0.23) 3.60 (0.21) 0.19 (0.539) 

Change: Round 3 – Round 2 0.41 (0.303) 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 376 ▌Appendix G: Analyses by Follow-Up Survey Round Abt Associates

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Sodium

Pooled (Rounds 2 & 3) 4.72 (0.16) 4.43 (0.17) 0.29 (0.215) 

Round 2 4.85 (0.19) 4.56 (0.21) 0.29 (0.308) 

Round 3 4.66 (0.23) 4.28 (0.24) 0.38 (0.253) 

Change: Round 3 – Round 2 0.09 (0.838) 

Refined grains

Pooled (Rounds 2 & 3) 7.08 (0.16) 6.38 (0.16) 0.71 (0.002)*** 

Round 2 7.21 (0.20) 6.32 (0.20) 0.88 (0.002)*** 

Round 3 6.84 (0.21) 6.50 (0.21) 0.34 (0.248) 

Change: Round 3 – Round 2 -0.54 (0.196) 

SoFAS calories

Pooled (Rounds 2 & 3) 12.90 (0.30) 12.57 (0.27) 0.33 (0.418) 

Round 2 12.40 (0.32) 12.32 (0.31) 0.07 (0.867) 

Round 3 13.72 (0.38) 12.78 (0.36) 0.94 (0.073)* 

Change: Round 3 – Round 2 0.87 (0.206) 

Total HEI-2010 score 

Pooled (Rounds 2 & 3) 61.85 (0.89) 57.12 (0.79) 4.73 (<0.001)*** 

Round 2 60.85 (0.88) 56.71 (0.90) 4.14 (0.001)*** 

Round 3 62.93 (1.20) 57.53 (1.00) 5.40 (0.001)*** 

Change: Round 3 – Round 2 1.26 (0.529) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across rounds may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

between rounds. 

Note that these analyses treat observations as independent rather than paired across rounds, and therefore 

likely somewhat understate actual statistical significance. a

All respondents in both treatment and control groups achieved the maximum score of 5 on the total protein 

foods component of the HEI-2010. 

Source: Participant Survey (AMPM dietary recall module), including 10% second-day subsamples for each round 

(unweighted N=3,919 recalls from 2,009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 377

Appendix H: Analyses by Subgroup 

Exhibit H5.1: Proportion of Primary Shoppers Who Had Ever Heard of HIP, by Treatment and 

Control Status, by Subgroup: Round 2 and 3 Participant Surveys 

Proportion (N) 

Total Treatment Control P-value 

Primary shopper employment status (N=3249)

Working full or part-time (N=635) 0.23 (270) 0.74 (216) 0.16 (54) [.]a

Not working (N=2614) 0.21 (1075) 0.66 (897) 0.14 (178) [<0.001]*** 

Difference: working – not working 

[p-value] 0.03 [0.311] 0.08 [0.017]** 0.02 [0.314] 

Household composition (N=3395) 

Children (and no elderly) in 

household (N=1400) 0.23 (604) 0.73 (496) 0.15 (108) [<0.001]*** 

Other household (N=1995) 0.21 (795) 0.63 (650) 0.15 (145) [<0.001]*** 

Difference: children in HH – other 

HH [p-value] 0.03 [0.150] 0.11 [<0.001]*** 0.01 [0.620] 

Household WIC status (past 30 days) (N=3243)

Participant (N=587) 0.22 (249) 0.71 (210) 0.14 (39) [.]a

Non-participant (N=2656) 0.21 (1097) 0.67 (905) 0.14 (192) [<0.001]*** 

Difference: participant – nonparticipant [p-value] 0.01 [0.594] 0.05 [0.186] >-0.01 [0.941] 

Household SNAP benefit amount (N=3395)

$200 or less (N=1923) 0.22 (779) 0.63 (633) 0.15 (146) [<0.001]*** 

Over $200 (N=1472) 0.22 (620) 0.72 (513) 0.14 (107) [<0.001]*** 

Difference: $200 or less – over 

$200 [p-value] >-0.01 [0.840] -0.09 [0.001]*** 0.01 [0.622] 

Baseline respondent fruit & vegetable intake (screener) (N=3395)

3+ servings/day (N=1713) 0.22 (700) 0.67 (571) 0.15 (129) [<0.001]*** 

<3 servings/day (N=1682) 0.22 (699) 0.67 (575) 0.15 (124) [<0.001]*** 

Difference: 3+ servings – <3 

servings [p-value] -0.01 [0.718] >-0.01 [0.962] >-0.01 [0.845] 

Baseline respondent fruit & vegetable intake (predicted) (N=2724)

High (N=1395) 0.22 (580) 0.68 (476) 0.15 (104) [<0.001]*** 

Low (N=1329) 0.23 (565) 0.70 (465) 0.15 (100) [<0.001]*** 

Difference: high – low [p-value] -0.01 [0.646] -0.02 [0.438] >-0.01 [.]a 

Baseline spending on TFV (N=2724)

High (N=1337) 0.23 (588) 0.74 (481) 0.16 (107) [<0.001]*** 

Low (N=1387) 0.21 (557) 0.65 (460) 0.14 (97) [.]a

Difference: high – low [p-value] 0.02 [0.338] 0.09 [0.002]*** 0.01 [ .]a 

Attitudes about food, fruits, & vegetables (N=3395)

High (N=1941) 0.23 (820) 0.68 (658) 0.16 (162) [<0.001]*** 

Low (N=1454) 0.20 (579) 0.65 (488) 0.13 (91) [<0.001]*** 

Difference: high – low [p-value] 0.03 [0.111] 0.03 [0.309] 0.04 [0.052]* 

Barriers to eating fruits & vegetables (N=3395)

High (N=1670) 0.22 (653) 0.62 (511) 0.16 (142) [<0.001]*** 

Low (N=1725) 0.22 (746) 0.71 (635) 0.13 (111) [<0.001]*** 

Difference: high – low [p-value] <0.01 [0.639] -0.08 [0.001]*** 0.03 [0.096]* 

Barriers to grocery shopping (N=3395)

High (N=1522) 0.22 (596) 0.63 (470) 0.16 (126) [<0.001]*** 

Low (N=1873) 0.22 (803) 0.70 (676) 0.13 (127) [<0.001]*** 

Difference: high – low [p-value] 0.01 [0.606] -0.07 [0.009]*** 0.03 [0.136] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 378 ▌Appendix H: Analyses by Subgroup Abt Associates

Proportion (N) 

Total Treatment Control P-value 

Fruits & vegetables at home (N=3395)

High (N=1481) 0.22 (627) 0.70 (510) 0.15 (117) [<0.001]*** 

Low (N=1914) 0.21 (772) 0.65 (636) 0.14 (136) [<0.001]*** 

Difference: high – low [p-value] 0.01 [0.669] 0.04 [0.094]* 0.01 [0.718] 

Pre-HIP shopping patterns 

(N=2202) 

Shopped primarily at HIP 

participating retailers (N=1320) 0.20 (528) 0.65 (444) 0.13 (84) [<0.001]*** 

Shopped primarily at non-HIP 

participating retailers (N=882) 0.20 (343) 0.64 (278) 0.14 (65) [.]a

Difference: HIP shoppers - nonHIP-shoppers [P-value] 0.01 [0.840] 0.01 [0.811] >-0.01 [0.818] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

Missing test statistic because of stratum with single sampling unit. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 379

Exhibit H5.2a: Proportion of Primary Shoppers Who Attended Meeting to Learn about HIP, by 

Subgroup: Round 2 and 3 Participant Surveys, HIP Participants 

Proportion (N) 

Primary shopper employment status (N=1654)

Working full or part-time (N=299) 0.03 (11) 

Not working (N=1355) 0.06 (82) 

Difference: working – not working [p-value] -0.03 [0.092] 

Household composition (N=1712) 

Children (and no elderly) in household (N=683) 0.04 (30) 

Other household (N=1029) 0.06 (64) 

Difference: children in HH – other HH [p-value] -0.02 [0.107] 

Household WIC status (past 30 days) (N=1651) 

Participant (N=298) 0.06 (18) 

Non-participant (N=1353) 0.05 (75) 

Difference: participant – non-participant [p-value] <0.01 [0.820] 

Household SNAP benefit amount (N=1712) 

$200 or less (N=997) 0.06 (60) 

Over $200 (N=715) 0.05 (34) 

Difference: $200 or less – over $200 [p-value] 0.01 [0.360] 

Baseline respondent fruit & vegetable intake (screener) (N=1712) 

3+ servings/day (N=853) 0.07 (59) 

<3 servings/day (N=859) 0.04 (35) 

Difference: 3+ servings – <3 servings [p-value] 0.03 [0.006]*** 

Baseline respondent fruit & vegetable intake (predicted) (N=1363) 

High (N=706) 0.07 (51) 

Low (N=657) 0.04 (29) 

Difference: high – low [p-value] 0.03 [0.060]* 

Baseline spending on TFV (N=1363) 

High (N=655) 0.06 (44) 

Low (N=708) 0.05 (36) 

Difference: high – low [p-value] 0.02 [0.209] 

Attitudes about food, fruits, & vegetables (N=1712) 

High (N=967) 0.05 (53) 

Low (N=745) 0.05 (41) 

Difference: high – low [p-value] >-0.01 [0.914] 

Barriers to eating fruits & vegetables (N=1712) 

High (N=825) 0.05 (46) 

Low (N=887) 0.05 (58) 

Difference: high – low [p-value] >-0.01 [0.958] 

Barriers to grocery shopping (N=1712) 

High (N=754) 0.05 (40) 

Low (N=958) 0.05 (54) 

Difference: high – low [p-value] -0.01 [0.653] 

Fruits & vegetables at home (N=1712) 

High (N=734) 0.07 (52) 

Low (N=978) 0.04 (42) 

Difference: high – low [p-value] 0.03 [0.026]** 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 380 ▌Appendix H: Analyses by Subgroup Abt Associates

Proportion (N) 

Pre-HIP shopping patterns (N=1108)

Shopped primarily at HIP participating retailers (N=682) 0.06 (46) 

Shopped primarily at non-HIP participating retailers (N=426) 0.04 (15) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] 0.03 [0.074]* 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. 

Includes participants who reported they had not heard of HIP.

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 381

Exhibit H5.2b: Proportion of Primary Shoppers Who Called EBT or HIP Hotline with Questions 

or Problems in Past Month, by Subgroup: Round 2 and 3 Participant Surveys, 

HIP Participants 

Proportion (N) 

Primary shopper employment status (N=1628) 

Working full or part-time (N=294) 0.04 (12) 

Not working (N=1334) 0.07 (86) 

Difference: working – not working [p-value] -0.02 [0.185] 

Household composition (N=1685) 

Children (and no elderly) in household (N=670) 0.05 (31) 

Other household (N=1015) 0.07 (68) 

Difference: children in HH – other HH [p-value] -0.02 [0.133] 

Household WIC status (past 30 days) (N=1625) 

Participant (N=295) 0.04 (12) 

Non-participant (N=1330) 0.07 (86) 

Difference: participant – non-participant [p-value] -0.02 [0.273] 

Household SNAP benefit amount (N=1685) 

$200 or less (N=983) 0.07 (66) 

Over $200 (N=702) 0.05 (33) 

Difference: $200 or less – over $200 [p-value] 0.02 [0.136] 

Baseline respondent fruit & vegetable intake (screener) (N=1685) 

3+ servings/day (N=842) 0.06 (52) 

<3 servings/day (N=843) 0.06 (47) 

Difference: 3+ servings – <3 servings [p-value] >-0.01 [0.888] 

Baseline respondent fruit & vegetable intake (predicted) (N=1342) 

High (N=701) 0.06 (42) 

Low (N=641) 0.05 (36) 

Difference: high – low [p-value] 0.01 [0.490] 

Baseline spending on TFV (N=1342) 

High (N=644) 0.06 (41) 

Low (N=698) 0.05 (37) 

Difference: high – low [p-value] 0.01 [0.640] 

Attitudes about food, fruits, & vegetables (N=1685) 

High (N=951) 0.06 (55) 

Low (N=734) 0.06 (44) 

Difference: high – low [p-value] 0.01 [0.689] 

Barriers to eating fruits & vegetables (N=1685) 

High (N=814) 0.07 (50) 

Low (N=871) 0.06 (49) 

Difference: high – low [p-value] 0.01 [0.604] 

Barriers to grocery shopping (N=1685) 

High (N=747) 0.06 (47) 

Low (N=938) 0.06 (52) 

Difference: high – low [p-value] 0.01 [0.672] 

Fruits & vegetables at home (N=1685) 

High (N=717) 0.06 (44) 

Low (N=968) 0.06 (55) 

Difference: high – low [p-value] <0.01 [0.766] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 382 ▌Appendix H: Analyses by Subgroup Abt Associates

Proportion (N) 

Pre-HIP shopping patterns (N=1084)

Shopped primarily at HIP participating retailers (N=667) 0.07 (43) 

Shopped primarily at non-HIP participating retailers (N=417) 0.05 (21) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] 0.02 [0.206] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. 

Includes participants who reported they had not heard of HIP.

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Exhibit H5.3a: Primary Shopper Report: “How easy or hard has it been to understand how HIP works?”, by Subgroup: Ro

Surveys, HIP Participants 

Very easy Easy 

Somewhat 

easy 

Somewhat 

hard Hard Very hard 

Primary shopper employment status Working 0.15 (48) 0.41 (115) 0.16 (53) 0.06 (15) 0.03 (8) 0.02 (6) Not working 0.14 (199) 0.34 (459) 0.19 (253) 0.07 (98) 0.04 (55) 0.04 (51) Household composition Children (and no elderly) in household 0.15 (107) 0.39 (259) 0.17 (117) 0.08 (48) 0.04 (25) 0.02 (10) Other household 0.14 (148) 0.33 (337) 0.19 (196) 0.06 (66) 0.04 (43) 0.05 (48) Household WIC status (past 30 days) Participant 0.15 (48) 0.43 (123) 0.14 (41) 0.08 (25) 0.03 (10) 0.01 (3) Non-participant 0.14 (199) 0.34 (453) 0.19 (262) 0.07 (87) 0.04 (53) 0.04 (54) Household SNAP benefit amount $200 or less 0.14 (142) 0.33 (327) 0.19 (189) 0.07 (67) 0.05 (44) 0.05 (47) Over $200 0.15 (113) 0.39 (269) 0.17 (124) 0.07 (47) 0.03 (24) 0.02 (11) Baseline respondent fruit & vegetable intake (screener)3+ servings/day 0.15 (129) 0.37 (309) 0.17 (149) 0.06 (49) 0.04 (33) 0.04 (33) <3 servings/day 0.14 (126) 0.34 (287) 0.19 (164) 0.08 (65) 0.04 (35) 0.03 (25) Baseline respondent fruit & vegetable intake (predicted)High 0.15 (112) 0.38 (256) 0.15 (114) 0.06 (43) 0.04 (29) 0.04 (26) Low 0.16 (104) 0.35 (234) 0.20 (129) 0.07 (45) 0.03 (18) 0.03 (17) Baseline spending on TFVHigh 0.17 (112) 0.37 (241) 0.17 (112) 0.08 (47) 0.03 (20) 0.03 (18) Low 0.14 (104) 0.35 (249) 0.18 (131) 0.06 (41) 0.04 (27) 0.03 (25) Attitudes about food, fruits, & vegetables High 0.16 (157) 0.37 (353) 0.17 (169) 0.06 (53) 0.04 (39) 0.03 (29) Low 0.13 (98) 0.33 (243) 0.19 (144) 0.08 (61) 0.04 (29) 0.04 (29) Barriers to eating fruits & vegetables High 0.09 (78) 0.34 (278) 0.18 (156) 0.08 (60) 0.06 (46) 0.04 (35) Low 0.19 (177) 0.36 (318) 0.18 (157) 0.06 (54) 0.02 (22) 0.03 (23) Barriers to grocery shopping High 0.13 (100) 0.32 (241) 0.20 (146) 0.07 (51) 0.05 (38) 0.04 (32) Low 0.16 (155) 0.38 (355) 0.17 (167) 0.07 (63) 0.03 (30) 0.03 (26) 
Evaluation of the Healthy In

pg. 384 ▌Appendix H: Analyses by Subgroup 

Very easy Easy 

Somewhat 

easy 

Somewhat 

hard Hard Very hard 

Fruits & vegetables at home High 0.16 (122) 0.38 (274) 0.17 (129) 0.06 (44) 0.03 (21) 0.03 (23) Low 0.13 (133) 0.33 (322) 0.19 (184) 0.07 (70) 0.05 (47) 0.04 (35) Pre-HIP shopping patterns (N=1115) Shopped primarily at HIP participating retailers 0.14 (97) 0.33 (227) 0.19 (126) 0.07 (49) 0.05 (29) 0.03 (24) Shopped primarily at non-HIP participating retailers 0.12 (53) 0.34 (150) 0.21 (91) 0.04 (17) 0.05 (19) 0.03 (13) 

Weighted proportions (unweighted Ns). Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. “Don’t know” and “refused” responses coded as missing. Due to rounding, reported proportions may not sum to one. Test statistics are adjusted for clustering at the individual respondent level. Includes participants who reported they had not heard of HIP. Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=1719). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Exhibit H5.3b: Primary Shopper Report: “How easy or hard is it remembering which fruits and vegetables earn the HIP re

and 3 Participant Surveys, HIP Participants 

Very easy Easy 

Somewhat 

easy 

Somewhat 

hard Hard Very hard 

Primary shopper employment status Working 0.11 (37) 0.37 (103) 0.25 (77) 0.11 (31) 0.06 (16) 0.02 (6) Not working 0.13 (182) 0.33 (446) 0.20 (273) 0.13 (175) 0.07 (98) 0.04 (50) Household composition Children (and no elderly) in household 0.13 (90) 0.33 (222) 0.24 (165) 0.13 (87) 0.07 (47) 0.02 (15) Other household 0.13 (138) 0.34 (349) 0.19 (195) 0.12 (122) 0.07 (71) 0.04 (42) Household WIC status (past 30 days) Participant 0.13 (41) 0.42 (117) 0.20 (61) 0.13 (36) 0.06 (17) 0.01 (4) Non-participant 0.13 (178) 0.32 (435) 0.22 (289) 0.12 (168) 0.07 (93) 0.04 (52) Household SNAP benefit amount $200 or less 0.13 (135) 0.33 (329) 0.19 (192) 0.12 (121) 0.07 (70) 0.04 (45) Over $200 0.12 (93) 0.34 (242) 0.24 (168) 0.13 (88) 0.07 (48) 0.02 (12) Baseline respondent fruit & vegetable intake (screener)3+ servings/day 0.13 (118) 0.35 (294) 0.21 (175) 0.12 (101) 0.06 (54) 0.03 (26) <3 servings/day 0.12 (110) 0.32 (277) 0.22 (185) 0.13 (108) 0.08 (64) 0.03 (31) Baseline respondent fruit & vegetable intake (predicted)High 0.13 (101) 0.36 (246) 0.21 (147) 0.11 (79) 0.07 (45) 0.03 (24) Low 0.14 (94) 0.31 (200) 0.22 (143) 0.15 (94) 0.07 (46) 0.03 (22) Baseline spending on TFVHigh 0.14 (95) 0.34 (217) 0.23 (150) 0.14 (85) 0.06 (36) 0.03 (19) Low 0.13 (100) 0.33 (229) 0.20 (140) 0.12 (88) 0.08 (55) 0.04 (27) Attitudes about food, fruits, & vegetables High 0.14 (138) 0.36 (343) 0.21 (207) 0.10 (99) 0.06 (58) 0.02 (24) Low 0.11 (90) 0.31 (228) 0.21 (153) 0.15 (110) 0.08 (60) 0.04 (33) Barriers to eating fruits & vegetables High 0.09 (77) 0.31 (254) 0.20 (169) 0.15 (125) 0.09 (70) 0.04 (35) Low 0.16 (151) 0.36 (317) 0.22 (191) 0.10 (84) 0.06 (48) 0.02 (22) Barriers to grocery shopping High 0.12 (97) 0.31 (235) 0.22 (162) 0.13 (96) 0.09 (61) 0.04 (33) Low 0.13 (131) 0.36 (336) 0.21 (198) 0.12 (113) 0.06 (57) 0.03 (24) 
Evaluation of the Healthy In

pg. 386 ▌Appendix H: Analyses by Subgroup 

Very easy Easy 

Somewhat 

easy 

Somewhat 

hard Hard Very hard 

Fruits & vegetables at home High 0.14 (107) 0.37 (266) 0.20 (149) 0.11 (77) 0.06 (43) 0.03 (20) Low 0.12 (121) 0.31 (305) 0.22 (211) 0.14 (132) 0.08 (75) 0.04 (37) Pre-HIP shopping patterns (N=1115) Shopped primarily at HIP participating retailers 0.12 (90) 0.33 (225) 0.20 (133) 0.13 (87) 0.08 (53) 0.03 (22) Shopped primarily at non-HIP participating retailers 0.11 (49) 0.33 (140) 0.25 (109) 0.10 (42) 0.06 (26) 0.04 (17) 

Weighted proportions (unweighted Ns). Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. “Don’t know” and “refused” responses coded as missing. Due to rounding, reported proportions may not sum to one. Test statistics are adjusted for clustering at the individual respondent level. Includes participants who reported they had not heard of HIP. Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=1719). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Exhibit H5.3c: Primary Shopper Report: “How easy or hard is it keeping track of the HIP rebates you earn?” by Subgroup

Surveys, HIP Participants 

Very easy Easy 

Somewhat 

easy 

Somewhat 

hard Hard Very hard 

Primary shopper employment status Working 0.20 (27) 0.54 (60) 0.17 (23) 0.04 (5) 0.03 (3) 0.00 (0) Not working 0.26 (122) 0.42 (201) 0.22 (103) 0.06 (25) 0.02 (11) 0.01 (4) Household composition Children (and no elderly) in household 0.25 (68) 0.48 (122) 0.18 (49) 0.05 (10) 0.03 (8) <0.01 (1) Other household 0.24 (86) 0.42 (148) 0.23 (79) 0.05 (20) 0.02 (7) 0.01 (3) Household WIC status (past 30 days) Participant 0.24 (26) 0.53 (59) 0.15 (17) 0.06 (7) 0.01 (1) 0.00 (0) Non-participant 0.25 (123) 0.42 (201) 0.23 (109) 0.05 (23) 0.03 (13) 0.01 (4) Household SNAP benefit amount $200 or less 0.25 (83) 0.43 (142) 0.21 (68) 0.06 (20) 0.01 (4) 0.01 (3) Over $200 0.24 (71) 0.46 (128) 0.20 (60) 0.04 (10) 0.04 (11) <0.01 (1) Baseline respondent fruit & vegetable intake (screener)3+ servings/day 0.26 (80) 0.44 (128) 0.22 (65) 0.05 (16) 0.01 (4) 0.01 (2) <3 servings/day 0.23 (74) 0.46 (142) 0.20 (63) 0.05 (14) 0.04 (11) 0.01 (2) Baseline respondent fruit & vegetable intake (predicted)High 0.23 (70) 0.44 (124) 0.21 (57) 0.06 (18) 0.03 (8) 0.01 (3) Low 0.26 (61) 0.45 (105) 0.20 (50) 0.04 (9) 0.02 (5) 0.00 (0) Baseline spending on TFVHigh 0.23 (66) 0.47 (127) 0.19 (55) 0.05 (11) 0.03 (8) 0.01 (3) Low 0.26 (65) 0.42 (102) 0.22 (52) 0.06 (16) 0.02 (5) 0.00 (0) Attitudes about food, fruits, & vegetables High 0.24 (89) 0.44 (159) 0.21 (75) 0.06 (19) 0.03 (10) 0.01 (3) Low 0.26 (65) 0.46 (111) 0.20 (53) 0.04 (11) 0.02 (5) <0.01 (1) Barriers to eating fruits & vegetables High 0.19 (49) 0.44 (105) 0.24 (60) 0.05 (12) 0.04 (10) 0.01 (2) Low 0.28 (105) 0.45 (165) 0.19 (68) 0.05 (18) 0.01 (5) <0.01 (2) Barriers to grocery shopping High 0.25 (65) 0.43 (112) 0.20 (55) 0.05 (12) 0.05 (12) 0.01 (4) Low 0.24 (89) 0.46 (158) 0.21 (73) 0.05 (18) 0.01 (3) 0.00 (0) 
Evaluation of the Healthy In

pg. 388 ▌Appendix H: Analyses by Subgroup 

Very easy Easy 

Somewhat 

easy 

Somewhat 

hard Hard Very hard 

Fruits & vegetables at home High 0.25 (75) 0.47 (133) 0.18 (52) 0.04 (11) 0.03 (7) 0.01 (3) Low 0.24 (79) 0.42 (137) 0.23 (76) 0.06 (19) 0.02 (8) <0.01 (1) Pre-HIP shopping patterns (N=377) Shopped primarily at HIP participating retailers 0.24 (58) 0.44 (105) 0.22 (52) 0.05 (10) 0.02 (6) <0.01 (1) Shopped primarily at non-HIP participating retailers 0.27 (37) 0.42 (60) 0.22 (29) 0.03 (5) 0.03 (4) 0.01 (1) 

Weighted proportions (unweighted Ns). Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. “Don’t know” and “refused” responses coded as missing. Due to rounding, reported proportions may not sum to one. Test statistics are adjusted for clustering at the individual respondent level. Includes participants who reported they had not heard of HIP. Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=614). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 389

Exhibit H5.4: Proportion of Primary Shoppers Who Keep Track of HIP Rebate Earned, by 

Subgroup: Round 2 and 3 Participant Surveys, HIP Participants 

Proportion (N) 

Primary shopper employment status (N=1522) 

Working full or part-time (N=279) 0.43 (120) 

Not working (N=1243) 0.38 (476) 

Difference: working – not working [p-value] 0.04 [0.264] 

Household composition (N=1574) 

Children (and no elderly) in household (N=642) 0.40 (263) 

Other household (N=932) 0.38 (351) 

Difference: children in HH – other HH [p-value] 0.02 [0.445] 

Household WIC status (past 30 days) (N=1519)

Participant (N=281) 0.40 (112) 

Non-participant (N=1238) 0.39 (483) 

Difference: participant – non-participant [p-value] 0.01 [0.863] 

Household SNAP benefit amount (N=1574)

$200 or less (N=907) 0.36 (328) 

Over $200 (N=667) 0.42 (286) 

Difference: $200 or less – over $200 [p-value] -0.06 [0.062]* 

Baseline respondent fruit & vegetable intake (screener) (N=1574)

3+ servings/day (N=778) 0.39 (301) 

<3 servings/day (N=796) 0.39 (313) 

Difference: 3+ servings – <3 servings [p-value] >-0.01 [0.976] 

Baseline respondent fruit & vegetable intake (predicted) (N=1261)

High (N=651) 0.45 (286) 

Low (N=610) 0.38 (234) 

Difference: high – low [p-value] 0.07 [0.037]** 

Baseline spending on TFV (N=1261)

High (N=612) 0.46 (277) 

Low (N=649) 0.37 (243) 

Difference: high – low [p-value] 0.08 [0.012]** 

Attitudes about food, fruits, & vegetables (N=1574)

High (N=887) 0.41 (363) 

Low (N=687) 0.36 (251) 

Difference: high – low [p-value] 0.05 [0.071]* 

Barriers to eating fruits & vegetables (N=1574)

High (N=743) 0.32 (244) 

Low (N=831) 0.45 (370) 

Difference: high – low [p-value] -0.13 [<0.001]*** 

Barriers to grocery shopping (N=1574)

High (N=686) 0.39 (263) 

Low (N=888) 0.39 (351) 

Difference: high – low [p-value] -0.01 [0.836] 

Fruits & vegetables at home (N=1574)

High (N=677) 0.42 (287) 

Low (N=897) 0.37 (327) 

Difference: high – low [p-value] 0.05 [0.084]* 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 390 ▌Appendix H: Analyses by Subgroup Abt Associates

Proportion (N) 

Pre-HIP shopping patterns (N=1013)

Shopped primarily at HIP participating retailers (N=627) 0.37 (237) 

Shopped primarily at non-HIP participating retailers (N=386) 0.36 (140) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] 0.01 [0.787] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. 

Includes participants who reported they had not heard of HIP.

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 391

Exhibit H5.5a: Proportion of Primary Shoppers Who Are Primary Users of EBT Card, by 

Treatment and Control Status, by Subgroup: Round 2 & 3 Participant Surveys 

Total Treatment Control P-value 

Primary shopper employment status (N=3274)

Working full or part-time (N=644) 0.90 (578) 0.90 (265) 0.90 (313) [ .]a

Not working (N=2630) 0.90 (2384) 0.91 (1243) 0.90 (1141) [0.510] 

Difference: working – not working [pvalue] <0.01 [0.979] -0.01 [0.778] <0.01 [0.940] 

Household composition (N=3421) 

Children (and no elderly) in 

household (N=1418) 0.90 (1292) 0.91 (627) 0.90 (665) [0.721] 

Other household (N=2003) 0.89 (1784) 0.89 (925) 0.89 (859) [0.656] 

Difference: children in HH – other HH 

[p-value] 0.02 [0.297] 0.02 [0.358] 0.02 [0.363] 

Household WIC status (past 30 days) (N=3268)

Participant (N=595) 0.91 (542) 0.91 (271) 0.90 (271) [ .]a

Non-participant (N=2673) 0.90 (2417) 0.91 (1236) 0.90 (1181) [0.388] 

Difference: participant – nonparticipant [p-value] <0.01 [0.817] >-0.01 [0.929] 0.01 [0.808] 

Household SNAP benefit amount (N=3421)

$200 or less (N=1934) 0.89 (1741) 0.91 (912) 0.89 (829) [0.130] 

Over $200 (N=1487) 0.90 (1335) 0.89 (640) 0.90 (695) [0.481] 

Difference: $200 or less – over $200 

[p-value] -0.01 [0.445] 0.02 [0.235] -0.02 [0.344] 

Baseline respondent fruit & vegetable intake (screener) (N=3421)

3+ servings/day (N=1726) 0.89 (1547) 0.89 (769) 0.89 (778) [0.994] 

<3 servings/day (N=1695) 0.90 (1529) 0.91 (783) 0.89 (746) [0.396] 

Difference: 3+ servings – <3 servings 

[p-value] <0.01 [0.839] -0.02 [0.359] <0.01 [0.949] 

Baseline respondent fruit & vegetable intake (predicted) (N=2741)

High (N=1407) 0.91 (1278) 0.90 (641) 0.91 (637) [0.684] 

Low (N=1334) 0.91 (1215) 0.91 (602) 0.91 (613) [0.754] 

Difference: high – low [p-value] <0.01 [0.992] -0.01 [0.526] <0.01 [ .]a 

Baseline spending on TFV (N=2741)

High (N=1346) 0.91 (1220) 0.90 (590) 0.92 (630) [0.501] 

Low (N=1395) 0.91 (1273) 0.91 (653) 0.90 (620) [ .]a

Difference: high – low [p-value] 0.01 [0.588] -0.01 [0.559] 0.01 [ .]a 

Attitudes about food, fruits, & vegetables (N=3421)

High (N=1955) 0.90 (1788) 0.92 (895) 0.90 (893) [0.247] 

Low (N=1466) 0.88 (1288) 0.88 (657) 0.88 (631) [0.806] 

Difference: high – low [p-value] 0.02 [0.144] 0.04 [0.014]** 0.02 [0.266] 

Barriers to eating fruits & vegetables (N=3421)

High (N=1683) 0.89 (1496) 0.88 (733) 0.89 (763) [0.810] 

Low (N=1738) 0.90 (1580) 0.92 (819) 0.90 (761) [0.287] 

Difference: high – low [p-value] -0.01 [0.408] -0.03 [0.063]* -0.01 [0.579] 

Barriers to grocery shopping (N=3421)

High (N=1536) 0.89 (1371) 0.89 (675) 0.89 (696) [0.942] 

Low (N=1885) 0.90 (1705) 0.91 (877) 0.90 (828) [0.411] 

Difference: high – low [p-value] -0.01 [0.468] -0.02 [0.164] -0.01 [0.601] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 392 ▌Appendix H: Analyses by Subgroup Abt Associates

Total Treatment Control P-value 

Fruits & vegetables at home (N=3421)

High (N=1489) 0.91 (1353) 0.90 (664) 0.91 (689) [0.378] 

Low (N=1932) 0.88 (1723) 0.90 (888) 0.88 (835) [0.111] 

Difference: high – low [p-value] 0.03 [0.064]* -0.01 [0.643] 0.03 [0.052]* 

Pre-HIP shopping patterns (N=2219)

Shopped primarily at HIP 

participating retailers (N=1332) 0.91 (1214) 0.92 (629) 0.91 (585) [0.428] 

Shopped primarily at non-HIP 

participating retailers (N=887) 0.90 (795) 0.88 (380) 0.91 (415) [ .]a

Difference: high – low [p-value] <0.01 [0.834] 0.04 [0.063]* >-0.01 [0.965] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Test statistics cannot be computed because of stratum with single sampling unit. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 393

Exhibit H5.5b: Proportion of Primary Shoppers Who Had Problems with EBT Card or Account 

in Past Month, by Treatment and Control Status, by Subgroup: Round 2 and 3 

Participant Surveys 

Total Treatment Control P-value 

Primary shopper employment status (N=3276)

Working full or part-time (N=643) 0.01 (5) 0.00 (1) 0.01 (4) [ .]a

Not working (N=2633) 0.04 (93) 0.03 (40) 0.04 (53) [0.132] 

Difference: working – not working [pvalue] -0.03 [0.005]*** -0.03 [0.018]** -0.03 [0.010]** 

Household composition (N=3423) 

Children (and no elderly) in 

household (N=1417) 0.03 (35) 0.02 (13) 0.03 (22) [0.155] 

Other household (N=2006) 0.04 (65) 0.03 (29) 0.04 (36) [0.368] 

Difference: children in HH – other HH 

[p-value] -0.01 [0.385] -0.01 [0.191] -0.01 [0.484] 

Household WIC status (past 30 days) (N=3270)

Participant (N=595) 0.03 (17) 0.02 (6) 0.03 (11) [ .]a

Non-participant (N=2675) 0.03 (80) 0.03 (36) 0.03 (44) [0.256] 

Difference: participant – nonparticipant [p-value] >-0.01 [0.994] >-0.01 [0.826] <0.01 [0.973] 

Household SNAP benefit amount (N=3423)

$200 or less (N=1935) 0.04 (62) 0.03 (27) 0.04 (35) [0.269] 

Over $200 (N=1488) 0.03 (38) 0.02 (15) 0.03 (23) [0.258] 

Difference: $200 or less – over $200 

[p-value] 0.01 [0.328] 0.01 [0.362] 0.01 [0.386] 

Baseline respondent fruit & vegetable intake (screener) (N=3423)

3+ servings/day (N=1725) 0.03 (48) 0.03 (24) 0.03 (24) [0.941] 

<3 servings/day (N=1698) 0.04 (52) 0.02 (18) 0.04 (34) [0.035]** 

Difference: 3+ servings – <3 servings 

[p-value] -0.01 [0.181] 0.01 [0.387] -0.01 [0.139] 

Baseline respondent fruit & vegetable intake (predicted) (N=2744)

High (N=1408) 0.03 (40) 0.02 (16) 0.03 (24) [0.169] 

Low (N=1336) 0.04 (40) 0.02 (14) 0.04 (26) [0.128] 

Difference: high – low [p-value] >-0.01 [0.736] >-0.01 [0.977] >-0.01 [ .][a] 

Baseline spending on TFV (N=2744)

High (N=1348) 0.04 (41) 0.02 (14) 0.04 (27) [0.073]* 

Low (N=1396) 0.03 (39) 0.02 (16) 0.04 (23) [ .]a

Difference: high – low [p-value] <0.01 [0.865] >-0.01 [0.726] <0.01 [ .]a 

Attitudes about food, fruits, & vegetables (N=3423)

High (N=1954) 0.03 (57) 0.02 (24) 0.03 (33) [0.351] 

Low (N=1469) 0.03 (43) 0.02 (18) 0.04 (25) [0.205] 

Difference: high – low [p-value] >-0.01 [0.769] <0.01 [0.808] >-0.01 [0.733] 

Barriers to eating fruits & vegetables (N=3423)

High (N=1683) 0.04 (48) 0.02 (17) 0.04 (31) [0.083]* 

Low (N=1740) 0.03 (52) 0.03 (25) 0.03 (27) [0.739] 

Difference: high – low [p-value] 0.01 [0.367] -0.01 [0.526] 0.01 [0.325] 

Barriers to grocery shopping (N=3423)

High (N=1541) 0.04 (50) 0.03 (19) 0.04 (31) [0.220] 

Low (N=1882) 0.03 (50) 0.02 (23) 0.03 (27) [0.362] 

Difference: high – low [p-value] 0.01 [0.259] <0.01 [0.629] 0.01 [0.286] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 394 ▌Appendix H: Analyses by Subgroup Abt Associates

Total Treatment Control P-value 

Fruits & vegetables at home (N=3423)

High (N=1492) 0.02 (31) 0.01 (10) 0.02 (21) [0.096]* 

Low (N=1931) 0.04 (69) 0.03 (32) 0.04 (37) [0.355] 

Difference: high – low [p-value] -0.02 [0.039]** -0.02 [0.011]** -0.02 [0.074]* 

Pre-HIP shopping patterns (N=2219)

Shopped primarily at HIP 

participating retailers (N=1336) 0.03 (39) 0.03 (17) 0.03 (22) [0.692]* 

Shopped primarily at non-HIP 

participating retailers (N=883) 0.03 (25) 0.01 (8) 0.04 (17) [ .]a

Difference: high – low [p-value] >-0.01 [0.853] 0.01 [0.150] >-0.01 [0.742] 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Test statistics cannot be computed because of stratum with single sampling unit. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 395

Exhibit H5.6: Proportion of Primary Shoppers Who Would Like to Keep Participating in HIP, 

by Subgroup: Round & 3 Participant Surveys, HIP Participants 

Proportion (N) 

Primary shopper employment status (N=1550) 

Working full or part-time (N=283) 0.96 (270) 

Not working (N=1267) 0.95 (1206) 

Difference: working – not working [p-value] 0.01 [0.371] 

Household composition (N=1603) 

Children (and no elderly) in household (N=648) 0.95 (622) 

Other household (N=955) 0.95 (901) 

Difference: children in HH – other HH [p-value] <0.01 [0.714] 

Household WIC status (past 30 days) (N=1547)

Participant (N=283) 0.96 (275) 

Non-participant (N=1264) 0.95 (1197) 

Difference: participant – non-participant [p-value] 0.02 [0.289] 

Household SNAP benefit amount (N=1603)

$200 or less (N=926) 0.95 (876) 

Over $200 (N=677) 0.95 (647) 

Difference: $200 or less – over $200 [p-value] >-0.01 [0.851] 

Baseline respondent fruit & vegetable intake (screener) (N=1603)

3+ servings/day (N=806) 0.96 (777) 

<3 servings/day (N=797) 0.93 (746) 

Difference: 3+ servings – <3 servings [p-value] 0.03 [0.022]** 

Baseline respondent fruit & vegetable intake (predicted) (N=1286)

High (N=671) 0.96 (647) 

Low (N=615) 0.95 (584) 

Difference: high – low [p-value] 0.01 [0.382] 

Baseline spending on TFV (N=1286)

High (N=626) 0.96 (607) 

Low (N=660) 0.95 (624) 

Difference: high – low [p-value] 0.01 [0.305] 

Attitudes about food, fruits, & vegetables (N=1603)

High (N=911) 0.95 (873) 

Low (N=692) 0.94 (650) 

Difference: high – low [p-value] 0.01 [0.315] 

Barriers to eating fruits & vegetables (N=1603)

High (N=760) 0.95 (720) 

Low (N=843) 0.95 (803) 

Difference: high – low [p-value] >-0.01 [0.980] 

Barriers to grocery shopping (N=1603)

High (N=709) 0.95 (678) 

Low (N=894) 0.94 (845) 

Difference: high – low [p-value] 0.01 [0.512] 

Fruits & vegetables at home (N=1603)

High (N=691) 0.95 (660) 

Low (N=912) 0.95 (863) 

Difference: high – low [p-value] <0.01 [0.724] 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 396 ▌Appendix H: Analyses by Subgroup Abt Associates

Proportion (N) 

Pre-HIP shopping patterns (N=1030)

Shopped primarily at HIP participating retailers (N=636) 0.97 (618) 

Shopped primarily at non-HIP participating retailers (N=394) 0.94 (369) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] 0.03 [0.021]** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don’t know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. 

Includes participants who reported they had not heard of HIP. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 397

Exhibit H6.1: Descriptive Results for EBT Analysis of Subgroups: Unweighted Percentage of 

All Households 

Round 2 Round 3 

HIP 

(%) 

Non-HIP 

(%) 

Total 

(N) 

HIP 

(%) 

Non-HIP 

(%) 

Total 

(N) 

Monthly SNAP benefit 

$1-$160 26.7 25.9 60,593 26.7 25.9 35,014 

$161–$200 37.3 37.8 87,904 37.1 37.5 50,435 

$201–$349 11.0 10.6 24,794 11.2 10.6 14,353 

$350 + 25.0 25.6 59,437 25.0 26.0 34,764 

Primary language 

Other primary language 73.3 73.5 170,930 73.3 73.4 98,739 

Spanish primary language 26.7 26.5 61,798 26.7 26.6 35,827 

Monthly income 

$0 21.8 22.1 51,408 21.4 21.8 29,257 

$1–$787 27.3 26.4 61,674 27.4 26.5 35,847 

$788–$1,083 27.2 27.1 63,171 27.5 27.4 36,894 

$1,084 + 23.7 24.4 56,475 23.7 24.3 32,568 

Household head age 

16–30 26.1 25.8 60,191 26.0 25.7 34,659 

31–40 21.3 20.7 48,305 21.4 20.6 27,918 

41–54 26.3 27.1 62,727 26.3 27.1 36,387 

55 + 26.4 26.4 61,505 26.2 26.5 35,602 

Household head race/ethnicity 

Hispanic 44.6 44.1 102,739 44.5 44.3 59,661 

White 35.5 36.4 84,425 35.6 36.2 48,605 

Black 13.1 12.8 29,777 13.1 12.8 17,233 

Other 6.8 6.8 15,787 6.8 6.7 9,067 

Disability status 

Not Disabled 46.3 46.7 108,635 45.9 46.3 62,213 

Disabled 53.7 53.3 124,093 54.1 53.7 72,353 

Household composition 

Household with children (no elderly) 36.4 36.2 84,301 36.6 36.5 49,148 

Household with elderly (with or 

without children) 13.1 13.5 31,267 12.9 13.5 18,049 

Other household 50.5 50.3 117,160 50.4 50.0 67,369 

Location 

Springfield 53.2 52.9 123,122 52.9 53.0 71,343 

Chicopee & Holyoke 24.8 24.9 57,971 25.2 24.9 33,603 

Other 22.0 22.2 51,635 22.0 22.0 29,620 

Household Size and Headship 

HH Size 2+ male head 4.9 5.0 11,551 4.7 4.9 6,534 

HH Size 2+ female head 40.0 40.2 93,409 40.4 40.6 54,549 

HH Size 1 male head 26.7 26.7 62,083 26.5 26.3 35,427 

HH Size 1 female head 28.4 28.2 65,685 28.4 28.3 38,056 

Source: 2011 case file data merged with EBT Transaction Data pooled across March-October 2012 (average of 

45,912 households per month). 
Evaluation of the Healt

pg. 398 ▌Appendix H: Analyses by Subgroup Exhibit H6.2: Differences in Impacts of SNAP Purchases and Eligible TFV Purchases at Supermarkets and Superstores by H

to October 2012 

SNAP purchases at participating supermarkets and 

superstores ($) 

TFV purchases at participating supermarkets and 

superstores ($) 

TFV pur

partici

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressio

adjusted 

treatmen

mean (SE

Monthly SNAP benefit $1–$160 97.74 

(1.59) 

94.64

(0.88) 

3.10

(0.043)** 

9.67

(0.25) 

8.37 

(0.11) 

1.29

(<0.001)*** 

10.97

(0.27) 

$161–$200 115.63 

(1.39) 

113.95

(0.78) 

1.682

(0.205) 

11.43

(0.23) 

10.44 

(0.10) 

0.99

(<0.001)*** 

9.93

(0.20) 

$201–$349 135.63 

(3.68) 

126.20

(1.53) 

9.43

(0.015)** 

13.21

(0.50) 

11.28 

(0.19) 

1.93

(<0.001)*** 

9.66

(0.28) 

$350 + 167.86 

(3.19) 

160.51

(1.43) 

7.36

(0.026)** 

14.86

(0.38) 

13.83 

(0.18) 

1.03

(0.008)*** 

9.44

(0.18) 

P-value for difference 0.139 0.375 Primary language Other primary language 129.54 

(1.28) 

124.39

(0.51) 

5.14

(<0.001)*** 

11.16

(0.16) 

10.10 

(0.06) 

1.06

(<.001)*** 

9.21

(0.12) 

Spanish primary language 118.14 

(1.95) 

116.03

(0.86) 

2.11

(0.300) 

14.49

(0.34) 

12.95 

(0.13) 

1.54

(<0.001)*** 

12.30

(0.25) 

P-value for difference 0.215 0.223 Monthly income $0 132.26 

(2.46) 

127.92

(1.08) 

4.33

(0.089)* 

12.09

(0.31) 

11.03 

(0.14) 

1.05

(0.001)*** 

9.80

(0.20) 

$1–$787 131.76 

(1.97) 

129.28

(0.85) 

2.48

(0.232) 

12.14

(0.28) 

11.27 

(0.11) 

0.88

(0.003)*** 

9.52

(0.20) 

$788–$1,083 137.10 

(1.74) 

130.59

(0.79) 

6.51

(<0.001)*** 

12.45

(0.26) 

11.43 

(0.11) 

1.01

(<0.001)*** 

9.33

(0.22) 

$1,084 + 103.67 

(2.56) 

99.71

(1.09) 

3.96

(0.145) 

11.46

(0.36) 

9.61 

(0.14) 

1.86

(<0.001)*** 

11.61

(0.24) 

P-value for difference 0.521 0.185 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Append

SNAP purchases at participating supermarkets and 

superstores ($) 

TFV purchases at participating supermarkets and 

superstores ($) 

TFV pur

partici

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressio

adjusted 

treatmen

mean (SE

Household head age 16–30 131.71 

(2.33) 

129.62

(0.97) 

2.08

(0.403) 

10.51

(0.26) 

9.53 

(0.11) 

0.97

(<0.001)*** 

8.15

(0.15) 

31–40 126.62 

(2.87) 

119.14

(1.05) 

7.48

(0.016)** 

10.99

(0.35) 

9.85 

(0.13) 

1.13

(0.002)*** 

8.78

(0.21) 

41–54 119.87 

(1.84) 

117.46

(0.78) 

2.41

(0.213) 

11.81

(0.27) 

11.20 

(0.11) 

0.61

(0.033)** 

9.83

(0.21) 

55 + 128.03 

(1.59) 

122.07

(0.84) 

5.96

(<0.001)*** 

14.61

(0.32) 

12.60 

(0.13) 

2.01

(<0.001)*** 

13.11

(0.30) 

P-value for difference 0.270 0.012** Household head race/ethnicity Hispanic 136.07 

(1.70) 

133.34

(0.68) 

2.72

(0.127) 

11.59

(0.20) 

10.85 

(0.08) 

0.75

(<0.001)*** 

8.99

(0.14) 

White 121.9 

(1.76) 

114.94

(0.78) 

6.96

(<0.001)*** 

12.43

(0.27) 

10.64 

(0.10) 

1.79

(<0.001)*** 

10.71

(0.21) 

Black 112.85 

(2.74) 

111.95

(1.19) 

0.91

(0.758) 

11.27

(0.34) 

10.78 

(0.16) 

0.49

(0.174) 

9.854

(0.28) 

Other 114.89 

(3.69) 

107.21

(1.49) 

7.68

(0.052)* 

14.51

(0.72) 

12.30 

(0.30) 

2.21

(0.005)*** 

13.78

(0.65) 

P-value for difference 0.177 0.004*** Disability status Not disabled 133.15 

(1.83) 

127.87

(0.69) 

5.28

(0.007)*** 

12.97

(0.24) 

11.52 

(0.09) 

1.45

(<0.001)*** 

10.27

(0.15) 

Disabled 120.74 

(1.24) 

117.21

(0.59) 

3.53

(0.005)*** 

11.25

(0.18) 

10.29 

(0.08) 

0.96

(<0.001)*** 

9.82

(0.16) 

P-value for difference 0.453 0.128 
Evaluation of the Healt

pg. 400 ▌Appendix H: Analyses by Subgroup 

SNAP purchases at participating supermarkets and 

superstores ($) 

TFV purchases at participating supermarkets and 

superstores ($) 

TFV pur

partici

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressio

adjusted 

treatmen

mean (SE

Household composition Household with children (no elderly) 126.14 

(2.53) 

118.08

(1.23) 

8.06

(0.002)*** 

11.54

(0.31) 

10.18 

(0.16) 

1.36

(<0.001)*** 

9.55

(0.16) 

Household with elderly (with or without children) 

127.92 

(2.14) 

124.36

(1.15) 

3.57

(0.085)* 

14.14

(0.49) 

12.12 

(0.20) 

2.02

(<0.001)*** 

13.46

(0.49) 

Other household 126.39 

(1.26) 

124.55

(0.79) 

1.84

(0.106) 

11.86

(0.19) 

11.02 

(0.11) 

0.84

(<0.001)*** 

9.51

(0.17) 

P-value for difference 0.082* 0.044** Location Springfield 122.50 

(1.44) 

121.01

(0.58) 

1.49

(0.33) 

11.49

(0.19) 

10.60 

(0.07) 

0.89

(<0.001)*** 

10.08

(0.14) 

Chicopee & Holyoke 137.19 

(2.19) 

130.3

(0.82) 

6.89

(0.003)*** 

12.00

(0.28) 

10.98 

(0.11) 

1.01

(<0.001)*** 

8.70

(0.19) 

Other 124.09 

(2.23) 

115.78

(0.96) 

8.31

(0.001)*** 

13.44

(0.39) 

11.34 

(0.14) 

2.10

(<0.001)*** 

11.47

(0.29) 

P-value for difference 0.025** 0.025** Household size and headship HH Size 2+ male head 127.25 

(6.09) 

117.37

(2.49) 

9.87

(0.131) 

16.43

(1.04) 

14.02 

(0.41) 

2.41

(0.032)** 

12.51

(0.66) 

HH Size 2+ female head 141.32 

(2.51) 

136.20

(1.32) 

5.12

(0.040)** 

12.80

(0.29) 

11.74 

(0.16) 

1.06

(<0.001)*** 

9.79

(0.16) 

HH Size 1 male head 107.17 

(1.62) 

104.52

(1.04) 

2.64

(0.066)* 

9.91

(0.25) 

8.87 

(0.14) 

1.04

(<0.001)*** 

9.16

(0.25) 

HH Size 1 female head 121.24 

(1.58) 

119.93

(1.03) 

1.31

(0.351) 

12.03

(0.28) 

10.95 

(0.14) 

1.07

(<0.001) *** 

10.78

(0.26) 

P-value for difference 0.374 0.697 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment and coStandard errors and test statistics are adjusted for clustering at the household level. Source: EBT Transaction Data, pooled across March-October 2012 merged with July 2011 case file data (average of 45,912 households per month). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 401

Exhibit H6.3a: Differences in Impacts of HIP on Self-Reported Monthly SNAP Expenditures, by 

Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) ($) 

Regressionadjusted control 

mean (SE) ($) 

Treatment-control 

impact 

(P-value) ($) 

Difference in 

impacts 

(P-value) ($) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 279.70 (7.53) 274.77 (5.49) 4.93 (0.569) 

<3 servings/day 270.72 (7.03) 279.14 (6.86) -8.43 (0.297) 

Impact: 3+ servings – <3 

servings 13.36 (0.261) 

Attitudes about food, fruits, & vegetables

High 279.29 (6.78) 276.43 (5.51) 2.86 (0.708) 

Low 269.76 (7.68) 277.43 (7.79) -7.67 (0.398) 

Impact: high – low 10.53 (0.370) 

Fruits & vegetables at home 

Frequently 271.86 (8.80) 266.38 (7.77) 5.48 (0.584) 

Infrequently 278.60 (6.71) 286.21 (6.64) -7.60 (0.288) 

Impact: freq. – infreq. 13.08 (0.293) 

Pre-HIP shopping patterns (N=2042 interviews from 1218 respondents)a

Shopped primarily at HIP 

participating retailers 261.62 (6.58) 259.77 (5.31) 1.86 (0.836) 

Shopped primarily at nonHIP participating retailers 283.87 (10.12) 276.13 (7.45) 7.74 (0.524) 

Impact: HIP shoppers – 

non-HIP-shoppers -5.88 (0.694) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don't know” and “refused” responses coded as 

missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may differ 

from differences between reported regression-adjusted means for the treatment and control groups and subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,131 interviews 

over 1,836 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 402 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.3b: Differences in Impacts of HIP on Self-Reported Monthly Expenditures on 

Groceries, by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) ($) 

Regressionadjusted control 

mean (SE) ($) 

Treatment-control 

impact 

(P-value) ($) 

Difference in 

impacts 

(P-value) ($) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 144.13 (6.15) 147.69 (6.42) -3.56 (0.645) 

<3 servings/day 154.15 (7.18) 148.94 (6.96) 5.21 (0.480) 

Impact: 3+ servings – <3 

servings -8.77 (0.414) 

Attitudes about food, fruits, & vegetables

High 146.89 (6.31) 144.57 (6.58) 2.32 (0.750) 

Low 151.85 (6.98) 153.45 (7.41) -1.61 (0.839) 

Impact: high – low 3.92 (0.717) 

Fruits & vegetables at home 

Frequently 147.19 (7.43) 147.91 (6.75) -0.72 (0.932) 

Infrequently 150.64 (6.42) 148.63 (6.24) 2.01 (0.768) 

Impact: freq. – infreq. -2.73 (0.802) 

Pre-HIP shopping patterns (N=1982 interviews from 1196 respondents)a

Shopped primarily at HIP 

participating retailers 145.73 (5.66) 138.73 (5.26) 7.00 (0.359) 

Shopped primarily at nonHIP participating retailers 149.17 (8.18) 145.97 (7.78) 3.21 (0.773) 

Impact: HIP shoppers – 

non-HIP-shoppers 3.79 (0.782) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don't know” and “refused” responses coded as 

missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may differ 

from differences between reported regression-adjusted means for the treatment and control groups and subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,051 interviews 

over 1,803 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 403

Exhibit H6.3c: Differences in Impacts of HIP on Self-Reported Monthly Expenditures on 

Groceries, Food Items, by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) ($) 

Regressionadjusted control 

mean (SE) ($) 

Treatment-control 

impact 

(P-value) ($) 

Difference in 

impacts 

(P-value) ($) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 101.46 (4.81) 107.55 (5.39) -6.09 (0.355) 

<3 servings/day 109.52 (5.39) 106.20 (5.04) 3.32 (0.613) 

Impact: 3+ servings – <3 

servings -9.41 (0.321) 

Attitudes about food, fruits, & vegetables

High 102.83 (5.35) 104.34 (5.64) -1.51 (0.807) 

Low 109.05 (6.11) 110.51 (6.54) -1.46 (0.837) 

Impact: high – low -0.05 (0.996) 

Fruits & vegetables at home 

Frequently 107.50 (6.47) 105.03 (6.06) 2.47 (0.726) 

Infrequently 104.08 (5.56) 108.59 (5.36) -4.51 (0.448) 

Impact: freq. – infreq. 6.98 (0.449) 

Pre-HIP shopping patterns (N=1915 interviews from 1166 respondents)a

Shopped primarily at HIP 

participating retailers 102.04 (5.07) 99.77 (4.91) 2.27 (0.743) 

Shopped primarily at nonHIP participating retailers 101.52 (7.18) 106.64 (7.26) -5.12 (0.604) 

Impact: HIP shoppers – 

non-HIP-shoppers 7.39 (0.546) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don't know” and “refused” responses coded as 

missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may differ 

from differences between reported regression-adjusted means for the treatment and control groups and subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=2,949 interviews 

over 1,758 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 404 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.3d: Differences in Impacts of HIP on Self-Reported Monthly Expenditures on 

Groceries, Non-Food Items, by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) ($) 

Regressionadjusted control 

mean (SE) ($) 

Treatment-control 

impact 

(P-value) ($) 

Difference in 

impacts 

(P-value) ($) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 44.14 (2.65) 42.91 (2.70) 1.24 (0.719) 

<3 servings/day 42.29 (2.48) 39.45 (2.39) 2.84 (0.342) 

Impact: 3+ servings – <3 

servings -1.61 (0.722) 

Attitudes about food, fruits, & vegetables 

High 42.07 (2.55) 39.16 (2.33) 2.91 (0.338) 

Low 44.93 (3.01) 44.22 (3.57) 0.71 (0.847) 

Impact: high – low 2.20 (0.649) 

Fruits & vegetables at home 

Frequently 42.16 (3.27) 44.01 (2.80) -1.85 (0.614) 

Infrequently 44.01 (2.56) 38.81 (2.48) 5.20 (0.064)* 

Impact: freq. – infreq. -7.05 (0.123) 

Pre-HIP shopping patterns (N=1915 interviews from 1166 respondents)a

Shopped primarily at HIP 

participating retailers 42.73 (2.55) 39.67 (2.26) 3.07 (0.361) 

Shopped primarily at nonHIP participating retailers 47.62 (3.62) 41.73 (3.25) 5.89 (0.212) 

Impact: HIP shoppers – 

non-HIP-shoppers -2.83 (0.633) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don't know” and “refused” responses coded as 

missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may differ 

from differences between reported regression-adjusted means for the treatment and control groups and subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=2,949 interviews 

over 1,758 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 405

Exhibit H6.3e: Differences in Impacts of HIP on Self-Reported Monthly Restaurant Expenditures, 

by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 32.09 (1.97) 36.16 (2.41) -4.07 (0.144) 

<3 servings/day 38.43 (2.03) 37.12 (2.44) 1.31 (0.646) 

Impact: 3+ servings – <3 

servings -5.38 (0.179) 

Attitudes about food, fruits, & vegetables 

High 33.53 (2.03) 35.70 (2.17) -2.17 (0.380) 

Low 37.36 (2.77) 37.93 (3.21) -0.56 (0.863) 

Impact: high – low -1.61 (0.691) 

Fruits & vegetables at home 

Frequently 32.60 (2.64) 36.77 (2.99) -4.18 (0.156) 

Infrequently 37.31 (2.56) 36.49 (2.73) 0.82 (0.783) 

Impact: freq. – infreq. -5.00 (0.262) 

Pre-HIP shopping patterns (N=1197 interviews from 1203 respondents)a

Shopped primarily at HIP 

participating retailers 32.14 (2.18) 36.40 (2.20) -4.26 (0.174) 

Shopped primarily at nonHIP participating retailers 35.86 (2.72) 36.34 (3.12) -0.48 (0.900) 

Impact: HIP shoppers – 

non-HIP-shoppers -3.78 (0.422) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don't know” and “refused” responses coded as 

missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may differ 

from differences between reported regression-adjusted means for the treatment and control groups and subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,088 interviews 

over 1,821 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 406 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.3f: Differences in Impacts of HIP on Self-Reported Monthly Fruit & Vegetable 

Expenditures, by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) ($) 

Regressionadjusted control 

mean (SE) ($) 

Treatment-control 

impact 

(P-value) ($) 

Difference in 

impacts 

(P-value) ($) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 76.38 (3.38) 68.28 (3.37) 8.10 (0.049)** 

<3 servings/day 80.43 (3.44) 76.32 (3.74) 4.11 (0.251) 

Impact: 3+ servings – <3 

servings 3.99 (0.471) 

Attitudes about food, fruits, & vegetables

High 78.27 (3.08) 72.57 (3.37) 5.70 (0.119) 

Low 78.00 (3.72) 71.21 (3.61) 6.79 (0.094)* 

Impact: high – low -1.09 (0.844) 

Fruits & vegetables at home 

Frequently 76.77 (3.85) 72.71 (3.60) 4.06 (0.365) 

Infrequently 79.27 (3.32) 71.37 (3.26) 7.90 (0.022)** 

Impact: Freq. - Infreq. -3.84 (0.509) 

Pre-HIP shopping patterns (N=1774 interviews from 1087 respondents)a

Shopped primarily at HIP 

participating retailers 71.80 (3.30) 69.37 (2.72) 2.42 (0.577) 

Shopped primarily at nonHIP participating retailers 78.25 (3.62) 72.72 (4.26) 5.53 (0.305) 

Impact: HIP shoppers – 

non-HIP-shoppers -3.10 (0.648) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Continuous outcomes: self-reported expenses in dollars per month; “don't know” and “refused” responses coded as 

missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may differ 

from differences between reported regression-adjusted means for the treatment and control groups and subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=2,708 interviews 

over 1,651 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: AppendExhibit H6.4: Differences in Impacts of SNAP Purchases by HIP Status, Retailer Type, and Subgroup, March to October 2012 

SNAP purchases at all retailers ($) SNAP purchases at participating retailers ($) SNAP p

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressio

adjusted 

treatmen

mean (SE

Monthly SNAP benefit $1–$160 205.45 

(2.07) 

202.19

(1.17) 

3.26

(0.097) 

106.89

(1.63) 

103.20 

(0.90) 

3.69

(0.018) 

98.56

(1.60) 

$161–$200 244.63 

(1.33) 

243.32

(0.97) 

1.30

(0.231) 

124.78

(1.38) 

123.31 

(0.79) 

1.47

(0.264) 

119.85

(1.37) 

$201–$349 273.72 

(4.43) 

264.13

(1.92) 

9.59

(0.039) 

146.28

(3.70) 

136.02 

(1.55) 

10.26

(0.008) 

127.44

(3.61) 

$350 + 348.25 

(3.28) 

340.72

(1.73) 

7.54

(0.020) 

180.81

(3.22) 

173.81 

(1.46) 

6.99

(0.036) 

167.45

(3.18) 

P-value for difference 0.106 0.092* Primary language Other primary language 265.41 

(1.29) 

260.57

(0.53) 

4.85

(<0.001) 

138.87

(1.28) 

134.02 

(0.51) 

4.85

(<0.001) 

126.54

(1.24) 

Spanish primary language 260.52 

(2.14) 

257.72

(0.95) 

2.80

(0.211) 

131.07

(1.98) 

127.86 

(0.88) 

3.21

(0.120) 

129.45

(2.01) 

P-value for difference 0.434 0.508 Monthly income $0 289.27 

(2.21) 

287.70

(1.50) 

1.57

(0.485) 

146.28

(2.47) 

141.72 

(1.10) 

4.56

(0.075) 

142.99

(2.42) 

$1–$787 281.09 

(1.79) 

275.49

(0.82) 

5.59

(0.003) 

143.2

(1.96) 

140.48 

(0.86) 

2.72

(0.189) 

137.88

(1.94) 

$788–$1,083 275.93 

(1.49) 

272.40

(0.74) 

3.53

(0.019) 

147.03

(1.75) 

140.89 

(0.78) 

6.13

(0.001) 

128.90

(1.60) 

$1,084 + 209.42 

(3.28) 

203.23

(1.29) 

6.19

(0.077) 

109.73

(2.62) 

105.5 

(1.109) 

4.23

(0.126) 

99.69

(2.64) 

P-value for difference 0.499 0.664 
Evaluation of the Healt

pg. 408 ▌Appendix H: Analyses by Subgroup 

SNAP purchases at all retailers ($) SNAP purchases at participating retailers ($) SNAP p

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressio

adjusted 

treatmen

mean (SE

Household head age 16–30 277.77 

(2.35) 

274.99

(1.00) 

2.79

(0.267) 

144.26

(2.36) 

141.47 

(0.97) 

2.78

(0.269) 

133.52

(2.27) 

31–40 261.60 

(3.15) 

255.40

(1.15) 

6.20

(0.069) 

136.27

(2.89) 

128.82 

(1.06) 

7.46

(0.017) 

125.33

(2.84) 

41–54 257.24 

(1.92) 

253.26

(0.83) 

3.98

(0.050) 

129.91

(1.85) 

127.54 

(0.78) 

2.37

(0.225) 

127.34

(1.83) 

55 + 259.72 

(1.39) 

255.14

(0.83) 

4.58

(0.001) 

136.87

(1.57) 

131.26 

(0.84) 

5.62

(<0.001) 

122.85

(1.56) 

P-value for difference 0.863 0.379 Household head race/ethnicity Hispanic 261.64 

(1.80) 

257.74

(0.72) 

3.89

(0.040) 

148.58

(1.70) 

145.75 

(0.69) 

2.82

(0.114) 

113.06

(1.59) 

White 266.78 

(1.73) 

259.99

(0.78) 

6.79

(<0.001) 

128.19

(1.76) 

121.66 

(0.79) 

6.52

(<0.001) 

138.60

(1.76) 

Black 262.66 

(2.92) 

261.96

(1.31) 

0.70

(0.824) 

126.28

(2.80) 

124.67 

(1.20) 

1.61

(0.589) 

136.38

(2.95) 

Other 269.12 

(3.42) 

268.31

(1.46) 

0.82

(0.825) 

126.11

(3.78) 

117.02 

(1.55) 

9.09

(0.025) 

143.01

(4.07) 

P-value for difference 0.246 0.232 Disability status Not disabled 273.58 

(1.92) 

267.81

(0.74) 

5.77

(0.005) 

142.96

(1.85) 

137.37 

(0.70) 

5.59

(0.005) 

130.62

(1.80) 

Disabled 255.9 

(1.24) 

252.86

(0.62) 

3.04

(0.015) 

131.45

(1.24) 

128.05 

(0.60) 

3.41

(0.007) 

124.44

(1.22) 

P-value for difference 0.259 0.353 Household composition Household with children (no elderly) 250.33 

(2.76) 

242.49

(1.43) 

7.84

(0.005) 

135.42

(2.55) 

127.27 

(1.25) 

8.15

(0.002) 

114.91

(2.47) 

Household with elderly (with or without children) 

269.63 

(1.91) 

264.40

(1.16) 

5.22

(0.004) 

137.86

(2.11) 

134.23 

(1.16) 

3.63

(0.075) 

131.76

(2.13) 

Other household 272.62 

(1.29) 

271.12

(0.89) 

1.50

(0.161) 

137.50

(1.27) 

135.59 

(0.80) 

1.91

(0.094) 

135.11

(1.31) 

P-value for difference 0.037** 0.084* 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Append

SNAP purchases at all retailers ($) SNAP purchases at participating retailers ($) SNAP p

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressionadjusted 

treatment 

mean (SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol 

impact 

(P-value) 

P-value for 

difference 

in impacts 

Regressio

adjusted 

treatmen

mean (SE

Location Springfield 262.35 

(1.52) 

260.75

(0.61) 

1.60

(0.325) 

130.58

(1.46) 

129.18 

(0.59) 

1.40

(0.366) 

131.76

(1.45) 

Chicopee & Holyoke 266.79 

(2.13) 

259.79 

(0.85) 

7.00 

(0.002) 

152.43 

(2.19) 

145.87 

(0.83) 

6.56 

(0.005) 

114.37 

(1.92) 

Other 265.34 

(2.31) 

257.60 

(0.99) 

7.74 

(0.002) 

134.08 

(2.24) 

124.85 

(0.97) 

9.23 

(<0.001) 

131.25 

(2.34) 

P-value for difference 0.047** 0.013** Household size and headship HH Size 2+ male head 276.66 

(7.98) 

282.45 

(3.60) 

-5.78 

(0.507) 

139.90 

(6.33) 

130.99 

(2.63) 

9.00 

(0.187) 

136.77 

(6.63) 

HH Size 2+ female head 283.28 

(3.37) 

278.79 

(1.93) 

4.48 

(0.161) 

150.63 

(2.57) 

145.46 

(1.35) 

5.17 

(0.042) 

132.65 

(2.59) 

HH Size 1 male head 240.09 

(1.78) 

237.82 

(1.42) 

2.27 

(0.066) 

119.07 

(1.62) 

115.96 

(1.06) 

3.11 

(0.028) 

121.01 

(1.65) 

HH Size 1 female head 251.82 

(1.74) 

250.31 

(1.40) 

1.51 

(0.227) 

130.76 

(1.58) 

129.82 

(1.05) 

0.94 

(0.497) 

121.05 

(1.64) 

P-value for difference 0.659 0.324 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment and coStandard errors and test statistics are adjusted for clustering at the household level. Source: EBT Transaction Data, pooled across March-October 2012 merged with July 2011 case file data (average of 45,912 households per month). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 410 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.5a: Differences in Impacts of HIP on Usual Grocery Store Type (Large Chain 

Grocery Store or Supermarket), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.813 (0.017) 0.820 (0.016) -0.007 (0.735) 

<3 servings/day 0.817 (0.018) 0.812 (0.017) 0.005 (0.808) 

Impact: 3+ servings – <3 

servings -0.012 (0.682) 

Attitudes about food, fruits, & vegetables

High 0.825 (0.017) 0.826 (0.017) -0.001 (0.972) 

Low 0.802 (0.022) 0.803 (0.022) >-0.001 (0.995) 

Impact: high – low -0.001 (0.986) 

Fruits & vegetables at home 

Frequently 0.839 (0.020) 0.817 (0.020) 0.022 (0.326) 

Infrequently 0.796 (0.019) 0.816 (0.019) -0.019 (0.352) 

Impact: freq. – infreq. 0.041 (0.177) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers 0.840 (0.016) 0.842 (0.015) -0.001 (0.963) 

Shopped primarily at nonHIP participating retailers 0.801 (0.023) 0.793 (0.022) 0.009 (0.779) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.010 (0.801) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 411

Exhibit H6.5b: Differences in Impacts of HIP on Usual Grocery Store Type (Natural or Organic 

Supermarket (such as Whole Foods Market), by Baseline Intake, Attitudes, and 

Pre-HIP Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.002 (0.002) 0.005 (0.003) -0.004 (0.244) 

<3 servings/day 0.011 (0.004) 0.004 (0.004) 0.007 (0.158) 

Impact: 3+ servings – <3 

servings -0.010 (0.088)* 

Attitudes about food, fruits, & vegetables

High 0.004 (0.003) 0.004 (0.002) <0.001 (0.915) 

Low 0.009 (0.004) 0.006 (0.004) 0.003 (0.543) 

Impact: high – low -0.003 (0.673) 

Fruits & vegetables at home 

Frequently 0.005 (0.004) 0.008 (0.004) -0.003 (0.496) 

Infrequently 0.008 (0.003) 0.002 (0.002) 0.005 (0.105) 

Impact: freq. – infreq. -0.009 (0.152) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers 0.007 (0.003) 0.002 (0.002) 0.004 (0.236) 

Shopped primarily at nonHIP participating retailers 0.009 (0.005) 0.006 (0.005) 0.003 (0.656) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.001 (0.873) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 412 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.5c: Differences in Impacts of HIP on Usual Grocery Store Type (Small Local Store 

or Corner Store), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.027 (0.006) 0.029 (0.007) -0.002 (0.806) 

<3 servings/day 0.017 (0.007) 0.022 (0.007) -0.005 (0.565) 

Impact: 3+ servings – <3 

servings 0.003 (0.806) 

Attitudes about food, fruits, & vegetables

High 0.023 (0.008) 0.021 (0.008) 0.002 (0.797) 

Low 0.021 (0.011) 0.032 (0.010) -0.011 (0.293) 

Impact: high – low 0.013 (0.316) 

Fruits & vegetables at home 

Frequently 0.019 (0.008) 0.020 (0.008) -0.001 (0.931) 

Infrequently 0.025 (0.007) 0.031 (0.009) -0.006 (0.505) 

Impact: freq. – infreq. 0.005 (0.657) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers 0.029 (0.007) 0.018 (0.005) 0.011 (0.181) 

Shopped primarily at nonHIP participating retailers 0.020 (0.009) 0.041 (0.009) -0.021 (0.113) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.031 (0.049)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,298 interviews over 

1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 413

Exhibit H6.5d: Differences in Impacts of HIP on Usual Grocery Store Type (Convenience Store 

(such as 7-11 or mini market), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.002 (0.002) <0.001 (0.001) 0.002 (0.260) 

<3 servings/day 0.002 (0.001) 0.003 (0.002) -0.001 (0.740) 

Impact: 3+ servings – <3 

servings 0.003 (0.375) 

Attitudes about food, fruits, & vegetables

High 0.002 (0.002) <0.001 (0.001) 0.002 (0.238) 

Low 0.002 (0.002) 0.003 (0.002) -0.001 (0.693) 

Impact: high – low 0.003 (0.390) 

Fruits & vegetables at home 

Frequently 0.002 (0.001) 0.002 (0.001) <0.001 (0.983) 

Infrequently 0.002 (0.002) 0.001 (0.002) 0.001 (0.676) 

Impact: freq. – infreq. -0.001 (0.676) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers 0.001 (0.001) 0.001 (0.001) >-0.001 (0.893) 

Shopped primarily at nonHIP participating retailers 0.002 (0.003) 0.003 (0.003) -0.001 (0.752) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.001 (0.769) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 414 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.5e: Differences in Impacts of HIP on Usual Grocery Store Type (Warehouse Club 

Store (such as Sam's Club or Costco), by Baseline Intake, Attitudes, and PreHIP Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.011 (0.006) 0.023 (0.007) -0.012 (0.128) 

<3 servings/day 0.025 (0.007) 0.024 (0.007) 0.001 (0.945) 

Impact: 3+ servings – <3 

servings -0.013 (0.256) 

Attitudes about food, fruits, & vegetables

High 0.026 (0.006) 0.029 (0.008) -0.003 (0.700) 

Low 0.007 (0.006) 0.016 (0.008) -0.009 (0.235) 

Impact: high – low 0.005 (0.635) 

Fruits & vegetables at home 

Frequently 0.011 (0.007) 0.028 (0.008) -0.017 (0.083)* 

Infrequently 0.023 (0.006) 0.020 (0.007) 0.003 (0.673) 

Impact: freq. – infreq. -0.020 (0.100) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers 0.008 (0.005) 0.014 (0.005) -0.005 (0.487) 

Shopped primarily at nonHIP participating retailers 0.025 (0.010) 0.048 (0.012) -0.023 (0.176) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.018 (0.330) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 415

Exhibit H6.5f: Differences in Impacts of HIP on Usual Grocery Store Type (Discount 

Superstore (such as Walmart), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.132 (0.014) 0.106 (0.013) 0.026 (0.125) 

<3 servings/day 0.119 (0.015) 0.131 (0.014) -0.011 (0.527) 

Impact: 3+ servings – <3 

servings 0.038 (0.130) 

Attitudes about food, fruits, & vegetables

High 0.103 (0.013) 0.100 (0.013) 0.003 (0.855) 

Low 0.154 (0.018) 0.143 (0.018) 0.011 (0.587) 

Impact: high – low -0.008 (0.748) 

Fruits & vegetables at home 

Frequently 0.115 (0.017) 0.116 (0.016) >-0.001 (0.992) 

Infrequently 0.133 (0.016) 0.120 (0.015) 0.013 (0.446) 

Impact: freq. – infreq. -0.013 (0.600) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers 0.112 (0.014) 0.117 (0.013) -0.004 (0.825) 

Shopped primarily at nonHIP participating retailers 0.123 (0.018) 0.097 (0.015) 0.026 (0.254) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.030 (0.322) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 416 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.5g: Differences in Impacts of HIP on Usual Grocery Store Type (Ethnic Market), by 

Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 0.006 (0.003) 0.010 (0.003) -0.004 (0.393) 

<3 servings/day 0.001 (0.002) 0.001 (0.002) -0.001 (0.720) 

Impact: 3+ servings – <3 

servings -0.003 (0.531) 

Attitudes about food, fruits, & vegetables

High 0.007 (0.003) 0.009 (0.003) -0.002 (0.564) 

Low -0.001 (0.002) 0.001 (0.003) -0.002 (0.364) 

Impact: high – low >-0.001 (0.916) 

Fruits & vegetables at home 

Frequently 0.002 (0.004) 0.003 (0.003) -0.002 (0.683) 

Infrequently 0.005 (0.002) 0.008 (0.004) -0.003 (0.339) 

Impact: freq. – infreq. 0.001 (0.844) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers <0.001 (0.001) 0.004 (0.002) -0.004 (0.131) 

Shopped primarily at nonHIP participating retailers 0.008 (0.006) 0.006 (0.004) 0.001 (0.839) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.006 (0.420) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 417

Exhibit H6.5h: Differences in Impacts of HIP on Usual Grocery Store Type (Farmers 

Market/Co-Op), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.001 (0.002) <0.001 (0.001) 0.001 (0.711) 

<3 servings/day 0.005 (0.003) 0.002 (0.001) 0.003 (0.355) 

Impact: 3+ servings – <3 

servings -0.002 (0.633) 

Attitudes about food, fruits, & vegetables

High 0.002 (0.002) 0.001 (0.001) <0.001 (0.943) 

Low 0.005 (0.003) 0.001 (0.001) 0.004 (0.211) 

Impact: high – low -0.004 (0.360) 

Fruits & vegetables at home 

Frequently 0.005 (0.004) 0.003 (0.002) 0.002 (0.669) 

Infrequently 0.001 (0.002) <0.001 (0.001) 0.002 (0.266) 

Impact: freq. – infreq. <0.001 (0.981) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers <0.001 (0.003) 0.001 (0.001) -0.001 (0.721) 

Shopped primarily at nonHIP participating retailers -0.001 (0.004) 0.003 (0.001) -0.004 (0.379) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.003 (0.562) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 418 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.5i: Differences in Impacts of HIP on Usual Grocery Store Type (Some Other 

Location), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.005 (0.003) 0.006 (0.003) -0.001 (0.895) 

<3 servings/day 0.001 (0.002) 0.001 (0.001) <0.001 (0.845) 

Impact: 3+ servings – <3 

servings -0.001 (0.852) 

Attitudes about food, fruits, & vegetables

High 0.005 (0.002) 0.007 (0.003) -0.002 (0.489) 

Low 0.001 (0.002) -0.002 (0.002) 0.003 (0.136) 

Impact: high – low -0.005 (0.166) 

Fruits & vegetables at home 

Frequently 0.003 (0.002) 0.005 (0.003) -0.002 (0.379) 

Infrequently 0.004 (0.002) 0.002 (0.002) 0.002 (0.576) 

Impact: freq. – infreq. -0.004 (0.280) 

Pre-HIP shopping patterns (N=2137 interviews from 1262 respondents)a

Shopped primarily at HIP 

participating retailers 0.001 (0.002) 0.002 (0.001) >-0.001 (0.918) 

Shopped primarily at nonHIP participating retailers 0.004 (0.004) 0.003 (0.002) 0.002 (0.711) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.002 (0.716) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,298 interviews 

over 1,912 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 419

Exhibit H6.6a: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Prices/Affordability), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.467 (0.023) 0.494 (0.022) -0.027 (0.346) 

<3 servings/day 0.500 (0.023) 0.532 (0.022) -0.032 (0.256) 

Impact: 3+ servings – <3 

servings 0.005 (0.894) 

Attitudes about food, fruits, & vegetables

High 0.471 (0.022) 0.492 (0.023) -0.021 (0.417) 

Low 0.498 (0.027) 0.541 (0.027) -0.042 (0.166) 

Impact: high – low 0.021 (0.612) 

Fruits & vegetables at home 

Frequently 0.479 (0.027) 0.513 (0.026) -0.034 (0.240) 

Infrequently 0.486 (0.026) 0.512 (0.025) -0.027 (0.336) 

Impact: freq. – infreq. -0.007 (0.856) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.508 (0.022) 0.520 (0.023) -0.012 (0.707) 

Shopped primarily at nonHIP participating retailers 0.444 (0.028) 0.498 (0.027) -0.054 (0.160) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.042 (0.402) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 420 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6b: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Close to Home), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.305 (0.020) 0.287 (0.019) 0.017 (0.495) 

<3 servings/day 0.317 (0.021) 0.362 (0.021) -0.045 (0.084)* 

Impact: 3+ servings – <3 

servings 0.063 (0.088)* 

Attitudes about food, fruits, & vegetables

High 0.293 (0.020) 0.325 (0.020) -0.031 (0.185) 

Low 0.329 (0.026) 0.322 (0.024) 0.007 (0.802) 

Impact: high – low -0.038 (0.295) 

Fruits & vegetables at home 

Frequently 0.323 (0.025) 0.325 (0.024) -0.002 (0.944) 

Infrequently 0.297 (0.022) 0.322 (0.022) -0.025 (0.307) 

Impact: freq. – infreq. 0.023 (0.528) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.327 (0.021) 0.346 (0.021) -0.019 (0.505) 

Shopped primarily at nonHIP participating retailers 0.317 (0.025) 0.324 (0.023) -0.008 (0.820) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.012 (0.793) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 421

Exhibit H6.6c: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Variety of Products), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.228 (0.017) 0.193 (0.017) 0.035 (0.104) 

<3 servings/day 0.256 (0.019) 0.206 (0.017) 0.050 (0.029)** 

Impact: 3+ servings – <3 

servings -0.015 (0.641) 

Attitudes about food, fruits, & vegetables

High 0.251 (0.018) 0.196 (0.017) 0.055 (0.009)*** 

Low 0.229 (0.021) 0.204 (0.021) 0.025 (0.289) 

Impact: high – low 0.029 (0.363) 

Fruits & vegetables at home 

Frequently 0.225 (0.020) 0.191 (0.019) 0.034 (0.126) 

Infrequently 0.256 (0.020) 0.207 (0.018) 0.049 (0.027)** 

Impact: freq. – infreq. -0.015 (0.633) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.248 (0.019) 0.190 (0.017) 0.057 (0.024)** 

Shopped primarily at nonHIP participating retailers 0.213 (0.023) 0.189 (0.019) 0.023 (0.413) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.034 (0.387) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 422 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6d: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Produce Better or Fresher), by Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.149 (0.016) 0.133 (0.014) 0.017 (0.385) 

<3 servings/day 0.174 (0.016) 0.134 (0.015) 0.039 (0.033)** 

Impact: 3+ servings – <3 

servings -0.023 (0.402) 

Attitudes about food, fruits, & vegetables

High 0.172 (0.016) 0.146 (0.016) 0.025 (0.168) 

Low 0.148 (0.018) 0.116 (0.017) 0.032 (0.088)* 

Impact: high – low -0.007 (0.791) 

Fruits & vegetables at home 

Frequently 0.189 (0.020) 0.163 (0.018) 0.026 (0.214) 

Infrequently 0.137 (0.017) 0.107 (0.015) 0.030 (0.086)* 

Impact: freq. – infreq. -0.004 (0.883) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.152 (0.016) 0.114 (0.014) 0.038 (0.072)* 

Shopped primarily at nonHIP participating retailers 0.173 (0.021) 0.165 (0.019) 0.008 (0.779) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.030 (0.407) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 423

Exhibit H6.6e: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Sales/Promotions in Store), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 0.105 (0.013) 0.141 (0.014) -0.036 (0.040)** 

<3 servings/day 0.094 (0.013) 0.122 (0.014) -0.028 (0.107) 

Impact: 3+ servings – <3 

servings -0.008 (0.756) 

Attitudes about food, fruits, & vegetables

High 0.103 (0.014) 0.145 (0.014) -0.042 (0.014)** 

Low 0.096 (0.016) 0.114 (0.017) -0.018 (0.320) 

Impact: high – low -0.024 (0.317) 

Fruits & vegetables at home 

Frequently 0.092 (0.016) 0.151 (0.018) -0.059 (0.001)*** 

Infrequently 0.104 (0.015) 0.115 (0.016) -0.010 (0.554) 

Impact: freq. – infreq. -0.049 (0.048)** 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.106 (0.014) 0.125 (0.015) -0.018 (0.346) 

Shopped primarily at nonHIP participating retailers 0.095 (0.016) 0.116 (0.016) -0.021 (0.364) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.003 (0.925) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 424 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6f: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Preferred Products Are Available), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.099 (0.013) 0.134 (0.014) -0.035 (0.041)** 

<3 servings/day 0.090 (0.013) 0.122 (0.014) -0.032 (0.043)** 

Impact: 3+ servings – <3 

servings -0.002 (0.923) 

Attitudes about food, fruits, & vegetables

High 0.091 (0.014) 0.123 (0.013) -0.031 (0.049)** 

Low 0.099 (0.015) 0.136 (0.017) -0.037 (0.026)** 

Impact: high – low 0.006 (0.796) 

Fruits & vegetables at home 

Frequently 0.112 (0.016) 0.147 (0.016) -0.035 (0.056)* 

Infrequently 0.080 (0.013) 0.112 (0.014) -0.032 (0.026)** 

Impact: freq. – infreq. -0.003 (0.910) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.097 (0.013) 0.139 (0.014) -0.042 (0.026)** 

Shopped primarily at nonHIP participating retailers 0.094 (0.018) 0.149 (0.018) -0.055 (0.028)** 

Impact: HIP shoppers – 

non-HIP-shoppers 0.013 (0.686) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 425

Exhibit H6.6g: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Familiarity with Store), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.064 (0.011) 0.067 (0.011) -0.002 (0.864) 

<3 servings/day 0.073 (0.012) 0.046 (0.010) 0.027 (0.081)* 

Impact: 3+ servings – <3 

servings -0.029 (0.159) 

Attitudes about food, fruits, & vegetables 

High 0.063 (0.011) 0.060 (0.011) 0.003 (0.822) 

Low 0.077 (0.016) 0.051 (0.012) 0.026 (0.128) 

Impact: high – low -0.023 (0.267) 

Fruits & vegetables at home 

Frequently 0.067 (0.013) 0.067 (0.011) 0.001 (0.965) 

Infrequently 0.070 (0.013) 0.047 (0.009) 0.022 (0.121) 

Impact: freq. – infreq. -0.022 (0.269) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.067 (0.011) 0.053 (0.011) 0.014 (0.332) 

Shopped primarily at nonHIP participating retailers 0.069 (0.013) 0.057 (0.012) 0.012 (0.522) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.003 (0.908) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 426 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6h: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(One Stop Shopping), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.065 (0.011) 0.057 (0.009) 0.009 (0.497) 

<3 servings/day 0.050 (0.010) 0.056 (0.010) -0.005 (0.630) 

Impact: 3+ servings – <3 

servings 0.014 (0.418) 

Attitudes about food, fruits, & vegetables

High 0.053 (0.010) 0.045 (0.009) 0.008 (0.438) 

Low 0.064 (0.013) 0.072 (0.013) -0.008 (0.572) 

Impact: high – low 0.016 (0.366) 

Fruits & vegetables at home 

Frequently 0.061 (0.012) 0.057 (0.012) 0.003 (0.797) 

Infrequently 0.055 (0.010) 0.055 (0.010) <0.001 (0.991) 

Impact: freq. – infreq. 0.003 (0.849) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.053 (0.010) 0.072 (0.011) -0.019 (0.175) 

Shopped primarily at nonHIP participating retailers 0.054 (0.012) 0.040 (0.010) 0.014 (0.359) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.033 (0.113) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 427

Exhibit H6.6i: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Easy to Get There), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 0.021 (0.005) 0.021 (0.006) <0.001 (0.982) 

<3 servings/day 0.025 (0.007) 0.014 (0.005) 0.010 (0.216) 

Impact: 3+ servings – <3 

servings -0.010 (0.354) 

Attitudes about food, fruits, & vegetables 

High 0.030 (0.007) 0.021 (0.006) 0.009 (0.197) 

Low 0.014 (0.008) 0.014 (0.006) 0.001 (0.951) 

Impact: high – low 0.009 (0.428) 

Fruits & vegetables at home 

Frequently 0.033 (0.009) 0.020 (0.007) 0.012 (0.118) 

Infrequently 0.015 (0.008) 0.015 (0.007) >-0.001 (0.966) 

Impact: freq. – infreq. 0.013 (0.249) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.019 (0.006) 0.018 (0.006) <0.001 (0.963) 

Shopped primarily at nonHIP participating retailers 0.021 (0.010) 0.017 (0.007) 0.003 (0.781) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.003 (0.826) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,281 interviews over 

1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 428 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6j: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Quality), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.023 (0.007) 0.047 (0.010) -0.024 (0.015)** 

<3 servings/day 0.059 (0.009) 0.052 (0.009) 0.007 (0.525) 

Impact: 3+ servings – <3 

servings -0.031 (0.040)** 

Attitudes about food, fruits, & vegetables

High 0.042 (0.008) 0.060 (0.010) -0.017 (0.075)* 

Low 0.039 (0.011) 0.035 (0.010) 0.003 (0.765) 

Impact: high – low -0.021 (0.178) 

Fruits & vegetables at home 

Frequently 0.048 (0.010) 0.056 (0.009) -0.008 (0.519) 

Infrequently 0.034 (0.008) 0.044 (0.008) -0.009 (0.320) 

Impact: freq. – infreq. 0.002 (0.901) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.042 (0.008) 0.052 (0.009) -0.010 (0.371) 

Shopped primarily at nonHIP participating retailers 0.040 (0.011) 0.037 (0.011) 0.003 (0.853) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.013 (0.514) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 429

Exhibit H6.6k: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Close to Work), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.006 (0.002) 0.005 (0.003) 0.001 (0.825) 

<3 servings/day 0.007 (0.004) 0.003 (0.002) 0.004 (0.400) 

Impact: 3+ servings – <3 

servings -0.003 (0.556) 

Attitudes about food, fruits, & vegetables

High 0.009 (0.004) 0.005 (0.002) 0.004 (0.320) 

Low 0.002 (0.003) 0.002 (0.002) <0.001 (0.997) 

Impact: high – low 0.004 (0.423) 

Fruits & vegetables at home 

Frequently 0.001 (0.003) 0.001 (0.002) >-0.001 (0.982) 

Infrequently 0.010 (0.004) 0.006 (0.002) 0.004 (0.313) 

Impact: freq. – infreq. -0.004 (0.387) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.007 (0.003) 0.004 (0.003) 0.003 (0.432) 

Shopped primarily at nonHIP participating retailers 0.007 (0.004) 0.003 (0.003) 0.004 (0.312) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.001 (0.910) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 430 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6l: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Close to Some Other Location), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.004 (0.002) 0.005 (0.002) -0.001 (0.829) 

<3 servings/day 0.003 (0.003) 0.003 (0.002) 0.001 (0.840) 

Impact: 3+ servings – <3 

servings -0.001 (0.771) 

Attitudes about food, fruits, & vegetables

High 0.006 (0.002) 0.005 (0.002) 0.001 (0.737) 

Low 0.001 (0.003) 0.002 (0.002) -0.001 (0.870) 

Impact: high – low 0.002 (0.760) 

Fruits & vegetables at home 

Frequently 0.001 (0.003) 0.002 (0.003) -0.001 (0.788) 

Infrequently 0.006 (0.003) 0.006 (0.003) 0.001 (0.798) 

Impact: freq. – infreq. -0.002 (0.702) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers <0.001 (<0.001) 0.002 (0.002) -0.001 (0.415) 

Shopped primarily at nonHIP participating retailers 0.007 (0.004) 0.002 (0.002) 0.005 (0.264) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.006 (0.199) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 431

Exhibit H6.6m: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Disability Accessible), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.002 (0.001) 0.001 (0.001) <0.001 (0.850) 

<3 servings/day 0.005 (0.002) 0.001 (0.001) 0.003 (0.228) 

Impact: 3+ servings – <3 

servings -0.003 (0.358) 

Attitudes about food, fruits, & vegetables 

High 0.003 (0.002) 0.001 (0.001) 0.002 (0.299) 

Low 0.003 (0.001) 0.002 (0.001) 0.001 (0.634) 

Impact: high – low 0.002 (0.586) 

Fruits & vegetables at home 

Frequently 0.001 (0.002) >-0.001 (0.001) 0.001 (0.529) 

Infrequently 0.005 (0.002) 0.003 (0.001) 0.002 (0.312) 

Impact: freq. – infreq. -0.001 (0.674) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.002 (0.002) 0.002 (0.001) <0.001 (0.915) 

Shopped primarily at nonHIP participating retailers 0.002 (0.001) 0.002 (0.001) <0.001 (0.990) 

Impact: HIP shoppers – 

non-HIP-shoppers <0.001 (0.912) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 432 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6n: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Hours of Operation Convenient), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.008 (0.003) 0.006 (0.003) 0.001 (0.720) 

<3 servings/day 0.005 (0.004) 0.009 (0.004) -0.004 (0.444) 

Impact: 3+ servings – <3 

servings 0.005 (0.395) 

Attitudes about food, fruits, & vegetables

High 0.005 (0.003) 0.006 (0.004) -0.001 (0.780) 

Low 0.008 (0.004) 0.010 (0.006) -0.002 (0.723) 

Impact: high – low 0.001 (0.884) 

Fruits & vegetables at home 

Frequently 0.009 (0.004) 0.007 (0.004) 0.002 (0.684) 

Infrequently 0.004 (0.003) 0.008 (0.004) -0.004 (0.278) 

Impact: freq. – infreq. 0.006 (0.314) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.003 (0.003) 0.008 (0.003) -0.004 (0.344) 

Shopped primarily at nonHIP participating retailers 0.004 (0.003) 0.003 (0.003) 0.001 (0.861) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.005 (0.438) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 433

Exhibit H6.6o: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(EBT Card Accepted), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.036 (0.008) 0.012 (0.004) 0.024 (0.003)*** 

<3 servings/day 0.021 (0.007) 0.017 (0.005) 0.004 (0.642) 

Impact: 3+ servings – <3 

servings 0.020 (0.077)* 

Attitudes about food, fruits, & vegetables

High 0.034 (0.007) 0.019 (0.006) 0.016 (0.051)* 

Low 0.021 (0.008) 0.009 (0.006) 0.012 (0.148) 

Impact: high – low 0.004 (0.723) 

Fruits & vegetables at home 

Frequently 0.034 (0.008) 0.017 (0.006) 0.017 (0.021)** 

Infrequently 0.024 (0.008) 0.013 (0.005) 0.011 (0.197) 

Impact: freq. – infreq. 0.006 (0.594) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.026 (0.007) 0.016 (0.005) 0.010 (0.245) 

Shopped primarily at nonHIP participating retailers 0.027 (0.009) 0.018 (0.006) 0.009 (0.428) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.001 (0.966) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 434 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6p: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Ethnic Foods Are Available), by Baseline Intake, Attitudes, and Pre-HIP 

Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.008 (0.004) 0.004 (0.003) 0.004 (0.367) 

<3 servings/day 0.005 (0.003) 0.011 (0.004) -0.006 (0.107) 

Impact: 3+ servings – <3 

servings 0.010 (0.084)* 

Attitudes about food, fruits, & vegetables

High 0.001 (0.003) -0.001 (0.002) 0.002 (0.618) 

Low 0.014 (0.004) 0.019 (0.006) -0.005 (0.328) 

Impact: high – low 0.007 (0.274) 

Fruits & vegetables at home 

Frequently 0.009 (0.004) 0.010 (0.004) -0.001 (0.769) 

Infrequently 0.004 (0.004) 0.005 (0.004) -0.001 (0.804) 

Impact: freq. – infreq. <-0.001 (0.993) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.006 (0.004) 0.005 (0.003) 0.001 (0.811) 

Shopped primarily at nonHIP participating retailers 0.010 (0.006) 0.008 (0.004) 0.003 (0.706) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.001 (0.845) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 435

Exhibit H6.6q: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Good Service), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.029 (0.007) 0.019 (0.005) 0.009 (0.258) 

<3 servings/day 0.026 (0.007) 0.016 (0.005) 0.010 (0.207) 

Impact: 3+ servings – <3 

servings -0.001 (0.932) 

Attitudes about food, fruits, & vegetables

High 0.028 (0.006) 0.024 (0.006) 0.004 (0.578) 

Low 0.027 (0.008) 0.010 (0.005) 0.017 (0.044)** 

Impact: high – low -0.013 (0.257) 

Fruits & vegetables at home 

Frequently 0.029 (0.008) 0.023 (0.007) 0.006 (0.409) 

Infrequently 0.026 (0.009) 0.013 (0.006) 0.013 (0.133) 

Impact: freq. – infreq. -0.006 (0.570) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.021 (0.006) 0.020 (0.006) 0.001 (0.923) 

Shopped primarily at nonHIP participating retailers 0.022 (0.009) 0.014 (0.007) 0.008 (0.491) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.007 (0.613) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 436 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6r: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Bulk Purchases), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.016 (0.006) 0.030 (0.007) -0.014 (0.103) 

<3 servings/day 0.027 (0.008) 0.020 (0.007) 0.007 (0.479) 

Impact: 3+ servings – <3 

servings -0.021 (0.104) 

Attitudes about food, fruits, & vegetables

High 0.030 (0.008) 0.037 (0.008) -0.007 (0.470) 

Low 0.010 (0.007) 0.009 (0.006) 0.001 (0.878) 

Impact: high – low -0.008 (0.507) 

Fruits & vegetables at home 

Frequently 0.020 (0.008) 0.032 (0.010) -0.012 (0.256) 

Infrequently 0.022 (0.007) 0.019 (0.008) 0.003 (0.730) 

Impact: freq. – infreq. -0.015 (0.277) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.012 (0.006) 0.021 (0.006) -0.010 (0.270) 

Shopped primarily at nonHIP participating retailers 0.027 (0.009) 0.030 (0.009) -0.003 (0.823) 

Impact: HIP shoppers – 

non-HIP-shoppers -0.007 (0.674) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 437

Exhibit H6.6s: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Clean), by Baseline Intake, Attitudes, and Pre-HIP Shopping Patterns 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.021 (0.007) 0.020 (0.006) 0.001 (0.884) 

<3 servings/day 0.019 (0.006) 0.015 (0.005) 0.004 (0.508) 

Impact: 3+ servings – <3 

servings -0.003 (0.781) 

Attitudes about food, fruits, & vegetables

High 0.021 (0.006) 0.017 (0.006) 0.003 (0.629) 

Low 0.019 (0.006) 0.017 (0.006) 0.002 (0.777) 

Impact: high – low 0.001 (0.921) 

Fruits & vegetables at home 

Frequently 0.013 (0.007) 0.004 (0.006) 0.009 (0.182) 

Infrequently 0.027 (0.006) 0.029 (0.008) -0.002 (0.775) 

Impact: freq. – infreq. 0.011 (0.299) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.011 (0.005) 0.014 (0.005) -0.003 (0.682) 

Shopped primarily at nonHIP participating retailers 0.025 (0.007) 0.029 (0.009) -0.005 (0.685) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.002 (0.892) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 438 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.6t: Differences in Impacts of HIP on Reason for Choosing Usual Place to Shop 

(Some Other Reason), by Baseline Intake, Attitudes, and Pre-HIP Shopping 

Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.009 (0.004) 0.004 (0.002) 0.005 (0.204) 

<3 servings/day 0.010 (0.005) 0.011 (0.004) -0.002 (0.771) 

Impact: 3+ servings – <3 

servings 0.007 (0.322) 

Attitudes about food, fruits, & vegetables

High 0.003 (0.004) 0.005 (0.003) -0.002 (0.711) 

Low 0.017 (0.006) 0.011 (0.005) 0.006 (0.318) 

Impact: high – low -0.008 (0.300) 

Fruits & vegetables at home 

Frequently 0.015 (0.007) 0.007 (0.004) 0.009 (0.189) 

Infrequently 0.004 (0.004) 0.008 (0.003) -0.004 (0.364) 

Impact: freq. – infreq. 0.012 (0.108) 

Pre-HIP shopping patterns (N=2130 interviews from 1258 respondents)a

Shopped primarily at HIP 

participating retailers 0.007 (0.003) 0.001 (0.002) 0.006 (0.128) 

Shopped primarily at nonHIP participating retailers 0.014 (0.007) 0.011 (0.005) 0.003 (0.725) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.003 (0.758) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,281 interviews 

over 1,904 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 439

Exhibit H6.7a: Differences in Impacts of HIP on Grocery Shopping Frequency, by Baseline 

Intake, Attitudes, and Pre-HIP Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 5.270 (0.041) 5.200 (0.036) 0.070 (0.158) 

<3 servings/day 5.209 (0.039) 5.237 (0.039) -0.029 (0.566) 

Impact: 3+ servings – <3 

servings 0.099 (0.168) 

Attitudes about food, fruits, & vegetables 

High 5.245 (0.040) 5.221 (0.037) 0.024 (0.610) 

Low 5.230 (0.048) 5.214 (0.047) 0.016 (0.757) 

Impact: high – low 0.007 (0.919) 

Fruits & vegetables at home 

Frequently 5.240 (0.048) 5.258 (0.043) -0.018 (0.719) 

Infrequently 5.235 (0.042) 5.183 (0.042) 0.052 (0.275) 

Impact: freq. – infreq. -0.071 (0.317) 

Pre-HIP shopping patterns (N=2151 interviews from 1270 respondents)a

Shopped primarily at HIP 

participating retailers 5.227 (0.039) 5.123 (0.036) 0.104 (0.041)** 

Shopped primarily at nonHIP participating retailers 5.179 (0.052) 5.209 (0.045) -0.030 (0.664) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.134 (0.120) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcome, 1=yearly or not at all, 2=2 to 3 times a year, 3=every other month, 4=once a month, 

5=every other week, 6=once a week, 7=more than once a week; “don’t know” and “refused” responses coded as 

missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,311 interviews 

over 1,921 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 440 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.7b: Differences in Impacts of HIP on Probability of Going Out of Way to Shop for 

Fruits & Vegetables at a Particular Store, by Baseline Intake, Attitudes, and PreHIP Shopping Patterns Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.380 (0.021) 0.418 (0.020) -0.038 (0.146) 

<3 servings/day 0.392 (0.023) 0.385 (0.021) 0.007 (0.792) 

Impact: 3+ servings – <3 

servings -0.045 (0.235) 

Attitudes about food, fruits, & vegetables

High 0.371 (0.021) 0.413 (0.020) -0.043 (0.088)* 

Low 0.408 (0.025) 0.386 (0.025) 0.021 (0.453) 

Impact: high – low -0.064 (0.093)* 

Fruits & vegetables at home 

Frequently 0.367 (0.026) 0.405 (0.025) -0.038 (0.168) 

Infrequently 0.403 (0.023) 0.399 (0.023) 0.004 (0.887) 

Impact: freq. – infreq. -0.042 (0.268) 

Pre-HIP shopping patterns (N=2151 interviews from 1270 respondents)a

Shopped primarily at HIP 

participating retailers 0.399 (0.022) 0.364 (0.020) 0.035 (0.232) 

Shopped primarily at nonHIP participating retailers 0.383 (0.028) 0.427 (0.026) -0.043 (0.244) 

Impact: HIP shoppers – 

non-HIP-shoppers 0.079 (0.102) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcome: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' and 'refused' 

responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,283 interviews 

over 1,907 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 441

Exhibit H6.8: Proportion of Primary Shoppers Who Reported Changing Store for Purchasing 

Fruits & Vegetables Because of HIP, by Subgroup: Round 2 & 3 Participant 

Surveys, HIP Participants 

Proportion (N) 

Baseline Respondent Fruit & Vegetable Intake (screener) (N=1611) 

3+ servings/day (N=802) 0.29 (236) 

<3 servings/day (N=809) 0.26 (209) 

Difference: 3+ servings – <3 servings [P-value] 0.04 [0.166] 

Attitudes about Food, Fruits, & Vegetables (N=1611)

High (N=914) 0.31 (284) 

Low (N=697) 0.23 (161) 

Difference: high – low [P-value] 0.07 [0.005]*** 

Fruits & Vegetables at Home (N=1611)

High (N=688) 0.27 (193) 

Low (N=923) 0.28 (252) 

Difference: high – low [P-value] -0.01 [0.742] 

Pre-HIP Shopping Patternsa (N=1030)

Shopped primarily at HIP participating retailers (N=636) 0.26 (164) 

Shopped primarily at non-HIP participating retailers (N=394) 0.32 (126) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] -0.05 [0.109]* 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 

 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 442 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.9a: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Other Store Has Greater Variety of Fruits and 

Vegetables, by Subgroup: Round 2 & 3 Participant Surveys, HIP Participants 

Proportion (N) 

Baseline Respondent Fruit & Vegetable Intake (screener) (N=444) 

3+ servings/day (N=236) 0.67 (157) 

<3 servings/day (N=208) 0.62 (130) 

Difference: 3+ servings – <3 servings [P-value] 0.05 [0.327] 

Attitudes about Food, Fruits, & Vegetables (N=444)

High (N=284) 0.69 (196) 

Low (N=16) 0.56 (91) 

Difference: high – low [P-value] 0.13 [0.013]** 

Fruits & Vegetables at Home (N=444) 

High (N=192) 0.61 (119) 

Low (N=252) 0.67 (168) 

Difference: high – low [P-value] -0.06 [0.249] 

Pre-HIP Shopping Patternsa (N=289) 

Shopped primarily at HIP participating retailers (N=163) 0.77 (126) 

Shopped primarily at non-HIP participating retailers (N=126) 0.55 (68) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] 0.22 [0.001]*** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 443

Exhibit H6.9b: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Price of Fruits and Vegetables More Affordable at 

Other Store, by Subgroup: Round 2 & 3 Participant Surveys, HIP Participants 

Proportion (N) 

Baseline Respondent Fruit & Vegetable Intake (screener) (N=442) 

3+ servings/day (N=235) 0.79 (189) 

<3 servings/day (N=207) 0.75 (153) 

Difference: 3+ servings – <3 servings [P-value] 0.04 [0.322] 

Attitudes about Food, Fruits, & Vegetables (N=442)

High (N=284) 0.77 (219) 

Low (N=158) 0.78 (123) 

Difference: High - Low [P-value] >-0.01 [0.916] 

Fruits & Vegetables at Home (N=442)

High (N=191) 0.76 (144) 

Low (N=251) 0.78 (198) 

Difference: high – low [P-value] -0.03 [0.519] 

Pre-HIP Shopping Patternsa (N=287)

Shopped primarily at HIP participating retailers (N=162) 0.86 (138) 

Shopped primarily at non-HIP participating retailers (N=125) 0.69 (86) 

Difference: HIP shoppers – non-HIP-shoppers [P-value] 0.17 [0.001]*** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 444 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.9c: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Other Store Has Fresh Fruits and Vegetables, by 

Subgroup: Round 2 & 3 Participant Surveys, HIP Participants 

Proportion (N) 

Baseline Respondent Fruit & Vegetable Intake (screener) (N=442) 

3+ servings/day (N=234) 0.79 (184) 

<3 servings/day (N=208) 0.71 (149) 

Difference: 3+ servings – <3 servings [P-value] 0.08 [0.109] 

Attitudes about Food, Fruits, & Vegetables (N=442)

High (N=282) 0.77 (217) 

Low (N=160) 0.72 (116) 

Difference: high – low [P-value] 0.06 [0.217] 

Fruits & Vegetables at Home (N=442) 

High (N=190) 0.77 (145) 

Low (N=252) 0.74 (188) 

Difference: high – low [P-value] 0.03 [0.543] 

Pre-HIP Shopping Patternsa (N=288) 

Shopped primarily at HIP participating retailers (N=162) 0.83 (136) 

Shopped primarily at non-HIP participating retailers (N=126) 0.66 (82) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] 0.16 [0.011]** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 445

Exhibit H6.9d: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables Because Other Store Participates in HIP, by Subgroup: Round 

2 & 3 Participant Surveys, HIP Participants 

Proportion (N) 

Baseline Respondent Fruit & Vegetable Intake (screener) (N=443) 

3+ servings/day (N=236) 0.21 (50) 

<3 servings/day (N=207) 0.23 (50) 

Difference: 3+ servings – <3 servings [P-value] -0.02 [0.655] 

Attitudes about Food, Fruits, & Vegetables (N=443)

High (N=283) 0.21 (61) 

Low (N=160) 0.24 (39) 

Difference: high – low [P-value] -0.02 [0.630] 

Fruits & Vegetables at Home (N=443)

High (N=191) 0.24 (44) 

Low (N=252) 0.21 (56) 

Difference: high – low [P-value] 0.03 [0.579] 

Pre-HIP Shopping Patternsa (N=288)

Shopped primarily at HIP participating retailers (N=163) 0.16 (26) 

Shopped primarily at non-HIP participating retailers (N=125) 0.31 (40) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] -0.14 [0.014]** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

 Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 446 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H6.9e: Proportion of Primary Shoppers Who Changed Stores for Purchasing Fruits 

and Vegetables for Some Other Reason, by Subgroup: Round 2 & 3 Participant 

Surveys, HIP Participants 

Proportion (N) 

Baseline Respondent Fruit & Vegetable Intake (screener) (N=443) 

3+ servings/day (N=236) 0.05 (13) 

<3 servings/day (N=207) 0.05 (10) 

Difference: 3+ servings – <3 servings [P-value] <0.01 [0.880] 

Attitudes about Food, Fruits, & Vegetables (N=443)

High (N=283) 0.04 (12) 

Low (N=160) 0.07 (11) 

Difference: high – low [P-value] -0.03 [0.170] 

Fruits & Vegetables at Home (N=443) 

High (N=191) 0.06 (13) 

Low (N=252) 0.04 (10) 

Difference: high – low [P-value] 0.03 [0.262] 

Pre-HIP Shopping Patternsa (N=288) 

Shopped primarily at HIP participating retailers (N=163) 0.01 (1) 

Shopped primarily at non-HIP participating retailers (N=125) 0.06 (9) 

Difference: HIP shoppers - non-HIP-shoppers [P-value] -0.05 [0.034]** 

Weighted proportions (unweighted Ns). 

Two-sided chi-square test: *p<0.1, **p<0.05, ***p<0.01. 

“Don't know” and “refused” responses coded as missing. 

Due to rounding, reported differences in proportions across subgroups may differ from differences between 

proportions for the subgroups. 

Test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 447

Exhibit H7.1a: Differences in Impacts of HIP on Self-Reported Receipt of Messages about 

Fruits and Vegetables in Past 3 Months, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.776 (0.019) 0.685 (0.019) 0.091 (<0.001) *** 

<3 servings/day 0.734 (0.021) 0.672 (0.020) 0.062 (0.016)** 

Impact: 3+ servings – <3 

servings 0.028 (0.422) 

Attitudes about food, fruits, & vegetables 

High 0.784 (0.020) 0.701 (0.020) 0.083 (<0.001) *** 

Low 0.719 (0.024) 0.649 (0.026) 0.070 (0.013)** 

Impact: high – low 0.013 (0.708) 

Fruits & vegetables at home 

Frequently 0.759 (0.025) 0.669 (0.026) 0.090 (0.001)*** 

Infrequently 0.753 (0.022) 0.686 (0.022) 0.067 (0.006)*** 

Impact: freq. – infreq. 0.023 (0.512) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,392 interviews over 

1,965 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 448 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.1b: Differences in Impacts of HIP on Self-Reported Attendance at Nutrition 

Education Class or Program in Past 3 Months, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.117 (0.014) 0.121 (0.014) -0.004 (0.809) 

<3 servings/day 0.084 (0.014) 0.095 (0.013) -0.011 (0.545) 

Impact: 3+ servings – <3 

servings 0.006 (0.796) 

Attitudes about food, fruits, & vegetables

High 0.106 (0.015) 0.105 (0.013) 0.001 (0.969) 

Low 0.095 (0.016) 0.112 (0.017) -0.017 (0.321) 

Impact: high – low 0.018 (0.466) 

Fruits & vegetables at home 

Frequently 0.112 (0.018) 0.116 (0.018) -0.003 (0.857) 

Infrequently 0.092 (0.016) 0.102 (0.015) -0.010 (0.550) 

Impact: freq. – infreq. 0.007 (0.783) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Binary outcomes, 1=yes, 0=no; “don’t know” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,406 interviews over 

1,966 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 449

Exhibit H7.2a: Differences in Impacts of HIP on Level of Agreement that “I enjoy trying new 

foods,” by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.920 (0.034) 3.845 (0.036) 0.075 (0.080)* 

<3 servings/day 3.878 (0.039) 3.823 (0.042) 0.056 (0.279) 

Impact: 3+ servings – <3 

servings 0.019 (0.779) 

Attitudes about food, fruits, & vegetables

High 3.918 (0.035) 3.830 (0.037) 0.089 (0.018)** 

Low 3.876 (0.048) 3.840 (0.053) 0.036 (0.540) 

Impact: high – low 0.052 (0.456) 

Fruits & vegetables at home 

Frequently 3.957 (0.045) 3.884 (0.046) 0.073 (0.133) 

Infrequently 3.854 (0.040) 3.794 (0.042) 0.060 (0.193) 

Impact: freq. – infreq. 0.013 (0.849) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,381 interviews over 

1,956 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 450 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.2b: Differences in Impacts of HIP on Level of Agreement that “I enjoy trying new 

fruits,” by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.941 (0.035) 3.957 (0.033) -0.016 (0.701) 

<3 servings/day 3.864 (0.037) 3.911 (0.037) -0.047 (0.342) 

Impact: 3+ servings – <3 

servings 0.030 (0.643) 

Attitudes about food, fruits, & vegetables

High 3.954 (0.037) 3.973 (0.036) -0.019 (0.586) 

Low 3.839 (0.053) 3.881 (0.049) -0.042 (0.470) 

Impact: high – low 0.023 (0.738) 

Fruits & vegetables at home 

Frequently 3.902 (0.044) 3.913 (0.045) -0.011 (0.820) 

Infrequently 3.905 (0.038) 3.952 (0.037) -0.047 (0.287) 

Impact: freq. – infreq. 0.036 (0.585) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,388 interviews over 

1,960 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 451

Exhibit H7.2c: Differences in Impacts of HIP on Level of Agreement that “I enjoy trying new 

vegetables,” by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.736 (0.038) 3.615 (0.039) 0.121 (0.012)** 

<3 servings/day 3.627 (0.043) 3.689 (0.041) -0.062 (0.234) 

Impact: 3+ servings – <3 

servings 0.183 (0.010)*** 

Attitudes about food, fruits, & vegetables

High 3.711 (0.039) 3.712 (0.039) -0.001 (0.977) 

Low 3.642 (0.052) 3.569 (0.052) 0.073 (0.226) 

Impact: high – low -0.074 (0.310) 

Fruits & vegetables at home 

Frequently 3.735 (0.051) 3.677 (0.050) 0.058 (0.261) 

Infrequently 3.638 (0.043) 3.631 (0.044) 0.007 (0.883) 

Impact: freq. – infreq. 0.051 (0.473) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,392 interviews over 

1,961 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 452 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.2d: Differences in Impacts of HIP on Level of Agreement that “I eat enough fruits to 

keep me healthy,” by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.778 (0.035) 3.726 (0.038) 0.052 (0.248) 

<3 servings/day 3.678 (0.042) 3.647 (0.041) 0.031 (0.576) 

Impact: 3+ servings – <3 

servings 0.021 (0.770) 

Attitudes about food, fruits, & vegetables

High 3.748 (0.039) 3.738 (0.039) 0.010 (0.811) 

Low 3.705 (0.052) 3.620 (0.050) 0.086 (0.146) 

Impact: high – low -0.076 (0.298) 

Fruits & vegetables at home 

Frequently 3.747 (0.051) 3.785 (0.050) -0.038 (0.446) 

Infrequently 3.713 (0.044) 3.610 (0.044) 0.102 (0.037)** 

Impact: freq. – infreq. -0.140 (0.044)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,383 interviews over 

1,957 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 453

Exhibit H7.2e: Differences in Impacts of HIP on Level of Agreement that “I eat enough 

vegetables to keep me healthy,” by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.798 (0.035) 3.732 (0.035) 0.065 (0.144) 

<3 servings/day 3.669 (0.044) 3.651 (0.038) 0.018 (0.735) 

Impact: 3+ servings – <3 

servings 0.047 (0.498) 

Attitudes about food, fruits, & vegetables

High 3.773 (0.039) 3.723 (0.035) 0.050 (0.243) 

Low 3.688 (0.052) 3.652 (0.049) 0.036 (0.534) 

Impact: high – low 0.014 (0.847) 

Fruits & vegetables at home 

Frequently 3.734 (0.050) 3.701 (0.045) 0.033 (0.517) 

Infrequently 3.736 (0.043) 3.686 (0.041) 0.050 (0.296) 

Impact: freq. – infreq. -0.017 (0.803) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,393 interviews over 

1,958 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 454 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.2f: Differences in Impacts of HIP on Level of Agreement that “I often encourage 

family/friends to eat fruits & vegetables,” by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.966 (0.036) 3.829 (0.039) 0.137 (0.003)*** 

<3 servings/day 3.764 (0.040) 3.808 (0.041) -0.044 (0.381) 

Impact: 3+ servings – <3 

servings 0.181 (0.008)*** 

Attitudes about food, fruits, & vegetables

High 3.836 (0.037) 3.767 (0.039) 0.068 (0.101) 

Low 3.905 (0.049) 3.889 (0.050) 0.016 (0.782) 

Impact: high – low 0.053 (0.460) 

Fruits & vegetables at home 

Frequently 3.864 (0.048) 3.767 (0.049) 0.097 (0.043)** 

Infrequently 3.870 (0.043) 3.860 (0.042) 0.010 (0.840) 

Impact: freq. – infreq. 0.087 (0.200) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,346 interviews over 

1,941 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 455

Exhibit H7.2g: Differences in Impacts of HIP on Positive Attitudes Toward Food, Fruits, & 

Vegetables Scale, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.855 (0.022) 3.783 (0.022) 0.072 (0.008)*** 

<3 servings/day 3.747 (0.025) 3.751 (0.025) -0.004 (0.896) 

Impact: 3+ servings – <3 

servings 0.076 (0.069)* 

Attitudes about food, fruits, & vegetables

High 3.823 (0.023) 3.788 (0.023) 0.034 (0.182) 

Low 3.775 (0.030) 3.739 (0.030) 0.036 (0.295) 

Impact: high – low -0.002 (0.969) 

Fruits & vegetables at home 

Frequently 3.821 (0.030) 3.785 (0.028) 0.036 (0.237) 

Infrequently 3.786 (0.025) 3.753 (0.025) 0.033 (0.251) 

Impact: freq. – infreq. 0.004 (0.931) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Scale is the mean of variables with these categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither 

Agree nor Disagree, 4=Agree, 5=Strongly Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,392 interviews over 

1,959 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 456 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.3a: Differences in Impacts of HIP on Level of Agreement that “Hard to eat 

vegetables because don't know how to prepare,” by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.400 (0.042) 2.404 (0.042) -0.005 (0.934) 

<3 servings/day 2.376 (0.045) 2.392 (0.040) -0.015 (0.777) 

Impact: 3+ servings – <3 

servings 0.011 (0.889) 

Attitudes about food, fruits, & vegetables

High 2.424 (0.047) 2.422 (0.043) 0.002 (0.968) 

Low 2.343 (0.052) 2.366 (0.052) -0.023 (0.686) 

Impact: high – low 0.025 (0.748) 

Fruits & vegetables at home 

Frequently 2.458 (0.054) 2.384 (0.052) 0.074 (0.195) 

Infrequently 2.335 (0.048) 2.409 (0.046) -0.074 (0.164) 

Impact: freq. – infreq. 0.149 (0.059)* 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,366 interviews over 

1,948 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 457

Exhibit H7.3b: Differences in Impacts of HIP on Level of Agreement that “Hard to eat 

vegetables because hard to find where I shop,” by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 2.206 (0.035) 2.203 (0.034) 0.003 (0.940) 

<3 servings/day 2.167 (0.037) 2.214 (0.036) -0.047 (0.317) 

Impact: 3+ servings – <3 

servings 0.050 (0.438) 

Attitudes about food, fruits, & vegetables

High 2.235 (0.036) 2.252 (0.036) -0.017 (0.708) 

Low 2.124 (0.041) 2.149 (0.042) -0.025 (0.596) 

Impact: high – low 0.008 (0.900) 

Fruits & vegetables at home 

Frequently 2.150 (0.044) 2.162 (0.044) -0.012 (0.796) 

Infrequently 2.216 (0.038) 2.245 (0.039) -0.029 (0.511) 

Impact: freq. – infreq. 0.017 (0.794) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,369 interviews over 

1,948 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 458 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.3c: Differences in Impacts of HIP on Level of Agreement that “Hard to eat fruits 

because hard to find where I shop,” by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.193 (0.035) 2.152 (0.032) 0.041 (0.348) 

<3 servings/day 2.134 (0.036) 2.153 (0.036) -0.019 (0.676) 

Impact: 3+ servings – <3 

servings 0.060 (0.345) 

Attitudes about food, fruits, & vegetables

High 2.191 (0.035) 2.186 (0.035) 0.005 (0.909) 

Low 2.129 (0.040) 2.106 (0.039) 0.023 (0.619) 

Impact: high – low -0.018 (0.776) 

Fruits & vegetables at home 

Frequently 2.142 (0.043) 2.159 (0.044) -0.016 (0.716) 

Infrequently 2.180 (0.039) 2.148 (0.037) 0.032 (0.458) 

Impact: freq. – infreq. -0.048 (0.438) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,372 interviews over 

1,950 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 459

Exhibit H7.3d: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits & 

vegetables as much as would like because cost too much,” by Baseline Intake 

and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 2.874 (0.047) 2.949 (0.050) -0.075 (0.222) 

<3 servings/day 2.967 (0.049) 2.933 (0.047) 0.035 (0.568) 

Impact: 3+ servings – <3 

servings -0.110 (0.208) 

Attitudes about food, fruits, & vegetables

High 2.965 (0.053) 2.979 (0.051) -0.014 (0.806) 

Low 2.866 (0.058) 2.890 (0.059) -0.024 (0.702) 

Impact: high – low 0.009 (0.913) 

Fruits & vegetables at home 

Frequently 2.896 (0.063) 2.904 (0.060) -0.008 (0.909) 

Infrequently 2.941 (0.051) 2.971 (0.052) -0.030 (0.599) 

Impact: freq. – infreq. 0.022 (0.800) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,379 interviews over 

1,953 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 460 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.3e: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits & 

vegetables as much as would like because they spoil,” by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.772 (0.046) 2.867 (0.044) -0.096 (0.095)* 

<3 servings/day 2.839 (0.047) 2.931 (0.049) -0.092 (0.125) 

Impact: 3+ servings – <3 

servings -0.003 (0.970) 

Attitudes about food, fruits, & vegetables

High 2.798 (0.049) 2.857 (0.048) -0.060 (0.279) 

Low 2.813 (0.056) 2.955 (0.058) -0.142 (0.023)** 

Impact: high – low 0.082 (0.321) 

Fruits & vegetables at home 

Frequently 2.782 (0.060) 2.842 (0.060) -0.060 (0.317) 

Infrequently 2.823 (0.053) 2.943 (0.053) -0.120 (0.035)** 

Impact: freq. – infreq. 0.060 (0.472) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,372 interviews over 

1,950 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 461

Exhibit H7.3f: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits & 

vegetables as much as would like because family dislikes,” by Baseline Intake 

and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.108 (0.032) 2.066 (0.029) 0.042 (0.289) 

<3 servings/day 2.174 (0.039) 2.147 (0.037) 0.027 (0.594) 

Impact: 3+ servings – <3 

servings 0.015 (0.814) 

Attitudes about food, fruits, & vegetables

High 2.174 (0.039) 2.093 (0.033) 0.081 (0.057)* 

Low 2.096 (0.041) 2.123 (0.042) -0.027 (0.549) 

Impact: high – low 0.108 (0.084)* 

Fruits & vegetables at home 

Frequently 2.158 (0.045) 2.108 (0.041) 0.050 (0.272) 

Infrequently 2.125 (0.039) 2.104 (0.037) 0.021 (0.624) 

Impact: freq. – infreq. 0.029 (0.642) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,230 interviews over 

1,889 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 462 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.3g: Differences in Impacts of HIP on Level of Agreement that “Don't eat fruits & 

vegetables as much as would like because I dislike,” by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.013 (0.031) 2.001 (0.031) 0.011 (0.773) 

<3 servings/day 2.048 (0.033) 2.119 (0.035) -0.071 (0.093)* 

Impact: 3+ servings – <3 

servings 0.082 (0.154) 

Attitudes about food, fruits, & vegetables 

High 2.062 (0.034) 2.045 (0.034) 0.017 (0.640) 

Low 1.987 (0.041) 2.078 (0.044) -0.091 (0.047)** 

Impact: high – low 0.108 (0.066)* 

Fruits & vegetables at home 

Frequently 2.056 (0.042) 2.088 (0.044) -0.032 (0.453) 

Infrequently 2.006 (0.035) 2.035 (0.035) -0.029 (0.452) 

Impact: freq. – infreq. -0.003 (0.952) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree nor Disagree, 4=Agree, 5=Strongly 

Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,379 interviews over 

1,954 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 463

Exhibit H7.3h: Differences in Impacts of HIP on Barriers to Eating Fruits & Vegetables Scale, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.359 (0.023) 2.376 (0.021) -0.016 (0.575) 

<3 servings/day 2.386 (0.024) 2.418 (0.023) -0.032 (0.282) 

Impact: 3+ servings – <3 

servings 0.015 (0.711) 

Attitudes about food, fruits, & vegetables

High 2.399 (0.024) 2.405 (0.023) -0.006 (0.834) 

Low 2.338 (0.025) 2.384 (0.027) -0.046 (0.106) 

Impact: high – low 0.040 (0.331) 

Fruits & vegetables at home 

Frequently 2.378 (0.029) 2.382 (0.029) -0.004 (0.895) 

Infrequently 2.368 (0.025) 2.407 (0.027) -0.040 (0.155) 

Impact: freq. – infreq. 0.035 (0.395) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Scale is the mean of variables with these categorical outcomes: 1=Strongly Disagree, 2=Disagree, 3=Neither 

Agree nor Disagree, 4=Agree, 5=Strongly Agree; “don’t know” and “does not apply” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module), pooled Round 2 and Round 3 sample (N=3,358 interviews over 

1,943 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 464 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.4a: Differences in Impacts of HIP on How Often Kept from Grocery Shopping, by 

Limited Transportation, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.223 (0.051) 2.154 (0.051) 0.069 (0.300) 

<3 servings/day 2.126 (0.055) 2.026 (0.050) 0.100 (0.145) 

Impact: 3+ servings – <3 

servings -0.031 (0.747) 

Attitudes about food, fruits, & vegetables

High 2.194 (0.053) 2.147 (0.052) 0.047 (0.429) 

Low 2.153 (0.065) 2.017 (0.067) 0.136 (0.074)* 

Impact: high – low -0.089 (0.355) 

Fruits & vegetables at home 

Frequently 2.143 (0.063) 2.129 (0.063) 0.014 (0.836) 

Infrequently 2.202 (0.061) 2.060 (0.055) 0.142 (0.034)** 

Impact: freq. – infreq. -0.129 (0.170) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcome: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' and 'refused' 

responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,275 interviews 

over 1,902 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 465

Exhibit H7.4b: Differences in Impacts of HIP on How Often Kept from Grocery Shopping, by 

Distance to Grocery Store, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 1.969 (0.050) 1.999 (0.053) -0.030 (0.646) 

<3 servings/day 1.967 (0.055) 1.900 (0.052) 0.068 (0.314) 

Impact: 3+ servings – <3 

servings -0.097 (0.293) 

Attitudes about food, fruits, & vegetables

High 1.979 (0.052) 2.006 (0.054) -0.027 (0.653) 

Low 1.957 (0.064) 1.875 (0.064) 0.082 (0.262) 

Impact: high – low -0.109 (0.249) 

Fruits & vegetables at home 

Frequently 1.906 (0.063) 1.999 (0.063) -0.092 (0.156) 

Infrequently 2.020 (0.060) 1.909 (0.056) 0.111 (0.092)* 

Impact: freq. – infreq. -0.203 (0.028)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcome: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' and 'refused' 

responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,253 interviews 

over 1,891 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 466 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.4c: Differences in Impacts of HIP on Barriers to Grocery Shopping (Composite 

Scale), by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.084 (0.046) 2.063 (0.048) 0.021 (0.730) 

<3 servings/day 2.048 (0.050) 1.962 (0.047) 0.086 (0.165) 

Impact: 3+ servings – <3 

servings -0.065 (0.448) 

Attitudes about food, fruits, & vegetables

High 2.079 (0.048) 2.065 (0.048) 0.014 (0.802) 

Low 2.052 (0.058) 1.945 (0.061) 0.108 (0.115) 

Impact: high – low -0.094 (0.283) 

Fruits & vegetables at home 

Frequently 2.024 (0.058) 2.063 (0.059) -0.039 (0.512) 

Infrequently 2.101 (0.056) 1.971 (0.052) 0.130 (0.034)** 

Impact: freq. – infreq. -0.169 (0.047)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Scale is the mean of variables with these categorical outcome: 1=never, 2=rarely, 3=sometimes, 4=most of the 

time, 5=always, 'don't know' and 'refused' responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,230 interviews 

over 1,880 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 467

Exhibit H7.5a: Differences in Impacts of HIP on How Often Fruits are Available at Home, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 4.345 (0.035) 4.248 (0.037) 0.097 (0.027)** 

<3 servings/day 4.256 (0.038) 4.133 (0.039) 0.123 (0.013)** 

Impact: 3+ servings – <3 

servings -0.026 (0.694) 

Attitudes about food, fruits, & vegetables

High 4.297 (0.039) 4.177 (0.042) 0.120 (0.004)*** 

Low 4.312 (0.048) 4.214 (0.044) 0.098 (0.064)* 

Impact: high – low 0.023 (0.735) 

Fruits & vegetables at home 

Frequently 4.231 (0.043) 4.171 (0.046) 0.061 (0.140) 

Infrequently 4.364 (0.044) 4.212 (0.044) 0.152 (0.003)*** 

Impact: freq. – infreq. -0.092 (0.163) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always; “don’t know” responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,314 interviews 

over 1,918 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 468 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.5b: Differences in Impacts of HIP on How Often Fruits are in Refrigerator or on 

Counter, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 4.062 (0.038) 4.025 (0.039) 0.037 (0.436) 

<3 servings/day 3.952 (0.043) 3.881 (0.042) 0.071 (0.185) 

Impact: 3+ servings – <3 

servings -0.033 (0.644) 

Attitudes about food, fruits, & vegetables

High 4.016 (0.041) 3.942 (0.042) 0.074 (0.122) 

Low 4.003 (0.049) 3.974 (0.047) 0.029 (0.596) 

Impact: high – low 0.044 (0.544) 

Fruits & vegetables at home 

Frequently 3.963 (0.047) 3.918 (0.049) 0.044 (0.366) 

Infrequently 4.051 (0.047) 3.988 (0.044) 0.063 (0.227) 

Impact: freq. – infreq. -0.019 (0.795) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,311 interviews 

over 1,918 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 469

Exhibit H7.5c: Differences in Impacts of HIP on How Often Vegetables are Available at Home, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 4.476 (0.034) 4.417 (0.033) 0.059 (0.164) 

<3 servings/day 4.526 (0.038) 4.419 (0.038) 0.107 (0.028)** 

Impact: 3+ servings – <3 

servings -0.048 (0.454) 

Attitudes about food, fruits, & vegetables

High 4.496 (0.034) 4.442 (0.036) 0.054 (0.174) 

Low 4.505 (0.045) 4.384 (0.042) 0.121 (0.022)** 

Impact: high – low -0.067 (0.318) 

Fruits & vegetables at home 

Frequently 4.419 (0.045) 4.393 (0.045) 0.026 (0.470) 

Infrequently 4.569 (0.049) 4.440 (0.040) 0.129 (0.011)** 

Impact: freq. – infreq. -0.103 (0.100) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' and “No 

refrigerator or freezer” responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,315 interviews 

over 1,918 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 470 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.5d: Differences in Impacts of HIP on How Often Ready-to-Eat Vegetables are 

Available on Counter or in Refrigerator, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.979 (0.047) 3.899 (0.048) 0.080 (0.181) 

<3 servings/day 3.809 (0.053) 3.751 (0.050) 0.058 (0.364) 

Impact: 3+ servings – <3 

servings 0.022 (0.800) 

Attitudes about food, fruits, & vegetables

High 3.953 (0.049) 3.830 (0.049) 0.123 (0.027)** 

Low 3.824 (0.061) 3.825 (0.064) -0.001 (0.990) 

Impact: high – low 0.124 (0.167) 

Fruits & vegetables at home 

Frequently 3.909 (0.064) 3.836 (0.063) 0.073 (0.202) 

Infrequently 3.888 (0.066) 3.820 (0.058) 0.067 (0.298) 

Impact: freq. – infreq. 0.005 (0.952) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,284 interviews 

over 1,909 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 471

Exhibit H7.5e: Differences in Impacts of HIP on Fruits and Vegetables Available at Home 

Scale, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 4.211 (0.029) 4.144 (0.030) 0.067 (0.067)* 

<3 servings/day 4.135 (0.033) 4.040 (0.032) 0.096 (0.017)** 

Impact: 3+ servings – <3 

servings -0.029 (0.594) 

Attitudes about food, fruits, & vegetables

High 4.186 (0.030) 4.092 (0.032) 0.094 (0.008)*** 

Low 4.161 (0.037) 4.096 (0.036) 0.065 (0.124) 

Impact: high – low 0.029 (0.597) 

Fruits & vegetables at home 

Frequently 4.133 (0.039) 4.082 (0.040) 0.051 (0.138) 

Infrequently 4.210 (0.041) 4.104 (0.035) 0.106 (0.010)*** 

Impact: freq. – infreq. -0.055 (0.309) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Scale is the mean of variables with these categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the 

time, 5=always, 'don't know' responses coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,318 interviews 

over 1,921 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 472 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.5f: Differences in Impacts of HIP on How Often Salty Snacks are Available at 

Home, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.245 (0.051) 3.225 (0.050) 0.019 (0.764) 

<3 servings/day 3.158 (0.052) 3.205 (0.049) -0.047 (0.452) 

Impact: 3+ servings – <3 

servings 0.066 (0.464) 

Attitudes about food, fruits, & vegetables

High 3.226 (0.052) 3.237 (0.050) -0.011 (0.848) 

Low 3.170 (0.062) 3.185 (0.062) -0.014 (0.836) 

Impact: high – low 0.003 (0.976) 

Fruits & vegetables at home 

Frequently 3.121 (0.064) 3.178 (0.060) -0.057 (0.389) 

Infrequently 3.271 (0.057) 3.248 (0.056) 0.022 (0.718) 

Impact: freq. – infreq. -0.080 (0.382) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) may differ from differences between reported regressionadjusted means for the treatment and control groups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,312 interviews 

over 1,922 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 473

Exhibit H7.5g: Differences in Impacts of HIP on How Often 1% Fat, Skim or Fat-Free Milk is 

Available at Home, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.362 (0.072) 3.348 (0.068) 0.014 (0.878) 

<3 servings/day 3.327 (0.073) 3.207 (0.073) 0.120 (0.178) 

Impact: 3+ servings – <3 

servings -0.106 (0.399) 

Attitudes about food, fruits, & vegetables

High 3.428 (0.072) 3.358 (0.070) 0.070 (0.401) 

Low 3.242 (0.088) 3.171 (0.087) 0.071 (0.463) 

Impact: high – low -0.001 (0.993) 

Fruits & vegetables at home 

Frequently 3.282 (0.085) 3.271 (0.081) 0.011 (0.905) 

Infrequently 3.401 (0.078) 3.288 (0.077) 0.113 (0.188) 

Impact: freq. – infreq. -0.102 (0.419) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,292 interviews 

over 1,910 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 474 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.5h: Differences in Impacts of HIP on How Often Soft Drinks are Available at Home, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener)

3+ servings/day 3.185 (0.060) 3.272 (0.058) -0.088 (0.246) 

<3 servings/day 3.110 (0.060) 3.201 (0.059) -0.091 (0.222) 

Impact: 3+ servings – <3 

servings 0.003 (0.978) 

Attitudes about food, fruits, & vegetables 

High 3.152 (0.062) 3.222 (0.060) -0.071 (0.318) 

Low 3.146 (0.071) 3.260 (0.072) -0.114 (0.158) 

Impact: high – low 0.043 (0.687) 

Fruits & vegetables at home 

Frequently 3.223 (0.075) 3.265 (0.075) -0.042 (0.583) 

Infrequently 3.087 (0.073) 3.214 (0.064) -0.126 (0.082)* 

Impact: freq. – infreq. 0.084 (0.424) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,312 interviews 

over 1,918 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 475

Exhibit H7.5i: Differences in Impacts of HIP on How Often All or Most of Family Sits Down 

and Eats Evening Meals Together at Home, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.861 (0.054) 3.914 (0.051) -0.053 (0.421) 

<3 servings/day 3.966 (0.054) 3.925 (0.051) 0.041 (0.540) 

Impact: 3+ servings – <3 

servings -0.094 (0.317) 

Attitudes about food, fruits, & vegetables

High 3.878 (0.055) 3.925 (0.051) -0.046 (0.459) 

Low 3.962 (0.063) 3.912 (0.065) 0.050 (0.488) 

Impact: high – low -0.097 (0.320) 

Fruits & vegetables at home 

Frequently 3.895 (0.060) 3.856 (0.059) 0.039 (0.564) 

Infrequently 3.936 (0.063) 3.984 (0.053) -0.049 (0.465) 

Impact: freq. – infreq. 0.087 (0.358) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' responses 

coded as missing. 

Asked only in households with more than one member. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=2,306 interviews 

over 1,379 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 476 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H7.5j: Differences in Impacts of HIP on How Often Evening Meals are Cooked at 

Home, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 4.381 (0.034) 4.434 (0.036) -0.053 (0.228) 

<3 servings/day 4.481 (0.035) 4.394 (0.036) 0.087 (0.056)* 

Impact: 3+ servings – <3 

servings -0.140 (0.027)** 

Attitudes about food, fruits, & vegetables

High 4.440 (0.037) 4.429 (0.037) 0.011 (0.798) 

Low 4.420 (0.042) 4.394 (0.044) 0.026 (0.577) 

Impact: high – low -0.015 (0.818) 

Fruits & vegetables at home 

Frequently 4.432 (0.041) 4.421 (0.040) 0.010 (0.806) 

Infrequently 4.431 (0.040) 4.409 (0.040) 0.023 (0.622) 

Impact: freq. – infreq. -0.012 (0.846) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Categorical outcomes: 1=never, 2=rarely, 3=sometimes, 4=most of the time, 5=always, 'don't know' responses 

coded as missing. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 sample (N=3,311 interviews 

over 1,918 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 477

Exhibit H8.1: Differences in Impacts of HIP on Consumption of Targeted Fruits (TF), CupEquivalents, by Demographic Subgroup, Preferred Restrictive Proxy Measure 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Respondent gender (N=3913 recalls from 2009 respondents)

Females 0.581 (0.041) 0.463 (0.035) 0.118 (0.008)*** 

Males 0.549 (0.066) 0.470 (0.057) 0.079 (0.262) 

Impact: females – males 0.039 (0.638) 

Respondent age group (N=3913 recalls from 2009 respondents)

16-40 years 0.180 (0.132) 0.108 (0.136) 0.072 (0.094)* 

41+ years 0.725 (0.062) 0.587 (0.059) 0.139 (0.023)** 

Impact: 16-40 years – 41+ 

years -0.067 (0.372) 

Respondent educational attainment (N=3892 recalls from 2000 respondents)a

Less than high school 

(including GED) 0.618 (0.047) 0.476 (0.037) 0.143 (0.014)** 

High school diploma 0.486 (0.061) 0.420 (0.046) 0.066 (0.389) 

More than high school 0.530 (0.044) 0.485 (0.043) 0.045 (0.457) 

P-value for difference c (0.481) 

Respondent race/ethnicity (N=3913 recalls from 2009 respondents)

Hispanic 0.589 (0.044) 0.482 (0.033) 0.107 (0.030)** 

Non-Hispanic white 0.479 (0.052) 0.418 (0.042) 0.061 (0.366) 

Non-Hispanic black 0.564 (0.073) 0.480 (0.083) 0.083 (0.415) 

Non-Hispanic other 0.993 (0.192) 0.577 (0.093) 0.416 (0.051)* 

P-value for difference c (0.466) 

Respondent disability status (N=3913 recalls from 2009 respondents)

Disabled 0.570 (0.044) 0.513 (0.023) 0.099 (0.058)* 

Non-disabled 0.621 (0.043) 0.513 (0.023) 0.108 (0.026)** 

Impact: disabled – non-disabled -0.005 (0.946) 

Primary shopper employment status (N=3751 recalls from 1916 respondents)b

Working full or part-time 0.542 (0.060) 0.495 (0.058) 0.047 (0.558) 

Not working 0.577 (0.035) 0.454 (0.027) 0.123 (0.006)*** 

Impact: working – not 

working -0.076 (0.416) 

Household composition (N=3913 recalls from 2009 respondents)

Children (and no elderly) in 

household 0.630 (0.053) 0.510 (0.044) 0.120 (0.018)** 

Other household 0.527 (0.045) 0.432 (0.035) 0.095 (0.077)* 

Impact: children in HH – other HH 0.026 (0.732) 

Household WIC status (N=3744 recalls from 1915 respondents)b

Participant 0.470 (0.057) 0.454 (0.053) 0.016 (0.818) 

Non-participant 0.589 (0.035) 0.462 (0.027) 0.128 (0.004)*** 

Impact: participant – nonparticipant -0.112 (0.164) 

Household SNAP benefit amount (N=3913 recalls from 2009 respondents)

$200 or less 0.583 (0.049) 0.471 (0.039) 0.112 (0.045)** 

Over $200 0.555 (0.050) 0.457 (0.042) 0.098 (0.042)** 

Impact: $200 or less – over 

$200 0.014 (0.850) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 478 ▌Appendix H: Analyses by Subgroup Abt Associates

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted fruit intake proxy measure includes intake of fruits acquired from the store. It excludes 100% juice, as 

well as mixed foods where the source of individual ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Sample size is smaller for this subgroup analysis due to missing data on educational attainment for some 

sampled respondents. 

b

 Sample size is smaller for this subgroup analysis because the primary shopper employment status and 

household WIC status items are included in the primary shopper survey, which was not completed in all 

households with a sampled respondent completing a dietary recall interview. 

c

 For demographic characteristics with more than two subgroup categories (respondent educational attainment 

and respondent race/ethnicity), p-value in parentheses represents significance level for joint test across all 

categories. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 479

Exhibit H8.2: Differences in Impacts of HIP on Consumption of Targeted Fruits (TF), CupEquivalents, by Baseline Fruit and Vegetable Behaviors and Preferences 

Subgroup, Preferred Restrictive Proxy Measure 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline fruit and vegetable intake (screener) (N=3913 recalls from 2009 respondents) 

3+ servings/day 0.607 (0.055) 0.467 (0.043) 0.140 (0.031)** 

<3 servings/day 0.534 (0.040) 0.464 (0.036) 0.071 (0.070)* 

Impact: 3+ servings – <3 

servings 0.069 (0.365) 

Baseline TFV intake (predicted) (N=3162 recalls from 1604 respondents) 

High (above median) 0.605 (0.061) 0.480 (0.042) 0.124 (0.081)* 

Low (below median) 0.510 (0.045) 0.425 (0.042) 0.085 (0.053)* 

Impact: high – low 0.039 (0.636) 

Baseline TFV spending (predicted) (N=3162 recalls from 1604 respondents) 

High (above median) 0.607 (0.058) 0.466 (0.042) 0.141 (0.029)** 

Low (below median) 0.513 (0.047) 0.440 (0.048) 0.073 (0.204) 

Impact: high – low 0.068 (0.438) 

Attitudes about food, fruits, and vegetables (N=3913 recalls from 2009 respondents) 

High (above median) 0.661 (0.045) 0.506 (0.036) 0.155 (0.002)*** 

Low (below median) 0.456 (0.052) 0.409 (0.042) 0.047 (0.384) 

Impact: high – low 0.108 (0.144) 

Barriers to eating fruits and vegetables (N=3913 recalls from 2009 respondents) 

High (above median) 0.590 (0.050) 0.459 (0.035) 0.131 (0.028)** 

Low (below median) 0.547 (0.034) 0.472 (0.034) 0.075 (0.121) 

Impact: high – low 0.055 (0.473) 

Barriers to grocery shopping (N=3913 recalls from 2009 respondents) 

High (above median) 0.598 (0.072) 0.458 (0.052) 0.141 (0.028)** 

Low (below median) 0.549 (0.049) 0.471 (0.044) 0.078 (0.088)* 

Impact: high – low 0.063 (0.430) 

Fruits and vegetables at home (N=3913 recalls from 2009 respondents) 

Frequently 0.522 (0.059) 0.494 (0.047) 0.028 (0.655) 

Infrequently 0.607 (0.047) 0.442 (0.043) 0.166 (<0.001) *** 

Impact: frequently – 

infrequently -0.138 (0.080)* 

Pre-HIP shopping patterns (N=2521 interviews from 1327 respondents)a

Shopped primarily at HIP 

participating retailers 0.596 (0.044) 0.416 (0.033) 0.180 (0.001)*** 

Shopped primarily at non-HIP 

participating retailers 0.617 (0.063) 0.454 (0.049) 0.163 (0.042)** 

Impact: HIP shoppers – nonHIP-shoppers 0.017 (0.857) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted fruit intake proxy measure includes intake of fruits acquired from the store. It excludes 100% juice, as 

well as mixed foods where the source of individual ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 480 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.3: Differences in Impacts of HIP on Consumption of Targeted Vegetables (TV), 

Cup-Equivalents, by Demographic Subgroup, Preferred Restrictive Proxy 

Measure 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Respondent gender (N=3913 recalls from 2009 respondents) 

Females 0.589 (0.034) 0.440 (0.027) 0.149 (<0.001)*** 

Males 0.555 (0.057) 0.456 (0.044) 0.098 (0.150) 

Impact: females – males 0.051 (0.521) 

Respondent age group (N=3913 recalls from 2009 respondents) 

16-40 years 0.186 (0.111) 0.065 (0.120) 0.121 (0.010)*** 

41+ years 0.737 (0.048) 0.593 (0.046) 0.144 (0.001)*** 

Impact: 16-40 years – 41+ 

years -0.023 (0.720) 

Respondent educational attainment (N=3892 recalls from 2000 respondents)a

Less than high school 

(including GED) 0.568 (0.034) 0.397 (0.026) 0.171 (<0.001)*** 

High school diploma 0.548 (0.054) 0.508 (0.050) 0.040 (0.587) 

More than high school 0.625 (0.053) 0.491 (0.052) 0.134 (0.058)* 

P-value for differencec (0.302) 

Respondent race/ethnicity (N=3913 recalls from 2009 respondents) 

Hispanic 0.580 (0.042) 0.472 (0.035) 0.108 (0.030)** 

Non-Hispanic white 0.534 (0.043) 0.364 (0.035) 0.170 (0.001)*** 

Non-Hispanic black 0.605 (0.074) 0.468 (0.048) 0.136 (0.109) 

Non-Hispanic other 0.730 (0.128) 0.664 (0.172) 0.067 (0.754) 

P-value for differencec (0.840) 

Respondent disability status (N=3913 recalls from 2009 respondents) 

Disabled 0.571 (0.037) 0.469 (0.033) 0.102 (0.027)** 

Non-disabled 0.586 (0.038) 0.421 (0.031) 0.165 (<0.001) *** 

Impact: disabled – nondisabled -0.063 (0.343) 

Primary shopper employment status (N=3751 recalls from 1916 respondents)b

Working full or part-time 0.537 (0.056) 0.430 (0.035) 0.107 (0.099)* 

Not working 0.589 (0.029) 0.440 (0.022) 0.149 (<0.001)*** 

Impact: working – not working -0.042 (0.573) 

Household composition (N=3913 recalls from 2009 respondents) 

Children (and no elderly) in 

household 0.682 (0.045) 0.520 (0.043) 0.162 (0.001)*** 

Other household 0.502 (0.036) 0.390 (0.027) 0.112 (0.010)*** 

Impact: children in HH – other 

HH 0.050 (0.448) 

Household WIC status (N=3744 recalls from 1915 respondents)b

Participant 0.570 (0.062) 0.405 (0.044) 0.166 (0.016)** 

Non-participant 0.582 (0.029) 0.449 (0.022) 0.133 (<0.001)*** 

Impact: participant – nonparticipant 0.033 (0.677) 

Household SNAP benefit amount (N=3913 recalls from 2009 respondents) 

$200 or less 0.552 (0.039) 0.437 (0.029) 0.115 (0.009)*** 

Over $200 0.611 (0.043) 0.455 (0.036) 0.156 (0.001)*** 

Impact: $200 or less – over 

$200 -0.041 (0.531) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 481

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as mixed foods where the source of individual ingredients was not identified 

by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

Sample size is smaller for this subgroup analysis due to missing data on educational attainment for some 

sampled respondents. 

b

Sample size is smaller for this subgroup analysis because the primary shopper employment status and 

household WIC status items are included in the primary shopper survey, which was not completed in all 

households with a sampled respondent completing a dietary recall interview. 

c

For demographic characteristics with more than two subgroup categories (respondent educational attainment 

and respondent race/ethnicity), p-value in parentheses represents significance level for joint test across all 

categories. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 482 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.4: Differences in Impacts of HIP on Consumption of Targeted Vegetables (TV), 

Cup-Equivalents, by Baseline Fruit and Vegetable Behaviors and Preferences 

Subgroup, Preferred Restrictive Proxy Measure 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline fruit and vegetable intake (screener) (N=3913 recalls from 2009 respondents) 

3+ servings/day 0.592 (0.040) 0.442 (0.038) 0.149 (0.003)*** 

<3 servings/day 0.564 (0.039) 0.448 (0.030) 0.116 (0.004)*** 

Impact: 3+ servings – <3 

servings 0.033 (0.600) 

Baseline TFV intake (predicted) (N=3162 recalls from 1604 respondents) 

High (above median) 0.642 (0.051) 0.429 (0.032) 0.213 (<0.001) *** 

Low (below median) 0.566 (0.041) 0.452 (0.037) 0.114 (0.008)*** 

Impact: high – low 0.099 (0.170) 

Baseline TFV spending (predicted) (N=3162 recalls from 1604 respondents) 

High (above median) 0.647 (0.046) 0.412 (0.029) 0.235 (<0.001)*** 

Low (below median) 0.565 (0.041) 0.470 (0.035) 0.095 (0.060)* 

Impact: high – low 0.140 (0.050)* 

Attitudes about food, fruits, and vegetables (N=3913 recalls from 2009 respondents) 

High (above median) 0.644 (0.042) 0.439 (0.032) 0.205 (<0.001)*** 

Low (below median) 0.494 (0.040) 0.453 (0.038) 0.041 (0.283) 

Impact: high – low 0.164 (0.010)*** 

Barriers to eating fruits and vegetables (N=3913 recalls from 2009 respondents) 

High (above median) 0.565 (0.040) 0.395 (0.025) 0.169 (<0.001)*** 

Low (below median) 0.585 (0.034) 0.496 (0.038) 0.089 (0.076)* 

Impact: high – low 0.080 (0.243) 

Barriers to grocery shopping (N=3913 recalls from 2009 respondents) 

High (above median) 0.552 (0.052) 0.428 (0.044) 0.124 (0.013)** 

Low (below median) 0.599 (0.045) 0.460 (0.038) 0.139 (0.001)*** 

Impact: high – low -0.015 (0.816) 

Fruits and vegetables at home (N=3913 recalls from 2009 respondents) 

Frequently 0.600 (0.047) 0.446 (0.041) 0.154 (0.003)*** 

Infrequently 0.561 (0.038) 0.445 (0.033) 0.116 (0.005)*** 

Impact: frequently – 

infrequently 0.038 (0.562) 

Pre-HIP shopping patterns (N=2521 interviews from 1327 respondents)a

Shopped primarily at HIP 

participating retailers 0.591 (0.040) 0.469 (0.041) 0.122 (0.026)** 

Shopped primarily at non-HIP 

participating retailers 0.641 (0.062) 0.432 (0.034) 0.209 (0.003)*** 

Impact: HIP shoppers – nonHIP-shoppers -0.087 (0.341) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as mixed foods where the source of individual ingredients was not identified 

by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 483

Exhibit H8.5: Differences in Impacts of HIP on Consumption of Total Fruits and Vegetables, 

Cup-Equivalents, by Demographic Subgroup 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Respondent gender (N=3913 recalls from 2009 respondents) 

Females 2.568 (0.084) 2.211 (0.074) 0.357 (<0.001)*** 

Males 2.708 (0.144) 2.458 (0.131) 0.250 (0.134) 

Impact: females – males 0.107 (0.578) 

Respondent age group (N=3913 recalls from 2009 respondents) 

16-40 years 1.811 (0.309) 1.499 (0.330) 0.312 (0.006)*** 

41+ years 2.902 (0.123) 2.569 (0.132) 0.333 (0.004)*** 

Impact: 16-40 years – 41+ 

years -0.021 (0.898) 

Respondent educational attainment (N=3892 recalls from 2000 respondents)a

Less than high school 

(including GED) 2.690 (0.087) 2.171 (0.077) 0.519 (<0.001)*** 

High school diploma 2.447 (0.128) 2.337 (0.110) 0.110 (0.512) 

More than high school 2.595 (0.110) 2.503 (0.118) 0.092 (0.559) 

P-value for differencec (0.045)** 

Respondent race/ethnicity (N=3913 recalls from 2009 respondents) 

Hispanic 2.623 (0.098) 2.202 (0.080) 0.421 (<0.001)*** 

Non-Hispanic white 2.540 (0.102) 2.225 (0.096) 0.315 (0.016)** 

Non-Hispanic black 2.594 (0.176) 2.589 (0.201) 0.004 (0.986) 

Non-Hispanic other 3.068 (0.306) 2.647 (0.305) 0.421 (0.329) 

P-value for differencec (0.470) 

Respondent disability status (N=3913 recalls from 2009 respondents) 

Disabled 2.619 (0.089) 2.379 (0.086) 0.240 (0.042)** 

Non-disabled 2.615 (0.091) 2.207 (0.073) 0.408 (<0.001)*** 

Impact: disabled – nondisabled -0.168 (0.306) 

Primary shopper employment status (N=3751 recalls from 1916 respondents)b

Working full or part-time 2.531 (0.118) 2.280 (0.108) 0.251 (0.106) 

Not working 2.643 (0.072) 2.300 (0.063) 0.343 (<0.001)*** 

Impact: working – not working -0.092 (0.611) 

Household composition (N=3913 recalls from 2009 respondents) 

Children (and no elderly) in 

household 2.734 (0.122) 2.355 (0.109) 0.379 (0.002)*** 

Other household 2.530 (0.094) 2.249 (0.087) 0.282 (0.010)** 

Impact: children in HH – other 

HH 0.097 (0.553) 

Household WIC status (N=3744 recalls from 1915 respondents)b

Participant 2.606 (0.148) 2.239 (0.122) 0.367 (0.032)** 

Non-participant 2.621 (0.070) 2.301 (0.062) 0.321 (0.001)*** 

Impact: participant – nonparticipant 0.046 (0.812) 

Household SNAP benefit amount (N=3913 recalls from 2009 respondents) 

$200 or less 2.625 (0.102) 2.277 (0.089) 0.347 (0.002)*** 

Over $200 2.605 (0.110) 2.314 (0.101) 0.291 (0.010)** 

Impact: $200 or less – over 

$200 0.056 (0.726) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 484 ▌Appendix H: Analyses by Subgroup Abt Associates

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

Sample size is smaller for this subgroup analysis due to missing data on educational attainment for some 

sampled respondents. 

b

Sample size is smaller for this subgroup analysis because the primary shopper employment status and 

household WIC status items are included in the primary shopper survey, which was not completed in all 

households with a sampled respondent completing a dietary recall interview. 

c

For demographic characteristics with more than two subgroup categories (respondent educational attainment 

and respondent race/ethnicity), p-value in parentheses represents significance level for joint test across all 

categories. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 485

Exhibit H8.6: Differences in Impacts of HIP on Consumption of Total Fruits and Vegetables, 

Cup-Equivalents, by Baseline Fruit and Vegetable Behaviors and Preferences 

Subgroup 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted 

control mean 

(SE) 

Treatmentcontrol impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline fruit and vegetable intake (screener) (N=3913 recalls from 2009 respondents) 

3+ servings/day 2.733 (0.104) 2.269 (0.089) 0.464 (<0.001)*** 

<3 servings/day 2.501 (0.089) 2.319 (0.084) 0.182 (0.064)* 

Impact: 3+ servings – <3 

servings 0.282 (0.083)* 

Baseline TFV intake (predicted) (N=3162 recalls from 1604 respondents) 

High (above median) 2.717 (0.112) 2.357 (0.095) 0.360 (0.007)*** 

Low (below median) 2.529 (0.111) 2.226 (0.102) 0.304 (0.010)*** 

Impact: high – low 0.056 (0.750) 

Baseline TFV spending (predicted) (N=3162 recalls from 1604 respondents) 

High (above median) 2.691 (0.105) 2.343 (0.085) 0.347 (0.006)*** 

Low (below median) 2.563 (0.105) 2.240 (0.098) 0.323 (0.011)** 

Impact: high – low 0.025 (0.892) 

Attitudes about food, fruits, and vegetables (N=3913 recalls from 2009 respondents) 

High (above median) 2.844 (0.095) 2.419 (0.091) 0.425 (<0.001) *** 

Low (below median) 2.327 (0.108) 2.123 (0.096) 0.204 (0.067)* 

Impact: high – low 0.221 (0.168) 

Barriers to eating fruits and vegetables (N=3913 recalls from 2009 respondents) 

High (above median) 2.625 (0.094) 2.205 (0.083) 0.419 (0.001)*** 

Low (below median) 2.597 (0.081) 2.383 (0.081) 0.214 (0.059)* 

Impact: high – low 0.205 (0.223) 

Barriers to grocery shopping (N=3913 recalls from 2009 respondents) 

High (above median) 2.656 (0.137) 2.373 (0.118) 0.282 (0.027)** 

Low (below median) 2.584 (0.110) 2.225 (0.098) 0.359 (0.001)*** 

Impact: high – low -0.077 (0.642) 

Fruits and vegetables at home (N=3913 recalls from 2009 respondents) 

Frequently 2.606 (0.114) 2.384 (0.106) 0.222 (0.074)* 

Infrequently 2.621 (0.098) 2.221 (0.098) 0.400 (<0.001)*** 

Impact: frequently – 

infrequently -0.178 (0.281) 

Pre-HIP shopping patterns (N=2521 interviews from 1327 respondents)a

Shopped primarily at HIP 

participating retailers 2.664 (0.095) 2.204 (0.081) 0.460 (<0.001)*** 

Shopped primarily at nonHIP participating retailers 2.722 (0.130) 2.321 (0.112) 0.401 (0.019)** 

Impact: HIP shoppers – nonHIP-shoppers 0.058 (0.782) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

 Smaller N for this subgroup because pre-HIP status information was not available for all respondents. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 486 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.7a: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables (TFV) from Mixed Foods, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.294 (0.019) 0.303 (0.020) -0.008 (0.733) 

<3 servings/day 0.319 (0.021) 0.306 (0.023) 0.013 (0.657) 

Impact: 3+ servings – <3 

servings -0.021 (0.566) 

Attitudes about food, fruits, & vegetables

High 0.295 (0.021) 0.287 (0.020) 0.008 (0.740) 

Low 0.321 (0.025) 0.328 (0.026) -0.007 (0.797) 

Impact: high – low 0.016 (0.671) 

Fruits & vegetables at home 

Frequently 0.337 (0.025) 0.320 (0.024) 0.017 (0.550) 

Infrequently 0.283 (0.023) 0.292 (0.020) -0.009 (0.749) 

Impact: freq. – infreq. 0.025 (0.517) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted fruit and vegetable intake (TFV) includes intake of fruits and vegetables acquired from the store. It 

excludes white potatoes, legumes, and 100% juice. Mixed foods where the source of individual ingredients was 

not identified by the respondent are excluded from our preferred restricted TFV measure but included in our 

alternative inclusive TFV measure. The outcome described in this table is fruit and vegetable intake from mixed 

foods, excluding white potatoes, legumes, and 100% juice. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 487

Exhibit H8.7b: Differences in Impacts of HIP on Consumption of Targeted Fruits and 

Vegetables (TFV), Alternative Inclusive Proxy Measure, Cup-Equivalents, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 1.493 (0.079) 1.212 (0.067) 0.281 (0.003)*** 

<3 servings/day 1.417 (0.067) 1.218 (0.059) 0.200 (0.004)*** 

Impact: 3+ servings – <3 

servings 0.082 (0.487) 

Attitudes about food, fruits, & vegetables

High 1.600 (0.072) 1.232 (0.059) 0.368 (<0.001) *** 

Low 1.271 (0.078) 1.191 (0.068) 0.080 (0.308) 

Impact: high – low 0.287 (0.013)** 

Fruits & vegetables at home 

Frequently 1.458 (0.089) 1.260 (0.072) 0.198 (0.037)** 

Infrequently 1.451 (0.070) 1.178 (0.067) 0.273 (<0.001) *** 

Impact: freq. – infreq. -0.075 (0.539) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits and vegetables acquired from 

the store, excluding white potatoes, legumes, and 100% juice. The preferred restrictive proxy measure 

additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was 

not identified by the respondent, while the alternative inclusive proxy measure includes fruit and vegetable intake 

from all mixed foods. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 488 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.7c: Differences in Impacts of HIP on Consumption of 100% Juice, Cup-Equivalents, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.610 (0.049) 0.447 (0.034) 0.163 (0.005)*** 

<3 servings/day 0.487 (0.038) 0.460 (0.035) 0.027 (0.532) 

Impact: 3+ servings – <3 

servings 0.136 (0.066)* 

Attitudes about food, fruits, & vegetables

High 0.573 (0.042) 0.472 (0.041) 0.101 (0.028)** 

Low 0.517 (0.057) 0.428 (0.049) 0.090 (0.137) 

Impact: high – low 0.012 (0.880) 

Fruits & vegetables at home 

Frequently 0.510 (0.047) 0.455 (0.046) 0.055 (0.284) 

Infrequently 0.578 (0.048) 0.452 (0.039) 0.126 (0.014)** 

Impact: freq. – infreq. -0.071 (0.327) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 489

Exhibit H8.7d: Differences in Impacts of HIP on Consumption of White Potatoes, CupEquivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.348 (0.028) 0.326 (0.022) 0.021 (0.540) 

<3 servings/day 0.326 (0.025) 0.393 (0.029) -0.067 (0.047)** 

Impact: 3+ servings – <3 

servings 0.089 (0.071)* 

Attitudes about food, fruits, & vegetables

High 0.356 (0.028) 0.406 (0.029) -0.049 (0.153) 

Low 0.311 (0.029) 0.296 (0.027) 0.014 (0.649) 

Impact: high – low -0.064 (0.169) 

Fruits & vegetables at home 

Frequently 0.410 (0.033) 0.397 (0.035) 0.013 (0.713) 

Infrequently 0.278 (0.029) 0.329 (0.025) -0.051 (0.099)* 

Impact: freq. – infreq. 0.064 (0.162) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 490 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.7e: Differences in Impacts of HIP on Consumption of Legumes, Cup-Equivalents, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.109 (0.011) 0.113 (0.011) -0.004 (0.790) 

<3 servings/day 0.102 (0.011) 0.115 (0.010) -0.013 (0.324) 

Impact: 3+ servings – <3 

servings 0.009 (0.643) 

Attitudes about food, fruits, & vegetables

High 0.111 (0.011) 0.118 (0.011) -0.007 (0.586) 

Low 0.099 (0.012) 0.108 (0.013) -0.009 (0.496) 

Impact: high – low 0.002 (0.905) 

Fruits & vegetables at home 

Frequently 0.090 (0.013) 0.095 (0.012) -0.005 (0.669) 

Infrequently 0.119 (0.012) 0.129 (0.012) -0.010 (0.445) 

Impact: freq. – infreq. 0.005 (0.788) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 491

Exhibit H8.7f: Differences in Impacts of HIP on Consumption of Other Fruits and Vegetables 

Acquired Outside Stores, Cup-Equivalents, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.172 (0.026) 0.171 (0.024) 0.001 (0.972) 

<3 servings/day 0.168 (0.025) 0.133 (0.023) 0.035 (0.222) 

Impact: 3+ servings – <3 

servings -0.034 (0.453) 

Attitudes about food, fruits, & vegetables

High 0.203 (0.027) 0.191 (0.028) 0.013 (0.693) 

Low 0.129 (0.030) 0.100 (0.027) 0.029 (0.326) 

Impact: high – low -0.017 (0.697) 

Fruits & vegetables at home 

Frequently 0.138 (0.027) 

0.177 (0.029) -0.039 (0.265) 

Infrequently 0.195 (0.028) 0.133 (0.024) 0.062 (0.033)** 

Impact: freq. – infreq. -0.101 (0.024)** 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 492 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.8a: Differences in Impacts of HIP on Consumption of Targeted Fruit from Citrus 

Fruits, Melons, and Berries, Preferred Restrictive Proxy Measure, CupEquivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.130 (0.018) 0.125 (0.024) 0.004 (0.872) 

<3 servings/day 0.132 (0.017) 0.112 (0.017) 0.020 (0.252) 

Impact: 3+ servings – <3 

servings -0.016 (0.623) 

Attitudes about food, fruits, & vegetables

High 0.140 (0.019) 0.115 (0.015) 0.025 (0.270) 

Low 0.121 (0.021) 0.124 (0.023) -0.004 (0.859) 

Impact: high – low 0.029 (0.342) 

Fruits & vegetables at home 

Frequently 0.111 (0.023) 0.118 (0.019) -0.007 (0.795) 

Infrequently 0.147 (0.017) 0.120 (0.019) 0.027 (0.101) 

Impact: freq. – infreq. -0.035 (0.268) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted fruit intake proxy measure includes intake of fruits acquired from the store. It excludes 100% juice, as 

well as fruit intake from mixed foods where the source of individual ingredients was not identified by the 

respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 493

Exhibit H8.8b: Differences in Impacts of HIP on Consumption of Targeted Fruit from Other 

Fruits, Preferred Restrictive Proxy Measure, Cup-Equivalents, by Baseline 

Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.477 (0.047) 0.341 (0.032) 0.136 (0.011)** 

<3 servings/day 0.402 (0.034) 0.351 (0.032) 0.051 (0.129) 

Impact: 3+ servings – <3 

servings 0.085 (0.178) 

Attitudes about food, fruits, & vegetables

High 0.521 (0.038) 0.391 (0.031) 0.130 (0.003)*** 

Low 0.335 (0.042) 0.285 (0.033) 0.051 (0.244) 

Impact: high – low 0.079 (0.195) 

Fruits & vegetables at home 

Frequently 0.411 (0.048) 0.376 (0.041) 0.035 (0.477) 

Infrequently 0.460 (0.042) 0.322 (0.034) 0.138 (0.001)*** 

Impact: freq. – infreq. -0.103 (0.115) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted fruit intake proxy measure includes intake of fruits acquired from the store. It excludes 100% juice, as 

well as fruit intake mixed foods where the source of individual ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 494 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.8c: Differences in Impacts of HIP on Consumption of Targeted Vegetables from 

Dark Green Vegetables, Preferred Restrictive Proxy Measure, Cup-Equivalents, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.078 (0.014) 0.037 (0.010) 0.041 (0.003)*** 

<3 servings/day 0.073 (0.013) 0.050 (0.011) 0.023 (0.025)** 

Impact: 3+ servings – <3 

servings 0.018 (0.296) 

Attitudes about food, fruits, & vegetables

High 0.080 (0.011) 0.033 (0.008) 0.046 (<0.001) *** 

Low 0.071 (0.011) 0.057 (0.012) 0.013 (0.284) 

Impact: high – low 0.033 (0.068)* 

Fruits & vegetables at home 

Frequently 0.072 (0.012) 0.049 (0.011) 0.023 (0.123) 

Infrequently 0.078 (0.011) 0.039 (0.007) 0.039 (<0.001) *** 

Impact: freq. – infreq. -0.015 (0.430) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as vegetable intake from mixed foods where the source of individual 

ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 495

Exhibit H8.8d: Differences in Impacts of HIP on Consumption of Targeted Vegetables from 

Red and Orange Vegetables, Preferred Restrictive Proxy Measure, CupEquivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.148 (0.016) 0.100 (0.012) 0.049 (0.011)** 

<3 servings/day 0.124 (0.013) 0.110 (0.012) 0.014 (0.313) 

Impact: 3+ servings – <3 

servings 0.035 (0.141) 

Attitudes about food, fruits, & vegetables

High 0.160 (0.017) 0.101 (0.011) 0.059 (0.002)*** 

Low 0.105 (0.011) 0.109 (0.014) -0.004 (0.715) 

Impact: high – low 0.064 (0.004)*** 

Fruits & vegetables at home 

Frequently 0.146 (0.018) 0.100 (0.014) 0.046 (0.020)** 

Infrequently 0.128 (0.013) 0.108 (0.013) 0.020 (0.175) 

Impact: freq. – infreq. 0.027 (0.280) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as vegetable intake from mixed foods where the source of individual 

ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 496 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.8e: Differences in Impacts of HIP on Consumption of Targeted Vegetables from 

Tomatoes, Preferred Restrictive Proxy Measure, Cup-Equivalents, by Baseline 

Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.096 (0.010) 0.076 (0.010) 0.020 (0.127) 

<3 servings/day 0.077 (0.009) 0.063 (0.008) 0.014 (0.141) 

Impact: 3+ servings – <3 

servings 0.006 (0.720) 

Attitudes about food, fruits, & vegetables

High 0.098 (0.010) 0.068 (0.008) 0.030 (0.014)** 

Low 0.071 (0.009) 0.071 (0.010) 0.001 (0.956) 

Impact: high – low 0.029 (0.058)* 

Fruits & vegetables at home 

Frequently 0.108 (0.013) 0.083 (0.012) 0.025 (0.068)* 

Infrequently 0.070 (0.009) 0.059 (0.009) 0.011 (0.267) 

Impact: freq. – infreq. 0.014 (0.400) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as vegetable intake from mixed foods where the source of individual 

ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 497

Exhibit H8.8f: Differences in Impacts of HIP on Consumption of Targeted Vegetables from 

Other Red and Orange Vegetables, Preferred Restrictive Proxy Measure, CupEquivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.052 (0.012) 0.023 (0.006) 0.029 (0.023)** 

<3 servings/day 0.047 (0.007) 0.047 (0.008) >-0.001 (0.998) 

Impact: 3+ servings – <3 

servings 0.029 (0.061)* 

Attitudes about food, fruits, & vegetables

High 0.062 (0.012) 0.033 (0.006) 0.029 (0.018)** 

Low 0.033 (0.007) 0.038 (0.008) -0.005 (0.502) 

Impact: high – low 0.034 (0.015)** 

Fruits & vegetables at home 

Frequently 0.038 (0.012) 0.017 (0.006) 0.021 (0.087)* 

Infrequently 0.058 (0.009) 0.050 (0.008) 0.009 (0.356) 

Impact: freq. – infreq. 0.012 (0.425) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as vegetable intake from mixed foods where the source of individual 

ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 498 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.8g: Differences in Impacts of HIP on Consumption of Targeted Vegetables from 

Starchy Vegetables, Preferred Restrictive Proxy Measure, Cup-Equivalents, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.092 (0.014) 0.079 (0.010) 0.014 (0.401) 

<3 servings/day 0.092 (0.012) 0.090 (0.011) 0.002 (0.891) 

Impact: 3+ servings – <3 

servings 0.012 (0.587) 

Attitudes about food, fruits, & vegetables

High 0.105 (0.014) 0.088 (0.012) 0.017 (0.289) 

Low 0.075 (0.012) 0.079 (0.014) -0.004 (0.784) 

Impact: high – low 0.021 (0.332) 

Fruits & vegetables at home 

Frequently 0.091 (0.016) 0.077 (0.014) 0.014 (0.423) 

Infrequently 0.093 (0.013) 0.090 (0.012) 0.003 (0.826) 

Impact: freq. – infreq. 0.011 (0.631) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as vegetable intake from mixed foods where the source of individual 

ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 499

Exhibit H8.8h: Differences in Impacts of HIP on Consumption of Targeted Vegetables from 

Other Vegetables, Preferred Restrictive Proxy Measure, Cup-Equivalents, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.273 (0.022) 0.227 (0.024) 0.046 (0.124) 

<3 servings/day 0.275 (0.024) 0.199 (0.017) 0.077 (0.003)*** 

Impact: 3+ servings – <3 

servings -0.031 (0.431) 

Attitudes about food, fruits, & vegetables

High 0.299 (0.025) 0.217 (0.020) 0.082 (0.008)*** 

Low 0.244 (0.025) 0.208 (0.022) 0.036 (0.119) 

Impact: high – low 0.046 (0.236) 

Fruits & vegetables at home 

Frequently 0.291 (0.029) 0.220 (0.025) 0.071 (0.021)** 

Infrequently 0.262 (0.023) 0.207 (0.020) 0.055 (0.032)** 

Impact: freq. – infreq. 0.016 (0.684) 

Two-sided test: *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Targeted vegetable intake proxy measure includes intake of vegetables acquired from the store. It excludes 

white potatoes and legumes, as well as vegetable intake from mixed foods where the source of individual 

ingredients was not identified by the respondent. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 respondents, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 500 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.9a: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, 100% Juice, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 1.142 (0.079) 1.025 (0.060) 0.117 (0.203) 

<3 servings/day 0.872 (0.071) 0.829 (0.070) 0.043 (0.501) 

Impact: 3+ servings – <3 

servings 0.074 (0.510) 

Attitudes about food, fruits, & vegetables

High 1.060 (0.062) 0.998 (0.069) 0.062 (0.414) 

Low 0.947 (0.080) 0.834 (0.084) 0.112 (0.182) 

Impact: high – low -0.050 (0.662) 

Fruits & vegetables at home 

Frequently 0.984 (0.088) 0.864 (0.082) 0.120 (0.180) 

Infrequently 1.033 (0.069) 0.981 (0.072) 0.052 (0.464) 

Impact: freq. – infreq. 0.068 (0.558) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3350 interviews from 1953 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 501

Exhibit H8.9b: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Fruit, Cup-Equivalents, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.734 (0.051) 0.632 (0.034) 0.103 (0.068)* 

<3 servings/day 0.617 (0.039) 0.584 (0.037) 0.033 (0.380) 

Impact: 3+ servings – <3 

servings 0.069 (0.317) 

Attitudes about food, fruits, & vegetables

High 0.724 (0.044) 0.620 (0.038) 0.104 (0.036)** 

Low 0.617 (0.044) 0.592 (0.039) 0.025 (0.568) 

Impact: high – low 0.079 (0.242) 

Fruits & vegetables at home 

Frequently 0.684 (0.049) 0.680 (0.042) 0.004 (0.931) 

Infrequently 0.668 (0.045) 0.550 (0.032) 0.118 (0.011)** 

Impact: freq. – infreq. -0.113 (0.098)* 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses, (unweighted N=3355 interviews from 1956 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 502 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.9c: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Salad, Cup-Equivalents, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.354 (0.019) 0.311 (0.017) 0.042 (0.070)* 

<3 servings/day 0.256 (0.014) 0.249 (0.016) 0.006 (0.707) 

Impact: 3+ servings – <3 

servings 0.036 (0.219) 

Attitudes about food, fruits, & vegetables

High 0.328 (0.017) 0.290 (0.018) 0.037 (0.085)* 

Low 0.278 (0.017) 0.268 (0.017) 0.010 (0.569) 

Impact: high – low 0.027 (0.340) 

Fruits & vegetables at home 

Frequently 0.324 (0.022) 0.261 (0.020) 0.063 (0.004)*** 

Infrequently 0.293 (0.018) 0.297 (0.021) -0.005 (0.811) 

Impact: freq. – infreq. 0.068 (0.023)** 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3371 interviews from 1954 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 503

Exhibit H8.9d: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Fried Potatoes, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.056 (0.006) 0.078 (0.014) -0.022 (0.162) 

<3 servings/day 0.055 (0.006) 0.066 (0.006) -0.012 (0.067)* 

Impact: 3+ servings – <3 

servings -0.010 (0.546) 

Attitudes about food, fruits, & vegetables

High 0.051 (0.006) 0.084 (0.012) -0.033 (0.022)** 

Low 0.061 (0.007) 0.056 (0.006) 0.005 (0.487) 

Impact: high – low -0.038 (0.021)** 

Fruits & vegetables at home 

Frequently 0.058 (0.007) 0.061 (0.007) -0.004 (0.588) 

Infrequently 0.054 (0.006) 0.081 (0.015) -0.027 (0.073)* 

Impact: freq. – infreq. 0.023 (0.174) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3389 interviews from 1961 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 504 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.9e: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Other Potatoes, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.235 (0.018) 0.212 (0.014) 0.022 (0.326) 

<3 servings/day 0.194 (0.013) 0.199 (0.013) -0.004 (0.738) 

Impact: 3+ servings – <3 

servings 0.026 (0.304) 

Attitudes about food, fruits, & vegetables

High 0.199 (0.014) 0.207 (0.013) -0.007 (0.660) 

Low 0.234 (0.018) 0.204 (0.018) 0.031 (0.122) 

Impact: high – low -0.038 (0.142) 

Fruits & vegetables at home 

Frequently 0.221 (0.018) 0.225 (0.019) -0.004 (0.840) 

Infrequently 0.209 (0.017) 0.190 (0.013) 0.020 (0.256) 

Impact: freq. – infreq. -0.024 (0.373) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3379 interviews from 1961 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 505

Exhibit H8.9f: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Beans, Cup-Equivalents, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.217 (0.019) 0.208 (0.014) 0.009 (0.678) 

<3 servings/day 0.165 (0.012) 0.166 (0.011) -0.001 (0.945) 

Impact: 3+ servings – <3 

servings 0.010 (0.686) 

Attitudes about food, fruits, & vegetables

High 0.180 (0.014) 0.182 (0.012) -0.002 (0.901) 

Low 0.207 (0.019) 0.194 (0.014) 0.012 (0.547) 

Impact: high – low -0.014 (0.580) 

Fruits & vegetables at home 

Frequently 0.183 (0.017) 0.178 (0.016) 0.004 (0.782) 

Infrequently 0.199 (0.018) 0.194 (0.014) 0.004 (0.811) 

Impact: freq. – infreq. >-0.001 (0.999) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3376 interviews from 1958 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 506 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.9g: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Other Vegetables, Cup-Equivalents, by Baseline Intake 

and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.552 (0.032) 0.501 (0.028) 0.051 (0.204) 

<3 servings/day 0.458 (0.028) 0.450 (0.027) 0.009 (0.765) 

Impact: 3+ servings – <3 

servings 0.042 (0.397) 

Attitudes about food, fruits, & vegetables

High 0.550 (0.031) 0.499 (0.028) 0.051 (0.145) 

Low 0.450 (0.032) 0.445 (0.031) 0.005 (0.879) 

Impact: high – low 0.047 (0.334) 

Fruits & vegetables at home 

Frequently 0.535 (0.038) 0.508 (0.036) 0.027 (0.526) 

Infrequently 0.483 (0.030) 0.450 (0.027) 0.032 (0.262) 

Impact: freq. – infreq. -0.005 (0.917) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3350 interviews from 1948 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 507

Exhibit H8.9h: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Tomatoes, Cup-Equivalents, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.113 (0.010) 0.108 (0.007) 0.006 (0.645) 

<3 servings/day 0.090 (0.007) 0.106 (0.009) -0.016 (0.074)* 

Impact: 3+ servings – <3 

servings 0.021 (0.159) 

Attitudes about food, fruits, & vegetables

High 0.100 (0.009) 0.111 (0.008) -0.010 (0.306) 

Low 0.103 (0.009) 0.102 (0.008) 0.002 (0.871) 

Impact: high – low -0.012 (0.404) 

Fruits & vegetables at home 

Frequently 0.105 (0.011) 0.107 (0.010) -0.002 (0.881) 

Infrequently 0.099 (0.010) 0.107 (0.009) -0.008 (0.448) 

Impact: freq. – infreq. 0.006 (0.680) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3391 interviews from 1945 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 508 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.9i: Differences in Impacts of HIP on Estimated Daily Intake from Fruit and 

Vegetable Screener, Salsa, Cup-Equivalents, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.010 (0.002) 0.010 (0.001) 0.001 (0.823) 

<3 servings/day 0.007 (0.001) 0.007 (0.002) >-0.001 (0.782) 

Impact: 3+ servings – <3 

servings 0.001 (0.734) 

Attitudes about food, fruits, & vegetables

High 0.011 (0.003) 0.011 (0.002) <0.001 (0.941) 

Low 0.006 (0.002) 0.005 (0.002) <0.001 (0.926) 

Impact: high – low >-0.001 (1.000) 

Fruits & vegetables at home 

Frequently 0.009 (0.002) 0.009 (0.002) 0.001 (0.846) 

Infrequently 0.008 (0.001) 0.009 (0.001) >-0.001 (0.815) 

Impact: freq. – infreq. 0.001 (0.792) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (respondent module, Modified Eating at America's Table Study (EATS) Fruit and 

Vegetable Screener), pooled Round 2 and Round 3 responses (unweighted N=3383 interviews from 1958 

respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 509

Exhibit H8.10a: Differences in Impacts of HIP on Probability of Any Fruit and Vegetable 

Intake in Past 24 Hours, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.971 (0.007) 0.965 (0.007) 0.006 (0.462) 

<3 servings/day 0.957 (0.008) 0.945 (0.010) 0.012 (0.301) 

Impact: 3+ servings – <3 

servings -0.006 (0.682) 

Attitudes about food, fruits, & vegetables

High 0.972 (0.007) 0.960 (0.009) 0.012 (0.159) 

Low 0.956 (0.011) 0.948 (0.010) 0.007 (0.570) 

Impact: high – low 0.005 (0.762) 

Fruits & vegetables at home 

Frequently 0.966 (0.010) 0.958 (0.011) 0.008 (0.437) 

Infrequently 0.964 (0.009) 0.953 (0.010) 0.011 (0.273) 

Impact: freq. – infreq. -0.003 (0.842) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 510 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.10b: Differences in Impacts of HIP on Probability of Fruit and Vegetable Intake of 

One Cup-Equivalent or More in Past 24 Hours, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.740 (0.018) 0.712 (0.017) 0.028 (0.186) 

<3 servings/day 0.746 (0.019) 0.716 (0.018) 0.030 (0.194) 

Impact: 3+ servings – <3 

servings -0.002 (0.958) 

Attitudes about food, fruits, & vegetables

High 0.780 (0.017) 0.724 (0.018) 0.056 (0.004)*** 

Low 0.697 (0.023) 0.701 (0.022) -0.004 (0.868) 

Impact: high – low     0.060 (0.062)* 

Fruits & vegetables at home 

Frequently 0.741 (0.023) 0.735 (0.022) 0.005 (0.822) 

Infrequently 0.745 (0.019) 0.697 (0.019) 0.047 (0.024)** 

Impact: freq. – infreq. -0.042 (0.185) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 511

Exhibit H8.10c: Differences in Impacts of HIP on Probability of Fruit and Vegetable Intake of 

2.5 Cup-Equivalents or More in Past 24 Hours, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.444 (0.020) 0.344 (0.019) 0.100 (<0.001) *** 

<3 servings/day 0.389 (0.020) 0.364 (0.019) 0.025 (0.293) 

Impact: 3+ servings – <3 

servings 0.075 (0.032)** 

Attitudes about food, fruits, & vegetables

High 0.467 (0.021) 0.373 (0.019) 0.094 (<0.001) *** 

Low 0.351 (0.023) 0.326 (0.023) 0.025 (0.326) 

Impact: high – low 0.069 (0.047)** 

Fruits & vegetables at home 

Frequently 0.415 (0.024) 0.360 (0.023) 0.055 (0.035)** 

Infrequently 0.417 (0.021) 0.348 (0.020) 0.068 (0.003)*** 

Impact: freq. – infreq. -0.013 (0.709) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 512 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.11a: Differences in Impacts of HIP on Total Number of Food Pattern Equivalent 

Fruit and Vegetable Groups Consumed in Past 24 Hours, by Baseline Intake 

and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 3.614 (0.066) 3.324 (0.062) 0.290 (<0.001) *** 

<3 servings/day 3.361 (0.070) 3.225 (0.066) 0.136 (0.103) 

Impact: 3+ servings – <3 

servings 0.153 (0.190) 

Attitudes about food, fruits, & vegetables

High 3.680 (0.066) 3.332 (0.066) 0.348 (<0.001) *** 

Low 3.246 (0.081) 3.198 (0.077) 0.048 (0.586) 

Impact: high – low 0.300 (0.011)** 

Fruits & vegetables at home 

Frequently 3.411 (0.080) 3.224 (0.075) 0.187 (0.032)** 

Infrequently 3.550 (0.071) 3.316 (0.070) 0.234 (0.002)*** 

Impact: freq. – infreq. -0.047 (0.688) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 513

Exhibit H8.11b: Differences in Impacts of HIP on Total Number of Food Pattern Equivalent 

Fruit Groups Consumed in Past 24 Hours, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 1.249 (0.039) 1.115 (0.038) 0.133 (0.007)*** 

<3 servings/day 1.199 (0.040) 1.103 (0.037) 0.096 (0.045)** 

Impact: 3+ servings – <3 

servings 0.037 (0.594) 

Attitudes about food, fruits, & vegetables

High 1.332 (0.041) 1.156 (0.039) 0.177 (<0.001) *** 

Low 1.087 (0.046) 1.045 (0.046) 0.041 (0.419) 

Impact: high – low 0.135 (0.053)* 

Fruits & vegetables at home 

Frequently 1.125 (0.049) 1.073 (0.044) 0.052 (0.328) 

Infrequently 1.301 (0.041) 1.138 (0.041) 0.163 (<0.001) *** 

Impact: freq. – infreq. -0.111 (0.114) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 514 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.11c: Differences in Impacts of HIP on Total Number of Food Pattern Equivalent 

Vegetable Groups Consumed in Past 24 Hours, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.365 (0.045) 2.209 (0.043) 0.156 (0.004)*** 

<3 servings/day 2.162 (0.050) 2.122 (0.045) 0.040 (0.498) 

Impact: 3+ servings – <3 

servings 0.116 (0.151) 

Attitudes about food, fruits, & vegetables

High 2.348 (0.045) 2.176 (0.044) 0.172 (0.001)*** 

Low 2.159 (0.058) 2.153 (0.053) 0.006 (0.918) 

Impact: high – low 0.165 (0.045)** 

Fruits & vegetables at home 

Frequently 2.286 (0.055) 2.152 (0.053) 0.135 (0.019)** 

Infrequently 2.249 (0.052) 2.178 (0.048) 0.071 (0.204) 

Impact: freq. – infreq. 0.064 (0.426) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 515

Exhibit H8.12a: Differences in Impacts of HIP on Consumption of Total Grains, OunceEquivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 5.267 (0.162) 5.710 (0.159) -0.443 (0.032)** 

<3 servings/day 4.856 (0.162) 5.306 (0.166) -0.450 (0.025)** 

Impact: 3+ servings – <3 

servings 0.007 (0.981) 

Attitudes about food, fruits, & vegetables

High 5.264 (0.175) 5.523 (0.183) -0.259 (0.162) 

Low 4.818 (0.224) 5.496 (0.199) -0.678 (0.002)*** 

Impact: high – low 0.419 (0.145) 

Fruits & vegetables at home 

Frequently 4.732 (0.195) 5.235 (0.203) -0.503 (0.020)** 

Infrequently 5.333 (0.165) 5.735 (0.173) -0.402 (0.035)** 

Impact: freq. – infreq. -0.101 (0.724) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 

Exhibit H8.12b: Differences in Impacts of HIP on Consumption of Whole Grains, Ounce 

Equivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.660 (0.047) 0.666 (0.045) -0.006 (0.920) 

<3 servings/day 0.602 (0.043) 0.628 (0.046) -0.026 (0.631) 

Impact: 3+ servings – <3 

servings 0.020 (0.805) 

Attitudes about food, fruits, & vegetables

High 0.663 (0.048) 0.656 (0.045) 0.006 (0.900) 

Low 0.592 (0.055) 0.635 (0.056) -0.043 (0.486) 

Impact: high – low 0.049 (0.541) 

Fruits & vegetables at home 

Frequently 0.649 (0.058) 0.647 (0.059) 0.002 (0.974) 

Infrequently 0.618 (0.045) 0.648 (0.047) -0.029 (0.561) 

Impact: freq. – infreq. 0.031 (0.692) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 516 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.12c: Differences in Impacts of HIP on Consumption of Other Grains, OunceEquivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 4.607 (0.151) 5.044 (0.148) -0.437 (0.023)** 

<3 servings/day 4.254 (0.155) 4.678 (0.159) -0.424 (0.026)** 

Impact: 3+ servings – <3 

servings -0.013 (0.962) 

Attitudes about food, fruits, & vegetables

High 4.601 (0.164) 4.867 (0.172) -0.266 (0.129) 

Low 4.225 (0.209) 4.861 (0.187) -0.635 (0.002)*** 

Impact: high – low 0.370 (0.171) 

Fruits & vegetables at home 

Frequently 4.083 (0.180) 4.588 (0.188) -0.505 (0.011)** 

Infrequently 4.715 (0.158) 5.087 (0.166) -0.372 (0.041)** 

Impact: freq. – infreq. -0.133 (0.622) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 

Exhibit H8.12d: Differences in Impacts of HIP on Consumption of Total Dairy (Milk, Yogurt, 

Cheese), Cup-Equivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 1.643 (0.072) 1.656 (0.065) -0.014 (0.879) 

<3 servings/day 1.512 (0.069) 1.460 (0.061) 0.052 (0.524) 

Impact: 3+ servings – <3 

servings -0.066 (0.597) 

Attitudes about food, fruits, & vegetables

High 1.673 (0.074) 1.573 (0.070) 0.100 (0.211) 

Low 1.462 (0.084) 1.543 (0.089) -0.081 (0.378) 

Impact: high – low 0.181 (0.140) 

Fruits & vegetables at home 

Frequently 1.450 (0.087) 1.502 (0.085) -0.052 (0.593) 

Infrequently 1.682 (0.068) 1.607 (0.067) 0.075 (0.336) 

Impact: freq. – infreq. -0.127 (0.314) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 517

Exhibit H8.12e: Differences in Impacts of HIP on Consumption of Meat, Poultry, Fish, Eggs, 

Soy, Nuts, Seeds, & Legumes, Ounce-Equivalents, by Baseline Intake and 

Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 5.011 (0.174) 5.004 (0.153) 0.007 (0.973) 

<3 servings/day 5.061 (0.193) 5.123 (0.180) -0.062 (0.791) 

Impact: 3+ servings – <3 

servings 0.069 (0.833) 

Attitudes about food, fruits, & vegetables

High 4.955 (0.183) 4.993 (0.165) -0.038 (0.858) 

Low 5.136 (0.218) 5.158 (0.186) -0.022 (0.923) 

Impact: high – low -0.015 (0.961) 

Fruits & vegetables at home 

Frequently 5.193 (0.206) 5.097 (0.202) 0.096 (0.669) 

Infrequently 4.912 (0.188) 5.035 (0.165) -0.123 (0.562) 

Impact: freq. – infreq. 0.219 (0.476) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 518 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.12f: Differences in Impacts of HIP on Consumption of Discretionary Oils, GramEquivalents, by Baseline Intake and Attitude Subgroup 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 18.12 (0.73) 18.75 (0.69) -0.63 (0.495) 

<3 servings/day 16.66 (0.74) 17.95 (0.82) -1.29 (0.201) 

Impact: 3+ servings – <3 

servings 0.67 (0.627) 

Attitudes about food, fruits, & vegetables 

High 19.15 (0.86) 20.07 (0.92) -0.91 (0.303) 

Low 15.19 (1.07) 16.03 (0.93) -0.84 (0.409) 

Impact: high – low -0.07 (0.959) 

Fruits & vegetables at home 

Frequently 18.04 (0.93) 18.77 (0.98) -0.73 (0.456) 

Infrequently 16.90 (0.83) 18.03 (0.86) -1.13 (0.218) 

Impact: freq. – infreq. 0.40 (0.758) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 

Exhibit H8.12g: Differences in Impacts of HIP on Consumption of Discretionary Solid Fats, 

Gram-Equivalents, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 30.29 (1.12) 32.34 (1.14) -2.05 (0.165) 

<3 servings/day 29.13 (1.18) 30.48 (1.07) -1.34 (0.333) 

Impact: 3+ servings – <3 

servings -0.71 (0.726) 

Attitudes about food, fruits, & vegetables

High 29.39 (1.10) 30.24 (1.14) -0.85 (0.522) 

Low 30.18 (1.41) 33.04 (1.27) -2.86 (0.062)* 

Impact: high – low 2.01 (0.320) 

Fruits & vegetables at home 

Frequently 28.28 (1.41) 30.52 (1.39) -2.25 (0.156) 

Infrequently 30.89 (1.11) 32.15 (1.18) -1.26 (0.346) 

Impact: freq. – infreq. -0.99 (0.637) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 519

Exhibit H8.12h: Differences in Impacts of HIP on Consumption of Added Sugar, Teaspoons, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 15.42 (0.72) 16.77 (0.68) -1.35 (0.143) 

<3 servings/day 14.84 (0.72) 14.77 (0.67) 0.07 (0.935) 

Impact: 3+ servings – <3 

servings -1.42 (0.266) 

Attitudes about food, fruits, & vegetables

High 14.21 (0.69) 15.16 (0.64) -0.95 (0.241) 

Low 16.35 (0.87) 16.64 (0.88) -0.28 (0.783) 

Impact: high – low -0.66 (0.614) 

Fruits & vegetables at home 

Frequently 14.70 (0.83) 15.56 (0.82) -0.86 (0.344) 

Infrequently 15.52 (0.78) 15.97 (0.71) -0.45 (0.609) 

Impact: freq. – infreq. -0.41 (0.750) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Added sugar includes, for example, the sugar added to sweetened soft drinks consumed. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 520 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.12i: Differences in Impacts of HIP on Consumption of Alcohola, Drinks, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 0.250 (0.049) 0.129 (0.024) 0.121 (0.022)** 

<3 servings/day 0.169 (0.040) 0.135 (0.027) 0.034 (0.467) 

Impact: 3+ servings – <3 

servings 0.087 (0.222) 

Attitudes about food, fruits, & vegetables

High 0.208 (0.044) 0.147 (0.031) 0.060 (0.181) 

Low 0.211 (0.054) 0.110 (0.038) 0.101 (0.070)* 

Impact: high – low -0.040 (0.575) 

Fruits & vegetables at home 

Frequently 0.273 (0.058) 0.148 (0.034) 0.124 (0.047)** 

Infrequently 0.160 (0.032) 0.118 (0.026) 0.042 (0.292) 

Impact: freq. – infreq. 0.082 (0.266) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. a

Reflects alcohol content in beverages consumed. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 521

Exhibit H8.13: Differences in Impacts of HIP on Total Energy Intake, Kilocalories, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 1793 (44) 1841 (40) -48 (0.385) 

<3 servings/day 1702 (43) 1751 (41) -50 (0.339) 

Impact: 3+ servings – <3 

servings 1 (0.985) 

Attitudes about food, fruits, & vegetables

High 1769 (43) 1794 (45) -24 (0.629) 

Low 1722 (55) 1802 (50) -80 (0.166) 

Impact: high – low 55 (0.469) 

Fruits & vegetables at home 

Frequently 1723 (53) 1782 (52) -59 (0.308) 

Infrequently 1769 (45) 1810 (45) -41 (0.417) 

Impact: freq. – infreq. -18 (0.816) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 

Exhibit H8.14a: Differences in Impacts of HIP on Total Fiber Intake, Grams, by Baseline 

Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 13.65 (0.39) 13.08 (0.37) 0.58 (0.237) 

<3 servings/day 13.28 (0.41) 13.11 (0.36) 0.18 (0.699) 

Impact: 3+ servings – <3 

servings 0.40 (0.552) 

Attitudes about food, fruits, & vegetables

High 14.10 (0.39) 13.31 (0.40) 0.80 (0.063)* 

Low 12.66 (0.50) 12.79 (0.49) -0.14 (0.799) 

Impact: high – low 0.93 (0.177) 

Fruits & vegetables at home 

Frequently 13.11 (0.46) 12.88 (0.43) 0.23 (0.646) 

Infrequently 13.75 (0.41) 13.26 (0.42) 0.49 (0.281) 

Impact: freq. – infreq. -0.26 (0.694) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 522 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.14b: Differences in Impacts of HIP on Total Beta Carotene Intake, Micrograms, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 1702 (167) 1340 (114) 363 (0.052)* 

<3 servings/day 1678 (122) 1651 (171) 27 (0.872) 

Impact: 3+ servings – <3 

servings 336 (0.188) 

Attitudes about food, fruits, & vegetables

High 1945 (165) 1563 (144) 382 (0.050)* 

Low 1356 (111) 1397 (130) -41 (0.727) 

Impact: high – low 423 (0.065)* 

Fruits & vegetables at home 

Frequently 1580 (177) 1490 (165) 91 (0.664) 

Infrequently 1766 (138) 1495 (128) 272 (0.052)* 

Impact: freq. – infreq. -181 (0.467) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 

Exhibit H8.14c: Differences in Impacts of HIP on Total Vitamin A Intake, Micrograms RAE, 

by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 578 (25) 558 (21) 20 (0.514) 

<3 servings/day 595 (30) 584 (30) 11 (0.777) 

Impact: 3+ servings – <3 

servings 9 (0.852) 

Attitudes about food, fruits, & vegetables

High 613 (28) 583 (26) 29 (0.387) 

Low 552 (30) 554 (27) -2 (0.961) 

Impact: high – low 31 (0.511) 

Fruits & vegetables at home 

Frequently 584 (38) 580 (38) 4 (0.928) 

Infrequently 588 (26) 564 (25) 24 (0.371) 

Impact: freq. – infreq. -20 (0.697) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 523

Exhibit H8.14d: Differences in Impacts of HIP on Total Vitamin C Intake, Milligrams, by 

Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 111 (6) 92 (4) 19 (0.005)*** 

<3 servings/day 103 (5) 94 (5) 9 (0.078)* 

Impact: 3+ servings – <3 

servings 10 (0.251) 

Attitudes about food, fruits, & vegetables

High 109 (5) 92 (5) 17 (0.002)*** 

Low 104 (6) 94 (6) 10 (0.148) 

Impact: high – low 8 (0.390) 

Fruits & vegetables at home 

Frequently 110 (6) 99 (6) 11 (0.079)* 

Infrequently 104 (6) 88 (5) 16 (0.005)*** 

Impact: freq. – infreq. -5 (0.542) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 

Exhibit H8.15a: Differences in Impacts of HIP on Intake of Sodium in Foods with Fruits & 

Vegetables, Milligrams, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 930 (39) 967 (44) -37 (0.484) 

<3 servings/day 929 (42) 995 (45) -66 (0.235) 

Impact: 3+ servings – <3 

servings 29 (0.706) 

Attitudes about food, fruits, & vegetables

High 952 (43) 1002 (49) -50 (0.330) 

Low 901 (55) 952 (51) -52 (0.361) 

Impact: high – low 1 (0.984) 

Fruits & vegetables at home 

Frequently 937 (54) 973 (54) -35 (0.521) 

Infrequently 923 (47) 987 (49) -64 (0.248) 

Impact: freq. – infreq. 28 (0.720) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 524 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.15b: Differences in Impacts of HIP on Intake of Discretionary Oils in Foods with 

Fruits & Vegetables, Gram-Equivalents, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 7.64 (0.45) 7.61 (0.44) 0.03 (0.954) 

<3 servings/day 6.45 (0.44) 7.72 (0.53) -1.27 (0.039)** 

Impact: 3+ servings – <3 

servings 1.31 (0.120) 

Attitudes about food, fruits, & vegetables

High 8.23 (0.49) 8.74 (0.58) -0.51 (0.387) 

Low 5.54 (0.55) 6.20 (0.57) -0.66 (0.257) 

Impact: high – low 0.14 (0.862) 

Fruits & vegetables at home 

Frequently 8.04 (0.58) 8.04 (0.65) -0.01 (0.993) 

Infrequently 6.27 (0.51) 7.36 (0.53) -1.09 (0.049)** 

Impact: freq. – infreq. 1.09 (0.178) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates: Appendix H: Analyses by Subgroup ▌pg. 525

Exhibit H8.15c: Differences in Impacts of HIP on Intake of Discretionary Solid Fats in Foods 

with Fruits & Vegetables, Gram-Equivalents, by Baseline Intake and Attitude 

Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 6.33 (0.46) 7.36 (0.61) -1.03 (0.144) 

<3 servings/day 6.74 (0.50) 6.54 (0.48) 0.21 (0.725) 

Impact: 3+ servings – <3 

servings -1.24 (0.169) 

Attitudes about food, fruits, & vegetables

High 6.65 (0.46) 6.88 (0.57) -0.23 (0.710) 

Low 6.42 (0.60) 7.05 (0.57) -0.63 (0.330) 

Impact: high – low 0.40 (0.649) 

Fruits & vegetables at home 

Frequently 6.19 (0.59) 6.89 (0.65) -0.70 (0.319) 

Infrequently 6.82 (0.50) 7.01 (0.62) -0.18 (0.784) 

Impact: freq. – infreq. -0.51 (0.609) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 526 ▌Appendix H: Analyses by Subgroup Abt Associates

Exhibit H8.15d: Differences in Impacts of HIP on Intake of Added Sugar in Foods with Fruits 

& Vegetables, Teaspoons, by Baseline Intake and Attitude Subgroups 

Regressionadjusted 

treatment mean 

(SE) 

Regressionadjusted control 

mean (SE) 

Treatment-control 

impact 

(P-value) 

Difference in 

impacts 

(P-value) 

Baseline respondent fruit & vegetable intake (screener) 

3+ servings/day 2.86 (0.25) 2.78 (0.23) 0.08 (0.782) 

<3 servings/day 3.04 (0.24) 2.94 (0.29) 0.11 (0.751) 

Impact: 3+ servings – <3 

servings -0.02 (0.956) 

Attitudes about food, fruits, & vegetables

High 2.93 (0.27) 2.61 (0.24) 0.32 (0.241) 

Low 2.98 (0.29) 3.20 (0.44) -0.22 (0.590) 

Impact: high – low 0.54 (0.256) 

Fruits & vegetables at home 

Frequently 2.54 (0.28) 2.36 (0.26) 0.18 (0.481) 

Infrequently 3.28 (0.28) 3.26 (0.30) 0.02 (0.951) 

Impact: freq. – infreq. 0.16 (0.730) 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. 

Due to rounding, reported impacts (T-C differences) and reported differences in impacts across subgroups may 

differ from differences between reported regression-adjusted means for the treatment and control groups and 

subgroups. 

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 

10% second-day subsamples for each round (unweighted N=3913 recalls from 2009 respondents. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary Non-Experimental Analyses ▌pg. 527

Appendix I: Supplementary Non-Experimental Analyses 

This appendix presents two supplementary sets of non-experimental analyses intended to provide 

additional context for understanding our main random assignment findings. First, our mediator 

analyses yield evidence on potential mechanisms by which HIP may work to increase fruit and 

vegetable intake. Second, within-treatment group analyses of the relationship between self-reported 

experiences with HIP and observed spending and consumption outcomes provide suggestive evidence 

on how outcomes might be affected by differences in awareness and understanding of HIP. 

I.1 Mediation Analyses 

We conducted mediation analysis to provide insight into the question of how HIP works to ultimately 

increase consumption of fruits and vegetables. The mediation analysis uses multivariate regression 

models to explore a variety of pathways through which HIP may be linked to fruit and vegetable 

consumption. In contrast to the main HIP impact estimates in the body of this Final Report, which 

capitalize on the random assignment research design and require fewer and more robust assumptions, 

this mediation analysis requires strong assumptions to infer causality, and thus should be interpreted 

with caution. 

This appendix section begins with an overview of the single-mediator modeling approach that we use 

to address the question of how HIP works. The second subsection describes the particular models we 

estimated. The third subsection provides our mediation results. The fourth subsection explains the 

assumptions required to interpret the estimates from mediator models as addressing the question of 

how HIP works, as well as the corresponding limitations to the conclusions we can make based on 

our results. Acknowledging these limitations, the final section provides an overall discussion of the 

mediation analyses. 

Single Mediator Models 

To test the pathways through which HIP may affect intake of targeted fruits and vegetables, we 

estimated single mediator models, testing one mediator at a time. Exhibit I.1 presents a depiction of a 

single mediator model. 

Exhibit I.1: Example of a Single Mediator Model 

M:  Intermediate Outcome

(Mediator;

e.g., Attitudes toward F&V)

Action Theory Link Conceptual Theory Link

T: HIP/no HIP Y:  TFV Intake Total Effect

(or Direct Effect)
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 528 ▌Appendix I: Supplementary Non-Experimental Analyses Abt Associates

Our primary analyses, presented in Chapter 8, estimate the “total effect” of HIP (T/Treatment) on the 

ultimate outcomes of interest (Y/TFV Intake). 

From a theoretical perspective, it is sometimes insightful to conceptualize this total effect as a 

combination of two distinct pathways: 

1. The Mediated Effect through M. For example, HIP may cause an increase in positive attitudes 

toward fruits and vegetables, and in turn, more positive attitudes may be linked to higher TFV 

intake. This mediated effect would suggest that HIP affects TFV intake through changes in 

attitudes. 

2. The Direct Effect; HIP affects TFV intake directly without, for example, changing attitudes, 

or through other unmeasured intermediate variables. 

In equations, we can express these ideas as follows. First, we express the Total Effect (i.e., the overall 

effect of HIP on the outcome, including the direct effect and any mediated effects) as: 

(1) 

i   i  i i Y   T  X  u 0 1 3

Second, we express the Action Theory Link (i.e., the effect of HIP on the mediator) as: 

(2) 

i   i  i i M   T  X  u 0 1 3

Third, we express the Conceptual Theory Link (i.e., the effect of the mediator on the outcome) as: 

(3) 

0 2 3

    i i i i Y   M  X  u

Fourth, we express the Mediator Model (from which we derive the direct effect and the mediator 

effect, net of HIP) as: 

(4) 

i   i  i  i i Y   T  M  X  u 0 1 2 3

Where each of the equations includes other covariates measured at baseline, X, and a regression 

residual, u (with appropriate superscript). 

In terms of this formulation, we can explain mediation as follows: Equation 2 implies that the 

intervention (T, in our case HIP) increases the mediator by 1. In addition, Equation 4 implies that 

each additional unit increase in the mediator, M (attitudes, in our example), increases the outcome by 

, net of the direct effect of T on the outcome. Thus, the mediated effect of T on Y can be expressed 

as 1. The direct effect (i.e., the effect of HIP on the outcome, net of the mediating effect) can be 

expressed as1, or calculated by subtracting the mediated effect from the total effect as 1 - 12. 

Finally, note that all four of these expressions are written in terms of observable variables and in 

linear form. It follows that they can be estimated by linear regression. Furthermore, we can calculate 

standard errors and p-values for 1 by the delta method (Cramer, 1946; Oehlert, 1992; Greene, 

2003). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary Non-Experimental Analyses ▌pg. 529

Testing Single Mediator Models 

To test single mediator models, a common approach is the Baron and Kenny (1986) method. 

Following this method, for each single mediation model, three regression equations are estimated, 

corresponding to equations 1, 2, and 4 above. To establish mediation, the following conditions must 

be met: 

1. The predictor (e.g., treatment/HIP) must affect the outcome in equation 1 (i.e., the total effect 

must be significant).175 

2. The predictor (e.g., treatment/HIP) must affect the mediator in equation 2 (i.e., the effect of 

the predictor on the mediator, 1, must be significant). 

3. The mediator must affect the outcome in equation 4 (i.e., the effect of the mediator on the 

outcome, , net of the effect of the predictor on the outcome, must be significant). 

4. The effect of the predictor (e.g., treatment/HIP) on the outcome must be smaller in equation 4 

(when the mediator is included) than in equation 1 (i.e., the direct effect, 1, must be smaller 

than the total effect, 1). When the direct effect is smaller than the total effect, this suggests 

that the mediator is accounting for some of the association between the predictor and the 

outcome, thereby providing support for the mediated pathway. 

In addition, the MacKinnon (1994) method, using the Sobel (1982) test, calculates the mediated effect 

and allows us to test the significance of the mediated effect. The mediated effect can be calculated by 

multiplying the effect of the predictor on the mediator (from equation 2 above) by the effect of the 

mediator on the outcome (net of the effect of the predictor on the outcome, from equation 4 above), or 

1 according to the above notation176. If the mediated effect is significant, we conclude that some of 

the link between the predictor and the outcome occurs through the mediator. If the mediated effect is 

not significant, but the conditions above are met, we conclude that there is some suggestive evidence 

of mediation. If the mediated effect is not significant and the conditions above are not met, we 

conclude that we have no evidence that the link between the predictor and the outcome occurs 

through mediator in question. 

Mediation Models Estimated 

All single mediator models for this report used the Baron & Kenny (1986) method to test the 

appropriate paths and the MacKinnon (1994) method, using the Sobel (1982) test, to test the 

significance of the mediated effect. All models were run on the primary shopper sample, using 

primary shopper weights, and restricted so that observations with missing data on the predictor, 

mediator, or outcome of a given mediator model were excluded. This implies that all equations for a 

particular mediator model are estimated on a common set of observations. 

 

175 Over time, this condition has been relaxed, because mediators may operate in opposite directions, making 

the total effect of the treatment on the outcome non-significant. 

176 This is also equivalent to subtracting the direct effect from the total effect, 1 - 1, according to the above 

notation. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 530 ▌Appendix I: Supplementary Non-Experimental Analyses Abt Associates

Mediation of the Link between HIP and Fruit and Vegetable Consumption 

We first tested mediation of the link between HIP and fruit and vegetable consumption. HIP was the 

predictor in all cases, and outcomes were Targeted Fruits and Vegetables (TFV), Targeted Vegetables 

(TV), Targeted Fruits (TF), and Total Fruit and Vegetable Intake 177. The mediators considered were: 

1. Positive attitudes toward fruits and vegetables178

2. Fruits and vegetables in the home178

3. Perceived barriers to fruit and vegetable consumption178

4. Perceived barriers to grocery shopping178

5. Fruit and vegetable spending179

6. Received messages about eating fruits and vegetables180

7. Attended nutrition education181

Other Mediation Pathways 

We also tested other mediation pathways, which were motivated by our logic model (reproduced from 

Chapter 1 as Exhibit I.2 below). For these models we tested all single mediator paths that applied to 

participants (i.e., mediation through retailers was not tested). 

 

177 We tested all four fruit and vegetable consumption outcomes, but only our confirmatory fruit and vegetable 

consumption outcome (targeted fruits and vegetables, preferred restrictive proxy measure) is presented in 

the results. Findings for the other fruit and vegetable consumption outcomes were similar, however. 

178 This variable is a scale that was created from multiple survey questions, as detailed in Appendix E. 

179 Measured with the question “What has been your household’s usual MONTHLY expense for fruits and 

vegetables?” 

180 Measured with the question “In the past three months, have you heard or seen any messages about eating 

more fruits and vegetables or the importance of fruits and veggies in a healthy diet?” Responses were 

binary (yes/no).

181 Measured with the question “In the past three months, did you attend any nutrition education or healthy 

eating programs or classes?” Responses were binary (yes/no). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary Non-Experimental Analyses ▌pg. 531

Exhibit I.2: Conceptual Model 

Timing of Mediation 

We tested each set of mediator models in two different ways. In order to take advantage of the added 

power from two rounds of follow-up data collection, we pooled Round 2 and Round 3 data (as we did 

for all impact analysis, resulting in 2-4 observations per household) for the mediator and outcome 

variables. These models assume that mediation occurs concurrently. For example, HIP influences 

fruit and vegetable spending, which influences fruit and vegetable consumption during the same time 

period; or spending influences fruits and vegetables in the home, which influences fruit and vegetable 

consumption, all during the same time period. These models used the normalized weights for Round 2 

or Round 3, where the normalization is the same as is used in the main analysis (i.e., forcing the sum 

of the weights in each round to equal the number of completed interviews). 

To test whether certain mediation pathways occur over a longer time period, we also examined 

temporal mediator models which assume that outcomes today were influenced by mediators earlier in 

time. For example, HIP may influence attitudes and then over time, changes in attitudes may 

influence fruit and vegetable consumption. For these models, Round 2 and Round 3 data were not 

pooled (resulting in 1 observation per household, or two for the 10 percent subsample that completed 

a second dietary recall), and the two rounds of data collection represented the two time points in the 

mediation pathways. We only tested temporal models for pathways that might plausibly occur over 

longer time periods. We assumed that links between fruit and vegetable spending, fruits and 

vegetables in the home, and fruit and vegetable consumption would only occur over a short time 

period (e.g., one would not consume fresh fruits and vegetables months after they were acquired), so 

these links were not examined at different time points in the temporal mediator models. In contrast, 

we hypothesized that the pathways of change involving attitudes and perceived barriers might take 

longer to occur. We therefore examined attitudes and barriers at different time points from other 

variables in temporal models. These models use the normalized Round 3 shopper weights. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 532 ▌Appendix I: Supplementary Non-Experimental Analyses Abt Associates

Covariates 

All models contain the following covariates: 

1. Gender of respondent 

2. Race of respondent 

3. Age of respondent 

4. Location 

5. Household size 

6. Gender of household head 

7. Household composition (elderly/children in the household) 

8. Number of adults in the household 

9. HIP wave 

10. Round 1 values of fruit and vegetable screener variables (estimated servings of juice, fruit, 

leafy green salad, fried potatoes, other potatoes, beans, other vegetables, tomato sauce, salsa) 

11. Round 1 values of all mediator variables collected at Round 1, including attitudes, fruits and 

vegetables in the home, barriers to fruit and vegetable consumption, barriers to grocery 

shopping, and fruit and vegetable spending 

For models predicting fruit and vegetable consumption, the following covariates were also included: 

1. Day of dietary recall interview (day 1 or day 2) 

2. Intake relative to usual levels (intake was more, less, or about the same as usual) 

Due to missing data for some covariates (fruit and vegetable screener variables, baseline attitudes, 

fruits and vegetables in the home, barriers to fruit and vegetable consumption, barriers to grocery 

shopping, and fruit and vegetable spending), the sample’s mean value for each of these variables was 

imputed when there was missing data. All models also include a flag for each imputed variable, 

which indicates whether the value for an observation was imputed.182

Mediation Results 

This section presents the results of the mediator analyses. The first section describes the information 

included in the tables, the second section describes the concurrent and temporal mediator models 

linking HIP to fruit and vegetable consumption, and the third section describes the concurrent and 

temporal mediator models that involve other pathways in our logic model. 

Information Included in the Tables 

Within each table, there is one row for each mediation analysis. Columns indicate the mediator in the 

model linking HIP to fruit and vegetable consumption, and the predictor, mediator, and outcome in 

the models of additional pathways. In the temporal models of additional pathways, the round in which 

 

182 There was only one case in this sample where the attitudes scale was imputed and this observation had a 

unique combination of all other imputation flags, such that the attitudes scale flag was perfectly predicted 

by all other flags and did not provide unique estimates. Therefore, the attitudes scale imputation flag was 

removed from all models. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary Non-Experimental Analyses ▌pg. 533

each variable was collected is also noted.183 The remaining columns in the tables are labeled as 

follows (see MacKinnon, 1994 for further reference to these terms in the context of a mediation 

model): 

 N = The sample size for each mediation tested 

 Total Effect (T) = effect of predictor on the outcome (1 in the above equations) 

 Direct Effect (T’) = effect of predictor on the outcome controlling for the mediator (1 in the 

above equations) 

 α = effect of the predictor on the mediator 

 β = effect of the mediator on the outcome (when the predictor is included in the model) 

 αβ = the mediated effect (α*β) 

 Proportion Mediated = proportion of the total effect that is mediated (αβ /(αβ + T’))184

For each effect, we present the estimate along with its standard error and p-value to indicate the 

statistical significance level of the impact estimate for a two-sided test. Asterisks indicate the level of 

statistical significance (*p < 0.10, **p < 0.05, ***p < 0.01). 

Mediation of the Link between HIP and Fruit and Vegetable Consumption 

Exhibit I.3 presents the concurrent mediation models linking HIP to fruit and vegetable consumption. 

Not surprisingly, these findings indicate that there is evidence that fruit and vegetable spending serves 

as a mediator between HIP and fruit and vegetable consumption (Exhibit I.3, model 5). In other 

words, HIP affects spending, such that HIP participants spend a greater amount on fruits and 

vegetables. In turn, greater spending on fruits and vegetables predicts more fruit and vegetable 

consumption. There is also suggestive evidence that having fruits and vegetables in the home serves 

as a mediator between HIP and fruit and vegetable consumption (Exhibit I.3, model 2; 0.05 < p < 

0.10); HIP causes participants to have more fruits and vegetables in the home, and when there are 

more fruits and vegetables in the home, there is more fruit and vegetable consumption. 

Over time (see temporal mediation models linking HIP to fruit and vegetable consumption in Exhibit 

I.4), there is also suggestive evidence that attitudes toward fruits and vegetables may serve as a 

mediator between HIP and fruit and vegetable consumption (Exhibit I.4, model 1; 0.05 < p < 0.10). 

HIP caused more positive attitudes toward fruits and vegetables among participants at Round 2, and 

more positive attitudes toward fruits and vegetables at Round 2 predicted more fruit and vegetable 

consumption at Round 3. No evidence emerged suggesting that barriers or messages about fruits and 

vegetables (including nutrition education) directly mediate the effect of HIP on fruit and vegetable 

consumption either concurrently or over time. 

 

183 In the temporal models linking HIP to fruit and vegetable consumption, all mediators were measured at 

Round 2 and all fruit and vegetable consumption variables were measured at Round 3. 

184 Calculated by dividing the absolute value of the mediated effect by the absolute value of the total effect. 

The accuracy of this proportion mediated measure is a function of the parameter values, and the proportion 

mediated measure is more accurate when there are large direct effects (MacKinnon, 1994). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 534 ▌Appendix I: Supplementary Non-Experimental Analyses Abt Associates

Additional Pathways through Which HIP May Operate 

Exhibit I.5 presents the concurrent mediation models that represent the additional pathways in the 

logic model. These analyses revealed evidence that fruit and vegetable spending serves as a mediator 

between HIP and fruits and vegetables in the home (Exhibit I.5, model 1). HIP participants spend 

more on fruits and vegetables, and more spending on fruits and vegetables is, in turn, linked to more 

fruits and vegetables in the home. Following this mediation of the effect of HIP on fruits and 

vegetables in the home by fruit and vegetable spending, there is also evidence that fruits and 

vegetables in the home serve as a mediator between spending and fruit and vegetable consumption 

(model 4), such that greater spending on fruits and vegetables predicts more fruits and vegetables in 

the home, which, in turn, predicts more fruit and vegetable consumption. 

There is also evidence that fruits and vegetables in the home serves as a mediator between spending 

and attitudes and barriers. Greater spending on fruits and vegetables predicts more fruits and 

vegetables in the home, which, in turn, predicts more positive attitudes toward fruits and vegetables 

(Exhibit I.5, model 3), fewer perceived barriers to eating fruits and vegetables (Exhibit I.5, model 10), 

and fewer perceived barriers to grocery shopping (Exhibit I.5, model 11). There is also evidence for 

these mediation pathways over time (see temporal mediation models of additional pathways in 

Exhibit I.6). Those who spent more on fruits and vegetables at Round 2 also had more fruits and 

vegetables in the home at Round 2; in turn, those who had more fruits and vegetables in the home at 

Round 2 had more positive attitudes toward fruits and vegetables (Exhibit I.6, model 2), fewer 

barriers to eating fruits and vegetables (Exhibit I.6, model 9), and fewer barriers to grocery shopping 

at Round 3 (Exhibit I.6, model 10) 

Following the above mediation of the link between spending and attitudes by fruits and vegetables in 

the home, evidence also emerged indicating that attitudes serve as a mediator between fruits and 

vegetables in the home and fruit and vegetable consumption (Exhibit I.5, model 16). When there are 

more fruits and vegetables in the home, there are more positive attitudes toward fruits and vegetables, 

and more positive attitudes are linked to more fruit and vegetable consumption. Temporal models also 

provided evidence for this mediation pathway over time. More fruits and vegetables in the home at 

Round 2 predicted more positive attitudes at Round 2, which predicted more fruit and vegetable 

consumption at Round 3 (Exhibit I.6, model 15). Similarly, there was suggestive evidence (0.05 < p < 

0.10) that those with more fruits and vegetables in the home at Round 2 also had more positive 

attitudes toward fruits and vegetables at Round 3, and those with more positive attitudes at Round 3 

consumed more fruits and vegetables at Round 3 (Exhibit I.6, model 16). 

There is some suggestive evidence185 that, over time, attitudes also serve as a mediator between HIP 

and fruit and vegetable spending186 (Exhibit I.6, model 1). The mediated effect in this case was not 

significant. However, HIP participants reported more positive attitudes toward fruits and vegetables at 

Round 2, and those with more positive attitudes at Round 2 tended to spend more on fruits and 

vegetables at Round 3. 

 

185 In cases where the mediated effect (αβ) did not reach the level of significance at p < .10, but the conditions 

required by Baron and Kenny (1986) were met, we concluded that there was some suggestive evidence of 

mediation. 

186 The total and direct effects are not significant in these cases. However, as noted above, this condition is 

often relaxed. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary NExhibit I.3: Concurrent Mediation Models Linking HIP to Fruit and Vegetable Consumption (TFV, Cup-Equivalents)

Mediator N 

Total Effect (T) Direct Effect (T') α β  

Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) 

1 Attitudes 3783 0.232 [0.057] (<0.001)*** 0.225 [0.057] (<0.001)*** 0.027 [0.022] (0.221) 0.237 [0.058] (<0.001)*** 2 FV home 3782 0.231 [0.057] (<0.001)*** 0.220 [0.057] (<0.001)*** 0.063 [0.029] (0.031)** 0.176 [0.042] (<0.001)*** 3 FV barriers 3774 0.228 [0.057] (<0.001)*** 0.226 [0.057] (<0.001)*** -0.026 [0.022] (0.244) -0.052 [0.059] (0.377) 4 Shop barriers 3750 0.238 [0.057] (<0.001)*** 0.235 [0.057] (<0.001)*** 0.051 [0.045] (0.256) 0.040 [0.036] (0.264) 5 FV spending 3287 0.261 [0.061] (<0.001)*** 0.242 [0.061] (<0.001)*** 8.130 [2.731] (0.003)*** 0.002 [0.001] (<0.001)*** 6 Messages 3770 0.229 [0.057] (<0.001)*** 0.225 [0.057] (<0.001*** 0.078 [0.019] (<0.001)*** 0.051 [0.067] (0.448) 7 Nutrition ed. 3787 0.231 [0.057] (<0.001)*** 0.231 [0.057] (<0.001)*** -0.001 [0.013] (0.940) 0.130 [0.116] (0.261) 

*p<.10; **p<.05; ***p<.01.Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store. It excludes white 

well as mixed foods where the source of individual ingredients was not identified by the respondent. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey, pooled Round 2 and Round 3 sample. 

Exhibit I.4: Temporal Mediation Models Linking HIP to Fruit and Vegetable Consumption (TFV, Cup-Equivalents)

Mediator N 

Total Effect (T) Direct Effect (T') α β  

Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) 

1 Attitudes 1625 0.239 [0.087] (0.006)*** 0.218 [0.088] (0.014)** 0.070 [0.029] (0.017)** 0.279 [0.086] (0.001)*** 2 FV barriers 1621 0.240 [0.088] (0.006)*** 0.240 [0.088] (0.006)*** >-0.001 [0.032] (0.995) -0.102 [0.100] (0.306) 3 Shop barriers 1613 0.246 [0.088] (0.005)*** 0.243 [0.088] (0.006)*** 0.033 [0.062] (0.596) 0.067 [0.041] (0.105) 4 Messages 1619 0.251 [0.088] (0.004)*** 0.245 [0.088] (0.005)*** 0.093 [0.026] (<0.001)*** 0.067 [0.102] (0.510) 5 Nutrition ed. 1628 0.238 [0.087] (0.006)*** 0.238 [0.087] (0.007)*** -0.002 [0.018] (0.917) -0.123 [0.124] (0.322) 

*p<.10; **p<.05; ***p<.01. Mediators measured at Round 2; fruit and vegetable consumption measured at Round 3. Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store. It excludes white potatoes, legumes, and 1where the source of individual ingredients was not identified by the respondent. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey, Round 2 and Round 3 sample. 
Evaluation of the Health

pg. 536 ▌ Appendix I: Supplementary Non-Experimental Analyses Exhibit I.5: Concurrent Mediation Models of Additional Pathways

Total Effect (T) Direct Effect (T') α Β 

Predictor Mediator Outcome N Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P

1 HIP FV Spending FV Home 3285 0.065 [0.031] (0.033)** 0.055 [0.030] (0.068)* 8.095 [2.732] (0.003)*** 0.001 [<0.001] (<2 HIP Attitudes FV Spending 3284 8.182 [2.729] (0.003)*** 8.096 [2.723] (0.003)*** 0.023 [0.024] (0.328) 3.746 [2.948] 3 FV Spending FV Home Attitudes 3281 <0.001 [<0.001] (0.210) <0.001 [<0.001] (0.668) 0.001 [<0.001] (<0.001)*** 0.133 [0.023] (<4 FV Spending FV Home TFV 3284 0.002 [0.001] (<0.001)*** 0.002 [0.001] (<0.001)*** 0.001 [<0.001] (<0.001)*** 0.137 [0.044] (05 Attitudes FV Spending FV Home 3281 0.208 [0.036] (<0.001)*** 0.204 [0.035] (<0.001)*** 3.815 [2.956] (0.197) 0.001 [<0.001] (<6 Attitudes FV Spending TFV 3283 0.236 [0.064] (<0.001)*** 0.228 [0.063] (<0.001)*** 3.833 [2.954] (0.195) 0.002 [0.001] (<7 FV Home Attitudes FV Spending 3281 12.238 [2.262] (<0.001)*** 12.064 [2.374] (<0.001)*** 0.134 [0.023] (<0.001)*** 1.301 [3.018] 8 HIP FV Barriers FV Spending 3282 8.177 [2.731] (0.003)*** 8.027 [2.724] (0.003)*** -0.040 [0.024] (0.096)* -3.735 [3.178] 9 HIP Shop Barriers FV Spending 3266 8.030 [2.732] (0.003)*** 8.148 [2.734] (0.003)*** 0.047 [0.047] (0.321) -2.517 [1.849] 10 FV Spending FV Home FV Barriers 3279 >-0.001 [<0.001] (0.233) >-0.001 [<0.001] (0.661) 0.001 [<0.001] (<0.001)*** -0.138 [0.022] (<11 FV Spending FV Home Shop Barriers 3265 -0.001 [<0.001] (0.175) >-0.001 [<0.001] (0.396) 0.001 [<0.001] (<0.001)*** -0.195 [0.045] (<12 FV Barriers FV Spending FV Home 3279 -0.205 [0.032] (<0.001)*** -0.200 [0.031] (<0.001)*** -3.884 [3.186] (0.223) 0.001 [<0.001] (<13 Shop Barriers FV Spending FV Home 3265 -0.077 [0.018] (<0.001)*** -0.074 [0.017] (<0.001)*** -2.472 [1.852] (0.182) 0.001 [<0.001] (<14 FV Barriers FV Spending TFV 3281 -0.070 [0.056] (0.209) -0.062 [0.056] (0.270) -3.874 [3.185] (0.224) 0.002 [0.001] (<15 Shop Barriers FV Spending TFV 3265 0.034 [0.033] (0.300) 0.040 [0.032] (0.218) -2.473 [1.852] (0.182) 0.002 [0.001] (<16 FV Home Attitudes TFV 3778 0.180 [0.042] (<0.001)*** 0.149 [0.042] (<0.001)*** 0.149 [0.021] (<0.001)*** 0.203 [0.058] (<17 FV Home FV Barriers TFV 3769 0.178 [0.042] (<0.001)*** 0.175 [0.045] (<0.001)*** -0.130 [0.020] (<0.001)*** -0.021 [0.062] 18 FV Home Shop Barriers TFV 3747 0.179 [0.043] (<0.001)*** 0.189 [0.043] (<0.001)*** -0.188 [0.043] (<0.001)*** 0.055 [0.036] 19 FV Home FV Barriers FV Spending 3279 12.268 [2.265] (<0.001)*** 12.071 [2.297] (<0.001)*** -0.140 [0.022] (<0.001)*** -1.414 [3.209] 20 FV Home Shop Barriers FV Spending 3265 12.289 [2.301] (<0.001)*** 11.979 [2.328] (<0.001)*** -0.200 [0.045] (<0.001)*** -1.550 [1.851] 

*p<.10; **p<.05; ***p<.01. Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store. It excludes white potatoes, legumes, and 1where the source of individual ingredients was not identified by the respondent. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey, pooled Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary NExhibit I.6: Temporal Mediation Models of Additional Pathways

Total Effect (T) Direct Effect (T') α β 

Predictor Mediator Outcome N Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P

1 HIP Attitudes (R2) FV spending 

(R3) 

1425 5.398 [3.646] (0.139) 4.957 [3.674] (0.178) 0.068 [0.031] (0.031)** 6.505 [3.856] (

2 FV spending 

(R2) 

FV home (R2) Attitudes (R3) 1393 0.001 [<0.001] (0.019)** 0.001 [<0.001] (0.071)* 0.002 [<0.001] (<0.001)*** 0.113 [0.035] (0

3 Attitudes (R2) FV spending 

(R3) 

FV home (R3) 1425 0.182 [0.049] (<0.001)*** 0.175 [0.049] (<0.001)*** 6.663 [3.844] (0.083)* 0.001 [<0.001] (0

4 Attitudes (R2) FV spending 

(R3) 

TFV (R3) 1425 0.232 [0.088] (0.009)*** 0.209 [0.086] (0.016)** 6.663 [3.844] (0.083)* 0.003 [0.001] (<0

5 FV home (R2) Attitudes (R3) FV spending 

(R3) 

1427 9.095 [3.351] (0.007)*** 8.775 [3.292] (0.008)*** 0.113 [0.033] (0.001)*** 2.824 [3.679] 

6 FV home (R2) Attitudes (R2) FV spending 

(R3) 

1425 9.043 [3.352] (0.007)*** 8.369 [3.339] (0.012)** 0.139 [0.031] (<0.001)*** 4.849 [3.782] 

7 HIP FV barriers 

(R2) 

FV spending (R3) 

1420 5.312 [3.655] (0.146) 5.242 [3.652] (0.151) -0.012 [0.035] (0.740) -6.070 [3.446] (

8 HIP Shop barriers 

(R2) 

FV spending (R3) 

1416 5.306 [3.670] (0.149) 5.268 [3.680] (0.153) 0.019 [0.067] (0.780) 2.059 [2.410] 

9 FV spending 

(R2) 

FV home (R2) FV barriers 

(R3) 

1391 >-0.001 [<0.001] (0.257) >-0.000 [<0.001] (0.755) 0.002 [<0.001] (<0.001)*** -0.140 [0.034] (<0

10 FV spending 

(R2) 

FV home (R2) Shop barriers 

(R3) 

1384 >-0.001 [0.001] (0.459) >-0.000 [0.001] (0.866) 0.002 [<0.001] (<0.001)*** -0.205 [0.064] (0

11 FV barriers 

(R2) 

FV spending (R3) 

FV home (R3) 1420 -0.119 [0.042] (0.005)*** -0.112 [0.041] (0.007)*** -6.096 [3.446] (0.077)* 0.001 [<0.001] (0

12 Shop barriers 

(R2) 

FV spending (R3) 

FV home (R3) 1416 -0.061 [0.024] (0.012)** -0.063 [0.024] (0.008)*** 2.070 [2.408] (0.390) 0.001 [<0.001] (0

13 FV barriers 

(R2) 

FV spending (R3) 

TFV (R3) 1420 -0.108 [0.105] (0.306) -0.086 [0.105] (0.411) -6.096 [3.446] (0.077)* 0.003 [0.001] (<0

14 Shop barriers 

(R2) 

FV spending (R3) 

TFV (R3) 1416 0.067 [0.039] (0.086)* 0.059 [0.039] (0.125) 2.070 [2.408] (0.390) 0.004 [0.001] (<0

15 FV home (R2) Attitudes (R2) TFV (R3) 1624 0.181 [0.072] (0.013)** 0.146 [0.069] (0.034)** 0.139 [0.029] (<0.001)*** 0.255 [0.080] (016 FV home (R2) Attitudes (R3) TFV (R3) 1627 0.182 [0.072] (0.012)** 0.159 [0.072] (0.028)** 0.121 [0.031] (<0.001)*** 0.192 [0.089] (017 FV home (R2) FV barriers 

(R2) 

TFV (R3) 1620 0.184 [0.072] (0.011)** 0.175 [0.075] (0.020)** -0.119 [0.029] (<0.001)*** -0.072 [0.103] 

18 FV home (R2) FV barriers 

(R3) 

TFV (R3) 1622 0.184 [0.073] (0.012)** 0.179 [0.080] (0.025)** -0.135 [0.030] (<0.001)*** -0.036 [0.109] 

19 FV home (R2) Shop barriers 

(R2) 

TFV (R3) 1612 0.188 [0.073] (0.010)** 0.198 [0.074] (0.008)*** -0.122 [0.058] (0.034)** 0.077 [0.043] (

20 FV home (R2) Shop barriers 

(R3) 

TFV (R3) 1613 0.183 [0.072] (0.012)** 0.189 [0.074] (0.011**) -0.158 [0.062] (0.012)** 0.040 [0.060] 

21 FV home (R2) FV barriers 

(R2) 

FV spending (R3) 

1420 9.265 [3.383] (0.006)*** 8.659 [3.452] (0.012)** -0.137 [0.032] (<0.001)*** -4.437 [3.498] 
Evaluation of the Health

pg. 538 ▌ Appendix I: Supplementary Non-Experimental Analyses 

Total Effect (T) Direct Effect (T') α β 

Predictor Mediator Outcome N Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P-value) Estimate [S.E.] (P

22 FV home (R2) Shop barriers 

(R2) 

FV spending (R3) 

1416 9.253 [3.359] (0.006)*** 9.583 [3.322] (0.004)*** -0.130 [0.062] (0.037)** 2.543 [2.424] 

23 FV home (R2) FV barriers 

(R3) 

FV spending (R3) 

1425 9.140 [3.356] (0.007)*** 8.476 [3.459] (0.014)** -0.148 [0.033] (<0.001)*** -4.483 [4.200] 

24 FV home (R2) Shop barriers 

(R3) 

FV spending (R3) 

1421 9.381 [3.348] (0.005)*** 9.399 [3.287] (0.004)*** -0.220 [0.066] (0.001)*** 0.079 [2.372] 

*p<.10; **p<.05; ***p<.01. Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store. It excludes white potatoes, legumes, and 1where the source of individual ingredients was not identified by the respondent. Standard errors and test statistics in models with TFV outcome are adjusted for clustering at the individual respondent level. Source: Participant Survey, Round 2 and Round 3 sample. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary Non-Experimental Analyses ▌pg. 539

There is also some suggestive evidence that, over time, spending serves as a mediator between both 

attitudes and perceived barriers to eating fruits and vegetables and having fruits and vegetables in 

the home. Again, these mediated effects were not significant, but those with more positive attitudes 

toward fruits and vegetables (Exhibit I.6, model 3) and those with fewer perceived barriers to eating 

fruits and vegetables (ExhibitI.6, model 11) at Round 2 tended to spend more on fruits and vegetables 

at Round 3; those who spent more on fruits and vegetables at Round 3 also had more fruits and 

vegetables in the home at Round 3. 

Some suggestive evidence also indicates that, over time, spending serves as a mediator between both 

attitudes and perceived barriers to eating fruits and vegetables and fruit and vegetable consumption.

Those with more positive attitudes towards fruits and vegetables (Exhibit I.6, model 4, mediated 

effect not significant) and those with fewer barriers to eating fruits and vegetables (Exhibit I.6, model 

13, 0.05 < p < 0.10) at Round 2 tended to spend more on fruits and vegetables at Round 3, and those 

who spent more on fruits and vegetables at Round 3 consumed more fruits and vegetables at Round 3. 

Implications, Limitations, and Assumptions 

The mediation analyses presented help us to understand how HIP works. However, these insights 

provided by the mediation analyses require a set of implicit assumptions. To understand those 

assumptions, this section considers three questions. First, if these findings have a causal 

interpretation, how could we use them for advancing theory and policy? Second, what assumptions 

are required to give these estimates a causal interpretation? Third, how valid are these assumptions? 

With respect to theory, knowledge of “how HIP works” can be useful for designing future 

interventions. Given that mediation analyses suggest that HIP may operate through changing 

attitudes, in addition to through prices, future programs might emphasize attitudinal change in 

addition to the incentive. Future experimental work would be needed, however, to establish an impact 

of any other program design and to further inform policy aimed at increasing fruit and vegetable 

consumption. 

Crucially, this use of mediation models requires a causal interpretation of the regression parameters. 

In particular, in order for the effect of HIP through the mediator to be expressible as αβ, two 

conditions must be satisfied. First, it must be true that α gives the causal effect of the predictor on the 

mediator, where by “causal effect,” we mean the change in the mediator with an exogenous (i.e., 

outside the system) change in the predictor. Second, it must be true that β gives the causal effect of 

the mediator on the outcome; where by “causal effect,” we mean the change in the outcome with an 

exogenous (i.e., outside the system) change in the mediator variable. 

It is not clear that both these conditions are satisfied. In a best linear predictor sense, regression gives 

the best fit between the included covariates and the dependent variable. However, standard analysis 

for linear regression suggests that only when there are no unobservable variables correlated with the 

included variables will regression estimate the causal effect (e.g., Greene, 2003). HIP treatment status 

is randomly assigned. So for α, the conditions required for a causal interpretation are minimal when 

HIP treatment status is the predictor. As long as randomization was properly conducted (as appears to 

be true; see the discussion in Chapter 2) and the other regressors are measured before random 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 540 ▌ Appendix I: Supplementary Non-Experimental Analyses Abt Associates

assignment (as they are), no omitted variable can be correlated with HIP treatment status. Thus, we 

can conclude that α gives the causal effect of HIP on the mediator.187

With respect to β, the assumptions are more problematic. The value of the mediator is determined by 

(randomly specified) intervention status, but also by anything else that affects the mediator. Anything 

that affects the mediator directly may also affect the outcome directly. To some extent observed 

covariates (i.e., X), included in the regression model, control for such factors that affect the mediator 

and the outcome directly. However, a causal interpretation would require that any omitted variables 

have no (or at least only a small) direct impact on the outcome. Given our limited covariates, this 

condition may not be met, and the assumption may be too strong to infer causality. For example, we 

only have limited proxies for underlying tastes for fruits and vegetables (i.e., baseline attitudes toward 

fruits and vegetables), which may be a key variable when predicting fruit and vegetable consumption 

in the absence of HIP. 

Similarly, measurement error in the mediator will induce problems with a causal interpretation. 

Classical measurement error biases coefficients toward zero. In addition, it induces omitted variable 

bias. Even if an effect works totally through a mediator, if the mediator is measured with error, then 

omitted variables will matter. Several of our mediators have been measured with multiple items to 

create scales with adequate reliability. However, we are unable to ensure that there is no measurement 

error. 

These issues have been widely noted in the literature on path analysis. The discussion in 

MacKinnon’s (2008, pp. 365-366) standard textbook on mediation is careful: 

The [Rubin Causal Model] demonstrates the problems in the interpretation of the relation 

between M and Y in mediation models, at least in part because this relation is not randomized 

but is self-selected in most applications. The main benefit of all these detailed causal approaches 

is the causal consideration of the limitations and strengths of different types of evidence of causal 

inference. ... At a minimum the causal inference approaches force researchers to consider the 

assumptions under which mediation is investigated. For the most part, the sensitivity of the 

estimates to violation of assumptions is not generally known[.] 

Several other scholars note the problematic assumptions required to infer causality from mediation 

analyses (see Imai, Keele, & Yamamoto, 2010; Bullock, Green, & Ha, 2010), even in the context of 

an experimental design, such as HIP. For example, in their abstract, Bullock, Green, and Ha (2010) 

note their skepticism: 

[E]xperiments cannot overcome certain threats to inference that arise chiefly or exclusively in the 

context of mediation analysis.... Our conclusion is that inference about mediators is far more 

difficult than previous research suggests, and best tackled by an experimental research program 

that is specifically designed to address the challenges of mediation analysis. 

It is important to note that the evaluation of HIP was not “specifically designed to address the 

challenges of mediation analysis.” Despite the strength of the random assignment design in that it 

requires only minimal assumptions to infer causality, these limited assumptions are not sufficient to 

 

187 When the predictor is not HIP treatment status, the assumptions are more problematic as detailed for the 

link between the mediator and the outcome. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary Non-Experimental Analyses ▌pg. 541

satisfy the assumptions required for mediation models. Thus, these mediation results should be 

interpreted with caution. 

Discussion 

Keeping in mind the limitations and necessary assumptions mentioned above, the analyses presented 

in this appendix section provide insight into how HIP might work by revealing several pathways 

through which HIP may influence its end goal of increasing fruit and vegetable consumption. Several 

pathways in our logic model were supported by these mediation analyses. Consistent with a price 

effect, HIP was shown to influence fruit and vegetable consumption through spending, and combining 

findings from several of the single mediator models implies that HIP increases fruit and vegetable 

spending which increases the amount of fruits and vegetables in the home, which then increases fruit 

and vegetable consumption. This is the expected primary pathway of HIP’s impact. 

Beyond this expected primary pathway, the mediation analyses also suggest that in addition to the 

presumed price effect, over time HIP may also operate to increase fruit and vegetable consumption by 

changing attitudes toward fruits and vegetables. HIP was shown to affect attitudes toward fruits and 

vegetables, and more positive attitudes were subsequently linked to greater fruit and vegetable 

consumption. In addition, the above-mentioned price effect may also involve a change in attitudes; by 

combining multiple mediator models, we infer that HIP increases fruit and vegetable spending, which 

increases fruits and vegetables in the home, which improve attitudes toward fruits and vegetables, and 

then more positive attitudes are linked to greater fruit and vegetable consumption. These findings 

imply that HIP may work in a way that goes beyond a pure price effect. 

It is also important to note that the percent of the effect of HIP on fruit and vegetable consumption 

that was mediated by any one of the mediators examined was quite low, leaving a large portion of this 

effect to be explained. However, some of our additional pathways revealed larger mediated 

proportions, which may indicate, as our logic model suggests, that HIP operates through multiple 

mediators and multiple steps. 

Overall, the mediation analyses presented in this appendix section contribute to the understanding of 

how HIP works to increase fruit and vegetable consumption. However, given the limitations and 

assumptions required of these models, it is important to keep in mind that any causal interpretations 

are only as valid as the required assumptions. 

I.2 HIP Awareness and Understanding 

The descriptive results on HIP participant experiences presented in Chapter 5 suggest varying and far 

from perfect awareness and understanding of HIP. It is therefore natural to ask to what extent impacts 

vary for those with greater or lesser awareness and understanding of the program. 

Chapter 5 showed that even at Round 3, nearly a quarter of treatment group respondents reported that 

they had not heard of HIP. To the extent that this reflects a true lack of awareness about the existence 

of HIP in this group (as opposed to, for example, simple confusion about the name of the pilot), one 

might expect impacts to be diminished as it is difficult to see how HIP could successfully change 

perceptions or behavior in a group that was completely unaware of the pilot’s existence! 

Similarly, HIP participants who reported that it was difficult to understand how HIP worked might be 

less affected by the pilot—if they truly could not understand what they needed to do to earn the 

incentive and therefore did not change their behavior at all. Alternatively, for some outcomes, those 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 542 ▌ Appendix I: Supplementary Non-Experimental Analyses Abt Associates

HIP participants who reported that it was difficult to understand how HIP worked might be more 

affected by the pilot, if they thought, for example, that they could earn the HIP incentive on fruit and 

vegetable purchases with cash and/or at non-participating retailers. 

Likewise, HIP participants who understood that they could earn the HIP incentive by purchasing 

fruits and vegetables in general, but found it difficult to understand exactly which fruits and 

vegetables qualified, might increase purchases of non-qualifying foods under the mistaken impression 

that those foods (e.g., white potatoes, 100% juice) did in fact earn the incentive. 

To explore these questions, we performed a series of non-experimental analyses within the treatment 

group only to assess how key spending and intake outcomes varied with available measures of HIP 

awareness and understanding. In particular, we examined differences in outcomes across the 

following subgroups of treatment group respondents: 

1. Respondents who reported that they had heard of HIP (67 percent) versus respondents who 

reported that they had never heard of HIP (33 percent) 

2. Respondents who reported that it was “very easy” or “easy” to understand how HIP works 

(61 percent) versus respondents who reported that it was “somewhat easy,” “somewhat 

difficult,” “difficult,” or “very difficult” (39 percent) 

3. Respondents who reported that it was “very easy” or “easy” to remember which fruits and 

vegetables qualified to earn the HIP incentive (52 percent) versus respondents who reported 

that it was “somewhat easy,” “somewhat difficult,” “difficult,” or “very difficult” (48 

percent) 

4. Respondents who reported that they kept track of their HIP incentive earnings (39 percent) 

versus those who did not keep track (61 percent)188

We estimate these within-treatment subgroup differences for each of the four sets of subgroups using 

linear regression, in a parallel approach to our main impact analyses. The key explanatory variable in 

these regressions is not HIP participation status, but a binary indicator equal to 1 for those with 

greater understanding/awareness as measured by the above proxies, and 0 for those with lesser 

understanding/awareness. All regressions additionally include the same set of covariates as in the 

main impact models. 

When interpreting these findings, it is important to note several key caveats. First, observed 

differences by HIP awareness or understanding do not necessarily imply a causal relationship. In the 

main body text and in Appendix H, we presented a number of analyses for subgroups defined 

exogenously; i.e., for characteristics defined from before HIP began. These subgroup analyses are 

intended to describe characteristics of participants associated with greater or lesser HIP impacts. 

These subgroup comparisons are unbiased within the random assignment framework because they 

depend on characteristics measured in both the treatment and control groups during the baseline 

period, prior to the start of HIP. 

 

188 This measure might be thought of as a proxy for the “intensiveness” of awareness/understanding and/or 

involvement with HIP; it seems likely that only those respondents who are most aware of and savvy about 

HIP would keep track of earnings. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary Non-Experimental Analyses ▌pg. 543

The situation is quite different for HIP awareness and understanding. For these measures, we can 

define HIP experiences within the treatment group only after the start of implementation, such that 

differences in both experiences and outcomes could plausibly be influenced by some third, 

unobserved factor. For example, someone who purchased a lot of fruits and vegetables even without 

HIP might have paid more attention to program details. Alternatively, a very budget-conscious person 

might be more responsive to the HIP incentive, and simultaneously more likely to take the time to 

understand exactly how the program works to make sure he or she is maximizing potential earnings. 

Thus, even if we observe greater TFV spending among those respondents with greater self-reported 

understanding of how HIP works, it is not necessarily the case that the greater understanding caused

greater spending. 

Second, in addition, it is important to note that our measures of awareness and understanding were 

not specifically developed to facilitate this type of subgroup analysis. For example, the survey asked 

respondents: “Have you heard of the Healthy Incentives Pilot?” As noted above, whether or not a 

respondent has “ever heard of HIP” is an imperfect measure of HIP “awareness” – we do not know 

whether the respondent is truly unaware of the pilot’s existence, or simply does not know it by name. 

In the remainder of this appendix subsection, we present results for spending measures (both surveyreported and EBT-transactions-based), followed by results for consumption measures. We conclude 

with a brief discussion of possible interpretations for these findings. 

Expenditures 

Exhibits I.7 through I.10, respectively, describe differences in expenditures for each of the four HIP 

awareness and understanding measures. 

There were no statistically significant differences in spending by our key awareness measure, i.e. 

whether or not HIP participants reported that they had heard of HIP (Exhibit I.7). This finding is 

somewhat counterintuitive, as one would not expect HIP to have any impacts on those who were 

unaware of its existence; it is possible, however, that this measure does not effectively capture true 

“awareness” about the pilot. 

EBT TFV expenditures in participating supermarkets were significantly higher for HIP participants 

who said that it was easy to understand HIP (Exhibit I.8), for those who said that it was easy to 

remember which fruits and vegetables qualified to earn the incentive (Exhibit I.9), and for those who 

reported that they tracked incentive earnings (Exhibit I.10). This pattern is consistent with the 

hypothesis that those who better understood the program were more likely to shift EBT spending in 

participating retailers toward TFV. (Though, as noted above, it is also possible that these correlations 

are explained by some other, unobserved factor.) For those who said it was easy to remember which 

fruits and vegetables qualified to earn the incentive, total EBT expenditures in participating 

supermarkets were additionally higher; point estimates for the other two understanding measure 

followed a similar pattern, though differences were not statistically significant. 

There were no differences in survey-based, self-reported spending measures by degree of 

understanding. This is not surprising. In Chapter 6, we noted that these measures appear to be 

imprecisely estimated relative to EBT based measures. 
Evaluation of the Healthy In

pg. 544 ▌ Appendix I: Supplementary Non-Experimental Analyses Exhibit I.7: Differences in Survey- and EBT-Transactions-Based Spending Measures, by Awareness of HIP, Treatment 

Regression-adjusted mean (S.E.) Differe

Heard of HIP 

Never heard of 

HIP Difference [S.E.] 

Usual survey-reported monthly spending for...aGroceries using only SNAP (N=1548 interviews over 922 respondents) 269.530 (4.565) 278.506 (8.109) -8.977 [8.881] Groceries not using SNAP (N=1513 interviews over 903 respondents) 146.410 (4.065) 145.175 (6.926) 1.235 [8.072] 

Food items (N=1460 interviews over 876 respondents) 103.147 (3.647) 101.294 (5.796) 1.853 [6.816] Nonfood items (N=1460 interviews over 876 respondents) 41.868 (1.762) 45.781 (3.062) -3.913 [3.589] 

Restaurants (N=1545 interviews over 921 respondents) 35.185 (1.483) 33.104 (2.184) 2.081 [2.669] Fruits and vegetables (N=1361 interviews over 833 respondents) 74.504 (2.036) 80.339 (3.582) -5.835 [4.161] EBT-recorded expenditures per household per month (N=1709 observations over 1001 respondents)Total EBT purchases 281.567 (5.317) 290.849 (7.448) -9.281 [8.616] EBT IECR purchases at HIP participating supermarkets & superstores 146.158 (4.291) 139.727 (4.734) 6.431 [5.701] EBT IECR TFV purchases at HIP participating supermarkets & superstores 13.419 (0.517) 13.463 (0.734) -0.045 [0.863] 

Weighted means (standard errors). Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

aContinuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded as missing. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 responses, and EBT Transactions Data, pooled across Roun

and Round 3 (average of August - October 2012), for survey respondents only. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary NonExhibit I.8: Differences in Survey- and EBT-Transactions-Based Spending Measures, by Self-Reported Understanding 

Group 

Regression-adjusted mean (S.E.) Differe

Easy to 

understand 

Not easy to 

understand Difference [S.E.] 

Usual survey-reported monthly spending for...aGroceries using only SNAP (N=1298 interviews over 840 respondents) 272.670 (5.600) 267.949 (6.726) 4.721 [8.377] Groceries not using SNAP (N=1268 interviews over 826 respondents) 145.867 (4.796) 149.212 (5.708) -3.344 [7.397] 

Food items (N=1225 interviews over 799 respondents) 101.464 (4.301) 106.757 (4.940) -5.293 [6.371] Nonfood items (N=1225 interviews over 799 respondents) 43.110 (2.019) 42.943 (2.822) 0.166 [3.554] 

Restaurants (N=1292 interviews over 834 respondents) 34.107 (1.649) 35.944 (2.148) -1.837 [2.653] Fruits and vegetables (N=1144 interviews over 757 respondents) 76.422 (2.328) 75.837 (2.939) 0.585 [3.744] EBT-recorded expenditures per household per month(N=1298 observations over 840 respondents) Total EBT purchases 279.467 (5.917) 289.906 (6.949) -10.439 [8.345] EBT IECR purchases at HIP participating supermarkets & superstores 148.834 (4.838) 140.812 (5.530) 8.022 [6.710] EBT IECR TFV purchases at HIP participating supermarkets & superstores 14.187 (0.612) 12.328 (0.629) 1.859 [0.809] 

Weighted means (standard errors). Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

aContinuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded as missing. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 responses, and EBT Transactions Data, pooled across Roun

and Round 3 (average of August - October 2012), for survey respondents only. 
Evaluation of the Healthy In

pg. 546 ▌ Appendix I: Supplementary Non-Experimental Analyses Exhibit I.9: Differences in Survey- and EBT-Transactions-Based Spending Measures, by Self-Reported Understanding 

Qualify to Earn the HIP Rebate, Treatment Group 

Regression-adjusted mean (S.E.) Differe

Easy to 

understand 

Not easy to 

understand Difference [S.E.] 

Usual survey-reported monthly spending for... aGroceries using only SNAP (N=1415 interviews over 878 respondents) 276.361 (5.534) 269.375 (6.067) 6.986 [7.524] Groceries not using SNAP (N=1382 interviews over 860 respondents) 136.298 (4.630) 159.825 (5.379) -23.527 [6.820] 

Food items (N=1333 interviews over 834 respondents) 91.729 (4.027) 116.476 (4.715) -24.746 [5.963] Nonfood items (N=1333 interviews over 834 respondents) 43.847 (2.233) 43.451 (2.304) 0.397 [3.211] 

Restaurants (N=1406 interviews over 875 respondents) 34.491 (1.832) 35.622 (1.810) -1.13 [2.566] Fruits and vegetables (N=1251 interviews over 794 respondents) 79.557 (2.520) 74.707 (2.688) 4.851 [3.624] EBT-recorded expenditures per household per month (N=1549 observations over 951 respondents) Total EBT purchases 285.361 (5.918) 288.735 (6.444) -3.374 [7.935] EBT IECR purchases at HIP participating supermarkets & superstores 152.676 (4.871) 139.721 (4.809) 12.956 [6.196] EBT IECR TFV purchases at HIP participating supermarkets & superstores 14.896 (0.632) 12.359 (0.599) 2.537 [0.810] 

Weighted means (standard errors). Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

aContinuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded as missing. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 responses, and EBT Transactions Data, pooled across Roun

and Round 3 (average of August - October 2012), for survey respondents only. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary NonExhibit I.10: Differences in Survey- and EBT-Transactions-Based Spending Measures, by Whether Primary Shopper Ke

Treatment Group 

Regression-adjusted mean (S.E.) Differe

Track rebate 

Don’t track 

rebate Difference [S.E.] 

Usual survey-reported monthly spending for...aGroceries using only SNAP (N=1435 interviews over 886 respondents) 270.922 (6.740) 276.749 (5.377) -5.828 [8.079] Groceries not using SNAP (N=1406 interviews over 870 respondents) 148.558 (5.770) 147.121 (4.594) 1.437 [7.301] 

Food items (N=1354 interviews over 843 respondents) 103.306 (4.945) 105.298 (4.090) -1.992 [6.205] Nonfood items (N=1354 interviews over 843 respondents) 43.759 (2.530) 42.357 (2.047) 1.402 [3.374] 

Restaurants (N=1431 interviews over 884 respondents) 33.488 (2.244) 36.268 (1.656) -2.78 [2.892] Fruits and vegetables (N=1261 interviews over 797 respondents) 77.864 (2.708) 76.162 (2.449) 1.702 [3.637] EBT-recorded expenditures per household per month (N=1578 observations over 963 respondents) Total EBT purchases 285.714 (6.962) 287.076 (5.940) -1.362 [8.672] EBT IECR purchases at HIP participating supermarkets & superstores 152.181 (5.718) 142.209 (4.552) 9.972 [6.946] EBT IECR TFV purchases at HIP participating supermarkets & superstores 14.946 (0.747) 12.593 (0.518) 2.353 [0.849] 

Weighted means (standard errors). Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Standard errors and test statistics are adjusted for clustering at the individual respondent level. 

aContinuous outcomes: self-reported expenses in dollars per month; “don’t know” and “refused” responses coded as missing. 

Source: Participant Survey (primary shopper module), pooled Round 2 and Round 3 responses, and EBT Transactions Data, pooled across Roun

and Round 3 (average of August - October 2012), for survey respondents only. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

pg. 548 ▌ Appendix I: Supplementary Non-Experimental Analyses Abt Associates

Consumption 

Exhibits I.11-I.14 provide results on differences in fruit and vegetable intake by awareness and 

understanding. In particular, we present findings for our primary TFV measure and other intake 

measures. Considering our primary TFV measure allows us to assess potential differences in intake of 

foods that qualified to earn the HIP incentive. Presenting results for other fruit and vegetable 

aggregates (white potatoes, legumes, 100% fruit juice, and other fruits and vegetables purchased 

outside stores) allows us to assess differences in intake of foods that did not qualify for the incentive. 

We find that TFV intake was higher among those who kept track of their HIP rebate earnings (Exhibit 

I.14), though there were no statistically significant differences in TFV intake by awareness of HIP 

(Exhibit I.11) or by the other two understanding measures (Exhibits I.12 and I.13). 

Interestingly, intakes of 100% fruit juice and of legumes were significantly higher among those who 

said they found it difficult to remember how HIP worked (Exhibit I.12). This finding would be 

consistent with confusion on the part of participants about exactly which foods qualified – that is, 

those that were confused about how HIP worked might think that juice and legumes earned the rebate. 

However, TFV intake was also lower in this group, though not significantly so. Thus, it may be that 

the types of respondents who had difficulty understanding HIP also had lower fruit and vegetable 

intake more generally for unrelated reasons. Additionally, we did not observe a statistically 

significant difference in intake of 100% fruit juice or legumes by whether or not the participant 

specifically reported that it was hard to remember which foods qualified to earn the rebate (Exhibit 

I.13). 

Discussion 

On balance, and subject to the important caveats noted above, these exploratory findings provide 

some limited evidence that confusion about HIP was associated with lower spending on TFVs, as 

well as higher intake of some categories of fruits and vegetables that did not qualify to earn the HIP 

incentive, though the evidence on the latter point is even more mixed. Where appropriate in the main 

text, we make reference to this exploratory evidence to provide further context for key findings. 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary NonExhibit I.11: Differences in Consumption of Fruits & Vegetables and Disaggregated Components, Cup-Equivalents, by A

Group 

Regression-adjusted mean (S.E.) Differe

Heard of HIP 

Never heard of 

HIP Difference [S.E.] 

Targeted Fruits & Vegetables (preferred restrictive proxy) 1.099 (0.048) 1.208 (0.075) -0.109 [0.087] Plus TFV from mixed foods 0.306 (0.016) 0.316 (0.021) -0.010 [0.026] Targeted Fruits & Vegetables (alternative inclusive proxy) 1.405 (0.051) 1.524 (0.080) -0.119 [0.092] Plus additional components 100% fruit juice 0.563 (0.038) 0.476 (0.039) 0.087 [0.056] White potatoes 0.337 (0.019) 0.353 (0.031) -0.015 [0.036] Legumes 0.102 (0.008) 0.111 (0.013) -0.009 [0.016] Other fruits & vegetables acquired outside stores 0.168 (0.019) 0.167 (0.026) 0.001 [0.033] All fruits and vegetables 2.575 (0.073) 2.630 (0.102) -0.055 [0.125] 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store, excluding white potatoes, legumes, and restrictive proxy measure additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was not idealternative inclusive proxy measure includes fruit and vegetable intake from all mixed foods. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10% second-day subsamples for ea

recalls from 979 respondents). 
Evaluation of the Healthy In

pg. 550 ▌ Appendix I: Mediator Analysis Exhibit I.12: Differences in Consumption of Fruits & Vegetables and Disaggregated Components, Cup-Equivalents, by S

of How HIP Works, Treatment Group 

Regression-adjusted mean (S.E.) Differe

Easy to 

understand 

Not easy to 

understand Difference [S.E.] 

Targeted Fruits & Vegetables (preferred restrictive proxy) 1.092 (0.053) 1.256 (0.084) -0.164 [0.100] Plus TFV from mixed foods 0.302 (0.017) 0.344 (0.027) -0.042 [0.033] Targeted Fruits & Vegetables (alternative inclusive proxy) 1.394 (0.056) 1.600 (0.088) -0.206 [0.104] Plus additional components 100% fruit juice 0.484 (0.033) 0.658 (0.062) -0.174 [0.070] White potatoes 0.338 (0.020) 0.350 (0.031) -0.013 [0.038] Legumes 0.090 (0.008) 0.115 (0.012) -0.025 [0.016] Other fruits & vegetables acquired outside stores 0.158 (0.020) 0.159 (0.027) -0.001 [0.034] All fruits and vegetables 2.464 (0.077) 2.883 (0.119) -0.419 [0.144] 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store, excluding white potatoes, legumes, and restrictive proxy measure additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was not idealternative inclusive proxy measure includes fruit and vegetable intake from all mixed foods. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10% second-day subsamples for ea

recalls from 879 respondents). 
Evaluation of the Healthy Incentives Pilot (HIP): Final Report 

Abt Associates Appendix I: Supplementary NonExhibit I.13: Differences in Consumption of Fruits & Vegetables and Disaggregated Components, Cup-Equivalents, by S

of Which Fruits & Vegetables Qualify to Earn the HIP Rebate, Treatment Group 

Regression-adjusted mean (S.E.) Differe

Easy to 

understand 

Not easy to 

understand Difference [S.E.] 

Targeted Fruits & Vegetables (preferred restrictive proxy) 1.113 (0.053) 1.160 (0.066) -0.046 [0.083] Plus TFV from mixed foods 0.302 (0.018) 0.325 (0.021) -0.023 [0.028] Targeted Fruits & Vegetables (alternative inclusive proxy) 1.415 (0.058) 1.485 (0.070) -0.069 [0.088] Plus additional components 100% fruit juice 0.512 (0.035) 0.584 (0.048) -0.072 [0.059] White potatoes 0.343 (0.024) 0.345 (0.025) -0.002 [0.036] Legumes 0.102 (0.009) 0.104 (0.010) -0.002 [0.014] Other fruits & vegetables acquired outside stores 0.133 (0.019) 0.187 (0.024) -0.053 [0.031] All fruits and vegetables 2.505 (0.079) 2.704 (0.097) -0.199 [0.124] 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store, excluding white potatoes, legumes, and restrictive proxy measure additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was not idealternative inclusive proxy measure includes fruit and vegetable intake from all mixed foods. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10% second-day subsamples for ea

recalls from 925 respondents). 
Evaluation of the Healthy In

pg. 552 ▌ Appendix I: Mediator Analysis Exhibit I.14: Differences in Consumption of Fruits & Vegetables and Disaggregated Components, Cup-Equivalents, by W

Keeps Track of HIP Rebate, Treatment Group 

Regression-adjusted mean (S.E.) Differe

Tracks rebate 

Does not track 

rebate Difference [S.E.] 

Targeted Fruits & Vegetables (preferred restrictive proxy) 1.299 (0.077) 1.036 (0.047) 0.262 [0.088] Plus TFV from mixed foods 0.319 (0.022) 0.308 (0.018) 0.011 [0.029] Targeted Fruits & Vegetables (alternative inclusive proxy) 1.618 (0.080) 1.344 (0.051) 0.273 [0.092] Plus additional components 100% fruit juice 0.584 (0.055) 0.507 (0.036) 0.077 [0.068] White potatoes 0.364 (0.029) 0.341 (0.022) 0.024 [0.037] Legumes 0.098 (0.010) 0.106 (0.009) -0.008 [0.013] Other fruits & vegetables acquired outside stores 0.134 (0.023) 0.182 (0.020) -0.048 [0.031] All fruits and vegetables 2.798 (0.108) 2.480 (0.073) 0.318 [0.129] 

Two-sided test; *p<0.1, **p<0.05, ***p<0.01. Due to rounding, reported differences between subgroups may differ from differences between reported regression-adjusted means.

Targeted fruit and vegetable (TFV) intake proxy measures include intake of fruits acquired from the store, excluding white potatoes, legumes, and restrictive proxy measure additionally excludes fruit and vegetable intake from mixed foods where the source of individual ingredients was not idealternative inclusive proxy measure includes fruit and vegetable intake from all mixed foods. Standard errors and test statistics are adjusted for clustering at the individual respondent level. Source: Participant Survey (AMPM dietary recall module), pooled Round 2 and Round 3 responses, including 10% second-day subsamples for ea

recalls from 941 respondents).