FCSM Session: 

Using Administrative Data to Examine 

Food Assistance Program 

Effectiveness 

Discussant	Remarks	

Constance	Newman,	USDA-FNS	
Outline

• Introduction	

• Findings	from	Cunnyngham	et	al,	Wheaton	et	al.	

• Admin,	QC,	and	Survey	data	quality	issues	

• SNAP	unit	composition	and	modeling	assumptions	

• SNAP	eligibility	and	access	rates	at	Census	

• SNAP	and	WIC	

• Future	research	
Findings on Data Quality 
Findings on Admin, QC, and Survey data: 

Cunnyngham et al. 

• Overall,	nice	to	see	the	NDB,	QC,	and	State	admin	data	closely	tracks.	

• NDB	vs.	State	files:	IL	exception:	strange	differences	between	NDB	

and	IL	state	data	around	2013.	Do	we	know	the	reason?		

• Why	would	the	benefits	be	lower	in	the	IL	state	data?	

• Are	those	2013	data	used	in	the	SNAP	unit	analysis?	

• QC	vs.	State	files:	Differences	in	age	and	case	type	distributions	imply	

issues	with	edits	or	actual	differences.	

• Shows	that	state	admin	data	have	issues;	they	aren’t	necessarily	the	

“gold	standard,”	but	the	QC,	NDB,	and	State	files	are	quite	consistent.	
Wheaton	et	al.	compare	across	MATH+	and	TRIM3	and	two	other	models.		

• They	find	that	certain	unit	subgroups	have	participation	rates	of	100	percent	or	

more	across	all	microsimulation	models	and	data	sources	examined:	

• Cases	with	single	adult	with	children,		

• Cases	with	countable	income	below	50	percent	of	the	poverty	guideline,		

• Cases	eligible	for	between	76	and	99	percent	of	the	maximum	benefit	for	

their	case	size.	

These	subgroups	also	have	high	participation	rates	in	the	MATH	CPS	estimates:		

• Cases	with	one	person,	

• Child-only	cases,	and		

• Cases	with	adults	aged	18	to	49	without	disabilities	in	childless	households

Findings on Admin, Models, and Survey 

data:
Findings from QC data: 

Cunnyngham et al. 

From	Slides	8,	9,	10:	Select	household	

characteristics	of	SNAP	cases:	

IL	 MS	 TN	

One	adult	+	children	 24	 29	 27	

Two	or	more	adults	+	children		 9	 14	 15	

One	member	(adult	or	child)	 56	 48	 52	

No	earnings	 53	 49	 52	

!The	bolded	groups	are	also	the	ones	with	the	overly	high	participation	rates.	

Bolded	groups	represent	large	

shares	of	SNAP	cases	overall.	

No	earnings	group,	not	mutually	

exclusive,	but	the	top	three	are.		

Shares	of	select	SNAP	case	types	in	the	QC	data	from	each	state	
Subgroup Shares of Admin Cases in Linked 

Data (Wheaton et al.) 

• One	adult	with	

kid(s)	cases	

represent	high	

share	of	all	SNAP	

admin	cases	

(previous	slide),	

and	the	linked	

sample	

underrepresents	

them.	

• Similar	situation	

with	Adults	with	

no	children:	high	

share	but	low	

representation	in	

linked	data.	

Source:	Slide	12,	Wheaton	et	al.	

Share	of	SNAP	Cases	Represented	in	Linked	Data,	2012	to	2016
CPS ASEC Interview Status by SNAP 

Case 

Wheaton et al. 

• Interesting	findings:	

• Slide	8:	Multiple	adults	with	kids	had	a	much	higher	rate	of	getting	

interviewed	among	those	sampled	(72%	compared	to	64%	for	all	types),	and	a	

higher	rate	of	whole	imputes	(18%).	

• Slide	8:	One	adult	with	kids	and	one	adult	cases	had	much	lower	Whole	

Impute	rates	(9%	each	compared	to	14%	for	all	types).	

• Slide	9:	One	adult,	with	income	less	than	50%	FPL	has	much	lower	rate	of	

getting	interviewed	(56%	compared	to	64%)	than	any	other	group	including	

other	types	below	50%	FPL.		

• Imputation	rate	is	also	lower	than	average	for	this	group,	suggesting	interviewing	is	the	

problem.	

• Upshot:	Imputes	don’t	explain	the	low	shares	of	One	adult	+	kids	or	

One	adults	or	Very-low-income	cases.	
Findings on SNAP Unit Composition 
Comment on Estimation Issues 

• Note	that	the	subgroups	with	overly	high	estimated	participation	

rates	are	the	most	disadvantaged	and	thus	most	likely	to	participate.		

• Mathematically,	the	high	participation	levels	affect	the	numerators	in	an	

expected	way,	but	the	estimates	of	eligible	units	of	these	subgroups	should	be	

easy	to	estimate.	Income	wouldn’t	be	close	to	the	threshold,	as	the	most	

obvious.	

• But	also,	because	of	the	high	participation	levels,	there	are	fewer	eligible	nonparticipants,	making	these	participation	rates	more	likely	to	be	“high”	and	

thus	go	over	the	100%	mark.	

• Even	more	reason	to	think	these	unit	types	are	missing	in	survey	data.	

• Still,	it	points	to	the	importance	of	correctly	identifying	these	case	types	as	

separate	units	from	other	household	members,	ineligible	or	not.	
• Major	Finding	on	Children	by	Cunnyngham/Czajka	et	al:		

• CPS	ASEC	is	widely	found	to	have	low	representation	of	children	

• Wheaton	et	al.,	Czajka	et	al.,	Meyer	et	al.	

• They	find	that	the	gap	is	sufficient	to	fully	explain	the	low	eligibility	estimates.	

• Multi-unit	cases	found	to	be	small	share	of	SNAP	cases		

• But	less	clear	findings	on	other	groups:		

• One-adult	+	child	units	

• Lack	of	children	in	the	survey	would	imply	there	would	too	many	One-adult	units,	but	

that’s	not	the	case.	

• One-adult,	non-elderly	

• No-income	or	0	–	50%	FPL	

Findings on SNAP Units 
One-adult	+	children	cases	are	found	to	be:	

• Less	likely	to	be	interviewed	when	part	of	the	CPS	ASEC	sample,	though	

less	likely	to	be	imputed.	(multiple	adult	with	children,	more	likely)	

• Less	likely	to	be	found	in	the	linked	data	as	well.	

• Among	these	cases,	the	TRIM3	estimates	found	(slide	17):	

• 52%	were	found	to	be	eligible	and	One-adult	+	children	cases.	

• 21%	were	found	to	be	eligible	but	another	type	of	case.	

• 20%	were	found	to	be	ineligible	and	another	type	of	case.	

• 7%	were	found	to	be	ineligible	and	One-adult	+	children	cases.	

• Blue	highlight	shows	that	42%	were	either	modeled	incorrectly	or	

participants	are	not	on	their	application,	including	spouses	who	should	be.		

Findings: One-adult + children SNAP 

cases 

Wheaton et al. 
Findings on One-adult + children SNAP 

cases: Wheaton et al. (cont.) 

Slide	18:	Inconsistencies	in	linked	data	for	

One	Adult	+	Children	cases/households	

Case	child	or	adult	not	in	ASEC	HH	 28%	+	15%	=	43%	

ASEC/TRIM3	spouse	not	in	Case	Unit	 20%	

ASEC/TRIM3	partner	not	in	Case	Unit	 28%	

ASEC/TRIM3	other	adult	not	in	Case	Unit	 15%	

The	first	row	suggests	survey	data	issue.	

The	second	row	is	a	potential	issue		

• of	a	SNAP	applicant	not	including	their	

spouse	as	they	should,	OR	

• The	spouse	is	not	eligible	(immigrant	

or	other	reason)	

The	third	row	could	be	similar	to	spouse.	

And	the	fourth	is	likely	a	modeling	issue.	

Upshot:	Need	to	understand	why	so	many	of	these	survey	households	have	more	people	than	in	the	

matching	admin	cases.		
How do linked SNAP participants look? 

Are they modeled as eligible?

• Wheaton	et	al.	examine	whether	matching	SNAP	cases	are	estimated	

as	eligible	or	not	in	TRIM3	and	to	explore	the	implications	of	

imputation.	

• ~75%	of	matches	are	eligible.	(73%	IL,	78%	MS,	73%	TN)	

• Eligible	units	are	more	likely	than	Ineligible	units	

• to	completely	match	SNAP	cases	on	all	members;	and	

• to	not	use	imputed	survey	data.	

• This	is	good	news.	However,	I	wish	the	estimates	of	eligibility	could	be	

better	aligned	with	the	admin	records	of	participation.		

• And	would	have	been	nice	to	compare	to	MATH+	estimates.	
Using Administrative Data to Examine CrossProgram Participation in SNAP and WIC 

Hodges et al. 

• Great	plan.	We	need	more	information	on	WIC	participation	in	other	

programs,	and	SNAP	is	a	good	place	to	start,	especially	since	children	are	

likely	to	be	covered	by	SNAP	and	yet	we	see	the	WIC	drop-off	as	children	

age.	

• Be	ready	for	double	trouble	in	matching	to	both	SNAP	and	WIC.	

• Why	only	three	states?	You’ll	need	as	much	data	as	possible	to	get	at	the	

numbers	in	both	programs.	

• Longitudinal	analysis!	Yay.	May	need	more	than	one	state.	

• Maybe	you	can	not	include	the	survey	data	and	link	on	SNAP	and	WIC	PIKs	to	

understand	the	dynamics.	
WIC admin data research 

• If	recent	data	become	available,	impact	of	COVID	on	participation	in	

both	programs	would	be	good	to	look	at.	

• Since	WIC	participants	didn’t	have	to	visit	offices,	that	may	have	made	it	

easier	to	participate.	

• WIC	eligibility	estimates	would	benefit	from	better	understanding	of	

monthly	income	changes.		
SNAP Eligibility and Access Rates 

Bhaskar et al.

• It	is	so	great	to	see	how	this	work	has	developed	and	become	a	

regular	product	provided	to	States	that	share	their	SNAP	data.		

• It	may	be	time	to	revisit	the	modeling	assumptions	given	the	insights	

raised	by	recent	studies	including	those	in	this	session.	

• Multi-units:	Though	these	studies	are	focused	on	linked	CPS	ASEC	data,	the	

general	insights	about	multi-units	should	be	re-examined.		

• And	similar	analysis	of	linked	ACS	SNAP	admin	data	is	warranted.	

• Are	children	and	members	of	lowest	income	households	also	missing	from	ACS	survey	

rosters?		
Future Research

• Income	dynamics	

• How	best	to	model	income	for	eligibility	purposes	when	we	have	(poor)	

annual	income	data	in	the	CPS	

• SNAP	unit	modeling	

• Compare	simulation	methods	more	directly	across	models

• Multi-units:	what	is	best?	

• Need	bigger	samples	

• PIK	error	implications	

• What	can	we	say	about	the	CPS/ACS	individuals	who	don’t	have	a	PIK?	

• Extend	the	work	matching	on	sex	and	age	for	those	who	don’t	have	PIKS.	
Future Research

• Survey	representation	of	low-response	groups	

• Expand	the	analysis	to	more	years	and	states,	and	extend	to	ACS	

• Admin	data	quality	(QC	and	State	files)	

• Add	information	to	SNAP	cases	about	ineligible	unit	members	when	they	are	

known	(as	suggested	by	Wheaton	et	al).	

• Match	CPS	ASEC	households	to	their	full	prior	year	of	SNAP	case	data	

to	see	how	case	composition	and	other	variables	changed	over	the	

year	(also	suggested	by	Wheaton).	

• How	does	that	inform	the	eligibility	model?		

• Pull	in	SNAP	data	with	income	data,	if	possible.	
THANKS	

Constance.newman@usda.gov	
SNAP-Eligible Units 

• We	should	keep	in	mind	that	when	using	the	household	composition	

of	SNAP	cases,	we	aren’t	looking	at	the	true	distribution	of	eligible	

participants	and	non-participants.	

• What	might	the	differences	be?		

• Most	Non-participant-Eligible	units	are	probably	closer	to	the	eligibility	threshold:	

• Higher	monthly	income,	more	adults,	those	eligible	for	only	a	small	benefit	

• Could	this	partially	explain	the	apparently	lower	rates	of	more-likely-toparticipate	cases	among	the	estimated	eligible?		
Subgroups in linked data by other unit 

members by type 

Czajka et al. 

Illinois	data	

Tables	8,	9,	10	

Child	+	one	adult	 Children	+	no	

earnings	

One	adult	 No	earnings	

Unrelated	individual	

or	subfamily	

17.6%	 20.7%	 14.7%	 14.2%	

Related	subfamily	 19.5%	 18.3%	 6.1%	 9.3%	

Unit	=	Case	 50.0%	 64.6%	 57.0%	 57.0%	

Rough	total	 87.1%	 103.6%	 77.8%	 80.5%