Re-engineering the SIPP:

Creating the SIPP-EHC

Jason Fields – US Census Bureau

Session 5: Designs that use “event history” methodology to improve recall and 

reduce measurement error in recall surveys.

June 1-2, 2011 

Committee on National Statistics

Household Survey Producers Workshop 

Conference Center, 20 F Street, NW, Washington DC
The Save the SIPP Campaign

New York Times Editorials

Discovering What Happens Next

Save our SIPP

The Continuing Saga of SIPP 

NY Representative, Carolyn Maloney stated:

“I want to thank all of the policy groups and economists who 

worked so hard to help reverse the Administration’s original SIPP 

decision. Because of their dedication and hard work, the 

Administration came to understand how important SIPP is to 

creating and implementing good public policy.” 
SIPP Re-engineering

Implement Improvements to SIPP

- reduce costs

- reduce R burden

- improve processing system

- modernize instrument

- expand/enhance use of admin records

Key Design Change:

- annual interview, 12-month reference pd.,

Event History Calendar (EHC) methods
EHC Interviewing

Human Memory

- structured/organized

- links and associations

EHC Exploits Memory Structure

- links between the occurrence and timing of events

EHC Encourages Active Assistance to Rs

- flexible approach to help elicit an 

autobiographical “story”

- Accuracy in placement of events by respondents

- Improvements in ability for respondents to recall details and timing

- Facilitate the interviewer‟s ability to recognize inconsistencies and probe

SIPP-EHC: A „hybrid‟ Event History Calendar.
Evaluations of EHC Methods 

Many EHC vs. “Q-List” Comparisons

- various methods

- in general: positive data quality results

BUT, Important Research Gaps

- data quality for need-based programs?

- comparison to 4-month reference period?
6

• Event history calendar instruments evaluated on numerous occasions 

• Reliability – High levels of agreement with previously collected 

questionnaire based data. 

• Domains

– Marriage, births, residential moves, school enrollment, labor force 

(all showed good agreement)

– Concerns - AFDC/Food Stamps (SNAP)

• Mode - Phone and in-person almost same degree of consistency.

• Validation – Limited record check studies – designed into the first SIPP 

field test evaluation.

• Reduction of seam bias

Evaluations of EHC Methods 
Measurement Errors 

- Missed data due to:

- Recall

- Ineffective conversational interviewing

- Inaccurately reported data due to:

- Timing accuracy

- Detail accuracy

- Proxies

- Memory decay

- Seam Errors

- Dependent data

- Revising based on longitudinal edits or administrative data post-hoc
The First SIPP Event History Calendar 

A form of Dependent Data

8

SIPP calendar aided interview test

1989 to 1991
SIPP Re-engineering Field Test Plans

- Proof of concept test 

- 2008 paper and pencil reinterview test

- EHC CAPI test

- 2010 Integrated Blaise and C# instrument prototype

- CAPI Revised test

- 2011 Test improvements to the wave 1 instrument, 

training, and expand sample to all regional offices.

- 2012 Test wave 2 concepts and instrument, examine movers 

and attrition issues, and refine training procedures.
Re-engineered 

SIPP -

Contents
Screenshot – 2010 SIPP-EHC Instrument 

For 2011...

• Extend the reference period to „Now‟

• Clean up the instrument navigation tasks

• Transition spell level questions to Blaise
Screenshot – 2011 SIPP-EHC Instrument 
Screenshot – 2012 SIPP-EHC Instrument 
Combined Conversational and 

Conventional Probing...

• Focus on spells

• Sub-monthly data for labor force section 

collected based on delineated weeks 

(calendar flash card provided to R‟s)

• Scripted – repeated probes for „more work‟

– Based on 2010 information that navigating 

multiple job lines conversationally was not 

clear, and evidence from administrative data 

that jobs were under-reported.
Defining the Spells
Sub-monthly Time Definition.
Within Spell Changes in Earnings / Hours

Probing – Now Scripted For More Work
Probing – Now Scripted For More Work
• Gaps calculations embedded into 

questionnaire flow.

• And one more scripted probe about work


• Completeness checks for „conversational‟

component

• Very useful in standard interviewing to 

reduce seam bias and respondent burden.

• Very burdensome to the instrument and to 

processing.

• Design, How to effectively build a calendar 

based interview with dependent data

Dependent Data
2011 SIPP-EHC Completed Calendar
EHC Dependent Data
2012 SIPP-EHC Calendar w/ Dependent Data
• Feedback will both utilize existing question 

structure and alternative fills to script 

questions based on the presence or 

absence of returning data.

• Evaluation and comparison of 

2010 SIPP-EHC data (in progress)
35

Food Stamp Participation 

Probability and 

Monthly Time Trend in 

Participation

2010 SIPP-EHC and

2008 Panel data matched in 

MSIPP
36
37
Challenges Faced

Developing new technical capacity.

- experimentation

- limitations

- evolution

Crisis planning.

- limited lead time / preparation

- changing goals and required flexibility

New procedures.

- training and acceptance

- development and refinement of procedures
Current Status

• Interviewing for 2011 SIPP-EHC in all 12 regional office 

areas (20 States) is complete.

• Reviewing and making changes to content and design 

(Dependent Interviewing for 2012 – Wave 2).

• Reviewing and revising training materials and methods

• Planning for 2013 Research Panel design and 

implementation

• Planning for 2014 Production Implementation of SIPPEHC
Assessing Users’ Needs

URL: http://www.census.gov/sipp