Background and Planning for Incorporating an Event History Calendar 

into the Re-Engineered SIPP. 

Jason Fields 

Mario Callegaro 

Jason Fields – US Census Bureau, Housing and Household Economic Statistics Division 

Jason.M.Fields@Census.GOV

Mario Callegaro – Knowledge Networks 

This report is released to inform interested parties of research and to encourage discussion of work in progress. 

The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. 

Introduction 

The Census Bureau is currently re-engineering the Survey of Income and Program Participation (SIPP). The reengineered SIPP will be an integrated data system with multiple components and methods: a survey-based data 

collection; an American Community Survey-based sampling frame; administrative records; and statistical 

modeling. Together, these components will yield a fuller, stronger, more richly detailed program of estimates of 

economic well-being. The re-engineered SIPP survey component will be evaluated to incorporate improvements to

the current SIPP. The survey’s mission is to provide a nationally representative sample for evaluating: 1) annual 

and sub-annual income dynamics, 2) movements into and out of government transfer programs, 3) family and 

social context of individuals and households, and 4) interactions among these items. These survey objectives are

pursued in the context of several goals - cost reduction and improved accuracy, relevance, timeliness, and 

accessibility. 

We present background information about the methodological choices that led to the current design. The reengineered SIPP survey is being designed around annual data collection that collects sub-annual dynamics in 

income and government program use. Annual survey administration is the primary source of significant cost 

savings compared with prior SIPP panels. Moving from the current design, with interviewing 3 times per year, to

an annual survey, raises significant concerns about recall accuracy and the ability of respondents to report the subannual dynamics SIPP traditionally measured. The current preferred option is to design a new survey based on

more streamlined content, minimal topical module-based content, and to utilize an Event History Calendar (EHC) 

module to improve respondents’ ability to recall the sub-annual dynamics the survey is tasked with collecting. We

discuss the background for choosing an EHC module, additional design considerations, and propose 

methodological evaluations that may be incorporated into strategies for evaluating EHC collected data.

Background 

The main objective of the SIPP has been to provide accurate and comprehensive information about the income and 

program participation of individuals and households in the United States. The SIPP collects detailed information

Confidential Page 1 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

on cash and non-cash income (including participation in government transfer programs) three times a year, and 

detailed data on taxes, assets, and liabilities are collected annually. A major use of the SIPP has been to evaluate 

the use of and eligibility for government programs and to analyze the impacts of options for modifying them. 

The Social Security Administration, for example, relies on SIPP data to project baby boomers’ retirement incomes 

and the likely timing of their retirement. The Department of Health and Human Services uses SIPP data to evaluate 

the impact of Welfare Reform and to measure the economic effect of disabling conditions on children and adults. 

The Department of Agriculture uses SIPP data to understand who is eligible for, but not participating in, assistance 

programs like Food Stamps, and to assess alternative eligibility requirements. Other SIPP stakeholders include the 

Small Business Administration, the Department of Labor, the Congressional Budget Office, and the Congressional

Research Service. While the SIPP has been used to research alternative measures of poverty, the official measure 

comes from the Current Population Survey (CPS). The American Community Survey (ACS) also provides some

data on poverty for small geographic areas. 

The Census Bureau’s continual efforts to improve its SIPP data collection program led to the decision to reengineer 

the SIPP. The re-engineered SIPP has had many names over the past year and half (Program of Income, Wealth

and Health Insurance Measurement, and Dynamics of Economic Well-Being System – DEWS are the most

referenced), but in this paper we will just refer to the re-engineered SIPP for simplicity. The SIPP’s longitudinal 

household design provides many advantages; it also imposes considerable burden on respondents and makes data

processing and review difficult, leading to long delays before the data can be released and analyzed. Furthermore,

the SIPP(like virtually all long-term survey programs) has been experiencing declining response rates. In 1996, 

nonresponse to the first SIPP interview was 8 percent; the 2001 rate was 13 percent; and the 2004 rate was 15 

percent. The Census Bureau has wanted to reengineer the SIPP for some time. Reengineering efforts to convert

SIPP to a new instrument language and new processing system were underway during 2005, however at the end of

that calendar year this work paused. Changes in resources necessitated new plans for SIPP. In early 2006 meetings 

were held with key federal stakeholders to discuss possible directions for the future collection of data that had been

part of SIPP, taking advantage of this time to consider other options to re-engineer the SIPP. 

The design goals for the re-engineered SIPP are focused on offering policymakers and researchers data that address 

the same basic issues as SIPP, and that can be used, in part, as SIPP data have been used, that is, to provide a 

nationally representative sample that can be used to evaluate the annual and sub-annual dynamics of income, the 

movements into and out of government transfer programs, the effect on family and social context of individuals and

households, and the interaction among these items. The improvements to SIPP were directed to streamline the 

collection and production systems, provide similar information at a reduced cost, with improved data quality, 

improved timeliness, and improved data accessibility. Several important decisions were made based on these 

directives; 1) The design of the data system would try to capitalize on existing data collection programs and 

simplify new collection systems to reduce cost, 2) the data processing system would be overhauled to eliminate the 

patches and workarounds that have accumulated over the years, 3) the structure of the data file presented to the 

public would be simpler, 4) and timeliness would be significantly improved through redesign of the processing

system and simplification of the data collection schedule.

The current SIPP program administers three waves or interviews in each year. The interview rotation design means 

that data are collected in each of the 12 months of the calendar year in order to level field resource requirements. 

This requires a constant level of production-readiness that encumbered several components of the survey

production and review processes, contributing to data products that were not timely, were error-prone, and were

difficult to use. 

Proposals for the re-engineered SIPP 

The process of developing alternatives for the current SIPP design was open and included many ideas that were

reviewed and dismissed before settling on a couple paths to develop more fully. Initially, following the decision to 

re-evaluate the current status and direction of SIPP, several options were brought forward based on data sharing 

and modeling advances. From January 2006 through April 2006, meetings were held with several key federal 

stakeholders, discussing several plans that could have provided economical alternatives to continuing an extensive 

and complex survey like SIPP. While the same domains as traditional SIPP would be present, compromises would

necessarily be required in terms of the scope and strict comparability to prior SIPP data. In June of 2006 the 

Confidential Page 2 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

leading options for a composite improvement to SIPP were presented in an open meeting at The Brookings 

Institution (see http://www.sipp.census.gov/sipp/dews.html). We will briefly discuss these alternatives, but return 

to focus on the methodological justification for our decision to pursue development of an instrument which utilizes 

an event history calendar to facilitate the collection of subannual information with an annual survey. 

There were several key components to the alternatives discussed. Throughout the first half of 2006 the discussion

of alternatives to SIPP were guided by a primary emphasis on cost reduction. The components include use of

existing survey data, that is data collected by another survey which could be used to provide information on SIPP

topics; administrative records as a replacement for SIPP type survey data; modeling or synthetic data; a simplified 

new survey; and combinations of the above. 

Using existing survey data collected as part of another data collection program was a first response to quickly 

replacing content that was collected by SIPP. Data for a number of SIPP topics are collected in the Annual Social 

and Economic (ASEC) supplement to the Current Population Survey (CPS). The CPS represents a monthly,

ongoing data collection, whose monthly sample design includes a subset of household addresses that were 

interviewed the preceding year. Taking advantage of this year-to-year overlap in the CPS-ASEC, about 40% of 

respondents would be available in adjacent years. The data from the respondents in both years would have 

represented the foundation of the longitudinal data. Geographic coverage in the American Community Survey

(ACS) far exceeds that which was possible in the SIPP. Utilizing the advances in image processing, forms 

handling, and infrastructure that supports the ACS could also be significant sources of cost savings and represent

additional returns for the development of mail-out supplemental data collections. These advances in survey 

administration and processing which enable the ACS forms to be quickly processed and coded could be adapted 

regardless of whether the CPS or the ACS was used as the foundation. Data collected in either the CPS or the ACS 

could then be supplemented with a follow-on survey that could include additional topics that could have expanded

the breadth of the data to include necessary domains that are not part of the CPS-ASEC or ACS data collections, as 

well as lengthening the period for which longitudinal data were available for these households. 

Administrative records data are data collected to serve an administrative purpose. The records under consideration

for use in this program come from federal or state government offices charged with administering programs or 

policies. Primary sources of federal administrative records data include the Social Security Administration and the 

Internal Revenue Service. There are many other sources of federal administrative records data, and where content 

in those records overlap with SIPP content, discussions were initiated to explore the possibility of agency 

partnerships which would facilitate obtaining access to those data. A significant portion of the program related 

content that users would like to see on any replacement or improvement to SIPP does not come from federal 

agencies, but from state agencies. In these cases distinct agreements for use and access to these records need to be 

negotiated with each state agency responsible for those data. An additional complication is the wide variability in

data format and quality from one state to another. The states have different levels of sophistication with their data 

collection and management systems. 

Proposals to use administrative records to replace, validate, or supplement data collected in SIPP are not new. 

Since the beginning of SIPP, internal SIPP personnel and external stakeholders and advisory groups (like the 

National Academy of Sciences) have recommended the use of administrative records for these purposes. In June 

2006, at a presentation at the Brookings Institution, our stakeholders expressed concerns about a lack of usefulness

of a survey enhanced with administrative records, citing comparability of measures, coverage differences, matching

rates, synthetic data concerns, and confidentiality issues. Additionally, the likelihood of a data gap and the 

potential length of the gap between the end of the current SIPP data and a possible improved SIPP became a 

significant issue. The importance of minimizing this data gap with data more similar to SIPP than could be 

generated from the existing data and administrative records became clear. By August 2006, we had refocused to 

utilize administrative records as a supplement to a new survey that could deliver more of the content stakeholders 

needed from SIPP. Administrative records will play a key role in validation activities for the test and dress 

rehearsal data collections; serve as a supplement to survey data in the editing process, for modeling activities, and

for evaluation activities. At this time a contract was also initiated with the National Academy of Sciences’ 

Committee on National Statistics (CNSTAT) to sponsor a panel that could advise the Census Bureau of the best use 

of administrative records in the new system.

Confidential Page 3 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Following the input from stakeholders and the outcry at the possible loss of SIPP data, we began development of a

new survey that would fundamentally redesign the collection of content collected by SIPP. After considering the 

existing 4 month recall period, or a six month or an annual recall period and the costs associated with each, we 

eventually opted to explore annual survey contacts in the household with a reference period long enough and

detailed enough to capture retrospective information on sub-annual dynamics. Choosing to focus on repeated

annual survey administrations provided the best source of cost reduction while continuing to include an

interviewer-administered survey-based data collection component. Addressing the issues raised by the stakeholders

with respect to the presence and size of the data gap were the driving force for moving through this process quickly 

and expediting the design and plans for a replacement data collection. There were insufficient funds and time to

design and implement a standard overlap and validation development process while also minimizing the data gap

and meeting stakeholders’ needs. 

The use of an event history calendar was not the only option discussed, nor was it even the first. Once the decision

was made that major cost savings would be achieved through reductions in field costs with an annual survey, and

that the content requirements still included subannual dynamics in jobs, programs, health insurance coverage, and

demographics, methodologies that could facilitate this were considered. The primary concern is that in an annual 

administration, spells on programs and the amounts received would be difficult to recall. Using traditional SIPP

type questionnaire techniques over this length of time would have certainly been deficient for many items. A 

literature review of survey methods related to improving respondent recall clearly points to the use of event history 

calendars as survey tools that facilitate better recall in nearly all cases examined. 

Event History Calendars 

The use of an event history calendar (EHC) to capture sub-annual transitions is a relatively new feature in

automated demographic survey instruments, The Panel Study of Income Dynamics (PSID) conducted by the 

University of Michigan, the Health and Retirement Survey, as well as the labor surveys of Statistics Canada (SLID) 

and Statistics Netherlands currently use or are planning to use this methodology, which ties life events to program

participation, health insurance, and job transitions. This type of methodology, described in more detail below, 

facilitates the use of annual instrument administration to collect sub-annual estimates rather than three interviews at 

four-month intervals, as is the current practice with SIPP. This was another clear advantage of the 12 month EHC, 

enabling the use of the same reference period across items, and further streamlining the instrument from one wave 

(year) of data collection to the next. Recent research also shows that EHC collected data can decrease seam effects 

between waves compared with traditional questionnaire interviewing, and also potentially reduce seam effects 

between different components of an the same instrument (Callegaro, 2007). Additional work is needed to evaluate 

whether additional reductions in seam biases can be realized by combining dependent interviewing with EHC

methodologies.

Calendar-based data collection originated in the late 1960s (Balán, Browning, Jelin, & Litzler, 1969; Blum, 

Karweit, & Sørensen, 1969) and was then rediscovered by Freedman and colleagues (1988) who called it the Life

History Calendar (Belli & Callegaro, in press-a). Authors have used inconsistent terminologies to refer to the same 

interviewing strategy. For the purpose of this discussion we will not enter into the terminological details but refer to

it as Event History Calendar or EHC. The interested reader can refer to Callegaro (2007) and Belli and Callegaro

(in press-a) for more details. 

An EHC interview is centered on a customized calendar that shows the reference period under investigation 

(Axinn, Pearce, & Ghimire, 1999). The calendar contains timelines for different domains, for example work 

history, residence history, household composition and other domains relevant to the topic of the study. Landmark 

events, such as holidays and birthdays can be used to aid the respondent’s memory (Belli, Shay, & Stafford, 2001) 

but not every researcher employs them (Axinn, Barber, & Ghimire, 1997). The interviewer guides the respondent in 

answering queries for each time line, starting with the first domain and continuing through all of the domains until 

the EHC is completed. The EHC also allows moving back and forth among the domains. The process uses previous

domain information and dates to help the respondent correctly place other events in the appropriate time frame. 

When, for example, unemployment history is queried, the respondent can use retrieval cues from landmark events, 

residential history, and household composition to retrieve the spell in which any unemployment had transpired. For 

instance, an unemployment spell can happen just before a move to a new location, or right after a pregnancy. The 

calendar does not necessarily have to be shown to the respondents. In the literature there are successful applications 

Confidential Page 4 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

using both face-to-face (Axinn, Barber, & Ghimire, 1997; Parrado & Zenteno, 2001) and telephone interviews 

(Belli, Shay, & Stafford, 2001; Belli, Smith, Andreski, & Agrawal, 2006). In a randomized assignment study, 

Callegaro and colleagues (2005) found initial support that showing the calendar improves the quality of reports in 

comparison to a completely auditory presentation (checked against validation data). In a computerized selfadministered panel, respondents were given a summary screen in the form of a calendar based on previous

responses collected with a conventional questionnaire (CQ) about major events in various domains of their life. 

After the summarized calendar screen the respondents were asked to make any necessary corrections. Twenty one 

percent of respondents corrected the order of the events and seventy nine percent corrected the dates after seeing

the information summarized on the calendar (Saris, 1998). 

The units of analysis are decided by the researcher and have included years (Axinn, Barber, & Ghimire, 1997; 

Furstenberg, Brooks-Gunn, & Morgan, 1987), months (Caspi et al., 1996; Engel, Keifer, & Zahm, 2001), and thirds

of a month (Belli, Shay, & Stafford, 2001; Pebley, 2004a). In some administrations, interviewers are encouraged to

permit a chronological order of retrieval at the respondents’ choosing such as moving forward in time, backward in

time, or across contemporary events. Every event or “spell” is entered into the time lines with a starting point and

an ending point as defined by the unit of analysis. In case the respondent remembers the exact dates those can be 

entered into the timeline or are converted directly into the unit of analysis. If those are not available, the interviewer 

is instructed to probe for a time location depending on the unit of analysis. In the case of thirds of a month, for 

example, the interviewer will ask if the event occurred at the beginning of the month, in the middle, or at the end of 

that month. 

Each spell can happen before, after, or contemporaneously with another spell. For example, a relocation event can 

occur contemporaneously with a new job event. The entire process of compiling the calendar focuses, by its nature, 

on coherence, consistency, sequential order, and attempts to correct for missing data (Belli, 1998). By coherence, it 

is intended that some events are less likely to occur together, such as having three jobs at the same time. Although 

that might be the case, it is also possible that the respondent made a mistake in the location of the jobs in time. The 

calendar instrument visualizes better than a CQ instrument events in the time line suggesting possible 

inconsistencies. The sequential nature of the EHC is revealed by the fact that an event should happen after the

preceding one and before the following one. For example if the respondents are unemployed, they will then look

for a job, and when found, become employed. The time line also highlights to the interviewer missing data in a 

more prominent way than a CQ. This is because each event in a time line should be adjacent to another event 

without any time unaccounted for (something must have been happened in each time frame). In case of time

unaccounted for, the interviewer can probe to investigate what happened in that time frame. 

The instrument is organized by domains (rows) and by months (columns). The calendar being designed for the reengineered SIPP will collect information for a one-year reference period and the units of analysis are months, with 

the exception of employment and unemployment, which are thirds of a month. It starts by asking for some

landmarks events; some of them are already filled in such as national holidays (e.g. Labor Day and Thanksgiving). 

In addition to the temporal placement of spells during the reference year, the EHC section of the instrument will 

collect specific details that are pertinent to each spell. For example in the residence history section of the EHC, a 

full address is collected for each residence to enable geographic coding and migration research. In addition, the 

type of living quarters, the tenure status (own/rent), residence in subsidized housing, and voucher receipt at this 

address are recorded. Across domains one of the characteristics that we are paying additional attention to is the 

starting date for spells. In the case of residences, the residence that is reported for January of the reference year will 

also generate a question for the date moved in. In designing this calendar component for the re-engineered SIPP, 

the Census Bureau staff has been informed by the work on the PSID by Belli et al. (2001), prior work on event 

history calendar use in SIPP by Kominski (1990) and by the work of Freedman and colleagues (1988). 

The calendar is accompanied by a questionnaire that provides a structure, suggests question wording and probes. 

The questions can be very detailed and standardized (Zahm et al., 2001) – as will be the case with the re-engineered

SIPP – or more narrative and open ended (Blum, Karweit, & Sørensen, 1969). The common denominator of EHC

studies is that the interviewers are not required to follow the question order exactly, but they are allowed to deviate 

from it with a series of probes and inquiring techniques that are adapted to the respondent’s way of remembering. 

The instrument’s flexibility promotes the recording of very complicated life histories, with the benefit of allowing

the respondents to remember the events in the chronological order that they prefer. Although the literature about 

which order of retrieval (forward or backward in time) leads to the best reports is still inconclusive (Bhandari & 

Confidential Page 5 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Wagner, 2006), some studies on the quality of order of retrieval suggest that when allowing respondents to choose 

their own order in comparison with suggesting an order of retrieval provides better data (Jobe et al., 1990; Loftus & 

Fathi, 1985). The EHC instrument does not necessarily have to be used alone. In fact, it is frequent that it is part of 

a longer instrument in which a CQ component is administered before and/or after the calendar (Belli, 2003; Pebley, 

2004b; Witterbrood & Nieuwbeerta, 2000). 

Quality of EHC Data 

Since its first applications, researchers have been examining the quality of data obtained by EHC interviewing and 

how it addresses the common problems of recall error in survey research (Belli, 1998). Freedman and colleagues 

(1988) published the first paper dealing with quality of EHC data by comparing retrospective EHC reports with

contemporaneous verbal reports collected from the same panel respondents five years earlier. For events such as

marital status and births the agreement was above 95% with discrepancies of one month. For school attendance and 

employment status measured on a 3-category scale (full-time, part-time, nonattendance/unemployment) the

reliability was .90 and .62 respectively (Alwin, 2007). Caspi et al. (1996) found over 90% agreement between EHC 

data and data reported in the same month three years earlier for living arrangements, cohabitation, schooling, 

employment, and job training in a longitudinal-epidemiological study. Similar rates of agreement with data 

collected 15 years earlier were found by Ensel and colleagues (1996) in a longitudinal study. Lin, Ensel & Lai 

(1997) asked 1993-95 Albany panel respondents to recall life experiences that happened in 1979-82 using Event

History Calendar data collection. They then compared the EHC reports with the contemporaneous reports of the 

same respondents for CQ data collected nearly 15 years before. The comparisons were done for ten life event 

categories such as marital changes, births, residential moves, education, employment, deaths, financial/legal work, 

health, sexual difficulties, arguments with partner or spouse, and changes in social activities and recreation. The 

analysis showed nearly no overreporting but some underreporting. The underreporting errors were no higher than

33% for health events and tended to be larger for chronic or routine activities rather than intimate events and family 

experiences. Interestingly, 25% of respondents made no recall errors in any of the ten categories and another

quarter made recall errors in only one of the ten categories. Only 8% made recall errors in five or more categories. 

When analyzing the precision of recall over time, the authors found nearly no backward or forward telescoping, 

concluding that recall errors were mostly due to forgetting or selective retention, rather than timing of their 

occurrence. 

Kessler and Wethington (1991) compared inter-respondent reliability of data collected with the EHC with a similar 

study using a CQ. The respondents were married couples who were separately asked to date several negative events 

such as illness, death, and job and financial troubles, for the previous 12 months before the interview. The 

interreliability rate (r) among the couple was in the range of .64 to .68 compared to .30 for a similar CQ study. 

When compared to CQ methods in randomized design studies, EHCs have shown better data quality for 

retrospective reports in terms of precision of the placement of events in time, and in terms of reducing 

underreporting. In a study with PSID respondents randomly assigned to EHC or CQ conditions, and using 

validation data collected from the same respondents (PSID) years before, Belli, Shay, and Shafford (2001) showed

that EHC reports were more precise (less underreporting, higher agreement between reports) than CQ reports on

number of moves, income, weeks unemployed, and weeks missing work resulting from personal illness, the illness 

of another, or the combination of the two. In a study with a design similar to that of Belli and colleagues (2001) 

with PSID respondents, this time using a computerized instrument and telephone interviews, Belli, Smith, Andreski

and Agrawal (2006) collected retrospective reports with a reference period of up to 30 years. The respondents were

randomly assigned to either a standardized CATI interview or a computerized EHC (C-EHC). The reports on social 

and economic variables of residential, marriage, cohabitation, and work history were compared to the previous data

collected from the same panel respondents. The C-EHC showed better overall data quality for cohabitation and

work history; no difference was found for residence change and CQ showed better data only for marriage history. 

Yoshihama and colleagues (2005) found that the EHC method facilitated the recall of domestic violence 

victimization. In particular, the EHC technique elicited more reports of lifetime experiences of intimate partner 

abuse (93-96%) than CQ format (61-64%). EHC was especially precise in revealing abuse that occurred earlier in

respondents’ lives. 

Testing and Evaluation of the re-engineered SIPP 

Confidential Page 6 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

As part of the development of the re-engineered SIPP we are incorporating two distinct field test activities. The 

first research project being undertaken at the Census Bureau is a reinterview and validation experiment using a

sample of cases from the SIPP 2004 panel. The purpose of this evaluation is to consider the ability of the EHC to 

successfully assist respondents to recall program related information. The EHC will be administered with a paper 

instrument to respondents from the 2004 SIPP panel and will focus on the 2007 calendar year experience for these 

respondents. In their wave 10, 11, and 12 SIPP interviews, each of which used a 4-month recall period, those 

respondents will have reported about their employment, program use, income sources, demographics, and other

subjects during calendar year 2007. The EHC reinterview, to be carried out in early 2008, will ask those same

respondents to report about the entire 2007 calendar year in a single interview, utilizing an event history calendar 

data collection instrument. Responses to the 12-month EHC will be compared with the same respondents’ SIPP 

interview reports covering the same calendar year. Missed events in one or the other interview method are likely 

evidence of reduced data quality. Other data quality differences may be indicated by more or less reasonable 

distributions of spell transitions across calendar months. Additional evaluation methods – respondent debriefings, 

interviewer debriefings and focus groups, interview observations, analysis of recorded interviews, etc. – will be

directed toward a better understanding of the EHC interview process, such as how landmark dates are introduced 

and used, the preferred “direction” of reporting, the extent to which events in one domain are used to pinpoint

transitions in another domain, etc. 

One limitation of this design is the possibility that the SIPP respondents’ EHC reports will be “primed” by their 

having just completed three waves of SIPP interviews covering the same time period. This study will yield data 

about the extent of such “priming” by including in the EHC interview sample a set of un-primed SIPP sample cases 

that were dropped from the 2004 SIPP sample after Wave 8 due to a budget-cutting exercise, and thus will not have

previously reported about calendar year 2007. 

To this point the design of this study is comparative; SIPP vs. EHC, and EHC (primed) vs. EHC (un-primed). We

also plan an assessment of data quality through validation against administrative records data. This data quality 

assessment with administrative record data will focus on subset of key characteristics and programs. In order to 

make this aspect of the design feasible, we will limit the EHC reinterview test sample to cases from one or two

states, where substantial groundwork has already been laid to be able to utilize administrative records for several 

programs (e.g., TANF, Food Stamps, Medicare, Medicaid, Social Security, SSI, and possibly unemployment 

information). This validation stage of the analysis will occur after the first stage comparisons due to the added time

necessary to obtain and match the necessary administrative records. 

By including direct comparisons across survey instruments, as well as an administrative-record-based validation 

component, this research will be able to add significantly to the literature on event history calendar survey

methodology, especially with respect to validating the SIPP and EHC reporting of income transfer program receipt 

and amounts over a calendar year. Results from the study will also inform the decision of whether to use EHC

methods in the re-engineered SIPP program. 

Following the 2008 paper instrument evaluation, (assuming a positive outcome) a broad dress-rehearsal evaluation

of the new electronic EHC instrument being designed for the re-engineered SIPP will be administered in September 

2009. The results from the 2008 EHC evaluation will be used to refine training procedures and make necessary 

adjustments to the new computer assisted personal interview (CAPI) EHC being prepared for the dress rehearsal. 

The planning and instrument development for the 2009 re-engineered SIPP dress rehearsal is well underway. The

survey will be administered in September – the earliest possible administration window for the dress rehearsal. It 

will collect information about jobs, programs, health insurance and demographics for the 2008 calendar year. The 

dress rehearsal will implement the lessons learned in developing field procedures for the 2008 EHC evaluation and

extend field implementation to each of the Regional Offices for this national test. The 2009 dress rehearsal 

instrument will be evaluated in several domains including field implementation issues and data comparability vis-àvis SIPP 2008 and administrative records. The administration of the 2009 dress rehearsal in September is not ideal,

but is the earliest in 2009 that the instrument can be ready for implementation. The production implementation of

an EHC in the re-engineered SIPP would be during the early part of the calendar year to minimize the length of

recall in the reporting of data for the prior calendar year. Results from both the 2008 evaluation and the 2009 dress

rehearsal will be used to make final decisions regarding the design and implementation of the re-engineered SIPP 

for production in 2011 or 2012. 

Confidential Page 7 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Keywords (SIPP, re-engineered SIPP, EHC, Census, Survey) 

References

Alwin, D. F. (2007). The margin of error: A study of reliability in survey measurement. Hoboken, NJ: Wiley. 

Axinn, W. G., Barber, J. S., & Ghimire, D. J. (1997). The neighborhood history calendar: A data collection method

designed for dynamic multilevel modeling. Sociological Methodology, 27, 355-392. 

Axinn, W. G., Pearce, L. D., & Ghimire, D. J. (1999). Innovations in life history calendar applications. Social

Science Research, 28, 243-264.

Balán, J., Browning, H. L., Jelin, E., & Litzler, L. (1969). A computerized approach to the processing and analysis

of life histories obtained in sample surveys. Behavioral Science, 14, 105-114.

Belli, R. F. (1998). The structure of autobiographical memory and the event history calendar. Potential 

improvements in the quality of retrospective reports in surveys. Memory, 6, 383-406. 

Belli, R. F. (2003). The integration of a computer assisted interviewing event history calendar in the Panel Study of 

Income Dynamics. Retrieved June, 2006, from

http://psidonline.isr.umich.edu/data/Documentation/Fam/2003/EHCManuscript.pdf

Belli, R. F., & Callegaro, M. (in press-a). The emergence of calendar interviewing: A theoretical and empirical 

rationale. In R. F. Belli, F. P. Stafford & D. Alwin (Eds.), Calendar and time diary methods: Measuring well being

in life course research. Newbury Park, CA: Sage. 

Belli, R. F., Shay, W. L., & Stafford, F. P. (2001). Event history calendars and question list surveys: A direct 

comparison of interviewing methods. Public Opinion Quarterly, 65, 45-74.

Belli, R. F., Smith, L., Andreski, P., & Agrawal, S. (2006). Methodological comparison between CATI Event 

History Calendar and conventional questionnaire instrument: Quality of life course and retrospective reports. 

Submitted for publication. 

Bhandari, A., & Wagner, T. (2006). Self-reported utilization of health care services: Improving measurement and 

accuracy. Medical Care Research and Review, 63, 217-235. 

Blum, Z. D., Karweit, n. L., & Sørensen, A. (1969). A method for the collection and analysis of retrospective life 

histories (No. 48). Baltimore, MD: Johns Hopkins University. 

Callegaro, M. (2007). Seam Effects Changes Due to Modifications in Question Wording and Data Collection

Strategies, A Comparison of Conventional Questionnaire and Event History Calendar Seam Effects in the PSID.

Lincoln, NE: University of Nebraska 

Callegaro, M., Yu, M., Cheng, F.-W., Liao, D., & Belli, R. F. (2005). Comparison of computerized event history 

calendar and question-list interviewing methods: A two year hospitalization history study. In 2004 Proceedings of 

the American Statistical Association, 59th Annual conference of the American Association for Public Opinion

Research (AAPOR) (CD-ROM) (pp. 4746-4753). Alexandria, VA: American Statistical Association. 

Caspi, A., Moffitt, T. E., Arland, T., Freedman, D., Amell, J. W., Harrington, H., et al. (1996). The life history 

calendar: A research and clinical assessment method for collecting retrospective event-history data. International 

Journal of Methods in Psychiatric Research, 6, 101-114. 

Engel, L. S., Keifer, M. C., & Zahm, S. H. (2001). Comparison of traditional questionnaire with an icon/calendarbased questionnaire to assess occupational history. American Journal of Industrial Medicine, 40, 502-511.

Confidential Page 8 9/14/2007 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Ensel, W. M., Peek, K. M., Lin, N., & Lai, G. W.-f. (1996). Stress in life course. A life history approach. Journal of

Aging and Health, 8, 389-416. 

Freedman, D., Thornton, A., Camburn, D., Alwin, D., & Young-DeMarco, L. (1988). The life history calendar: a 

technique for collecting retrospective data. Sociological Methodology, 18, 37-68. 

Furstenberg, F. F. J., Brooks-Gunn, J., & Morgan, P. S. (1987). Adolescent mothers in later life. New York:

Cambridge University Press. 

Jobe, J. B., White, A. A., Kelley, C. L., Mingay, D. J., Sanchez, M. J., & Loftus, E. F. (1990). Recall strategies and 

memories for health-care visits. The Milbank Quarterly, 68, 171-189.

Kessler, R. C., & Wethington, E. (1991). The reliability of life reports in a community survey. Psychological 

Medicine, 21, 723-738. 

Kominski, R. (1990). The SIPP Event History Calendar: Aiding respondents in the dating of longitudinal events. In

Proceedings of the Section of Survey Research Methods (pp. 553-558). Washington D.C.: American Statistical 

Association. 

Lin, N., Ensel, W. M., & Lai, G. W.-f. (1997). Construction and use of the life history calendar: Reliability and

validity of the recall data. In I. H. Gotlib & B. Wheaton (Eds.), Stress and adversity over the life course. 

Trajectories and turning points (pp. 249-272). Cambridge: Cambridge University Press. 

Loftus, E. F., & Fathi, D. C. (1985). Rerieving multiple authobiographical memories. Social Cognition, 3, 280-295. 

Parrado, E. A., & Zenteno, R. M. (2001). Economic restructuring, financial crises, and women's work in Mexico. 

Social Problems, 48, 456-477.

Pebley, A. R. (2004a). EHC user's guide. Retrieved June, 2006, from

http://www.lasurvey.rand.org//EHC%20Users%20Guide%20final.pdf

Pebley, A. R. (2004b). The Los Angeles family and neighborhood survey. Retrieved December, 2005, from

http://www.lasurvey.rand.org/

Saris, W. E. (1998). Ten years of Interviewing without Interviewers: The Telepanel. In M. P. Couper, R. P. Baker, 

J. Bethlehem, C. Z. F. Clark, J. Martin, W. L. Nichols II & J. M. O'Reilly (Eds.), Computer assisted survey 

information collection (pp. 409-430). New York: Wiley. 

Witterbrood, K., & Nieuwbeerta, P. (2000). Criminal victimization during one's life course: The effects of previous

victimization and patterns of routine activities. Journal of Research in Crime and Delinquency, 37, 91-122. 

Yoshihama, M., Gillespie, B., Hammok, A. C., Belli, R. F., & Tolman, R. M. (2005). Does the life history calendar 

method facilitate the recall of intimate partner violence? Comparison of two methods of data collection. Social

Work Research, 29, 151-163. 

Zahm, S. H., Colt, J. S., Engel, L. S., Keifer, M. C., Alvarado, A. J., Burau, K., et al. (2001). Development of a life 

events/icon calendar questionnaire to ascertain occupational histories and other characteristics of migrant 

farmworkers. American Journal of Industrial Medicine, 40, 490-501.

Confidential Page 9 9/14/2007