Evaluation of a 12-Month Reference Period in the 

National Crime Victimization Survey (NCVS) 

Lisa Lee and James Carr1

NORC at the University of Chicago 

55 East Monroe 

Chicago, IL 60603 

lee-lisa@norc.org, carr-james@norc.org 

Introduction 

In 1973 BJS introduced the National Crime Victimization Survey (NCVS, formerly NCS), with the purpose 

of learning “more about crimes and the victims of crime [and] to measure crimes not reported to police as 

well as those that are reported” (BJS 1988:11). The NCVS collects data twice a year from a nationally 

representative sample to obtain information about incidents of crime, victimization, and trends involving 

victims 12 years of age and older and their households. The survey underwent an “intensive 

methodological redesign” in 1993 to “improve the questions used to uncover crime, update the survey 

methods, and broaden the scope of the crimes measured” (BJS 2004:1; see also, Taylor & Rand 1995). 

BJS has recently set forth four priority areas for methodological research on potential improvements to the 

National Crime Victimization Survey (NCVS). One of these priority areas is a recommendation to explore 

the possibility of changing the 6-month reference period to a 12-month period to increase the cost 

efficiency of the survey. 

The recommendation reads as follows: 

“Changing from a 6-month reference period to a 12-month reference period has the potential for improving 

the precision per-unit cost in the NCVS framework, but the extent of loss of measurement quality is not 

clear from existing research based on the post-1992-redesign NCVS instrument. BJS should sponsor 

additional research-involving both experimentation as well as analysis of the timing of events in extant 

data-to inform this trade-off.” 

A challenge in determining the reference period for the NCVS is to find the best balance between cost, data 

quality and respondent burden. Increasing the length of the reference period from 6 to 12 months has an 

obvious favorable impact on cost, since the data can be collected in fewer interviews. However, an 

increased reference period also risks data quality. The further crime incidents are from the date of the 

interview, the more likely they will be forgotten, leading to underreporting. A competing effect on data 

quality with a longer reference period is increased telescoping, in which errors in the recall of date of crime 

incidents may result in overreporting. 

NORC is conducting an investigation of the impact of a 12-month reference period on data quality and 

respondent burden in the NCVS. NORC is developing and testing methods of improving the accuracy of 

recall and minimizing respondent burden through the use of enhanced contextual priming and event history 

calendars. These memory aids will be tested in both web and telephone modes. In our paper we will 

describe the cognitive issues posed by a 12-month reference period for the NCVS, present the memory aids 

that are under development, and outline the plan for testing these memory aids in a field test. 

1

 Norman Bradburn and Henry Brownstein contributed to sections of this manuscript. 
Understanding the Problem 

Information Retrieval 

The process that is most important for research on reference periods is that of information retrieval. 

Remembering is a process by which the memory storehouse is searched to retrieve a particular item that is 

being sought. If we think of memory as a big storehouse, it is clear that it must be organized in some way in 

order for us to be able to retrieve things from it. Just as we must label files when we put them in file 

drawers, we also must attach some kind of labels to information in the memory storehouse. The labeling 

process, often called “encoding,” refers to various aspects of the information or the experience, including 

emotional tone, attached to the item when we stored is it memory so that we can retrieve it. (For a more 

complete discussion of memory models see Tourangeau, Rips and Rasinski, 2000, Chapter 3). 

Information, such as the wording of the question and any explanatory material available to respondents at 

the time they are asked to recall an event, acts as retrieval cues. Retrieval cues are any words, images, 

emotions, and other cues that activate or direct the memory search process. If retrieval cues do not specify 

the event type, e.g., a crime, then the event type must be inferred before the search can begin. This 

inference can come from the wording of the question or from the larger context in which the question is 

asked, including the preceding questions or the introductory material to the survey. 

Cues presented in a question may facilitate retrieval. Cue lists are assumed to reduce survey measurement 

error in a number of ways (e.g., Tulving and Pearlstone, 1966; Hudson and Davis 1972; Bellezza and 

Hartwell, 1981). The presence of a cue list may aid the respondent at the comprehension stage in 

understanding the question and determining what should be included in the answer (Dashen & Fricker, 

2001). Cues may also help respondents with retrieval and estimation, in which they recall the information 

needed to answer the question and decide whether the information is relevant to the question. Further, 

according to models of memory that describe memory as an associative network (e.g., Collins & Loftus, 

1975), the mention of a particular example may activate memories of other, related events the respondent 

may have experienced. 

However, cue lists can have a negative impact on response for a number of reasons (Nickerson, 1984; 

Slamecka, 1968; Roediger, 1978; Marsh, Dolan, Balota, & Roediger, 2004). First, omitting items from the 

cue list may lead respondents to assume those items should not be included. Further, the presentation of 

some items as cues may prime those items, enhancing the activation and easing retrieval of those items in 

comparison to other, non-primed items. 

Because information, both in episodic and semantic memory, is encoded in many different ways, the cues 

in the question or in the context surrounding the question including previous questions, may facilitate or 

constrain the activation and produce better or less good retrieval. Tulving (1983) developed the idea of 

“encoding specificity” in which all of the aspects of events at the time of their occurrence, including, 

importantly, emotional states, get encoded in the representation of the event in memory. Retrieval is more 

likely to be successful based on the degree to which cues activate specific aspects of the coding. 

Retrieval takes time. One clear empirical finding is that giving respondents more time to answer questions 

produces more accurate reports, particularly for behavioral questions. But time is not all there is to it. 

Memories for events in one’s life appear to be organized in event sequences which are hierarchically 

organized (Barsalou, 1988). Giving respondents cues to remind them about the sequence is more effective 

than trying to get them to retrieve information about a specific event. For example, in questions about 

alcohol consumption, giving examples of the kinds of situations in which one might drink increases 

consumption reports (Bradburn, Sudman & Associates, 1979). 

Telescoping 

While one might expect lengthening the reference period for recall of events to lower the report of events, 

an offsetting phenomenon may in fact produce an overreporting of events in a particular reporting period. 

This phenomenon is called “telescoping.” Telescoping was first reported by Neter and Waksberg (1964) in 

a study of response errors in the Consumer Expenditure Survey. Rubin and Baddeley (1989) proposed a 

model to account for telescoping based on the dating of autobiographical events in calendar time. 
Huttenlocher, Hedges & Bradburn (1999) proposed a model to account for telescoping based on the dating 

of autobiographical events in terms of elapsed time from the present. The two models share the same 

general approach but differ in details because of their focus on how the events are represented in temporal 

memory. 

Both models assume there are no systematic errors in dating events (that is, dates associated with events are 

stored in memory correctly) but rather that observed errors in reporting are the result of errors in the recall 

process. The observed telescoping errors that result in overreporting are caused by the combination of 

three independent factors. The first is the normal forgetting process. The second is that, even when events 

are remembered, errors in dating occur randomly and increase linearly with time. The third is that 

intrusions often occur from events outside the reference period, but cannot occur from events that have not 

yet happened. In other words, intrusion occurs in only one direction—from the past forward. 

Telescoping can be reduced by “bounding,” that is reminding respondents of what they reported in the 

previous interview. The design of the NCVS, with its repeated interviews with the same household, 

reflects a concern for telescoping. In the NCVS, the first of the seven interviews is a bounding interview. 

Previously, the data from this interview were not included in the data set, but used to bound the data in the 

subsequent interview. However, the current procedure is to apply a correction factor to the first, 

unbounded, interview to adjust for telescoping. Whenever a crime incident report is completed in the 

NCVS, the interviewer also enters information (called “bounding information”) about the incident onto a 

control card (Field Interviewer Manual, page B1-97; U. S. Census Bureau, 2003). After completing the 

control card, the interviewer checks the bounding information for previous enumeration periods. If a 

review of the bounding information shows that there may be duplicate incidents, the interviewer must ask 

probing questions of the respondent to determine whether they are duplicates or different incidents. With a 

longer reference period, the amount of telescoping will increase, because respondent uncertainty about 

when events happened becomes greater the further away from the present the event occurred (see 

Huttenlocher, Hedges, and Bradburn, 1994 for a full explanation of the phenomenon). Bounding may 

reduce the telescoping produced by the longer reference period but will not totally eliminate it. 

Developing Memory Aids for the NCVS 

Event History Calendar 

The event history calendar (EHC) is a well-tested tool for improving recall of retrospective data (Freedman, 

Thornton, Camburn, Alwin, & Young-DeMarco, 1988). In this method of interviewing, a calendar is used 

to record major life events. The purpose of the calendar is to stimulate the recall of autobiographic 

information and to aid the respondent in dating life events. This methodology is to be contrasted with the 

standard “question-list” (Q-list) approach in which scripted questions are asked and the respondent’s 

answers are recorded. In comparison to standard interviewing, the EHC approach offers more retrieval 

strategies for the respondent (Belli, Lee, Stafford, & Chou, 2004). Standard questions are more restricted 

in the type of cues they provide to respondents. The more conversational style of EHC interviewing 

affords the respondent more cues for use in retrieving information. In their study of the types of retrieval 

cues present in EHC and standard Q-list interviewing, Belli and colleagues (2004) note that most of the 

sequential and parallel interviewer probes they coded appeared more often in EHC interviews than in Q-list 

interviews. Further, probing was more frequent with EHC than with Q-list interviewing. 

EHCs have been used widely as a tool to improve event dating; that is, retrieval of forgotten events is not 

so much the focus as the accurate dating of events. The events that the EHC is often used with include 

residence, periods of employment and unemployment, and periods during which benefits such as food 

stamps were received. These events are continuous in nature and the duration of the event, such as 

employment at a particular job, and the timing of a transition, such as to another job, are of primary interest 

in the data collection. For example, the National Longitudinal Survey of Youth2

 (NLSY) collects data on 

dates of employment, training and other variables. The Panel Study of Income Dynamics3

 (PSID) collects 

 

2

 Information about the NLSY is available at http://www.bls.gov/nls/home.htm. 

3

 Information about the PSID is available at http://psidonline.isr.umich.edu/. 
data on employment, participation in government programs, schooling, family formation and many other 

variables. In surveys such as these, the data collected with the aid of the calendar are the data of interest in 

the survey. The kind of data that the NCVS collects is different from the data often collected with the 

calendar. Unlike employment, schooling, residence, receipt of benefits and other variables typically 

collected via calendar, crime events are random, often one-time events, and not a continuous state. 

In the literature on the EHC, there is some discussion of the collection of data on events that are similar in 

character to the crime events collected by the NCVS (intimate partner violence, Yoshihama, 2009; 

hospitalizations, violent events, and drinking, Roberts & Mulvey, 2009; drug use, ADAM survey, 

Yacoubian & Peters, 2002). Overall, the evidence on the use of the calendar is somewhat mixed. Much 

evidence supports its use for the collection of data on continuous events such as employment, schooling 

and residence. However, the smaller number of studies that have used the calendar to collect data on 

discrete events, such as intimate partner violence episodes or arrests, have varied in their findings on the 

effectiveness of the calendar in improving recall. 

The EHC will be used as part of the NCVS screener help the respondent define the reference period of the 

survey and to recall the major life events that occurred during that period. With an increased reference 

period, events that are further from the interview date become more difficult to recall accurately. The 

intent of using the EHC is to address these recall issues by enhancing the respondent’s memory for the 

events that happened during the reference period. Recalling autobiographical events for the reference 

period may lead to recall of a crime event related in time, location, people or activities. 

In the EHC version of the screener, calendar questioning has been woven into the original NCVS screener 

questions. No additional enhancements to the screener questions have been made. Rather, the completion 

of the calendar and reference to the calendar is built around the exact wording of the original screener 

questions. The respondent fills in the calendar with events from the past 12 months, attempting to place an 

event in each quarter of the year. If the respondent cannot name enough personal events to populate the 

calendar, cultural landmarks such as major holidays are used to fill in the gaps. Events on the calendar are 

referenced during the crime screening. 

The approach that was developed includes segmenting of the 12-month reference period into smaller 

intervals to encourage respondents to increase time spent on retrieval and sharpen their focus on specific 

time periods within the year. This segmented approach is included for some but not all of the crime 

screening questions because of the excessive time it would require to cycle through all the screener 

questions repeatedly for each segment of the year. This approach combines the need to provide 

respondents with more time for recall and balance that against level of burden. 

Completion of the EHC may influence recall in two ways. First, using a calendar is typically thought to 

improve the accuracy of the dating of events. Whereas errors in event dating due to forward telescoping 

could spuriously increase crime reporting with a lengthened reference period, the EHC may alleviate these 

errors. At the same time, the activation of memory structures that occurs as the respondent recalls 

significant personal events as part of the calendaring could enhance recall of events during the reference 

period. Thus, in addition to improved dating of events, enhanced recall of crime events could result from 

the use of the calendar. 

Enhanced Contextual Priming 

The context within a question can influence response. In a study of reporting of sensitive behaviors such as 

drinking and sex, question length was found to influence reporting of sensitive behaviors (Blair, Sudman, 

Bradburn, & Stocking, 1977). For example, a shorter question simply asked how often the respondent 

became intoxicated while drinking alcoholic beverages. In contrast, in the longer questioning, respondents 

were probed for the term they used to describe when people drink too much and act differently from usual. 

The respondent’s term was then used in a survey question indicating that people sometimes drink on an 

empty stomach or drink too much and become intoxicated (using the respondent’s word), and asking how 

 
often this happened to the respondent. The added context and time for processes offered by the longer 

questions may have contributed to the higher reporting observed. 

The context provided from prior questions within a survey can also influence response. Tourangeau and 

Rasinski (1988) describe how the context of prior questions affects response to attitude questions. The 

framework provided by preceding questions may influence interpretation of later questions. Also, through 

a priming effect, specific information from those questions may be more accessible for retrieval and play a 

larger role in the process of making a judgment. As an example, question order effects have been found for 

two questions related to abortion (Schuman & Presser, 1981). One question mentions the possibility of 

birth defects and the other mentions that the woman does not want any more children. When the question 

on birth defects was asked first, support for abortions for women who do not want more children decreased. 

In prior work on the National Crime Survey (NCS), specific cues have been found to influence recall. 

However, the effects varied by crime severity. Taylor and Rand (1995) found that with the redesigned 

NCS screener, which included many short cues to different kinds of crime, reports of crime increased, but 

they did not increase uniformly. There was some evidence that reports of more minor crimes increased 

relative to more major crimes. Reports of crime by people known to the victim increased relative to reports 

of crime by strangers. 

Between 1972 and 1975, the NCS included supplemental attitude questions, which were administered to 

half the sample (Cowan, Murphy, & Wiener, 1978). The attitude questions appeared prior to the crime 

screening questions. The rationale for having the attitude supplement precede the crime screening was to 

avoid any influence of the victimization questions on attitudes to crime. Overall, the attitude supplement 

was found to increase reports of crime. For example, robbery showed an effect with a sign test and higher 

reports of theft were found. Reports of household crime were also higher for households given the 

supplement. 

The findings reported by Cowan et al. (1978) suggest a number of ways in which the attitude supplement 

could increase reports of crime. First, the supplement acts generally to prime thoughts about crime and 

activate memories of crime related-events before the crime screening, increasing the accessibility of 

memories for particular incidents. It also provides cues for specific types of crimes, such as relatively less 

serious crimes, that are more easily forgotten. The supplement may also clarify for respondents what counts 

as a crime and what should be reported. 

In summary, contextual cues can enhance recall in a number of ways. The context can signal what types of 

incidents constitute a crime. Cues such as location, offender, weapons used, time of day, type of crime, and 

others can aid in retrieval of specific incidents. In designing the Enhanced Contextual Priming (ECP) 

materials, a broad variety of cues will be provided. The ECP version includes a set of introductory 

questions at the beginning, before the crime screen questions. The first two introductory questions are 

items on the respondent’s feelings of safety at home and in the neighborhood at night. These two questions 

have been used in the General Social Survey (GSS; see variables FEAR and FEARHOME)4

. Since the 

questions have been asked in prior rounds of the GSS, comparison data on the questions is available. For 

example, it is possible to examine the distribution of responses to these questions on the GSS for various 

years and by various demographic groups and compare to the data obtained for the NCVS sample. 

Additional introductory questions remind respondents that crime can happen in different locations and asks 

respondents to report the different places they go on a regular basis, whether they have traveled, and their 

feelings of safety when not at home. These questions were written by NORC. Finally, respondents are 

reminded that crimes can be committed both by people they know and by strangers; they are asked about 

their trust in other people. The questions on trust are adapted from the crime victimization module (module 

5

on trust and reciprocity) of the Canadian General Social Survey.

 

4

 For more information about the General Social Survey see http://www.norc.org/GSS+Website/. 

5

 For more information see http://www.statcan.gc.ca/cgibin/imdb/p2SV.pl?Function=getSurvey&SDDS=4504&lang=en&db=imdb&adm=8&dis=2. 
The intent of the introductory questions is to provide respondents with an opportunity to spend time 

thinking about crime in general, about the locations where crimes can occur, and the people who can 

commit crimes. The selection of questions that emphasize locations where crimes can occur and offenders 

(including non-strangers) moves to the forefront two types of cues already present in the screener. The 

placement of these questions before the first screener question on crime allows the respondent the 

opportunity to recall relevant information before being asked whether they experienced a particular crime. 

As discussed in the literature review, time spent on the recall task and context are both key to enhancing 

recall of crime events. It is important to note that different introductory questions could have been chosen 

for the ECP version. Currently, no data are available that supports the use of one set of questions over 

another. Rather, the literature supports the general approach of providing more time and context for recall 

to occur. 

Field Testing of the Memory Aids 

The ECP and EHC versions of the screener will be tested in cognitive interviews. Cognitive pretesting 

provides an opportunity to test the screener questions along a variety of dimensions. The goal is to uncover 

potential problems with question wording, response categories, and instructions to the respondent. Once 

the ECP and EHC versions of the screener have been finalized, these versions and a control version will be 

administered to respondents in either the telephone or web mode. We expect to complete a total of 3000 

interviews. This test is intended to reveal whether the two approaches, priming with enhanced context and 

event history calendaring, yield better reporting of crime events relative to the control condition. 

Reference Period Mode 

Memory Aid 

Control 

(no memory aid) 

Event History 

Calendar Contextual Priming 

12 months Telephone/CATI 850 850 850 

12 months Web XXX 225 225 

NORC plans on using an RDD sample of telephone numbers in the Chicago area. Based on area codes and 

prefixes, we will be able to select specific neighborhoods in Chicago and specific suburbs around the 

Chicago area. The households will be in areas that vary in racial/ethnic composition, income, and crime 

rate as reported by local law enforcement agencies. The sampled households will be systematically 

assigned to one of the five cells of the experiment. 

To be able to test the effectiveness of a memory aid in the telephone and web modes, the two samples must 

be comparable. This would not be the case if specific types of respondents (such as lower income or older, 

who may be less likely to complete a web survey) are not represented equally in the telephone and web 

samples. We plan on screening respondents based on access to the internet. Respondents without web 

access who are selected for that condition will be reassigned to the telephone condition. However, when 

making comparisons across the telephone and web modes, only those respondents in both modes who have 

internet access will be included in the comparison. 

In the current NCVS, any household members ages 12 and over is eligible to participate. In this 

experiment, we will interview only the household respondent. This respondent will be 18 or over and will 

be the most knowledgeable about crimes against the household and about household composition. 

The information learned from the field test will form the basis for future work on a 12-month reference 

period for the NCVS; the results will inform the development of a plan for further study and making the 

transition to the 12-month reference period. 
References 

Barsalou, L.W., (1988). “The content and organization of autobiographical memories.” In U. Neisser and 

E. Winograd (Eds.), Remembering reconsidered: Ecological and traditional approaches to the study of 

memory. Cambridge, England: Cambridge University Press 

Bellezza, F. S., & Hartwell, T. C. (1981). “Cuing subjective units.” Journal of Psychology, 107, 209-218. 

Belli, R.F., Lee, E.H., Stafford, F.P., Chou, C. (2004). “Calendar and question-list survey methods: 

Association between interviewer behaviors and data quality.” Journal of Official Statistics, 20(2), 185-218. 

Blair, E., Sudman, S., Bradburn, N. M., & Stocking, C. (1977). How to ask questions about drinking and 

sex: Response effects in measuring consumer behavior. Journal of Marketing Research, 14 (3), 316-321. 

Bradburn, N.M., Sudman, S. and Associates, (1979). Improving interview method and questionnaire 

design. San Francisco: Jossey-Bass. 

Bureau of Justice Statistics (BJS). (1988). Report to the Nation on Crime and Justice. (2nd ed.) NCJ105506. Washington, DC: US Department of Justice. 

Bureau of Justice Statistics (BJS). (2004). The Nation’s Two Crime Measures. NCJ-122705. Washington, 

DC: US Department of Justice. 

Collins, A., & Loftus, E. (1975). “A spreading-activation theory of semantic processing.” Psychological 

Review, 18, 407-428. 

Cowan, C.D., Murphy, L.R., & Wiener, J. (1978). “Effects of supplemental questions on victimization 

estimates from the National Crime Survey.” Proceedings of the American Statistical Association, (pp. 277-

282). 

Dashen, M., and Fricker, S. (2001). “Understanding the cognitive processes of open-ended categorical 

questions and their effects on data quality”. Journal of Official Statistics, 17, 457-477. 

Freedman, D., Thornton, A., Camburn, D., Alwyn, D., & Young-DeMarco, L. (1988). “The life history 

calendar: A technique for collecting retrospective data.” Sociological Methodology, 18, 37-68. 

Hudson, R. L., & Davis, J. L. (1972). “ The effect of intralist cues, extralist cues, and category names on 

categorized recall.” Psychonomic Science, 29, 71-75. 

Huttenlocher, J., Hedges, L.V., and Bradburn, N.M., (1990) “Reports of elapsed time: Bounding and 

rounding processes in estimation.” Journal of Experimental Psychology, Learning, Memory and Cognition, 

16, 196-213. 

Marsh, E. J., Dolan, P. O., Balota, D. A., and Roediger III, H. A. (2004). Part-set cuing effects in younger 

and older adults,” Psychology and Aging, 19, 134-144. 

Neter, J. & Waksberg, J. (1964). “A study of response errors in expenditure data from household 

interviews,” Journal of the American Statistical Association, 59, 18-55. 

Nickerson, R. S. (1984). “Retrieval inhibition from part-set cuing: A persisting enigma in memory 

research.” Memory & Cognition, 12, 531-552. 

Roberts, J., & Mulvey, E. P. (2009). “Reports of life events by individuals at high risk for violence.” In R. 

F. Belli, F. P. Stafford, and D. F. Alwin (Eds.) Calendar and Time Diary Methods in Life Course Research 

(pp. 191-206). Los Angeles: Sage. 

Roediger, H. L., III. (1978). “Recall as a self-limiting process.” Memory & Cognition, 6, 54-63. 
Rubin, D.C., & Baddeley, A.D. (1989). “Telescoping is not time compression: A model of the dating of 

autobiographical events.” Memory and Cognition, 17(6), 653-661. 

Schuman, H. & Presser, S., (1981) Questions and answers in attitude surveys: Experiments on question 

from, wording and context. Orlando: Academic Press 

Slamecka, N. J. (1968). “An examination of trace storage in free recall.” Journal of Experimental 

Psychology, 76, 504-513. 

Taylor, B.M. and M.R. Rand. (1995). “The National Crime Victimization Survey redesign: New 

understandings of victimization dynamics and measurement.” Paper presented at the annual meeting of the 

ASA, August, Orlando, FL. 

Tourangeau, R., and Rasinski, K. (1988) “Cognitive processes underlying context effects in attitude 

measurement.” Psychological Bulletin. 103, 299-314. 

Tourangeau, R., Rips, L.J., and Rasinski, K., (2000). The psychology of survey response. Cambridge, 

England: Cambridge University Press. 

Tulving, E. (1983). Elements of episodic memory. Oxford: Clarendon Press. 

Tulving, E., & Pearlstone, Z. (1966). “Availability versus accessibility of information in memory for 

words.” Journal of Verbal Learning and Verbal Behavior, 5, 381-391. 

U.S. Census Bureau (2003). National Crime Victimization Survey: Interviewing manual for field 

representatives. pp. B1-97. 

Yacoubian, G. S., & Peters, R. J. (2002). “Does the calendar method enhance illegal drug use reporting 

among arrestees?” International Journal of Drug Testing, Volume 2. 

Yoshihama, M. (2009). “Application of the life history calendar approach: Understanding women’s 

experiences of intimate partner violence over the life course. In R. F. Belli, F. P. Stafford, and D. F. Alwin 

(Eds.) Calendar and Time Diary Methods in Life Course Research (pp. 135-155). Los Angeles: Sage.