Bureau of Justice Statistics Research and Development Series

Determining the Optimal Number of 

Interview Waves in the National Crime 

Victimization Survey: Evaluation and 

Recommendations

Marcus Berzofsky, DrPH

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 2770

Ivan Carrillo-Garcia

Statistics Canada

100 Tunney’s Pasture

Ottawa, ON K1Y17

R&D-2016-5, NCJ 249878, November 2016 

Research papers have been reviewed by the Bureau of Justice Statistics (BJS) to ensure the 

general accuracy of information presented and adherence to confidentiality and disclosure 

standards. This paper is released to inform interested parties of research and to encourage 

discussion. Any opinions and conclusions expressed herein are those of the author(s) and do 

not necessarily represent the views of BJS and the U.S. Department of Justice. 

This document was prepared using federal funds provided by the U.S. Department of Justice 

under Cooperative Agreement number 2011-NV-CX-K068. The BJS Project Managers were 

Lynn Langton and Michael Planty. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

ii

Abstract

The National Crime Victimization Survey (NCVS) currently uses a seven-wave or timein-sample (TIS) design. That is, households are asked to participate every six months over a 

three year period. In an effort to maintain or reduce costs and improve data quality, the Bureau of 

Justice Statistics commissioned a Panel Design Study to evaluate the effects of changing the 

NCVS from a 7-TIS design to a 5-TIS, 4-TIS, 3-TIS, or 1-TIS design. Panel surveys need to 

balance the benefits of repeated measurements (e.g., bounded interview, reduced cost, increased 

response rates) with the drawbacks that may eventually occur (e.g., respondent fatigue, attrition, 

nonresponse). The optimal number of interview waves for a panel survey needs to maximize the 

advantages while minimizing the potential for bias due to incorporating sampling units for too 

many interview waves. This study used a set of simulations to mimic different panel design

options for the NCVS. The simulation assumptions were constructed using NCVS data from 

1999 to 2011, and included assumptions about sample sizes, costs, response rates, household 

replacement, type of interview, demographics, and victimization propensities. Samples were 

simulated with different panel designs and summary victimization propensities, and standard 

errors were computed for key estimates. Simulations considered cost models for potential 

constraints: (1) the need to keep the cost constant and (2) the need to keep the number of 

interviews constant across the different panel design options. In this paper, we show the impact 

of changing the number of panel TISs on property and violent victimization rates in terms of 

point estimates, variability, sample sizes, and costs, by several population characteristics. 

Simulation results found that a 4-TIS design is optimal for the NCVS.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

iii

Contents

Section Page

1. Introduction ............................................................................................................................1

1.1 Background ..............................................................................................................1

1.2 Purpose of study .......................................................................................................3

2. Methods..................................................................................................................................6

2.1 Cost models ..............................................................................................................6

2.2 Cost model assumptions ..........................................................................................6

2.3 Cost estimates for current design ...........................................................................10

2.4 Cost model for keeping the number of interviews constant ..................................12

2.5 Cost model for keeping the cost constant ..............................................................12

2.6 Population parameter assumptions for victimization rates ....................................13

2.7 Data for determining population assumptions .......................................................14

2.8 Population parameters ............................................................................................14

2.9 Monte Carlo simulation .........................................................................................16

3. Results ..................................................................................................................................18

3.1 Cost models ............................................................................................................18

3.2 Victimization rates .................................................................................................19

3.3 Comparing overall victimization rates by design ..................................................19

3.4 Comparing subpopulation victimization rates by design .......................................24

4. Conclusions ..........................................................................................................................27

5. Next Steps ............................................................................................................................28

References ................................................................................................................................29
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

iv

Appendix A: Alternative rotation schemes..............................................................................31

Appendix B: Distributions of number of persons age 12 or older living in a household ........33

Appendix C. Impact of alternative designs, by type of crime under keeping the 

number of interviews constant conditions......................................................................35

Appendix D. Impact of alternative designs, by type of crime under keeping the cost 

constant conditions .........................................................................................................41
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

v

List of Tables

Table number Page

Table 2-1.Household response rate and status by time in sample and previous time in 

sample’s response status .................................................................................................8

Table 2-2.Person response rate and interview type by time in sample, household status, 

and whether the household responded in a previous time in sample ..............................9

Table 2-3.Number of selected households in a rotation scheme for a 7-time in sample 

design ............................................................................................................................11

Table 2-4.Design effects for property crime .................................................................................16

Table 2-5.Design effects for violent crime....................................................................................16

Table 3-1.Costs and number of interviews for keeping number of interviews constant by 

design ............................................................................................................................18

Table 3-2.Costs and number of interviews for keeping costs constant by design.........................19

Table 3-3.Victimization rate, standard error, and relative standard error by type of crime 

and design alternative for keeping the number of interviews constant designs............21

Table 3-4.Victimization rate, standard error, and relative standard error by type of crime 

and design alternative for keeping the cost constant designs........................................24

Table 3-5.Violent crime and property crime victimization rates when keeping the number 

of interviews constant by selected demographic characteristic and alternative 

design ............................................................................................................................25

Table 3-6.Violent crime and property crime victimizations when keeping the cost 

constant by selected demographic characteristic and design ........................................26
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

vi

List of Figures

Figure number Page

Figure 3-1. Violent victimization rate, nominal and effective sample sizes, and cost when 

keeping the number of interviews constant by alternative design .................................................20

Figure 3-2. Property victimization rate, nominal and effective sample sizes, and cost 

when keeping the number of interviews constant by alternative design .......................................21

Figure 3-3. Violent victimization rate, nominal and effective sample sizes, and cost when 

keeping the cost constant by alternative design.............................................................................22

Figure 3-4. Property victimization rate, nominal and effective sample sizes, and cost 

when keeping the cost constant by alternative design ...................................................................23
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

1

1. Introduction

1.1 Background

The National Crime Victimization Survey (NCVS) is the nation’s leading measure of 

reported and unreported crime victimization rates in the United States. Sponsored by the 

Department of Justice’s Bureau of Justice Statistics (BJS) and conducted by the U.S. Census 

Bureau, the NCVS is a nationally representative, probability-based household survey that 

interviews all persons age 12 or older in a selected household. Each year in the NCVS, 

interviews are conducted in approximately 90,000 households and 160,000 individuals are 

interviewed (U.S. Census Bureau, 2014; Truman, Langton, & Planty, 2013).

Similar to other national benchmark surveys,1 the NCVS uses a rotating panel design, 

where equally sized sets of sampling units (i.e., rotation groups) are brought in and out of the 

sample in a specified pattern (Kasprzyk et al., 1989; Cantwell, 2008). In the NCVS, samples of 

about 50,000 households are generated every 6 months and allocated across seven rotation 

groups with about 7,000 households per group. Once the households rotate in, they remain in the 

sample for 3.5 years and are interviewed seven times (at 6-month intervals) during that period 

(Truman, Langton, & Planty, 2013). In other words, seven is the maximum time in sample (TIS) 

for a household in the NCVS.

Rotating panel designs offer three key benefits to a survey design (Kasprzyk et al., 1989; 

Kalton & Citro, 1995):

1. Bounded interviews. For studies where there is a concern that the outcome of 

interest may be highly susceptible to recall bias (e.g., telescoping), bounding an 

interview (i.e., tying a previous interview to a specific point in time) to the sampling 

unit’s previous interview better ensures that events of interest occurring before the 

 1 For example, the Current Population Survey (U.S. Department of Labor, 2006) and Consumer Expenditure Survey 

(U.S. Department of Labor, 2011).
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

2

bounding period will not be reported in the current period (Biderman & Cantor, 1984; 

Gaskell, Wright, & O’Muircheartaigh, 2000). 

2. Cost. A panel design can reduce survey costs in two ways. First, panel designs 

often have higher response rates because after the initial contact, participating 

households are more motivated to remain in the study (Cantwell, 2008). A higher 

response rate allows fewer sampling units to be initially selected to achieve the 

desired number of interviews, thereby reducing data collection costs. Second, a 

panel design allows the study to alter the interview mode to a lower-cost mode 

after the initial contact is made. For example, in the NCVS, the initial interview 

with a sampling unit may be conducted in person (i.e., the interviewer goes to the 

sampled address in person to interview all eligible household members) to better 

recruit the household into the study and explain the study and its purpose. Followup interviews may be conducted by telephone (i.e., the same interviewer calls the 

household and conducts interviews via the telephone with each eligible household 

member) to reduce survey costs.

3. Longitudinal design. By interviewing a sampling unit multiple times, a rotating 

panel design allows for longitudinal data analysis in addition to serial crosssectional analysis. This benefit allows analysts to better take into account the 

correlation between a sampling unit and the outcome of interest over time. 

However, some logistical considerations may reduce the impact of these benefits 

(Kasprzyk et al., 1989). For instance, because the NCVS includes the initial, unbounded 

interview, measurement error in the form of recall bias may be introduced. Similarly, mobility in 

the sample may reduce the benefits of bounding and the longitudinal nature of the data 

(Addington, 2005). If there is a large amount of household turnover requiring replacement 

households (i.e., new families that have moved into a selected address), then the cost benefits of 

changing interview mode may not be realized because the first interview with a new replacement 

household will be conducted in person and will negate the potential cost savings by switching 

modes. Also, conducting initial interviews and replacement household interviews in person and 

other interviews over the telephone operates the potential for a mode effect between in-person 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

3

and telephone interviews. In addition, if there is a large amount of panel attrition during the data 

collection period, the ability to conduct a longitudinal analysis may be reduced due to an increase 

in bias and a reduction in precision. Furthermore, respondents that remain in the panel may 

suffer from rotation group bias or panel conditioning (Bailar, 1975; Hart, Rennison, & Gibson,

2005). Although the exact impact of panel conditioning is not consistent in all surveys, it may 

contribute to changes in respondents’ behavior over time (Cantor, 2007). In the NCVS, 

Berzofsky et al. (forthcoming) found that respondents in later interview waves report fewer 

victimizations than in earlier waves. Since telescoping in the NCVS is controlled for with a 

bounding adjustment in the first wave and by prior interviews in subsequent waves, this may 

indicate respondent fatigue is occurring in the NCVS panel, which could lead to suppressed 

victimization rates. Also, because BJS conducts most analyses in a cross-sectional or serial 

cross-sectional manner (Planty et al., 2013; Hardison-Walters et al., 2013), having a larger 

number of interview waves (referred to here as TISs) may not be helpful analytically. This paper 

addresses the issue of cost and the longitudinal design. The effect of including unbounded 

interviews is beyond the scope of this paper.2

1.2 Purpose of study

Because a rotating panel design has both benefits and limitations, it is necessary to assess 

the current NCVS design to see if the number of panel waves (TISs) for a household in the 

sample is optimal. Therefore, the purpose of this study is to determine the optimal number of 

TISs for the NCVS while ensuring that study estimates (i.e., crime victimization rates), precision 

levels, and study costs are not negatively altered. 

To understand how changing the number of TISs for sampled households in the NCVS 

will affect the cost of data collection, four alternative designs were considered in addition to the 

current 7-TIS design:

• 5-TIS design

• 4-TIS design

 2 Because the adjustment for unbounded cases is not incorporated (published estimates for the NCVS include this 

adjustment), estimates in this analysis will increase across designs compared to published estimates. BJS is 

examining bounding adjustment considerations through other lines of research.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

4

• 3-TIS design

• 1-TIS or serial cross-sectional design.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

5

To assess the main study goal, four secondary study questions were considered:

1. Do violent and property victimization estimates differ across different TIS 

designs, and what would be the impact of reducing the number of TISs on these 

estimates? 

2. To what extent do response rates differ across TIS designs, and what would be the 

impact of reducing the number of TISs on response rates? 

3. Given household turnover and attrition, what are the cost savings associated with 

keeping households in the sample for 7 TISs compared to 5 TISs, 4 TISs, 3 TISs, 

and 1 TIS? 

4. How would key sample demographic distributions be affected if the NCVS 

moved from a 7-TIS design to a 5-TIS, 4-TIS, 3-TIS, or 1-TIS design?
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

6

2. Methods

Study questions were answered through a three-step process. First, cost models were 

developed to determine the change in survey costs or the number of interviews that could be 

conducted under the current and alternative designs. Second, key characteristics related to the 

probability of reporting a crime were determined. Third, a Monte Carlo simulation was 

conducted using the cost models and key characteristics to assess the change in the NCVS 

estimates and precision levels caused by modifying the number of TISs.

2.1 Cost models

To assess the cost of modifying the number of TISs, two types of cost models were 

developed: 1) keeping the number of interviews constant (KNIC) and 2) keeping the cost 

constant (KCC). In the KNIC model, the number of interviews is fixed on the basis of the 

average number of interviews in the 7-TIS (current) design. The model adjusts the cost of each 

design based on this fixed number of interviews. In the KCC model, the cost of each design is 

fixed on the basis of the estimated cost of the 7-TIS (current) design. The model adjusts the 

number of interviews in each design based on the fixed cost. Because both models depend on 

knowledge about the current 7-TIS design, the first step was to determine, within these model 

frameworks, the number of interviews and the cost of the survey under the current design.

2.2 Cost model assumptions 

Each cost model is based on assumptions grounded in how the NCVS is currently 

conducted. Two main assumptions were needed for the cost models: 1) the probability of a 

sampled person participating in a particular TIS and 2) the cost of conducting an interview. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

7

Because the field procedures and analysis of the NCVS have changed over time, the data 

used to determine the probability of participating were restricted to a period that best reflects 

current practices. Characteristics that needed to be based on current practices included—

• distribution of interview types (i.e., mode of interview)

• household status in previous TIS3

• response rate

• cost per interview type.

Using the data that met the cost assumption study criteria,4 the following response and 

participation distributions were determined:

• Household response rate and household status by TIS and the person’s previous TIS 

status (table 2-1)

• For households responding in the current TIS, the distribution of person-level

response status by TIS for each possible pattern of response in the previous TISs, 

based on 1) the household’s response status (i.e., whether the same household is 

responding or if it is a replacement household), 2) the person’s previous participation 

status for a household (i.e., whether or not at least one person in the household 

participated in the survey during the previous TIS), and 3) mode of interview (i.e., in

person or telephone) (table 2-25

).

 3

Either “First TIS,” “Same HH [household] interviewed the previous TIS,” “Replacement HH since the previous 

TIS,” or “Noninterview in the previous TIS.”

4

For the cost portion of the analysis, data were restricted to the years 2007 to 2011 and included only sample and 

rotation groups for which all seven TISs were publicly available. Additionally, reinstated cases were excluded from 

the analysis.

5

Appendix Table B-1 presents the counts associated with the percentages presented in this table.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

8

Table 2-1. Household response rate and status by time in sample and previous time in 

sample’s response status

HH Status

TIS

Address 

responded in 

previous TIS

Address 

responded

Number 

in 

sample

First 

TIS

Same HH 

interviewed 

the previous 

TIS

Replacement 

HH since the 

previous TIS

Noninterview in 

the previous

TIS All HHs

1 — No 22,346 — — — — 28.4%

Yes 56,423 100% — — — 71.6%

2 No No 11,513 — — — — 69.8%

Yes 4,977 — 0.8% 10.1% 89.1% 30.2%

Yes No 4,312 — — — — 8.9%

Yes 44,297 — 95.6% 4.4% — 91.1%

3 No No 9,484 — — — — 71.5%

Yes 3,780 — 2.3% 31.6% 66.1% 28.5%

Yes No 3,735 — — — — 8.8%

Yes 38,724 — 95.7% 4.3% — 91.2%

4 No No 8,111 — — — — 72.2%

Yes 3,122 — 2.6% 39.1% 58.3% 27.8%

Yes No 3,126 — — — — 8.6%

Yes 33,416 — 95.7% 4.3% — 91.4%

5 No No 6,781 — — — — 72.3%

Yes 2,592 — 2.4% 43.1% 54.6% 27.7%

Yes No 2,565 — — — — 8.2%

Yes 28,628 — 95.6% 4.4% — 91.8%

6 No No 5,439 — — — — 72.2%

Yes 2,093 — 1.4% 46.1% 52.5% 27.8%

Yes No 2,073 — — — — 8.1%

Yes 23,564 — 95.7% 4.3% — 91.9%

7 No No 4,108 — — — — 71.2%

Yes 1,658 — 1.5% 45.8% 52.7% 28.8%

Yes No 1,486 — — — — 7.3%

Yes 22,346 — 95.5% 4.5% — 92.7%

Note: HH = household, TIS = time in sample, — = not applicable.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

9

Table 2-2. Person response rate and interview type by time in sample, household status, and whether the household

responded in a previous time in sample

First TIS

Same HH interviewed the 

previous TIS*

Replacement HH since the 

previous TIS 

Non-interview in the 

previous TIS: HH has

responded previously 

Non-interview in the previous 

TIS: HH has not responded 

previously

TIS

Person 

Resp. IP Phone

Noninterview IP PhoneNoninterview IP PhoneNoninterview IP Phone

Noninterview IP Phone

Noninterview

1 No — — 11.3% — — — — — — — — — — — —

Yes 65.7% 23.0% — — — — — — — — — — — — —

2 No — — — — — 12.5% — — 12.9% — — 14.3% — — —

Yes — — — 27.3% 60.3% — 67.9% 19.2% — 59.1% 26.6% — — — —

3 No — — — — — 13.0% — — 11.7% — — 15.1% — — 20.8%

Yes — — — 26.2% 60.8% — 69.9% 18.4% — 58.7% 26.2% — 40.0% 39.2% —

4 No — — — — — 13.0% — — 11.7% — — 14.6% — — 25.1%

Yes — — — 25.6% 61.4% — 71.9% 16.4% — 58.2% 27.2% — 37.5% 37.4% —

5 No — — — — — 12.9% — — 11.3% — — 13.4% — — 22.2%

Yes — — — 26.1% 61.0% — 71.5% 17.2% — 60.1% 26.5% — 38.7% 39.0% —

6 No — — — — — 12.7% — — 10.0% — — 13.2% — — 24.7%

Yes — — — 26.0% 61.3% — 73.2% 16.8% — 58.4% 28.4% — 36.2% 39.1% —

7 No — — — — — 11.7% — — 9.2% — — 11.6% — — 22.8%

Yes — — — 29.2% 59.1% — 75.7% 15.1% — 62.0% 26.4% — 41.9% 35.3% —

Note: HH = household, IP = in person, TIS = time in sample, — = not applicable. See Appendix Table B-1 for interview counts for each category.

*Possibly due to coding error, a few households are coded as “Same HH Interviewed in the Previous TIS” and “never previously responded.” Those households

are not included in the table because there are very few of them (51 cases in this restricted dataset). For these cases, the same interview type distribution as for 

those households that previously responded was applied, but a cost of $250 is assumed.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

10

With these two pieces of information, a person’s probability of participating in the NCVS 

for a particular TIS by interview mode was determined. 

For the cost per interview, this study assumed $250 for an in-person interview and $120 

for a telephone interview. BJS provided these cost assumptions, which were based on actual total 

Census Bureau costs to administer the survey in 2013 and the approximate distribution of inperson and telephone interviews. 

2.3 Cost estimates for current design 

To make a fair comparison with the simulated samples (for 5, 4, 3, or 1 TISs), the actual 

sample (of 7 TISs) is not used to calculate the cost of the current design. Instead, a simulated

sample of seven TISs similar to the current design is generated. Using the simulated sample 

removes any noise from the actual sample caused by cases that were excluded from the analysis. 

This allows for an equal comparison between the current design and the alternative designs. 

Approximately 50,000 households are selected every 6 months and distributed among seven

rotation groups (across two samples). This means that a sample of 7,143 households per rotation 

group by sample number is selected. Table 2-3 shows a typical rotation pattern for households 

for the simulation of the current 7-TIS design, with 2 sample numbers and 18 semesters (9 

years). When the maximum seven rotation groups are active (see periods 7 to 12 in table 2-3), 

there are 50,000 households for which interviewers are attempting to conduct the survey. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

11

Table 2-3. Number of selected households in a rotation scheme for a 7-time in sample

design

Sample 1 Sample 2

Rotation group Rotation group

Period 1 2 3 4 5 6 1 2 3 4 5 6

1 7,143

2 7,143 7,143

3 7,143 7,143 7,143

4 7,143 7,143 7,143 7,143

5 7,143 7,143 7,143 7,143 7,143

6 7,143 7,143 7,143 7,143 7,143 7,143

7 7,143 7,143 7,143 7,143 7,143 7,143 7,143

8 7,143 7,143 7,143 7,143 7,143 7,143 7,143

9 7,143 7,143 7,143 7,143 7,143 7,143 7,143

10 7,143 7,143 7,143 7,143 7,143 7,143 7,143

11 7,143 7,143 7,143 7,143 7,143 7,143 7,143

12 7,143 7,143 7,143 7,143 7,143 7,143 7,143

13 7,143 7,143 7,143 7,143 7,143 7,143

14 7,143 7,143 7,143 7,143 7,143

15 7,143 7,143 7,143 7,143

16 7,143 7,143 7,143

17 7,143 7,143

18 7,143

Given this rotation pattern; the number (n) of sampled households in a rotation group and 

sample; the response propensity (r) for a person given the TIS, household status, household 

response status in the previous TIS, and interview type; and the cost (c) of the person interview 

given the interview type, the total cost (TC) model can be written as

TTTT = � ������rriiiiiiiiiicc |pp�

PP

pp=1

2

=1

4

ii=1

nnllll

h=1

6

=1

2

=1

where h = household; i = household status (1 = first TIS, 2 = household in the previous TIS, 

3 = replacement household since the last TIS, 4 = noninterview in the previous TIS);

j = household response status in the previous TIS (1 = responded, 2 = nonrespondent);

k = interview type (1 = in person or equivalent, 2 = telephone) for person p; l = rotation group;

m = sample number; p = person in the given household; P = number of persons in the given 

household; and nn = 7,143.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

12

On the basis of the rotation chart for the 7-TIS design, the cost model, and the 

distribution of the number of persons age 12 or older living in a household (based on the NCVS 

sample, the average is 2.04 persons per household), the survey cost for a 6-month period is 

approximately $12,200,000, with about 67,200 interviews conducted. 

2.4 Cost model for keeping the number of interviews constant 

On the basis of the model for the current 7-TIS design, designs that kept the number of 

interviews constant fixed the number of interviews for a 6-month period at 67,200 and let the 

cost vary based on the mixture of in-person and telephone interviews. For each alternative 

design, a rotation chart (similar to the one in Table 2-1) was developed to determine the number 

of households that would need to be selected per rotation group. The average number of 

households needed per sample (m) and rotation group (l) is 

nn = nnpp

rr����pp̅

Given the number of households sampled, the total cost for the alternative models for a 6-month 

period is

TTTT = � �TTTT

LL

=1

MM

=1

= � ��nn ���rriiiiiicc |pp

PP

pp=1

2

=1

4

ii=1

�

LL

=1

MM

=1

2.5 Cost model for keeping the cost constant 

On the basis of the model for the current 7-TIS design, designs that kept the cost constant 

fixed the total cost for a 6-month period at $12,200,000 and let the number of interviews vary 

based on the mixture of in-person and telephone interviews. For each alternative, when KCC, the 

number of interviews per sample (m) and rotation group (l) can be written as

nn = TTTT

∑ ∑ ∑ rriiiiiicc |pp

PP

pp=1

2

=1

4

ii=1
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

13

However, this formula will lead to a sample size that will vary across rotation groups. Therefore, 

for ease of implementation, the average number of interviews can be written as

�nn���� = ∑ ∑ nn LL

=1

MM=1

MMMM

where M is the number of samples selected during the 6-month period and L is the number of 

rotation groups per sample.

2.6 Population parameter assumptions for victimization rates

The simulations that were carried out to assess the impact of reducing the number of 

NCVS interview TISs required a set of population distribution assumptions. Because the sample 

population is fixed before any data are collected, the population distribution for the simulations 

needs to be based on attributes about the population that are knowable before data are collected.

The assumptions about the population distribution will be applied to all of the samples used 

throughout the simulation study.

It is not practical, or feasible, to use all of the available variables when creating the 

population for the simulations. Therefore, it is necessary to restrict the variables for simulation to 

the ones that best predict the outcome of interest: reporting a victimization. Characteristics 

associated with the propensity to experience property and violent victimization are likely to be

different. Additionally, the subjects to which the two types of victimization apply are also 

different (households vs. persons). Therefore, the population distribution assumptions will be 

created separately for property and violent victimization.

Once the characteristics that are most strongly associated with experiencing victimization 

have been identified, they will be used to determine the distribution of characteristics for the 

simulated samples. The victimization outcome will then be generated on the basis of those 

characteristics and the victimization propensity for the group to which the household or person 

belongs.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

14

2.7 Data for determining population assumptions 

As crime victimization is relatively rare, especially within groups of interest (e.g., 

Hispanic males ages 18 to 29), the data used for the cost models were deemed inadequate (too 

sparse) to determine the most significant population parameters. The cost models dealt with 

estimating nonresponse patterns and sample sizes across TIS designs, so it was necessary to use a 

dataset for which the TIS variable could be calculated without error. However, for the population 

parameters, the most important requirement is for the propensity estimates to be as accurate as 

possible within propensity groups; therefore, a larger dataset was needed. Thus, to estimate the 

victimization propensity and to find the most important variables affecting those propensities, the 

analyses used all of the dataset that contains TIS 1 through TIS 7 responses for survey years 

1999 to 2011.

2.8 Population parameters

For property and violent victimizations, the first step is to determine how many distinct 

victimization propensities exist in the population and the variables that are most strongly 

associated with experiencing victimization. The subjects (either households or persons) in the 

NCVS sample will then be split according to differential propensity groups, defined by the most 

important variables. All of the subjects within a group will have the same victimization 

propensity, but the propensity will differ across groups.

To identify the characteristics most strongly associated with the propensity to experience 

a property or violent victimization, respectively, the outcome of interest is defined as whether or 

not the household was the victim of at least one property crime in the reference period or at least 

one violent crime in the reference period. The set of characteristic variables that will be 

evaluated includes all population characteristics collected in the NCVS. These variables are 

potentially associated with 1) whether or not a household experienced property victimization or 

2) whether a person experienced a violent victimization. For certain variables, some category 

collapsing was done beforehand due to either small cell counts or preliminary exploratory 

analyses that revealed that some categories did not differ with respect to the outcome of interest: 

victimization. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

15

It was then necessary to establish the number of groups to be used for estimating the 

likelihood of experiencing a violent or property victimization. As mentioned above, the 

propensity will vary among groups but will be constant within groups. One way to decide how 

many groups to use is by evaluating the reduction in deviance that increasing the number of 

groups produces. The deviance is a statistical measure of the error associated with a model. For 

example, in this case the model might specify that the data can be divided into a number of 

groups (for instance, 10) within which all of the subjects have the same victimization propensity, 

and across which the propensity to experience victimization differs. Another model might

specify that the data are divided into 11 groups (rather than 10), and so on.

Once the reduction in deviance was equal to at least 80% of the total possible reduction, 

it was clear that the largest reduction in deviance occurred when there were 25 to 27 groups for 

property victimizations and 12 to 14 groups for violent victimizations. Therefore, the property 

victimization model will include at least 27 groups and the violent victimization model will 

include at least 14 groups. For each victimization type, a recursive partitioning tree was used to 

determine the best set of groups. The partitioning tree for property crime included 12 different 

household characteristics that were identified as correlated with reporting a property 

victimization. The partitioning tree for violent crime included nine person and household 

characteristics identified as correlated with reporting a violent victimization.

In addition to the variables used for predicting the probability of reporting a 

victimization, other key demographic characteristics that BJS uses for analysis (e.g., sex, age 

category) were randomly assigned to the sample population based on their marginal distributions 

in the population (i.e., the probability of being ages 18 to 29 was not conditioned on any other 

characteristic, such as sex or race). These variables were used for subpopulation analysis.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

16

2.9 Monte Carlo simulation

Once the propensities to respond and the population parameters were determined, a 

Monte Carlo simulation was conducted to produce victimization estimates by type of crime 

(TOC). The simulation produced estimates for each detailed TOC. For property crime, estimates 

were produced for household burglary, theft, and motor vehicle theft. For violent crime, 

estimates were produced for rape and sexual assault, aggravated assault, robbery, and simple 

assault. 

Because of the complex nature of the NCVS household sample design, it was not feasible 

to incorporate the actual design into the simulation. Therefore, a simple random sample was used 

to select households from the population. To get appropriate standard errors, design effects from 

the population were estimated from the unbounded 1999 to 2011 data. For each design, only 

responses from the corresponding TISs were used to estimate the design effects. For example, for 

robbery, the design effect for the 4-TIS design was based only on robbery victims in TIS 1 

through TIS 4. Table 2-4 and Table 2-5 present the design effects for the property crime and 

violent crime types, respectively, that were analyzed. 

Table 2-4. Design effects for property crime

Design alternative

Type of crime 7-TIS 5-TIS 4-TIS 3-TIS 1-TIS

All property crime 2.51 2.20 2.07 1.90 1.42

Household burglary 1.48 1.39 1.32 1.28 1.29

Motor vehicle theft 1.21 1.12 1.14 1.18 1.11

Theft 2.55 2.27 2.15 2.04 1.54

Note: TIS = time in sample.

Table 2-5. Design effects for violent crime

Design alternative

Type of crime 7-TIS 5-TIS 4-TIS 3-TIS 1-TIS

All violent crime 1.94 1.80 1.78 1.68 1.63

Rape and sexual assault 1.25 1.22 1.21 1.26 1.31

Robbery 1.31 1.27 1.20 1.21 1.30

Aggravated assault 1.41 1.43 1.44 1.40 1.47

Simple assault 1.99 1.80 1.75 1.72 1.47

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

17

Furthermore, because no bounding adjustment (i.e., an adjustment applied to respondents 

in TIS 1 who report a victimization to account for potential recall bias) can be applied to a 1-TIS 

(cross-sectional) design, no bounding adjustment was applied to any of the designs. 

The simulation of samples was conducted 1,000 times. For each simulation, households 

are assigned a rotation group and survey characteristics. Within each rotation group, response 

and participation characteristics are assigned on the basis of the cost model assumptions. Given 

these characteristics and the simulated TIS for a household, a household (or person) is assigned a 

victimization status. Among those identified as victims, the number of victimizations reported 

was simulated on the basis of the simulated household (or person) characteristics. 

Estimates were the average victimization rate over the 1,000 simulations. In other words, 

if the victimization rate for one realization of the simulation for TOC V is VVrr rr = 1,2, ⋯ ,1000,

then the average victimization rate across all simulations was calculated as

VV� = 1

1000 � VVrr

1000

rr=1

The standard error is the product of the design effect for TOC V (DDDDDDDDVV) and the 

standard error under a simple random sample. In other words,

SS�(VV) = �DDDDDDDDVV × VVVVVV�SSSSSS(VV�) = �DDDDDDDDVV ×

1

1000 �(VVrr − VV�)2

1000

rr=1
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

18

3. Results

3.1 Cost models

Table 3-1 presents the results of the cost models by alternative design when KNIC. The 

table shows that as the number of TISs decreases, the cost to maintain the same number of 

interviews increases compared with the current design. For instance, with the 4-TIS design, the 

cost increases to $12.8 million (a 4.9% increase). The cost increase is 37.7% for the 1-TIS design

but the change in cost is less than 10% for all other designs. The total cost grows because as the 

number of households per sample and rotation group increases, the number of in-person 

interviews also increases (i.e., there are more first interviews with an address).

Table 3-1. Costs and number of interviews for keeping number of interviews constant 

by design

Design

Households per 

sample and RG

Households per 

6 months

Interviews per 

6 months

Cost per 6 

months

Percent change 

in cost

7-TISs 7,143 50,000 67,200 $12,200,000 

5-TISs 10,108 50,540 67,200 $12,500,000 2.5

4-TISs 12,695 50,780 67,200 $12,800,000 4.9

3-TISs 16,990 50,970 67,200 $13,300,000 9.0

1-TIS 51,470 51,470 67,200 $16,800,000 37.7

Note: RG = rotation group, TIS = time in sample.

Table 3-2 presents the results of the cost models by alternative design when KCC. The 

table shows that as the number of TISs decreases, the number of interviews per 6-month period 

decreases. For instance, for the 4-TIS design, only 48,400 interviews can be conducted for the 

$12.2 million cost of the current design (a 4.8% decrease). The decrease in the number of 

interviews is 27.4% for the 1-TIS design, but the change is less than 10% for all other designs.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

19

Table 3-2. Costs and number of interviews for keeping costs constant by design

Design

HHs per 

sample and 

RG

HHs per 6 

months

Interviews per 6 

months

Cost per 

6 months

Percent change in 

number of 

interviews

7-TISs 7,143 50,000 67,200 $12,200,000 

5-TISs 9,860 49,300 65,500 $12,200,000 −2.5

4-TISs 12,100 48,400 64,000 $12,200,000 −4.8

3-TISs 15,567 46,701 61,600 $12,200,000 −8.3

1-TIS 37,370 37,370 48,800 $12,200,000 −27.4

Note: HH = household, RG = rotation group, TIS = time in sample.

3.2 Victimization rates

Two types of analyses were conducted to assess victimization rates: 1) comparing overall 

victimization rates by design and 2) comparing subpopulation victimization rates by design.

3.3 Comparing overall victimization rates by design

To assess the quality of the estimates under each design, given that no gold standard (i.e., 

error-free estimate) of crime victimization exists, only relative comparisons to the current design 

could be made. Therefore, to compare victimization estimates in each design, the following 

measures were used—

• estimates by type of crime 

• statistical difference of estimates 

• relative standard errors (RSEs) 

• nominal and effective sample sizes.

Figure 3-1 and Figure 3-2 show (for violent crimes and property crimes, respectively)

the victimization rates, nominal and effective sample sizes, and costs for the KNIC designs. In 

both figures, the victimization rate increases as the number of TISs decreases because as the 

number of TISs decreases, the influence of the unbounded interview (i.e., the first TIS) is greater. 

Unbounded interviews have more victimizations reported due to potential recall bias. For violent 

victimization, the differences in the victimization rates are not significantly different from one 

another. However, for property victimization, the 1-TIS design has a significantly higher rate 

than the other designs because the design effects for property crimes decrease more sharply than 

for violent crimes as the number of TISs decreases (table 2-4 and table 2-5). Violent crimes 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

20

have additional correlation as a result of interviewing all persons age 12 or older in a household. 

This additional correlation offsets the benefits, in terms of variance reduction, of having fewer 

repeat interviews over time. These findings are similar when violent crimes are separated into 

more detailed types of crime (e.g., aggravated assault, household theft). Moreover, although the 

nominal sample size is intentionally the same for each design, due to the decreasing design 

effects, the effective sample size increases as the number of TISs decreases for both violent and 

property crime.

Figure 3-1. Violent victimization rate, nominal and effective alternative sample sizes, and 

cost when keeping the number of interviews constant by design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

21

Figure 3-2. Property victimization rate, nominal and effective sample sizes, and cost when 

keeping the number of interviews constant by alternative design

Note: TIS = time in sample.

Table 3-3 presents the RSEs for violent crime and property crime, respectively, by 

design when KNIC. In both cases, the RSE decreases as the number of TISs decreases. This is 

mainly a result of the reduction in design effect as the number of TISs decreases. 

Table 3-3. Victimization rate, standard error, and relative standard error by type of 

crime and design alternative for keeping the number of interviews constant 

designs

Violent crime Property crime

Design

Estimated rate 

per 1,000

Standard 

error RSE

Estimated rate 

per 1,000

Standard 

error RSE

7-TIS 33 2.64 8.00 174 6.51 3.74

5-TIS 34 2.55 7.50 183 6.16 3.37

4-TIS 35 2.58 7.37 189 6.29 3.33

3-TIS 37 2.62 7.08 200 5.93 2.97

1-TIS 44 2.94 6.68 254 6.19 2.44

Note: RSE = relative standard error, TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

22

Figure 3-3 and Figure 3-4 show, for violent crime and property crime, respectively, the 

victimization rates, nominal and effective sample sizes, and costs for the KCC designs. As with 

the KNIC designs, the victimization rates increase as the number of TISs decreases. For violent 

victimization, the differences in the victimization rates are not significantly different from one 

another. However, for property victimization, the 1-TIS design has a significantly higher rate 

than the other designs. These findings are similar for more detailed types of crimes (e.g., 

aggravated assault, household theft). Furthermore, the nominal sample size decreases as the 

number of TISs decreases in order to keep costs fixed. Nonetheless, for 3-TIS, 4-TIS, and 5-TIS 

designs, the effective sample size increases relative to the 7-TIS design for both types of crime. 

However, due to the smaller decrease in the design effect and the large (27.4%) decrease in the 

nominal sample size for violent crime, the effective sample size is lower for the 1-TIS design 

than the 7-TIS design. The 1-TIS effective sample size is larger than the 7-TIS design for 

property crime because the design effect is much smaller for the 1-TIS design than for the 7-TIS 

design.

Figure 3-3. Violent victimization rate, nominal and effective sample sizes, and cost when 

keeping the cost constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

23

Figure 3-4. Property victimization rate, nominal and effective sample sizes, and cost when 

keeping the cost constant by alternative design

Note: TIS = time in sample.

Table 3-4 presents the RSEs for violent crime and property crime, respectively, by 

design when KCC. For violent crime, the RSEs (100 × standard error/estimate) remain relatively 

flat across design options, while for property crime, the RSEs decrease as the number of TISs 

decreases. In the KCC designs, the nominal sample size must decrease to maintain costs because 

the number of in-person interviews increases. When the change in the design effect is negligible 

(as it is with violent crime) the standard errors increase rather than remain flat (i.e., the negative 

impact of the decrease in sample size on the standard errors is greater than the positive impact of 

the smaller design effect), thus leading to flat RSEs. However, for property crime, the decrease 

in the design effect has greater influence on the standard errors than the decrease in nominal 

sample size, leading to lower RSEs as the number of TISs decreases. These findings were 

consistent across the more detailed types of crime. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

24

Table 3-4. Victimization rate, standard error, and relative standard error by type of 

crime and design alternative for keeping the cost constant designs

Violent crime Property crime

Design

Estimated rate 

per 1,000

Standard 

error RSE

Estimated rate per 

1,000

Standard 

error RSE

7-TIS 33 2.64 8.00 174 6.51 3.74

5-TIS 34 2.62 7.71 183 6.22 3.39

4-TIS 35 2.65 7.58 189 6.33 3.35

3-TIS 37 2.82 7.62 201 6.48 3.23

1-TIS 44 3.65 8.30 254 6.82 2.68

Note: RSE = relative standard error, TIS = time in sample.

3.4 Comparing subpopulation victimization rates by design

In addition to comparing overall victimization rates by design, it is necessary to know 

whether the alternative designs affect subpopulation estimates. For this comparison, 

subpopulation estimates for key population characteristics (e.g., age category, income) were 

computed for each design and statistically compared to the current 7-TIS design. 

Table 3-5 and Table 3-6 present the violent and property victimization rates by gender 

and race for KNIC and KCC models, respectively, by design alternative. For both the KNIC and 

KCC models, the only differences (except for whites in the 3-TIS design) between the alternative 

designs and the current 7-TIS design were in the 1-TIS design. This finding held true for all other 

characteristics compared. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

25

Table 3-5. Violent crime and property crime victimization rates when keeping the 

number of interviews constant by selected demographic characteristic and 

alternative design

Violent crime victimizations Property crime victimizations

Characteristic 7-TIS 5-TIS 4-TIS 3-TIS 1-TIS 7-TIS 5-TIS 4-TIS 3-TIS 1-TIS

Sex

Male 33.3 34.8 35.7 37.8 44.6 * — — — — —

Female 32.2 33.5 34.4 36.4 43.2 * — — — — —

Race

White 30.0 31.2 32.1 34.0 40.5 * 162.5 170.9 176.4 186.5 * 235.6 **

Black 41.9 43.9 44.8 47.1 55.3 191.6 201.2 208.1 222.7 285.4 **

Hispanic 38.1 40.0 41.0 43.5 50.5 217.8 228.8 237.7 252.4 319.8 **

Other 35.1 36.3 37.1 38.7 46.8 190.5 202.2 209.7 221.3 280.3 *

Age

12–19 56.9 60.0 61.6 65.5 73.9 — — — — —

20–29 51.0 52.9 53.2 56.2 67.0 — — — — —

30–39 32.9 34.8 35.4 37.2 45.1 — — — — —

40–49 29.1 30.5 31.4 33.3 39.8 — — — — —

50–59 26.9 28.4 29.5 31.0 37.6 — — — — —

60+ 8.1 8.3 8.6 9.2 11.2 — — — — —

Household income

Less than $14,999 56.0 57.9 59.2 60.9 78.1 179.9 189.6 199.1 211.7 278.6 **

$15,000–24,999 37.3 38.9 39.2 42.2 51.9 177.9 187.2 195.4 209.2 270.9 **

$25,000–34,999 33.5 34.8 35.2 37.7 42.8 167.9 176.4 181.6 191.9 243.3 **

$35,000–49,999 31.5 33.0 34.0 35.7 41.0 167.2 175.2 179.8 190.4 240.1 **

$50,000–74,999 29.2 30.8 31.3 33.3 38.4 169.8 178.5 183.7 194.7 244.2 **

$75,000+ 26.4 27.9 28.7 30.5 35.7 178.3 188.1 193.6 204.0 252.8 **

Region

Northeast 30.4 31.9 32.9 34.7 41.0 158.9 166.7 171.9 181.9 231.8 **

Midwest 32.8 34.3 34.8 37.0 44.1 171.8 181.7 187.9 198.4 251.8 **

South 33.0 34.6 35.3 37.5 44.3 174.4 182.9 189.5 201.1 254.5 **

West 34.2 35.6 36.5 38.4 45.5 187.5 197.6 203.9 216.1 272.4 **

Place size

Not in a place 27.9 29.3 30.2 32.0 37.8 155.5 163.4 168.8 178.0 222.2 **

Under $10,000 31.9 33.3 34.3 36.0 43.3 156.9 163.8 169.4 179.7 227.7 **

$10,000–99,999 33.5 35.0 35.6 37.8 44.7 159.4 167.6 172.4 183.5 232.7 **

$100,000–249,999 37.3 39.0 39.6 41.8 49.2 200.6 211.3 219.2 231.9 294.4 **

$250,000–999,999 38.1 40.0 40.8 43.0 51.3 206.4 218.4 226.7 239.1 305.0 **

$1,000,000+ 36.5 38.5 38.9 41.3 48.8 200.8 211.5 218.6 233.3 298.3 **

Tenure

Owned or being 

bought

— — — — — 162.0 170.5 176.1 185.7 * 231.7 **

Rented — — — — — 198.1 208.1 215.5 230.1 * 298.4 **

Note: TIS = time in sample, — = not applicable.

*Significantly different from 7-TIS design at the 95% confidence level.

**Significantly different from 7-TIS design at the 99% confidence level.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

26

Table 3-6. Violent crime and property crime victimizations when keeping the cost 

constant by selected demographic characteristic and design 

Violent crime victimizations Property crime victimizations

Characteristic 7-TIS 5-TIS 4-TIS 3-TIS 1-TIS 7-TIS 5-TIS 4-TIS 3-TIS 1-TIS

Sex

Male 33.3 34.8 35.7 37.8 44.6 * — — — — —

Female 32.2 33.5 34.4 36.4 43.2 * — — — — —

Race

White 30.0 31.2 32.1 34.0 40.5 * 162.5 170.9 176.4 186.5 * 235.6 **

Black 41.9 43.9 44.8 47.1 55.3 191.6 201.2 208.1 222.7 285.4 **

Hispanic 38.1 40.0 41.0 43.5 50.5 217.8 228.8 237.7 252.4 319.8 **

Other 35.1 36.3 37.1 38.7 46.8 190.5 202.2 209.7 221.3 280.3 *

Age

12–19 56.9 60.0 61.6 65.5 73.9 — — — — —

20–29 51.0 52.9 53.2 56.2 67.0 — — — — —

30–39 32.9 34.8 35.4 37.2 45.1 — — — — —

40–49 29.1 30.5 31.4 33.3 39.8 — — — — —

50–59 26.9 28.4 29.5 31.0 37.6 — — — — —

60+ 8.1 8.3 8.6 9.2 11.2 — — — — —

Household income

Less than $14,999 56.0 57.9 59.2 60.9 78.1 179.9 189.6 199.1 211.7 278.6 **

$15,000–24,999 37.3 38.9 39.2 42.2 51.9 177.9 187.2 195.4 209.2 270.9 **

$25,000–34,999 33.5 34.8 35.2 37.7 42.8 167.9 176.4 181.6 191.9 243.3 **

$35,000–49,999 31.5 33.0 34.0 35.7 41.0 167.2 175.2 179.8 190.4 240.1 **

$50,000–74,999 29.2 30.8 31.3 33.3 38.4 169.8 178.5 183.7 194.7 244.2 **

$75,000+ 26.4 27.9 28.7 30.5 35.7 178.3 188.1 193.6 204.0 252.8 **

Region

Northeast 30.4 31.9 32.9 34.7 41.0 158.9 166.7 171.9 181.9 231.8 **

Midwest 32.8 34.3 34.8 37.0 44.1 171.8 181.7 187.9 198.4 251.8 **

South 33.0 34.6 35.3 37.5 44.3 174.4 182.9 189.5 201.1 254.5 **

West 34.2 35.6 36.5 38.4 45.5 187.5 197.6 203.9 216.1 272.4 **

Place size

Not in a place 27.9 29.3 30.2 32.0 37.8 155.5 163.4 168.8 178.0 222.2 **

Under $10,000 31.9 33.3 34.3 36.0 43.3 156.9 163.8 169.4 179.7 227.7 **

$10,000–99,999 33.5 35.0 35.6 37.8 44.7 159.4 167.6 172.4 183.5 232.7 **

$100,000–249,999 37.3 39.0 39.6 41.8 49.2 200.6 211.3 219.2 231.9 294.4 **

$250,000–999,999 38.1 40.0 40.8 43.0 51.3 206.4 218.4 226.7 239.1 305.0 **

$1,000,000+ 36.5 38.5 38.9 41.3 48.8 200.8 211.5 218.6 233.3 298.3 **

Tenure

Owned or being bought — — — — — 162.0 170.5 176.1 185.7 * 231.7 **

Rented — — — — — 198.1 208.1 215.5 230.1 * 298.4 **

Note: TIS = time in sample, — = not applicable.

*Significantly different from 7-TIS design at the 95% confidence level.

**Significantly different from 7-TIS design at the 99% confidence level.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

27

4. Conclusions

For both violent crimes and property crimes, the 4-TIS design achieves the largest 

effective sample sizes while still ensuring that the (overall) victimization estimates are not 

significantly different from the current estimates. The 3-TIS and 1-TIS designs, on the other 

hand, sometimes achieve larger effective sample sizes, but both designs produce estimates that 

are significantly different from the current estimates at either the subpopulation level (3-TIS 

design) or subpopulation and overall level (1 TIS). Moreover, the 4-TIS design reduces the RSEs 

for all types of crime. 

The conclusion that the 4-TIS design has preferred properties compared with all other 

designs holds for both the KNIC and KCC models. In general, the recommendation is to use a 

design that maintains the current number of interviews per year. When maintaining the same 

number of interviews, a lower number of TISs results in higher effective sample sizes. Designs 

with fewer TISs produce lower design effects in general. However, maintaining the same number 

of interviews costs more due to the increase in the number of in-person interviews. Therefore, 

the decision about which 4-TIS design is preferred (KNIC or KCC) is a matter of cost. When 

KNIC, the cost of the 4-TIS design is 4.8% greater ($1.2 million per year) than the 7-TIS design. 

However, for the additional cost of $1.2 million, the 4-TIS design only reduces the RSE 

for violent crime by 7.9% and reduces the RSE for property crime by 11.0% compared with the 

7-TIS design. The KCC 4-TIS design provides improvements in the effective sample size, 

although not as great as the KNIC 4-TIS design, while having only a slightly lower reduction in 

the RSE (violent victimization has a reduction of 5.25%, while reduction in the RSE is the same 

as the KNIC reduction for property crime). Moreover, unlike the 3-TIS and 1-TIS designs, when 

costs are kept constant, the 4-TIS design produces estimates that are not significantly different 

from the 7-TIS design for all subpopulation characteristics and types of crimes.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

28

5. Next Steps

If BJS would consider transitioning the NCVS to a 4-TIS design because of these 

findings, several additional steps would be required to determine the feasibility of this change. 

First, a more thorough cost assessment would need to be conducted using accurate and detailed 

cost data provided by the Census Bureau. Second, considerations regarding the impact of 

integrating a 4-TIS design into the existing rotating panel scheme would have to be taken into 

account. Because more households would be sampled each rotation period, it would be necessary 

to assess whether a design with additional primary sampling units (PSU) is more effective than 

increasing the caseload in the existing PSUs. Additional considerations include how the 

allocation of Census Bureau field interview staff would be affected, potential changes to the 

instrument to adjust for items currently administered in every other interview, and potential 

modifications to bounding and weighting adjustments. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

29

References

Addington, L. A. (2005). Disentangling the effects of bounding and mobility on reports of 

criminal victimization. Journal of Quantitative Criminology, 21(3), 321-343. 

Bailar, B. A. (1975). The effects of rotation group bias on estimates from panel surveys. Journal 

of the American Statistical Association, 70, 23-30

Berzofsky, M. E., Moore, A., Couzens, G. L., Heller, D., Burfeind, C., & Krebs, C. 

(forthcoming). A total survey error review of the National Crime Victimization Survey 

sample design.

Biderman, A. D., & Cantor, D. (1984). A longitudinal analysis of bounding, respondent 

conditioning and mobility as sources of panel bias in the National Crime Victimization 

Survey. Proceedings of the Joint Statistical Meetings, Survey Research Methods Section

(pp. 708-713). Alexandria, VA: American Statistical Association. Retrieved from

http://www.amstat.org/sections/srms/Proceedings/papers/1984_134.pdf

Cantor, D. (2007). A review and summary of studies on panel conditioning. In S. Menard (Ed.), 

Handbook of Longitudinal Research: Design, Measurement, and Analysis (pp. 124-137). 

Amsterdam: Academic Press.

Cantwell, P. J. (2008). Rotating panel design. In P. Cantwell (Ed.), The Encyclopedia of Survey 

Research Methods (pp. 772-774). Thousand Oaks, CA: Sage Publications. doi: 

10.4135/9781412963947.

Couzens, G. L., Berzofsky, M. & Krebs, C. P. (2014). Analyzing potential mode effects in the 

National Crime Victimization Survey. Proceedings of the Joint Statistical Meetings, 

Survey Research Methods Section (pp. 2915-2923). Alexandria, VA: American Statistical 

Association. Retrieved from

https://www.amstat.org/membersonly/proceedings/2014/data/assets/pdf/312744_89876.pdf

Gaskell, G. D., Wright, D. B., & O’Muircheartaigh, C. A. (2000). Telescoping of landmark 

events: Implications for survey research. Public Opinion Quarterly, 64, 77-89. 

Hardison-Walters, J., Moore, A., Berzofsky, M., & Langton, L. (2013). Household Burglary, 

1994–2011. Washington, DC: U.S. Department of Justice, Bureau of Justice Statistics. 

Retrieved from http://www.bjs.gov/content/pub/pdf/hb9411.pdf

Hart, T. C., Rennison, M. C. & Gibson, C. (2005). Revisiting respondent “fatigue bias” in the 

National Crime Victimization Survey. Journal of Quantitative Criminology, 21(3), 345-

363.

Kalton, G., & Citro, C. F. (1995). Panel surveys: The fourth dimension. The European Journal of 

Social Science Research, 8(1), 25-39. doi: 10.1080/13511610.1995.9968429.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

30

Kasprzyk, D., Duncan, G., Kalton, G., & Singh, M. P. (Eds.). (1989). Panel Surveys. New York, 

NY: John Wiley & Sons.

Planty, M., Langton, L. Krebs, C., Berzofsky, M., & Smiley-McDonald, H. (2013). Female 

Victims of Sexual Violence, 1994–2010. Washington, DC: U.S. Department of Justice, 

Bureau of Justice Statistics. Retrieved from http://www.bjs.gov/content/pub/pdf/hb9411.pdf

Truman, J., Langton, L., & Planty, M. (2013). Criminal Victimization, 2012. Washington, DC: 

U.S. Department of Justice, Bureau of Justice Statistics. Retrieved from

http://www.bjs.gov/content/pub/pdf/cv12.pdf 

U.S. Census Bureau. (2014). National Crime Victimization Survey: Technical Documentation.

Washington, DC: U.S. Census Bureau.

U.S. Department of Labor. (2006). Current Population Survey: Design and Methodology

(Technical Paper 66). Washington, DC: U.S. Department of Labor, Bureau of Labor 

Statistics. 

U.S. Department of Labor. (2011). Consumer Expenditure Survey Anthology, 2011 (Report 

1030). Washington, DC: U.S. Department of Labor, Bureau of Labor Statistics. 
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

31

Appendix A: Alternative rotation schemes

Table A-1. Rotation scheme for a 5-time in sample design

SeqSem

Sample number

1 Rotation group 2 Rotation group

1 2 3 4 1 2 3 4

N N N N N N N N

1 10,108

2 10,108 10,108

3 10,108 10,108 10,108

4 10,108 10,108 10,108 10,108

5 10,108 10,108 10,108 10,108 10,108

6 10,108 10,108 10,108 10,108 10,108

7 10,108 10,108 10,108 10,108 10,108

8 10,108 10,108 10,108 10,108 10,108

9 10,108 10,108 10,108 10,108

10 10,108 10,108 10,108

11 10,108 10,108

12 10,108

Table A-2. Rotation scheme for a 4-time in sample design

Sample number

1 Rotation group 2 Rotation group

1 2 3 1 2 3

SeqSem N N N N N N

1 1,250

2 1,250 1,250

3 1,250 1,250 1,250

4 1,250 1,250 1,250 1,250

5 1,250 1,250 1,250 1,250

6 1,250 1,250 1,250 1,250

7 1,250 1,250 1,250

8 1,250 1,250

9 1,250
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

32

Table A-3. Rotation scheme for a 3-time in sample design

SeqSem

Sample number

1 Rotation group 2 Rotation group

1 2 1 2

N N N N

1 16,990

2 16,990 16,990

3 16,990 16,990 16,990

4 16,990 16,990 16,990

5 16,990 16,990

6 16,990

.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

33

Appendix B: Distributions of number of persons age 12 or older living in a household

Table B-1. Number of sampled persons by interview type, time in sample, household status, and whether the household

responded in a previous time in sample

First TIS

Same HH interviewed the 

previous TIS*

Replacement HH since the 

previous TIS

Non-interview in the 

previous TIS: HH has

responded previously

Non-interview in the previous 

TIS: HH has not responded 

previously

TIS

Person 

Resp. IP Phone

Noninterview IP PhoneNoninterview IP PhoneNoninterview IP Phone

Noninterview IP PhoneNoninterview Total

1 No — — 13,156 — — — — — — — — — — — — 13,156

Yes 76,271 26,665 — — — — — — — — — — — — — 102,936

2 No — — — — — 11,029 — — 591 — — 1,183 — — — 12,816

Yes — — — 24,092 53,257 — 3,113 880 — 4,892 2,205 — — — — 88,497

3 No — — — — — 10,008 — — 596 — — 426 — — 382 11,413

Yes — — — 20,182 46,749 — 3,563 938 — 1,660 740 — 736 722 — 75,305

4 No — — — — — 8,601 — — 574 — — 224 — — 511 9,910

Yes — — — 16,888 40,607 — 3,511 801 — 896 419 — 762 760 — 64,647

5 No — — — — — 7,244 — — 498 — — 132 — — 399 8,279

Yes — — — 14,704 34,294 — 3,143 754 — 590 260 — 695 700 — 55,150

6 No — — — — — 5,890 — — 365 — — 84 — — 376 6,716

Yes — — — 12,038 28,381 — 2,659 610 — 372 181 — 550 594 — 45,387

7 No — — — — — 4,287 — — 261 — — 53 — — 277 4,878

Yes — — — 10,746 21,757 — 2,151 429 — 282 120 — 510 430 — 36,427

Note: HH = household, IP = in person, TIS = time in sample, — = not applicable. *

Possibly due to a coding error, a few households were coded as both “Same HH Interviewed in the Previous TIS” and “never previously responded.” Those

households are not included in the table because there are very few of them (51 cases in this restricted dataset). For these cases, the same interview type 

distribution as for those households that previously responded was applied, but a cost of $250 was assumed.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

34

Table B-2. Distribution of number of persons age 12 or older per household

Number of persons age 12 or older Percent

1 31.079

2 46.355

3 13.348

4 6.5195

5 1.9959

6 0.5362

7 0.1084

8 0.0351

9 0.0141

10 0.0053

11 0.0027

12 0.0004

Total 100
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

35

Appendix C. Impact of alternative designs, by type of crime under keeping the number of 

interviews constant conditions

Figure C-1. Property victimization rate, nominal and effective sample sizes, and cost when 

keeping the number of interviews constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

36

Figure C-2. Household burglary victimization rate, nominal and effective sample sizes, and 

cost when keeping the number of interviews constant by alternative design

Note: TIS = time in sample.

Figure C-3. Motor vehicle theft victimization rate, nominal and effective sample sizes, and 

cost when keeping the number of interviews constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

37

Figure C-4. Theft victimization rate, nominal and effective sample sizes, and cost when 

keeping the number of interviews constant by alternative design

Note: TIS = time in sample.

Figure C-5. Violent victimization rate, nominal and effective sample sizes, and cost when 

keeping the number of interviews constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

38

Figure C-6. Rape and sexual assault victimization rate, nominal and effective sample sizes,

and cost when keeping the number of interviews constant by alternative design

Note: TIS = time in sample.

Figure C-7. Robbery victimization rate, nominal and effective sample sizes, and cost when 

keeping the number of interviews constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

39

Figure C-8. Aggravated assault victimization rate, nominal and effective sample sizes, and 

cost when keeping the number of interviews constant by alternative design

Note: TIS = time in sample.

Figure C-9. Simple assault victimization rate, nominal and effective sample sizes, and cost 

when keeping the number of interviews constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

40

Figure C-10. Intimate partner violence victimization rate, nominal and effective sample 

sizes, and cost when keeping the number of interviews constant by 

alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

41

Appendix D. Impact of alternative designs, by type of 

crime under keeping the cost constant conditions

Figure D-1. Property victimization rate, nominal and effective sample sizes, and cost when 

keeping the cost constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

42

Figure D-2. Household burglary victimization rate, nominal and effective sample sizes, and 

cost when keeping the cost constant by alternative design

Note: TIS = time in sample.

Figure D-3. Motor vehicle theft victimization rate, nominal and effective sample sizes, and 

cost when keeping the cost constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

43

Figure D-4. Theft victimization rate, nominal and effective sample sizes, and cost when 

keeping the cost constant by alternative design

Note: TIS = time in sample.

Figure D-5. Violent victimization rate, nominal and effective sample sizes, and cost when 

keeping the cost constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

44

Figure D-6. Rape and sexual assault victimization rate, nominal and effective sample sizes,

and cost when keeping the cost constant by alternative design

Note: TIS = time in sample.

Figure D-7. Robbery victimization rate, nominal and effective sample sizes, and cost when 

keeping the cost constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

45

Figure D-8. Aggravated assault victimization rate, nominal and effective sample sizes, and 

cost when keeping the cost constant by alternative design

Note: TIS = time in sample.

Figure D-9. Simple assault victimization rate, nominal and effective sample sizes, and cost 

when keeping the cost constant by alternative design

Note: TIS = time in sample.
Determining the Optimal Number of Interview Waves in the National Crime Victimization Survey: Evaluation and Recommendations

46

Figure D-10. Intimate partner violence victimization rate, nominal and effective sample 

sizes, and cost when keeping the cost constant by alternative design

Note: TIS = time in sample.
Office of Justice Programs

Innovation • Partnerships • Safer Neighborhoods

www.ojp.usdoj.gov

The Bureau of Justice Statistics of the U.S. Department of Justice is the principal federal agency responsible for measuring crime, criminal victimization, 

criminal offenders, victims of crime, correlates of crime, and the operation of 

criminal and civil justice systems at the federal, state, tribal, and local levels. 

BJS collects, analyzes, and disseminates reliable and valid statistics on crime 

and justice systems in the United States, supports improvements to state and 

local criminal justice information systems, and participates with national and 

international organizations to develop and recommend national standards 

for justice statistics. Jeri M. Mulrow is acting director.

NCJ249878