Assessing Sample Design Options for the 

National Crime Victimization Survey

Robert E. Fay and Jianzhu Li1

Westat, Inc. 

1600 Research Blvd, Rockville, MD 20850-3129, BobFay@westat.com, JaneLi@westat.com.

Abstract

The National Crime Victimization Survey (NCVS) began full-scale data collection in 1972. Over time, increasing costs of

data collection and a diminishing budget have forced a gradual reduction of the sample size from 72,000 households in 1972 

to 38,600 in 2005, leaving the NCVS far less able to meet its initial goals. The sponsor of the NCVS, the Bureau of Justice

Statistics (BJS), asked a panel of the National Academy of Sciences to review the survey. The panel’s interim report, released 

in 2008, recommended to BJS a series of actions, including a systematic review of a range of sample design options for the 

survey. 

Since the beginning of the NCVS, the Census Bureau has been the primary architect and data collector. The panel report

addressed the question of whether BJS should consider an alternative collector, but it detailed reasons for and against

continuing with the Census Bureau. The report recommended, however, that the Census Bureau provide BJS with detailed 

information on costs to allow external assessment of survey design options. The report also recommended that BJS consult

external sources to evaluate design options. On that basis, BJS has funded the research to be reported here. 

This paper summarizes our research design for the study. The panel report provided two forms of guidance for this research. 

For one, it listed the essential components of the design of a national household survey—the design, stratification, and 

selection of primary sampling units; methods for selecting households; the choice between a rotating panel or cross-sectional 

design; and possible subsampling of individuals within households. The panel report also recommended consideration of

specific alternative designs, including the design of the British Crime Survey. Our planned research responds to both forms of

guidance, attempting to build on the panel’s thoughtful review of the current status of the NCVS. 

1. Introduction

The National Criminal Victimization Survey (NCVS), begun as the National Crime Survey (NCS) in 1972, collects data on

the frequency and impact of crime by surveying the victims of crime. The survey has enjoyed a long history of research-based 

methodological improvements; for example, during the 1980s it was the first federal survey where measurement issues were 

viewed through the lens of cognitive psychology. During this entire period, the U.S. Census Bureau has had the primary

responsibility for the sample design and data collection, while the Bureau of Justice Statistics (BJS) has sponsored the survey

and published its findings. 

Over time, increasing costs of data collection and a diminishing budget have forced a gradual reduction of the sample size

from 72,000 households in 1972 to 38,600 in 2005, leaving the NCVS far less able to meet its initial goals. BJS requested 

that the Committee on National Statistics and the Committee of Law and Justice of the National Academy of Sciences, 

National Research Council (NRC), convene a panel to review BJS programs, including the NCVS. BJS specifically stated “A

focus of the panel’s work will be to consider alternative options for conducting the National Crime Victimization Survey, 

which is the largest BJS program” (quoted in NRC, 2008, p. 23). In its interim report, the panel reviewed the history of the 

NCVS, summarized key methodological literature, and assessed its current status. The panel proposed several design options

for consideration (NRC, 2008).
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The panel’s report raised a wide range of methodological issues, but several focused on the sample design. One of their 

recommendations (NRC, 2008, p. 6) stated 

Recommendation 4.7: BJS should investigate changing the sample design to increase efficiency, thus allowing

more precision for a given cost. Changes to investigate include: 

(i) changing the number or nature of the first-stage sampling units; 

(ii) changing the stratification of the primary sampling units;

(iii) changing the stratification nits of housing u ;

(iv) selecting housing units with unequal probabilities, so that probabilities are higher where victimization rates 

are higher; and 

(v) alternative person-level sampling schemes (sampling or subsampling persons within housing units). 

The effect of the recommendation is to provide both encouragement and broad advice to investigate alternative designs. 

Other parts of the report offer more detailed suggestions of possible research directions for improving the NCVS sample 

design. But the panel’s report stops short of advocating specific design changes without support from additional research. 

The primary contribution of the panel’s report, then, is to frame a series of researchable questions concerning the sample 

design. 

The NRC report also proposed a “design competition” to further the sample design research (NRC, 2008, p. 13), 

Recommendation 5.8: BJS should consider a survey design competition in order to get a more accurate reading 

of the feasibility of alternative NCVS redesigns. The design competition should be administered with the 

assistance of external experts, and the competition should include private organizations under contract and the 

Census Bureau under an interagency agreement. 

In effect BJS accepted the general point of this advice but applied it in a modified form. In place of a general design 

competition, BJS divided the major research tasks suggested by the panel’s report into four research grants, one of which 

concerned the sample design and was directly derived from Recommendation 4.7. BJS awarded the sample design grant to 

Westat in late 2008, and our work effectively began in 2009. The final report is due at the end of March, 2010, and we are 

sharing preliminary results with our sponsor on an ongoing basis. 

At the time of the writing of this paper, we expect to continue to refine the preliminary empirical findings we currently have, 

and we have not attempted to include them here. By agreement with our sponsor, we will focus in this paper on describing the 

design of our study and its intended contribution to the NCVS design. 

In the next section, we will summarize the goals for the project and our primary research strategies. Section 3 summarizes the 

current NCVS design and identifies aspects of the current design meriting review. Before summarizing our analytic approach 

to the project as a whole, we will describe an auxiliary data series that we hope will prove useful in the study. Section 4 

introduces the FBI’s Uniform Crime Reporting (UCR) Program. The early history of the NCVS can be told in part as an 

effort to develop an alternative to the UCR, particularly in areas in which the UCR has been acknowledged to be clearly 

deficient. Perhaps as a consequence, previous NCVS design efforts have largely avoided use of the UCR. But a basic 

principle of sample survey design is that auxiliary information does not need to be perfect in order to be useful. Our 

preliminary research has focused on strengths and weaknesses of the FBI’s UCR series for NCVS design purposes. 

Section 5 revisits the recommendations in the panel report related specifically to the question of sample design and outlines 

the program of research we have planned in response. Section 6 focuses on one aspect of these plans, our approach to the 

analysis of the relationship between the FBI’s UCR and the NCVS. 

2. An Overview of the Project

2.1 Project Goals

This first subsection will elaborate the goals of the project, which have already been sketched in the introduction. The

principal goal is provide a bridge from the advice provided in the panel’s report to specific interpretations and implications 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

applicable to the redesign of the NCVS. As previously noted, the panel’s report does not advocate a specific approach to the 

redesign but instead identifies alternatives and fruitful paths for research. The goal of the project is to clarify how and the 

extent to which these suggestions could be implemented in the upcoming redesign or later in the future. By assembling 

statistical evidence unavailable to the panel at the time of the report, the intention is to form specific evidence-based 

recommendations wherever possible. 

Several aspects of the project will consider the merits of various designs in terms of their relative cost and reliability tradeoffs. To the extent the Census Bureau is able to provide cost data to assist in answering these questions, we will be able to 

analyze the alternatives in terms of the current NCVS cost structure. We also intend to state the results more generally in 

terms of critical cost ratios. For example, a cost ratio might express the relative costs of a first-wave interview conducted 

primarily by personal visit with subsequent interviews conducted primarily by telephone. If BJS chooses to consider 

alternative data collectors for the survey, formulations in terms of cost ratios will provide BJS with a set of transferable 

findings. 

An addition goal of the project is to provide BJS with a final report that extends what the panel’s report started, namely, to 

document the underlying rationale for choosing among design alternatives. Although the current features of the NCVS 

sample design are adequately described, the Academy panel had relatively few documents to draw from in terms of the 

underlying rationale for the current design, particularly with respect to comparisons among alternatives. Initial technical work 

on the design from the 1980s and early 1990s has become dated to some degree because the properties and context of the 

NCVS survey have changed considerably since then (Rennison and Rand, 2007). The final report is intended to extend the 

panel’s work in addressing this gap. 

2.2 Methods

We plan to rely on the following approaches to achieving the project goals: 

2.2.1 Inventory the recommendations and relevant evidence reviewed by the NRC panel. The panel’s report addresses 

a variety of issues, but the project specifically requires focus on the sample design. In planning our project, we have found it 

useful to inventory the relevant recommendations, citations, and arguments in the panel report. A detailed account is planned 

as an appendix to the final report, but Section 5 provides a summary. 

2.2.2 Link the recommendations to researchable questions. Developing researchable questions appears in almost every 

research project in one form or another. But it is worth explicitly noting this step here. To be clear, the panel’s report does 

identify, in broad terms, research directions likely to be fruitful, but this project can contribute by translating the panel’s 

general advice into specific analyses to be carried out. Section 5 notes both general direction already provided by the panel 

report and the project’s subsequent contributions to identifying specific researchable questions. 

2.2.3 Identify publicly available evidence. Another general research strategy that nonetheless bears mention here is to 

review publicly available evidence relevant to the NCVS sample design. The panel’s report substantially accomplishes this 

task, but a logical first step is to identify in the panel report the key findings and citations relevant to the NCVS sample 

design. We will also attempt to identify additional material that either escaped mention or was published too recently to be 

included in the panel’s report. 

2.2.4 Identify questions that can be answered with public use data. A number of research questions can be addressed 

through public use data. Where possible, this approach is our method of choice because it facilitates replication of our 

findings by other researchers both inside and outside the Census Bureau. The empirical findings can readily be revisited in the 

future. 

BJS releases NCVS data through the National Archive of Criminal Justice Data (NACJD), which in turn is closely tied to the 

Inter-university Consortium for Political and Social Research (ICPSR). We have accessed the data through the standard 

agreement required of all participating researchers to use the data solely for statistical purposes rather than to obtain 

information about individuals. 

2.2.5 Where possible, address remaining questions in cooperation with the Census Bureau. Because the level of 

geographic detail is highly restricted on the public use files, a number of questions can only be addressed with data on the 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Census Bureau’s internal files. For example, the project will revisit the important question of whether the UCR provides 

county-level data sufficiently related to the concepts measured by the NCVS that the UCR could be incorporated in 

stratification, oversampling, or estimation at the county level. Section 6 will describe the plans for this analysis in more 

detail, but interest in county-level detail requires use of the Census Bureau’s internal NCVS data. 

In passing, we note that the Census Bureau has fully cooperated and willingly assisted in our use of the internal data. 

Currently, two Westat staff members associated with the project, including one of the authors (JL), is authorized to analyze 

specific NCVS data sets on the Census Bureau’s computers. Each specific critical finding we intend to remove from the 

Census Bureau and share publicly will be submitted to the Census Bureau’s Disclosure Review Board. 

3. The Current NCVS Sample Design

3.1 Fixed Design Features

Some aspects of the NCVS sample design arise almost inevitably from the core goals and survey methods of the NCVS. We 

plan to treat these fixed features simply as given. A starting point is the NCVS questionnaire, which we interpret as an 

expression of the measurement goals of the survey. Its basic design is divided into a set of screening questions intended to 

identify incidents of criminal victimization and a set of extended questionnaire items designed to collect specific information 

about each identified incident. (There is an exception for repeated incidents.) Respondents who report no victimizations 

during the screening phase are not asked extended items. The questionnaire is clearly designed for administration by 

interviewers—taken as a whole, the skip patterns and general flow of the questionnaire are complex, particularly the extended 

questionnaire items. The NCVS is not unique in this respect; even the questions on the venerable Current Population Survey, 

which measures monthly labor force status, require interviewer administration. 

The questionnaire is administered by interviewers either through face-to-face interviewing or by telephone. Over time, the 

telephone component has become the dominant mode, with a face-to-face interviewing being the primary mode only for the 

first in the series of seven NCVS interviews, which are spaced six months apart. Nonetheless, subsequent interviews may be 

completed by face-to-face interviewing if the household is unwilling or unable to be interviewed by phone. Replacement 

households who move into a sample address in later waves are also initially interviewed in person on their first interview. 

The current design is based on sampling housing units by address, rather than sampling telephone numbers. Instead, 

telephone numbers from the sampled households are collected during the initial interview. Although it is logically possible 

that a telephone-based sample could be considered as an alternative to the address-based approach, this alternative is 

essentially out of the scope of the project for two reasons. First, BJS directed the project to work within the framework of the 

current address-based approach. Secondly, achieving high response rates and population coverage through telephone-based 

sampling has become progressively more difficult, and it is questionable whether a telephone-based approach could achieve 

the current NCVS response rates. 

The combination of address sampling and a personal-visit component virtually forces the same basic strategy shared by other 

large national sample surveys that include a personal visit component. Surveys the size of the NCVS, including surveys that 

are several times larger, cannot efficiently sample persons for personal visit from each of the 3,000+ counties in the United 

States. In each case, a number of self-representing populous counties (including the largest, Los Angeles County, California) 

are included in the sample with certainty, but other non-self-representing counties are grouped into strata from which samples 

are drawn. Thus, the national estimate is composed of data from counties that represent themselves and data from other 

counties that represent entire strata, including non-sample counties. (A single notable exception to this generalization is the 

American Community Survey (ACS), which over a decade of operation will collect data from more households than the long 

form in the previous decennial census, Census 2000. The ACS draws sample from—and will produce estimates for—every 

county in the country. The much larger sample size of the ACS makes this design feasible.) 

Within the basic principle that some large counties should be sampled with certainty and small counties sampled within strata, 

there is latitude for variation in strategy. In many federal surveys, counties are grouped into primary sampling units of one or 

more counties. Primary sampling units (PSUs) are grouped into strata, and one or sometimes two PSUs are randomly selected 

from each stratum. Many national surveys adopt the principle of grouping all counties of a Metropolitan Statistical Area 

(MSA) into a single PSU, but not all do. Surveys may sometimes group two or more non-MSA counties together similarly to 

treat them as a single PSU. Thus, within the broad outline of the first-stage selection of sample counties, numerous 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

precedents define a range of specific strategies for forming and sampling PSUs. Section 5.1 will continue the discussion of 

this point. 

We will accept the use of counties in defining PSUs as a given, rather than some other system of sub-state geography. In a 

few states, particularly in New England, township or other geographic divisions may be used, but in general no other system 

of partitioning substate geography has proven as uniform or as stable across time as counties. 

3.2 Potentially Changeable Features of the NCVS Design

The previous section argues that some basic features of the design emerge naturally from the basic requirement for 

interviewers to administer the questionnaires with face-to-face interviewing in a significant portion of the overall sample. But 

the review by the NRC panel proposed re-examining many other aspects of the NCVS design, including some that have been 

stable fixtures in the design since the start of the NCS in 1972. To illustrate, Rennison and Rand (2007) summarized the 

evolution of the NCS/NCVS over its history, including changes to the sample size. They conclude their review by 

highlighting features in the current design that have been largely constant over the entire period (p. 48). 

The survey is still conducted for BJS by the Census Bureau using a sample of American households drawn 

from each decennial census. The NCVS uses the same sampling design as the NCS. Addresses remain in 

sample for three years and are interviewed seven times at six-month intervals. Interviews are conducted with 

every household member aged 12 and older. The interview is conducted in two parts. The first consists of a 

screening interview to identify and crimes experienced during the previous six months, and the second collects 

detailed information about the incidents. 

In relatively few words, the authors summarize persistent features of the sample design that are nonetheless within the scope 

of this project. To illustrate, the decision to interview the household at a sample address seven times was the likely outcome 

of earlier judgments to determine a near-optimum compromise. Assuming a constant response rate, cost considerations favor 

multiple contacts with the same household because the cost of an interview by telephone of an established sample household 

can be expected to be less than an initial contact in person for a first interview. These cost considerations are balanced by 

increasing nonresponse in subsequent waves and an apparent decline in reported incidents for many types of crime. The 

qualitative argument may still be made that sampled housing units should be included in sample more than once, but it is an 

open question whether seven has remained the optimal number of interviews. 

4. The Uniform Crime Reporting Program

In 1927, the International Association of Chiefs of Police convened a committee to propose methods to monitor trends in 

crime (Barnett-Ryan, 2007). Work of the committee led to the establishment of the Uniform Crime Reporting Program, with 

a first year of data collection in 1930. Since that beginning, the UCR Program has grown to cover most, although not all, of 

the country. Barnett-Ryan (2007) reviewed important changes in concepts and coverage since its beginning. Throughout this 

period, however, the UCR has been essentially a form-based system. Local enforcement agencies have summarized incidents 

of crime by category and other aggregate information on a series of forms. The program provided standards for classifying 

crimes, such as the distinction between aggravated assault (involving a weapon or resulting in serious physical injury) and 

simple assault (without a weapon and causing minor or no injury). Agencies are instructed to apply the Hierarchy Rule in 

classifying an incident in order to avoid counting the same incident twice. Similarly, they apply the Separation of Time and 

Place Rule to determine whether a sequence of events represents one incident or a series of incidents. To a large degree, the 

NCVS inherited these conceptual distinctions rooted in the development of the UCR. 

Starting in the 1980s, there has been an effort to convert the reporting of aggregate-level statistics to the National IncidentBased Reporting System (NIBRS). The basic unit in this reporting system is the incident, enabling much more detailed 

recording of information than possible under the form-based UCR. NIBRS electronic records can be summarized to the detail 

required by the UCR. As of yet, however, NIBRS covers only a fraction of the country, primarily states with small 

populations (Barnett-Ryan, 2007, Table 3.3, p. 83). Should the coverage of NIBRS extend to half of the country or more, 

then its more detailed data could benefit a future redesign of the NCVS, but that threshold has not yet been crossed. We plan 

no specific analysis of the NIBRS data. 

The overall quality of UCR statistics and their relationship to the NCVS has been a matter of considerable study and debate. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The NCVS succeeds in measuring a level of crime higher than reports to the police indicate. Nonetheless, we are 

investigating the extent to which UCR data can be used for sample design purposes. Our position is similar to that of Lynch 

and Addington (2007b, p. 3), who comment on the debate on the differences between the NCVS and UCR. 

One of the most significant works arising from this body of work is the 1991 book Understanding Crime 

Incidence Statistics: Why the UCR Diverges from the NCS written by Albert Biderman and one of the 

coeditors of the current volume, James Lynch. Biderman and Lynch’s work used the divergence of the UCRNCS trends as a vehicle for explaining how each data series measures crime differently and for emphasizing 

that it was acceptable (and even expected) for the two data series to diverge. Their work had two important 

results. Fist, it helped quell the “which is better” debates. Second, their work established a foundation for 

today’s commonly held perception that the two indicators are complementary as opposed to competing and the 

each system should enlighten the portion of the crime problem it is best equipped to address. 

We mention this position in defense of our approach to investigate the possible use of the UCR to improve the NCVS design, 

in spite of recognized differences between the two series. As we summarize our overall research strategy in the next section, 

we include analysis of the UCR as a candidate source of auxiliary information. Section 6 describes an important limitation of 

the UCR series, namely missing data, and our strategy to analyze the potential use of the UCR in NCVS design. 

5. Building upon the Panel’s Direction

Recommendation 4.7 from the Academy panel’s report, quoted in the first section, called for an investigation of ways to 

increase the precision of the survey under the constraint of a fixed cost. The recommendation suggested five broad areas for 

possible improvement, apparently ordered in a top-down fashion. We comment on how empirical work can build and sharpen 

the panel’s general direction. 

5.1 Investigate Changing the Number or Nature of the First-Stage Sampling Units

The general distinction between self-representing (SR) and non-self-representing (NSR) primary sampling units was 

introduced in Section 3.1. For a given total sample size, a typical strategy is to design so that the sample size required in each 

sampled NSR PSU represents a cost-efficient workload for a single interviewer. For surveys such as the NCVS, the 

interviewers, who must be recruited and trained, constitute important human capital. If paid by the hour or completed case, 

survey interviewers would typically want an adequate level of employment, so workloads must reach a certain average size to 

retain interviewers and amortize the fixed expenses in recruiting and training them. On the other hand, unrealistically large 

workloads present obstacles to timely completion and may force the hiring of additional interviewers. 

If the total sample size and the number of NSR strata are fixed, there remains some latitude in setting definitions of PSU 

boundaries to group counties. An interviewer may experience difficulty covering an assignment in a PSU composed of 

several geographically dispersed counties. In this situation, the time and expense devoted to travel become major components 

of the overall cost. On the other hand, to some degree combining counties may improve sampling efficiency. For example, to 

the extent that some types of crime may concentrate in central cities of MSAs, combining all of the counties in an NSR MSA 

together may result in lower variance as a result of balancing out the mix of counties between central-city counties and 

balance-of-MSA counties. 

Two related analyses, requiring access to internal Census Bureau files, can attempt to quantify these general principles. The 

UCR measures of crime may be used as a proxy for the expect values of the NCVS measurements, and alternative sample 

design strategies tested on this universe. To some degree, county-level NCVS estimates may also support limited analysis if 

averaged over a 10-year period. 

5.2 Investigate Changing the Stratification of the Primary Sampling Units

There are three major choices in defining a stratification of PSUs: (1) the division of the universe into self-representing 

counties and strata of non-self-representing counties; (2) the number of non-self-representing strata to form; and (3) the 

strategies to be used in forming the strata. 

The panel report (NRC, 2008, p. 154) notes that in 2005 the NCVS design was based on 110 NSR PSUs and 93 SR PSUs 

(which are effectively strata from the perspective of sampling theory). It also notes, however, that the original design based 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

on the 1990 census defined 152 NSR strata, with one PSU sampled from each. In October, 1996, the number of NSR PSUs 

was reduced by 42, leaving 110. The number of SR PSUs did not change in this reduction. Although somewhat disruptive, 

this decrease was presumably driven by the need to cut costs. In general, the optimum number of strata is tied to both cost 

considerations and the extent to which between-PSU variance contributes to overall variance. A small overall sample size 

discourages using too many NSR strata, but attempting to estimate national characteristics using too few NSR PSUs risks a 

large impact of between-PSU variance on the estimates. 

Previously, demographic and other characteristics have been used to define the sampling strata. We plan to investigate 

whether UCR data can be used to improve the outcome. A possibility we are researching is whether NSR strata should be 

defined on the basis of UCR data, either directly or in combination with other variables previously used. 

5.3 Investigate Changing the Stratification of Housing Units

Conditional on the first-stage selection of sample PSUs, housing units must be sampled within the sampled counties. In 

selecting the NCS/NCVS sample, the Census Bureau has used the previous census as the primary address frame, and 

supplemented by additional samples to cover new construction or areas where the census frame was insufficient. We plan to 

review the current methods used by the Census Bureau. 

If reliable subcounty estimates of crime were available, then this information could be useful in stratifying housing units 

within sampled counties. Our initial research has focused on county-level estimates from the UCR, but we plan to investigate 

whether adequately reliable subcounty estimates of crime could be useful. 

5.4 Investigate Selecting Housing Units with Unequal Probabilities

The full phrase in Recommendation 4.7 was “selecting housing units with unequal probabilities, so that probabilities are 

higher where victimization rates are higher.” The problem of optimal sample allocation is a classic one in the sampling 

literature, yet its findings are often not applied to large, multi-purpose national surveys for a mixture of appropriate and 

inappropriate reasons. (A notable exception should be made for economic surveys, which apply these methods routinely.) 

The Academy panel reasoned that if the reliability of a few key statistics, such as the violent crime rate and the property crime 

rate, were of central importance to the survey, then selective oversampling should be considered. 

Although the recommendation is stated in terms of sampling households at differential rates, reliable information at the county 

level can be the basis for the design of an unequal probability sample. County-level data can be the basis for differentially 

sampling counties with higher crime rates in SR PSUs relative to counties with low rates. For example, if the crime rate is 

twice as high in a central-city county as in a surrounding county, then sampling rates differing by a factor of 1.4 could 

improve the overall efficiency. Similarly, reliable county-level data can be converted to a measure of size to oversample NSR 

PSUSs with high crime rates. (By building the oversampling into the stratum definitions and the first stage of selection, 

interviewer workloads in high-crime NSR PSUs would not need to increase relative to low crime PSUs.) 

The UCR Program is the only likely source of such data, and the next section outlines our research plans to investigate 

whether it appears sufficiently reliable to be productively used for these purposes. 

Reliable within-county estimates of crime could be the basis of similar oversampling. The UCR collects reports from 

reporting enforcement agencies, most of which cover a county or part of a county. The geographic complexities of the UCR 

at the subcounty level are daunting, but it appears that in some cases it might be possible to distinguish large cities within 

counties from the balances of the county. We will also attempt to study whether statistics from a commercial vendor could be 

used. 

5.5 Investigate Alternative Person-Level Sampling Schemes

Since its beginning, the NCS/NCVS has interviewed the households residing a sample addresses a total of seven times. Until 

the 2007 estimates, the first interview was used as a “bounding interview” and excluded from the official estimates. 

Essentially all persons age 12 and over are to be interviewed. Except for series incidents involving six or more occurrences of 

the same crime, each reported incident identified in the screening leads to extended interview questions about the details of 

the incident. The Academy panel, in effect, asked whether improvements could emerge from changing one or more of these 

features. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

For example, subsampling could increase efficiency under favorable cost ratios; the subsampling could be of persons in the 

household prior to screening, of incidents reported during screening, reducing the extended interview component, or some 

combination of the two. The general topic includes the question of the cost/benefit trade-offs in the current NCVS strategy of 

including sampled housing units in sample for seven times. 

6. Missing Data in the UCR

In a recent paper, Lynch and Jarvis (2008) remark that comparatively less is known about measurement error in the UCR than 

the NCVS. They make an exception, however, for research on missing data in the UCR pursued by Michael Maltz and his 

collaborators. Maltz and Targonski (2002) identified missing data as a significant limitation on county-level data, in part as a 

methodological critique of work of Lott and Whitney. Lott and Whitney (2003) replied, including an a revised analysis based 

on only counties with more completely reported data, but Maltz and Targonski (2003) argued that problems would still 

remain as a result of the remaining influence of missing data. In subsequent years, Maltz and others (e.g., Maltz, 2006) have 

continued to research the problem, but no fully imputed data set appears to have emerged from their effort. As Lynch and 

Jarvis (2008) note, Maltz has approached the problem at lowest level of reporting, monthly figures by type of crime and 

jurisdiction; yet, an alternative strategy is to approach the problem at a higher level of aggregation. 

Our project goal is to improve the NCVS sample design, not to impute the UCR, but if we establish that the UCR is 

potentially useful we will have to specifically confront the issue of missing UCR data. We have taken a two-phase approach. 

Interestingly, our first phase currently parallels work of Lott and Whitney (2003), in trying to first limit the effect of missing 

data. We have identified a subset of approximately half of the counties in the U.S. where the UCR data appear least subject to 

missing data in recent years, up to 2006. Using internal Census Bureau files with county-level geography, we are currently 

investigating the extent to which the UCR predicts NCVS county-level crime rates. Because NCVS county-level estimates 

have very high sampling variance, a low correlation with UCR values is expected as a result of attenuation. Instead, we are 

using unweighted and weighted regression to investigate the strength of the statistical relationship, with the NCVS value as 

the dependent variable and the UCR value the independent variable. If our first-phase findings are negative, we will not 

further pursue use of the UCR. A set of positive findings, however, will encourage us to develop in the second-phase a 

simplified strategy to impute missing UCR county rates in a manner useful for design purposes, and then to propose an 

effective strategy for incorporating the information into the sample design. 

References

Barnett-Ryan, C. (2007), “Introduction to the Uniform Crime Reporting Program,” in M.A. Lynch and L.A. Addington (eds.) 

Understanding Crime Statistics (2007), pp. 55-89. 

Lott, J.R., Jr., and Whitley, J. (2003), “Measurement Error in County-Level Data,” Journal of Quantitative Criminology, 19,

185-198. 

Lynch, J.P. and Addington, L.A. (eds) (2007a), Understanding Crime Statistics: Revisiting the Divergence of the NCVS and

UCR, Cambridge University Press, New York, NY. 

______ (2007b), “Introduction,” in M.A. Lynch and L.A. Addington (eds.) Understanding Crime Statistics (2007), pp. 3-13. 

Lynch, J.P. and Jarvis, J.P. (2008), “Missing Data and Imputation in the UCR Crime Reports and the Effects on National

Estimates,” Journal of Contemporary Criminal Justice, 24, 69-85. 

Maltz, M.D. (2006), “Analysis of Missingness in UCR Crime Data,” Document NCJ 215343, NCJRS, National Institute of 

Justice, August 2006. 

Maltz, M.D. and Targonski, J. (2002), “A Note on the Use of County-Level UCR Data,” Journal of Quantitative 

Criminology, 18, 297-318. 

______ (2003), “Measurement and Other Errors in County-Level UCR Data: A Reply to Lott and Whitley,” Journal of

Quantitative Criminology, 19, 199-206. 
 

 

 

 

 

National Research Council (2008), Surveying Victims: Options for Conducting the National Crime Victimization Survey, 

Robert M. Groves and Daniel L. Cork, (eds.), Panel to Review the Programs of the Bureau of Justice Statistics, National 

Academy of Sciences, National Academy Press, Washington, DC. 

Rennison, C.M. and Rand, M. (2007), “Introduction to the National Crime Victimization Survey,” in M.A. Lynch and L.A. 

Addington (eds.) Understanding Crime Statistics (2007), pp. 17-54.