Rethinking the NCVS: 

Subnational Goals through Direct Estimation 

Robert E. Fay and Jianzhu Li1

Westat, Inc. 

1600 Research Blvd, Rockville, MD 20850-3129, BobFay@westat.com, JaneLi@westat.com

Abstract

Now almost 40 years old, the National Crime Victimization Survey (NCVS) provides annual estimates of the number 

of victimizations by different types of crime and estimates of the characteristics of the victims. The Bureau of Justice 

Statistics (BJS) is currently investigating several strategies to expand the usefulness of the survey. A major goal is to 

shift the almost exclusive focus of the NCVS as a national survey to a revised program capable of also providing 

subnational detail relevant to state and local governments. 

Until now, the NCVS has been conducted exclusively by the Census Bureau. Currently, BJS is investigating a

strategy that includes retaining a core NCVS preserving many of the basic features of the current design, including 

address sampling and personal visit as the primary mode for an initial interview. The core NCVS would continue to 

be conducted by the Census Bureau. At the same time, BJS is supporting research to determine the possibility of 

integrating the core NCVS with auxiliary or supplemental data collected by quite different survey strategies, modes, 

and collection agents. 

This paper will detail an investigation of one aspect of BJS’s overall strategy: To what extent can strategic sample 

boosting and allocation of the core NCVS meet a key set of subnational estimation goals through direct estimation? 

Direct survey estimates of annual crime rates for each of the states would require an unrealistic expansion of the 

survey, but the analysis reported here examines to what extent defining a restricted set of areas, such as states of over 

8 million population and using 3- or 5-year period averages in a manner similar to the American Community Survey, 

could achieve a set of useful results to address the needs of many NCVS users. 

1. Introduction

The Bureau of Justice Statistics (BJS) of the Department of Justice funds and publishes results from the National 

Crime Victimization Survey (NCVS), which is conducted by the U.S. Census Bureau. (The survey was called the 

National Crime Survey (NCS) prior to 1992.) By questioning a national sample of persons from the household 

population about crimes committed against them, it provides annual estimates of the number of victimizations by 

different types of crime and the characteristics of the victims. Launched in 1972, the survey has a long history 

(Rennison and Rand 2007), including a major redesign of the questionnaire implemented in 1992 after almost a 

decade of methodological research. 

Since the survey’s beginning, a number of aspects of the design have remained fixed: a panel design based on 

sampling housing units; panel interviews spaced 6 months apart, for a total of 7 interviews over 3 years; and selfresponse for ages 14 and over. (Now, ages 12-13 are also self-response with parental consent). The first interview 

has always been by personal visit, but subsequent interviews have been accomplished with multiple modes—

personal visit, telephone, or (for a period of time) centralized CATI—in varying proportions. 
Throughout its history, the survey has been analyzed almost exclusively at the national level. The sample size and 

design of the NCVS limit the geographic detail that can be usefully extracted from the survey. The principal 

publications from the survey emphasize national estimates. Until 2006, a set of detailed tables published by BJS 

(e.g., BJS (2006) for the 2005 estimates) included a few tables cross-classified by census region (Northeast/ 

Midwest/ South/West) and residence (urban/suburban/rural), but beginning in 2006 most of these tables have been 

dropped from the publication. (As noted in BJS (2006), the distinction urban/suburban/rural corresponds to central 

city/balance of metro area/non-metro as used by the Census Bureau.) 

A small number of NCVS studies focus on subnational analysis. In one, a separate RDD sample was interviewed as a 

supplement to the NCVS in 12 cities in 1998 (Smith et al. 1999) to obtain city-level estimates of crime and 

perceptions of community safety. In another study, two BJS researchers (Langan and Durose 2004) compared 1980-

1999 NCVS rates for New York City specifically with the NYPD crime statistics, finding general agreement in the 

trends over the time period examined. In a third study, Lauritsen and Schaum (2005) published estimates for the 

three largest metropolitan areas in the country, New York, Chicago, and Los Angeles. Their analysis was restricted 

to the core counties remaining continuously in the metropolitan areas during 1980-1998. To dampen the effect of 

sampling variability, the report presented 3-year moving averages for each area, which produced relatively smooth 

series when presented graphically. 

Each of the preceding three studies required special tabulations by the Census Bureau and could not have been 

carried out at the time with publicly available data. In 2007, BJS released through the Inter-University Consortium 

for Political and Social Research (ICPSR) a microdata file for the 40 largest MSAs for the period 1979-2004. The 

file contains person and incident-level data. With the release of the 2007 MSA file for 1979-2004, parts of the 

Lauritsen and Schaum (2005) study can now be replicated, but newer MSA data have not been released. The primary 

annual NCVS files available to researchers through the ICPSR have not carried state or specific metropolitan 

geographic codes, and it also appears that state estimates based on the NCVS have never been published. 

Instead of the NCVS, researchers and other data users interested in subnational analysis generally rely on the 

Uniform Crime Report (UCR) series of the FBI. The UCR is based on aggregating data reported by a large number 

of law enforcement agencies, over 18,000 in 2010. Participation is voluntary, however, and the completeness of 

reporting varies by area. For some purposes, including state estimates, the FBI imputes for missing data. Thus, the 

UCR provides far more geographic detail than the NCVS. But the UCR has well-recognized limitations. The degree 

to which law enforcement agencies consistently implement the same definitions in classifying crime remains an open 

question. In addition, UCR statistics can only reflect crimes reported to law enforcement agencies, whereas the 

NCVS collects data from respondents on both reported and unreported victimizations. Consequently, the ability to 

produce subnational estimates from the NCVS would enhance the knowledge of the incidence of crime and thereby 

also the value of the NCVS. 

Over the last few years BJS has pursued a research program to improve the NCVS’s capacity to produce subnational 

estimates of crime. A separate paper to be presented in this session (Li, Diallo, and Fay 2012) reports on our effort to 

develop model-based small area estimation approaches to the NCVS data, with the goal of estimating victimization 

rates and related characteristics for states, large counties, and perhaps large cities. The paper builds on previous work 

(Cantor, et al. 2010), which outlined possible approaches to apply small area methods to NCVS, and on a study 

(Krenzke, Li, and Cantor, 2009) that included an initial attempt to develop a model for the 2003 estimates from the 

1979-2004 MSA file. 

Adopting a different strategy to subnational estimation, another paper in this session (Brick, Edwards, and Lohr 

2012) describes research funded by BJS to develop a lower-cost crime survey in parallel with the core NCVS. 

Results from the parallel (or “companion”) survey would then be combined with core NCVS data, possibly through 

an appropriate statistical model, to produce subnational estimates of crime. 
This paper reports on our research to examine the degree to which some subnational estimation goals can be met by 

expanding the size and revising the sample design for the NCVS. The question of the NCVS sample design is a 

timely one, because the Census Bureau in consultation with BJS is in the process of redesigning the NCVS sample. 

The next section of this paper selectively reviews U.S. surveys that illustrate the transformation of a national sample 

design to a design serving both national and subnational purposes. A prime example is the American Community 

Survey (ACS), the foremost survey providing sample data at highly detailed geographic levels, with the exception of 

the decennial census itself. To do so, however, the ACS makes extensive use of time-averaging, which provides an 

important precedent for NCVS. The section also examines other precedents for direct subnational estimation, 

including the Current Population Survey (CPS). 

Whatever modifications the upcoming redesign may incorporate, the NCVS will remain a complex, multi-stage 

sample with each stage contributing to the overall variance. The third section summarizes our research to assess the 

effect of the design at each stage. A finding of particular importance is the considerable contribution of between-PSU 

variance to the total variance. Consequently, we have incorporated allowances for between-PSU variance in our 

consideration of direct state estimates. 

The fourth section describes how the results are used to develop a “public” model of NCVS sampling. The model 

uses publicly available data from the census, NCVS, and other sources to approximate the methods used by the 

Census Bureau in the NCVS design. The intent is only to approximate, rather than fully anticipate, design decisions 

to be made by the Census Bureau. The public model of the design does not disclose any potentially sensitive 

information about the current actual design, which the Census Bureau increasingly protects because of confidentiality 

concerns. The sensitivity of this public model to variations in the assumptions can be readily tested.

The fifth section reports on the results from the simulation. Except for a refinements implemented for this paper, the 

results are quite similar to those we provided to BJS several months ago as guidance in design decisions. A 

discussion section remarks on potential extensions of this research to other decisions to be faced in the upcoming 

NCVS redesign, and potentially to the redesign of other national surveys. 

2. Surveys Designed For Subnational Estimation

On an annual basis, victimization, particularly by violent crime, is fortunately a relatively rare event. Consequently, 

the reliable measurement of violent crime requires large samples. 

Reaching essentially its intended size in 2005, the American Community Survey (ACS) draws an initial sample of 

approximately 3 million housing units each year. The survey is a successor to the decennial census long form and 

produces annual estimates for large areas, including states. All ACS estimates are based on averaging over time: 

Even the 1-year estimates represent a period estimate of characteristics collected over the course of a year, and 

smaller geographic levels are only published as 3- or 5-year period estimates. In spite of some initial concerns over 

the public acceptance of the time-averaged estimates (e.g., National Research Council, 2007), the approach has 

largely proven successful, perhaps in part as a result of the Census Bureau’s efforts to educate users on the 

interpretation of the results. These efforts included collaborations between the Census Bureau and outside 

researchers to produce a series of 12 handbooks offering guidance to data users, included as part of the ACS’s 

Compass Products. The handbooks are published online at the Census Bureau’s ACS website. 

Averaging of results over time has appeared in a number of previous survey applications, but the emergence of the 

ACS doubtless has had the effect of increasing the familiarity of many users with the underlying concept. The ACS 

estimates are now widely consulted as the primary source for basic subnational demographic and social statistics for 

the U.S., and the ACS illustrates the potential utility for NCVS subnational estimates based on accumulating data 

over a multi-year period. We focus in this paper on 3-year NCVS state estimates. Using 3-year periods permits 

alignment with 3-year ACS estimates and provides greater timeliness than 5-year estimates, the other precedent 

offered by the ACS. 
BJS has identified the victimization rate for violent crime as a key statistic to be produced for states or other 

subnational areas. In considering a sample redesign to achieve this goal, the logical question becomes: How reliable 

should these estimates be? For example, if the victimization rate were 2% (20 per 1,000), what target coefficient of 

variation (CV) should set as a goal—10% (corresponding to a standard error of 2 per 1,000), 8%, 12%, or some 

other percent? There is no single “textbook” answer to this question, but we attempted to review some of the relevant 

past experience. 

For ACS users, a somewhat similar choice in reliability is often available for large areas, namely whether to use the 

most recent 1-year estimates or choose instead 3- or even 5-year estimates in order to reduce the impact of sampling 

variance by accepting a reference period reaching somewhat further back in time. We consulted four of the 

handbooks from the Census Bureau’s Compass Products (U.S. Census Bureau, 2008a, 2008b, 2009a, 2009b), “What 

General Data Users Need to Know,” “What Federal Agencies Need to Know,” “What State and Local Governments 

Need to Know,” and “What Researchers Need to Know”, to locate any specific guidance on tradeoffs between 

reliability and timeliness in the use of multi-year estimates. 

Of the four, only the state and local government handbook offers any explicit guidance on the level of reliability 

users should seek: “For some applications, CVs that are in excess of 15 percent should be a cause for concern” (p. 

5). Elsewhere on the same page, the handbook also states: 

While there is no hard-and-fast rule, for the purposes of this handbook, estimates with CVs of 

more than 15 percent are considered cause for caution when interpreting patterns in the data. The 

choice of a CV level threshold that distinguishes a reliable estimate from an unreliable estimate 

will vary by application. While CVs for 3- and 5-year estimates would be generally lower than 

that of the 1-year estimate, one could go with the most recent 1-year estimate if its CV were 15 

percent or lower. 

This passage appears to endorse estimates with CVs below 15 percent as acceptable. Otherwise, the handbooks 

appear to have avoided characterizing any specific level of reliability as desirable. In fact, each of the four 

handbooks reviewed includes an Appendix 1 with an illustration of the greater smoothness to be expected from 3- 

and 5-year period estimates compared to 1-year estimates for the same area. Thus, even if a 1-year estimate should 

meet a specific standard, such as 15%, multi-year averages may be more interpretable by dampening much of the 

random variation from sampling variance. 

Although most large surveys are multi-purpose, the ACS is particularly so, and it is not designed to meet any one 

standard of reliability for a specific variable. Because its sample size is so large, no consideration of the reliability of 

state estimates is explicitly built into the allocation of the ACS sample. Instead, the ACS is allocated by considering 

the reliability of much smaller geographic units. 

The evolution of the design of the Current Population Survey (CPS) provides a more useful precedent in redesigning 

a national survey for state estimates. The CPS was designed initially as a monthly national labor force sample to 

meet a specific goal for estimating the monthly national unemployment rate. The survey was redesigned, over a 

period of years, to meet state estimation goals, while continuing to meet the national objectives. The details are 

provided in Technical Papers No. 40, No. 63, and No. 66 (U.S. Census Bureau, 1978, 2000, 2006, respectively). 

In 1973, annual averages were published for 19 states meeting the requirement of a 10% CV for an unemployment 

rate of 5%. These states were typically the most populous at the time. For 1974-1975, the requirement was changed 

to a 10% CV for an unemployment rate of 6% and, in combination with a revision to the estimation procedure, 8 

more states were added to the list for publication. In 1976, a supplemental sample was added to the remaining 23 

states and DC to achieve the 1974-1975 standard (10% CV on a 6% rate) for all states. Subsequently, in the redesign 

of the CPS sample following the 1980 Census, all states were state-stratified to simplify state estimation. The 10% 

CV standard survived for a number of years, but Technical Paper 63 in 2000 reported (p. 3-1) that in the redesign 
following the 1990 Census, completed in 1995, part of the CPS sample in the most populous states was reallocated to 

the smaller states to achieve an 8% CV in every state. Unfortunately, Technical Paper 63 outlined this change 

without explaining in detail the underlying considerations. It would be reasonable to assume, however, that the 

Bureau of Labor Statistics was not entirely satisfied with the results of the 10% CV standard and was willing to 

reallocate the sample to achieve a tighter 8% standard in each state, even at the cost of a small loss in national 

precision. Technical Paper 66 in 2006 (p. 3-1) confirmed that the same requirements for state-level accuracy were 

still in force in July 2005 at the end of the redesign based on Census 2000. 

Another Census Bureau survey, the Survey of Income and Education (SIE) in 1976, was also designed for specific 

state estimation goals. The estimates of children in poverty age 5-17 were to be used to allocate federal education 

funds to states. In consultation with the Department of Health, Education, and Welfare, the survey specification was 

set to produce a 10% CV in each state on the required estimate, far more precise than produced by the CPS. One of 

the authors (Fay) worked on the evaluation of the survey. The variability of the SIE estimates turned out to be an 

obstacle, however, and many questioned specific estimated changes in poverty since the 1970 census. Congress 

implemented a hold-harmless provision substantially weakening the impact of the survey estimates on the allocation. 

In this case, which involved the allocation of funds, a 10% CV may not have been sufficient to convince skeptical 

users. This example is cautionary but does not rule out the usefulness of estimates with a 10% CV. The SIE was a 

one-time survey, making its single set of estimates a particular target of scrutiny. A program of continued publication 

of state estimates from a redesigned NCVS would offer the opportunity to track unusual estimates over time, so that 

3-year estimates that appeared to be outliers might gradually be either confirmed or replaced with less unusual 

estimates. 

In summary, there is some precedent for designing the NCVS to meet target CVs on the 3-year period estimates for 

state rates of victimization by violent crime in the range of 8% to 10%, although CVs slightly above this range may 

also provide acceptable alternatives. During the 1970s, CPS annual estimates of unemployment were published for 

some but not all states for a few years, providing a precedent to NCVS to publish results only for a set of large states. 

The reallocation and increase of the CPS sample in the 1990s to improve the state estimates to achieve an 8% in 

place of a 10% CV is cautionary, because at the time of this decision considerable empirical experience had 

accumulated on the stability of the annual estimates under the 10% target. With a 10% CV target, the SIE as a onetime survey may have fallen just short of full acceptance by Congress; nonetheless, the survey results were widely 

used at the state level for other purposes. Because an ongoing NCVS would continue to produce estimates over a 

longer period, the accumulated data would be additionally informative relative to achieving only a single 3-year 

period estimate for each state, so that a 10% target may prove acceptable. To illustrate a balance between cost and 

reliability, a 10% CV was used in the simulation to be reported here. 

3. Design Effects in NCVS

As is typical of national household surveys that include personal visit as one of their modes, the sample design for 

the NCVS is quite complex. At the first stage of selection, counties are grouped into primary sampling units (PSUs). 

Under the current design, very roughly half of the population is in self-representing (SR) PSUs that are, in effect, 

sampling strata. Essentially all of these SR PSUs are metropolitan areas, although the Census Bureau does not 

publish the exact details for reasons of confidentiality. 

The other PSUs are composed of the remaining, generally smaller, metropolitan areas and groups of nonmetropolitan counties. In many cases, non-metropolitan PSUs are single counties. These non-self-representing (NSR) 

PSUs are grouped into NSR strata. In the past and current NCVS designs, the NSR strata typically cut across states. 

In general, one PSU is selected from each NSR stratum, which also has been the practice for the CPS and some of 

other national surveys conducted by the Census Bureau. In 1996, the number of NSR PSUs was reduced from 152 to 

110 by grouping some of the NSR strata and randomly deleting one of the sampled NSR PSUs from each group. 

This aspect of the design persisted until the subsequent redesign implemented in 2005/2006. 
Within sampled PSUs, segments of 4 housing units constitute the next stage of selection. The primary sampling 

frame for the segments is based on the previous decennial census, but other frames represent new construction and 

areas where the census enumeration lacks sufficient detail to serve as an adequate frame. In the current design, there 

is no further subsampling: All households in the segment are eligible for interview and all persons age 12 and over 

within each household are eligible. 

Estimates over time are correlated from two sources. The NSR PSUs remain fixed until the next decade redesign, 

when new PSUs replace the old during the course of about a year. Within PSUs, segments remain in sample for 

seven interviews, but the segments that replace them are drawn from neighboring areas within the PSU in most cases. 

Thus, correlation over time arises from both the first-stage selection of NSR PSUs and within PSUs from the panel 

nature of the design and the method of replacing retiring segments. 

An earlier phase of the project focused on identifying aspects of the design where significant improvements were 

possible. As sworn agents of the Census Bureau, we used the internal files to estimate the contribution to overall 

variance from the different components of the design, using information unavailable on the external files. Our 

approach was to estimate the sampling variance under the current design and under various misspecifications of 

design. For example, we computed variances for the crime rate separately in both the SR and NSR universes. By 

treating the NSR PSUs as separate strata, we were able to estimate the within-PSU variances. Similarly, we estimated 

the effect of clustering housing units into segments of size four, the clustering effect of sampling all eligible persons 

within households, and the clustering effect from interviewing the same persons twice in a year in many cases rather 

than replacing the sample every six months. 

Victimization rates in the NCVS are true rates, that is, they are ratios of events occurring to a population in the 

numerator and the exposed population in the denominator. They are not, however, true proportions, because 

individuals can report more than one event and thus contribute more than once to the numerator of the rate. 

Nonetheless, it is convenient to model the observed variances relative to a binomial-like variance, p p n ˆ ˆ (1 ) /  , 

where pˆ is the weighted NCVS estimate of the victimization rate, and n is the unweighted count of individuals. 

Additional complications arise from the fact that many individuals are interviewed twice over the course of a year 

but others only once. Informally, the ratio of the actual estimated variance to the binomial-like variance might be 

termed a design effect, although this ratio is not a true design effect as Leslie Kish originally defined the term. 

We found that the first stage of selection has a particularly important effect on the reliability of NCVS estimates. By 

using variance estimates over the period 1996-2009, the variance ratio/design effect in the SR PSUs was about 1.7 

for the annual violent crime victimization rate. The corresponding value in NSR counties was about 3.2. In other 

words, empirically the NSR strata appear to contribute variance at almost twice the rate of the SR strata. (The NSR 

estimate of 3.2 incorporated an adjustment to remove the transient effect of weight variation 1997-2005 due to the 

1996 reduction in the NSR PSUs noted earlier. Thus, 3.2, rather than the original value, 3.6, is a more accurate 

representation of the intrinsic design situation. We also dropped 1996 and 2005 because of unusual patterns in the 

pseudo-stratum and half-sample assignments in these two years which may have distorted the estimated NSR 

variances.) 

In considering the use of 3-year averages for states, we computed the variance ratios/design effects for the 3-year 

national average rates falling within the ranges 1997-2004 and 2006-2009. The results were 2.3 and 6.0 in SR and 

NSR strata, respectively. In other words, because of correlation over time, 3-year averages are approximately 77% as 

efficient as 3 independent annual samples in SR areas and 53% as efficient as 3 independent annual samples in NSR 

areas. 

4. Simulating Sample Allocation Decisions with a Public Model

As previously noted, the Census Bureau restricts the publication of details about the NCVS sample design for 

reasons of respondent confidentiality. The degree of this protection has increased over time. For example, Technical 
Paper 40 (U.S. Census Bureau, 1978) included maps showing the sampled counties for the Current Population 

Survey (CPS) as of March 1973 and August 1976, but now the identity of sampled PSUs—particularly NSR PSUs—

is treated as confidential. Some geographic information from the CPS nonetheless remains available. The public use 

file from the Annual Demographic Supplement includes state codes and geographic codes for numerous large 

metropolitan areas. 

As already noted in the introduction, limited geographic information is currently available for the NCVS. Some facts 

about the NCVS may be reasonably assumed from publicly available information. Statisticians with experience in 

sampling would presume that the very largest metropolitan areas, such as New York and Los Angeles, are entirely or 

almost entirely included in the sample with certainty. In fact, the previous publication of microdata for the 40 largest 

MSAs assures that the NCVS sample falls in each, and it may reasonably be assumed, without risk of disclosure, that 

all or virtually all of them are certainty. The current number of NSR PSUs is not publicly available, but the counts in 

1995 and 1996, 152 and 110, respectively, are. 

In order to provide a means to consider the implications of different possible designs, we have built a “public model” 

of the NCVS design based on publicly available data and facts about the NCVS. The model simulates the process by 

which the Census Bureau may construct the design, without intending to either exactly replicate either the current 

design or dictate the exact course of the future redesign. We are using the results to assist BJS to set goals for the 

redesign and to inform the Census Bureau on how a set of alternatives might be implemented. 

When it was initially developed last spring, the model incorporated the 2010 state apportionment counts and 2009 

population estimates for counties and New England townships. (The 2009 estimates have now been replaced with the 

2010 census counts.) It also employs the 2009 definitions of Metropolitan CBSAs (Core Based Statistical Areas) and 

NECTAs (New England City and Town Areas). It also notes whether these areas are identified on the Current 

Population Survey public use file.

The model constructs a hypothetical design for the national sample to the extent that it proposes a set of SR areas 

chosen from the largest metropolitan areas. One choice made in developing the model—which may disagree with the 

Census Bureau’s approach—was to separate state pieces of metropolitan areas, such as dividing the New York 

CBSA into its New York, New Jersey, and Pennsylvania components. A second step was to estimate the population 

size of a typical NSR stratum. We have used the figure 900,000 as a rough guess, derived from the observation that 

the SR component is likely to be just a bit over 50% of the population and that 152 NSR PSUs, the number of NSR 

PSUs in 1995 before a sample reduction, might be a more representative value that 110 NSR PSUs remaining after a 

significant NSR sample reduction. 

We also assumed that the Census Bureau would classify as certainty any CBSA/state area that was above the NSR 

stratum size, because the area would constitute an NSR stratum by itself. Additionally, we assumed that it would be 

reasonable to extend that bar downward somewhat to include areas that were close to the NSR stratum size. We used 

90% as a threshold (that is, all CBSA/state areas above 810,000), but other values could be considered. We also 

imposed what might be thought of as a “DC rule”—if (i) any CBSA/state area was not in the initially defined set of 

SR PSUs, but one or more pieces of the CBSA were already SR in another state, and (ii) the CBSA/state area in 

question was a CPS publication area (indicating a size considered not a disclosure risk in the CPS context), then the 

piece in question was also classified as SR. In the initial identification of SR areas, both the Maryland and Virginia 

components of the Washington-Arlington-Alexandria-DC-VA-MD-WV Metro area are included, but the District of 

Columbia is added under this rule. At this point, the model then calculates the population in the SR portion for each 

state. 

The model incorporates a projection, 95,211 persons, for the 2011 sample size, based on earlier estimates of the 

sample increases for 2011 compared to 2010. It currently uses a base rate of .02 for the annual violent crime rate and 

the variance ratios/design effects for the SR and NSR areas previously noted. 
Counting CBSA/state pieces separately, the model proposes 75 SR PSUs and 148 NSR PSUs in the initial sample, 

with 57% of the population in SR areas. Strictly speaking, only three states are estimated to have CVs under 10%: 

California, Texas, and New York, although Florida at 10.1% would also qualify for all practical purposes.

To consider state estimation goals, the model assumes that state stratification would be applied in any state to be 

published, including the largest four states. 

An iterative process was used to determine the state sample size for states requiring supplementation. A ratio of the 

variance under the model for the national design to the desired variance was used as an initial factor to increase the 

state allocation. The factor was also used to adjust the threshold to classify a PSU as self-representing; for example, 

in a state whose sample size were to double from the original national allocation, the threshold for a SR PSU would 

be proportionately reduced to 450,000 (=900,000/2). Both metropolitan areas and large NSR counties were

candidate SR areas. When the revised threshold would increase the proportion of the SR population, the variance 

model would predict that the sample size adjustment was now too large, and it would then revise the adjustment to 

compensate. The number of SR PSUs might decrease as a result of this second adjustment. The model iterated this 

adjustment process 10 times, ending with a sample size adjustment to achieve the CV goal based on the SR PSUs 

currently identified. 

5. Results 

Based on the 2010 apportionment counts, we grouped states by population boundaries to determine possible 

groupings of states to consider for direct state estimation. The 2010 counts show a natural boundary at 8 million 

because no state falls between 7 million and 8 million. Table 1 shows the groupings of states added for each 

alternative design that we considered. Figure 1 shows the relative contributions to the U.S. population from this 

grouping of states. 

Table 1. Grouping of states considered for publication. The first row reports the states included in an 

initial proposal, and each subsequent row shows the additional states added by lowering the threshold. 

Population 

cutoff 

Number of 

added states 

Added states 

8 million+ 12 California, Texas, New York, Florida, Illinois, Pennsylvania, Ohio, 

Michigan, Georgia, North Carolina, New Jersey, Virginia 

6 million+ 5 Washington, Massachusetts, Indiana, Arizona, Tennessee 

5 million+ 5 Missouri, Maryland, Wisconsin, Minnesota, Colorado 

4 million+ 4 Alabama ,South Carolina, Louisiana, Kentucky 

3 million+ 4 Oregon, Oklahoma, Connecticut, Iowa 

2 million+ 6 Mississippi , Arkansas, Kansas, Utah, Nevada, New Mexico 

1 million+ 7 West Virginia, Nebraska, Idaho, Hawaii, Maine, New Hampshire, 

Rhode Island 

All 8 Montana, Delaware, South Dakota, Alaska, North Dakota, Vermont, 

District of Columbia, Wyoming 

Figure 2 shows the sample size and national CV on the annual estimate under the projected 2011 sample size (at the 

far left), where 3 states would be publishable, and the alternatives in Table 1. The increase in the sample size to yield 

12 estimated states is comparatively small, but the sample size increases to almost 3 times the original to reach the 

goal of all states. There are also some gains in national precision, but they are less than if the sample had been 

distributed proportionately. 
Fig. 1 Shares of the total U.S. population for states grouped by size. 

Figure 3 shows the increase in the number of SR and of NSR PSUs under the alternatives. The count of SR PSUs 

may be exaggerated in a few New England states—Connecticut (14), New Hampshire (14), Rhode Island (4), and 

Vermont (14)—because each is forced to be 100% SR when supplemented, but the resulting counts of SR PSUs 

might overstate how they would be later counted by the Census Bureau. 

6. Discussion

BJS is currently considering proposing to supplement all of the states, the rightmost option shown in each of the 

figures. Because design work must begin well before Congress will consider this proposed increase, the design work 

needs to reflect BJS’s intended direction but prudently develop fallback plans in case the necessary funds are only 

partially approved or disallowed. Under this scenario, every state would be state-stratified. The Census Bureau could 

also simultaneously lay the foundation for a fallback national design by determining what PSUs should be SR under 

that situation. After the identification of SR PSUs and the formation of strata in NSR PSUs for the state-based 

design, every SR PSU, except national SR PSUs, and every initial NSR stratum could be assigned a permanent 

random number for use in case the proposed sample might be cut. 

In the possible scenario that Congress should entirely disallow the increase but support the survey at the current 

level, the state-based design could remain intact in the four largest states but reduced in the others. In some of the 

largest states, it may be possible to reduce the sample within the state by retaining the national SR PSUs (but 

reducing the sampling rates within them to the national rate), stratifying the remaining SR PSUs and NSR strata (now 

represented by sampled NSR PSUs) into super-strata, and drawing an appropriate first-stage sample. This approach 

would retain some effect of state stratification. In most states, however, the remaining SR PSUs (except for the 

national SR PSUs) and NSR strata could be appropriately grouped into super-strata at the regional level and 

subsampled to national rates. 

Alternatively, a compromise might be achieved, supporting state estimates for some but not all states. In this case, 

the initial design would be retained in all of the publishable states, but in the other states the SR PSUs (except again 

for national SR PSUs) and the NSR strata would be grouped into strata at the regional level and again subsampled to 

national rates. 
The large variance ratios/design effects in NSR areas for 3-year averages invite a broader consideration of the NCVS 

design. Sampling a larger number of NSR PSUs with reduced workload per PSU may be cost-effective in terms of 

the resulting reliability for the 3-year averages. Whether this possibility should be pursued will depend on assessing 

any risk associated with reducing the average assignment workload for NCVS interviewers in NSR PSUs. 

Fig. 2 Estimated sample size and percent CV on the national rate of violent crime under supplementation 

alternatives. The results show three states publishable without supplementation at the projected size of 

the 2011 sample at the left-hand side, and the other points denote results for states above 8, 6, 5, 4, 3, 

2, and 1 million and for all states at the right. 
Fig. 3 Estimated number of SR and NSR PSUs under the scenarios of Fig 1 and Table 1. As new states are 

supplemented, additional SR PSUs are identified. The number of NSR PSUs also grows as states are 

supplemented.

Acknowledgements 

We wish to thank the Bureau of Justice Statistics for their support of this research and Dr. Michael Planty for his 

help and guidance. We also acknowledge considerable assistance from current and former members of the Census 

Bureau staff, including Stephen Ash, Barbara Blass, Jeremy Shimer, and Meagan Wilson. The views expressed here 

are ours and do not reflect those of the Bureau of Justice Statistics, the Census Bureau, or Westat. 
References 

Barnett-Ryan, C. (2007), “Introduction to the Uniform Crime Reporting Program,” in M.A. Lynch and L.A.

Addington (eds.) Understanding Crime Statistics (2007), pp. 55-89. 

Brick, J.M., Edwards, W.S., and Lohr, S. (2012), “Designing a Low(er)-Cost Complement to the National Crime

Victimization Survey,” to be presented at the Federal Committee on Statistical Methodology Research 

Conference, Washington, DC, Jan. 10-12, 2012. 

Bureau of Justice Statistics (2006), “Criminal Victimization in the United States, 2005, Statistical Tables,” NCJ 

215244, issued by the Bureau of Justice Statistics, Dec. 2006. 

Cantor, D., Krentzke, T., Stukel, D., and Rizzo, L. (2010), “NCVS Task 4 Report: Summary of Options Relating to 

Local Area Estimation,” issued by Westat to the Bureau of Justice Statistics, May 19, 2010. 

Kalton, G., Brick, J.M., and Lê, T. (2005). “Estimating Components of Design Effects for Use in Sample Design,” 

Household Sample Surveys in Developing and Transition Countries, Chapter VI, United Nations, New York, pp. 

95-121.

Krenzke, T., Li, J., and Cantor, D. (2009). “NCVS Task 3: Alternative designs for local area estimates,” draft 

memorandum from Tom Krenzke, Jane Li, and David Cantor of Westat to Michael Rand, Allen Beck, and Erika 

Harrell of the Bureau of Justice Statistics. Rockville, MD: Westat, June 2, 2009. 

Langan, P.A. and Durose, M. (2004), “The Remarkable Drop in Crime in New York City,” dated Oct. 21, 2004 

prepared for the International Conference on Crime, Rome, December 3-5, 2003, and downloaded on Nov. 11, 

2011from http://www3.istat.it/istat/eventi/2003/perunasocieta/relazioni/Langan_rel.pdf. 

Lauritsen, J.L. and Schaum, R.J. (2005), “Crime and Victimization in the Three Largest Metropolitan Areas, 1980-

1998,” NCJ 208075, issued by the Bureau of Justice Statistics, March 2005. 

Lê, T., Brick, J.M., and Kalton, G. (2002), “Decomposing Design Effects,” Proceedings of the Survey Research 

Methods Section, American Statistical Association, Alexandria, VA, pp. 2007-2012.

Li, J., Diallo, M.S., and Fay, R.E. (2012), “Rethinking the NCVS: Small Area Estimation Approaches to Estimating 

Crime,” to be presented at the Federal Committee on Statistical Methodology Research Conference, Washington, 

DC, Jan. 10-12, 2012. 

Lynch, J.P. and Addington, L.A. (eds) (2007a), Understanding Crime Statistics: Revisiting the Divergence of the 

NCVS and UCR, Cambridge University Press, New York, NY. 

National Research Council (2007), Using the American Community Survey: Benefits and Challenges, C.F. Citro and 

G. Kalton, (eds.), Panel on the Functionality and Usability of Data from the American Community Survey, 

National Academy of Sciences, National Academy Press, Washington, DC. 

National Research Council (2008), Surveying Victims: Options for Conducting the National Crime Victimization 

Survey, Robert M. Groves and Daniel L. Cork, (eds.), Panel to Review the Programs of the Bureau of Justice 

Statistics, National Academy of Sciences, National Academy Press, Washington, DC. 

Rennison, C.M. and Rand, M. (2007), “Introduction to the National Crime Victimization Survey,” in M.A. Lynch 

and L.A. Addington (eds.) Understanding Crime Statistics (2007), pp. 17-54. 
Smith, S.K., Steadman, G.W., Minton, T.D., and Townsend, M. (1999), “Criminal Victimization and Perceptions of 

Community Safety in 12 Cities, 1998,” NCJ 173940, issued by the Bureau of Justice Statistics, May 1999. 

U.S. Census Bureau (1978), “Current Population Survey, Design and Methodology,” Technical Paper 40, issued 

January, 1978. 

______ (2000), “Current Population Survey, Design and Methodology,” Technical Paper 63RV, issued March 2000. 

______ (2006), “Current Population Survey, Design and Methodology,” Technical Paper 66, issued October 2006. 

______ (2008a), “A Compass for Understanding and Using the American Community Survey Data: What General

Data Users Need to Know,” issued Oct. 2008, downloaded 24 Mar 2011 from 

http://www.census.gov/acs/www/Downloads/handbooks/ACSGeneralHandbook.pdf. 

______ (2008b), “A Compass for Understanding and Using the American Community Survey Data: What Federal

Agencies Need to Know,” issued Dec. 2008, downloaded 24 Mar 2011 from 

http://www.census.gov/acs/www/Downloads/handbooks/ACSFed.pdf. 

______ (2009a), “A Compass for Understanding and Using the American Community Survey Data: What State and 

Local Governments Need to Know,” issued Feb. 2009, downloaded 24 Mar 2011 from 

http://www.census.gov/acs/www/Downloads/handbooks/ACSstateLocal.pdf. 

______ (2009b), “A Compass for Understanding and Using the American Community Survey Data: What 

Researchers Need to Know,” issued May 2009, downloaded 24 Mar 2011 from 

http://www.census.gov/acs/www/Downloads/handbooks/ACSResearch.pdf.