Impact of Influential Observations on Enumeration and Variance 

Estimation in the National Crime Victimization Survey

Michael Planty

Lynn Langton 

Bureau of Justice Statistics

810 7th St., NW Washington, DC 20531

Michael.Planty@usdoj.gov

Lynn.Langton@usdoj.gov

Abstract

The National Crime Victimization Survey (NCVS) collects information from respondents about the number and 

type of victimizations they experience each year. A small percentage of respondents experience a high number of 

victimizations within the reference period, referred to as series incidents. These chronic victims are substantively 

important groups as they represent victims of violence from intimate partners and incidents that occur routinely in 

the workplace and at school. These victims, however, introduce the possibility of extreme weights in the NCVS 

data. These extreme weight outliers can be influential and significantly increase both point estimates and variance of 

estimates. One way to control for these impacts is to trim and smooth extreme weights. Trimming weights has the 

potential to increase bias, but the increase in bias can be offset by the decrease in variance. This paper describes 

various approaches to incorporating series incidents into the enumeration scheme through trimming while balancing 

concerns about point estimates and variance.

1. Introduction: Outliers and influential or extreme observations

Outliers in samples are defined as data points that deviate substantially from other observations in a variable or 

population (Chamber 1986; Dixon 1950; Eltinge 2006). Three general types of outliers include: gross measurement 

errors which are observations that are not true values; representative outliers which are true values and not entirely 

unique to the population; and non-representative outliers which are considered unique to the population (Chambers, 

1985; Eltinge, 2006; Dixon 1950). Similarly, Anscombe (1960) identified outliers arising from errors in the data and 

those arising from the inherent variability of the data. The sources of outliers include data errors (i.e., coding errors), 

respondent response errors (i.e., intentional errors or misreporting due to misunderstanding the question or response 

categories), sampling error (e.g., sampling units outside of the target population or scope), interviewing or field 

collection operations, and legitimate cases from the correct population (a true value in the tail of a distribution). 

Other variations include inliers and fringeliers which tend to be more difficult to identify through simple univariate 

distributions (e.g., Wainer 1976; Chamber 1986; Eltinge 2006). These outlier types do not appear as outliers when 

full population estimates are produced, but emerge as outliers for subdomain estimates. 

Outliers are concerning because these observations can potentially distort point estimates and inflate variance

estimates, increasing volatility and reducing the statistical power of the data. Outliers with extreme weights can 

disproportionately influence the overall estimate or subdomain estimates. This results in the potential for volatile 

shifts from year to year and instability in estimates caused by an increase in variance due to the processing and 

weighting procedure. 

However, putting aside gross errors in the collection or editing process, discarding outlier observations is not a 

straight-forward process. There is a need to balance legitimate true values and the reduction of bias with a reduction 

in variance. To address such concerns, the statistical field has focused on the identification and handling of extreme 

influential observations. As Eltinge (2006) noted, to be considered an outlier the observation must deviate from the 

norm but it also must be influential by having “a major effect on analysis of the data.” Techniques for identifying 

extreme influential observations include a reliance on technical standards related the standard deviations from the 
2 

mean, visual inspection, and more sophisticated statistical approaches to identifying influential cases. Solutions to

addressing outliers due to coding and measurement errors focus on design and field operations to improve response 

and processing accuracy (e.g., improving item and questionnaire designs; validating responses in the field; 

implementing logic checks; eliminating hand coding or type errors through automation). Approaches to handling

outliers in the processing (e.g., weighting) and analysis phases including elimination, transformation, capping, 

trimming, and imputation (e.g., Little; Barnett and Lewis, 1994). 

This paper focuses on one set of extreme observation from the National Crime Victimization Survey (NCVS) that 

impacts the counting of criminal victimization, victims who suffer high-volume, repeated victimizations or series 

incidents. Series incidents occur when a victim experiences a large number of similar crimes within a given 

reference period. Research has demonstrated the validity of these observations as true values and has found them to 

be substantively important to the level and characterization of crime (Lynch, Berbaum, and Planty, 1998; Planty and 

Strom, 2007; Dodge, 1984a; 1984b; 1987; Lauritsen et al. 2012). NCVS data collection procedures reduce the 

collection burden associated with series incidents, but these procedures can result in processing, enumeration, and 

classification issues. In addition, decisions related to how the final weighting procedure will handle series incidents 

can result in extreme weights. 

The paper provides an overview of the series incident collection procedure and the need to account for these 

incidents in national estimates of crime, and then considers various weighting options for handling series incidents 

for enumeration and statistical testing purposes. The weighting options are compared and assessed in terms of their 

impact on violent crime counts, estimate variance, and the coefficients of variation. While there are legitimate

concerns about the impact these relatively rare series incidents have on point estimates and variance, eliminating 

these accurate accounts of crime may also result in distortions in the true crime rate and the characterization of crime 

victims and incidents.

2. Victimization estimates and series incidents

The measurement and enumeration of the number and type of criminal victimizations occurring during a year is 

complicated by the challenges associated with repeat victimization. Many crimes types are discrete events with a 

defined beginning and end. For example, an offender in an armed street robbery may take only a few seconds to 

threaten, harm, and steal property from a victim. The incident is clearly defined in space and time. In addition, most 

persons are victimized only once in a given year, if at all. These single, discrete events are relatively easy to recall 

and describe. 

However, for some victims, crime may be a continuous condition (Biderman, 1980; Skogan, 1981). For example, a 

woman may suffer periodic violence within an on-going intimate partner relationship or victims of bullying at 

school may suffer continuous assaults during the school year. Victims in these situations may have difficulty 

defining and recalling the details of each victimization (Dodge, 1984), and these cognitive challenges become 

apparent when respondents are asked to recall their victimizations during victim surveys. Yet details about each 

incident are necessary for counting and classifying victimizations that occur each year. Another challenge with

these types of high-frequency repeat victimizations, also known as series incidents, is that they can have a 

significant impact on point estimates and the potential for measuring annual change. A small number of repeat 

victims can be very influential on the size and precision of the national estimates of crime. 

Most crime surveys have data collection limits or a cap on the number of incidents recorded during an interview or 

in the estimation process in an effort to balance measurement error and accuracy with burden and cost (Lauritsen et 

al. 2012). The British Crime Survey (BCS), for example, allows up to five incident reports to be completed for any 

respondent. Respondents who reported being victimized more than once are subsequently asked if they would 

identify the victimization as a series of crimes that were very similar, done under the same circumstances, and 

probably by the same people (Bolling, Grant, and Donovan, 2009). When estimating victimization rates, BCS 

publications typically limit the inclusion of series victimizations to a maximum of five. The national survey 

conducted for Mexico also uses a cap of five incidents when estimating victimization rates (Encuestas Nacionales 

Sobre Iseguridad), while the Canadian Victimization Survey limits the number of incident reports to 10 per 

respondent for each individual crime type and to a total of 20 per respondent. In addition, the Canadian survey also 

utilizes a series protocol, incorporated in the survey’s weighting procedure, which caps the number of similar 

incidents in the series at three (Nazaretian, Z., & Merolla, D. M., 2013).
3 

The U.S. NCVS utilizes an alternative protocol to cap series incidents. If the respondent was victimized six or more 

times during the reference period in a similar manner (i.e. by the same offender or in the same general sort of way)

and cannot recall the details of every incident, the interviewer uses the series victimization protocol, recording the 

number of victimizations that occurred and collecting detailed information for the last occurrence. Since specific 

information used to classify the crime type is only gathered for the most recent victimization, all incidents in the 

series are classified by the most recent incident. The series protocol addresses the recall and burden issues for the 

respondent, but at the cost of having less available information about these experiences in the data. It can also result 

in a highly skewed distribution of the number of incidents experienced by victims (figure 1, 2). 

These series incidents are not considered errors, but rather true estimates of victimization with substantive 

importance (Planty and Strom, 2007; Lauritsen et al. 2012). When examined by crime type and incident 

characteristics we find there are substantively important instances of repeat victimization. Prior research examined 

the characteristics of violent series victimizations and found series consisted of three typical categories of assaults: 

those that occurred in the home and involved intimate partners or domestic violence; those that occurred at school; 

and those associated with workplace violence (Lynch, Berbaum, and Planty, 1998; Planty and Strom, 2007; Dodge, 

1984a; 1984b; 1987). Further, series incidents are not periodic anomalous outliers occurring occasionally, but rather, 

are consistently reported each year and under similar conditions. The fact that the NCVS procedures routinely 

capture these victims is evidence of a real phenomenon and not a gross measurement error (figure 3, 4). 

2.1 NCVS weighting and capping series incidents

Despite evidence that high-frequency repeat victimizations are a real and important phenomenon, inclusion of these 

series incidents can have a disproportionately high impact on estimates of criminal victimization. Capping these 

incidents and limiting the number of separate criminal incidents that can be included in the total estimates of crime

clearly reduces the resulting survey estimates. Therefore, efforts to place an upper limit on the number of incident 

reports that can be collected in the field or to restrict the inclusion of series incidents during data processing or 

analysis have been criticized by some researchers for undercounting the extent of crime victimization (Planty and 

Strom, 2007; Farrell and Pease 2007). However, capping remains a common practice applied in most victimization 

surveys and is regarded by survey methodologists as a means to provide more consistent comparisons and trend 

measures that are less impacted by relatively rare extreme outliers. The question then becomes how to best cap the 

estimate in a manner that balances estimate accuracy and volatility. 

At various points in the past decade, the NCVS has utilized several different approaches to handling series incidents. 

One practice was to exclude series incidents completely from all estimates of crime in the NCVS annual Criminal 

Victimization bulletin. The exclusion approach was applied because of concerns that classifying all incidents in the 

series according to the most recent could result in some misclassification. In other NCVS analyses the entire series

was counted as one incident, to ensure that the single, most recent incident was correctly classified and that 

characteristics of that incident were captured in estimates. Finally, acknowledging that these prior approaches 

underestimated the annual crime numbers, the final weight has also been adjusted to count the actual number of 

incidents in in the series, up to a maximum of ten. 

3. Methodology

This paper examines several capping strategies for the NCVS series incidents: 1. Exclude entirely (count as 0); 2. 

Count as one incident; 3. Cap the estimate at the median response for a given year; or 4. Record the actual number 

of incidents as reported by the respondent. The paper then compares point estimates generated from each of these

approaches for the major violent crime counts: total violence; rape or sexual assault; robbery; aggravated assault; 

and simple assault. In order to measure of estimate volatility, it also examines the coefficient of variation (CV) or 

relative standard error (RSE) of each estimate, which is the ratio of the standard error to the mean, with larger CVs 

meaning less precision. 

3.1 Analysis

The analysis is based on NCVS data from 1993-2012. For the purpose of examining the handling of series incidents, 
4 

the data are restricted to violent crime, which includes attempted and completed rape, sexual assault, robbery, and

both aggravated and simple assault. In the NCVS, the type of crime classification is based upon detailed 

characteristics of the event as provided by the respondent. During data processing an algorithm classifies each 

incident into a type of crime, depending on the victim’s responses to a number of items on the survey questionnaire. 

The NCVS sample estimates are weighted to represent the national person (12 or older) and household populations 

in a given year. While each component of the weighting process could result in an outlying and influential 

observation, the analysis primarily focuses on the last stage, series incident adjustment. The weighting steps include:

Components of the NCVS Weights

Household-Level 

Estimates

Person-Level

Estimates

HouseholdVictimizationIncidentPersonVictimizationIncident

Base Weight × × × × × × 

Special Weighting Adjustments × × × × × × 

Household Non-interview Factor × × × × × × 

Within- Household Non-interview 

Factor

 × × × 

First-Stage Ratio × × × × × × 

Second-Stage Ratio × × × × × × 

Bounding Adjustment × × × × 

Multiple Victim Adjustment × 

Series Incident Adjustment × × 

4. Findings

Before examining the impact of series enumeration rules on counts of violent crime, the first step was to examine 

the basic descriptive statistics for series incidents and the impact of the various series weighting approaches on these 

descriptives. The 2,269 victims of high-frequency repeat violent victimizations from 1993 through 2012 experienced 

an average of 21 incidents and a median of 10 incidents in a six month period (table 1). Estimates of skewness and 

kurtosis suggest that the data are highly skewed with extreme deviations. The number of incidents in the series 

ranged from six to 750.

When looking next at the weighted number of victimizations contributed by series incidents for each of the last three

weighting options, the impact of these series incidents becomes more apparent (table 2). With series set equal to one 

the variance is much lower than with series capped at the median or with the actual number of series incidents used. 

However, the number of victimizations is also substantially lower. As seen in figure 5, the mode and median sample 

case weight is fairly consistent across the three approaches for handling series cases, while, as expected, the mean is 
5 

highest when the actual number of events is used. 

When the four weighting options are applied to generate estimates of violent crime counts, the impact of series 

incidents is even clearer (figures 6, 7, 8). Excluding series incidents or counting them as one incident clearly

underestimates the total number of victimizations, though the degree of underestimations was more pronounced in 

1993 than in 2012. This is due to the fact that the number of series incidents declined slightly during the 20 year 

period, as did the number of events that occurred within each series incident (figures 3, 4). While using the actual 

number of incidents results in the higher crime counts, the CVs are also higher, meaning more variance and greater 

volatility of estimates. This is especially apparent when examining the rape and sexual assault estimates for a single 

year (figure 7). The CV when series incidents are capped at ten, the median number of incidents, is under 25%, 

while the CV is approximately 35% when the actual number is used. However, compared to other crime types, 

rape/sexual assault estimates have relatively large CVs regardless as to how series are treated. 

Figures 9 and 10 and tables 3, 4, 4a, and 4b summarizes the percent change in violent crime estimates and CVs 

using the different series weighting approaches. Again, estimates of rape and sexual assault are the most sensitive to 

the inclusion of series incidents. When series incidents are included in counts of rape and sexual assault the counts 

increase by less than 10% to over 130% depending on the weighting approach used. The increases are less 

pronounced for crimes like robbery and aggravated assault that have larger sample sizes and are less likely to 

involve high frequency repeat victimizations. However, even for these crime types, using the capped series weight 

results in a more than 10% increase in the victimization count and using the actual number of series events results in 

about a 20% increase. 

The need to balance accuracy and increased variance is also apparent from these tables and figures. Across the board 

the CVs increase when series incidents are capped, rather than set equal to zero or one. When all series events are 

counted, the CVs are even greater and suggest that for rarer events, like rape and sexual assault, and for crimes were 

series incidents are more prevalent, the estimates became more volatile and measuring year-to-year change becomes 

more difficult. However, this problem in detecting annual change for rare events exists regardless of the series 

treatment. Other approaches, such as generating rolling annual averages, help to stabilize the volatility (Planty et al. 

2013). 

5. Discussion

The findings presented here suggest that the capping mechanism used to handle NCVS outlier cases contains the 

volatility of the estimates, results in a one-time level change, and for most estimates results in minor or reasonable 

increases to the standard errors and measured by the CVs. For example, compared to series counted as one, total 

violent crime, rape and sexual assault, robbery, aggravated assault, and simple assault 2012 estimates increased 

between 30 to 60% using the capping approach and saw relatively small increases to the CVs. As expected some 

estimates do incur substantial increases to point estimates and variance by these influential cases. However, for these 

cases, the impact of series incidents is driven from estimates already prone to small sample sizes, low statistical 

power, and high standard errors. For example, the number of rape/sexual assault increases by 59% with an increase 

of 187% in the standard error. Even without the inclusion of series cases with relatively high standard errors and 

CVs, this estimate performed poorly, though. The new series capping rule made a poor measure slightly worse. 

To summarize: 

• High volume repeat victims pose challenges to data collection and estimation

• Reporting actual respondent counts produces extreme and influential weights resulting in:

 Large increases in point estimates

 Large shifts in annual change

 Large increases in variance, reduction in statistical power

 Relatively rare crimes such as rape/sexual assault experience large increases in volatility, 

other crime types less susceptible (e.g., robbery)

 Relatively common incidents such as simple assault account for a large number of series, 

but less prone to large shifts because of the large number of total observations.

• Capping series incidents with the median estimate reduces the impact on the estimate and variance while 

balancing substantive accuracy.
6 

This research focused on one particular type of outlier in NCVS data. However, series incidents do not represent the 

only extreme and influential values in the data. Particularly as the NCVS begins to be used to measure crime at the 

subnational level, future research is needed to address other issues such as the impact of inliers for subdomains and 

particular small areas. 

Disclaimer

The views expressed here are the author’s and do not reflect those of the Bureau of Justice Statistics, U.S. 

Department of Justice. 

References

Anscombe (1960). Rejection of Outliers. Technometrics, 2, 123. 

Barnett, V. and Lewis, T.: 1994, Outliers in Statistical Data. John Wiley & Sons, 3rd edition.

Chambers, R.L. (1986). Outlier Robust Finite Population Estimation. Journal of the American Statistical 

Association 81, 1063-1069.

Sadeq Chowdhury, Meena Khare, Kirk Wolter (2007). Weight Trimming in the National Immunization Survey

Proceedings of the Section on Survey Research Methods, American Statistical Association, 

http://www.amstat.org/sections/srms/proceedings/y2007/Files/JSM2007-000077.pdf

Dodge, R.W. (1984a). A preliminary inquiry into series victimization. In R.G. Lehnen & W. Skogan (Eds.), The

National Crime Survey working papers, 2. Washington, D.C.: Bureau of Justice Statistics.

Dodge, R.W. (1984b). Series victimization – what is to be done? In R.G. Lehnen & W.Skogan (Eds.), The National

Crime Survey working papers, 2. Washington, D.C.: Bureau of Justice Statistics.

Dodge, R.W. (1987). Series crimes: report of a field test. Washington, D.C.: Bureau of Justice Statistics.

Dodge, R.W. & Lentzner, H.R. (1980). Patterns of personal series incidents in the National Crime Survey. Review of

public data use, 8, 27-41.

Elliott, M.R. and R.J.A. Little (2000). Model-Based Alternatives to Trimming Survey Weights. Journal of Official 

Statistics 16, 191-209.

Eltinge, J.L. and Cantwell, P.J. (2006): Outliers and influential observations in establishment surveys. Federal 

Economic Statistics Advisory Committee. http://www.bls.gov/bls/fesacp3060906.pdf

EUICS Consortium (2005). Methodology of the European Crime and Safety Survey. Retrieved from http://

www.europeansafetyobservatory.eu/downloads/WP_methodology.pdf.

General Social Survey Cycle 18: Victimization, 2004 (2005). User’s guide to the public use microdata file. Ottawa,

Ontario: Statistics Canada.

ICESI (2011). Cuadernos del ICESI 9: Analisis de la Septima Encuesta Nacional Sobre Inseguridad ENSI-7/2010.

Lauritsen, Janet L., Jennifer Gatewood Owens, Michael Planty, Michael R. Rand, and Jennifer L. Truman (2012). 

“Methods for Counting High-Frequency Repeat Victimizations in the National Crime Victimization Survey.” 

Washington DC: U.S. Department of Justice, Bureau of Justice Statistics, NCJ 237308. 

Little, R.J.A. and P.J. Smith (1987). Editing and Imputation for Quantitative Survey Data. Journal of the American 

Statistical Association 82, 58-68.

Little, R.J.A., Lewitzky, S., Heeringa, S., Lepkowski, J., Kessler, R.C. (1997).Assessment of Weighting 
7 

 

Methodology of the National Comorbidity Survey. American Journal of Epidemiology, 146, 439-449.

Lynch, J.P., Berbaum, M.L., & Planty, M. (1998). Investigating repeated victimization with the NCVS. Final report 

for National Institute of Justice Grant 97-IJ-CX-0027. Washington, D.C.: U.S. Department of Justice. 

Nazaretian, Z., & Merolla, D. M. (2013). Questioning Canadian Criminal Incidence Rates: A Re-analysis of the 

2004 Canadian Victimization Survey. Canadian Journal of Criminology and Criminal Justice/La Revue canadienne 

de criminologie et de justice pénale, 55(2), 239-261.

Planty, M. & Strom, K.J. (2007). Understanding the role of repeat victims in the production of annual US

victimization rates. Journal of Quantitative Criminology, 23, 179-200.

Planty, M., Langton, L., Krebs, C., Berzofsky, M., & Smiley-McDonald, H. (2013). Female victims of sexual 

violence, 1994-2010. (NCJ 240655) Washington, DC: U.S. Department of Justice, Bureau of Justice Statistics.

Potter, F.J. (1988). Survey of Procedures to Control Extreme Sampling Weights. Proceedings of the Section on 

Survey Research Methods, American Statistical Association, 453-458.

Potter, F.J. (1993). The Effect of Weight Trimming on Nonlinear Survey Estimates. Proceedings of the Section on 

Survey Research Methods, American Statistical Association, 758-763.
0

5000

10000

15000

20000

25000

30000

35000

40000

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

32

33

35

36

37

39

40

41

45

48

50+

Number of victims

Number of victimizations

Figure 1. Number of victimization per victim reported in the 6-month reference period, NCVS 

1993-2012
9 

 

0

100

200

300

400

500

600

700

800

68

101214161820222426283033363941485260667278909699

102108122130136150160180196240300360480750

Number of victims

Number of victimizations

Figure 2. Number of victimizations by series victim, NCVS 1993-2012 
10

0

50

100

150

200

250

300

350

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Number of series incidents

Year

Figure 3. Number of series incidents reported per year, NCVS, 1993-2012 
11

 

0

1

2

3

4

5

6

7

8

9

10

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Percent

Year

Figure 4. Percent of violent crimes that are series incidents, NCVS, 1993-2012 
12

Table 1. Series incident summary statistics, 

1993-2012

N 

 2,269 

Mean 21.4 

Median 10.0 

Mode 6 

Std. Deviation 38.9 

Skewness 7.4 

Kurtosis 88.1 

Minimum 6 

Maximum 750 

Percentiles 25 6 

50 10 

75 20

 

 
13

Table 2. Summary statistics for weighting by series enumeration scheme, 1993-2012

 series=1 series=capped series=actual

Mean 2,732 3,107 3,621 

Median 2,394 2,409 2,408 

Mode 2,219 1,977 1,977 

Std. Deviation 1,090 3,313 16,924 

Variance 1,187,558 10,976,526 286,406,594 

Skewness 

4 

 

10 71 

Kurtosis 

36 

 

148 8,244 

Range 37,871 153,385 2,733,224 

Minimum 

101 

 

101 101 

Maximum 37,972 153,487 2,733,326 

Percentiles 25 2,084 2,088 2,088 

50 2,394 2,409 2,408 

75 3,044 3,095 3,094 

 

 

 

 
14

 

 -

 500

 1,000

 1,500

 2,000

 2,500

 3,000

 3,500

 4,000

series=1 series=capped series=actual

weight size

Figure 5. Measures of central tendency for victimization weights by series 

enumeration scheme, 1993-2012 

Mode Median Mean