Bureau of Office of Community

Justice Statistics Oriented Policing Services

 U.S. Department of Justice

 

Conducting Community

Surveys

A Practical Guide 

for Law Enforcement

Agencies

 
 

U.S. Department of Justice

Office of Justice Programs

810 Seventh Street, N.W.

Washington, D.C. 20531

Janet Reno

Attorney General

Raymond C. Fisher

Associate Attorney General

Laurie Robinson

Assistant Attorney General

No1l Brennan

Deputy Assistant Attorney General

Office of Justice Programs

World Wide Web Homepage:

http://www.ojp.usdoj.gov

_______

Jan M. Chaiken, Ph.D.

Director

Bureau of Justice Statistics

World Wide Web Homepage:

http://www.ojp.usdoj.gov/bjs/

For information:

BJS Clearinghouse

1-800-732-3277

Mary Lou Leary

Acting Director

Office of Community Oriented

Policing Services

World Wide Web Homepage:

http://www.usdoj.gov/cops/

For grant and funding information:

Department of Justice 

Response Center

1-800-421-6770
 

U.S. Department of Justice

Conducting Community

Surveys

A Practical Guide for Law

Enforcement Agencies

by 

Deborah Weisel

October 1999, NCJ 178246

A joint project by the

Bureau of Justice Statistics

and the 

Office of Community Oriented 

Policing Services
 

Acknowledgments

This report was written by Deborah Weisel. It was prepared

under the supervision of Marshall DeBerry and Steven K.

Smith, Ph.D., of the Bureau of Justice Statistics and by Pam

Cammarata of the Office of Community Oriented Policing

Services. Meg Townsend, formerly with the COPS office,

also contributed to the project.

The project was supported by BJS under contract number

OJP-99-135-M. Tom Hester, Lea Gifford, and Marika Litras 

of BJS provided editorial review. Jayne Robinson of BJS

administered final publication.

The Crime Victimization Software may be obtained by

contacting 1-800-732-3277 or ASKCVS@ncjrs.org.

ii Conducting Community Surveys
Introduction

Why undertake a community survey 1

How was the survey developed? 2

What is the local-level Crime Victimization Survey? 3

How have surveys been used by police 3

Use of telephone surveys 4

Overcoming fears of surveys 5

Survey development

Thinking about and setting goals 7

What do you really want to know from the survey 8

Tailoring the survey to fit your needs 9

Modifying the level of analysis 9

Modifications to reflect local conditions 9

Modifications to incorporate measures of planned police initiatives 10

Modifications to survey should be consistent and neutral 10

Who ya gonna call? The critical issue of sample size 10

Population and samples 11

Issues of probability sampling 11

Generating the sample frame 13

What is the appropriate sample size? 13

How representative is the sample? 15

Survey administration

Carrying out a survey 16

Coordination and quality control 16

Selection, training, and supervision 17

Analyzing and interpreting survey results

Frequencies and crosstabulations 19

More sophisticated analyses 20

Conclusion 20

Appendixes

Appendix A. Terminology

Appendix B. References

Appendix C. Seven basic steps for conducting telephone surveys

Appendix D. Costs

iii Conducting Community Surveys iii

Contents
Why undertake a community survey?

 

The widespread adoption of community policing across the nation has increased

the interest in law enforcement agencies’ conducting community surveys. Many

police engaged in community policing want to know “how we're doing” from the

citizens’ perspective. Community surveys provide descriptive information that

goes beyond the traditional measures of police workload, arrest activity, reported

offenses, and calls for service. Besides, since community police officers are trying

to be responsive to community concerns, any complete measure of success would

have to include asking the members of the community themselves. 

 

Community surveys at a city-wide or county-wide level can be designed to provide

police with reliable feedback from citizens about perceptions of police

performance. In addition, these surveys collect information about criminal

victimization, residents’ views about crime, and their willingness to report crime to

the police. From a police management perspective, these surveys collect

information about the most effective approaches to dealing with crime. Since

these surveys provide a measure of police performance, they can be used to

analyze the way police deliver services and possibly change the allocation of

resources where needed. Some community surveys provide detailed information

about specific problems affecting parts of a city, which helps in focusing police

resources.

 

Survey research is a science and requires that accepted practices be followed.

But carrying out surveys need not be intimidating. Even terms such as field

testing, validation, sampling and statistical analyses need not be barriers to you if

you want to conduct survey research to shed light on police practices. You can

partner with a local college, university, or research firm to assist you in developing

and administering the survey as well as reporting the findings. Local colleges,

universities, and research firms can assist you with survey issues addressed

throughout this guide, including validity, reliability, purpose, probability sampling,

survey administration, and analyzing and interpreting survey results. 

 

Survey research is straightforward, although it can be time-consuming to follow

acceptable practices. To make available good community surveys and reliable

survey methods to cities, counties, and states, the Office of Community Oriented

Policing Services (COPS) and the Bureau of Justice Statistics (BJS) in the U.S.

Department of Justice have developed a software package that includes a

standardized community survey you can administer by telephone. The entire

questionnaire is included at no cost in the software package available to

government agencies and researchers.

 

This guide comes with the software and its technical how-to manual. It gives

practical, basic pointers for police in conducting community surveys. It provides 

an overview of key issues involved in conducting survey research. This guide 

provides some basic do’s and don’ts for conducting surveys that can withstand

Introduction

Conducting Community Surveys 1
close scrutiny by police leaders, academicians, community groups, politicians, and

the media and that can be compared with similar surveys by other jurisdictions.

The goal of this guide is to help you identify the issues in conducting useful

surveys. However, this guide addresses only the basic elements of survey

research. To learn more about research and statistics for police managers, see

Appendix B. Police agencies may also consult with professionals about the

technical aspects of survey research. The step-by-step procedures in this guide

will help you and whoever will conduct the survey to address the following basic

but critical issues:

 

Why are we doing a survey and what do we want to know?

What kind of resources and commitment do we have for this effort?

Who should be surveyed and how?

How many people should be surveyed?

What do we do with the responses to the survey?

 

Answering these questions will make conducting surveys a relatively painless

process to collect valuable information from the community. This guide also

provides the reader with a brief description of the critical issues affecting

community surveys. 

How was the survey developed?

 

Some background on this survey project is in order. The community survey

discussed in this guide builds on the work currently part of the National Crime

Victimization Survey, or NCVS. The NCVS has been conducted since 1972 by 

the Bureau of the Census for the Bureau of Justice Statistics. The results of the

NCVS are widely available to police and others, and are developed to provide

national estimates of victimization, illuminating how actual victimization varies from

crime reported to the police. Because much crime is not reported to police, the

NCVS findings clarify patterns of crime that are actually experienced by citizens

and give the victim’s perspective on what happened and the consequences.

Because the NCVS is based on a nationwide sample designed to produce

estimates for the entire United States, the NCVS does not provide insight into

crime problems at the city, county, or state level.

 

To assist localities with collecting this important victimization information and other

information at the city level, COPS and BJS have developed a standardized

software package. Known as the Crime Victimization Survey or CVS, the CVS

software duplicates all the questions in the NCVS. In addition, the CVS includes a

component which asks specific questions about community policing. Moreover,

both the victimization component and the community policing component of

the CVS can be modified to meet specific local needs.

 

2 A Practical Guide for Police
What is the local-level Crime Victimization Survey?

 

The CVS is a generic telephone survey questionnaire covering what police

engaged in community policing often seek to learn from the community:

 

What is the extent of residents’ exposure to crime and perceptions of crime? 

What are local perceptions of community disorder and quality of life? 

What are local priorities for addressing neighborhood conditions of disorder? 

How fearful are residents of crime? 

What self-protective steps have been taken by citizens? 

What is the extent and nature of police contact with residents?

What are citizens' perceptions of police activities?

How satisfied are citizens with police performance?

What are public attitudes toward and knowledge of community policing?

 

The community policing questions supplement the core victimization questions

taken from the NCVS. The core victimization questions from NCVS examine in

great detail the nature of victimization experiences within a household & identifying

victim characteristics, victim behaviors, offense characteristics, reporting behavior

including an explanation of why the incident may have been unreported, offender

characteristics, use of weapons, and a host of other information about the

victimization experience.

 

The reader should be cautioned that the CVS is a basic or generic community

survey instrument. Survey research is not typically “one size fits all.” Later in this

guide, we’ll discuss how to revise the basic survey to fit the specific needs of your

jurisdiction. There are important benefits to conducting a standardized survey,

however. The principal benefit is that many of the questions related to scientific

methodology have been addressed by experts. You can have greater

confidence that the results of the survey are valid. A second reason is that it is

possible to make comparisons with other jurisdictions which have

conducted similar surveys. For many situations such as addressing crime and

disorder problems, such comparisons may not be helpful. However, to interpret

the measures of citizen satisfaction with your department resulting from this

survey, it may be useful to compare your agency to others.

 

How have surveys been used by police?

 

Police have long been interested in citizens’ views of police. To evaluate police

programs, academic researchers have used surveys of citizens & in person, by

telephone or by mail, with different response rates and costs. But police agencies 

have only intermittently surveyed citizens, with some increase in these surveys

beginning in the 1980s. During the 1980s police began to conduct their own &

albeit often informal & surveys. Many of these surveys were door-to-door surveys

designed to document the extent and nature of fear of crime. Among others, fear

Conducting Community Surveys 3
reduction programs by police in Houston, Newark and Baltimore County made use

of community surveys.

 

During the late 1980s, many police came to view the public as customers or

consumers of police service. Police often used citizen or customer satisfaction

surveys to gauge the reaction of citizens to contact with police. These surveys

were often mailed surveys, sent to citizens who had complained, received traffic

citations, or had other formal contact with police. Telephone surveys were also

used for this purpose. 

 

As part of the trend towards community policing and problem-solving in the midand late 1990s, community surveys have increasingly been used to collect

local-level information from citizens about specific problems in their communities.

For the most part, these surveys have been administered to sub-units within cities

& public housing developments, neighborhoods or other geographic entities.

Many of these surveys have been administered face-to-face at the homes of

residents; however, door-to-door surveys are very costly to carry out.

 

While all of these approaches to collecting information from citizens have been

used, surveys have not always provided reliable information to police. Because of

the way people were selected for surveys, police surveys have typically not met

scientific standards of reliability, and some have been subject to criticism. 

 

To provide feedback about community policing, there has been increasing interest

among police, city councils, and mayors’ offices in obtaining reliable survey

information that meets scientific standards by using probability samples. In fact,

some cities now routinely conduct citizen surveys to provide feedback on police

performance.

 ̆ Scottsdale, Arizona, conducts (through a contractor) an annual citizen

satisfaction survey to assess service delivery. (See the questionnaire at

www://ci.scottsdale.az.us). The telephone survey queries approximately 400

citizens through a random digit dialing approach. The sampling procedure is so

standardized and the survey so routinized that information is used to guide the

city’s budgetary process.

 ̆ Similarly, police in Reno, Nevada, completed their 17th semi-annual

community survey in early 1999. The Reno department primarily uses

volunteers, such as university students and seniors, to carry out the survey of

430 citizens. (For more information contact the deputy chief for planning,

training and research via http://www.reno.gov)

 

Use of telephone surveys

 

As a method of research, telephone surveys have many benefits. The telephone

interview is typically non-threatening to the respondent and allows the respondent

to feel the degree of confidentiality or anonymity assured by the interviewer.

Telephone surveys also preclude the need for interviewers to conduct interviews 

4 A Practical Guide for Police
in high crime areas where the risks to personal safety may be high. Typically,

telephone surveys are inexpensive relative to other types of survey research, and

response rates to telephone surveys are generally higher. 

Despite the strengths of telephone surveys, there are some limitations which

should not be overlooked. Although the vast majority of Americans have

telephones in their homes, some do not, and in some neighborhoods the

proportion of households with phones is low. Telephone surveys are inherently

biased against lower-income citizens, since a larger proportion of low-income

families have no telephone. Telephone surveys may also underrepresent persons

with language barriers, in some ethnic groups or in age groups like the very young

or the very old. Some groups such as the homeless, young inner-city males, or

college students may be severely underrepresented in phone surveys. Since

these persons may have different views of police service or different exposure to

crime, this inherent bias in telephone survey research should be recognized

throughout the survey process. There are procedures to weight demographic

groups which are not fully represented in telephone surveys. These procedures

are discussed later in this guide.

 

Overcoming fears of surveys

 

Conducting surveys can be intimidating and for some good reasons. The results 

of community surveys come under a great deal of scrutiny by media, politicians,

community groups, academicians, and police leaders. If police or sheriff’s

departments develop their own surveys, they have to be concerned about their

perceived bias as survey administrators. Surveys can be viewed as self-serving

devices, designed to make the police look good, especially if there are concerns

about police brutality, police attention to minority areas or police treatment of

minorities, and other sensitive issues. 

 

To address perceived bias, police must focus attention on every aspect of survey

research, including question wording, field testing of questionnaires to identify

problems, and other processes of survey development. In addition, police must

identify an unbiased sampling frame and develop procedures for contacting

prospective respondents. This is a big task & one with which a local university or

college can assist.

 

Using the CVS and a standardized sampling procedure will relieve many of the

external and internal pressures on police in conducting surveys. By following

sound sampling procedures, city-level findings can be comparable to those of

other jurisdictions. Replicating the survey on a periodic basis will provide police

agencies with regular and reliable information about citizens' views of crime 

and public safety.

Conducting Community Surveys 5
The CVS eliminates the need for many of the procedures that occupy survey

researchers. The survey instrument has already been rigorously pre-tested to

ensure that questions are neutral and understandable, and responses have been

scaled and validated. In addition, the software includes a method for generating a

simple random sample of telephone numbers through a random digit dial approach

by using the telephone prefixes for the area.

 

Telephone surveys are a promising method for police to collect quality information

from citizens on a routine basis and at relatively low cost. Conducting survey

research that meets standards of reliability is not difficult but requires a significant

amount of effort. Importantly, the commitment and resources for this task

should be carefully evaluated by police before undertaking survey research.

Only if sufficient resources are available to police to conduct reliable scientific

surveys can the benefits of telephone surveys be fully realized.

 

6 A Practical Guide for Police
Carrying out the community survey involves installation and use of the CVS

software provided by COPS/BJS. While this task should be carried out by

someone knowledgeable about computers, it is a straightforward procedure 

that involves following clearly specified steps. The technical manual

accompanying the software includes detailed installation and use instructions. 

Thinking about and setting goals

 

The first and most important step of survey research, and the most often

overlooked step, is determining the goals of the survey. Surveys for police can be

used for different purposes and it is important to thoroughly discuss and articulate

these goals prior to conducting the research. Avoiding or rushing this step will

invariably result in problems. 

 

Articulating survey goals invariably saves time, money and headaches. For

example, if your jurisdiction is not interested in collecting information about

victimization for different types of crime, much effort would be expended for

collecting data with little value to your agency. Similarly, if your agency is not

engaged in community policing & and has no interest in implementing community

policing & you won’t want to waste time asking respondents about community

policing initiatives. If your agency is concerned about its relationship among

different groups of Asian heritage & perhaps Lao, Cambodian and Vietnamese &

the survey must ask about country of origin. Thinking about these needs after the

survey is conducted is too late. 

So, thinking through and articulating goals help police develop a survey instrument

that is most appropriate for the jurisdiction. Articulating goals also facilitates

analysis. For example, if you’re interested in the rate of crime in the Northeast

area of your city, the sampling plan must collect enough responses from that

neighborhood to yield valid information. If you’re particularly interested in the

opinions of a small ethnic group, or a particular age group, this affects your sample

size and sampling plan. It’s best to know these needs at the beginning of the

survey, not after the survey is completed.

 

During the discussion of goals, police should also plan whether the survey is to be

repeated periodically. While there are many benefits to conducting periodic

surveys of citizens, this replication may be beyond the resources of the jurisdiction.

It may be more feasible to conduct the community survey every other year or even

every three years. While the longer time frame may wash out some of the

information about police services, such a survey will still provide valuable

information which is a snapshot of a point in time. If possible, decisions about

replication should be made before conducting the first survey as plans to replicate

the survey can have implications for what questions are asked. 

 

While your jurisdiction may not be exceptionally interested in victimization rates for

a one-time survey, declines in victimization rates & as reflected in a periodic

survey & may provide useful information for evaluating crime control and public

Survey development

Conducting Community Surveys 7
safety initiatives. An example of this utility is often heard in discussions of

community policing. Many police leaders believe that reported crime will first rise

when community policing initiatives are implemented, as residents become more

willing to report offenses. Victimization surveys may provide evidence for this

hypothesis, revealing that as actual victimization declines reported crime rises.

Without victimization information, police will be hard pressed to document this

explanation of community policing impact and will be able only to provide

anecdotal support for the hypothesis that crime is actually declining.

 

So, thinking through the survey goals is a necessity for conducting good survey

research. This step ensures that the survey accomplishes the following:

 

Makes most efficient use of resources by selection of appropriate sample size 

Provides the most meaningful information for local needs 

Assists with development of an analysis plan 

Plans for future information needs.

What do you really want to know from the survey?

 

In practical terms, the answer to this question may be affected by knowing what

kinds of things you can learn from a community survey. Among the possible

objectives of a police department using the standardized CVS including the

community policing section are the following: 

 

 ̆ Victimization & differences between reported offenses and actual victimization

 by individual characteristics such as race, gender, age and income

 

 ̆ Comparison of local victimization rates with those of other jurisdictions

 

 ̆ Measures of citizen willingness to report crime

 ̆ Indicators of citizen fearfulness 

 

 ̆ Analysis of crime and disorder problems

 ̆ Measures of police performance: survey measures citizen knowledge and

attitudes, satisfaction reflects how well the police are doing their job from the

citizens' perspective 

 ̆ Measures of public information efforts

 

 ̆ Comparison of a jurisdiction's community policing efforts & and

impact on victimization and fear & with those of other jurisdictions

8 A Practical Guide for Police
Tailoring the survey to fit your needs 

 

The goals of the survey will affect the kinds of questions which the police agency

will want included in the survey. The basic survey questionnaire & either the CVS

portion or the community policing section & can be modified to fit different police

objectives. Some questions may be deleted or altered, while additional questions

may be added. The technical guide that accompanies the CVS software includes

directions for carrying out these modifications.

 

Modifying the level of analysis

 

One modification which may be desired by local jurisdictions is to alter the level 

of analysis. The CVS survey is designed to collect information about a city. The

survey currently does not contain area-level information smaller than a city. While

city-level information is certainly easiest to collect, there is no reason why the level

of analysis cannot be changed. However, if police wish to collect and analyze

information about smaller areas within the city, such as neighborhoods or other

geographic areas, typically a larger sample must be surveyed. Sample size is

discussed inlater in this guide. 

 

Since many police agencies are interested in differences in crime and victimization

in specific areas within their city, agencies may wish to add a question identifying

the respondent’s area within the city. Addresses are requested in the survey or

respondents may also be asked to identify their zip code, neighborhood by name,

region or quadrant of the city (linking the survey to a GIS may facilitate data

collection by neighborhood). 

 

Police agencies may want to collect information only from specific neighborhoods

rather than undertaking a city-wide survey. If information about geographic areas

for telephone prefixes is available, a sampling frame may be constructed making

use of this knowledge. Sample size will vary accordingly. Alternatively,

interviewers may screen calls to identify only respondents who live in certain

portions of the city. This task would be time consuming.

 

Modifications to reflect local conditions 

 

Other modifications to the survey will reflect differences in jurisdictions across the

nation. On occasion, these differences may consist of wording or particular

localized conditions. For example, the survey asks respondents about steps they

have taken to prevent crime in the home. In warm areas of the country where

windows and doors may be left open, simply keeping doors and windows shut may

be a deterrent to crime. While the survey asks if the respondent has taken other

precautions, the current survey questions and responses should be carefully

evaluated to ensure they reflect local conditions and objectives. The local crime

and public safety scenario in cities varies based on a variety of conditions. For

example, crime varies by housing stock since high rises feature different problems

than single-family residences; crime varies by the presence of alleys or other

Conducting Community Surveys 9
street configurations and conditions; by topography, such as canyons; by

development, such as proximity to retail areas, and by many other features.

 

Modifications to incorporate measures of planned police initiatives

 

An important element for modifying the questionnaire is to incorporate measures

of planned police initiatives. Say, for example, the police plan to mount an

extensive public information effort to encourage residents to keep windows and

doors shut. If this type of effort is planned, a baseline question can be included in

the survey to determine what proportion of residents currently keep windows and

doors shut. This baseline measure will then provide a before-and-after measure of

the impact of the police public safety measure. This research design, known as a

pre-test post-test, provides excellent information about the impact of prevention or

intervention efforts. Crackdowns on gangs, street-level drug dealing, truancy,

public drinking and a host of other community-policing initiatives can be well

documented by using this design.

 

Modifications to survey should be consistent and neutral 

 

Modifying the questionnaire requires care and attention to question construction

and response options. In general, questions should be neutral and consistent with

other questions in the survey questionnaire. Response options should also be

consistent with the responses in the questionnaire. For example, if concern about

gangs in your jurisdiction is a major issue, the survey may be modified to ask the

respondent: 

 

"How fearful are you about gangs in your city (and/or your neighborhood)?" 

 

Since response options to similar questions in the survey are worded as

(1) Very fearful

(2) Somewhat fearful

(3) Not very fearful

(4) Not at all fearful

this same sequence of responses should be used for the new question.

 

Who ya gonna call? The critical issue of sample size

 

Once the survey has been modified to be consistent with the goals and objectives

in a particular jurisdiction, attention must turn to the selection of respondents. By

far, the most common question of police conducting community surveys is "How

many people must I survey?" The answer to this question relates to the goals and

objectives of your survey. In this section, we will provide some general guidelines

for decision making. 

 

We mentioned previously that police surveys are often criticized because they are

not scientific or probability samples, thus their findings are not representative of

anything in particular. Investing the time to develop a defensible sampling strategy

10 A Practical Guide for Police
and a probability sample is an important procedure. So, a brief and very basic

discussion about sampling procedures is necessary.

 

Population and samples 

 

When you sample people, you select a portion or subset of everyone you are

interested in, then make generalizations from the subset to all people of this type.

The larger group is called a population while a sample is the subset or a portion 

of the population. If you were interested in the experiences of women who have

been victims of theft, the population is all women. Since it is not possible to

interview all women, you will want to interview a subset of women. The subset is

your sample, and it is made up of cases. In the example so far, 

the cases are those women selected for interviewing.

 

If possible, you will want to know something about the population. For example,

what proportion of the population have been victims of theft in the prior six

months? To answer this question with a sample, you need to be sure that the

sample is representative of the population. Characteristics of the population are

called parameters. In this example, the proportion of women in the population who

have been sexually assaulted in the prior six months is a parameter. Though this

number exists, it is unknown. You will need to estimate it from the sample. The

estimate of the population parameter from the sample is called a statistic. If the

sample is unrepresentative of the whole population, then the statistic you calculate

is unlikely to be an accurate estimate of the population parameter you care about.

For example, if you interview women at a women's crisis clinic, you may

overestimate the proportion of women who have been victims of theft because it is

quite likely that this sample or subgroup has a higher than average theft rate than

women in the population.

 

How do you get a representative sample from your population? All sampling

begins with the translation of a defined population into a sampling frame. A

sampling frame is either a list of possible cases or a procedure for finding eligible

cases. It is from this sampling frame that you will select your cases. Ideally,

everyone in the population should show up in the sampling frame. Say you were

interested in the experience of college undergraduate women who have been

victims of theft. There is one university in your jurisdiction, but there are too

many female undergraduates to interview. The university might be able to provide

you with a list of undergraduates for you to use as a sampling frame.

 

Issues of probability sampling

 

Probability sampling includes a variety of techniques that produce cost-effective

data sets that are representative of the population from which they are drawn and

yield information that quantitatively describes our confidence in how well the

sample estimates describe population characteristics. Non-probability samples

can be cheap or expensive, but we often have no basis for claiming they are

Conducting Community Surveys 11
representative of the population, and we are not able to measure how well our

sample results reflect the whole population. 

 

A probability sample is one that is selected in such a way that each member 

of the population's sampling frame has a known probability of being selected and

that probability is greater than zero. Note that this does not necessarily mean an

equal probability of being selected, though some probability samples have that

characteristic. In some circumstances it is desirable to select some members of

population with certainty. If you are using something other than equal probability

of selection, weights should be used to take into account the differing sampling

probabilities. 

 

A non-probability sample is a sample in which you either do not know the chances

that each member of the population has of being selected, or there are some

members of the population that will not be selected. Interviewing the women in

your office about their theft victimization experiences might help you think about

the problem of women as victims of theft, but this group is unlikely to be

representative of the larger population in your city. 

 

When using a probability sample, two things heavily influence the confidence in

your estimates. The first is sample size. This is the number of cases in your

sample. The bigger your sample, the greater the confidence that your statistic is

an accurate measure of the population parameter. You have some control over

your sample size, at least within the bounds of your budget or the budget of the

agency funding your research. 

The second factor which influences confidence in estimates is the prevalence of

the characteristics of interest in the population. The greater the prevalence, the

smaller the sample you will need to get the same level of confidence. The more

rare the characteristic is in the population, the larger the sample required to get the

same level of confidence. While you have no control over the prevalence in your

jurisdiction, your knowledge about how common the characteristic is in your

population can be used to design your sampling strategy. 

 

Simple random sampling is one of four common probability sampling designs.

Each has its advantages and disadvantages. Simple random sampling is the type

of sampling design we usually think about, and the sampling strategy which is

recommended in software package included with this guide. 

 

Once the sample size has been determined and the sampling frame has been

developed by inputting the jurisdiction's telephone prefixes to generate a list of

telephone numbers, each item on the list is given a unique number. Using the

software's capacity to produce random numbers, cases (telephone numbers) are

selected for the sample. Each case (telephone number) in the sampling frame 

(list of telephone numbers) has exactly the same probability of selection and that

probability is equal to the sample size divided by the population size (number 

of entries in the sampling frame). It is important to include all prefixes that 

12 A Practical Guide for Police
significantly cover the area of interest. Leaving out some will lessen the

representativeness of the sample. 

Simple random sampling is most useful when you are not investigating rare events

or small population segments. For example, if you think that only about 2 percent

of the female undergraduates at a certain college have been the victim of a theft in

the prior six months, then simple random sampling is a very costly approach. To

be sure that we have 30 women who have been victims of theft in our sample, we

would have to interview 1,500 female students. In contrast, if 40 percent of female

undergraduates are likely to have been theft victims, then simple random sampling

may be quite efficient. This, of course, is the difficulty for a city-wide sample for

victimization; despite prominence in the media, violent crime events are relatively

rare in a jurisdiction. Very few persons are victimized by violent crime.

 

Generating the sampling frame 

 

Now that we've discussed the issues underlying determining sample size, we’re

back to the question & How many are enough? Since violent crime rates & that

is, rates per 1,000 population & are fairly well established and consistent across

different places, we know that violent crime is a relatively rare event.

 

Using the CVS software, the sampling frame for the CVS and community policing

survey consists of randomly generated telephone numbers which, by and large,

represent separate households. (Some may represent multiple phones within a

single household while others serve multiple households, such as a rooming

house.) This method of telephone number generation is known as random-digit

dialing or RDD. In developing a sampling frame, RDD avoids the limitations of

using phone books, since many telephone numbers are unlisted. 

 

In generating the sampling frame, enough numbers must be generated to account

for refusals & that is, the prospective respondents who decline to be interviewed &

and ineligibles such as non-residential numbers, non-working numbers, facsimile,

computer lines, and cell phones and numbers which duplicate other lines within a

household. These ineligible numbers may constitute a very large portion of the

sampling frame, particularly in moderate to high-income neighborhoods. There is

no reliable information about ineligibles generated through RDD. An estimate 

of about 60 percent may be appropriate for purposes of planning. Actual

experience will probably vary a great deal by location and depend on the effort

applied to obtaining participation from the survey respondents. 

 

What is the appropriate sample size? 

 

Deciding on an appropriate sample size requires effort and planning. There are no

standardized sample sizes for surveys. Recommended sample sizes are even 

independent of city size. Since violent crime is a relatively rare event and is

Conducting Community Surveys 13
somewhat equally as rare in large cities as in small cities, recommended sample

sizes will not vary much based upon the total population of the city. In general, a

final sample size of at least 200-250 will be the absolute minimum necessary for

making inferences to the larger population. In the 12 large cities in which the CVS

questions were tested, final samples consisted of approximately 800 households

(in each city) to produce a sample sufficient for making inferences about

victimization experience. These numbers were necessary to generate a sufficient

number of victims of violent crime in the sample since violent crime victimization is

statistically a rare event. In general, an appropriate sample size will fall

somewhere between these points, depending of course on the objectives of the

survey. If the police agency is not interested in violent crime, and desires only to

collect information about property crime victims, a smaller sample may be used

since property crime is statistically more common than violent crime. 

One way to think of selecting sample size is to develop a "best guess" of the level

of victimization experience being sought in the sample. Using Table 1, say that

you estimate that half of 1 percent of persons in your

community experience the type of victimization you

wish to examine. If your population is 500,000, this

best guess of victimization suggests that about 2,500

persons are victims. To find 30 of the 2,500

victimized persons, you will need to call or interview 

at least 6,000 persons & your sample size is 6,000. 

A minimum number of 30 cases is an absolute

requirement for large sample analysis with statistical

procedures normally used. 

If you are interested in looking at subgroups such as

race or gender by type of victimization, then doubling

the sample size is appropriate. This requires that

your minimum number of cases increases

proportionally as indicated in the top row of Table 1. 

14 A Practical Guide for Police

25% 120 200 400 600

20% 150 250 500 750

15% 200 333 666 1,000

10% 300 500 1,000 1,500

5% 600 1,000 2,000 3,000

1% 3,000 5,000 10,000 15,000

0.5% 6,000 10,000 100,000 300,000

n=30 n=50 n=100 n=150

Best guess of Minimum sample size to find n cases with victimization

percent of

people

victimized

Table 1. Best guess for sample size

Reno, Nevada

Reno police conduct

community surveys

twice each year, and a

sample size of 430

provides reliable

information. In general,

the larger the sample

size the greater the

accuracy in the survey

estimates, assuming the

sample is increased in a

random manner.
If the subgroup of interest is a very small proportion of the overall population, then

tripling or quadrupling the sample size may be in order. In these situations, simple

random sampling is prohibitively expensive since you need more and more

interviews & and a more efficient method of probability sampling should be used.

Professional assistance should be sought under these conditions.

 

If the victimization experience you seek to examine is more common than the

previous example, you may call fewer people. For example, if the victimization

experience is estimated at 15%, looking at Table 1, you will see that you only need

to call 200 persons to find 30 persons with that type of experience. 

 

How representative is the sample?

As telephone numbers are randomly generated for the survey, no advance

information is available about the population. While the generation of random

numbers (RDD) will provide a defensible sampling strategy, the randomness alone

will be insufficient to insure that the final sample is actually representative of the

jurisdiction's population. Nonrepresentativeness may occur either in the original

sampling frame if, for example, low-income persons or minorities are less likely to

have phones, or in the non-responses or refusals if victimized persons or some

other group are less likely to participate in the survey. The resulting level of error in

the survey estimates will depend on how many persons are missed and how

different they are from those that you are able to interview. Missing a large portion

of a distinctive subgroup in the population can greatly reduce the usefulness of the

survey data. 

 

For most police agencies, some demographic characteristics of citizens are 

of critical importance in collecting information. The exposure of citizens to police

and their opinions of police, as well as victimization experience, vary based on

demographic and neighborhood characteristics. Thus information about these

characteristics must be collected during any survey and must be monitored as the

survey is being conducted to make sure that enough respondents with those

characteristics are being surveyed.

 

For most police agencies, these relevant characteristics may include gender, age,

race, and income level. Additional variables may be critical in a particular

jurisdiction. Once the survey is administered and demographic characteristics of

the respondents are developed, these findings can be compared with census data

for your jurisdiction to ensure that respondents are representative. For groups

which may be underrepresented in the final sample, the responses may be

weighted to increase their relative proportion for the entire sample.

Conducting Community Surveys 15
Carrying out a survey

 

Conducting a large survey is a substantial task. This section of the guide

discusses the highlights of survey administration. You may want to refer to the

appendix for additional resources about this step of the survey. Resources must

be identified and capacity for conducting the survey must be assessed. Cities 

or police agencies often contract with local colleges, universities or research

organizations to conduct surveys. These organizations have the resources in

terms of telephones, computers and trained personnel to carry out the survey. 

While this approach may be expensive, this may be the most efficient way to

conduct the research. In addition, many police and city leaders feel that there 

is increased legitimacy attached to research findings conducted external to the

police department. 

 

The perceived legitimacy associated with survey research is not a minor issue. 

Conducting their own survey of the community may open the city or police

department to criticism or make results suspect. However, if results are designed

primarily for use within the police agency for example, feedback for allocating

police resources or providing annual evaluation information & an in-house survey

may be appropriate. 

 

Using the CVS software companion to this guide relieves some of the criticism

applied to police surveys. The CVS is a standardized survey which builds on the

National Crime Victimization Survey, conducted for over 25 years by the U.S.

Census Bureau for the Bureau of Justice Statistics. The community policing

questions in the CVS were rigorously field tested by the U.S. Department of

Justice in 12 jurisdictions. The validation processes carried out have standardized

the survey instrument, so that the same survey will likely be used by other police

agencies in the nation. The survey instrument has a great deal of validity on its

own merits.

 

Effort and care must be expended to set up and carry out the survey. Resources

such as telephones and computers must be allocated and located. Personnel

must be selected and thoroughly trained. Methods of supervision and quality

control must be developed to insure the accuracy of the surveys, and issues such

as time frames must be established.

 

Coordination and quality control

 

Administration of a telephone survey is not complicated but requires careful

planning and close adherence to a series of quality control procedures. Planning a

survey also involves estimating the time needed to carry out the survey in a

relatively compressed period of time. Surveys which are allowed to drag out for

months and months run the risk of a major crime event occurring which will alter

respondents' perceptions of crime or police in the midst of the survey. To avoid

this situation, every effort should be made to conduct the survey as quickly as

possible. 

Survey administration

16 A Practical Guide for Police
Estimations can be made of the amount of time necessary to carry out a survey

based on the desired sample size. The national CVS requires approximately 10

minutes to administer. This figure, of course, varies from one individual to the next

while the Community Policing section will add approximately 5-10 minutes to the

interview time.

 

The number of surveys to be conducted will be determined through the sampling

procedure (see above), guiding survey administrators in estimating the amount of

staff necessary to conduct the survey. Estimates of the time to conduct a

community survey should follow a formula which includes:

 

The number of respondents/cases desired & e.g., 600

Estimation of refusal rates (30 percent of 12-city CVS)

Estimates of eligible respondents (that is, ineligible numbers are screened out

& perhaps around 60 percent)

Total number of telephone numbers to be generated and called 

 

In this example seeking 600 respondents, 2400 numbers will be generated; 60

percent or 1,440 will be ineligible while 960 will be eligible; and 30 percent or 288

will refuse while 70 percent or 672 will agree to the interview. As general rule,

based on the sample size for the 12-city field test of the CVS questions, telephone

numbers should be generated for approximately four times the number of surveys

desired. This ratio incorporates both ineligibles and refusals. 

 

Selection, training, and supervision

 

The selection and training of interview staff and their supervision are critical to

administering the survey. All interviewers should be well-supervised. Survey

administration should be monitored on a random basis and quality control checks,

such as making call-backs to respondents should be carried out to ensure the

accuracy of interviews.

 

Training of interviewers should be carried out to ensure that all surveys are

conducted as similarly as possible. In general, interviewers should be thoroughly

trained to be neutral, precise, and thorough in administering each survey. 

 

Training should also emphasize the confidential nature of information being

collected. Interviewers should be aware that respondents may be suspicious of

providing personal or sensitive information over the telephone particularly about

their experience with victimization. Evidence with telephone surveys suggests that

these surveys are typically less threatening to respondents than face-to-face

surveys. Nonetheless, interviewers should fully brief prospective respondents

about the purposes of the survey and the identity of the survey organization. Data

management practices should be established to maintain the anonymity of the

respondent and the confidentiality of all information collected through the interview.

Conducting Community Surveys 17
The respondent should be fully assured of anonymity associated with his or her

individual experience and opinions. Good interviewers who are able to address

respondents' concerns about anonymity may lower refusal rates. 

 

Consistent with the practices of the U.S. Census Bureau in carrying out surveys,

telephone interviews should adhere to the following practices:

 

 ̆ Be friendly and businesslike

 

 ̆ Assure the respondent of confidentiality

 

 ̆ Refrain from expressing (via words or tone) a personal opinion about answers

received & any comments should be neutral

 

 ̆ Pace the interview and always avoid rushing the interview & surveys can be

rescheduled for completion at a later time, if necessary

 

 ̆ Ask all questions in a deliberate and objective way & asking no additional

questions

 

 ̆ Ask all questions in order

 

 ̆ Ask all questions exactly as worded & even if the respondent has previously

provided the information

 

 ̆ Avoid leading respondents by changing words

 

 ̆ Read each question clearly

 

 ̆ As instructed, follow up questions with "Any other way?" or other prompts

to elicit additional responses 

 ̆ Probe, as necessary & stating "yes, I see" or repeating questions as necessary

may encourage a more complete response to questions. Probes are especially

useful if the respondent says "I don't know"

 ̆ Use neutral probes such as "I don't understand" or "Can you explain that?"

 ̆ Close the interview by thanking the respondent(s) for their time.

 

Compliance with these basic requirements for conducting telephone surveys

should be assessed through routine monitoring of survey administration. As a

technique of quality control, all surveyors should be monitored periodically to

ensure that the survey is being conducted as designed. Failure to comply with

these basic administration requirements may invalidate the survey findings.

18 A Practical Guide for Police
Analyzing and interpreting survey results is a step of survey research that makes

many lay persons panic. To help ease the pain of analysis, the CVS has

developed some canned analysis packages to report survey findings.

 

Frequencies and crosstabulations

 

Frequencies report how many persons (and what percent of respondents) 

responded a certain way to each question in the survey. This is the most basic

output from surveys and is a step taken by every survey analyst. For some folks,

this may even be the last piece of analysis done on the survey findings! The

following table is a frequency table reporting citizen attitudes about drug problems.

 

Concerns about drugs

 

 Agree Neutral Disagree Total 

Drug sales and use in my 61.02% 23.73% 15.25% 100%

neighborhood are problems (288) (112) (72) (472)

 

The numbers in the table show the rare situation in which all respondents

answered the question. Typically, in a survey with many respondents, each

question will have at least a few respondents who do not answer. The number of

non-responses is often shown separately, particularly if it is large, and the percents

are of the people who did answer the question. While this type of table is

interesting, frequencies are most useful for pointing to directions for further

analysis. Wouldn't it be useful to know what kinds of citizens are most concerned

about drug problems?

 

Crosstabulations compare categories of responses on a particular characteristic.

Demographic characteristics frequently influence the way results are interpreted.

 

Concerns about drugs, by age of respondent

 

Drug sales and use in my 

neighborhood are problems Agree Neutral Disagree Total 

Age 18-29 21% 18% 61% 100%

(30) (26) (87) (143) 

Age 30-59 43% 21% 36% 100%

(74) (36) (62) (172) 

Age 60 or older 80% 11% 9% 100%

(126) (17) (14) (157)

Totals 100% 100% 100% 100%

 (288) (112) (72) (472)

 

Analyzing and interpreting survey results

Conducting Community Surveys 19
This type of table is available as a standardized report with the CVS software.

Tables can be generated by the demographic variables such as age, gender and

race. The technical manual provides guidance on how to use the software to

carry out these tasks.

More sophisticated analyses 

 

Frequencies and crosstabulations tell most police agencies much of what they

want to know from community surveys. We have not discussed significance

testing, the criteria for determining the likelihood that an observed relationship is

due to chance. You may want to evaluate significance of findings to establish

support for an observed relationship. The mathematical formula used to estimate

significance depends on a host of factors, including the size of the sample or

sub-sample. 

 

In addition to frequencies and crosstabulations, more sophisticated analysis may

be used by jurisdictions with the interest and ability to do so. For example, surveys

which opt to collect location information from residents (such as zipcode or

neighborhood) may be able to use GIS systems to map response categories.

Analyses should also be framed so that replication of findings from year to year will

generate results which provide meaningful comparisons over time. 

 

Conclusion

 

Surveys of the public can provide valuable information to police managers and law

enforcement policy makers. There are some types of information, such as citizen

attitudes and the volume of unreported crime, that cannot be gathered in any other

way. 

This guide has introduced the basic concepts and procedures needed to

undertake a community survey. The survey software provided by COPS and BJS

incorporates some of the best practices available, thus making the process

simpler. Nevertheless, a competent and valid survey, particularly of rare events

such as criminal victimization, is a difficult and resource-intensive undertaking. If

the sample is too small, there can be little confidence in the results even if it is a

random sample. If an important segment of the population refuses to answer the

questions or cannot be reached by telephone then making population estimates

from your sample is highly problematic. 

The software and this guide focus on simple random sampling which is

appropriate for telephone samples in which nothing is known about the

respondents. For many circumstances, simple random sampling is more

expensive than other more complex sampling strategies. We strongly recommend

that you discuss the substantive problem you want to investigate with a trained

survey researcher before starting your project.

 

20 A Practical Guide for Police
CVS & Crime Victimization Survey & This is the two-part survey provided by

BJS/COPS. It includes the community policing section and replicates the

questions contained in the National Crime Victimization Survey conducted by the

Bureau of the Census.

 

NCVS & The National Crime Victimization Survey is the victimization survey

conducted by the Bureau of the Census.

 

Parameter & characteristics of the population of interest, e.g., one gender or one

race with victimization experience.

 

Population & the group in which you are interested.

 

RDD & Random Digit Dialing & This is a method for generating a random list 

of telephone numbers for carrying out the survey. The Windows-based CVS

software uses this method to generate a list of numbers for calls. 

 

Sample &- a subset or portion of the group you’re interested in.

 

Sampling frame & a list of all possible cases from which you select your sample.

 

Significance testing & the criteria for determining the likelihood that an

observed relationship is due to chance.

 

Statistic & the estimate of a population parameter from your sample.

 

Probability sample & a sample in which everyone selected had a known chance

of being selected; often probability samples reflect an equal probability of

selection. 

Appendix A. Terminology

Conducting Community Surveys 21
For further reading on survey research, especially telephone surveys, see &

 

Babbie, Earl (1999). The Basics of Social Research, 8th edition. Belmont, CA: 

Wadsworth Publishing.

 

Czaja, Ronald and Johnny Blair (1996). Designing Surveys: A Guide to Decisions

and Procedures. Thousand Oaks, CA: Pine Forge Press.

 

Eck, John E. and Nancy G. LaVigne (1994). Using Research: A Primer for Law

Enforcement Managers. Washington, DC: Police Executive Research Forum.

 

Lavrakas, Paul (1993). Telephone Survey Methods: Sampling, Selection, and

Supervision. Newbury Park, CA: Sage Publications.

 

Lipsey, Mark W. (1990). Design Sensitivity: Statistical Power for Experimental

Research. Newbury Park, CA: Sage Publications.

 

Maisel, Richard and Caroline Hodges Persell (1995). How Sampling Works.

Thousand Oaks, CA: Pine Forge Press.

 

Saris, Willem E. (1991). Computer-Assisted Interviewing. Newbury Park, CA: 

Sage Publications.

 

Schutt, Russell K. (1999). Investigating the Social World: The Process and

Practice of Research. Thousand Oaks, CA: Pine Forge Press.

 

Webb, Kenneth and Harry P. Hatry (1973). Obtaining Citizen Feedback: The

Application of Citizen Surveys to Local Governments. Washington, DC: The

Urban Institute.

 

Appendix B. References

22 A Practical Guide for Police
1. Think about and set goals. Take time to be exceptionally clear about what 

you really want to know from a survey. Modify the survey questionnaire based 

on your specific objectives.

 

2. Select a sampling frame by generating telephone numbers with RDD software.

 

3. Determine the sample size and the sampling method based on your goals.* 

Select the sample & that is, select who will be called.

 

4. Set up the survey administration. Select interviewers such as students or

volunteers or contract for administration. Find resources, including telephones,

computers and space. Develop data management procedures which secure the

data and protect the anonymity of respondents.

 

5. Administer the survey. Make telephone calls and carry out quality control

steps.

 

6. Analyze the survey data.

 

7. Report the findings.

 

*You may need some assistance with this step. You can contact a local college,

university, or research firm for assistance.

 

Appendix C. Seven basic steps for conducting telephone surveys

Conducting Community Surveys 23
Telephone surveys are the least expensive way to get reliable survey information.

But there are costs involved. Costs vary from one place to another, but you should

be able to predict costs by examining the following cost factors &

 

Personnel & telephone callers and supervisors. To save money, some police

agencies use volunteers such as students or seniors or other community

members. The number of personnel you need is based, of course, on the

sample size you select and the time necessary to conduct a single survey.

You will also need personnel to be involved in planning the survey,

coordinating survey tasks, analysis, and report preparation.

 

Training & Training of callers and supervisors will require time and staff, and

materials.

 

Telephones & Each caller needs a telephone. Calls can be scheduled for evening

hours, thus permitting the use of telephones which may be otherwise used

during daytime hours.

 

Space & Sufficient space is necessary for callers and their necessary equipment.

You may have space available within the police agency or elsewhere in

local government. For at least a portion of the calls, evening or weekend

access to buildings will be necessary. 

 

Computers & Windows-based PCs for recording survey information. One

computer per caller.

 

Software & There is a small cost associated with the survey software. You will 

also need software such as SPSS or SAS for any sophisticated analysis. 

Although the CVS includes some standardized report features, these are

limited.

 

Contractual services & You should include costs for any assistance you may need 

with sampling, making calls, or analyzing data. These costs may vary 

from enough to carry out the entire survey task to nothing, if you have 

the expertise in-house.

 

Appendix D. Costs

24 A Practical Guide for Police