United States Government Accountability Office

GAO Report to Congressional Requesters

BUREAU OF JUSTICE 

STATISTICS 

Quality Guidelines 

Generally Followed 

for Police-Public 

Contact Surveys, but 

Opportunities Exist to 

Help Assure Agency 

Independence 

March 2007 

 

GAO-07-340 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

March 2007

BUREAU OF JUSTICE STATISTICS

Quality Guidelines Generally Followed for 

Police-Public Contact Surveys, but 

Opportunities Exist to Help Assure 

Agency Independence 

Highlights of GAO-07-340, a report to 

congressional requesters 

The Bureau of Justice Statistics

(BJS), a statistical agency of the 

Department of Justice’s Office of 

Justice Programs, produces a

recurring national Police-Public

Contact Survey documenting 

contacts between the police and 

the public, including instances

involving the use or threat of force 

by police. BJS issues public reports

and sometimes press releases from

survey results. 

For reports and a press release 

issued from the 1999 and 2002

surveys (the most recent available), 

GAO reviewed (1) the extent to

which BJS followed quality 

guidelines to ensure the accuracy

and integrity of its survey-related 

products, and (2) factors that 

affected whether and how BJS 

followed available guidelines. 

 

GAO reviewed applicable federal 

data quality guidelines, policy and 

procedure documents, and 

interviewed current and former 

officials familiar with BJS.

What GAO Recommends

In a May 2006 report, GAO

recommended that the Office of

Management and Budget (OMB) 

issue a directive to further 

safeguard the integrity of federal

statistical data and improve

guideline clarity. Such a directive 

could help address conflicting 

interpretations about the

applicability of quality guidelines. 

Since OMB is still working on this 

directive, GAO makes no new 

recommendations in this report. 

BJS followed nearly all quality guidelines for its 1999 and 2002 Police-Public 

Contact Surveys. Specifically, for the four public reports issued from these 

surveys, BJS fully followed all data quality guidelines available for reviewing

statistical information, obtaining the approval of key decision makers, and 

publicly disseminating information. These guidelines were issued by the 

National Research Council, Department of Justice, Justice’s Office of Justice 

Programs, and BJS itself. GAO believes that because BJS followed these 

guidelines, proper steps were taken to help ensure the accuracy and integrity 

of the reports. BJS followed 7 of the 10 quality guidelines available for the 

one press release issued from its 1999 survey, but was not in a position to 

fully follow 3 other guidelines for reasons discussed below. 

Two key factors affected whether and how BJS followed quality guidelines. 

The first concerned different interpretations about certain guideline 

applicability. BJS considered its survey-related reports—but not its press

releases—to be statistical products covered by the National Research 

Council’s guidelines. BJS cited a lack of specificity in these guidelines, which 

did not specifically state that they were applicable to statistical agency press 

releases, as a basis for concluding that the survey press releases need not 

conform to guidelines for statistical products. We believe BJS’s position was 

not unreasonable, and did not find fault with the agency. However, we 

determined nonetheless that the single press release issued from the 1999 

survey was a statistical product, and therefore believe the council’s 

guidelines appropriately applied. Second, certain noncareer appointees 

outside BJS may, in accordance with Justice Department policy, make 

decisions about the review, approval, and dissemination of press releases, 

and BJS press releases are jointly issued with the Justice Department, with 

input from its Office of Justice Programs. Both conditions can potentially 

affect BJS’s independence. Owing to these conditions, BJS was not, in our 

view, in a position to meet 3 council quality guidelines related to statistical 

agency independence, including that it be able to issue statistical products 

without prior clearance, and control the scope and content of its products. 

Justice affirmed several of GAO’s findings but disagreed with certain GAO 

conclusions about the applicability of guidelines to a press release. Justice’s 

detailed comments and GAO’s response are contained in the report. 

Guidelines BJS Followed for Products Issued from 1999 and 2002 Police-Public Contact 

Surveys 

Reports Press releases

Number of 

reports

issued 

Number of 

guidelines 

available 

Number of 

guidelines fully 

followed

Number of 

press releases

issued 

Number of 

guidelines 

available

Number of 

guidelines

fully followed

4 Up to 23a All 1b 10 7

Source: GAO analysis of agency data. 

a

Total guidelines were 24 across the different points in time; 23 was maximum at any given time. 

b

From 1999 survey information. 

www.gao.gov/cgi-bin/getrpt?GAO-07-340.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Brian Lepore at

(202) 512-4523 or leporeb@gao.gov. 
Contents 

Letter 1

Results in Brief 3

Background 6

BJS Followed Nearly All Available Quality Guidelines to Help 

Ensure Accuracy and Integrity of Products Issued from PolicePublic Contact Surveys 11 

Differing Views on Applicability of Guidelines Raised Questions 

about Clarity, while Roles of Certain Noncareer Appointees 

during Product Issuance Process Affected BJS’s Independence 14 

Agency Comments and Our Evaluation 23 

Appendix I Objectives, Scope, and Methodology 28 

Appendix II Applicable Quality Guidelines for Statistical 

Products Issued by the Bureau of Justice Statistics 

and Other Agencies 32 

Appendix III Press Release Issued in 2001 Based on BJS’s 1999 

Police-Public Contact Survey 38 

Appendix IV BJS Processes for the Review, Approval, and 

Dissemination of Police-Public Contact Survey 

Products 40 

Appendix V Changes in the Scope and Methodology of the PolicePublic Contact Surveys 43 

Appendix VI Comments from the Department of Justice 47 

Page i GAO-07-340 Bureau of Justice Statistics
Appendix VII GAO Contact and Acknowledgments 52 

Tables 

Table 1: Public Products Issued Based on 1999 and 2002 PolicePublic Contact Surveys 8 

Table 2: Guidelines That BJS Fully Followed for the Reports Issued 

from Its 1999 and 2002 Police-Public Contact Surveys 11 

Table 3: Noncareer Appointees’ Roles in the Review, Approval, and 

Dissemination of BJS Press Releases 20 

Table 4: Extent to Which BJS Followed Available Quality 

Guidelines for 1999 and 2002 Police-Public Survey 

Products 33

Table 5: Police-Public Contact Survey (PPCS) Methodologies for 

the Four Surveys: 1996, 1999, 2002, and 2005 45 

Figures 

Figure 1: Office of Justice Programs Organizational Structure 7 

Figure 2: Review, Approval, and Dissemination Process for BJS 

Survey Reports 40 

Figure 3: Review, Approval, and Dissemination Process for BJS 

Survey Press Releases 41 

Page ii GAO-07-340 Bureau of Justice Statistics
Abbreviations 

BJS Bureau of Justice Statistics 

CATI computer-assisted telephone interview 

DOJ Department of Justice 

FBI Federal Bureau of Investigation 

NRC National Research Council 

OCOM Office of Communications 

OJP Office of Justice Programs 

OMB Office of Management and Budget 

PAPI paper-and-pencil in-person face-to-face interview 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page iii GAO-07-340 Bureau of Justice Statistics
United States Government Accountability Office

Washington, DC 20548 

March 30, 2007 

Congressional Requesters: 

The Bureau of Justice Statistics (BJS), a statistical agency within the 

Department of Justice’s Office of Justice Programs, collects, analyzes, 

publishes, and disseminates information on crime, criminal offenders, 

victims of crime, and the operation of justice systems at all levels of 

government. A critical aspect of the agency’s mission is to produce 

impartial, timely, and accurate statistics. An ongoing BJS data collection 

activity is the Police-Public Contact Survey—one of many statistical 

surveys the agency administers. This recurring, nationally representative 

survey of the public is designed to document contacts between the police 

and the public and the outcomes of those contacts, including instances 

involving the use of force or the threat of force by the police. BJS issues 

reports to the public based on the results of this survey, and these may be

accompanied by a press release. To date, BJS has conducted the survey 

four times—a pilot survey was conducted in 1996, and more extensive 

surveys were done in 1999, 2002, and 2005.1

The Bureau of Justice Statistics (BJS), a statistical agency within the 

Department of Justice’s Office of Justice Programs, collects, analyzes, 

publishes, and disseminates information on crime, criminal offenders, 

victims of crime, and the operation of justice systems at all levels of 

government. A critical aspect of the agency’s mission is to produce 

impartial, timely, and accurate statistics. An ongoing BJS data collection 

activity is the Police-Public Contact Survey—one of many statistical 

surveys the agency administers. This recurring, nationally representative 

survey of the public is designed to document contacts between the police 

and the public and the outcomes of those contacts, including instances 

involving the use of force or the threat of force by the police. BJS issues 

reports to the public based on the results of this survey, and these may be

accompanied by a press release. To date, BJS has conducted the survey 

four times—a pilot survey was conducted in 1996, and more extensive 

surveys were done in 1999, 2002, and 2005.1

The Police-Public Contact Survey involves conducting interviews in 

person and by telephone with tens of thousands of individuals to obtain

information about their encounters with police officers. The most 

frequently reported reason for such contacts has involved traffic stops. In 

this context, interviewers have gathered self-reported data from motorists 

on such factors as the reason for the traffic stop and what occurred after 

the traffic stop—for example, whether the individual was ticketed, 

handcuffed, or searched, and whether the police used or threatened to use 

force. 

The Police-Public Contact Survey involves conducting interviews in 

person and by telephone with tens of thousands of individuals to obtain

information about their encounters with police officers. The most 

frequently reported reason for such contacts has involved traffic stops. In 

this context, interviewers have gathered self-reported data from motorists 

on such factors as the reason for the traffic stop and what occurred after 

the traffic stop—for example, whether the individual was ticketed, 

handcuffed, or searched, and whether the police used or threatened to use 

force. 

In the spring of 2005, a disagreement arose between the then-director of 

BJS and an official outside of BJS but within the Department of Justice 

about the statistical content of a draft press release concerning the results 

of the 2002 Police-Public Contact Survey (which at the time was the most 

recent survey to have been completed), and no press release was 

ultimately issued from this survey. 

In the spring of 2005, a disagreement arose between the then-director of 

BJS and an official outside of BJS but within the Department of Justice 

about the statistical content of a draft press release concerning the results 

of the 2002 Police-Public Contact Survey (which at the time was the most 

recent survey to have been completed), and no press release was 

ultimately issued from this survey. 

Bureau of reau of Justice Statistics

 

7-340

 1

 At the time of our review, BJS had not issued any products from the 2005 survey. 

Page 1 GAO-07-340
As a result of this incident, some members of Congress, the research 

community, and the media raised questions about the methods and 

procedures BJS follows in reviewing and approving reports and other 

products based on the agency’s statistical information, and the extent of

BJS’s independence as a statistical agency. 

In light of the questions raised about BJS’s ability to meet its self-stated 

mission as a statistical agency committed to maintaining public trust and 

confidence in its statistics, we reviewed what processes are in place at BJS 

to help the agency ensure the integrity and independence of its statistical 

studies. This report specifically addresses the following two questions for 

the 1999 and 2002 Police-Public Contact Surveys, the two surveys for 

which products had been issued as of February 2007: (1) To what extent 

did BJS follow guidelines to help ensure the accuracy and integrity of the

review, approval, and dissemination of reports and press releases based 

on its surveys? (2) What key factors affected whether and how BJS 

followed available guidelines? 

To assess the extent to which BJS followed guidelines to help ensure the 

accuracy and integrity of its product issuance processes for the 1999 and 

2002 Police-Public Contact Surveys, we reviewed quality guidelines 

pertaining to the review, approval, and dissemination of written products 

issued by the National Research Council (a private, nonprofit institution 

that is a component of the congressionally chartered National Academies), 

the Department of Justice, the Office of Justice Programs, and BJS itself. 

The guidelines that we reviewed covered the period between February 

2001, when the first product based on the 1999 survey was issued, and 

June 2006, when the most recent product based on the 2002 survey was 

issued. We initially identified a total of 63 guidelines from the four 

organizations. Because the organizations had a number of similar 

guidelines, we eliminated the overlap by reducing the list to 24 

nonduplicative guidelines. To determine the extent to which BJS followed 

each guideline, we developed a data collection instrument for recording 

information on whether BJS fully, partially, or did not at all follow the 

guideline. For BJS’s 1999 and 2002 Police-Public Contact Survey reports 

and the one press release that was issued, we asked BJS to assess the 

extent to which it followed each of 15 guidelines, and provide us with 

documentary evidence supporting its assessment. We completed the 

instrument for the remaining 9 guidelines for which we already had 

sufficient information from BJS and Office of Justice Programs documents 

and interviews, and asked BJS to confirm or not confirm our assessments. 

Two GAO analysts reviewed BJS’s responses and all available supporting 

documentary and testimonial evidence, and made a final determination on 

Page 2 GAO-07-340 Bureau of Justice Statistics
the extent to which we believed BJS followed each guideline. We provided

our findings to BJS for review and comment. We also reviewed guidelines 

issued by the Office of Management and Budget (OMB), but did not 

specifically assess BJS’s practice with respect to following OMB’s 

guidelines, because OMB issued governmentwide policy and procedural 

guidance to federal agencies that called for agencies to develop their own 

agency-specific guidelines. To determine key factors that affected whether 

and how BJS followed applicable guidelines, we reviewed BJS procedural

documents and interviewed both current and former officials involved in 

the review, approval, and dissemination of BJS products, including past 

and present BJS directors and assistant attorneys general within the Office

of Justice Programs. In addition, we reviewed pertinent statutory 

provisions relating to the roles and responsibilities of officials with respect 

to BJS. It was beyond the scope of this review to address any personnel 

issues that may have arisen in connection with the disagreement over the 

content of the draft press release. Appendix I provides additional details 

on our scope and methodology. We conducted our work between April 

2006 and February 2007 in accordance with generally accepted 

government auditing standards. 

In addition, in response to your interest regarding changes that have 

occurred in the scope and methodology of the Police-Public Contact 

Surveys between 1996 and 2005, and the reasons for any changes, 

appendix V provides detailed information on this issue. 

For the four issued reports and the one issued press release based on the 

1999 and 2002 Police-Public Contact Surveys, BJS followed nearly all 

quality guidelines related to product issuance developed by various 

organizations. These reports and press release were issued between 2001

and 2006, and are the most recent products available from the surveys. 

Specifically, for the four reports issued from these surveys, we found that 

BJS fully followed all product issuance guidelines of the National 

Research Council, the Department of Justice, the Office of Justice 

Programs, and BJS itself, pertaining to how agencies should review 

statistical information, obtain the approval of key decision makers, and 

publicly disseminate the information. Because BJS followed these 

guidelines, we believe it took proper steps to help ensure the accuracy and 

integrity of the processes associated with issuing public reports based on 

Police-Public Contact Surveys. BJS concurred with our analysis. With 

respect to the single press release issued based on the Police-Public 

Contact Survey, we found that BJS was in a position to fully follow 7 of 10

quality guidelines available for the one press release issued from the 1999

Results in Brief 

Page 3 GAO-07-340 Bureau of Justice Statistics
survey.2

 BJS did not, in our view, fully follow 3 other guidelines pertaining

to agency independence for reasons discussed below. 

Two key factors affected whether and how BJS followed quality guidelines 

during the review, approval, and dissemination of products issued from 

the 1999 and 2002 Police-Public Contact Surveys. First, BJS believed 

certain quality guidelines, including those issued by the National Research

Council, applied to its survey-related reports, but not to press releases. 

This was because BJS considered its reports to be statistical products 

covered by the guidelines, and press releases not to be statistical products, 

and therefore not covered by the guidelines. BJS cited a lack of specificity

in the National Research Council’s guidelines as a basis for this 

conclusion. While we believe that BJS’s conclusion was not unreasonable, 

our analysis of the press release issued led us to conclude—based on its 

content as opposed to its label as a press release, and regardless of the 

procedures used to develop and issue it—that it was primarily a statistical 

product. In our view, it was therefore appropriate to apply the council’s 

guidelines to it. Given that the National Research Council’s printed 

guidelines did not specifically state that they covered press releases based 

on the statistical reports of statistical agencies, we do not fault BJS for 

concluding that the 2001 press release based on the 1999 Police-Public 

Contact Survey was exempt from the guidelines. Indeed, we noted in our

May 2006 report on data quality that 2 of 14 statistical agencies we 

surveyed stated that there was ambiguity as to whether a statistical press 

release was a statistical product. BJS was among the 14 statistical agencies 

surveyed, but it was not 1of the 2 agencies reporting ambiguity in whether

a statistical press release was a statistical product. The second factor that 

affected whether and how BJS followed guidelines in its product issuance 

process concerned the role that certain noncareer appointees outside BJS 

can play in the product issuance process. In particular, certain noncareer 

appointees outside of BJS, including the Assistant Attorney General in the 

Office of Justice Programs, have the ability under Department of Justice 

policy to become involved in the review, approval, and dissemination of 

press releases; and press releases are issued jointly by BJS, the 

Department of Justice, and the Office of Justice Programs. Owing to these 

circumstances, BJS was not in a position to follow the National Research 

Council guidelines recommending that statistical agencies issue statistical 

 2

 Of the consolidated list of 24 review, approval, and dissemination guidelines issued by the 

National Research Council, Department of Justice, Office of Justice Programs, and Bureau

of Justice Statistics, only 10 were available in 2001, when the press release based on the

1999 survey was issued. 

Page 4 GAO-07-340 Bureau of Justice Statistics
products without prior clearance; and control the scope, content, and 

timing of their products—issues relating to BJS’s independence as a 

statistical agency. 

To address conflicting interpretations of certain quality guidelines, we 

recommended in a May 2006 report on the quality of federal data that OMB 

issue a directive that, among other things, would provide clear definitions 

about what guidelines cover.3

 As of January 2007, according to an OMB 

analyst assigned to the effort, OMB was still working on this directive. 

Because our May 2006 recommendation to OMB has not yet been 

implemented, we are not making any new recommendations in this report. 

We continue to believe, however, that implementing our recommendation 

could help safeguard the integrity of federal statistical data. We also 

believe it would reduce the likelihood that the type of disagreement 

discussed in this report would recur, and help assure the independence of

BJS as a statistical agency. 

We provided a draft of this report to the Department of Justice and 

received written comments from the Justice Department’s Office of 

Justice Programs’ Assistant Attorney General. The Assistant Attorney 

General affirmed several of our findings and agreed that a need exists for 

clear definitions about what constitutes a statistical product and that the 

federal statistical community would benefit from clarity in this area. The 

agency disagreed, however, with our characterization of the 2001 PolicePublic Contact Survey press release as a statistical product, and therefore 

did not believe that the National Research Council’s or BJS’s own quality 

guidelines were applicable to it. We maintain that we made a sound 

decision in applying these guidelines to the Police-Public Contact Survey 

press release because we felt that the content of the press release, which 

was almost entirely statistical in nature, was a more important 

determinant of whether or not it was a statistical product than the label 

attached to it. The Assistant Attorney General’s comments appear in 

appendix VI and our detailed response to these comments is contained in 

the report. 

 3

 GAO, Data Quality: Expanded Use of Key Dissemination Practices Would Further 

Safeguard the Integrity of Statistical Data, GAO-06-607 (Washington, D.C.: May 31, 2006). 

Page 5 GAO-07-340 Bureau of Justice Statistics
BJS was established by the Justice Systems Improvement Act of 1979.4

 In 

1995, OMB identified BJS as one of 10 principal statistical agencies within 

the federal government. As defined by OMB,5

 the statistical activities of 

statistical agencies include the planning of statistical surveys and studies; 

and the collection, processing, or tabulation of statistical data for 

publication, dissemination, research, analysis, or program management 

and evaluation. BJS publishes annual data on criminal victimization, 

populations under correctional supervision, and federal criminal offenders 

and case processing. It provides periodic data series on the administration

of law enforcement agencies and correctional facilities, prosecutorial 

practices and policies, state court case processing, felony convictions, the 

characteristics of correctional populations, criminal justice expenditure 

and employment, civil case processing in state courts, and special studies

on other criminal justice topics. 

Background 

Overview of BJS Mission 

and Organization 

BJS is organizationally located within the Department of Justice’s Office of 

Justice Programs (see fig. 1). The highest-ranking executives of BJS (BJS

Director) and the department’s Office of Justice Programs (Assistant 

Attorney General) are both noncareer officials appointed by the President 

and confirmed by the Senate. Within BJS, only the Director is a noncareer

appointee. 

 4

 Pub. L. No. 96-157, 93 Stat. 1167 (1979). 

5

 Office of Management and Budget, Statistical Programs of the United States

Government: Fiscal Year 2006, pp. 3-4 (Washington, D.C.: 2005). 

Page 6 GAO-07-340 Bureau of Justice Statistics
Figure 1: Office of Justice Programs Organizational Structure 

United States

Department of Justice

Attorney General

Deputy Attorney General

Associate Attorney General

Office of Justice

Programs

Assistant Attorney General

 Deputy Assistant Attorney General

Chief of Staff

Bureau of Justice

Assistance

Bureau of Justice

Statistics

Director

National Institute 

of Justice

Office of Juvenile 

Justice and Delinquency

Prevention

Community Capacity

Development

Office

Office for

Victims of Crime

Office for

Civil Rights

Office of 

General Counsel

Office of 

Communications

Chief 

Information Officer

Office of

Management and

Administration

Source: BJS.

The BJS Police-Public 

Contact Survey 

BJS initiated the Police-Public Contact Survey pursuant to a mandate in

the Violent Crime Control and Law Enforcement Act of 1994,

6

 which 

required the Attorney General to collect information on the use of 

excessive force by law enforcement officers. The data were to be used 

only for research or statistical purposes and were not to contain any 

information that could reveal the identity of the victim or any law 

enforcement officer. BJS fielded its first pilot survey in 1996 with the goal 

of better understanding the types and frequency of contacts between the

police and the public, and the conditions under which force may be 

threatened or used. The pilot survey consisted of 6,421 respondents. The 

 6

 Pub. L. No. 103-322, 108 Stat. 1796, 2071 (1994) (codified at 42 U.S.C. 14142). 

Page 7 GAO-07-340 Bureau of Justice Statistics
three subsequent surveys (in 1999, 2002, and 2005) consisted of 80,543, 

76,910, and 63,943 respondents, respectively. Multiple reports and press 

releases may be issued in connection with any of the surveys. The years in 

which reports and a single press release associated with the 1999 and 2002 

surveys were issued are shown in table 1. 

Table 1: Public Products Issued Based on 1999 and 2002 Police-Public Contact 

Surveys 

Date of survey Years reports issued Year press release issued

1999 2001, 2002a,b 2001e

2002 2005, 2006c,dNone issued

Sources: GAO, BJS.

a

Contacts between Police and the Public, Findings from the 1999 National Survey, February 2001. 

b

Characteristics of Drivers Stopped by Police, 1999, March 2002. 

c

Contacts between Police and the Public, Findings from the 2002 National Survey, April 2005. 

d

Characteristics of Drivers Stopped by Police, 2002, June 2006. 

e

Force or Threatened Force Used in Less than 1 Percent of All Police-Public Interactions, March 

2001. 

Quality Guidelines for 

Statistical Products 

Over the last several years, various types of guidance have been developed 

to help federal agencies such as BJS ensure the integrity of statistical 

information. In 1992, in response to requests from Congress and others as 

to what constitutes an effective statistical agency, the National Research

Council began issuing best-practice guidelines.7

 According to the 

Committee on National Statistics,8

 which authored the guidelines, the 

guidelines have been widely cited and used by Congress and federal 

agencies, and have shaped legislation and executive actions to establish 

and evaluate statistical agencies. These recommended guidelines, which 

BJS and other statistical agencies may choose to voluntarily follow, cover

the review, approval, and dissemination processes of products issued by 

 7

 For ease of presentation, in this report we use the term “guidelines” to refer to the

principles and practices described in the National Research Council document, as well as 

to guidelines issued by the Department of Justice, Office of Justice Programs, and BJS. 

8

 The National Research Council’s Committee on National Statistics was established in 1972

at the recommendation of the President’s Commission on Federal Statistics in order to

improve the statistical methods and information on which public policy decisions are 

based. 

Page 8 GAO-07-340 Bureau of Justice Statistics
federal statistical agencies. In its guideline document, Principles and 

Practices for Federal Statistical Agencies, the National Research Council 

indicated, among other things, that statistical agencies should provide 

high-quality data, take a strong position of independence, be perceived to 

be free of political interference and policy advocacy, and strive for wide 

dissemination of their results.9

 In particular, according to the National 

Research Council, the quality guidelines are to cover the 

• review process and include verification of sources and results, 

disclosure of limitations, and accuracy of results; 

• approval process and include who has authority over the content and 

timing of the release of a product, and separation of policy from

statistical information; and 

• dissemination process and include the usability of information and its 

accessibility to a wide range of people. 

In February 2002 and September 2006, pursuant to the Information Quality 

Act of 2001, OMB issued policy and procedural guidance to federal 

agencies, including statistical agencies such as BJS, directing them to 

develop their own quality guidelines to help maximize the quality, 

objectivity, utility, and integrity of the information they disseminate.10 OMB 

stated that it was essential that federal statistics be collected, processed, 

and published in a manner that guarantees and inspires confidence in their 

reliability.11 Specifically, OMB directed federal agencies to “adopt a basic 

standard of quality ... as a performance goal,” and “take appropriate steps 

to incorporate information quality criteria into agency dissemination 

practices.” 

In response to OMB’s February 2002 guidance, the Department of Justice,

Office of Justice Programs, and BJS issued their own guidelines later that

year. BJS issued a second edition of its guidelines in 2005. In formulating 

its guidelines, BJS stated that it sought to provide the public with 

 9

 Margaret E. Martin, Miron L. Straf, and Constance F. Citro (eds.), Principles and

Practices for a Federal Statistical Agency, (3rd ed.) (Washington, D.C.: The National 

Academies Press, 2005). 

10 Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 

2001 (Pub. L. No. 106-554, 114 Stat. 2763, 2763A-125 (2000)), known as both the Information

Quality Act and the Data Quality Act, required OMB to issue governmentwide quality 

guidelines including the requirement that such agencies issue their own quality guidelines. 

11 See Notice of revisions to OMB’s Standards and Guidelines for Statistical Surveys, 71 Fed.

Reg. 55522 (Sept. 22, 2006). 

Page 9 GAO-07-340 Bureau of Justice Statistics
additional information regarding its methods for ensuring the quality,

utility, objectivity, and integrity of the statistics it publicly disseminates. 

As a component of the Department of Justice, BJS is governed by its own 

data quality guidelines, as well as the information quality guidelines 

promulgated by the Office of Justice Programs, Department of Justice, and 

Office of Management and Budget. The Department of Justice’s 

Information Quality Guidelines are intended to (1) provide the 

department’s components with a foundation for developing their own, 

more detailed procedures, (2) provide guidance to component staff, and 

(3) inform the public of the agency’s policies and procedures.

The Office of Justice Programs’ information quality guidelines require its 

components—including BJS—to (1) assess the usefulness of the 

information to be disseminated to the public by continuously monitoring 

information needs, developing new information sources, or revising 

existing methods, models, and information products where appropriate; 

(2) ensure disseminated information is accurate, clear, complete, 

reproducible, and presented in an unbiased manner by using reliable data

sources, sound analytical techniques, and documenting methods and data 

sources; and (3) protect information from unauthorized access, 

corruption, or revision. As in the case of the Department of Justice’s 

guidelines, the Office of Justice Programs provides guidance to its 

components in developing their own, more specific quality guidelines. 

Page 10 GAO-07-340 Bureau of Justice Statistics
For all four reports issued from the two Police-Public Contact Surveys, we 

found that BJS fully followed all of the review, approval, and

dissemination guidelines available at the time of issuance. We considered 

a guideline to have been fully followed if our independent analysts 

determined that all aspects of the guideline were followed. (Our 

methodology for how we determined the extent to which BJS followed the 

guidelines is explained in app. I.) The extent to which BJS followed 

applicable, available guidelines when it issued its Police-Public Contact 

Survey reports is shown in table 2. 

BJS Followed Nearly

All Available Quality 

Guidelines to Help 

Ensure Accuracy and 

Integrity of Products 

Issued from PolicePublic Contact 

Surveys

BJS Fully Followed All 

Applicable Quality 

Guidelines for the Four 

Reports Issued from the 

1999 and 2002 PolicePublic Contact Surveys 

Table 2: Guidelines That BJS Fully Followed for the Reports Issued from Its 1999 

and 2002 Police-Public Contact Surveys 

Year survey 

conducted Year report issued

Number of 

 applicable 

guidelines

 available when 

report was issued 

Number of 

 applicable 

guidelines that

 BJS fully followed 

1999 2001 10 10

2002 12 12

2002a2005 23 23

2006 23 23

Source: GAO analysis 

a

The number of guidelines varied across the time periods covered. One National Research Council 

guideline was dropped when the council issued the second edition of its guidelines. Had this guideline 

not been dropped, 13 guidelines would have been applicable to BJS’s 2002 survey report, instead of

12, and 24 guidelines would have been applicable to the 2005 and 2006 survey reports, instead of 

23. Table 4 in appendix II reflects which guidelines were applicable to each of the survey products. 

For the first report issued from the 1999 survey, we found that BJS 

voluntarily followed the National Research Council’s 10 applicable 

Page 11 GAO-07-340 Bureau of Justice Statistics
existing guidelines; for the second report, we found that BJS voluntarily 

followed those 10, as well as 2 additional guidelines issued since the first 

report, for a total of 12. For each of the two reports based on the 2002 

survey, we found that BJS followed all 23 available data quality guidelines 

that had by then been issued by the National Research Council, the 

Department of Justice, the Office of Justice Programs, and BJS itself.12

The data quality guidelines that BJS followed describe how agencies 

should review statistical information, obtain the approval of key decision 

makers, and publicly disseminate the information. While not all of the 

guideline-issuing organizations addressed the review, approval, and 

dissemination process, in total across the four organizations—the National

Research Council, Department of Justice, Office of Justice Programs, and 

Bureau of Justice Statistics—all three areas were addressed. Some 

examples of the guidelines that BJS fully followed in its report issuance 

process are listed below. (For a complete list of all available data quality 

guidelines, see appendix II.) 

• Components of the Department of Justice and Office of Justice 

Programs will review all information dissemination products for their 

quality (including objectivity, utility, and integrity) before they are 

disseminated. 

• All BJS reports and other statistical products must be subject to an 

objective and appropriate verification process conducted by qualified 

BJS staff other than the author of the report. 

• The statistical agency has recognition by policy officials outside the 

statistical agency of its authority to release statistical information 

without prior clearance.

• The statistical agency has authority for professional decisions over the 

scope, content, and frequency of data compiled, analyzed, or published. 

• The objectivity of BJS statistics must be vigilantly protected at all times 

by BJS staff. 

On the basis of our analysis, BJS successfully followed all applicable 

quality guidelines for these survey-based statistical reports, which both 

BJS and we consider to be statistical products covered by the guidelines. 

Thus, we believe the agency took proper steps to help ensure the accuracy 

and integrity of the review, approval, and dissemination processes 

associated with issuing public reports based on the two surveys we 

 12 After OMB issued policy and procedural guidance to federal agencies in 2002, the

Department of Justice, Office of Justice Programs, and BJS developed their own guidelines. 

Page 12 GAO-07-340 Bureau of Justice Statistics
reviewed. BJS concurred with our analysis. All of the reports were posted 

to the BJS Web site, where the information is to be accessible to the 

general public.13

BJS Fully Followed 7 of 10 

Applicable Data Quality 

Guidelines for the Single 

Press Release Issued from 

Its Police-Public Contact 

Surveys 

For the single press release that was issued—that is, the 2001 press release 

based on BJS’s 1999 Police-Public Contact Survey—we determined that 

BJS fully followed 7 of the 10 applicable National Research Council 

guidelines available at the time.14 The 7 federal data quality guidelines that

BJS fully followed are listed below. 

(1) A statistical agency should develop an understanding of the validity 

and accuracy of its data and convey the resulting measures of quality 

to users in ways that are comprehensible to nonexperts. 

(2) A statistical agency should use modern statistical theory and sound 

statistical practice in all technical work. 

(3) A statistical agency has maintenance of a clear distinction between 

statistical information and policy interpretations of such information 

by the President, the secretary of the department, or others in the 

executive branch. 

(4) A statistical agency should follow good practice, in reports and other 

data releases, in documenting concepts, definitions, data collection 

methodologies, and measures of uncertainty, and in discussing 

possible sources of error. 

(5) Effective dissemination programs include policies for the preservation 

of data that guide what data to retain and how they are to be archived 

for future secondary analysis. 

(6) An agency should have an established publications policy that 

describes, for a data collection program, the types of reports and other 

 13 Information posted on the BJS Web site includes hypertext links to information that is

created and maintained by other public and private organizations in the United States and 

by other nations. 

14 Guidelines developed by BJS, the Office of Justice Programs, and the Department of

Justice were not available at the time the press release, based on the 1999 survey, was 

released. 

Page 13 GAO-07-340 Bureau of Justice Statistics
data released to be made available, the audiences served, and the 

frequency of release. 

(7) Dissemination of data and information (basic series, analytic reports, 

press releases, public use tapes) should be timely and public. Avenues 

of dissemination should be chosen to reach as broad a public as 

reasonably possible. 

There were 3 other applicable quality guidelines that we determined BJS 

was not in a position to follow in connection with this issued press 

release, and a press release based on the 2002 Police-Public Contact 

Survey findings was not issued. It is important to note that, for reasons 

discussed later in this report, BJS officials did not believe these guidelines

were applicable to its press releases in the first place. 

Two key factors affected whether and how BJS followed quality guidelines 

during the review, approval, and dissemination of products issued from 

the 1999 and 2002 Police-Public Contact Surveys. First, while BJS 

believed, as noted earlier, that its survey reports were statistical products

covered by the quality guidelines, it did not believe that the survey-related 

press release was a statistical product covered by the quality guidelines. 

BJS cited a lack of specificity in the National Research Council’s 

guidelines as a basis for this conclusion. We believe, however, that while 

BJS’s interpretation of the guidelines was not unreasonable, there was 

nonetheless sufficient evidence for a different interpretation; namely, that

this press release was a statistical product, that the available guidelines 

did apply, and that BJS was not in a position to meet 3 of 10 guidelines for 

the single press release issued from the 1999 survey, owing to a second 

factor. This second factor was the role that certain noncareer appointees 

outside BJS have the ability to play, pursuant to Department of Justice 

policy, in the product issuance process. In certain instances, the roles of 

these non-BJS officials meant that BJS was not in a position to fully follow 

all guidelines related to agency independence, and this holds the potential 

for future actual or perceived political interference in BJS’s product 

issuance process for statistical products. 

Differing Views on 

Applicability of 

Guidelines Raised 

Questions about 

Clarity, while Roles of 

Certain Noncareer 

Appointees during 

Product Issuance 

Process Affected 

BJS’s Independence 

Page 14 GAO-07-340 Bureau of Justice Statistics
In both written documentation and oral comments, BJS officials stated 

that they believed they were in full conformance with the National 

Research Council’s guidelines and disagreed with our determination that 

the agency was not in a position to follow 3 of 10 guidelines for the 2001 

Police-Public Contact Survey press release that was issued from the 1999 

survey. The guidelines that we determined BJS was not in the position to

fully follow all pertain to the agency’s independence and, in particular, to 

its control over the issuance of press releases. These guidelines were: 

BJS Interpreted Guidelines 

as Not Applying to Press 

Releases, Raising 

Questions about the 

Applicability and Clarity of 

Certain Guidelines 

(1) The statistical agency has recognition by policy officials of its 

authority to release statistical information without prior clearance. 

(2) The statistical agency has authority for professional decisions over the 

scope, content, and frequency of data compiled, analyzed, or 

published. 

(3) The release of information should not be subject to actual or perceived

political interference. In particular, the timing of the public release of 

data should be the responsibility of the statistical agency. 

BJS officials asserted that, based on their interpretation of the National 

Research Council’s guidelines, BJS press releases did not qualify as 

statistical products and, therefore, press releases did not fall within the 

purview of the council’s guidelines. They also asserted that neither BJS’s 

own quality guidelines, nor those issued by the Department of Justice and 

the Office of Justice Programs, apply to BJS press releases. Both BJS and 

Office of Justice Programs officials stated that the applicability of the 

council’s guidelines to BJS press releases was, at a minimum, open to 

question because the council did not state that press releases are data 

disseminations. In other words, according to BJS and the Office of Justice

Programs, press releases are not publications of data, but rather they are 

simply announcements that a data publication is forthcoming. In its 

communications with us, BJS stated that many of the guidelines do not 

apply to press releases but apply only to statistical products.

Based on its content rather than its label as a press release, and 

notwithstanding that the policies and procedures for developing and 

issuing products labeled by the Office of Justice Programs as press 

releases differed from policies and procedures for products it labeled as 

statistical products, we believe there is sufficient evidence for us to 

conclude that the press release issued from the 1999 Police-Public Contact 

Survey qualified as a statistical product to which the National Research 

Council’s quality guidelines appropriately apply. Our analysis of this press 

Page 15 GAO-07-340 Bureau of Justice Statistics
release indicated that it was a data-based statistical product, more than 

simply an announcement that a data publication was forthcoming. In its 

entirety, the press release consisted of 20 sentences and one table 

describing the survey’s statistical findings; 3 sentences on the survey’s 

methodology; and 5 sentences on who prepared the report and how to 

obtain copies. We found that this press release was a compilation of 

statistical data that contained no interpretations, conclusions, or policy 

statements.15 (See Appendix III for a reproduction of the press release.) In 

accordance with the council’s guidelines, the release maintained “a clear 

distinction between statistical information and policy interpretations of 

such information.” 

To understand whether the National Research Council was purposeful in 

not stating that its guidelines were applicable to statistical agency press 

releases, we contacted the council to seek clarification. Officials from the 

council’s Committee on National Statistics, which authored the data 

quality guidelines, stated that although the Principles and Practices

document did not specifically state that the guidelines covered the 

content, scope, and timing of press releases issued by statistical agencies, 

it was not the committee’s intent to exclude press releases from the 

guidelines. They stated that, in their view, press releases issued by BJS are 

statistical products to which it is appropriate to apply the guidelines. 

BJS and we agree that the National Research Council’s guidelines apply, in

general, to statistical products. In asserting that the press release that BJS 

jointly issued with the Department of Justice and Office of Justice 

Programs was not a statistical product, BJS correctly noted that the 

National Research Council did not explicitly state that the guidelines 

covered press releases. However, given the strong statistical content of the 

Police-Public Contact Survey press release, we did not believe that such 

an explication was necessary. Nonetheless, we acknowledge that it is not 

unreasonable for BJS to reach a different conclusion given the lack of 

specificity that existed in the council’s printed guidelines. Because BJS’s

own data quality guidelines, issued in 2002, state that they “govern all 

justice statistics that BJS produces and disseminates for the general 

public, including all statistics that are featured in BJS publications, on the

website, and in BJS press releases,” we considered the BJS guidelines to

 15 Our work focused exclusively on the Police-Public Contact Survey, and we did not

conduct a systematic review of other press releases that have been issued based on BJS’s

work, to make an independent determination of whether they constituted statistical 

products. 

Page 16 GAO-07-340 Bureau of Justice Statistics
be applicable to press releases, as well. BJS, however, did not hold this 

view. 

It is important to note that we are not finding fault with BJS for the 

conclusions it drew with respect to the applicability of the quality 

guidelines to its press release issuance process because the National 

Research Council’s guidelines were not explicit on this matter. Indeed, we 

noted in our May 2006 report on data quality that 2 of 14 statistical 

agencies we surveyed stated that there was ambiguity as to whether a 

statistical press release was a statistical product, and if so, whether 

statistical agencies could issue them without first getting releases cleared

at the departmental level. BJS was among the 14 statistical agencies 

surveyed, but it was not one of the two agencies reporting ambiguity in 

whether a statistical press release was a statistical product. Overall, we 

believe that BJS made a good faith effort to follow the guidelines it 

deemed to be applicable to the Police-Public Contact Survey products. 

Deciding which guidelines a statistical agency like BJS should follow is 

further complicated by the fact that BJS’s parent organizations—the 

Department of Justice and Office of Justice Programs—have explicitly 

stated that their own guidelines do not apply to press releases. However, 

these organizations’ guidelines are intended to be broadly applicable to

both statistical and nonstatistical agencies. For example, the Department 

of Justice comprises 38 separate component organizations that produce a 

variety of types of information, both statistical and nonstatistical in nature. 

The Office of Justice Programs is composed of 6 bureaus and program 

offices, and these, too, produce both statistical and nonstatistical 

information. Because we believe that press releases issued by the 

department and the Office of Justice Programs may in some, but not all, 

instances be statistical products, we do not hold the view that statistical 

guidelines should be universally applicable to all press releases issued by 

the Department of Justice and Office of Justice Programs. However, 

because different interpretations can arise, we believe that clarification 

regarding which guidelines should be applied under which 

circumstances—and, specifically, to press releases—would be helpful to 

statistical agencies that are in situations similar to BJS’s. 

To address potential discrepancies such as these, in a May 2006 report on

the quality of federal data,

16 we recommended that to help improve 

 16 See GAO-06-607. 

Page 17 GAO-07-340 Bureau of Justice Statistics
governmentwide data dissemination practices that would further 

safeguard the integrity of federal statistical data, OMB should consider 

how best to address the gaps we identified between agencies’ data 

dissemination practices and the National Research Council’s guidelines. 

We noted in that report that OMB, in concert with federal statistical 

agencies, was developing a governmentwide directive on the release and 

dissemination of statistical products that, according to OMB officials, 

parallels the council’s and other generally accepted dissemination 

practices. We pointed out that it will be important for OMB’s directive to 

consider, for example, whether the directive should cover principal 

statistical agencies only, the statistical functions of all agencies, or only 

statistical products. OMB officials indicated that the guidance is intended

to help ensure that statistical products are policy-neutral, timely, and 

accurate. We recommended that, among other things, OMB include in this

directive (1) clear definitions of what is and is not covered, (2) the extent

to which agencies should document their data dissemination guidance and 

how often the guidance should be reviewed, (3) the amount of flexibility 

agencies have in implementing OMB’s guidance, and (4) procedures for 

monitoring agencies’ adherence to its directive. To the extent that 

statistical agencies appropriately follow these practices, the directive 

could promote more consistent adherence to practices that facilitate 

broader dissemination of statistical data and enhance its credibility. 

Although OMB did not provide comments on the recommendations in our 

2006 report, an OMB official told us that as of January 2007, OMB was still 

working on this directive.17 We believe it remains important for OMB to 

complete its directive on the release and dissemination of statistical 

products in order to help safeguard the integrity of federal statistical data, 

reduce the likelihood that the type of disagreement discussed in this 

report would recur, and help assure both the actual and perceived 

independence of BJS. 

 17 OMB’s February 2002 guidelines for federal agencies exempted press releases from the

guidelines. These guidelines, however, were designed to apply to a wide variety of 

government information dissemination activities, and be generic enough to fit all types of 

media. As we noted in our May 2006 report (GAO-06-607), the directive on which OMB is

currently working pertains directly to the release and dissemination of statistical products. 

Page 18 GAO-07-340 Bureau of Justice Statistics
The second key factor that affected whether and how BJS followed 

guidelines concerned the involvement of noncareer appointees outside of 

BJS in the press release issuance process, and had implications for BJS’s 

independence as a statistical agency. Specifically, we determined that BJS

was not in a position to fully follow the 3 National Research Council 

guidelines listed in the previous section for the 2001 press release based 

on the 1999 survey (the only applicable, available data quality guidelines in 

place in 2001) because certain noncareer appointees outside of BJS and 

within the Department of Justice, are vested—pursuant to the Department 

of Justice’s and Office of Justice Programs’ policies defining the roles and 

responsibilities of their noncareer appointees—with the ability to 

participate in the review, approval, and dissemination of press releases.18

In certain cases, the roles and responsibilities of these noncareer 

appointees precluded BJS from being in the position to fully follow certain 

guidelines. The Assistant Attorney General within the Department of 

Justice’s Office of Justice Programs has general statutory responsibilities 

with respect to coordinating the activities of that office and its various 

components, such as BJS.19 These statutory provisions do not specifically 

address the Office of Justice Programs’ role with respect to the review, 

approval, and dissemination of press releases. However, under 

departmental policy, noncareer appointees within the Department of 

Justice and outside of BJS have the ability to participate in the press 

release issuance process.

The Roles Played by 

Certain Outside Noncareer 

Appointees in the Press 

Release Process Affected 

BJS Carrying Out Its Role 

as an Independent 

Statistical Agency 

Table 3 shows the type of involvement that the Assistant Attorney General 

in the Office of Justice Programs and other noncareer appointees generally

have had in the press release review, approval, and dissemination process. 

Appendix IV describes in more detail the responsibilities of these various 

officials associated with review, approval, and dissemination procedures 

for both BJS reports and press releases. 

 18 We did not review the extent to which BJS followed guidelines for the 2005 draft press 

release (based on the 2002 survey) that was the source of disagreement between the thenBJS Director and the then-Acting Assistant Attorney General, because the press release 

was not issued. 

19 U.S. Code, Title 42, Chapter 46, Subchapter I (42 U.S.C. Sections 3711 through 3715a). 

Page 19 GAO-07-340 Bureau of Justice Statistics
Table 3: Noncareer Appointees’ Roles in the Review, Approval, and Dissemination of BJS Press Releases 

Entity involved in press release process 

Bureau of

Justice Statistics Office of Justice Programs 

Department

of Justice 

Area of

involvement Director Chief of Staff 

Deputy Assistant 

Attorney General 

Assistant Attorney 

General 

Office of Public

Affairs Director

Review X X X X X

Approval X X X X X

Dissemination XaXb

Source: BJS and Office of Justice Programs.

a

To the BJS web site, after approval by the BJS Director. 

b

To Congress, the media, and executive department press offices, after verification by the BJS 

Director. 

With respect to the first of the three guidelines, which calls for a statistical 

agency to have authority to release information without prior clearance, it

is our view that BJS was not in a position to follow this independencerelated guideline at all because it did not have the ability to do so. This is 

because press releases are subject to review and approval by not only the 

BJS Director, but also by other Department of Justice noncareer 

appointees.20 Outside of BJS, the noncareer appointees participating in the 

clearance process are located in the Department of Justice’s Office of 

Justice Programs (these include the Office’s Chief of Staff, Deputy 

Assistant Attorney General, and Assistant Attorney General) and Office of 

Public Affairs. 

The current Assistant Attorney General and two former Assistant 

Attorneys General in the Office of Justice Programs told us that there is no 

written, formal policy or guidance that bounds their input and decisionmaking roles and responsibilities with respect to BJS press releases. BJS 

and OJP officials indicated that the Office of Justice Programs’ Assistant 

Attorney General has ultimate responsibility for the review and approval 

of BJS press releases. Press releases are issued jointly on letterhead listing 

BJS and the Department of Justice. The current BJS Director confirmed 

that publication and dissemination functions for press releases are 

 20 In contrast to the process followed for press releases, the only noncareer appointee 

involved in the review, approval, and dissemination of reports from Police-Public Contact

Surveys is the BJS Director. Appendix IV provides details on the roles of the Director and 

other employees in this process.

Page 20 GAO-07-340 Bureau of Justice Statistics
considered to be within the Assistant Attorney General’s oversight 

authority. Because the National Research Council stated that an aspect of 

independence includes “recognition by policy officials outside the 

statistical agency of its authority to release statistical information without 

prior clearance,” we concluded that BJS was not in the position to follow 

this guideline because, as we have stated, we believe the Police-Public 

Contact Survey press release was a statistical product that BJS could not 

issue independently. 

In practice, the ways in which Assistant Attorneys General of the Office of

Justice Programs have exercised their authority have varied. For instance, 

one former Office of Justice Programs’ Assistant Attorney General told us 

that she placed “self-imposed” limits on her decisions to modify the 

content of a BJS press release based on her awareness of congressional 

support for, and her own belief in, the independence of statistical 

agencies. The current Office of Justice Programs’ Assistant Attorney 

General told us that she reviews only press releases that contain quotes 

from the Attorney General. She said that since she assumed her position in 

2005, there have been no BJS press releases that have quoted the Attorney

General, and she has relied on her Deputy Assistant Attorney General, the 

BJS Director, and others to ensure the accuracy and clarity of press 

releases. Nevertheless, the BJS Director must obtain the approval of the 

Office of Justice Programs’ Assistant Attorney General and other Justice 

noncareer appointees to issue a press release. 

With respect to the second guideline, pertaining to the agency’s decisions 

over the scope, content, and frequency of data compiled, analyzed, or 

published, we found that BJS was not in a position to fully follow this 

independence-related guideline. Specifically, we found that BJS could 

exercise professional decisions about the frequency of data analyzed and

published (within available budgets), but did not always have complete 

control over the scope and content of survey press releases to be issued. 

As noted above, this was due to the fact that press releases are joint 

products of BJS, the Office of Justice Programs, and the Department of 

Justice, and noncareer appointees outside of BJS can become involved in

the press release process. BJS’s situation with respect to this second 

guideline came to the fore during the drafting of a press release in 2005 

based on the 2002 Police-Public Contact Survey. The press release that 

BJS sought to publish would have included the following statistical 

findings from the accompanying Police-Public Contact Survey report: (1) 

there was no statistically significant difference between the rates that 

white and minority drivers reported being stopped by police, and (2) once 

stopped, a larger percentage of black and Hispanic minority drivers 

Page 21 GAO-07-340 Bureau of Justice Statistics
reported police using or threatening to use force against them than did 

whites. The then-BJS Director and the then-Acting Assistant Attorney 

General had a difference of opinion regarding the presentation of the 

second statistical finding, which was included in the Police-Public Contact 

Survey report. Despite reported efforts on the part of both parties to 

negotiate alternative language with respect to the content of the press 

release, they could not resolve their differences and the BJS Director 

decided that a press release would not be issued. The current BJS Director 

told us that it is “inconceivable” that the Assistant Attorney General would 

issue a press release without the BJS Director’s prior approval.

According to current BJS officials (both career and noncareer) and the 

Office of Justice Programs’ Office of Communications staff, during the 

period 1996-2006, this was the only instance in which a BJS press release 

was prepared but not issued because the Office of Justice Programs and 

BJS could not agree on the contents. In all other instances during this 

period, according to these officials, when the parties disagreed on the 

content of a press release, they were able to resolve their differences. 

With respect to the third guideline, pertaining to actual or perceived 

political interference and the timing of a release, we similarly believe BJS 

was in not in a position to fully follow this independence-related guideline 

for the 2001 press release, which, as discussed earlier, we believe to be a 

statistical product. Although we found no evidence of political

interference with the timing of the 2001 survey press release issued from 

the 1999 survey, we found that BJS does not have complete control over 

the timing of press releases, as recommended by the National Research 

Council. Since both noncareer appointees and career officials in the Office 

of Justice Programs and the Department of Justice have a role in reviewing 

and approving BJS press releases, they can affect the date that a press 

release is issued. According to BJS, career and noncareer appointees 

outside of BJS can delay the issuance of a press release for reasons having 

nothing to do with political interference, such as a determination that the 

press release is not sufficiently newsworthy at the time that it was 

designated to be issued.

On balance, we believe that the noncareer appointees who played 

decision-making roles in the Police-Public Contact Survey press release 

process that we reviewed acted within the scope of the roles and 

responsibilities accorded them under Department of Justice policies, and 

that BJS made a reasonable effort to adhere to all applicable data quality 

guidelines. The fact that certain noncareer officials have the ability to 

make decisions that affect BJS’s ability to fully meet federal data quality 

Page 22 GAO-07-340 Bureau of Justice Statistics
guidelines suggests, however, that the potential exists for BJS’s review, 

approval, and dissemination process for statistical products to be subject 

to political interference. Thus, certain actions by noncareer appointees—

though made on the basis of professional judgment—could put them at 

odds with the very guidelines designed to ensure the statistical

independence and integrity of agencies such as BJS. 

We provided a draft of this report to the Department of Justice for review 

and comment. On March 13, we received written comments on the draft

report from the Office of Justice Programs’ Assistant Attorney General, 

and the comments are reproduced in full in appendix VI. 

Agency Comments 

and Our Evaluation 

In her letter, the Assistant Attorney General affirmed several of our 

findings and agreed that a need exists for clear definitions about what 

quality guidelines cover. She noted that competing interpretations exist 

about what constitutes a statistical product and that the federal statistical 

community would benefit from clarity in this area. 

However, the Assistant Attorney General disagreed with our 

characterization of the 2001 Police-Public Contact Survey press release as 

a statistical product and, therefore, with our conclusion that the National

Research Council’s quality guidelines applied to this press release. The 

Assistant Attorney General stated that “a press release ... is a public 

relations announcement issued to encourage media coverage. The mere 

presence of statistics in a press release does not transform a press release 

into a statistical product.” We do not believe and have not stated that the 

mere presence of statistics in a press release in and of itself transforms it 

into a statistical product any more than we believe or have stated that 

labeling a document lacking in statistics but called a statistical product 

necessarily transforms it into one. The Assistant Attorney General also 

stated that we “mischaracterized” BJS’s data quality guidelines as applying 

to press releases because the guidelines apply only to the statistics 

contained in BJS press releases, and because BJS conforms with OMB, the 

Department of Justice, and the Office of Justice Programs in considering 

press releases to be outside the scope of the guidelines. For the following

reasons, we maintain that we made a sound decision in applying BJS’s 

guidelines to the Police-Public Contact Survey press release: (1) BJS’s 

guidelines state that they “govern all justice statistics that BJS produces 

and disseminates for the general public, including all statistics that are 

featured in BJS publications, on the website, and in BJS press releases;” 

and (2) the Police-Public Contact Survey press release was made up 

almost entirely of survey statistics, indicating to us that it was a statistical 

Page 23 GAO-07-340 Bureau of Justice Statistics
product. In determining that the Police-Public Contact Survey was a 

statistical product, we felt that the content of the press release was a more 

important determinant than the label attached to it, or the fact that the 

processes and staff involved in developing the press release were different 

from those in BJS reports. 

The Assistant Attorney General also noted that the National Research 

Council’s written guidelines did not explicitly cover press releases. 

Because we agree, we contacted the National Research Council and 

consulted with officials of the Council’s Committee on National Statistics

(the authoring committee of the Principles and Practices). The officials 

concurred with our view that BJS press releases referring to statistical 

products (as opposed to press releases about the announcement of a new 

agency head, for example) are statistical products to which it is 

appropriate to apply the guidelines. Although the Principles and 

Practices document did not specifically state that the guidelines covered

the content, scope, and timing of press releases issued by statistical 

agencies, according to these officials it was not the Committee’s intent to

exclude such press releases from the guidelines. 

The Assistant Attorney General also felt that the draft report overstated 

the potential threats to BJS’s independence because we used the term 

“statistical products” to refer to press releases. She was concerned with 

our observation that the potential exists for BJS’s review, approval, and 

dissemination process for statistical products to be subject to political 

interference since noncareer officials can affect BJS’s ability to meet 

federal data quality guidelines. We stand by this conclusion. Department 

of Justice policy permits noncareer appointees within the Department but 

outside of BJS to participate in the press release process. At the same 

time however, we are unaware of anything that prevents future 

modifications to that policy to similarly allow noncareer appointees to 

participate in BJS’s report issuance. Thus, we believe that we have 

correctly assessed the risk of potential or actual threats to BJS’s 

independence. 

Finally, the Assistant Attorney General stated that even if the council’s 

written guidelines explicitly applied to press releases, the BJS director 

would not adhere to them and no current law can make him do so. We 

recognize that they are voluntary and not legally required and never have 

said otherwise. 

Page 24 GAO-07-340 Bureau of Justice Statistics
As agreed with your offices, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days from its 

issue date. At that time, we will send copies of the report to the Attorney 

General, the Director of the Office of Management and Budget, and other 

interested parties. In addition, the report will be available at no charge on

GAO’s home page at http://www.gao.gov. 

Page 25 GAO-07-340 Bureau of Justice Statistics
Please contact Brian Lepore at (202) 512-4523 or leporeb@gao.gov if you 

or your staff have any questions concerning this report. Contact points for 

our Offices of Congressional Relations and Public Affairs may be found on

the last page of this report. Key contributors to this report are listed in 

appendix VII. 

Brian Lepore, Acting Director 

Homeland Security and Justice Issues 

Page 26 GAO-07-340 Bureau of Justice Statistics
List of Congressional Addressees 

The Honorable John Conyers 

Chairman 

Committee on the Judiciary 

House of Representatives 

The Honorable William Delahunt 

The Honorable Sheila Jackson Lee 

The Honorable Zoe Lofgren 

The Honorable Martin Meehan 

The Honorable Jerrold Nadler 

The Honorable Linda Sanchez 

The Honorable Debbie Wasserman Schultz 

The Honorable Robert C. “Bobby” Scott 

The Honorable Mel Watt 

The Honorable Robert Wexler 

House of Representatives 

Page 27 GAO-07-340 Bureau of Justice Statistics
Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology 

Methodology 

This report addresses the following two objectives for the 1999 and 2002 

Police-Public Contact Surveys, the two surveys for which products had 

been issued as of February 2007: (1) To what extent did the Bureau of 

Justice Statistics (BJS) follow available guidelines to help ensure the 

accuracy and integrity of the review, approval, and dissemination of 

reports and press releases based on its surveys? (2) What key factors 

affected whether and how BJS followed available guidelines? In addition, 

we provide information on scope and methodology changes in the PolicePublic Contact Surveys over time (see app. V). 

To address the first objective, regarding the extent to which BJS followed 

guidelines, we obtained quality guideline documents from BJS, the 

Department of Justice’s Office of Justice Programs, the Department of 

Justice, and the National Research Council. The guidelines that we 

obtained from these organizations covered the period between February 

2001, when the first product based on the 1999 Police-Public Contact 

Survey was issued, and June 2006, when the most recent product based on

the 2002 survey was issued. We included these federal organizations in our 

review because BJS is a component of the Office of Justice Programs, 

which in turn is a component of the Department of Justice, and BJS 

considers itself to be “governed by” the information quality guidelines of 

these organizations. We included the National Research Council in our 

review because it is a widely recognized organization that issued 

guidelines that were intended to be statements of best practice and 

provide information on what constitutes an effective statistical 

organization. We also reviewed guidance and directives issued by the 

Office of Management and Budget (OMB) because OMB is charged with 

issuing governmentwide policy and procedural guidance to federal 

agencies, which are then encouraged to issue their own implementation 

guidelines. 

We took several steps to determine the extent to which BJS followed the 

specific quality guidelines that it, the Office of Justice Programs, the 

Department of Justice, and the National Research Council had issued. 

From the documents provided by these four organizations, a GAO analyst 

initially identified a total of 63 guidelines that pertained to product review,

approval, and dissemination processes. For verification purposes, a GAO 

methodologist also reviewed the guideline documents. The GAO 

methodologist agreed with the auditor that all 63 guidelines were 

appropriate for inclusion in our review. Because many of the guidelines 

issued by the four organizations were similar and overlapping, the GAO 

auditor reduced the list to 24 nonduplicative guidelines. The GAO 

methodologist again reviewed the work of the auditor, and in all cases 

Page 28 GAO-07-340 Bureau of Justice Statistics
Appendix I: Objectives, Scope, and 

Methodology 

agreed with the auditor that similar guidelines were being appropriately 

grouped. 

We then developed a data collection instrument to determine whether BJS

was following guidelines for the 1999 and 2002 Police-Public Contact 

Surveys, on which information could be recorded as to whether BJS fully 

followed, partially followed, or did not at all follow each of the guidelines. 

We defined “fully” as all aspects of the guideline being followed; “partially”

as some, but not all, aspects of the guideline being followed; and “not at 

all” as no aspects of the guideline being followed. We asked BJS to 

complete a separate data collection instrument with respect to each of its 

1999 and 2002 Police-Public Contact Survey reports and one press release, 

and to support each response by providing documentary evidence. To 

decrease the burden on BJS, GAO analysts completed the data collection 

instrument for 9 of the 24 guidelines, for which we already had sufficient 

information (for example, documents describing agency processes and 

procedures, and interviews regarding the roles and responsibilities of 

noncareer appointees).1 We provided our assessments regarding these 

guidelines to BJS and asked officials to either confirm or not confirm 

them. Two GAO analysts reviewed BJS’s responses and all available 

supporting documentary and testimonial evidence, and determined 

whether BJS fully, partially, or did not at all meet each guideline. We 

provided our findings to BJS for review and comment. 

BJS’s 1996 and 2005 Police-Public Contact Surveys were outside the scope

of our work. We excluded the 1996 survey because that was a relatively 

small-scale pilot study; and we excluded the 2005 survey, the most recent 

Police-Public Contact Survey conducted, because no reports or press 

releases have yet been issued from this work. 

To address the second objective, regarding key factors that affected 

whether and how BJS followed guidelines, we reviewed processes and 

procedures that described the review, approval, and dissemination 

processes for BJS-generated reports and press releases, with particular 

 1

 The number of guidelines that were applicable to the review, approval, and dissemination 

of Police-Public Contact Surveys products varied across the time periods covered. In 

general, the number of guidelines increased between February 2001, when the first survey

report was issued, and June 2006, when the latest survey report was issued. However, one 

National Research Council guideline, pertaining to the dissemination of data and

information to as broad a public as reasonably possible, was dropped when the council 

issued the second edition of its guidelines between the time that BJS issued its first report

(in February 2001) and second report (in March 2002) from the 1999 survey. 

Page 29 GAO-07-340 Bureau of Justice Statistics
Appendix I: Objectives, Scope, and 

Methodology 

interest in identifying the roles of noncareer appointees involved in each 

of these processes. We also reviewed pertinent statutory provisions 

relating to the roles and responsibilities of officials with respect to BJS. 

We conducted in-person interviews with, or obtained written responses to 

our questions from, noncareer appointees in BJS, the Office of Justice 

Programs, and the Department of Justice’s Office of Public Affairs. 

Specifically, we conducted in-person interviews with the current BJS 

Director and the BJS Director who was involved in the disagreement with 

the Acting Assistant Attorney General, as well as with the current 

Assistant Attorney General and Deputy Assistant Attorney General in the 

Office of Justice Programs. We obtained detailed written responses to our 

questions from a former BJS Director, the Acting Assistant Attorney 

General who was involved in the disagreement with the BJS Director, and 

two former Assistant Attorneys General from the Office of Justice 

Programs. We conducted a telephone interview with the current Deputy 

Director of the Department of Justice’s Office of Public Affairs. Among 

other things, we asked these noncareer appointees to provide us with 

information about BJS’s process for reviewing, approving, and

disseminating reports and press releases; the roles and responsibilities of 

noncareer appointees in that process; changes, if any, that had occurred in 

the roles played by noncareer appointees; procedures used to help ensure 

that BJS reports and press releases were accurate, reliable, and unbiased; 

and any factors that may have affected BJS’s independence in the product 

issuance process. Finally, we reviewed the guidelines of BJS, the 

Department of Justice’s Office of Justice Programs, the Department of 

Justice, and the National Research Council to determine that they 

reflected the product issuance processes and to consolidate them in order 

to eliminate duplication.

To determine what changes, if any, have occurred in the scope and 

methodology of the Police-Public Contact Surveys between 1996 and 2006, 

which we present in appendix V, we initially developed a matrix of key 

scope and methodology dimensions, based on a review of the standard 

social science literature. We then conducted interviews and reviewed 

documents with respect to these dimensions, for all four Police-Public 

Contact Surveys—the 1996 pilot survey and the surveys of 1999, 2002, and 

2005. We interviewed the current BJS Director and a former BJS Director, 

and available report authors and the key statistician participating in 

administrations of the survey, to ascertain their views concerning the 

intended scope of the four surveys, the methodologies used, scope and

methodology changes that were made, and reasons for any changes. We 

also obtained written responses to our questions from these officials. We 

conducted a detailed documentary review of the scoping and methodology 

Page 30 GAO-07-340 Bureau of Justice Statistics
Appendix I: Objectives, Scope, and 

Methodology 

sections of the issued Police-Public Contact Survey reports and press 

releases, and extracted information about changes in the data collection 

instruments used (for example, the numbers and types of questions asked 

about searches and the use of force). In addition, we reviewed documents 

prepared by the American Statistical Association and the U.S. Bureau of 

the Census, which conducted field tests to ensure that Police-Public 

Contact Survey questions were appropriately devised. In cases where we 

noted that changes had been made between surveys, we reviewed Census 

Bureau documentation and interviewed staff and officials at BJS. 

It was beyond the scope of this review to address any personnel issues 

that may have arisen in connection with the disagreement over the content 

of the 2005 draft press release based on the 2002 Police-Public Contact 

Survey. We conducted our work between April 2006 and January 2007 in 

accordance with generally accepted government auditing standards. 

Page 31 GAO-07-340 Bureau of Justice Statistics
Appendix II: Applicable Quality Guidelines 

for Statistical Products Issued by the Bureau Appendix II: Applicable Quality Guidelines for of Justice Statistics and Other Agencies 

Statistical Products Issued by the Bureau of 

Justice Statistics and Other Agencies 

BJS followed numerous recommended data quality guidelines designed to 

help ensure the accuracy and integrity of the Police-Public Contact Survey 

products that it issued in 2001, 2002, 2005, and 2006 based on its 1999 and 

2002 surveys. The product issuance guidelines were used to aid BJS’s 

efforts to review, approve, and disseminate these statistical products to 

the public and others. 

The guidelines were issued at various points in time by the following 

organizations: 

• the National Research Council, 

• the Bureau of Justice Statistics, 

• the Department of Justice, and 

• the Office of Justice Programs. 

In addition to reviewing the guidelines of these four organizations, we also 

reviewed guidelines and directives issued by the Office of Management 

and Budget (OMB). However, we did not specifically assess BJS’s practice

with respect to following OMB’s guidelines because OMB issued 

governmentwide policy and procedural guidance to federal agencies that 

called for agencies to develop their own implementing guidelines. 

Table 4 shows the guidelines that were available at the time BJS’s 1999 and 

2002 Police-Public Contact Survey products were issued, and which 

guidelines BJS was in a position to follow. 

Page 32 GAO-07-340 Bureau of Justice Statistics
Appendix II: Applicable Quality Guidelines 

for Statistical Products Issued by the Bureau

of Justice Statistics and Other Agencies 

Table 4: Extent to Which BJS Followed Available Quality Guidelines for 1999 and 2002 Police-Public Survey Products 

Products issued from 1999 survey 

Products issued from 2002 

survey 

Products issued in 2001-2002 Products issued in 2005-2006 

Guideline

Issuing

entity Press releasea

Report AbReport BcReport CdReport De

A statistical agency should 

follow good practice, in reports

and other data releases, in 

documenting concepts, 

definitions, data collection 

methodologies, and measures

of uncertainty, and in

discussing possible sources of

error. 

NRC z z z z z

An effective dissemination 

program includes policies for 

the preservation of data that 

guide what data to retain and 

how they are to be archived 

for future secondary analysis. 

NRC z z z z z

An agency should have an 

established publications policy 

that describes, for a data 

collection program, the types 

of reports and other data 

released to be made 

available, the audience 

served, and the frequency of 

release. 

NRC z z z z z

Dissemination of data and 

information (basic series, 

analytic reports, press 

releases, public-use tapes) 

should be timely and public. 

Avenues of dissemination 

should be chosen to reach as 

broad a public as reasonably 

possible. 

NRC z z { { {

Statistical agency has 

dissemination policies that 

foster regular, frequent 

release of major findings from

an agency’s statistical 

programs to the public via the

media, the Internet, and other 

means. 

NRC { { z z z

Page 33 GAO-07-340 Bureau of Justice Statistics
Appendix II: Applicable Quality Guidelines 

for Statistical Products Issued by the Bureau

of Justice Statistics and Other Agencies 

Products issued from 1999 survey 

Products issued from 2002 

survey 

Products issued in 2001-2002 Products issued in 2005-2006 

Guideline

Issuing

entity Press releasea

Report AbReport BcReport CdReport De

The statistical agency has 

maintenance of a clear 

distinction between statistical

information and policy 

interpretations of such 

information by the President, 

the secretary of the 

department, or others in the 

executive branch. 

NRC z z z z z

The statistical agency has 

authority for professional 

decisions over the scope, 

content, and frequency of data 

compiled, analyzed, or 

published. Most statistical 

agencies have broad 

authority, limited by budgetary

constraints, departmental

requirements, OMB review, 

and congressional mandates. 

NRC S z z z z

The statistical agency has 

recognition by policy officials 

outside the statistical agency 

of its authority to release 

statistical information without 

prior clearance.

NRC  z z z z

Release of information should

not be subject to actual or 

perceived political 

interference. In particular, the 

timing of the public release of

data should be the 

responsibility of the statistical 

agency. 

NRC S z z z z

The objectivity of BJS 

statistics must be vigilantly 

protected at all times by BJS 

staff. 

BJS { { { z z

All BJS reports and other 

statistical products must be 

subject to an objective and 

appropriate verification 

process conducted by 

qualified BJS staff other than 

the author of the report. 

BJS { { { z z

Page 34 GAO-07-340 Bureau of Justice Statistics
Appendix II: Applicable Quality Guidelines 

for Statistical Products Issued by the Bureau

of Justice Statistics and Other Agencies 

Products issued from 1999 survey 

Products issued from 2002 

survey 

Products issued in 2001-2002 Products issued in 2005-2006 

Guideline

Issuing

entity Press releasea

Report AbReport BcReport CdReport De

BJS staff must archive all 

analysis and data sets as 

soon as they become final in 

order to ensure the 

reproducibility of the published

statistics. 

BJS { { { z z

BJS statistical products must 

provide sufficient information

concerning instances where 

data have been imputed. 

BJS { { { z z

Effective dissemination 

program includes a variety of 

avenues for data 

dissemination, chosen to 

reach as broad a public as 

reasonably possible. 

Channels of dissemination 

include, but are not limited to, 

an agency’s Internet Web site,

government depository 

libraries, conference exhibits 

and programs, newsletters 

and journals, e-mail address 

lists, and the media for regular

communication of major 

findings. 

BJS, 

NRC 

{ { z z z

A statistical agency should 

develop an understanding of 

the validity and accuracy of its 

data and convey the resulting

measures of quality to users in

ways that are comprehensible

to nonexperts. 

NRC, 

BJS 

z z z z z

An effective dissemination 

program includes release of 

data in a variety of formats so 

that the information can be 

accessed by users with 

varying skills and needs for 

data retrieval and analysis. 

BJS, 

NRC 

{ { z z z

A statistical agency should 

use modern statistical theory 

and sound statistical practice 

in all technical work. 

DOJ, 

NRC 

z z z z z

Page 35 GAO-07-340 Bureau of Justice Statistics
Appendix II: Applicable Quality Guidelines 

for Statistical Products Issued by the Bureau

of Justice Statistics and Other Agencies 

Products issued from 1999 survey 

Products issued from 2002 

survey 

Products issued in 2001-2002 Products issued in 2005-2006 

Guideline

Issuing

entity Press releasea

Report AbReport BcReport CdReport De

OJP/DOJ will allow adequate 

time for reviews, consistent 

with the level of standards 

required for the type of 

information to be 

disseminated. 

DOJ, 

OJP 

{ { { z z

OJP/DOJ will ensure 

compliance with relevant 

quality guidelines (i.e., utility, 

objectivity, and integrity 

requirements). 

DOJ, 

OJP 

{ { { z z

OJP/DOJ will provide 

methodologies, origins of 

data, limitations of the 

information, etc., whenever 

possible, as part of 

information dissemination. 

DOJ, 

OJP, BJS 

{ { { z z

OJP/DOJ components will

ensure disseminated 

information, as a matter of 

substance and presentation is 

accurate, reliable and 

unbiased. Objectivity is 

achieved by using reliable 

data sources, sound analytical

techniques, and documenting

methods and data sources. 

DOJ, 

OJP 

{ { { z z

OJP/DOJ components will

review all information 

dissemination products for 

their quality (including 

objectivity, utility, and integrity) 

before they are disseminated.

DOJ, 

OJP 

{ { { z z

OJP/DOJ components will

assess the usefulness of the 

information to be 

disseminated to the public. 

DOJ, 

OJP 

{ { { z z

OJP/DOJ will ensure that the 

information fulfills the 

intentions stated and that the 

conclusions are consistent 

with the evidence. 

DOJ, 

OJP 

{ { { z z

Source: GAO analysis of DOJ, OJP, and BJS data.

Page 36 GAO-07-340 Bureau of Justice Statistics
Appendix II: Applicable Quality Guidelines 

for Statistical Products Issued by the Bureau

of Justice Statistics and Other Agencies 

Legend: 

zGuideline was available and BJS followed it fully. 

SBJS was not in the position to follow this guideline fully. 

BJS was not in the position to follow this guideline at all. 

{Guideline was unavailable during the product issuance stage. 

NRC National Research Council

DOJ Department of Justice 

OJP Office of Justice Programs 

a

Force or Threatened Force Used in Less Than 1 Percent of All Police-Public Interactions (March 11, 

2001). 

b

Contacts between Police and the Public, Findings from the 1999 National Survey (February 2001). 

c

Characteristics of Drivers Stopped by Police, 1999 (March 2002). 

d

Contacts between Police and the Public, Findings from the 2002 National Survey (April 2005). 

e

Characteristics of Drivers Stopped by Police, 2002 (June 2006). 

Page 37 GAO-07-340 Bureau of Justice Statistics
Appendix III: Press Release Issued in 2001 

Based on BJS’s 1999 Police-Public Contact Appendix III: Press Release Issued in 2001 Survey

Based on BJS’s 1999 Police-Public Contact 

Survey 

Page 38 GAO-07-340 Bureau of Justice Statistics
Appendix III: Press Release Issued in 2001 

Based on BJS’s 1999 Police-Public Contact 

Survey

Page 39 GAO-07-340 Bureau of Justice Statistics
Appendix IV: B

Approval, a

Contact Sur

JS Processes for the Review, 

nd Dissemination of Police-Public

vey Products 

Page 40 GAO-07-340

Appendix IV: BJS Processes for the Review, 

Approval, and Dissemination of Police-Public 

Contact Survey Products 

Since the inception of the Police-Public Contact Survey in 1996, the BJS 

Director has been the single noncareer appointee who has had a decisionmaking role in BJS’s review, approval, and dissemination processes for 

reports. The BJS Director is a noncareer presidential appointee subject to 

Senate confirmation. Figure 2 provides an overview of the process 

followed by BJS in the review, approval, and dissemination of PolicePublic Contact Survey reports. 

Review, Approval, and 

Dissemination Process for 

Reports Issued from 

Police-Public Contact 

Surveys 

Figure 2: Review, Approval, and Dissemination Process for BJS Survey Reports 

BJS

Author/Supervisor

BJS

Chief of Publication 

and 

Dissemination Unit

BJS

 Director

BJS Director posts 

report to web site

Memorandum of notification signed by 

OJP Assistant Attorney General

DOJ

Deputy Attorney 

General

DOJ

Associate 

Attorney 

General

DOJ

Attorney 

General

Memorandum sent to 

Source: BJS.

Memorandum of notification 

to issue report sent by 

BJS director

Denotes career employee

Denotes noncareer appointees

BJS Bureau of Justice Statistics

OJP Office of Justice Programs

DOJ Department of Justice 

As indicated by the figure, the BJS report author and supervisor prepare 

the draft report for review and approval. The BJS Director reviews the 

draft, requests any changes, approves the final draft, and transmits a 

memorandum of notification through the Office of Justice Program’s 

Assistant Attorney General up the chain of command to the Attorney 

General. The memorandum contains an abstract of the report, selected

survey findings, and a projected release date for the report. BJS sets the 

release date for 30 days from the date that the Assistant Attorney General 

signs the memorandum of notification. The report is posted to the Web 

site at that time, or sooner if the date and time are specified in the 

notification memo. 

Bureau of Justice Statistics
Appendix IV: BJS Processes for the Review, 

Approval, and Dissemination of Police-Public

Contact Survey Products 

In contrast to the process followed for survey reports, several noncareer 

appointees are involved in the organizational review, approval, and 

dissemination process, as shown in figure 3. 

Review, Approval, and 

Dissemination Process for 

Press Releases Issued from 

Police-Public Contact 

Surveys 

Figure 3: Review, Approval, and Dissemination Process for BJS Survey Press Releases 

Review/approve

BJS

Author/Supervisor

BJS

Director

OJP

Office of Communications

(OCOM ) Staff

DOJ

Director, Office of

Public Affairs

OJP

OCOM Director 

Jointly 

prepare press

release

OJP

Chief of Staff

OJP

Deputy Assistant

Attorney General

OJP

Assistant

Attorney General

Disseminate to

 Congress, the media

and executive 

department press

offices

Post to web site

OJP

OCOM Director Of 

Public Affairs

Director

forwards to 

other reviewers

Review/approve:

Verify accuracy

Sources: BJS and Office of Justice Programs. 

Denotes career employee

Denotes noncareer appointees

BJS Bureau of Justice Statistics

OJP Office of Justice Programs

DOJ Department of Justice 

Page 41 GAO-07-340 Bureau of Justice Statistics
Appendix IV: BJS Processes for the Review, 

Approval, and Dissemination of Police-Public

Contact Survey Products 

As indicated in the figure, in addition to the Director of BJS, there are 

three noncareer appointees within the Office of Justice Programs who 

participate in the review and approval process—the Chief of Staff, the 

Deputy Assistant Attorney General, and the Assistant Attorney General,

and at least one noncareer appointee within Department of Justice 

headquarters: the Director of the Office of Public Affairs. 

The BJS report author and supervisor jointly work with staff from the 

Office of Justice Program’s Office of Communications to prepare the press 

release. The BJS director reviews the draft press release, requests any 

changes, approves the final draft, and transmits the press release up the 

chain of command to the Office of Justice Programs’ Assistant Attorney 

General for review and approval. Following approval by the Assistant 

Attorney General, the Department of Justice’s Office of Public Affairs 

reviews the press release for clarity, and the BJS Director then verifies 

that the information in the press release is accurate. The Department of 

Justice’s Office of Public Affairs is then responsible for disseminating the 

press release to Congress, the media, and executive department press 

offices, while BJS is responsible for disseminating the press release 

through its Web site. 

Page 42 GAO-07-340 Bureau of Justice Statistics
Appendix V: Ch

Methodology of

Surve

anges in the Scope and 

 the Police-Public Contact 

ys 

Page 43 GAO-07-340

Appendix V: Changes in the Scope and 

Methodology of the Police-Public Contact 

Surveys 

BJS has conducted four Police-Public Contact Surveys as supplements to 

the National Crime Victimization Survey. The first Police-Public Contact 

Survey was conducted as a pilot in 1996. Three subsequent, more 

extensive surveys were conducted at 3-year intervals: 1999, 2002, and 2005. 

Although we do not discuss the 2005 survey in this report because no 

reports or press releases have yet been issued from this survey, we present 

information on the 2005 survey in this appendix because information is 

available on this survey’s scope and methodology. The scope of the PolicePublic Contact Surveys has consistently expanded over time, while the 

methodology has remained generally consistent. 

The Scope of the PolicePublic Contact Survey Has 

Expanded with Each 

Additional Survey 

The pilot survey was designed to test whether the survey could be 

effectively used as a supplement to the National Crime Victimization 

Survey to collect data on (a) the types of contacts the public have with the 

police, and (b) police use of force. To conduct this test, BJS employed a 

representative sample of 6,421 U.S. residents. The pilot survey yielded 

useful information on the various types of contacts the public had with the 

police, and whether force was used by the police. However, the sample 

size of the pilot survey was not sufficiently large for BJS to draw 

inferences about the extent to which the population at large would report 

that they experienced “excessive” use of force by police. 

For its Police-Public Contact Survey in 1999, BJS increased the sample size 

to a representative sample of 80,543 U.S. residents. The scope of the 

survey was further enhanced by adding questions about traffic stops (the 

most common form of public-police contact, as determined in the 1996 

pilot survey), and including a question on whether the police used 

excessive force during any contact with the public. BJS officials told us 

that they added the traffic stop questions, at least in part, to “address the 

growing public concern about racial profiling in connection with traffic 

stops.” 

In its 2002 survey, BJS expanded and refined its survey questions further. 

Specifically, according to BJS officials, they added questions that would 

help BJS estimate the extent to which U.S. residents nationwide would say 

that (1) they were stopped by the police while driving, (2) they or their 

vehicle were searched by the police without their permission during a 

traffic stop, and (3) they were arrested as a result of the search. In 

addition, BJS officials said that they added questions to estimate 

differences, if any, among racial groups in their rates of traffic stops at 

various times of the day, and whether police used force in situations 

Bureau of Justice Statistics
Appendix V: Changes in the Scope and 

Methodology of the Police-Public Contact 

Surveys 

where persons were engaged in such behaviors as arguing with, cursing, or 

disobeying police. 

In 2005, according to BJS officials, the scope of the Police-Public Contact 

Survey was further extended in several ways, including the following: (a) 

respondents were asked whether they had been arrested for driving under 

the influence of alcohol during the year (in order to make comparisons 

with Federal Bureau of Investigation (FBI) arrest rates, so that potential 

undercounting rates could be determined); (b) respondents were 

permitted to group themselves into any combination of racial categories 

(rather than choosing a single category) to better refine respondent 

demographic status; (c) respondents were asked whether police used 

force during any of their police contacts during the year, as opposed to the

more limiting question in 2002, which was directed only toward the most 

recent contact with police; and (d) respondents were provided open-ended

response fields on the survey instrument to indicate any ways they 

believed that the police had acted inappropriately toward them. 

The Police-Public Contact 

Survey Methodology Has 

Remained Consistent 

Following the 1996 Survey 

The 1999, 2002, and 2005 Police-Public Contact Surveys have consistently 

maintained a similar methodology. The methodological dimensions of the 

surveys that have remained consistent are geographic coverage, target 

population, sampling design, data collection method, sample size/response 

rate, survey administration, and sample characteristics (as indicated in

table 5). To illustrate, all three surveys involved selecting nationally 

representative, stratified, multi-cluster samples from the population of U.S. 

residents 16 years of age or older.1

 The data were collected either through 

face-to-face or computer-assisted telephone interviews. The surveys were 

administered during the last 6 months of the year, and the demographic 

characteristics of the samples were similar across time periods. 

The 1996 pilot survey differed in several ways from the subsequent three 

surveys. Specifically, the pilot survey included residents younger than 16, 

included far fewer people than the subsequent surveys, and limited the 

sampling to individuals who had participated in the last round of the 

National Crime Victimization Survey. In addition, the percentage of faceto-face interviews was lower, and the survey administration period was 

 1

 This included persons living in group quarters, but excluded institutionalized persons and 

some members of the armed services. 

Page 44 GAO-07-340 Bureau of Justice Statistics
Appendix V: Changes in the Scope and 

Methodology of the Police-Public Contact 

Surveys 

shorter, and during a different time of the year, than in the other three 

surveys. 

Table 5: Police-Public Contact Survey (PPCS) Methodologies for the Four Surveys: 1996, 1999, 2002, and 2005 

Dimensions 1996 surveya1999 survey 2002 survey 2005 survey 

Geographic Coverage National survey National survey National survey National survey

Target populations Included: residents ages 

>12, living in the United

States, including persons 

living in group quarters 

(e.g., dormitories, rooming 

houses, religious group 

dwellings). 

Excluded: crew members 

of merchant vessels, 

armed forces personnel 

living in military barracks, 

and institutionalized 

persons (e.g., correctional 

facility inmates) 

Same as 1996, except 

the target population age 

range is >16 

Same as 1996, except 

the target population age 

range is >16 

 Same as 1996, except 

the target population age 

range is >16 

Sampling design Nationally representative, 

stratified, multi-stage 

cluster sample, limited to 

respondents participating 

in the outgoing rotation 

panels of the National 

Crime Victimization 

Survey (i.e., the seventh 

and last round of 

interviews) 

Nationally representative, 

stratified, multi-stage 

cluster sample, not 

limited to the out-rotation 

panels of the National 

Crime Victimization 

Survey 

Nationally representative, 

stratified, multi-stage 

cluster sample, not 

limited to the out-rotation 

panels of the National 

Crime Victimization 

Survey 

Nationally representative, 

stratified, multi-stage 

cluster sample, not 

limited to the out-rotation 

panels of the National 

Crime Victimization 

Survey 

Data collection method All survey interviews were

collected using either the 

CATI (computer-assisted 

telephone interview) or 

PAPI (paper-and-pencil

interview either in person 

or by telephone) 

Interviews: 

In person: 19%

Telephone: 81% 

All PPCS interviews were

collected using either the 

CATI or PAPI 

Interviews: 

In person: 31%

Telephone: 69% 

All PPCS interviews were

collected using either the 

CATI or PAPI

Interviews: 

In-person: 34%

Telephone: 66% 

All PPCS interviews were

collected using either the 

CATI or PAPI 

Interviews: 

In-person: 37%

Telephone: 63% 

Sample size/response 

rate 

6,421 (99%) 80,543 (85%) 76,910 (82%) 63,943 (80%) 

Survey administration 

period 

5/1/96 to 7/31/96 7/1/99 to 12/31/99 7/1/02 to 12/31/02 7/1/05 to 12/31/05 

Page 45 GAO-07-340 Bureau of Justice Statistics
Appendix V: Changes in the Scope and 

Methodology of the Police-Public Contact 

Surveys 

Dimensions 1996 surveya1999 survey 2002 survey 2005 survey 

Sample characteristics 

(unweighted percent) 

Gender: 

Male: 50.7 

Female: 49.3 

Race/ethnicity:

White: 83.0 

Black: 7.4 

Hispanic: 5.7 

Other: 3.9 

Age: 

16-19: 11.1 

20-39: 44.5 

40-59: 32.8 

> 60: 11.6 

Gender: 

Male: 50.1 

Female: 49.9 

Race/ethnicity:

White: 79.3 

Black: 9.1 

Hispanic: 8.4 

Other: 3.2 

Age: 

16-19: 8.1 

20-39: 45.6 

40-59: 35.3 

> 60: 11.1 

Gender: 

Male: 50.2 

Female: 49.8 

Race/ethnicity:

White: 77.5 

Black: 9.5 

Hispanic: 9.8 

Other: 3.2 

Age: 

16-19: 7.5 

20-39: 44.6 

40-59: 36.3 

> 60: 11.7 

Gender: 

Male: 50.4 

Female: 49.6 

Race/ethnicity:

White: 76.5 

Black: 9.1 

Hispanic: 9.6 

Other: 3.9 

 

Age: 

16-19: 7.6 

20-39: 41.3 

40-59: 38.5 

> 60: 12.6 

Source: BJS data. 

Note: Percentages may not sum to 100 percent because of rounding. 

a

Pilot survey. 

Page 46 GAO-07-340 Bureau of Justice Statistics
Appendix VI: Comments from the Department Appendix VI: Comments from the of Justice 

Department of Justice 

Page 47 GAO-07-340 Bureau of Justice Statistics
Appendix VI: Comments from the Department 

of Justice 

Page 48 GAO-07-340 Bureau of Justice Statistics
Appendix VI: Comments from the Department 

of Justice 

Page 49 GAO-07-340 Bureau of Justice Statistics
Appendix VI: Comments from the Department 

of Justice 

Page 50 GAO-07-340 Bureau of Justice Statistics
Appendix VI: Comments from the Department 

of Justice 

Page 51 GAO-07-340 Bureau of Justice Statistics
Appendix VII: 

A

GAO Contact and 

cknowledgments 

Page 52 GAO-07-340

Appendix VII: GAO Contact and 

Acknowledgments 

Brian J. Lepore (202) 512-4523 

In addition to the above, Evi. L. Rezmovic, Assistant Director; Ronald S. 

Fecso, Chief Statistician; Jared A. Hermalin; Karen A. Jarzynka; Amanda K. 

Miller; Amy L. Bernstein; Geoffrey R. Hamilton; Robert Alarapon; and 

Tracy J. Harris made key contributions to this report. 

Bureau of Justice Statistics

GAO Contacts 

Acknowledgments 

(440500) 
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER