a

GAO

United States Government Accountability Office

Report to Congressional Committees

September 2005 INFORMATION 

QUALITY ACT

National Agricultural 

Statistics Service 

Implements First 

Steps, but 

Documentation of 

Census of Agriculture 

Could Be Improved

GAO-05-644
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-05-644.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Orice Williams 

at (202) 512-6806 or williamso@gao.gov. 

Highlights of GAO-05-644, a report to 

congressional committees 

September 2005

INFORMATION QUALITY ACT 

National Agricultural Statistics Service 

Implements First Steps, but 

Documentation of Census of Agriculture 

Could Be Improved 

NASS fulfilled its various procedural responsibilities and reporting 

requirements under the Office of Management and Budget’s (OMB) 

guidelines for implementing the act. For example, NASS drafted its own 

implementation guidance, and developed a mechanism allowing affected 

parties to request the correction of information they believe is of poor 

quality. As a result of our review, NASS has also taken steps to better 

document the criteria it uses to evaluate data users’ input on the content of 

the Census of Agriculture. 

The Census of Agriculture Provides a Detailed Picture of U.S. Farms and Ranches 

Building on these efforts, better documentation could improve the 

transparency of census data products. For example, the nine key products 

from the 2002 Census we examined lacked, among other things, discussions 

of any data limitations. This is contrary to NASS’s own guidelines for 

ensuring transparency, which stress the importance of describing the 

methods, data sources, and other items to help users understand how the 

information was designed and produced. 

Although NASS complied with OMB’s requirement to establish a mechanism 

under IQA to address requests to correct information, NASS has not 

documented its approach for handling correction requests not filed under 

IQA (NASS handles these correction requests using an existing, informal 

method). Agency officials told us that data users have been satisfied with 

the way NASS had responded to these requests. However, because NASS 

does not document its informal procedures for handling correction requests 

and lacks a recordkeeping system to log and track them, NASS could not 

provide us with specific data on the number of such requests it has handled, 

the nature of those requests, and whether and how they were addressed. 

The Information Quality Act (IQA) 

required the Office of Management 

and Budget to issue guidelines for 

ensuring the quality, objectivity, 

utility, and integrity of information 

disseminated by federal agencies. 

As part of our long-term 

examination of the quality of 

federal information, under the 

Comptroller General’s authority, 

we reviewed how the act was 

implemented by the National 

Agricultural Statistics Service 

(NASS), and assessed the 

transparency of the documentation 

supporting its Census of 

Agriculture. NASS is part of the 

U.S. Department of Agriculture 

(USDA). 

What GAO Recommends

To help enhance the transparency 

of the Census of Agriculture, we 

recommend that the Secretary of 

Agriculture direct NASS to 

(1) ensure that census products 

fully address NASS’s guidelines for 

data documentation or at least 

contain links to such information, 

and (2) document and post on 

NASS’s Web site its procedures for 

handling data correction requests 

not filed under IQA. NASS agreed 

with our findings and described the 

steps it is taking in response to our 

recommendations. Additional 

actions, consistent with our 

recommendations, would enhance 

NASS’s efforts. 
Page i GAO-05-644 Information Quality Act

Contents

Letter 1

Results in Brief 3

Background 5

Objectives, Scope, and Methodology 7

NASS Met the Procedural and Reporting Requirements of OMB’s 

IQA Guidelines 9

Better Documentation Could Improve the Transparency of Data 

Products and Correction Procedures 10

Conclusions 18

Recommendations for Executive Action 19

Agency Comments and Our Evaluation 19

Appendixes

Appendix I: Comments from the Department of Agriculture 21

Appendix II: How Census of Agriculture Reports Address Various 

Documentation Elements 23

Appendix III: GAO Contact and Staff Acknowledgments 25

Tables Table 1: NASS Addressed OMB’s Agencywide Guidelines for 

Implementing IQA 9

Table 2: Census Reports Need More Robust Documentation 11

Table 3: NASS Is Using More Extensive Outreach to Develop the 

2007 Census Compared to 2002 16

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately.
Page 1 GAO-05-644 Information Quality Act

United States Government Accountability Office

Washington, D.C. 20548

A

September 23, 2005 Letr

The Honorable Saxby Chambliss

Chairman

The Honorable Tom Harkin

Ranking Democratic Member

Committee on Agriculture, Nutrition and Forestry

United States Senate

The Honorable Collin C. Peterson

Ranking Democratic Member

Committee on Agriculture

House of Representatives

The information disseminated by federal agencies is a critical strategic 

asset. For example, data collected for statistical purposes provide 

indicators of the economic and social well-being of the nation, while health, 

safety, environmental, and other scientific data help inform agencies’ rulemaking activities. Given the widespread use and impact of federal 

information, it is important for it to meet basic quality standards. 

Section 515 of the Treasury and General Government Appropriations Act 

for Fiscal Year 2001—legislation that has come to be known as the 

Information Quality Act (IQA)1

—required the Office of Management and 

Budget (OMB) to issue governmentwide guidelines that provide policy and 

procedural guidance to federal agencies for ensuring and maximizing the 

quality, objectivity, utility, and integrity of information disseminated by 

federal agencies. 

OMB’s guidelines, issued in final form in February 2002, directed agencies 

covered by the act to issue their own quality guidelines, and noted that, 

where appropriate, agencies should support their data with transparent 

documentation. OMB’s guidelines also required agencies to, among other 

actions, report annually to the Director of OMB on the number and nature 

of complaints received regarding compliance with the guidelines, and 

establish an administrative mechanism whereby affected parties can 

request the correction of information they deem to be of poor quality.

1

Consolidated Appropriations – Fiscal Year 2001, Pub. L. No. 106-544, § 515, 114 Stat. 2763A153 to 2763A-154 (2000) (44 U.S.C. § 3516 note).
Page 2 GAO-05-644 Information Quality Act

As part of our long-term examination of the collection, dissemination, and 

quality of federal information, we are reviewing the governmentwide 

implementation of the IQA. As an initial step in our research on the quality 

of statistical data, under the Comptroller General’s authority, we conducted 

a case study of the National Agricultural Statistics Service (NASS), and its 

Census of Agriculture. NASS is a statistical agency within the U.S. 

Department of Agriculture (USDA). We selected the census because it is 

one of the largest government surveys with a universe of 2.3 million 

respondents and an estimated paperwork burden in excess of 1.3 million 

burden hours.2

 The last census took place in 2002 and the next census is 

scheduled for 2007.

Specifically, our objectives were to (1) review how NASS met OMB’s 

guidelines covering the IQA, and (2) examine the transparency of the 

documentation behind the Census of Agriculture’s processes and products, 

both for the recently completed work on the 2002 Census and the efforts 

underway for the 2007 Census. To achieve both objectives, we reviewed 

OMB’s and NASS’s information quality guidelines and other relevant 

documents. We also interviewed senior agency officials and other 

personnel responsible for implementing the census including the NASS 

Administrator, Associate Administrator, and Deputy Administrator for 

Programs and Products.

To evaluate the transparency of census products, we reviewed nine census 

products--eight reports and the Frequently Asked Questions (FAQ) section 

on NASS’s 2002 Census Web site--to determine the extent to which NASS 

followed its own documentation guidelines. To obtain an external 

perspective of how NASS processes and products address the IQA 

guidelines, we interviewed six data users from different types of 

agricultural and research organizations. We selected these six because they 

use census data on a regular basis and have attended NASS’s outreach 

meetings. Additional information on our approach is provided in the 

Objectives, Scope, and Methodology section below. 

We performed our work in Washington, D.C., from August 2004 through 

August 2005 in accordance with generally accepted government auditing 

standards. 

2

Under the Paperwork Reduction Act, agencies must estimate the burdens their data 

collections impose on the public.
Page 3 GAO-05-644 Information Quality Act

Results in Brief NASS fulfilled its various procedural responsibilities and reporting 

requirements under OMB’s guidelines. For example, NASS (1) drafted its 

own implementation guidelines and posted them on its Web site, 

(2) developed an administrative mechanism allowing affected persons to 

seek the correction of information, (3) designated an official responsible 

for ensuring NASS’s compliance with OMB’s requirements, and (4) reported 

to OMB the number of complaints received regarding data quality they 

received under the IQA.3

With respect to the transparency of the documentation underlying the 

Census of Agriculture’s data products and processes, as a result of our 

review, NASS has taken steps to better document the criteria it uses to 

evaluate data users’ suggestions on the questionnaire content. The 

development of the 2002 and 2007 Censuses was led by the Census Content 

Team, which consisted of experienced NASS statisticians. The 2002 Team 

assessed users’ input using a documented set of criteria, which considered 

such factors as whether the questionnaire items were mandated by 

Congress or whether they would provide data on current agricultural 

issues. However, because of staff turnover and reassignments, the 2007 

Team was unaware of the 2002 criteria, and initially relied on professional 

judgment rather than documented factors to evaluate input on the 2007 

Census. According to NASS, our review raised the 2007 Team’s awareness 

of the earlier criteria, and it has since developed similar documentation 

that it will use in the future. This approach is more consistent with NASS’s 

own IQA guidelines concerning transparency, and could help create a 

closer link between the questions included in the census and evolving 

agricultural policy requirements, and thus a more cost-effective data 

collection program. Documenting the content selection criteria will also 

guard against the loss of institutional memory to the extent there is further 

turnover in Content Team membership. 

Building on these efforts, better documentation could improve the 

transparency of census data products. Although NASS’s guidelines for 

ensuring transparency stress the importance of describing the methods, 

data sources, assumptions, and other items in order to help users 

understand how information was designed and produced, the eight census 

reports as well as the FAQ section we examined lacked a discussion of 

such important documentation practices as the limitations of the data; the 

3

NASS officials reported that they have not received any complaints under the IQA.
Page 4 GAO-05-644 Information Quality Act

impact of imputations, by item; and whether any of the collected data have 

been suppressed for data quality reasons. 

The transparency of NASS’s procedures for handling data correction 

requests filed outside of the IQA could also be improved. NASS complied 

with OMB’s requirement to establish a mechanism to process requests for 

correction of disseminated information under the IQA. As part of this 

process, an individual must state that their request is being submitted 

under the IQA. To date, no individual has done so. NASS handles all other 

correction requests using its existing informal, undocumented procedures. 

Agency officials told us that it has resolved the informal requests it has 

handled so far to the data users’ satisfaction. Nevertheless, because NASS 

does not document its informal procedures for handling correction 

requests and lacks a recordkeeping system to log and track them, NASS 

could not provide us with specific data on the number of requests it has 

handled, the nature of those requests, and whether and how they were 

addressed. 

To help enhance the transparency of the Census of Agriculture’s processes 

and products, we recommend that the Secretary of Agriculture direct NASS 

to (1) ensure its products fully address its own requirements for 

transparent data documentation or at least contain links to such 

information and (2) document and post on its Web site its procedures for 

handling data correction requests not filed under the IQA and track the 

disposition of those requests. 

The NASS Administrator provided written comments on a draft of this 

report (see app. I). NASS said the information was insightful, and noted it 

will be used to strengthen the transparency of its methods and procedures. 

In particular, NASS agreed with our findings and, consistent with one of our 

two recommendations, said it will take steps to better document its 

specialized reports. NASS also said it plans to make a list of “common 

issues” raised by data users available on its Web site, which is in line with 

our second recommendation to improve the transparency of its procedures 

for handling data correction requests not filed under the IQA. NASS’s 

commitment to continually improve its products is commendable, and its 

efforts to improve the transparency of its processes and products would be 

further enhanced if, consistent with our recommendations, it (1) ensures 

that all of its census products fully address NASS’s own guidelines for data 

documentation and (2) posts on its Web site its procedures for handling 

correction requests not filed under the IQA. 
Page 5 GAO-05-644 Information Quality Act

Background The IQA directed OMB to issue guidelines to federal agencies covered by 

the Paperwork Reduction Act designed to ensure the “quality, objectivity, 

utility, and integrity” of information disseminated to the public.4

 The IQA 

also directed OMB to include in its guidelines requirements for agencies to 

(1) develop their own information quality guidelines, (2) establish 

administrative mechanisms for affected persons to seek correction of 

information that does not comply with OMB’s guidelines, and (3) annually 

report to OMB the number and nature of complaints they receive regarding 

the accuracy of the information they disseminate.5

Prior to the IQA, there were several governmentwide actions aimed at 

improving agency data. For example, Statistical Policy Directive No. 2, first 

issued in 1952, required statistical agencies to inform users of conceptual 

or other limitations of the data, including how the data compare with 

similar statistics. In 1996, the Federal Committee on Statistical 

Methodology—an OMB-sponsored interagency committee dedicated to 

improving the quality of federal statistics—established a subcommittee to 

review the measurement and reporting of data quality in federal data 

collection programs. The results of the subcommittee’s work were 

published in a 2001 report that addressed such issues as what information 

on sources of error federal data collection programs should provide, and 

how they should provide it.6

 For all federal government information 

collections, the 1995 amendments to the Paperwork Reduction Act called 

on federal agencies to manage information resources with the goal of 

improving “the integrity, quality, and utility of information to all users 

within and outside the agency.”7

4

Agencies covered by the IQA include all agencies subject to the Paperwork Reduction Act 

(PRA) – cabinet departments, independent regulatory agencies (e.g., Federal 

Communications Commission), and other independent agencies (e.g., the Environmental 

Protection Agency). 44 U.S.C. § 3502(1).

5

Discussion of the IQA often centers on its impact on agencies’ regulatory activities. 

Supporters of IQA, many of whom represent businesses and other regulated entities, 

maintain that IQA could enhance the quality of agency science and improve the rule-making 

process. Critics of IQA, including some environmental and public interest groups, view the 

law as a device to curtail health, safety, and other regulations.

6

OMB, Measuring and Reporting Sources of Error in Surveys, Statistical Policy Working 

Paper 31, July 2001.

7

44 U.S.C. § 3506(b)(1)(C).
Page 6 GAO-05-644 Information Quality Act

OMB’s IQA guidelines were issued in final form in February 2002.8 They 

required agencies subject to the IQA to take such steps as

• issue information quality guidelines designed to ensure the quality, 

objectivity, utility, and integrity of information disseminated to the 

public; 

• establish administrative mechanisms for affected persons to seek 

correction of information they believe is not in compliance with the 

guidelines; 

• report annually to the Director of OMB on the number and nature of 

complaints received regarding compliance with the guidelines and how 

the agencies handled those complaints; and 

• designate an official responsible for ensuring compliance with OMB’s 

guidelines.

The OMB guidelines defined quality as an encompassing term comprising 

• utility, which is the usefulness of the information to its intended users; 

• integrity, which refers to the security of information and its protection 

from unauthorized access or revision; and

• objectivity, which addresses both presentation (i.e., whether the 

information is being presented in an accurate, clear, complete, and 

unbiased manner) and substance (i.e., whether the information is 

accurate, reliable, and unbiased).

In addition, OMB addresses transparency within the definition of 

objectivity and utility. As recognized in OMB’s guidelines, agencies that 

disseminate influential scientific, financial, or statistical information must 

demonstrate a high degree of transparency about data and methods. These 

measures are in place to facilitate the information’s reproducibility by an 

outside party or reanalysis of an agency’s results.

The National Research Council of the National Academies considers 

transparency a key principle for federal statistical agencies, and stated in a 

8

67 Fed. Reg. 8452 (Feb. 22, 2002).
Page 7 GAO-05-644 Information Quality Act

recent report that transparency, which it defines as “an openness about the 

sources and limitations of the data,” is particularly important for instilling 

credibility and trust among data users and providers. 9

As an agency within USDA, NASS is required to comply with the IQA. One 

statistical program administered by NASS is the quinquennial Census of 

Agriculture. According to NASS, the census provides a detailed picture of 

U.S. farms and ranches every 5 years and is the only source of uniform, 

comprehensive agricultural data at the county level. The results are 

published in 18 reports divided among three categories: Geographic Area 

Series, Census Quick Stats, and Specialty Products and Special Studies. 

Users of this information include federal agencies (for program and 

statistical purposes), farm organizations, businesses, universities, state 

departments of agriculture, elected representatives, legislative bodies at all 

levels of government, and academia. The next Census of Agriculture is 

scheduled for 2007. 

Objectives, Scope, and 

Methodology

Our objectives were to (1) review how NASS met OMB’s guidelines 

covering the IQA and (2) examine the transparency of the documentation 

behind the Census of Agriculture’s processes and products, including the 

recently completed work on the 2002 Census, and the efforts currently 

underway for the 2007 Census. 

To achieve both of these objectives, we reviewed OMB’s and NASS’s 

information quality guidelines, Census of Agriculture reports,10

submissions to OMB, and other relevant documents. We also interviewed 

NASS officials about how NASS conducted the 2002 Census and how it is 

planning for the 2007 Census. The officials included the NASS 

Administrator, Associate Administrator, and Deputy Administrator for 

Programs and Products.

In addition, to evaluate the transparency of Census of Agriculture products, 

we reviewed eight census reports and the Frequently Asked Questions area 

of the 2002 Census Web site, to determine the extent to which NASS 

9

Margaret E. Martin, Miron L. Straf, and Constance F. Citro eds., Principles and Practices 

for a Federal Statistical Agency, (3rd ed.) (Washington, D.C.: National Academies Press, 

2005), pp. 8, 29.

10Reports can be obtained from USDA’s Web site; see www.usda.gov/nass.
Page 8 GAO-05-644 Information Quality Act

followed its own procedures for ensuring the transparency of its 

information products. NASS’s IQA guidelines define transparency as, “a 

clear description of the methods, data sources, assumptions, outcomes, 

and related information that allows a data user to understand how an 

information product was designed and produced.”

NASS’s guidelines state that its survey activities include such activities as 

sample design, questionnaire design, pre-testing, analysis of sampling, and 

imputation of missing data. However, the guidelines were not clear as to 

the specific activities to be documented. Consequently, we reviewed the 

practices employed by such statistical agencies as the National Academies 

of Sciences, International Monetary Fund, and U.S. Census Bureau, and 

developed a set of 20 practices associated with transparent documentation 

that encompassed the items NASS laid out in its own guidelines. The 

practices include such actions as defining data items, discussing sample 

design, and describing how the content of the survey differs from past 

iterations (see app. II). 

We looked for the presence or absence of these practices in 9 out of the 18 

census reports and related forms of data that NASS disseminates, and 

verified the results with a second, independent analysis. In instances where 

a report did not include a particular documentation practice, we reviewed 

whether the report instead informed data users where to obtain this 

information. We chose these 9 reports because they all stem from the 

original census data collection, represent different product categories, and 

were available on the census Web site as of February 1, 2005. 

To obtain an external perspective of how NASS processes and products 

address the IQA guidelines, we interviewed six data users from different 

types of agricultural and research organizations. We selected these data 

users from lists of registrants for USDA and NASS outreach meetings 

within the past 5 years. We selected these six data users because they use 

information from the census on a regular basis. Moreover, these data users 

attended the most recent NASS outreach meeting, which specifically 

addressed the 2002 and 2007 Censuses. Some data users had also provided 

NASS with feedback on the content of the agricultural census. Their views 

cannot be projected to the larger population of census data users. 

We requested comments on a draft of this report from the Secretary of 

Agriculture. On September 8, 2005, we received the NASS Administrator’s 

written comments and have reprinted them in appendix I. They are 
Page 9 GAO-05-644 Information Quality Act

addressed in the Agency Comments and Our Evaluation section of this 

report.

NASS Met the 

Procedural and 

Reporting 

Requirements of OMB’s 

IQA Guidelines

NASS fulfilled the various procedural responsibilities and reporting 

requirements under OMB’s guidelines. For example, NASS released its own 

IQA guidelines for public comment on March 27, 2002. NASS officials 

stated they received no substantive comments on them and OMB approved 

the guidelines with only minimal changes. The officials also noted that no 

revisions have been made since then. Table 1 shows in greater detail how 

NASS addressed OMB’s guidelines.

Table 1: NASS Addressed OMB’s Agencywide Guidelines for Implementing IQA

Source: GAO analysis of OMB and NASS documents.

a

67 Fed. Reg. 38,467.

OMB directed agencies to: NASS’s response

• Prepare a draft report explaining how 

their guidelines will ensure and 

maximize the quality of information. 

• Publish a notice in the Federal 

Register announcing the availability of 

this report on the agency’s Web site 

for public comment. 

• NASS posted its draft report and guidelines to 

its Web site from March 27, 2002, until 

September 30, 2002.

• Notice of availability was published in the June 

4, 2002, Federal Register a and the report itself 

was available on NASS’s Web site. 

• Post their final report and guidelines 

to their Web sites.

• Final guidelines are on the NASS Web site.

• Develop administrative mechanisms 

allowing affected persons to correct 

disseminated information that does 

not comply with the OMB guidelines. 

• NASS outlines its correction procedures in 

detail on its Web site. 

• Submit a report by January 1 of each 

year on the number and nature of 

complaints received in the prior fiscal 

year.

• USDA submitted a report to OMB both years 

since the guidelines have been in effect; NASS 

reported no complaints. 

• Designate an official to be responsible 

for the agency’s compliance with 

OMB’s guidelines.

• USDA has designated its Chief Information 

Officer (CIO) as this official. The CIO in turn 

delegates compliance questions to lower-level 

offices, including the Standards Officer within 

the office of the Associate Administrator at 

NASS.
Page 10 GAO-05-644 Information Quality Act

Better Documentation 

Could Improve the 

Transparency of Data 

Products and 

Correction Procedures 

NASS’s IQA guidelines define transparency as, “a clear description of the 

methods, data sources, assumptions, outcomes, and related information 

that allows a data user to understand how an information product was 

designed and produced.” NASS’s guidelines also note that “NASS will make 

the methods used to produce information as transparent as possible” and 

that its “internal guidelines call for clear documentation of data and 

methods used in producing estimates and forecasts. . . .”

To assess the extent to which NASS processes help ensure the 

transparency of the information it publishes, we examined key publications 

from the 2002 Census of Agriculture. Census reports vary in terms of scope 

and intended audience (see table 2). On the one hand, the United States 

Summary and State Data report contains over 100 data tables, an 

introduction, and four appendices. On the other hand, County Profile

reports summarize each county’s agricultural situation on two pages.

Overall, we assessed eight census reports within three product categories, 

as well as the Frequently Asked Questions (FAQ) section of the 2002 

Census Web site, to determine the extent to which NASS followed its own 

guidelines for ensuring the transparency of its products. As shown in table 

2, the transparency of the data documentation in the reports we reviewed 

varied between the Geographic Area Series reports—which are the most 

comprehensive of NASS’s products and addressed 15 of the 20 data 

documentation practices—and the Specialty Products and Special Studies

which, depending on the specific product, addressed no more than 1 of the 

practices. 
Page 11 GAO-05-644 Information Quality Act

Table 2: Census Reports Need More Robust Documentation

Source: GAO analysis of NASS reports.

All eight reports and the FAQ Web site lacked a discussion of four 

documentation practices, including the following: 

1. Questionnaire testing. NASS produced a separate, internal report that 

discusses questionnaire testing in detail; however, publicly available 

census publications do not address this topic. 

2. Limitations of the data. NASS does not discuss data limitations in the 

census reports we reviewed. 

3. Impact of imputations, by item. When a statistical agency receives a 

report form with missing values, it normally estimates or “imputes” 

those values based on comparable data sources such as a similar farm 

operation. Although NASS uses a complex editing and imputation 

process to estimate missing values, and describes this process in the

Product category Product title General description 

Portion of 20

documentation

practices

addressed

Geographic Area 

Series

United States Summary and 

State Data

Contains over 100 national and state data tables. 15

State and County Data Contains over 100 state and county data tables. 15

Census Quick Stats: 

Ag Statistics Data 

Base

2002 Census of Agriculture 

Downloadable Application 

A database for users to download and generate data 

tables at the national, state, and county levels. 

14

Specialty Products and 

Special Studies

State and County Profiles Two-page reports containing summary data about a 

state or county. 

0

Quick Facts from the 2002 

Census of Agriculture

This report presents national data in 16 charts or 

graphs.

1

Ranking of Market Value of 

Agricultural Products Sold

This report contains state tables that rank the 

agricultural products sold by market value. The report 

also includes table definitions.

1

Congressional District Profiles Each profile is a two-page report that contains summary 

data about one congressional district. 

0

Ranking of Congressional 

Districts

Contains tables for 46 data items, such as number of 

farms, and ranks the congressional districts for each of 

these data items. 

1

Additional Information Frequently Asked Questions This section of the census Web site contains questions 

and answers grouped into four categories.

6
Page 12 GAO-05-644 Information Quality Act

United States Summary and State Data report appendix, it does not 

quantify the impact of imputations by item in reports. 

4. Whether any of the collected data have been suppressed for data 

quality reasons. Without information on whether any of the data had 

been suppressed because the quality was lacking, data users must 

assume that reports include all data items collected in the census had 

met agency publication standards.

Although NASS appropriately recognizes the variation in data user needs 

by publishing several types of specialized reports, none of the reports we 

reviewed direct data users where to find either a complete set of 

documentation or additional documentation. For example, given the short 

length and summary format of the County Profile reports, it is not 

surprising that they lack documentation. However, in order for users to 

assess the quality of the data contained in the reports, it is important for 

NASS to at least provide links on its Web site or to other publications 

where users can access definitions, response rates, and other relevant 

information. 

NASS Should Document Its 

Procedures for Handling 

Correction Requests Not 

Filed under the IQA 

NASS has two methods for handling data correction requests, depending 

on how they are submitted: a formal approach prescribed by OMB for 

correction requests filed under IQA, and an informal approach that NASS 

uses to address correction requests that are not filed under IQA. NASS’s 

informal correction procedures lack transparency because they are not 

documented and individual cases are not tracked. As a result, we could not 

determine the nature of these correction requests or whether or how they 

were addressed. 

Consistent with OMB’s guidelines, NASS detailed its procedures to request 

corrections under IQA on its Web site, and posted appropriate Federal 

Register notices. For example, NASS’s Web site explains that to seek a 

correction under IQA, petitioners must, among other steps: (1) state that 

their request for correction is being submitted under IQA, (2) clearly 

identify the information they believe to be in error, and (3) describe which 

aspects of NASS’s IQA guidelines were not followed or were insufficient.11

11See www.usda.gov/nassinfo/infocorrection.htm.
Page 13 GAO-05-644 Information Quality Act

According to the instructions posted on its Web site, NASS’s IQA 

procedures are triggered only when petitioners explicitly state they are 

submitting a correction request under IQA. To date, none have done so. 

NASS addresses all other correction requests using informal, 

undocumented procedures that were in place before IQA was enacted. 

NASS officials explained that such requests are forwarded to the agency 

official responsible for preparing the report containing the information in 

question. That official, in turn, determines if the request can be resolved by 

clarifying the data, or whether a correction is needed. If a data item needs 

to be corrected, NASS has a set of procedures for documenting errors and 

issuing errata reports that are detailed in its Policy and Standards 

Memorandum No. 38. The memorandum describes the circumstances 

under which errata reports will be printed, and provides a mechanism for 

NASS staff to describe the nature of the error, its cause, and the action 

taken to resolve it. 

According to the Administrator, Associate Administrator, and other senior 

NASS officials we interviewed, the requests it has handled from the 2002 

Census have so far been resolved to the petitioners’ satisfaction, and none 

resulted in any corrections to the data from the 2002 Census. However, 

because NASS does not document its informal procedures for handling 

inquiries and data correction requests, and lacks a recordkeeping system to 

log and track them, NASS could not provide us with firm information on 

the number of inquiries it has handled, the nature of those inquiries, and 

whether and how they were addressed. 

This is not to say that all complaints should follow the same procedures 

required by the IQA mechanism. For efficiency’s sake, it is important for 

agencies to respond to complaints in accordance with the magnitude of the 

problem. However, to provide a more complete picture of the questions 

NASS receives about its data and how those questions were handled, it will 

be important for NASS to better document its approach for handling 

correction requests not filed under IQA, and track their disposition.

NASS Has Taken Steps to 

Better Document Its Criteria 

for Assessing Input on 

Census Content

The 2002 Census of Agriculture was the first in which NASS developed the 

questionnaire (the 1997 Census of Agriculture was moved from the Census 

Bureau to NASS after the content had been determined). In doing so, NASS 

went to great lengths to obtain input from data users on what questions to 

ask, and evaluated their suggestions using a documented set of criteria. In 

preparing for the 2007 Census, NASS sought feedback on the questionnaire 

content from a broader spectrum of data users, in part because NASS 
Page 14 GAO-05-644 Information Quality Act

solicited suggestions via the Internet. However, unlike the 2002 cycle, the 

criteria NASS used to assess the feedback were not initially documented, 

which is contrary to NASS’s IQA guidelines. However, as a result of our 

review, NASS has developed documented criteria similar to that used 

during the previous census. 

Under the Paperwork Reduction Act, agencies must obtain OMB’s approval 

prior to collecting information from the public. As part of this process, 

agencies must certify to OMB that, among other things, the effort is 

necessary for the proper performance of agency functions, avoids 

unnecessary duplication, and reduces burden on small entities. Agencies 

must also provide an estimate of the burden the information collection 

would place on respondents.12

For the 2002 Census, NASS submitted its request for approval—a form 

called “OMB 83-I”—in August 2001, and OMB approved it in October 2001. 

NASS estimated that the census would require a cumulative total of more 

than 1.3 million hours for respondents to complete and would cost them, in 

terms of their time, in excess of $21 million. 

OMB’s approval process also requires agencies to solicit input from 

external sources. NASS obtained input on the 2002 Agricultural Census 

content through a Federal Register notice, meetings with data users, and 

by contacting federal and state agencies that use census statistics to 

discuss data needs.

Likewise, NASS is obtaining input on the content of the 2007 Census 

through a variety of channels. According to an agency official, the process 

began around June 2004, when NASS began releasing publications from the 

2002 Census. NASS sent an evaluation form to its state offices requesting 

feedback on the census, including their suggestions for changing the 

content. NASS also asked the state offices to identify users from whom it 

could obtain additional feedback. 

NASS solicited further input by reaching out to data users within USDA and 

other federal agencies, querying organizations included in a list of “typical” 

data users maintained by NASS’s Marketing and Information Services 

Office, and holding periodic regional meetings with data users. NASS also 

1244 U.S.C. § 3506(c).
Page 15 GAO-05-644 Information Quality Act

has a “hot button” on its Web site where visitors are asked what items, if 

any, should be added or deleted from the census.13

In all, NASS obtained input on the 2007 Census through 10 distinct 

conduits. Moreover, compared to the process used to develop the content 

of the 2002 Census, its 2007 efforts were open to a wider spectrum of 

customers, and involved more direct contact with data users during the 

planning phase. Indeed, as shown in table 3, NASS’s outreach via the 

Internet, regional meetings, and queries to data users was over and above 

the steps it took when developing the 2002 Census. This openness was 

reflected in the comments of the six data users we interviewed. Five of the 

six users said NASS’s approach to eliciting input was adequate, while three 

of the six had requested new content items for the 2007 Census to better 

meet the needs of their organizations.

The content evaluation process began in December 2004, and NASS is 

currently testing the questionnaire content. Following any refinements, 

mail-out of the actual census is scheduled for December 2007. 

13See http://www.nass.usda.gov/census/census02/feedback.htm.
Page 16 GAO-05-644 Information Quality Act

Table 3: NASS Is Using More Extensive Outreach to Develop the 2007 Census Compared to 2002

Source: GAO analysis of NASS data.

For both the 2002 and 2007 Census cycles, the solicitation, review, and 

ultimate determination of the questionnaire content was led by the Census 

Content Team, a group consisting of experienced NASS statisticians 

representing different segments of the agency such as livestock, crops, and 

marketing. The 2002 Content Team used specific, documented criteria to 

inform its decisions. Specifically, suggestions were assessed according to 

the following factors, which were also made available to data users: 

• items directly mandated by Congress or items that had strong 

congressional support;

• items proposed by other federal agencies where legislation called for 

that agency to provide data for Congress;

• items needed for evaluation of existing federal programs;

• items which, if omitted, would result in additional respondent burden 

and cost for a new survey for other agencies or users;

• items required for classification of farms by historical groupings;

• items needed for improving coverage in the census; and

Method of outreach 2002 Census 2007 Census

Posted Federal Register notices X X

Solicited input from state agricultural statistical offices X X

Solicited input from state governors X

Solicited input from Advisory Committee on Agricultural Statistics X X

Solicited input from land grant universities X X

Solicited input from federal data users X X

Held federal data user working group meetings X X

Held USDA-wide national data user outreach meeting X X

Solicited input from a list of “typical” census users maintained by NASS’s Marketing and Information 

Services Office

X

Solicited input via Web site feedback form X

Held NASS-specific, regional data user meeting X
Page 17 GAO-05-644 Information Quality Act

• items that would provide data on current agricultural issues. 

However, the criteria the 2007 Team used to assess input on the 

questionnaire content were not initially documented. According to agency 

officials we interviewed, NASS largely relied on professional judgment to 

evaluate the feedback it received, considering such factors as the need to 

keep the data comparable to past censuses and not increase the length of 

the questionnaire. 

Although a certain amount of professional judgment will invariably be used 

in making determinations on questionnaire content, the absence of 

documented assessment criteria is inconsistent with NASS’s guidelines. 

Indeed, these guidelines note that transparent documentation “allows a 

data user to understand how an information product was designed and 

produced.” Moreover, without documented criteria, it is not clear whether 

members of the Content Team are considering the same set of factors, or 

even if they are weighing those factors in the same manner.

According to NASS, the shift in approach stemmed from staff turnover and 

reassignments of members of the 2002 Team and, as a result, the 2007 Team 

was not aware of the criteria used in 2002. Our review made the 2007 Team 

aware of the earlier set of criteria, and the Team has since developed 

similar documentation. NASS noted that all future content teams will use 

and update these criteria when developing the content of subsequent 

censuses.

It will be important for NASS to continue with this approach because it is 

more consistent with its own IQA guidelines, and will also help NASS to do 

the following: 

Ensure the utility and relevance of information. A key principle for federal 

statistical agencies is to provide information relevant to issues of public 

policy.14 However, the nation’s information needs are constantly evolving, 

and it is important for statistical agencies to adapt accordingly. This is 

particularly true with agriculture, where a variety of factors such as 

changing technology and agricultural trends can affect what information 

should be collected. Rigorous content selection criteria could help NASS 

14Margaret E. Martin, Miron L. Straf, and Constance F. Citro eds., Principles and Practices 

for a Federal Statistical Agency, (3rd ed.) (Washington, D.C.: National Academies Press, 

2005), p. 4.
Page 18 GAO-05-644 Information Quality Act

methodically evaluate the needs of different users, establish priorities, and 

keep the census synchronized with changing public policy requirements.

Maximize cost-effectiveness and reduce public burden. As with all federal 

surveys, there are financial and nonfinancial costs to conducting the 

Census of Agriculture. These costs include the direct expenditures related 

to planning, implementing, and analyzing the census, as well as 

disseminating the information. There is also a cost to respondents in terms 

of the time they take to complete the questionnaire. Additionally, there are 

opportunity costs in that for every question that is included in the census, 

another question might need to be excluded so as not to increase the length 

of the census. Rigorous, consistently applied criteria can help promote 

cost-effectiveness because they can ensure that only those questions that 

meet a particular, previously identified need are included in the census. 

Applying such criteria also help inform decisions on the appropriate role of 

the federal government in collecting the data, and whether a particular 

question might be more appropriately addressed by a different survey, 

government organization, or the by the private sector. 

Maintain credibility. Content selection criteria provide a basis for 

consistent decision making on what to include in the census and what gets 

left off. This is especially important for maintaining NASS’s credibility 

given the input it receives from various sources. Without documented 

criteria, NASS’s actions could be perceived as arbitrary or 

disproportionately swayed by one particular interest or another; thus, 

NASS’s decisions would be more defensible. 

Further, documented criteria will guard against the loss of institutional 

memory to the extent there is further turnover in Content Team 

membership. 

Conclusions NASS satisfied the procedural responsibilities and reporting requirements 

under OMB’s IQA guidelines. Moreover, to the extent that NASS continues 

to use the documented criteria it developed to inform future decisions on 

the content of the Census of Agriculture, it could help establish a closer 

alignment between the questions included in the census and evolving 

agricultural policy requirements, resulting in a more cost-effective data 

collection program.

Building on these efforts, the transparency of census data products could 

be improved with more robust documentation. NASS’s procedures for 
Page 19 GAO-05-644 Information Quality Act

addressing correction requests not filed under IQA could be more 

transparent as well. More than just a paperwork issue, greater transparency 

will help enhance NASS’s accountability to public data users and increase 

the credibility of census information. 

Recommendations for 

Executive Action

To help enhance the transparency of the Census of Agriculture’s processes 

and products, we recommend that the Secretary of Agriculture direct NASS 

to take the following two steps:

1. Ensure that census products fully address NASS’s own guidelines for 

data documentation or at least contain links to such information. The 

list of 20 documentation practices that we developed, while not 

necessarily exhaustive, represents sound actions used by other 

statistical agencies and could form a starting point for NASS.

2. Document and post on NASS’s Web site its procedures for handling 

data correction requests not filed under IQA, and track the disposition 

of those requests. 

Agency Comments and 

Our Evaluation

The NASS Administrator provided written comments on a draft of this 

report on September 8, 2005, which are reprinted in appendix I. NASS 

noted that our “report and recommendations are insightful and will be used 

to further strengthen the transparency of NASS methods and procedures.”

In particular, NASS concurred with our finding that the methods and 

procedures in its specialized reports should be better documented and, 

consistent with our recommendation, stated that these products “will now 

provide links to this information.” NASS’s efforts, if fully implemented, 

should make it easier for data users to understand how these products 

were designed and produced, and NASS should be commended for its 

actions to continually improve its products and better meet the needs of its 

customers. 

While NASS’s more comprehensive products were better documented, our 

analysis found that they could also benefit from more robust 

documentation. Thus, in keeping with our recommendation, it will be 

important for NASS to ensure that all of its census products—its larger 

reports and more focused studies--fully address NASS’s own guidelines for 

data documentation. 
Page 20 GAO-05-644 Information Quality Act

In commenting on our recommendation for NASS to document and post on 

its Web site its procedures for handling data correction requests not filed 

under IQA, NASS concurred with our view that this information would 

provide it with a better sense of the questions it receives about its data, but 

added that “a detailed recordkeeping system to log and track every inquiry” 

would not be the best use of its resources. Instead, NASS plans to “compile 

a listing of the more common issues” and make them available on its Web 

site in the form of frequently asked questions. NASS believes this approach 

would be useful for future planning, as well as provide answers to 

questions most likely to arise among other data users. 

As noted in our report, our recommendation stemmed from our finding that 

NASS could not provide us with information on the number of inquiries not 

filed under IQA, the characteristics of those inquiries, and how they were 

addressed. Although the details remain to be seen, NASS’s proposed 

approach could provide this information and, consistent with the intended 

outcome our recommendation, address the need for greater transparency. 

NASS’s efforts will be further strengthened if, consistent with our 

recommendation, it posts on its Web site its procedures for handling 

correction requests not filed under IQA. 

We will send copies of this report to other interested congressional parties, 

the Secretary of Agriculture, and the NASS Administrator. Copies will be 

made available to others on request. This report will also be available at no 

charge on GAO’s Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact me 

at (202) 512-6806 or williamso@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to this report are 

listed in appendix III.

Orice M. Williams

Director

Financial Markets and

Community Investments
Page 21 GAO-05-644 Information Quality Act

Appendix I

Comments from the Department of Apendxies

Agriculture ApendxiI
Appendix I

Comments from the Department of 

Agriculture

Page 22 GAO-05-644 Information Quality Act
Page 23 GAO-05-644 Information Quality Act

Appendix II

How Census of Agriculture Reports Address 

Various Documentation Elements ApendxiI

Documentation 

practices

Product titles

United 

States 

Summary 

and State 

Report

State and 

County 

Reports 

(Alabama)

Quick 

Stats: Ag 

Statistics 

Database

State and 

County 

Profiles 

Quick 

Facts

Ranking: 

Market 

Value Ag 

Products 

Sold

Congress. 

District 

Profiles

Ranking of 

Congress. 

Districts

Frequently 

Asked 

Questions

Discussion of 

how NASS 

developed the 

content of the 

census.

XX X X

Discussion of 

2002 Census 

content 

consistency with 

1997.

XX X X

Description of 

how the content 

differs.

XX X X

Discussion of 

why NASS made 

content changes.

XX X

Definition of the 

population.

XX X X X

Definition of data 

items.

XX X X X

Discussion of 

sample design.

XX X

Discussion of 

questionnaire 

design.

XX X

Discussion of 

questionnaire 

testing.

Copy of the 

questionnaire.

X X

Discussion of 

data collection 

procedures.

XX X

Discussion of 

nonresponse 

followup.

XX X

Discussion of 

data entry 

procedures.

XX X
Appendix II

How Census of Agriculture Reports Address 

Various Documentation Elements

Page 24 GAO-05-644 Information Quality Act

Source: GAO analysis of NASS data.

Note: See the Objectives, Scope, and Methodology section of this report for a complete explanation of 

our analysis.

Discussion of 

data editing 

procedures.

XX X

Quantify the 

impact of 

imputations, by 

item.

Discussion of 

response rates.

XX X X

Discussion of 

nonsampling 

error.

XX X

Discussion of the 

limitations of the 

data.

Discussion of 

whether any of 

the collected 

data have been 

suppressed for 

data quality 

reasons.

Comparison of 

census results to 

other survey 

results for 

consistency.

X

(Continued From Previous Page)

Documentation 

practices

Product titles

United 

States 

Summary 

and State 

Report

State and 

County 

Reports 

(Alabama)

Quick 

Stats: Ag 

Statistics 

Database

State and 

County 

Profiles 

Quick 

Facts

Ranking: 

Market 

Value Ag 

Products 

Sold

Congress. 

District 

Profiles

Ranking of 

Congress. 

Districts

Frequently 

Asked 

Questions
Page 25 GAO-05-644 Information Quality Act

Appendix III

GAO Contact and Staff Acknowledgments ApendxiI

GAO Contact Orice M. Williams, (202) 512-6806

Acknowledgments In addition to the contact named above, Robert Goldenkoff, Assistant 

Director; David Bobruff; Jennifer Cook; Richard Donaldson; Andrea 

Levine; Robert Parker; John Smale; and Michael Volpe made key 

contributions to this report.

(450344)
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go to 

www.gao.gov and select “Subscribe to Updates.”

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to:

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional 

Relations

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548

Public Affairs Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548