Data Quality: Expanded Use of Key Dissemination Practices Would Further Safeguard the Integrity of Federal Statistical Data
Published: 2006-05-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 2003, the Bureau of the Census (Bureau) changed the day and location of the release of its Income and Poverty Estimates. Some data users believed the change was an effort to suppress unfavorable news and questioned the Bureau's data dissemination practices. GAO was asked to assess whether (1) the Bureau adhered to its dissemination practices for the 2003 and later releases, and (2) the Bureau and 13 other federal statistical agencies follow data release practices recommended by the National Research Council (NRC). GAO reviewed the Bureau's dissemination process for the 2003 thru 2005 Income and Poverty Estimates.
section_title: What GAO Found
paragraphs: While not all of the Bureau's data dissemination practices are documented, GAO was able to determine through discussions with Bureau officials and review of available documentation, that the Bureau adhered to most of its long-standing data release practices. However, the Bureau did depart from the traditional day and location for the release of the Income and Poverty Estimates report in 2003 and subsequent years. According to the Bureau, the day of the 2003 release was changed because of a delay in producing a companion report, and the location was changed from Washington, D.C., to Suitland, Maryland, in part, because the Director of the Census Bureau stated that he wanted to raise awareness that the construction of its new headquarters had just started. Some of the Bureau's documented practices, such as guidance on who has authority to choose the release date and location, lacked specificity. Also, the Bureau's documented Income and Poverty practices are outdated as they are contained in a 21-year-old memo. The Bureau is updating it, to among other things, reflect current technology. Most of the 14 statistical agencies in GAO's review generally adhered to NRC's guidance, important for (1) the wide dissemination of data, and (2) maintaining a strong position of independence. Still, there were some notable gaps. For example, 6 of the 14 agencies lacked dissemination policies (as recommended by NRC) that promote the regular and frequent release of major findings from an agency's statistical program. The Office of Management and Budget (OMB), in concert with other statistical agencies, is developing governmentwide guidance on the release and dissemination of statistical products that, according to OMB officials, parallels NRC's and other generally accepted release practices. OMB's guidance could foster more consistent adherence to practices that promote broader dissemination of statistical data and enhance its credibility, especially to the extent they address gaps GAO found between agencies' data dissemination practices and NRC's guidance.

section_title: Letter
paragraphs: Data collected and disseminated by federal statistical agencies are a  Data collected and disseminated by federal statistical agencies are a  critical strategic asset because they provide indicators of the economic  critical strategic asset because they provide indicators of the economic  and social well-being of the nation and help inform agencies’ rule-making  and social well-being of the nation and help inform agencies’ rule-making  activities. Given the widespread use and the impact of federal information,  activities. Given the widespread use and the impact of federal information,  the data itself, the timing of reports, and any news releases that  the data itself, the timing of reports, and any news releases that  accompany them must be accurate and objective. Moreover, statistical  accompany them must be accurate and objective. Moreover, statistical  agencies must be viewed as credible to secure the public’s trust. One way  agencies must be viewed as credible to secure the public’s trust. One way  statistical agencies foster credibility is by maintaining a position of  statistical agencies foster credibility is by maintaining a position of  independence; that is, taking appropriate steps that help ensure their data  independence; that is, taking appropriate steps that help ensure their data  products and the timing of their release are free from even an appearance  products and the timing of their release are free from even an appearance  of political influence.  of political influence.
Because of the sensitivity of certain statistical information, seemingly  Because of the sensitivity of certain statistical information, seemingly  mundane actions, such as the timing of a data release, can be called into  mundane actions, such as the timing of a data release, can be called into  question, as interested parties may perceive the change as a maneuver to  question, as interested parties may perceive the change as a maneuver to  obscure or manipulate the information for partisan purposes. The  obscure or manipulate the information for partisan purposes. The  Department of Commerce, which has a number of data gathering  Department of Commerce, which has a number of data gathering  programs under its purview, oversees the U.S. Bureau of the Census  programs under its purview, oversees the U.S. Bureau of the Census  (Bureau). Among its many surveys, the Bureau has responsibility to report  (Bureau). Among its many surveys, the Bureau has responsibility to report  estimates of the nation’s income and poverty. In 2003, the Bureau changed  estimates of the nation’s income and poverty. In 2003, the Bureau changed  the day and location for releasing this information. Because the estimates  the day and location for releasing this information. Because the estimates  showed that poverty rates had risen, some data users believed that the  showed that poverty rates had risen, some data users believed that the  change was an effort to suppress unfavorable news, and called into  change was an effort to suppress unfavorable news, and called into  question the Bureau’s practices for disseminating such data, and the  question the Bureau’s practices for disseminating such data, and the  Department of Commerce’s role in the 2003 release.  Department of Commerce’s role in the 2003 release.
This letter responds to your request that we review the Bureau’s level of  This letter responds to your request that we review the Bureau’s level of  independence in releasing its annual Income and Poverty Estimates over  independence in releasing its annual Income and Poverty Estimates over  the last several years. As agreed with your offices, we determined the  the last several years. As agreed with your offices, we determined the  extent to which (1) the Bureau adhered to its dissemination practices for  extent to which (1) the Bureau adhered to its dissemination practices for  the issuance of the 2003 annual Income and Poverty Estimates and  subsequent releases in 2004 and 2005, and (2) the Bureau and 13 other  federal statistical agencies follow data dissemination practices that the  National Academy of Sciences’ National Research Council (NRC)  recommend in a 2005 report.
To address the first objective, we reviewed relevant program documents  and interviewed Bureau officials responsible for disseminating the Income  and Poverty Estimates and other data releases. The dissemination process  includes the steps from approval of the report content up to and including  public distribution of the report. For the second objective, we surveyed  key officials at the Bureau and 13 other federal statistical agencies  collecting information on their practices for releasing data. We compared  the 14 agencies’ practices to those developed by NRC that are important  for (1) the wide dissemination of data, and (2) maintaining a strong  position of independence. According to NRC, elements within these  practices facilitate the provision of timely, credible, and politically neutral  information into the hands of data users. Appendix I provides additional  details on our scope and methodology. We conducted our work between  March 2005 and April 2006 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: In September 2003, the Bureau broke from its tradition of releasing its  Income and Poverty Estimates on a Tuesday or Thursday at a news  conference at the National Press Club (see table 1). The data were instead  released at a news conference on a Friday at Bureau Headquarters in  Suitland, Maryland. Although the Bureau provided the media and other  attendees with bus service from the National Press Club to Suitland,  because the data showed that poverty levels had risen, some data users  expressed concern that the change in day and location was an attempt to  suppress unfavorable information by releasing it at a more remote location  and before a weekend, when the public tends to pay less attention to the  news.
The Income and Poverty Estimates, like other kinds of federal statistical  information, provide key measures of the health and well-being of our  society. As a result, the data need to be accurate, timely, accessible,  relevant, and objective. At the same time, according to NRC, the manner in  which agencies release the data is also important, and needs to be free  from even an appearance of bias and political manipulation. Failure to  meet this goal could undermine public confidence in the information and  erode an agency’s credibility.
That said, although various guidance and laws have been developed to  safeguard the overall quality of federal data, few governmentwide  provisions directly address the data dissemination process itself, and  agencies have largely been left to develop their own practices. For  example, while OMB’s Statistical Policy Directive Number 3, “Compilation,  Release, and Evaluation of Principal Federal Economic Indicators”  provides detailed guidance on the dissemination of data, it only applies to  38 market sensitive principal economic indicators. Statistical Policy  Directive Number 3 is highly regarded in the statistical community  because it provides statistical agencies with comprehensive data  dissemination guidance, requiring agencies to, among other actions,  promptly release data according to an established schedule, and announce  and fully explain any schedule changes in advance.
Under the Information Quality Act, OMB was required to issue  governmentwide guidelines that provide policy and procedural guidance  to federal agencies for ensuring and maximizing the quality, objectivity,  utility, and integrity of information disseminated by federal agencies.  OMB’s guidelines, issued in final form in February 2002, directed agencies  covered by the act (statistical agencies and most others) to issue their own  quality guidelines. OMB’s guidelines imposed certain core responsibilities  on agencies, including incorporating quality into their information  dissemination practices. OMB noted that quality consists of several  dimensions, including objectivity (which focuses on whether the  disseminated information is accurate, reliable, and unbiased in  presentation and substance).
More generally, OMB helps ensure that the activities of the statistical  agencies are in line with federal statistical policy by coordinating agency  budget requests and interagency groups working on statistical issues,  issuing statistical standards, and reviewing agency requests to collect  information.
This report is the latest of several studies we have issued on the quality of  federal data. See Related GAO Reports at the end of this report for a list of  selected products we have issued to date.
section_title: Release of Income and Poverty Estimates Adhered to Most of the Bureau’s Data Dissemination Practices
paragraphs: While not all of the Bureau’s data dissemination practices are documented,  we were able to determine through discussions with Bureau officials and  review of available documentation, that the Bureau adhered to most of its  long-standing data release practices. In changing the date and location of  the 2003 and subsequent releases of the Income and Poverty Estimates,  the Bureau did depart from its tradition of releasing this information on a  Tuesday or Thursday at a news conference at the National Press Club.  That said, under the Bureau’s documented data dissemination practices  (1) there is no requirement for the Bureau to release this information at a  particular location on a given day and, (2) no particular official is  designated authority to choose the release date and location. Bureau  officials stated that the date of the 2003 release was changed from  September 23rd to September 26th for several reasons, including delays in  producing a companion report on supplemental measures of expenditures,  consumption, and poverty that was to be released at the same time. Also,  the 2004 and 2005 estimates were released a month earlier than in prior  years to coincide with the release of data from the American Community  Survey. The documented practices for disseminating the Income and  Poverty Estimates are contained in a memo that is 21 years old so the  Bureau is updating them, to among other things, reflect current  technology.
subsections: section_title: The Bureau’s Income and Poverty Estimate Dissemination Practices Are Derived from Several Sources
paragraphs: The Bureau has several sources of documented, agencywide practices for  disseminating data to the public. For example, in accordance with OMB’s  guidelines for implementing the Information Quality Act, the Bureau  developed its own set of quality guidelines that include provisions aimed  at ensuring the objectivity and integrity of its data. The Bureau also has a  series of data dissemination practices available on its Intranet site and it  has issued four standards governing the dissemination of data products,  including Dissemination of Census and Survey Data Products. We found  that the only documented practices specific to the release of the Income  and Poverty Estimates are contained in a 1985 memorandum that was  included as one of several appendixes to the Bureau’s Administrative  Manual. The manual provides Bureau policy on the release of data and  guidance for divisions to follow in responding to requests for such  information.
The 1985 memorandum, which was signed by the Director of the Census  Bureau at the time, includes eight broad steps, covering the process for  disseminating the Income and Poverty Estimates. The eight steps include  the time period from approval of the report content up to and including  distributing the report at the press conference.  1.  The Associate Director for Demographic Fields approves the final  content of the report prepared by the Population Division after review  by the Statistical Methods Division.  2.  The Public Information Office receives a copy of the final content to  draft a press release. This draft release is approved within the Census  Bureau, by the Public Affairs Specialist in the Under Secretary for  Economic Affairs’ office, and by the Commerce Department’s  newsroom.  3.  The report is prepared for camera-ready form.  4.  Camera-ready copy is sent to the printer.  5.  When the completion time for this report is known, the Census Bureau  establishes the release date and time with Commerce Department  concurrence.  6.  Approximately 48 hours before report release date and time, the  Census Bureau briefs the Deputy Secretary for Economic Affairs on  the principal findings.  7.  The Census Bureau makes the report and accompanying press release  available to the media on the established date at 9 a.m. for 10 a.m.  release.  8.  The Census Bureau distributes the report and press release to the  Congress and the OMB at the same time as the media.
section_title: Release of 2003 and Subsequent Income and Poverty Estimates
paragraphs: In releasing the 2003 Income and Poverty Estimates, the Bureau adhered  to most of its data dissemination practices. The change in the date and  location of the 2003 and subsequent releases of the Income and Poverty  Estimates was a departure from the Bureau’s tradition of releasing this  information on a Tuesday or Thursday at a news conference at the  National Press Club. That said, under the Bureau’s documented data  dissemination practices there is no requirement for the Bureau to release  this information at a particular location on a given day. Based on our  review of available documentation and our interviews with officials  involved with the Income and Poverty Estimates, the Bureau followed the  steps in the 1985 memo in the 2003, 2004 and 2005 releases, with the  exception of the release time as previously described.
While the Bureau complied with its documented practices for the  dissemination of the Income and Poverty Estimates, they lacked  specificity. For example, clear and specific documentation does not exist  for how and when the release date and location are to be determined for  the Income and Poverty Estimates and who should make those decisions.  In actuality, as discussed in greater detail subsequently, in 2003, the  Director of the Census Bureau chose the location and the Associate  Director for Communications chose the date. However, because this was  not thoroughly documented (the 1985 memo only provides general  guidance), it is unclear to the public who made these decisions and how  they were made.
Furthermore, Bureau officials told us that they did not retain any internal  memos or e-mails that documented the decision to change the 2003  Income and Poverty Estimates release, which would have provided  evidence to support the Bureau’s narrative of the events leading up to the  release.
section_title: According to the Bureau, Delays in the Companion Report Caused Change in Timing of the 2003 Release
paragraphs: Based on our review of available Bureau documents and interviews with  key Bureau officials, several factors led to the change in the timing of the  release of the 2003 and subsequent Income and Poverty Estimates.
The Chief of the Census Bureau’s Housing and Household Economic  Statistics Division at the time of the 2003 release of Income and Poverty  Estimates, and other senior officials we spoke to, stated that the 2003  Income and Poverty Estimates release was different from years past  because the Bureau decided earlier that year to issue the report at the  same time as a multi-agency report on supplemental measures of  expenditures, consumption, and poverty. This decision was made before  the findings of the Income and Poverty Estimates report were known.  Bureau officials stated that although the original target date for releasing  both reports was September 23, 2003, complications with finalizing the  supplemental measures report kept it from being ready for release on that  day.
According to Bureau officials and documents we reviewed, because the  supplemental measures of expenditures, consumption, and poverty report  involved several statistical agencies, there was a different clearance  process than that used for the Income and Poverty Estimates report. As a  result, while the Bureau had completed its review of the latter report, all  the members of a steering committee still needed to review the report on  supplemental measures.
At the same time, based on our discussions with Bureau officials involved  with the Income and Poverty Estimates report, as well as available  documents, the Commerce Department’s Under Secretary for Economic  Affairs wanted to release both reports simultaneously in an effort to  broaden the public’s understanding of social well-being. The Under  Secretary’s decision was consistent with the Bureau’s ongoing effort to  provide alternative estimates of poverty, which itself stemmed from a 1995  report by the National Academy of Sciences that recommended revising  how poverty is measured.
Because of the additional time required to clear the supplemental  measures report, Bureau officials responsible for the Income and Poverty  Estimates asked for a later date to issue their report. Consequently, the  Bureau’s Associate Director for Communications, with the Director’s  consent, scheduled Friday, September 26, 2003, as the release date for the  Income and Poverty Estimates, and both reports were issued on that  date.
Under the Bureau’s guidance for dealing with the media, Census Bureau  analysts are to arrange their work schedules to be available for inquiries  for 2 to 3 days after a data release. This is why, prior to 2003, the Bureau  tended to release the Income and Poverty Estimates earlier in the week: it  obviated the need for analysts to work on the weekend. Additionally,  Bureau officials said that because of the Internet and cable television, the  news cycle is no longer viewed as a cycle and has instead become a 24  hours a day, 7 days a week operation. Thus, many of the media’s inquiries  occur the same day as the data are released. While it seldom does so, the  Bureau has released other reports on Fridays, such as its 2001 health  insurance report.
For the 2004 and 2005 releases of Income and Poverty Estimates, the data  were released in August at the same time as data from the American  Community Survey. Bureau officials reported the Income and Poverty  Estimates (which come from the Bureau’s Current Population Survey) are  one of several sources of income and poverty information issued by the  Bureau. Starting in 2003, the Bureau began releasing income and poverty  information from the American Community Survey, which produces data  independent from the Current Population Survey. Bureau officials  reported that for methodological and other reasons, estimates from the  Current Population Survey, in some cases, did not match estimates from  the American Community Survey, causing confusing press coverage. In  August 2004, when the Bureau released the two data sets at the same time,  the press release that accompanied the estimates explained why the two  sets of numbers might not match. (According to Bureau officials, the plans  to move the release date from September 2004 to August 2004 were in  place well before the actual release.) Going forward, the Bureau plans to  continue its practice of releasing the American Community Survey data  and the Income and Poverty Estimates simultaneously around the last  Thursday in August.
section_title: According to the Bureau, Several Factors Affected the Change in Location of the 2003 Release
paragraphs: According to a senior official we interviewed in the Bureau’s Public  Information Office, the location of the 2003 Income and Poverty Estimates  news conference was changed from the National Press Club in  Washington, D.C., at the request of the Director of the Census Bureau, to  help raise awareness of the Bureau’s new headquarters building, which  was under construction. The groundbreaking ceremony at the new site on  the Bureau’s campus in Suitland, Maryland, had taken place several weeks  earlier, and a Bureau official reported the Director wanted the media to  see the improvements the Bureau was making at its headquarters location,  as well as to foster a spirit of good feeling, and highlight how Bureau  officials hoped that the new building would help improve the morale of  Bureau employees. The Bureau provided bus service for attendees from  the original location at the National Press Club in downtown D.C., to  Bureau headquarters in Suitland, a distance of around 8 miles.
Additionally, according to the Bureau’s Associate Director for Strategic  Planning and Innovation, the location of the news conference is no longer  as relevant as it once was because of changes in technology. The 2003  news conference was broadcast in real time via the Internet, and materials  were made available on the Bureau’s Web site. The Associate Director for  Strategic Planning and Innovation noted that because of these advances  and accommodations, news media on-location attendance has declined  over recent years. Yet, overall media participation has increased via the  availability of Web casts, satellite-feed transmissions and telephone-audio  access. Consequently, the Suitland, Maryland headquarters is now the  primary location for this annual news conference.
section_title: Bureau Officials Made Key Decisions on Releasing Income and Poverty Estimates
paragraphs: Because the Bureau did not maintain a written record of the release  decision, a precise list of the personnel involved and time line of events is  unavailable. However, according to the Bureau officials we interviewed,  the following Bureau employees were involved in the process for releasing  the Income and Poverty Estimates in 2003:    Director of the Census Bureau;    Deputy Director/Chief Operating Officer;    Chief of the Bureau’s Housing and Household Economic Statistics    Assistant Division Chief for Income, Poverty, and Health Statistics;    Associate Director for Demographic Programs, now serving as the  Associate Director for Strategic Planning and Innovation;    Associate Director for Communications;    Staff from the Bureau’s Housing and Household Economic Statistics    Staff from the Bureau’s Administrative and Customer Services Division;    Chief and Deputy Chief of the Bureau’s Public Information Office.
Bureau officials said that prerelease access to the Income and Poverty  Estimates is tightly controlled because of the possible economic impact of  the data. They stated its contents are shared with staff on a need-to-know  basis, where only those individuals who are involved with drafting the  report or the accompanying press release have access to the information.  They noted further that key steps in preparing and releasing the report  included the following:  1.  Program staff from the Bureau’s Housing and Household Economic  Statistics Division drafted the Income and Poverty Estimates report.  2.  A branch chief reviewed and approved the draft followed by the  Associate Division Chief, the Division Chief, and ultimately the  Associate Director for Demographic Programs, who reports to the  Bureau Director. These senior officials reviewed the report for such  things as clarity and presentation.  3.  When the content of the report was finalized, the Bureau’s Public  Information Office was sent a copy so it could draft a press release.  4.  The final draft was sent to the Bureau’s Administrative and Customer  Service Division, which designed the tables and figures, edited the text,  and prepared a camera-ready version of the report for printing.
According to Bureau officials, the Department of Commerce had only  limited access to information from the Income and Poverty Estimates  report before it was issued, and Commerce officials played no role in the  decision-making process surrounding its release. For example,  Commerce’s Office of Public Affairs reviewed the press release that  accompanied the report and thus had access to some of the numbers as  well as the key findings in the report. However, the office did not have  access to any of the tables that are placed on the Internet. (According to  the Bureau, Commerce usually provides a “hook” for the news media. In  2003, the press release was issued Friday, September 26, and noted, on the  first line, that the nation’s poverty rate rose from 11.7 percent in 2001 to  12.1 percent in 2002.) Moreover, the Bureau considers the press release  part of the report and holds it to the same standards for statistical quality  as the report itself.
Additionally, according to Bureau officials, the Division Chief and the  Assistant Division Chief briefed the Director of the Census Bureau on the  report about a week before the September 26, 2003, press conference.  Commerce’s Under Secretary for Economic Affairs was briefed a day or  two before the press conference and the Under Secretary’s staff were  provided with the final report at that time. (The report was also provided  to the Council of Economic Advisers the afternoon before the press  conference.)
section_title: The Bureau Is Updating Its Practices for Releasing the Income and Poverty Estimates
paragraphs: The then Chief of the Census Bureau’s Housing and Household Economic  Statistics Division told us the Bureau is updating its practices for releasing  the Income and Poverty Estimates. The official stated that the Bureau was  prompted to revisit the 1985 memo by the fact the memo does not include  all of the Bureau’s long-standing data dissemination practices; that some  of the practices in the 1985 memo are obsolete given the age of the  guidance; and the rise of the Internet and other technological advances  have had an effect. The official added that the process for releasing  Income and Poverty Estimates has become more formalized over time.
Bureau officials began drafting these revisions after the 2004 release. In  addition to updating the obsolete practices, Bureau officials stated they  planned to document the current practice of combining the Income and  Poverty Estimates release with the American Community Survey release.  The Bureau plans to issue its updated practices prior to the next release of  the Income and Poverty Estimates expected in August 2006.
section_title: Governmentwide Guidance Is Being Developed That May Improve Statistical Agencies’ Data Dissemination Practices
paragraphs: Most of the 14 statistical agencies we reviewed reported general  adherence to NRC’s guidance, important for (1) the wide dissemination of  data, and (2) maintaining a strong position of independence, although  there were some notable gaps. OMB, in concert with the statistical  agencies, has developed draft guidance on the release and dissemination  of statistical products that, according to OMB officials, parallel NRC’s  guidance. To the extent it is comparable to NRC’s guidance and other  widely accepted procedures for disseminating data, the proposed OMB  directive could promote more consistent adherence to practices that  promote broader dissemination of statistical data and enhance the data’s  credibility.
subsections: section_title: Most Agencies’ Data Dissemination Practices Generally Adhered to NRC’s Guidance for Disseminating Information and Maintaining Their Independence
paragraphs: According to NRC, statistical agencies must have “vigorous and well- planned dissemination programs to get information into the hands of users  who need it on a timely basis.” Attributes of a good dissemination program  include using a variety of mechanisms to inform the widest possible  audience about available data products and how to acquire them. Agencies  should also have arrangements for archiving the information so that it is  available for future use, as well as a publications policy that describes,  among other things, the types of data products that will be made available,  the frequency of their release, and the audiences they serve.
NRC also notes that a statistical agency needs to be politically  independent; that is, it “must be impartial and avoid even the appearance  that its collection, analysis, and reporting processes might be manipulated  for political purposes. . . .” Elements of this practice include having the  authority for decisions associated with the scope, content, and publication  of the data, as well as the authority for the selection and promotion of  professional, operational, and technical staff.
As shown in table 2, the data dissemination procedures of the 14 statistical  agencies we reviewed included elements that were generally aligned with  NRC’s guidance for the wide dissemination of data and maintaining a  strong position of independence.
Twelve or more of the agencies reported having data dissemination  practices possessing four of the five elements related to the wide  dissemination of data. All 14 agencies reported their data dissemination  practices followed NRC’s guidance for (1) having multiple avenues for  disseminating data, (2) releasing data in a variety of formats, and (3)  having policies to guide what data should be preserved and how it should  be archived. Similarly, 12 or more of the agencies’ dissemination practices  had characteristics associated with five of the eight elements  corresponding to maintaining a strong position of independence. These  elements include (1) adherence to predetermined data release schedules,  and (2) authority to make decisions over the scope, content, and frequency  of the data compiled, analyzed, or published.
A greater number of agencies’ data dissemination practices lacked certain  elements important for maintaining a strong position of independence. An  example of one of these elements is NRC’s guidance suggesting statistical  agencies should have the “authority to release statistical information and  accompanying materials (including press releases) without prior clearance  by department policy officials” so there is “no opportunity for or  perception of political manipulation of any of the information.”
However, 10 of the 14 selected agencies reported varying degrees of  clearance required by department officials. For example, at 2 agencies, the  department rather than the statistical agency releases statistical  information. Other agencies have the authority to release statistical  information, except for press releases, without departmental clearance,  although in some cases, the department’s clearance process is limited to  reviewing the grammar, punctuation, and other editorial aspects. (Among  the agencies in our review, 11 agencies use press releases; 1 of these 11  agencies first publishes data from all of its major programs via a press  release; and the 3 remaining agencies reported they do not use press  releases as a vehicle to disseminate data.) With other agencies the  clearance process is more involved. For example, one agency said it  summarizes the data for the press release making sure it is fair and  complete, while officials at the departmental level might insert comments  from the cabinet secretary into the release. Further, 6 of the 14 agencies  lacked dissemination policies that promote the regular and frequent  release of major findings from an agency’s statistical program.
As for the Bureau, officials reported that their agency generally adhered to  NRC’s recommended guidelines. A notable gap was that Bureau officials  did not report adhering to announcing and explaining modifications to a  customary release schedule in advance (7b in table 2). Bureau officials  also lacked the authority to release statistical information and  accompanying materials (including press releases) without prior clearance  by department policy officials (8 in table 2). Also, while the Bureau’s  established publications policy describes the frequency of release of data  collection programs, the Bureau reported the policy does not describe the  types of reports to be made available, the data releases to be made  available, or the audience to be served (4a-c in table 2).
section_title: OMB Is Developing Governmentwide Data Dissemination Guidance That Could Help Strengthen the Credibility of Statistical Information
paragraphs: OMB has been working with the federal Interagency Council on Statistical  Policy to develop guidance for the release and dissemination of statistical  products. According to OMB officials, the guidance is intended to help  ensure statistical products are policy-neutral, timely, and accurate. OMB  officials told us their directive is similar to the NRC’s recommended  practices, as well as to OMB’s Statistical Policy Directive Number 3, which  applies only to the 38 market-sensitive principal economic indicators  produced by the Departments of Agriculture, Commerce, Labor, and  Treasury, as well as the Federal Reserve Board. However, OMB officials  told us this new directive will not be as stringent as Statistical Policy  Directive Number 3, because the data covered by the directive are released  less frequently than the principal economic indicators, and the data are  not considered to be market-sensitive. OMB expects to issue the directive  for public comment in the spring of 2006.
To the extent OMB’s dissemination directive appropriately addresses the  principles underlying NRC’s guidance and Statistical Policy Directive  Number 3, the directive could enhance the quality and credibility of  federal statistical data, in part, by replacing the patchwork of agency- specific guidance with a more transparent, commonly accepted, and  consistently applied framework for disseminating data. For example,  OMB’s directive could help promote more consistent adherence to key  data release practices such as the wide dissemination of data and  maintaining an agency’s independent position.
As noted in the previous section, the dissemination procedures at several  statistical agencies we examined lacked elements important for these  practices, including (1) authority to release statistical information without  prior clearance by department policy officials, (2) data dissemination  policies that foster the frequent release of major findings from an agency’s  statistical programs, and (3) an established publications policy that  describes the types of reports and other data releases to be made  available. As a result, their data products could be better protected, with  the directive, from the appearance of, or actual political involvement.  More specifically, OMB’s new directive could address how best to address  the gaps that exist between agencies’ data dissemination practices on the  one hand, and NRC’s guidance on the other.
As OMB moves forward with its new directive, our interviews with OMB  and statistical agency officials, as well as our past work on data quality  guidance and internal control standards, identified the following questions  that will be important for OMB’s dissemination directive to consider:  Coverage: What will be covered by the directive?—principal statistical  agencies only?—the statistical functions of all agencies?—or only  statistical products? It will be important for OMB’s directive to clearly  define what it does and does not cover so that both statistical agencies and  their parent organizations share the same understanding of their  respective authorities, and help ensure dissemination procedures are  consistently implemented. Certain roles, responsibilities, and processes  need to be clarified as well. Indeed, officials at two statistical agencies we  spoke with said there is ambiguity as to whether a statistical press release  is a statistical product and if so, whether statistical agencies can issue  them with or without first getting releases cleared at the departmental  level. Additionally, OMB has issued a number of guidelines, directives, and  standards on federal statistics. Are there any gaps and overlaps among  them, and can they be better integrated?
Documentation: To what extent, and how, should agencies document  their data dissemination procedures and policies, and how often should  they be reviewed and updated? The agencies we examined did not always  document their processes for disseminating statistical data, relying instead  on professional practice. However, as NRC points out: “Although a long- standing culture of data quality contributes to professional practice, an  agency should also seek to develop and document standards through an  explicit process.”
Moreover, documented guidance would lend more transparency to the  data dissemination process, and thus provide a basis for agencies to  explain their dissemination decisions to policy makers, news media, and  the public. Indeed, an OMB official told us that Statistical Policy Directive  Number 3 is a useful tool for explaining to high level policy officials the  procedures agencies must follow to maintain the integrity of the data, and  why the officials may not access principal economic data before it is  released to the public nor comment on it until after its release.  Documented guidance could also help ensure continuity in the face of  employee turnover.
The importance of documenting agencies’ data dissemination practices  can be seen in the Bureau’s experience in releasing data from the 2000  Census on the homeless and others without conventional housing, when  the Bureau was criticized for shifting its position on reporting components  of this population. In our 2003 report, we noted that although the Bureau’s  decision stemmed from its concerns over the reliability of the underlying  data, the Bureau’s lack of documented, clear, transparent, and consistently  applied guidance governing the release of data from the 2000 Census  hampered the Bureau in explaining its actions. Had such guidance been in  place, it could have helped the Bureau be more accountable and  consistent in its dealings with the public, and helped to ensure that the  Bureau’s decisions both were, and appeared to be, totally objective.
Flexibility: How much leeway should agencies have in implementing  OMB’s directive? Agency officials we spoke with noted the different  missions of the various statistical agencies and cautioned against a one- size-fits-all approach. As a result, it might not be practical to require all  agencies to meet predetermined release dates because it could lead to  additional workload burdens and staffing issues.
Monitoring: How will OMB ensure agencies comply with its directive?  Indeed, the effectiveness of the policies and procedures laid out in OMB’s  directive will rest in large part on the extent to which agencies and their  parent departments adhere to them. Related questions include whether  there should be a regular assessment of agencies’ compliance, and if so,  how often should it occur, and whether this should be done by OMB, or by  the agencies through a self-assessment.
Posting Data: Should agencies’ dissemination policies include written  guidance for releasing information via specific channels? Indeed, although  NRC’s guidance calls on agencies to disseminate data using a variety of  outlets so that the information reaches as wide an audience as possible,  should agencies also have a standard set of conduits where the public will  know an agency’s data will always be available? Such conduits might  include, among others, an agency’s Web site. Because all of an agency’s  data products would be, at a minimum, available from a central point of  access, it could help strengthen an agency’s credibility because the public  would always know where to find it.
section_title: Conclusions
paragraphs: A key lesson learned from the Bureau’s experience is the importance of  fully documented, specific practices for maintaining the integrity of data  products, and by extension, the credibility of the agencies that release  them. Thus, as the Bureau updates its practices for releasing the Income  and Poverty Estimates, it will be important for the Bureau to more  thoroughly document its dissemination procedures so they are clear to the  public.
Further, OMB’s efforts to develop governmentwide guidance on data  dissemination is a positive step toward enhancing the credibility of federal  statistical data, especially to the extent the directive mirrors NRC’s  guidance and Statistical Policy Directive Number 3, as it would replace  each statistical agency’s procedures with a more transparent, commonly  accepted, and consistently applied framework for disseminating  information. As OMB works to complete its directive, it will be important  for it to pay particular attention to those elements dealing with the wide  dissemination of data and maintaining a strong position of independence  that, our survey of statistical agencies suggests can be adhered to by a  greater number of agencies. Likewise, OMB should also consider other  aspects of agencies’ data dissemination efforts that could make its  directive more comprehensive.
section_title: Recommendations for Executive Action
paragraphs: To help improve the Bureau’s data dissemination practices and thus  enhance the agency’s actual and perceived position of independence, we  recommend that the Secretary of Commerce direct the Bureau to, as part  of its efforts to update its practices for releasing the Income and Poverty  Estimates, fully document its key data dissemination practices for  releasing the Income and Poverty Estimates.
Further, to help improve governmentwide data dissemination practices  that would further safeguard the integrity of federal statistical data, we  recommend that the Director of OMB ensure his agency, in completing its  draft directive on the release of federal statistical products, considers  whether and how to address areas where our survey indicates there are  gaps between NRC’s existing guidance and agencies’ practices. These  areas include the extent to which agencies should have (1) full authority to  release statistical information without prior clearance by their respective  departments, (2) data dissemination policies that foster the frequent  release of major findings from agency’s statistical programs, and (3) an  established publications policy describing the types of reports and other  releases an agency has available.
We are also recommending that the Director of OMB direct his agency to  include in its directive additional elements and characteristics important  for agencies’ data dissemination practices, including (1) clear definitions  of what is, and what is not covered by the directive, (2) the extent to  which agencies should document their data dissemination guidance and  how often the guidance should be reviewed, (3) the amount of flexibility  agencies have in implementing OMB’s guidance, (4) procedures for  monitoring agencies’ adherence to its directive, and (5) the feasibility of  requiring agencies to distribute data products through a standard set of  channels as well as through other outlets as appropriate, so that the public  will always know at least one source it can turn to and obtain agency data.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, Commerce neither agreed  nor disagreed with our recommendation for the Bureau to fully document  its key data dissemination practices for releasing the Income and Poverty  Estimates. However, Commerce reiterated the point we made in our report  that the Bureau is updating its practices for releasing the Income and  Poverty Estimates. Commerce noted that the updated document—which  details the dissemination practices for the Income and Poverty  Estimates—is under review. The Bureau plans to issue it prior to the next  release of the Income and Poverty Estimates expected in August 2006.  Commerce also provided some technical corrections and suggestions  where additional context might be needed, and we revised the report to  reflect these comments as appropriate. Commerce’s comments are  reprinted in their entirety in appendix II.
The Director of OMB did not have comments on our recommendations to  them. However, OMB officials provided suggestions for technical  corrections and we revised the report to reflect these suggestions as  appropriate.
As agreed with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days from its  issue date. At that time, we will send copies of the report to interested  congressional committees, the Director of the U.S. Census Bureau, and the  Director of the Office of Management and Budget. Copies will be made  available to others on request. This report will also be available at no  charge on GAO’s home page at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-6806 or farrellb@gao.gov. Contact points for our Office of  Congressional Relations and Public Affairs can be found on the last page  of this report. GAO staff that made major contributions to this report are  listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To address the extent to which the U.S. Bureau of the Census (Bureau)  adhered to its dissemination practices for the release of the 2003 annual  Income and Poverty Estimates and subsequent releases we asked Bureau  officials (in the Housing and Household Economic Statistics Division and  the Bureau’s Public Information Office, among others) to identify the  Bureau and Department of Commerce officials who participated in the  data dissemination decisions, and interviewed the identified officials to  determine their role in the decision-making process, and whether they had  prerelease access to the information. We compared their actions to the  Bureau’s data dissemination practices. The dissemination process includes  the steps from approval of the report content up to and including public  distribution of the report. Some of these practices are documented in the  Bureau’s Policy and Procedures Manual, while others are undocumented  practices that we identified by interviewing cognizant Bureau officials.
Because written records of key activities related to the release (e.g.   e-mails, meeting agendas and notes) were either not retained or never  created, much of our reconstruction of the release was based on  interviews with the officials involved. We interviewed many of these  officials both as a group (by department) and individually to obtain as  complete a picture of the events as possible, and corroborated the  information we received from the various parties involved.
To assess the extent to which the Bureau and other federal statistical  agencies followed data dissemination practices that the National  Academy of Sciences’ National Research Council (NRC) recommended in  its 2005 report, Principles and Practices for a Federal Statistical Agency,  we surveyed officials at 14 federal statistical agencies. (NRC prepared the  report to assist statistical agencies in making their products as sound as  possible.)
Specifically, we surveyed officials at the Bureau, and 13 additional federal  statistical agencies to collect information on the procedures they followed  when releasing data. These 14 agencies comprise the Interagency Council  on Statistical Policy, a body that coordinates federal statistical work and  advises Office of Management and Budget (OMB) on statistical matters.  The 14 agencies are:  1. Bureau of Economic Analysis, U.S. Department of Commerce  2. Bureau of Justice Statistics, U.S. Department of Justice  3. Bureau of Labor Statistics, U.S. Department of Labor  4. Bureau of Transportation Statistics, U.S. Department of Transportation  5. Bureau of the Census, U.S. Department of Commerce  6. Economic Research Service, U.S. Department of Agriculture  7. Energy Information Administration, U.S. Department of Energy  8. National Agricultural Statistics Service, U.S. Department of Agriculture  9. National Center for Education Statistics, U.S. Department of Education  10. National Center for Health Statistics, U.S. Department of Health and  Human Services  11. Office of Environmental Information, Environmental Protection  Agency  12. Office of Research, Evaluation, and Statistics, Social Security  Administration  13. Science Resources Statistics Division, National Science Foundation  14. Statistics of Income Division, Internal Revenue Service, U.S.  Department of the Treasury  In surveying the agencies, we reviewed relevant documents such as  agency policy manuals, and interviewed key officials who included,  depending on the agency, top management officials, chief statisticians, as  well as management staff from program, communications, or public affairs  offices.
We compared the information they provided us to certain practices that  the NRC has determined are important to federal statistical agencies in the  successful conduct of their missions. Specifically, we focused on two NRC  practices (1) wide dissemination of data, and (2) a strong position of  independence, because the 13 guidelines or elements associated with  these two practices are particularly important for data dissemination. The  first practice, the wide dissemination of data, is associated with the  mechanics of making the information available to the public, including the  media for releasing the information, as well as how it is formatted and  archived. The elements of the second practice, a strong position of  independence, are essential for maintaining the credibility of statistical  agencies, as well as for providing an unimpeded flow of information to  data users.
To obtain a broader perspective on the governmentwide framework for  helping to ensure data quality, we also interviewed OMB officials about  OMB’s role in coordinating and overseeing the data dissemination  activities and reviewed appropriate OMB documents such as Statistical  Policy Directive Number 2 and Number 3.
We conducted our work between March 2005 and April 2006 in accordance  with generally accepted government auditing standards.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the individual named above, Robert Goldenkoff, Assistant  Director, as well as Timothy Wexler, April Thompson, Robert Parker, Jay  Smale, Michael Volpe, Andrea Levine, and Amy Rosewarne made key  contributions to this report.
section_title: Related GAO Reports
paragraphs: Information Quality Act: National Agricultural Statistics Service  Implements First Steps, but Documentation of Census of Agriculture  Could Be Improved. GAO-05-644. Washington, D.C.: September 23, 2005.
Data Mining: Agencies Have Taken Key Steps to Protect Privacy in  Selected Efforts, but Significant Compliance Issues Remain. GAO-05-866.  Washington, D.C.: August 15, 2005.
Data Quality: Improvements to Count Correction Efforts Could Produce  More Accurate Census Data. GAO-05-463. Washington, D.C.: June 20, 2005.
Data Quality: Census Bureau Needs to Accelerate Efforts to Develop and  Implement Data Quality Review Standards. GAO-05-86. Washington,  D.C.: November 17, 2004.
Decennial Census: Methods for Collecting and Reporting Hispanic  Subgroup Data Need Refinement. GAO-03-228. Washington, D.C.: January  17, 2003.
Decennial Census: Methods for Collecting and Reporting Data on the  Homeless and Others without Conventional Housing Need Refinement.  GAO-03-227. Washington, D.C.: January 17, 2003.  2000 Census: Refinements to Full Count Review Program Could Improve  Future Data Quality. GAO-02-562. Washington, D.C.: July 3, 2002.