Data Quality: Expanded Use of Key Dissemination Practices Would Further Safeguard the Integrity of Federal Statistical Data
Released: 2006-06-05T00:00:00

Letter

Background

In September 2003, the Bureau broke from its tradition of releasing its Income and Poverty Estimates on a Tuesday or Thursday at a news conference at the National Press Club (see table 1). The data were instead released at a news conference on a Friday at Bureau Headquarters in Suitland, Maryland. Although the Bureau provided the media and other attendees with bus service from the National Press Club to Suitland, because the data showed that poverty levels had risen, some data users expressed concern that the change in day and location was an attempt to suppress unfavorable information by releasing it at a more remote location and before a weekend, when the public tends to pay less attention to the news.

The Income and Poverty Estimates, like other kinds of federal statistical information, provide key measures of the health and well-being of our society. As a result, the data need to be accurate, timely, accessible, relevant, and objective. At the same time, according to NRC, the manner in which agencies release the data is also important, and needs to be free from even an appearance of bias and political manipulation. Failure to meet this goal could undermine public confidence in the information and erode an agency’s credibility.

That said, although various guidance and laws have been developed to safeguard the overall quality of federal data, few governmentwide provisions directly address the data dissemination process itself, and agencies have largely been left to develop their own practices. For example, while OMB’s Statistical Policy Directive Number 3, “Compilation, Release, and Evaluation of Principal Federal Economic Indicators” provides detailed guidance on the dissemination of data, it only applies to 38 market sensitive principal economic indicators. Statistical Policy Directive Number 3 is highly regarded in the statistical community because it provides statistical agencies with comprehensive data dissemination guidance, requiring agencies to, among other actions, promptly release data according to an established schedule, and announce and fully explain any schedule changes in advance.

Under the Information Quality Act, OMB was required to issue governmentwide guidelines that provide policy and procedural guidance to federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information disseminated by federal agencies. OMB’s guidelines, issued in final form in February 2002, directed agencies covered by the act (statistical agencies and most others) to issue their own quality guidelines. OMB’s guidelines imposed certain core responsibilities on agencies, including incorporating quality into their information dissemination practices. OMB noted that quality consists of several dimensions, including objectivity (which focuses on whether the disseminated information is accurate, reliable, and unbiased in presentation and substance).

More generally, OMB helps ensure that the activities of the statistical agencies are in line with federal statistical policy by coordinating agency budget requests and interagency groups working on statistical issues, issuing statistical standards, and reviewing agency requests to collect information.

This report is the latest of several studies we have issued on the quality of federal data. See Related GAO Reports at the end of this report for a list of selected products we have issued to date.

Release of Income and Poverty Estimates Adhered to Most of the Bureau’s Data Dissemination Practices

While not all of the Bureau’s data dissemination practices are documented, we were able to determine through discussions with Bureau officials and review of available documentation, that the Bureau adhered to most of its long-standing data release practices. In changing the date and location of the 2003 and subsequent releases of the Income and Poverty Estimates, the Bureau did depart from its tradition of releasing this information on a Tuesday or Thursday at a news conference at the National Press Club. That said, under the Bureau’s documented data dissemination practices (1) there is no requirement for the Bureau to release this information at a particular location on a given day and, (2) no particular official is designated authority to choose the release date and location. Bureau officials stated that the date of the 2003 release was changed from September 23rd to September 26th for several reasons, including delays in producing a companion report on supplemental measures of expenditures, consumption, and poverty that was to be released at the same time. Also, the 2004 and 2005 estimates were released a month earlier than in prior years to coincide with the release of data from the American Community Survey. The documented practices for disseminating the Income and Poverty Estimates are contained in a memo that is 21 years old so the Bureau is updating them, to among other things, reflect current technology.

The Bureau’s Income and Poverty Estimate Dissemination Practices Are Derived from Several Sources

The Bureau has several sources of documented, agencywide practices for disseminating data to the public. For example, in accordance with OMB’s guidelines for implementing the Information Quality Act, the Bureau developed its own set of quality guidelines that include provisions aimed at ensuring the objectivity and integrity of its data. The Bureau also has a series of data dissemination practices available on its Intranet site and it has issued four standards governing the dissemination of data products, including Dissemination of Census and Survey Data Products. We found that the only documented practices specific to the release of the Income and Poverty Estimates are contained in a 1985 memorandum that was included as one of several appendixes to the Bureau’s Administrative Manual. The manual provides Bureau policy on the release of data and guidance for divisions to follow in responding to requests for such information.

The 1985 memorandum, which was signed by the Director of the Census Bureau at the time, includes eight broad steps, covering the process for disseminating the Income and Poverty Estimates. The eight steps include the time period from approval of the report content up to and including distributing the report at the press conference. 1. The Associate Director for Demographic Fields approves the final content of the report prepared by the Population Division after review by the Statistical Methods Division. 2. The Public Information Office receives a copy of the final content to draft a press release. This draft release is approved within the Census Bureau, by the Public Affairs Specialist in the Under Secretary for Economic Affairs’ office, and by the Commerce Department’s newsroom. 3. The report is prepared for camera-ready form. 4. Camera-ready copy is sent to the printer. 5. When the completion time for this report is known, the Census Bureau establishes the release date and time with Commerce Department concurrence. 6. Approximately 48 hours before report release date and time, the Census Bureau briefs the Deputy Secretary for Economic Affairs on the principal findings. 7. The Census Bureau makes the report and accompanying press release available to the media on the established date at 9 a.m. for 10 a.m. release. 8. The Census Bureau distributes the report and press release to the Congress and the OMB at the same time as the media.

Release of 2003 and Subsequent Income and Poverty Estimates

In releasing the 2003 Income and Poverty Estimates, the Bureau adhered to most of its data dissemination practices. The change in the date and location of the 2003 and subsequent releases of the Income and Poverty Estimates was a departure from the Bureau’s tradition of releasing this information on a Tuesday or Thursday at a news conference at the National Press Club. That said, under the Bureau’s documented data dissemination practices there is no requirement for the Bureau to release this information at a particular location on a given day. Based on our review of available documentation and our interviews with officials involved with the Income and Poverty Estimates, the Bureau followed the steps in the 1985 memo in the 2003, 2004 and 2005 releases, with the exception of the release time as previously described.

While the Bureau complied with its documented practices for the dissemination of the Income and Poverty Estimates, they lacked specificity. For example, clear and specific documentation does not exist for how and when the release date and location are to be determined for the Income and Poverty Estimates and who should make those decisions. In actuality, as discussed in greater detail subsequently, in 2003, the Director of the Census Bureau chose the location and the Associate Director for Communications chose the date. However, because this was not thoroughly documented (the 1985 memo only provides general guidance), it is unclear to the public who made these decisions and how they were made.

Furthermore, Bureau officials told us that they did not retain any internal memos or e-mails that documented the decision to change the 2003 Income and Poverty Estimates release, which would have provided evidence to support the Bureau’s narrative of the events leading up to the release.

According to the Bureau, Delays in the Companion Report Caused Change in Timing of the 2003 Release

Based on our review of available Bureau documents and interviews with key Bureau officials, several factors led to the change in the timing of the release of the 2003 and subsequent Income and Poverty Estimates.

The Chief of the Census Bureau’s Housing and Household Economic Statistics Division at the time of the 2003 release of Income and Poverty Estimates, and other senior officials we spoke to, stated that the 2003 Income and Poverty Estimates release was different from years past because the Bureau decided earlier that year to issue the report at the same time as a multi-agency report on supplemental measures of expenditures, consumption, and poverty. This decision was made before the findings of the Income and Poverty Estimates report were known. Bureau officials stated that although the original target date for releasing both reports was September 23, 2003, complications with finalizing the supplemental measures report kept it from being ready for release on that day.

According to Bureau officials and documents we reviewed, because the supplemental measures of expenditures, consumption, and poverty report involved several statistical agencies, there was a different clearance process than that used for the Income and Poverty Estimates report. As a result, while the Bureau had completed its review of the latter report, all the members of a steering committee still needed to review the report on supplemental measures.

At the same time, based on our discussions with Bureau officials involved with the Income and Poverty Estimates report, as well as available documents, the Commerce Department’s Under Secretary for Economic Affairs wanted to release both reports simultaneously in an effort to broaden the public’s understanding of social well-being. The Under Secretary’s decision was consistent with the Bureau’s ongoing effort to provide alternative estimates of poverty, which itself stemmed from a 1995 report by the National Academy of Sciences that recommended revising how poverty is measured.

Because of the additional time required to clear the supplemental measures report, Bureau officials responsible for the Income and Poverty Estimates asked for a later date to issue their report. Consequently, the Bureau’s Associate Director for Communications, with the Director’s consent, scheduled Friday, September 26, 2003, as the release date for the Income and Poverty Estimates, and both reports were issued on that date.

Under the Bureau’s guidance for dealing with the media, Census Bureau analysts are to arrange their work schedules to be available for inquiries for 2 to 3 days after a data release. This is why, prior to 2003, the Bureau tended to release the Income and Poverty Estimates earlier in the week: it obviated the need for analysts to work on the weekend. Additionally, Bureau officials said that because of the Internet and cable television, the news cycle is no longer viewed as a cycle and has instead become a 24 hours a day, 7 days a week operation. Thus, many of the media’s inquiries occur the same day as the data are released. While it seldom does so, the Bureau has released other reports on Fridays, such as its 2001 health insurance report.

For the 2004 and 2005 releases of Income and Poverty Estimates, the data were released in August at the same time as data from the American Community Survey. Bureau officials reported the Income and Poverty Estimates (which come from the Bureau’s Current Population Survey) are one of several sources of income and poverty information issued by the Bureau. Starting in 2003, the Bureau began releasing income and poverty information from the American Community Survey, which produces data independent from the Current Population Survey. Bureau officials reported that for methodological and other reasons, estimates from the Current Population Survey, in some cases, did not match estimates from the American Community Survey, causing confusing press coverage. In August 2004, when the Bureau released the two data sets at the same time, the press release that accompanied the estimates explained why the two sets of numbers might not match. (According to Bureau officials, the plans to move the release date from September 2004 to August 2004 were in place well before the actual release.) Going forward, the Bureau plans to continue its practice of releasing the American Community Survey data and the Income and Poverty Estimates simultaneously around the last Thursday in August.

According to the Bureau, Several Factors Affected the Change in Location of the 2003 Release

According to a senior official we interviewed in the Bureau’s Public Information Office, the location of the 2003 Income and Poverty Estimates news conference was changed from the National Press Club in Washington, D.C., at the request of the Director of the Census Bureau, to help raise awareness of the Bureau’s new headquarters building, which was under construction. The groundbreaking ceremony at the new site on the Bureau’s campus in Suitland, Maryland, had taken place several weeks earlier, and a Bureau official reported the Director wanted the media to see the improvements the Bureau was making at its headquarters location, as well as to foster a spirit of good feeling, and highlight how Bureau officials hoped that the new building would help improve the morale of Bureau employees. The Bureau provided bus service for attendees from the original location at the National Press Club in downtown D.C., to Bureau headquarters in Suitland, a distance of around 8 miles.

Additionally, according to the Bureau’s Associate Director for Strategic Planning and Innovation, the location of the news conference is no longer as relevant as it once was because of changes in technology. The 2003 news conference was broadcast in real time via the Internet, and materials were made available on the Bureau’s Web site. The Associate Director for Strategic Planning and Innovation noted that because of these advances and accommodations, news media on-location attendance has declined over recent years. Yet, overall media participation has increased via the availability of Web casts, satellite-feed transmissions and telephone-audio access. Consequently, the Suitland, Maryland headquarters is now the primary location for this annual news conference.

Bureau Officials Made Key Decisions on Releasing Income and Poverty Estimates

Because the Bureau did not maintain a written record of the release decision, a precise list of the personnel involved and time line of events is unavailable. However, according to the Bureau officials we interviewed, the following Bureau employees were involved in the process for releasing the Income and Poverty Estimates in 2003: Director of the Census Bureau; Deputy Director/Chief Operating Officer; Chief of the Bureau’s Housing and Household Economic Statistics Assistant Division Chief for Income, Poverty, and Health Statistics; Associate Director for Demographic Programs, now serving as the Associate Director for Strategic Planning and Innovation; Associate Director for Communications; Staff from the Bureau’s Housing and Household Economic Statistics Staff from the Bureau’s Administrative and Customer Services Division; Chief and Deputy Chief of the Bureau’s Public Information Office.

Bureau officials said that prerelease access to the Income and Poverty Estimates is tightly controlled because of the possible economic impact of the data. They stated its contents are shared with staff on a need-to-know basis, where only those individuals who are involved with drafting the report or the accompanying press release have access to the information. They noted further that key steps in preparing and releasing the report included the following: 1. Program staff from the Bureau’s Housing and Household Economic Statistics Division drafted the Income and Poverty Estimates report. 2. A branch chief reviewed and approved the draft followed by the Associate Division Chief, the Division Chief, and ultimately the Associate Director for Demographic Programs, who reports to the Bureau Director. These senior officials reviewed the report for such things as clarity and presentation. 3. When the content of the report was finalized, the Bureau’s Public Information Office was sent a copy so it could draft a press release. 4. The final draft was sent to the Bureau’s Administrative and Customer Service Division, which designed the tables and figures, edited the text, and prepared a camera-ready version of the report for printing.

According to Bureau officials, the Department of Commerce had only limited access to information from the Income and Poverty Estimates report before it was issued, and Commerce officials played no role in the decision-making process surrounding its release. For example, Commerce’s Office of Public Affairs reviewed the press release that accompanied the report and thus had access to some of the numbers as well as the key findings in the report. However, the office did not have access to any of the tables that are placed on the Internet. (According to the Bureau, Commerce usually provides a “hook” for the news media. In 2003, the press release was issued Friday, September 26, and noted, on the first line, that the nation’s poverty rate rose from 11.7 percent in 2001 to 12.1 percent in 2002.) Moreover, the Bureau considers the press release part of the report and holds it to the same standards for statistical quality as the report itself.

Additionally, according to Bureau officials, the Division Chief and the Assistant Division Chief briefed the Director of the Census Bureau on the report about a week before the September 26, 2003, press conference. Commerce’s Under Secretary for Economic Affairs was briefed a day or two before the press conference and the Under Secretary’s staff were provided with the final report at that time. (The report was also provided to the Council of Economic Advisers the afternoon before the press conference.)

The Bureau Is Updating Its Practices for Releasing the Income and Poverty Estimates

The then Chief of the Census Bureau’s Housing and Household Economic Statistics Division told us the Bureau is updating its practices for releasing the Income and Poverty Estimates. The official stated that the Bureau was prompted to revisit the 1985 memo by the fact the memo does not include all of the Bureau’s long-standing data dissemination practices; that some of the practices in the 1985 memo are obsolete given the age of the guidance; and the rise of the Internet and other technological advances have had an effect. The official added that the process for releasing Income and Poverty Estimates has become more formalized over time.

Bureau officials began drafting these revisions after the 2004 release. In addition to updating the obsolete practices, Bureau officials stated they planned to document the current practice of combining the Income and Poverty Estimates release with the American Community Survey release. The Bureau plans to issue its updated practices prior to the next release of the Income and Poverty Estimates expected in August 2006.

Governmentwide Guidance Is Being Developed That May Improve Statistical Agencies’ Data Dissemination Practices

Most of the 14 statistical agencies we reviewed reported general adherence to NRC’s guidance, important for (1) the wide dissemination of data, and (2) maintaining a strong position of independence, although there were some notable gaps. OMB, in concert with the statistical agencies, has developed draft guidance on the release and dissemination of statistical products that, according to OMB officials, parallel NRC’s guidance. To the extent it is comparable to NRC’s guidance and other widely accepted procedures for disseminating data, the proposed OMB directive could promote more consistent adherence to practices that promote broader dissemination of statistical data and enhance the data’s credibility.

Most Agencies’ Data Dissemination Practices Generally Adhered to NRC’s Guidance for Disseminating Information and Maintaining Their Independence

According to NRC, statistical agencies must have “vigorous and well- planned dissemination programs to get information into the hands of users who need it on a timely basis.” Attributes of a good dissemination program include using a variety of mechanisms to inform the widest possible audience about available data products and how to acquire them. Agencies should also have arrangements for archiving the information so that it is available for future use, as well as a publications policy that describes, among other things, the types of data products that will be made available, the frequency of their release, and the audiences they serve.

NRC also notes that a statistical agency needs to be politically independent; that is, it “must be impartial and avoid even the appearance that its collection, analysis, and reporting processes might be manipulated for political purposes. . . .” Elements of this practice include having the authority for decisions associated with the scope, content, and publication of the data, as well as the authority for the selection and promotion of professional, operational, and technical staff.

As shown in table 2, the data dissemination procedures of the 14 statistical agencies we reviewed included elements that were generally aligned with NRC’s guidance for the wide dissemination of data and maintaining a strong position of independence.

Twelve or more of the agencies reported having data dissemination practices possessing four of the five elements related to the wide dissemination of data. All 14 agencies reported their data dissemination practices followed NRC’s guidance for (1) having multiple avenues for disseminating data, (2) releasing data in a variety of formats, and (3) having policies to guide what data should be preserved and how it should be archived. Similarly, 12 or more of the agencies’ dissemination practices had characteristics associated with five of the eight elements corresponding to maintaining a strong position of independence. These elements include (1) adherence to predetermined data release schedules, and (2) authority to make decisions over the scope, content, and frequency of the data compiled, analyzed, or published.

A greater number of agencies’ data dissemination practices lacked certain elements important for maintaining a strong position of independence. An example of one of these elements is NRC’s guidance suggesting statistical agencies should have the “authority to release statistical information and accompanying materials (including press releases) without prior clearance by department policy officials” so there is “no opportunity for or perception of political manipulation of any of the information.”

However, 10 of the 14 selected agencies reported varying degrees of clearance required by department officials. For example, at 2 agencies, the department rather than the statistical agency releases statistical information. Other agencies have the authority to release statistical information, except for press releases, without departmental clearance, although in some cases, the department’s clearance process is limited to reviewing the grammar, punctuation, and other editorial aspects. (Among the agencies in our review, 11 agencies use press releases; 1 of these 11 agencies first publishes data from all of its major programs via a press release; and the 3 remaining agencies reported they do not use press releases as a vehicle to disseminate data.) With other agencies the clearance process is more involved. For example, one agency said it summarizes the data for the press release making sure it is fair and complete, while officials at the departmental level might insert comments from the cabinet secretary into the release. Further, 6 of the 14 agencies lacked dissemination policies that promote the regular and frequent release of major findings from an agency’s statistical program.

As for the Bureau, officials reported that their agency generally adhered to NRC’s recommended guidelines. A notable gap was that Bureau officials did not report adhering to announcing and explaining modifications to a customary release schedule in advance (7b in table 2). Bureau officials also lacked the authority to release statistical information and accompanying materials (including press releases) without prior clearance by department policy officials (8 in table 2). Also, while the Bureau’s established publications policy describes the frequency of release of data collection programs, the Bureau reported the policy does not describe the types of reports to be made available, the data releases to be made available, or the audience to be served (4a-c in table 2).

OMB Is Developing Governmentwide Data Dissemination Guidance That Could Help Strengthen the Credibility of Statistical Information

OMB has been working with the federal Interagency Council on Statistical Policy to develop guidance for the release and dissemination of statistical products. According to OMB officials, the guidance is intended to help ensure statistical products are policy-neutral, timely, and accurate. OMB officials told us their directive is similar to the NRC’s recommended practices, as well as to OMB’s Statistical Policy Directive Number 3, which applies only to the 38 market-sensitive principal economic indicators produced by the Departments of Agriculture, Commerce, Labor, and Treasury, as well as the Federal Reserve Board. However, OMB officials told us this new directive will not be as stringent as Statistical Policy Directive Number 3, because the data covered by the directive are released less frequently than the principal economic indicators, and the data are not considered to be market-sensitive. OMB expects to issue the directive for public comment in the spring of 2006.

To the extent OMB’s dissemination directive appropriately addresses the principles underlying NRC’s guidance and Statistical Policy Directive Number 3, the directive could enhance the quality and credibility of federal statistical data, in part, by replacing the patchwork of agency- specific guidance with a more transparent, commonly accepted, and consistently applied framework for disseminating data. For example, OMB’s directive could help promote more consistent adherence to key data release practices such as the wide dissemination of data and maintaining an agency’s independent position.

As noted in the previous section, the dissemination procedures at several statistical agencies we examined lacked elements important for these practices, including (1) authority to release statistical information without prior clearance by department policy officials, (2) data dissemination policies that foster the frequent release of major findings from an agency’s statistical programs, and (3) an established publications policy that describes the types of reports and other data releases to be made available. As a result, their data products could be better protected, with the directive, from the appearance of, or actual political involvement. More specifically, OMB’s new directive could address how best to address the gaps that exist between agencies’ data dissemination practices on the one hand, and NRC’s guidance on the other.

As OMB moves forward with its new directive, our interviews with OMB and statistical agency officials, as well as our past work on data quality guidance and internal control standards, identified the following questions that will be important for OMB’s dissemination directive to consider: Coverage: What will be covered by the directive?—principal statistical agencies only?—the statistical functions of all agencies?—or only statistical products? It will be important for OMB’s directive to clearly define what it does and does not cover so that both statistical agencies and their parent organizations share the same understanding of their respective authorities, and help ensure dissemination procedures are consistently implemented. Certain roles, responsibilities, and processes need to be clarified as well. Indeed, officials at two statistical agencies we spoke with said there is ambiguity as to whether a statistical press release is a statistical product and if so, whether statistical agencies can issue them with or without first getting releases cleared at the departmental level. Additionally, OMB has issued a number of guidelines, directives, and standards on federal statistics. Are there any gaps and overlaps among them, and can they be better integrated?

Documentation: To what extent, and how, should agencies document their data dissemination procedures and policies, and how often should they be reviewed and updated? The agencies we examined did not always document their processes for disseminating statistical data, relying instead on professional practice. However, as NRC points out: “Although a long- standing culture of data quality contributes to professional practice, an agency should also seek to develop and document standards through an explicit process.”

Moreover, documented guidance would lend more transparency to the data dissemination process, and thus provide a basis for agencies to explain their dissemination decisions to policy makers, news media, and the public. Indeed, an OMB official told us that Statistical Policy Directive Number 3 is a useful tool for explaining to high level policy officials the procedures agencies must follow to maintain the integrity of the data, and why the officials may not access principal economic data before it is released to the public nor comment on it until after its release. Documented guidance could also help ensure continuity in the face of employee turnover.

The importance of documenting agencies’ data dissemination practices can be seen in the Bureau’s experience in releasing data from the 2000 Census on the homeless and others without conventional housing, when the Bureau was criticized for shifting its position on reporting components of this population. In our 2003 report, we noted that although the Bureau’s decision stemmed from its concerns over the reliability of the underlying data, the Bureau’s lack of documented, clear, transparent, and consistently applied guidance governing the release of data from the 2000 Census hampered the Bureau in explaining its actions. Had such guidance been in place, it could have helped the Bureau be more accountable and consistent in its dealings with the public, and helped to ensure that the Bureau’s decisions both were, and appeared to be, totally objective.

Flexibility: How much leeway should agencies have in implementing OMB’s directive? Agency officials we spoke with noted the different missions of the various statistical agencies and cautioned against a one- size-fits-all approach. As a result, it might not be practical to require all agencies to meet predetermined release dates because it could lead to additional workload burdens and staffing issues.

Monitoring: How will OMB ensure agencies comply with its directive? Indeed, the effectiveness of the policies and procedures laid out in OMB’s directive will rest in large part on the extent to which agencies and their parent departments adhere to them. Related questions include whether there should be a regular assessment of agencies’ compliance, and if so, how often should it occur, and whether this should be done by OMB, or by the agencies through a self-assessment.

Posting Data: Should agencies’ dissemination policies include written guidance for releasing information via specific channels? Indeed, although NRC’s guidance calls on agencies to disseminate data using a variety of outlets so that the information reaches as wide an audience as possible, should agencies also have a standard set of conduits where the public will know an agency’s data will always be available? Such conduits might include, among others, an agency’s Web site. Because all of an agency’s data products would be, at a minimum, available from a central point of access, it could help strengthen an agency’s credibility because the public would always know where to find it.

Conclusions

A key lesson learned from the Bureau’s experience is the importance of fully documented, specific practices for maintaining the integrity of data products, and by extension, the credibility of the agencies that release them. Thus, as the Bureau updates its practices for releasing the Income and Poverty Estimates, it will be important for the Bureau to more thoroughly document its dissemination procedures so they are clear to the public.

Further, OMB’s efforts to develop governmentwide guidance on data dissemination is a positive step toward enhancing the credibility of federal statistical data, especially to the extent the directive mirrors NRC’s guidance and Statistical Policy Directive Number 3, as it would replace each statistical agency’s procedures with a more transparent, commonly accepted, and consistently applied framework for disseminating information. As OMB works to complete its directive, it will be important for it to pay particular attention to those elements dealing with the wide dissemination of data and maintaining a strong position of independence that, our survey of statistical agencies suggests can be adhered to by a greater number of agencies. Likewise, OMB should also consider other aspects of agencies’ data dissemination efforts that could make its directive more comprehensive.

Recommendations for Executive Action

To help improve the Bureau’s data dissemination practices and thus enhance the agency’s actual and perceived position of independence, we recommend that the Secretary of Commerce direct the Bureau to, as part of its efforts to update its practices for releasing the Income and Poverty Estimates, fully document its key data dissemination practices for releasing the Income and Poverty Estimates.

Further, to help improve governmentwide data dissemination practices that would further safeguard the integrity of federal statistical data, we recommend that the Director of OMB ensure his agency, in completing its draft directive on the release of federal statistical products, considers whether and how to address areas where our survey indicates there are gaps between NRC’s existing guidance and agencies’ practices. These areas include the extent to which agencies should have (1) full authority to release statistical information without prior clearance by their respective departments, (2) data dissemination policies that foster the frequent release of major findings from agency’s statistical programs, and (3) an established publications policy describing the types of reports and other releases an agency has available.

We are also recommending that the Director of OMB direct his agency to include in its directive additional elements and characteristics important for agencies’ data dissemination practices, including (1) clear definitions of what is, and what is not covered by the directive, (2) the extent to which agencies should document their data dissemination guidance and how often the guidance should be reviewed, (3) the amount of flexibility agencies have in implementing OMB’s guidance, (4) procedures for monitoring agencies’ adherence to its directive, and (5) the feasibility of requiring agencies to distribute data products through a standard set of channels as well as through other outlets as appropriate, so that the public will always know at least one source it can turn to and obtain agency data.

Agency Comments and Our Evaluation

In written comments on a draft of this report, Commerce neither agreed nor disagreed with our recommendation for the Bureau to fully document its key data dissemination practices for releasing the Income and Poverty Estimates. However, Commerce reiterated the point we made in our report that the Bureau is updating its practices for releasing the Income and Poverty Estimates. Commerce noted that the updated document—which details the dissemination practices for the Income and Poverty Estimates—is under review. The Bureau plans to issue it prior to the next release of the Income and Poverty Estimates expected in August 2006. Commerce also provided some technical corrections and suggestions where additional context might be needed, and we revised the report to reflect these comments as appropriate. Commerce’s comments are reprinted in their entirety in appendix II.

The Director of OMB did not have comments on our recommendations to them. However, OMB officials provided suggestions for technical corrections and we revised the report to reflect these suggestions as appropriate.

As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies of the report to interested congressional committees, the Director of the U.S. Census Bureau, and the Director of the Office of Management and Budget. Copies will be made available to others on request. This report will also be available at no charge on GAO’s home page at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6806 or farrellb@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs can be found on the last page of this report. GAO staff that made major contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To address the extent to which the U.S. Bureau of the Census (Bureau) adhered to its dissemination practices for the release of the 2003 annual Income and Poverty Estimates and subsequent releases we asked Bureau officials (in the Housing and Household Economic Statistics Division and the Bureau’s Public Information Office, among others) to identify the Bureau and Department of Commerce officials who participated in the data dissemination decisions, and interviewed the identified officials to determine their role in the decision-making process, and whether they had prerelease access to the information. We compared their actions to the Bureau’s data dissemination practices. The dissemination process includes the steps from approval of the report content up to and including public distribution of the report. Some of these practices are documented in the Bureau’s Policy and Procedures Manual, while others are undocumented practices that we identified by interviewing cognizant Bureau officials.

Because written records of key activities related to the release (e.g. e-mails, meeting agendas and notes) were either not retained or never created, much of our reconstruction of the release was based on interviews with the officials involved. We interviewed many of these officials both as a group (by department) and individually to obtain as complete a picture of the events as possible, and corroborated the information we received from the various parties involved.

To assess the extent to which the Bureau and other federal statistical agencies followed data dissemination practices that the National Academy of Sciences’ National Research Council (NRC) recommended in its 2005 report, Principles and Practices for a Federal Statistical Agency, we surveyed officials at 14 federal statistical agencies. (NRC prepared the report to assist statistical agencies in making their products as sound as possible.)

Specifically, we surveyed officials at the Bureau, and 13 additional federal statistical agencies to collect information on the procedures they followed when releasing data. These 14 agencies comprise the Interagency Council on Statistical Policy, a body that coordinates federal statistical work and advises Office of Management and Budget (OMB) on statistical matters. The 14 agencies are: 1. Bureau of Economic Analysis, U.S. Department of Commerce 2. Bureau of Justice Statistics, U.S. Department of Justice 3. Bureau of Labor Statistics, U.S. Department of Labor 4. Bureau of Transportation Statistics, U.S. Department of Transportation 5. Bureau of the Census, U.S. Department of Commerce 6. Economic Research Service, U.S. Department of Agriculture 7. Energy Information Administration, U.S. Department of Energy 8. National Agricultural Statistics Service, U.S. Department of Agriculture 9. National Center for Education Statistics, U.S. Department of Education 10. National Center for Health Statistics, U.S. Department of Health and Human Services 11. Office of Environmental Information, Environmental Protection Agency 12. Office of Research, Evaluation, and Statistics, Social Security Administration 13. Science Resources Statistics Division, National Science Foundation 14. Statistics of Income Division, Internal Revenue Service, U.S. Department of the Treasury In surveying the agencies, we reviewed relevant documents such as agency policy manuals, and interviewed key officials who included, depending on the agency, top management officials, chief statisticians, as well as management staff from program, communications, or public affairs offices.

We compared the information they provided us to certain practices that the NRC has determined are important to federal statistical agencies in the successful conduct of their missions. Specifically, we focused on two NRC practices (1) wide dissemination of data, and (2) a strong position of independence, because the 13 guidelines or elements associated with these two practices are particularly important for data dissemination. The first practice, the wide dissemination of data, is associated with the mechanics of making the information available to the public, including the media for releasing the information, as well as how it is formatted and archived. The elements of the second practice, a strong position of independence, are essential for maintaining the credibility of statistical agencies, as well as for providing an unimpeded flow of information to data users.

To obtain a broader perspective on the governmentwide framework for helping to ensure data quality, we also interviewed OMB officials about OMB’s role in coordinating and overseeing the data dissemination activities and reviewed appropriate OMB documents such as Statistical Policy Directive Number 2 and Number 3.

We conducted our work between March 2005 and April 2006 in accordance with generally accepted government auditing standards.

Appendix II: Comments from the Department of Commerce

Appendix III: GAO Contact and Acknowledgments

GAO Contact

Acknowledgments

In addition to the individual named above, Robert Goldenkoff, Assistant Director, as well as Timothy Wexler, April Thompson, Robert Parker, Jay Smale, Michael Volpe, Andrea Levine, and Amy Rosewarne made key contributions to this report.

Related GAO Reports

Information Quality Act: National Agricultural Statistics Service Implements First Steps, but Documentation of Census of Agriculture Could Be Improved. GAO-05-644. Washington, D.C.: September 23, 2005.

Data Mining: Agencies Have Taken Key Steps to Protect Privacy in Selected Efforts, but Significant Compliance Issues Remain. GAO-05-866. Washington, D.C.: August 15, 2005.

Data Quality: Improvements to Count Correction Efforts Could Produce More Accurate Census Data. GAO-05-463. Washington, D.C.: June 20, 2005.

Data Quality: Census Bureau Needs to Accelerate Efforts to Develop and Implement Data Quality Review Standards. GAO-05-86. Washington, D.C.: November 17, 2004.

Decennial Census: Methods for Collecting and Reporting Hispanic Subgroup Data Need Refinement. GAO-03-228. Washington, D.C.: January 17, 2003.

Decennial Census: Methods for Collecting and Reporting Data on the Homeless and Others without Conventional Housing Need Refinement. GAO-03-227. Washington, D.C.: January 17, 2003. 2000 Census: Refinements to Full Count Review Program Could Improve Future Data Quality. GAO-02-562. Washington, D.C.: July 3, 2002.