Federal Statistical System: Agencies Can Make Greater Use of Existing Data, but Continued Progress Is Needed on Access and Quality Issues
Released: 2012-03-26T00:00:00

Letter

Background

In contrast to many other countries, the United States does not have a primary statistical agency. Instead, the statistical system is decentralized, with statistical agencies generally located in different government departments. This structure keeps statistical work within close proximity to the various cabinet-level departments that use the information. There are 13 federal agencies, referred to as the principal statistical agencies, which have statistical activities as their core mission. These agencies conduct much of the government’s statistical work, though there are more than 80 additional federal agencies that carry out some statistical work in conjunction with their primary missions. The 13 principal statistical agencies are all attached to a cabinet-level department or an independent agency that reports to the president. As shown in figure 1, they are located at different levels within their respective departments and agencies.

For fiscal year 2011, $6.83 billion was requested for statistical work, which includes the collections in our scope as well as work that focuses on entities other than households and individuals, such as businesses and farms. This amount is about 0.2 percent of that year’s total federal budget request. Much of this work is concentrated in the 13 principal statistical agencies, which account for approximately 40 percent of requested funding. The budget request for the Census Bureau is among the highest of the principal statistical agencies. Excluding funding related to the decennial census, the fiscal year 2011 budget request for the Census Bureau was $558 million. In addition to conducting its own statistical activities, the Census Bureau also performs statistical work for other agencies on a reimbursable basis.

Most of the collections in our scope have relatively modest annual costs. In a sample of 112 information collections that fell within our scope and were active as of September 22, 2011, the majority cost less than $500,000 annually, and fewer than one in five cost more than $1 million annually. There are a few, more expensive, large and broad-based collections such as the Current Population Survey and National Health Interview Survey, both of which cost tens of millions of dollars each year, and the ACS, which costs over $200 million annually (see fig. 2).

Various statutes and guidance from OMB and other entities establish standards for quality and privacy that apply to the federal statistical system. One of the most significant statutes is the PRA, which designates OMB as the coordinating body of the federal statistical system. The PRA establishes requirements that agencies must meet in order to administer information collections, and OMB must meet in overseeing the system, including that it issue guidance to agencies. Other entities also provide guidance to agencies that conduct statistical work. In addition, use of information must be balanced with protection of privacy and confidentiality. Statutes such as the Confidential Information Protection and Statistical Efficiency Act (CIPSEA) apply to the federal statistical system and focus on ensuring the privacy and confidentiality of respondents’ information. Agency-specific statutes also protect the privacy and confidentiality of data collected by those agencies. For example, Title 13 of the U.S. Code authorizes the Census Bureau to request and collect information from individuals but also guarantees the confidentiality of these data and establishes penalties for unlawfully disclosing this information. Additional statutes are described in more detail in appendix III.

OMB and Agencies Take a Number of Steps to Ensure Efficient Information Collections, Though Opportunities Exist for Refinements

OMB Uses Its Oversight Authority to Improve Efficiency

Under PRA, OMB, through its Office of Information and Regulatory Affairs (OIRA), has responsibility and broad authority to improve the efficiency and effectiveness of federal information resources.is charged with the oversight and coordination of federal agencies’ statistical activities. Specifically, these oversight functions are carried out by OIRA’s Statistical and Science Policy Branch, headed by the Chief Statistician, which includes five staff members who work closely on these oversight and coordination activities with approximately 25 other OIRA desk officers. OMB exercises four key authorities that contribute to the efficiency of the federal statistical system: In this regard, OMB

Oversight and approval of information collections: OMB generally must approve information collections that are to be administered to 10 or more people. OMB staff review agencies’ information collection requests to determine whether proposed collections meet PRA standards by assessing such factors as whether they are necessary for the mission of the agency and do not unnecessarily duplicate existing information. This review also enables OMB to identify opportunities for improvement. For example, according to OMB and agency officials, if it determines that it is necessary to ask similar questions in multiple collections, then OMB works to ensure that agencies ask them in a consistent manner, when appropriate.

Standard-setting and guidance to agencies: OMB is responsible for developing and implementing governmentwide policies, principles, standards, and guidelines related to statistical issues, such as procedures and methods for collecting data and disseminating information. Specifically, OMB issues directives, guidance, and memorandums, and provides additional information through information sessions and presentations, to guide federal data collection and promote the quality and efficiency of information collections. For example, OMB published “Questions and Answers When Designing Surveys for Information Collections,” a set of 81 questions and answers on the OMB review process for agency information collection requests required by PRA.Standards and Guidelines for Statistical Surveys, which outlines 20 standards and related guidelines for the design and methodology of statistical surveys. Finally, OMB issues memorandums focusing on various topics, with recent ones clarifying the guidance for complying with PRA and encouraging agencies to coordinate efforts to share data.

Budget development and reporting: Although agency budgets are initiated within agencies, OMB is responsible, under PRA, for ensuring that agency budget proposals are consistent with systemwide priorities for maintaining and improving the quality of federal statistics. In addition to the budgets themselves, OMB reports information to the public and Congress about the identification of key priorities through key documents. OMB annually reports on the paperwork burden federal collections impose on the public in the Information Collection Budget of the United States Government. In addition, OMB annually describes statistical program funding and proposed program changes for statistical activities in the Statistical Programs of the United States Government.

Other statistical-policy coordination activities: The Chief Statistician and staff in OMB’s Statistical and Science Policy Branch participate in both formal and informal coordination activities with agencies. OMB’s role and participation in the formal interagency committees are discussed later in this report. In general, it maintains regular contact with staff at principal statistical agencies. Additionally, OMB encourages agencies that are designing information collections to collaborate with principal statistical agencies because they can help improve survey design and methodology. For example, according to OMB and Census Bureau officials, OMB encouraged the Corporation for National and Community Service to work with the Census Bureau and BLS to sponsor a supplement to the Current Population Survey rather than a stand-alone survey. OMB indicated that sponsoring this supplement likely resulted in cost savings, improved data quality, and greater utility.

The Reliability of Information in OMB’s Information Collections Database Needs to Be Improved

One tool that OMB uses to facilitate its oversight and coordination functions under PRA is an internal system called the Regulatory Information Service Center and OIRA Consolidated Information System (ROCIS), which contains information on all active collections and those pending OMB approval. Agencies use the system to submit information collection requests. This system also facilitates OIRA’s review of the requests and underlies the information provided on the public website Reginfo.gov. Agency submissions to OMB typically include a copy of the data-collection instrument (e.g., a survey) and supporting documentation that, in a standardized form, provides information on the collection, such as the estimated annual burden hours and cost to the federal government. Further, under PRA, agencies must certify that the collection satisfies the act’s standards, for example that the collection avoids unnecessary duplication. Making this information transparent and easily accessible to other agencies facilitates coordination and can potentially help agencies avoid duplication and identify opportunities for improvement. Furthermore, OMB uses the information contained in its internal system to track reviews of information collections, and to compile quantitative data for the Information Collection Budget of the United States Government.

Despite the benefits of this electronic system, our review identified some discrepancies between the Reginfo.gov website’s data and the underlying documentation for certain key variables. Specifically, we reviewed a systematic random sample of 56 of the 555 collections in our scope and checked the reported information for annual cost to the federal government and annual burden hours. For 11 of the 56 information collections, the information on cost or burden, or both, did not match between the two sources. In cases where annual cost did not match, the differences ranged from $1,000 to $19.3 million. In cases where annual burden hours did not match, the differences ranged from 30 to almost 500,000 hours. OMB confirmed that the information in the external Reginfo.gov system is the same as in its internal ROCIS system. As a result, these discrepancies raise questions about the confidence that users can have in both the internal and external databases and may affect OMB’s ability to track information collection requests. OMB officials told us that responsibility for ensuring data reliability is shared between OMB and agencies. The Regulatory Information Service Center has issued detailed guidance to agencies on how to upload information into ROCIS, and the system has a function that allows agencies to check the completeness of data for individual information collections to ensure that no required data are missing. Entering this information is not always straightforward, however, and some interpretation of the underlying documentation may be required. The discrepancies that we identified indicate that additional actions, such as edit checks, review by an informed staff member, or increased clarification in supporting documents are necessary to ensure the reliability of Reginfo.gov and ROCIS data.

Agencies Identify Duplication and Solicit Input to Enhance Efficiency

Our analysis indicated that agencies addressed PRA standards related to duplication and public comment in their information collection requests to OMB, and in many cases went beyond the actions specifically described in PRA and related OMB guidance.

The elements of PRA most directly related to our review were identifying duplication and soliciting external input on proposed collections. To analyze agencies’ actions in these two areas, we reviewed a generalizable sample of supporting statements from 106 active statistical information collections administered to households and individuals.Each of the supporting statements we reviewed addressed those PRA standards, as required, and in many cases included detailed descriptions, the content of which we analyzed in order to identify the range of actions that agencies took.

Although agencies must address how their proposed collections meet PRA standards, the act and OMB guidance do not prescribe many specific actions that agencies need to take in addressing these standards. Regarding duplication, PRA does not dictate how agencies should address the standard. Regarding external input, PRA does require that agencies at a minimum provide notice in the Federal Register to allow the public to comment on proposed collections as well as consult with members of the public and affected agencies. OMB guidance expands somewhat on ways that agencies can address these standards, particularly in the case of surveys using statistical methods. However, just as with PRA, much is left to the discretion of agencies and little is specifically required. For example, OMB guidance states that agencies should review existing studies and consult with survey methodologists and data users.

Identifying potential duplication: Our analysis showed that agencies took various steps to comply with the PRA requirement that information collections do not unnecessarily duplicate an available information source. Specifically, based on our analysis, we estimated the following for the universe of collections in our scope:

77 percent included detailed explanations of the actions taken to identify potential duplication.

The 95 percent confidence interval for this estimate is (68, 85). not include detailed explanations were generally for information collections that had unique scopes or other characteristics that made them unlikely to duplicate existing information.

57 percent reported reviewing other surveys when looking for duplication.seven other surveys that collected information similar to that of a Current Population Survey supplement on tobacco use and explained why the data from these surveys could not replace those collected through the supplement.

For example, the National Cancer Institute identified

46 percent indicated that the agency considered administrative data as a potential source of data.

About a quarter indicated that they consulted with other entities, such as agencies, and a similar number reported that they conducted literature searches. In addition, for six of the information collections in our sample, agencies sponsored a collection in the form of a supplement to the Current Population Survey rather than creating a stand-alone survey, thus piggybacking onto another survey vehicle to potentially avoid duplication.

The 95 percent confidence interval for this estimate is (32, 59). evaluate surveys’ data quality. In order to facilitate comparisons among surveys, OMB encourages asking consistent questions, when possible, about certain characteristics such as race and ethnicity. Further, when considered from an individual’s perspective, duplication of survey questions is relatively rare. This is because in a given year a very small percentage of households are selected to participate in a single collection within our scope. The likelihood that a household would be selected for participation in more than one collection, and thus household members be asked the same question more than once, is considerably lower.

The 95 percent confidence interval for this estimate is (46, 67). and sponsoring or participating in workshops, panels, or other events.

Agencies in our sample reported making changes in response to input, potentially resulting in improvements to their information collections. For example, in response to recommendations made by the Committee on National Statistics on a Current Population Survey supplement about food security, ERS reported entering into an agreement with Iowa State University to study food security measurement issues. This collaborative project is exploring alternatives to an aspect of the supplement’s current design, which could result in alternatives to methods used to estimate food security prevalence and potentially improve measurement precision and reliability. The U.S. Geological Survey also reported incorporating changes in response to feedback on its Landsat Survey.

Agencies’ actions to find duplication and solicit input that we identified in our review, as well as others that OMB may identify, could be useful for OMB to share with other agencies that sponsor information collections. Offering more-detailed guidance in a single document that outlines different actions agencies can take to identify duplication and solicit input would help ensure that agencies are aware of the various options. It would also allow them to easily access and reference this information. OMB could include this information in one of its periodic memorandums related to compliance with the PRA. We previously reported on the importance of establishing ways to operate across agency boundaries, and promoting these actions is one way OMB can do this. Also, just as OMB’s guidance to agencies in complying with the Information Quality Act gives agencies flexibility to determine the most appropriate actions, it is important that any new guidance continue to give agencies discretion in the number and types of actions they take to identify duplication and solicit input. This is because the most appropriate actions will vary based on the characteristics of the collection.

Interagency Committees Facilitate Collaboration, but Better Communication Could Increase Effectiveness

Interagency statistical committees offer opportunities for broader collaboration to increase the efficiency of the federal statistical system. Three key committees are the Interagency Council on Statistical Policy (ICSP), the Federal Committee on Statistical Methodology (FCSM), and the Statistical Community of Practice and Engagement (SCOPE), all of which are either chaired or sponsored by OMB. Importantly, the activities of the interagency committees are consistent with key collaborative practices we identified in our previous work. For example, each of these committees has defined roles and responsibilities, and the committees serve as a vehicle for the agencies to operate across agency boundaries. Specifically, ICSP serves an advisory function to the Chief Statistician and focuses on broader issues related to the federal statistical system. In addition, ICSP provides overarching guidance to FCSM and SCOPE. FCSM investigates statistical practices and methodologies used in federal statistical programs, while SCOPE focuses on cross-agency activities of data management and dissemination. Table 1 provides an overview of these committees.

The committees study statistical issues and methods through subcommittees and working groups, most of which rely on volunteers from member agencies who take on these responsibilities in addition to their current job duties. The work of the subcommittees and working groups has been useful to other agencies. For example, an FCSM subcommittee produced a checklist that, according to OMB, is used around the world to determine whether a public-use data product sufficiently protects the confidentiality of individuals’ data.

The interagency committees use various methods to disseminate information on their activities and products, but they do not do so in a timely or comprehensive manner. The committees’ work is summarized in OMB’s annual report Statistical Programs of the United States Government, but the report does not always communicate key information about it. For example, the fiscal year 2011 report states that one of ICSP’s activities over the past year was identifying the highest- priority statistical-program improvements, but does not provide information about all of these improvements. In addition, interagency committees present information about their work at statistical seminars. For example, according to OMB officials, FCSM has presented work at the biennial FCSM Statistical Policy Seminars. Additionally, agency officials noted that members of interagency statistical committees utilize a limited-access web-based system to facilitate information sharing. Information about FCSM’s work is also posted on the committee’s website or the FedStats website. Neither ICSP nor SCOPE has a dedicated website, though OMB believes that this is not necessary or appropriate because the work of these groups is deliberative. While the FCSM website offers the potential to effectively disseminate information, it is not comprehensive or timely. For example, it provides links to the sites of various interagency and advisory committees, including three FCSM permanent working groups, but does not have pages for any of the active FCSM subcommittees. Moreover, the websites do not appear to be regularly updated with new products produced by the committees that could be useful for other agencies. For example, the subcommittee on the statistical uses of administrative data published a paper in April 2009 highlighting examples of successful data-sharing projects using administrative data for statistical purposes, but this product is not yet available on the FCSM website.

Providing more-comprehensive and timely information on interagency activities could offer benefits. As identified in our previous work, developing mechanisms to monitor and report on results is a necessary element of a collaborative relationship.committee activities and products could offer benefits to those who are not involved in committee activities, as well as committee members. Membership in the committees is made up almost exclusively of representatives from the 13 principal statistical agencies, so most agencies are not directly involved in committee activities. It makes sense that agencies that have statistics as their primary focus are the most- heavily involved, but those agencies for which statistics is a supporting function to their primary mission, and possibly academics and the broader public, could benefit from greater access to information and products related to the committees’ work and priorities. More easily accessible information would also benefit member agencies, as it would offer a centralized place to maintain committee work and communicate priorities. Much work goes into developing the committees’ products, and making them easily accessible maximizes their value.

Administrative Data Could Help Improve Federal Surveys, but Continued Progress Is Needed on Access and Quality Issues

Administrative Data Have Greater Potential to Supplement, Rather than Replace, Federal Surveys

Administrative data, typically collected to administer a program or business, are a growing source of information on individuals and households. For example, the Social Security Administration collects data on the earnings of U.S. workers from employers and the Internal Revenue Service to calculate the amount of benefits for retired workers, spouses, children, and other beneficiaries, while businesses obtain data, for example, on item and amount of purchases when customers use credit cards and store loyalty cards. According to the Census Bureau, the amount of administrative data held by private companies exceeds the amount held by the government. Researchers recently estimated that the amount of digital data in existence, which includes some types of administrative data such as retail customer databases, more than doubles every 2 years. Administrative data have been identified as an important resource for the future of the statistical system, as some of these publicly and privately held data may be analyzed or reported with survey data to yield greater value. Furthermore, the increasing capacity to store and process administrative data has facilitated this potential use.

For decades, agencies have been working to expand the use of administrative data in conjunction with data collected from surveys, but certain characteristics of administrative data make it difficult to use them to replace surveys or sections of surveys administered to households and individuals. There is interest in exploring how administrative data may be used to improve data quality, hold down costs, and reduce respondent burden. For example, as part of the redesign of the Consumer Expenditure Surveys, BLS is investigating the potential for replacing some portions of the survey with external sources of expenditure data to reduce respondent burden and potentially improve data quality. However, agencies we contacted have not replaced surveys or sections of surveys administered to households and individuals with administrative data because data: (1) are often not representative of a survey’s population of interest; (2) may not correspond to information collected through survey questions; (3) are vulnerable to program cancellation or changes; and (4) may take a long time to obtain, which delays use and in some cases could cause agencies to miss required reporting dates.

Administrative data currently show greater promise for supplementing federal surveys. Indeed, the agencies we contacted identified four major opportunities to enhance surveys with administrative data in order to create efficiencies and enhance data quality.administrative data include the following: Current uses of

Creating new data products: Agencies link survey data and administrative data to create new, more robust, statistical data products, which increases efficiency in two key ways. First, according to OMB and agency officials, agencies can use these new data products to evaluate and potentially improve federal policies and programs, especially those related to the source of the administrative data, without adding to respondent burden. Second, combining administrative data with survey data can increase efficiency by enhancing previously collected survey data. For example, the National Center for Health Statistics’s (NCHS) record-linkage program links survey data from various health-related surveys to different administrative datasets to create new data products for studying factors that influence health-related outcomes, such as disability, health care, and mortality.

Supplementing surveys’ sample frames: Using administrative data to supplement surveys’ sample frames—the sources from which a survey’s sample is drawn—can create efficiencies, reduce costs, and enhance the quality of surveys. For example, the National Household Food Acquisition and Purchase Survey uses administrative data from the Supplemental Nutrition Assistance Program to develop a sample frame of participating households to potentially include in the survey. ERS officials said that using these data to help develop the survey’s sample frame costs less than the alternative of screening a broader group of respondents to determine if they are participating. In addition, agencies can use administrative data to augment sample frames in areas where the sample is not large enough to fully support a survey. For example, the Census Bureau’s pilot project studying the potential to use ACS data as a sample frame for the National Immunization Survey used commercial data to supplement ACS data in a county that had a limited ACS sample.

Comparing data to improve survey accuracy and design: By comparing survey data to similar administrative datasets and identifying reasons for any discrepancies that may exist, agencies can improve the quality of survey data. For example, researchers identified opportunities for improving surveys’ designs and methodologies after agencies found that surveys of enrollment in health-insurance programs provided lower estimates than those compiled from administrative data. Agencies can also improve the efficiency of their surveys by using administrative data as part of nonresponse follow-up activities.

Modeling estimates: Agencies combine administrative data and survey data to create, or model, estimates that are designed to be more accurate than estimates based on survey data alone. The main benefit of modeling is that it provides the ability to produce estimates for smaller geographic areas than is possible using a survey alone. For example, the Census Bureau conducts the Small Area Income and Poverty Estimates Program to provide updated data on poverty and income, which is used to administer federal programs and allocate federal funds to local areas. The Census Bureau combines survey data from the ACS with population estimates and administrative data and has found that this approach produces consistent and reliable data more reflective of current conditions than data produced only by existing surveys.

Agencies Are Addressing Issues That Hamper Use of Administrative Data, but Additional Actions Could Facilitate Progress

Despite the benefits of using administrative data to supplement federal surveys, agencies face five key constraints related to data access and quality:

Statutory restrictions on data sharing: Federal and state statutes sometimes prohibit or limit sharing of data for statistical purposes. In cases where specified authorized uses do not include statistical use, nothing short of a statutory change can overcome the constraint. In other cases, statutes limit sharing to purposes related to program administration. For example, the 2008 Farm Bill restricts access to data on participants in certain nutrition-assistance programs to uses for the “administration or enforcement” of the programs. Similarly, the Higher Education Act of 1965, as amended, restricts federal student aid data to purposes related to the “application, award, and administration of aid.” However, agencies holding such restricted data can differ on whether statistical uses are related to program administration. The Census Bureau successfully negotiated access to the nutrition assistance data because it could demonstrate that the linked data would help the federal sponsor and state agencies develop better measures of outcomes, such as poverty, inequality, and the receipt of government transfers. Conversely, the Census Bureau was unable to gain access to the federal student aid data for statistical uses because the Department of Education did not consider that any of the planned uses related to the program’s administration.

Consent: Individuals’ consent to allow their administrative and survey data to be linked affects uses of administrative data for statistical purposes. Seeking consent derives from a core concept of personal privacy: the notion that each individual should have the ability to control personal information about himself or herself. Moreover, there can be issues regarding the privacy and confidentiality of data collected for one purpose and used for another, and agencies use different practices, wording, and level of detail to meet consent requirements, according to OMB officials. At the time administrative data are collected, an agency can inform individuals that their data may be used for statistical purposes, but, according to ERS officials, agencies collecting administrative data often do not consider possible future statistical uses and therefore may not provide such notice. Obtaining consent after data have been collected can be time- consuming and costly. In addition, an agency can ask survey respondents for permission to link their survey data with certain administrative data. Some respondents may not consent, which can substantially limit the number of respondents eligible for linkage and as a result potentially affect the quality of the linked data.

Costs and infrastructure: Because the primary cost of collecting administrative data has already been incurred, using these data can, in some cases, be more efficient and less costly than new survey efforts. However, there still are costs to using administrative data for statistical purposes, including up-front and ongoing investments to purchase and maintain hardware and software to link data and protect their confidentiality. Agencies identified various factors that can affect costs. These include but are not limited to negotiations with the agency holding the data, the quality of the administrative data, and the ease with which they can be linked to other data. BLS officials said that in some cases the costs of using administrative data with survey data may outweigh any savings and that evaluation of administrative data options always requires careful consideration of a wide range of quality and cost issues, including the costs of specialized personnel and infrastructure. According to an FCSM study that profiled examples of successful statistical uses of administrative data, agencies wanting to share data also may not have the necessary staff, policies, or procedures. For example, negotiating data-sharing agreements may require significant time. Moreover, many key administrative datasets are held by states, further complicating the data-sharing process because agencies have to negotiate under different policies and procedures as well as work with numerous staff across states.

Documentation of datasets: OMB and agency officials said that agencies holding administrative data do not uniformly document information about their datasets in a way that is always useful or efficient for use outside of the agency. This lack of documentation of datasets makes evaluating their potential for statistical uses challenging. For example, definitions of key variables of research interest or information about how frequently the agency updates the data may not be available. ERS officials also noted that private companies typically do not disclose detailed information about the sources of their data, making it difficult to assess their quality. As a result, agencies interested in using these data for statistical purposes may have to spend additional time and resources to understand the content and structure of the datasets.

Quality of data: Agency officials and experts identified reasons why the quality of administrative data can vary, which can affect their potential use with survey data. Specifically, different agencies may use different systems, definitions, and time frames when collecting administrative data. For example, states may collect and evaluate the quality of data in different ways, making it complicated to aggregate the data across states as well as to compare state-level data. In addition, several factors can influence the accuracy of data reported in administrative data. For example, agencies that collect data for the purpose of program administration may be concerned with the accuracy of only the variables used for such purpose. Moreover, reporting incentives may influence data quality. For example, individuals may underreport income on tax forms, and program agencies may pay less attention to the accuracy of information collected from applicants when it does not affect their participation in a program.

Agencies and interagency committees have been taking numerous actions to address these constraints. For example, ERS, in collaboration with the Census Bureau, NCHS, and OMB is undertaking a pilot project to address data quality concerns with state-level administrative data, and FCSM is working on a project to clarify legal requirements for informed consent (see fig. 3).

One theme that cuts across many of these efforts, and where additional short-term actions could accelerate progress, is identifying ways to facilitate the process of deciding whether to share data among agencies. FCSM published a paper describing successful data-sharing arrangements between various federal and state agencies. One of the four core elements of success that FCSM identified in these arrangements was mutual interest, in that each participant—in particular the agency providing the data—evaluates a proposed data-sharing agreement from its own perspective. On the one hand, agencies may share data because the linked data can benefit program administration, as noted earlier. On the other hand, OMB and agency officials noted that agencies may decide against sharing because perceived disadvantages, such as policy concerns and potential identification of weaknesses in program administration, outweigh the possible benefits. In such a case, an individual agency’s interests may be at odds with the broader efficiency of the whole federal statistical system. As illustrated in figure 3, FCSM and agencies are developing tools to approach these decisions in a more standardized way, such as developing checklists for evaluating the quality of administrative data and a template for executing data- sharing agreements. However, these individual tools focus on particular aspects of data sharing—for example, the checklist focuses on data quality. Separately, they may not be sufficient for agencies to efficiently identify potential datasets with the greatest potential for mutual benefit and address all factors involved in the decision-making process.

The benefits of having more-comprehensive centralized guidance could include greater consistency, clarification, and efficiency. A more- comprehensive standardized framework that ties together existing tools with additional resources in order to cover major aspects of the data- sharing process could bring consistency to the decision-making process. Similar to the checklist that FCSM is developing for agencies to use in evaluating data quality, the framework could include a template outlining a list of key questions for all agencies involved in the proposed data sharing, including federal and state agencies that hold data, to address issues such as: (1) the steps to take to ensure data reliability; (2) any statutory limitations on planned uses of the data (including confidentiality protections); (3) whether consent has already been obtained for additional use of the data, or how it will be obtained; and (4) methods to fully account for the costs associated with obtaining and using the data. To be comprehensive, such guidance would not need to be voluminous, but it should identify each of these major aspects of data sharing, provide advice to agencies, and reference any tools available to assist agencies during the process. It should also be kept up-to-date, reflecting changes in legislation or other factors that affect data sharing, as well as any new tools that are developed. Although such a framework may not lead to sharing in all cases, the framework could better ensure that agencies weigh the related benefits and costs in a more balanced, consistent, and transparent fashion. Such guidance could also clarify ways that agencies could resolve disagreements over data sharing. It could also improve efficiency, given that agency officials we spoke with cited examples in which it took multiple years to reach a resolution on data sharing, by helping agencies evaluate available data and determine those that have the greatest potential for mutual benefit.

While agencies can take steps to address some constraints on sharing data, in other cases only policy actions on the part of the executive branch or Congress can lift barriers. One of the primary examples of such action is Congress’s enactment of CIPSEA in 2002, which authorized the Census Bureau to share selected business data with BLS and the Bureau of Economic Analysis for statistical purposes. However, CIPSEA is limited because the Census Bureau’s business data are based in large part on tax data, and as a result the tax code would need to be amended for the Census Bureau to also share these data with other statistical agencies. There have been proposals to amend the tax code to further expand the scope and coverage of CIPSEA, but action has not yet been taken by Congress.

Prospects for Enhanced Use of the ACS with Other Surveys Are Mixed

The ACS Provides Unique Coverage of the Nation’s Population

The Census Bureau’s full implementation of the ACS in 2005 was a major change to the statistical system. The survey is unique among other surveys of households and individuals because of its size—the monthly surveys add to an annual sample of 3.54 million addresses. The ACS provides annual estimates of social and economic characteristics for all areas of the country and is a primary source of information on small areas, such as towns and tribal lands, down to the neighborhood level. The ACS covers a broad range of topics, such as housing, education, and employment. The information provided by the ACS was previously only available once a decade from the decennial census long form, which the ACS replaced. Users of ACS information include all levels of government, the private and nonprofit sectors, and researchers. According to the Census Bureau, ACS estimates are currently used to help allocate more than $400 billion in federal funding annually. Table 2 lists some of the key characteristics of the ACS.

Several of the ACS’s characteristics lend to its appeal for use for other surveys, including that it produces annual estimates on a broad range of topics at finer geographic levels than other surveys, and agencies and others identified five areas of opportunity in which surveys can make use of ACS data and resources. Two of these areas, which generally rely on publicly-available ACS estimates and do not require changes to the survey’s design or methodology, have the greatest potential for widespread use. The Census Bureau has provided users with various resources to guide their use of ACS estimates. These include a guide to comparing estimates, handbooks directed to specific types of users, training presentations, and a tutorial. The two areas with the most potential for use are as follows:

Evaluating and supplementing other surveys’ results: Survey administrators and data users can also use ACS estimates to evaluate information collected by other surveys. For example, survey administrators can use ACS information to evaluate the quality of responses to other surveys that include some questions that are the same as or similar to ACS questions. Additionally, data users and survey administrators can use ACS data to supplement information collected by other surveys. For example, a recent report based on analysis of ACS data describes how median earnings vary by the field in which people obtain their bachelor’s degrees. Such information can complement results from other surveys. In this case, NCSES also produces information on earnings by degree type, based on information in its Scientists and Engineers Statistical Data System database, which contains data on people with a science or engineering degree and those who work in related fields. NCSES information collection is less frequent than ACS estimates and pertains to a more-narrowly defined population, but allows more detailed analysis of issues such as how people use their college degrees at work. Together, these two sources of information offer more-timely and more-detailed information than a single source.

Designing other surveys: There is also widespread potential to use ACS data to more-efficiently design other surveys. Because many of the topics included in the ACS are covered in more detail by other surveys or relate to other surveys’ target populations, survey administrators can use ACS estimates at different demographic or geographic levels to stay up-to-date on changes that may affect their surveys. These estimates can also be used when designing and selecting a survey’s sample. Census Bureau officials told us that, when designing a survey, survey administrators can use the data to guide the selection of a survey’s sample so that it better represents individuals or households with certain characteristics. For example, the Survey of Income and Program Participation can use ACS estimates at different demographic or geographic levels to identify and more-efficiently sample geographic areas with disproportionately large numbers of low-income households because this is a population of interest for the survey. Because these data are available for small geographic areas, agencies can use the data when samples include more-local geographic levels.

Uses of ACS That Require Design and Methodology Changes Have Limited Potential

Agencies and others identified three uses of ACS data and resources that, while offering potential benefits to other surveys, face such constraints that more widespread use is likely not possible under current ACS design. These uses are more intensive than the ones described above, in that they affect the survey’s design and methodology or respondent burden, or both. Because the ACS has a large sample size and a complex methodology, there are logistical challenges involved in changing its design and methodology. Additionally, any changes that affect the survey’s respondent burden also have limited potential, as there are already concerns about the burden that the ACS places on respondents. Uses with more-limited potential are as follows:

Adding or modifying ACS content: Adding a question to the ACS or modifying existing questions can improve the efficiency of other surveys, though doing so involves trade-offs with factors such as respondent burden. This use of ACS could provide information that would inform the design of other surveys or facilitate the use of ACS data for another survey’s sample frame. For example, NCSES worked with the Census Bureau to add a question to the ACS about the field in which respondents earned their bachelor’s degrees in order to identify respondents that are in the target population for the National Survey of College Graduates. Despite the potential benefits of adding or modifying ACS content, adding a question to the ACS would increase respondent burden and have operational impacts, as it requires the Census Bureau to change the questionnaire design and processing and editing systems. If these actions result in additional pages for the questionnaire, it could affect costs and the response rate. Modifying questions poses an additional challenge because ACS estimates reflect multiple years of data, and a change in a question may affect the Census Bureau’s ability to cumulate data.

Adding supplements to the ACS: Another possible use of the ACS by other surveys is adding supplements to the ACS, though this use faces several obstacles. While the ACS currently does not include supplements, doing so could enable surveys to leverage the resources of the ACS. Other surveys, such as the Current Population Survey, allow other agencies or entities to sponsor supplemental surveys that are added on to the survey’s core set of questions. According to officials at BLS, which sponsors the Current Population Survey, in their experience it costs less to add a supplement to an existing survey than to conduct a separate stand-alone survey. Additionally, the agencies sponsoring the supplements gain the benefit of the experience of BLS or Census staff, or both, in designing and implementing surveys. Although the Current Population Survey successfully incorporates supplements, the ACS is different in several key ways, and adding supplements to the ACS would involve significant challenges. For example, the ACS is mandatory, meaning that responses are required by law. Assuming a supplement to the ACS would be voluntary, Census Bureau officials told us that they would have to determine how to distinguish between the mandatory and voluntary sections, which would create complexity. Additionally, the Census Bureau processes ACS data on a yearly basis and does not have a process in place for producing estimates from a single month’s data, which would be a challenge if the supplement was administered along with only 1 month’s ACS mailout. Finally, including supplements raises concerns about respondent burden and respondent fatigue. BLS officials noted the potential of matrix sampling, in which a set of additional questions, as in a supplement, is added to a month of collection (or all months) but differs from a supplement in that it is only administered to a subset of the survey sample in a given collection period. This option could reduce burden and increase efficiency; however, such an option involves logistical considerations in administering the survey and processing the data, and adds complexity for analysts using the data for research.

Creating sample frames: Using ACS data to develop sample frames for follow-on surveys has been identified as a potential use of ACS data, but several factors limit this use. This involves using ACS data to identify ACS respondents with certain characteristics for potential inclusion in a follow-on survey and requires the approval of the Census Bureau and OMB. At present only NCSES uses ACS data for this purpose. Agency officials told us that using ACS data to create a sample frame, as opposed to census long-form data, which they used previously, has improved the agency’s coverage of its target population and has reduced costs and respondent burden.benefit of this use is expanded analysis, as agencies, under appropriate Title 13 restrictions, can analyze respondents’ answers to the ACS along with responses to the follow-on surveys, and can analyze the characteristics (from ACS data) of those who do and do not respond to the follow-on survey to determine if they have different characteristics, which might cause bias in the survey. Surveys such as the National Survey of College Graduates that focus on populations that are costly to identify are likely to realize higher gains in efficiency from using ACS data for this purpose. Despite these benefits, opportunities for other surveys to use ACS data for this purpose are limited. ACS’s sample size, although large compared to most surveys, can be too small for another survey to use for a sampling frame. This is especially an issue if a survey targets a rare population or targets members similar to those of surveys already drawing from the ACS for their frame, because there would be too much chance of drawing individuals into both follow-on surveys, and current policy does not allow for that. Census Bureau policy states that, when agencies conduct follow-on surveys, they may not contact any member of a household that has already responded to the ACS and also had a member selected for a follow-on survey. With certain households in the ACS excluded from potential selection, it becomes more difficult for other surveys to draw samples because the data no longer reflect respondents with certain characteristics.

In the long run, more-intensive uses of ACS data and resources may require difficult decisions and entail trade-offs with factors such as cost and respondent burden. Further, they risk affecting ACS response rates and overall data quality. However, redesign of the scope and methodology of ACS might overcome some of these constraints. After the release of the survey’s first 5-year data products in 2010, the Census Bureau and others began evaluating the survey and exploring options for increased uses. In addition to its own evaluation of the ACS, at the Census Bureau’s request the National Academy of Sciences is organizing workshops with data users to assess the survey. Also, OMB, in cooperation with the Census Bureau, created an ACS subcommittee of the ICSP with the goal of investigating trade-offs of options such as adding questions to the ACS and rotating questions in and out of the survey. If the Census Bureau changes the survey’s design or methodology, these changes may become more feasible.

Conclusions

To ensure the provision of high-quality, timely statistical data for public- and private-sector users, OMB and the agencies that make up the federal statistical system must continue to identify opportunities for efficiency in federal surveys of households and individuals. Most of the surveys and other information collections in our scope have relatively modest costs, but challenges such as declining survey response rates will strain available resources unless agencies find more-effective and less-costly ways to collect and analyze the needed information, while maintaining critical protections of respondents’ privacy and confidentiality. In the long term, addressing the key challenges and constraints that agencies have identified will necessitate broader public debates and policy decisions about balancing trade-offs among competing values, such as quality, cost, timeliness, privacy, and confidentiality. In the short term, our review indicated that two promising avenues to sustain the progress that OMB and agencies are making include (1) facilitating collaboration and coordination among agencies and (2) combining existing data from both survey and administrative sources.

The federal statistical system already exhibits many collaborative traits and practices, in particular through projects sponsored by OMB and interagency committees that facilitate coordination and the development of new policies and tools. However, additional efforts could help enhance the effectiveness of these efforts. Going forward, it will be important for OMB to supplement existing guidance to clarify the range of options available to address PRA standards. Supplementing the guidance could increase agencies’ awareness of these options, in particular those that were cited less frequently. At the same time, interagency committees could do more to improve accessibility and timeliness of their work products. Doing so could maximize the usefulness of committees’ work. Additionally, OMB’s ability to oversee and coordinate information collections across the government would benefit from additional steps to ensure the reliability of data on collections’ costs and burdens. Doing so would also benefit users of the information, whether they access it through the website or though OMB reports.

Agencies identified multiple ways that combining survey and administrative data can improve the efficiency and quality of their work, and they are already pursuing such opportunities. Importantly, they have demonstrated that using existing datasets to supplement each other can add value for all agencies involved in data sharing. But agencies also face serious constraints to expanded uses of existing data. One of the more-significant barriers is the complexity of the process through which they make decisions about sharing data. Though agencies and interagency committees are working to create tools to facilitate parts of this process, more-comprehensive and centralized guidance for agencies to follow when negotiating and making decisions regarding data-sharing opportunities could help facilitate the process.

A standard protocol or framework could accelerate progress in this area by helping agencies to (1) evaluate the growing array of administrative data to identify those datasets that have the greatest potential for mutual benefit of the participating agencies, and (2) consider a common set of criteria and key questions when weighing the pros and cons of sharing data. A key benefit would be to encourage agencies to consider, in a uniform manner, all relevant aspects of these decisions, such as whether or not proposed uses would be consistent with applicable law, maintain confidentiality protections, be cost-effective, and serve to increase the broader efficiency of the federal statistical system.

Recommendations for Executive Action

In order to maintain progress in maximizing the efficiency of existing data sources, we recommend that the Director of OMB, in consultation with the Chief Statistician, work with the ICSP to take the following four actions: To improve the broader efficiency of the federal statistical system and improve communication among agencies and others, when OMB next updates guidance on agency survey and statistical information collection and dissemination methods, include additional details on actions agencies can take to meet requirements to identify duplication, to consult with persons outside of the agency, and address other requirements as appropriate; and create new methods or enhance existing methods to improve the dissemination of information and resources produced by interagency statistical committees. For example, such enhancements could include increasing the timeliness and availability of information on websites to better capture the full range of products and identify committee priorities.

To increase the reliability of the information presented on the Reginfo.gov website and in OMB’s internal system, implement quality-control procedures designed to identify and remedy any differences between cost and burden information provided on the website and in the related supporting statement documentation that underlies this information.

To accelerate progress in sharing administrative data for statistical purposes, where appropriate, develop comprehensive guidance for both statistical agencies and agencies that hold administrative data to use when evaluating and negotiating data sharing, such guidance should include key questions focused on issues such as statutory authority, confidentiality, cost, and usefulness in order to ensure agencies consider all relevant factors and the broader interest of the federal government.

Agency Comments and Our Evaluation

We provided a draft of this report to the Secretaries of Commerce and Health and Human Services, the Director of OMB, the Commissioner of BLS, the Administrator of ERS, and the Director of the National Science Foundation for their review and comment. We received written comments on the draft report from the Secretary of Commerce that are reprinted in appendix V. We also received comments from OMB staff that are summarized below. The Department of Health and Human Services, BLS, National Science Foundation, OMB, and agencies on the ICSP also provided technical comments and suggestions that we incorporated as appropriate.

Commerce stated that our observations illuminate future opportunities for using administrative records within the federal statistical system to increase efficiency and better meet informational needs and that our suggested actions would enhance the ability of statistical agencies to realize these opportunities. Regarding our recommendation on standard protocols and procedures to facilitate data sharing, the department noted that policies and other initiatives can also play a role in achieving cooperation. Finally, the department noted that our report’s acknowledgement of related concerns about the quality of administrative data, and the level of support and resources necessary to maintain a statistical and administrative data infrastructure, underscore the importance of our recommendations.

OMB generally agreed with our recommendations and said that the agency hopes to pursue these in the future. More specifically,

OMB agreed that it is worth considering good practices for reducing duplication. As we suggested, OMB indicated that when its survey guidance is next updated it will include additional details and examples of actions agencies can take to identify duplication and consult with persons outside the agency.

OMB said that it shared our concerns about timely and easily accessible dissemination of information resources produced by interagency statistical committees, and that our recommendation underscores the need for addressing this issue.

With respect to our recommendation that OMB implement quality- control procedures designed to identify and remedy any differences between cost and burden information provided on Reginfo.gov and in the related supporting statement documentation that underlies this information, OMB noted that PRA requires OMB to weigh the burdens imposed on the public by information collections against the legitimate needs of the federal agencies. OMB said that this requires a careful assessment of the estimates of paperwork burden that agencies provide to OMB as part of their information collection requests and, further, that these estimates are subject to public scrutiny and comment in Federal Register notices, in the PRA statements provided on information collections, and on Reginfo.gov. OMB pointed out that, because the burden estimates provided on Reginfo.gov and in the underlying supporting statements are all made public, discrepancies such as those found by us are public as well. OMB said that it will investigate and address any such discrepancies that are brought to its attention by GAO or any member of the public.

Finally, OMB concurred that administrative records can be a valuable supplement to, though usually not a replacement for, household surveys. OMB believes that our recommendation to develop comprehensive guidance for statistical and administrative agencies to use when evaluating and negotiating data-sharing agreements would be constructive, but cautioned that this involves a very complex set of issues and said it will take some time to develop such guidance.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Commissioner of the Bureau of Labor Statistics (BLS), the Director of the U.S. Census Bureau, the Administrator of the Economic Research Service (ERS), the Secretary of Health and Human Services, the Director of the National Science Foundation, the Director of OMB, the Secretary of Commerce, and the Under Secretary of Economic Affairs. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact Robert Goldenkoff at (202) 512-2757 or goldenkoffr@gao.gov, or Ronald S. Fecso at (202) 512-7791 or fecsor@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors are listed in appendix VI.

Appendix I: Scope and Methodology

The objectives of this report were to (1) review the ways in which the Office of Management and Budget (OMB) and agencies identify opportunities for improvement and increased efficiency of selected information collections; (2) evaluate opportunities and constraints for the statistical agencies to use administrative data in conjunction with selected surveys; and (3) evaluate ways in which American Community Survey (ACS) data and resources can be used in selected surveys, and the associated benefits and constraints.

To achieve our objectives, we focused on statistical information collections administered to households and individuals and subject to the Paperwork Reduction Act (PRA), which requires OMB approval of certain federal data collections. Although in many cases the information and views provided by agencies during our review and our general findings may also apply to statistical information collections outside of our scope, such as those administered to businesses, all of the specific collections and surveys we reviewed were administered to households and individuals. The majority of the collections within our scope include a survey, though some also include other methods of information collection such as focus groups. To examine the issues related to our objectives, we performed case studies of five federal surveys: the Consumer Expenditure Surveys, sponsored by the Bureau of Labor Statistics; the National Health and Nutrition Examination Survey and the National Health Interview Survey, both sponsored by the National Center for Health Statistics, part of the Centers for Disease Control and Prevention; the National Survey of College Graduates, sponsored by the National Center for Science and Engineering Statistics, part of the National Science Foundation; and the Survey of Income and Program Participation, sponsored by the Census Bureau. We selected these surveys based on several factors, such as their size and cost and whether they use or have the potential to use administrative data or ACS data.

For the first objective, to review the ways in which OMB and agencies identify opportunities for improvement and increased efficiency of selected statistical information collections, we examined the PRA, OMB guidance to agencies, and prior GAO work on the federal statistical system. We interviewed officials at OMB and the four agencies that administer the case-study surveys to learn about coordination among agencies, efforts agencies take to identify improvement, and OMB’s role. We also interviewed officials at the Department of Agriculture’s Economic Research Service, which is a member of several interagency statistical committees and the lead agency for the Statistical Community of Practice and Engagement. In addition, we interviewed experts on the federal statistical system to learn about their perspectives on the efficiency of the federal statistical system and agency and OMB coordination. In evaluating OMB, agency, and interagency actions, we used as criteria the requirements of the PRA and practices identified in prior GAO work on agency collaboration.

To address the second objective, to evaluate opportunities and constraints for agencies to use administrative data in conjunction with selected surveys, we reviewed statutes that govern the sharing and use of administrative data, documentation from case-study surveys, and various papers and reports. We interviewed officials at OMB and experts in the field of federal statistics to learn about their perspectives on the current and potential uses of administrative data. We also interviewed officials at the Economic Research Service and the agencies that sponsor the case-study surveys to learn about ways in which their surveys use or could potentially use administrative data. For this objective and the third we used OMB guidance, relevant statutes, and prior GAO work as criteria in our evaluation.

For the third objective, to evaluate the ways in which ACS data and resources can be used in selected surveys, we reviewed Census Bureau documentation, National Science Foundation reports, prior GAO work, and reports issued by the Committee on National Statistics. We interviewed officials at the Census Bureau, which sponsors the ACS, and at OMB to learn about their perspectives on potential uses of the survey and its data. We also interviewed officials at the Economic Research Service and the agencies that administer the case-study surveys to learn about ways in which their surveys use or could potentially use ACS data and resources, and experts in the field of federal statistics to learn about their assessment of the uses and potential uses of the ACS.

To gain a broader perspective on the information collections in our scope and to inform our work across all three objectives, we obtained and analyzed publicly-available data from Reginfo.gov, a government website that provides access to information on agency requests for OMB approval of information collections. We used the website’s search feature to download all of the collections that were classified as (1) active, meaning that they are currently approved by OMB for use by agencies; (2) employing statistical methods; and (3) directed to households and individuals. We downloaded data on all information collections that met these criteria from Reginfo.gov on two dates, May 17, 2011, and September 22, 2011.

We performed more in-depth analyses of the 507 information collections in our May 17, 2011, download. First, we reviewed the supporting statements for each of these collections, and on the basis of information in these documents classified them according to the subject matter on which they focus. Next, we grouped the collections into categories, based on information on the sponsoring agency in Reginfo.gov and the supporting statements. Depending on the sponsoring agency, we put the collections into one of four categories: (1) those that are sponsored by one of the 13 principal statistical agencies; (2) those that are sponsored by another agency that shares a parent agency with one of the 13 principal statistical agencies (for example, agencies in the Department of Health and Human Services would fall into this category because it is the parent agency of the National Center for Health Statistics); (3) those that are not a principal statistical agency and do not share a parent agency with one; and (4) unknown, for those whose sponsoring agency we could not determine based on the available information. We also used the information in the supporting statements to determine if the collections included a survey component and found that 481 of the 507 did.

We divided the 481 collections that included a survey component into three strata that reflect the type of sponsoring agency. Of the 481, we were not able to determine agency type for 7 collections so we dropped these records, leaving a population of 474 statistical information collections. The number of collections by stratum is shown in table 3. In order to estimate the prevalence of certain characteristics in this population—for example, the percentage of information collections for which the sponsoring agency reported steps taken to identify potential duplication—we drew a stratified sample of 106 collections. Within each stratum, we estimated the sample size required to yield a 95 percent confidence interval of plus or minus 14 percent around such an estimate. For the overall population of 474, the approximate precision for an estimated percentage of 50 percent is plus or minus 8.4 percent, at the 95 percent level of confidence.

We reviewed the supporting statements of each of the information collections in our sample of 106, focusing on agencies’ reported efforts to identify duplication and to consult with persons outside the agency to obtain their views. Because agencies follow a standard format in preparing supporting statements, we focused our analysis on the sections of the supporting statements in which OMB instructs agencies to include this information (sections 4 and 8, respectively, of section A of the supporting statement). To review agencies’ reported actions, we used a data-collection instrument that contained a series of “yes-no” questions about the types of efforts reported. For example, we reviewed whether agencies had reported considering administrative data as a potential source of duplication, and whether agencies reported that they had consulted with other agencies when describing consultations outside of the agency. We did not evaluate whether agencies actually took the actions they reported taking. Estimates produced from the sample of the collections are subject to sampling error. We express our confidence in the precision of our results as a 95 percent confidence interval. This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. As a result, we are 95 percent confident that each of the confidence intervals in this report will include the true values in the study population.

We took several steps to evaluate the reliability of the data we accessed through the Reginfo.gov website. We interviewed OMB officials and reviewed documentation of the Reginfo.gov website and the Regulatory Information Service Center and OIRA Consolidated Information System,which is the system that agencies use to track information collection requests and that underlies information provided on Reginfo.gov. As part of our review of the subject matter of the collections in the May 17, 2011, download, we confirmed that the collections were within our scope. We also used the information in the September 22, 2011, download to evaluate the reliability of data on collections’ cost and annual burden. To do this, we drew a systematic random sample of 56 (approximately 10 percent) of the 555 collections in the download.

We found a number of inconsistencies between the cost and burden information available on the website and that provided in supporting statement documentation. According to an official at OMB, the two sources should match, but the supporting statement documentation is more accurate than that on the website. On the basis of our assessment, we determined that the information from the website was not sufficiently reliable for the purpose of describing the annual cost or annual burden to respondents of the collections in our scope. However, through this review and the other steps we took, we found that the other information provided on the Reginfo.gov site was sufficiently reliable for our other intended purposes of identifying the collections within our scope and obtaining information on their subject matter and reported actions taken to identify unnecessary duplication and solicit input from outside persons and entities.

Because the cost information on the Reginfo.gov website was not sufficiently reliable, we used cost information from the supporting statements of the collections in our sample to provide background information on the costs of the collections in our scope. In addition to using information from the supporting statements in our initial sample of 56 collections, we drew another systematic random sample of 56 additional collections from the September download. In total, we obtained cost information from the supporting statements of 112 (approximately 20 percent) of the 555 collections in our scope that were active as of our September 22, 2011, download.

We conducted this performance audit from December 2010 until February 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audits to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Description of Case-Study Surveys

Consumer Expenditure (CE) Surveys: Quarterly Interview Survey (CEQ) and Diary Survey (CED)

Purpose: To collect information on expenditures and households’ characteristics Sponsoring agency: Bureau of Labor Statistics (BLS)

This amount reflects the approximate fiscal year 2010 cost of collecting, processing, reviewing, and publishing data collected through the CE Surveys. Survey costs vary somewhat from year to year.

BLS estimates that respondents will take an average of 60 minutes to complete one interview survey of the CEQ. Since the CEQ is administered to the same sample of households four times in a year, the annual burden for respondents who complete all four surveys is roughly 4 hours. In addition, a certain number of respondents who complete the interview surveys are reinterviewed, a process that adds 10 minutes to these selected respondents’ burden times.

Price Index, the most widely used measure of inflation.government agencies, private companies, policymakers, and researchers use data from the CE Surveys in a variety of ways. For example, the Department of Defense uses data from the CE Surveys to update cost-of- living adjustments for military families. Congressional committees also use the data to inform decision making, such as the potential effect of increases in the minimum wage.

National Health and Nutrition Examination Survey (NHANES)

NCHS estimates that the annual cost to the federal government of NHANES for fiscal year 2010 was $37.8 million, including both direct and reimbursable funding provided by other agencies for NCHS statistical services. Survey costs vary somewhat from year to year.

NCHS estimates that the total annual burden for the NHANES is 37,626 hours, including screening, household interviews, physical examinations, and any follow-up interviews. In addition, tests of procedures and special studies account for an additional 12,000 hours, for a total annual burden of 49,626 hours. NCHS estimates that respondents who participate in all aspects of the NHANES, including the screener survey, household interview, and physical examination, can expect a burden of 6.7 hours. In addition to those who complete all aspects of the NHANES, some respondents may only participate in the screener survey and be screened out of the sample, while other respondents may participate in the screener survey and the household interview but not the physical examination. NCHS includes all respondents at these varying levels of participation in its calculation of the annual burden hours.

Target population: Nationally-representative sample of individuals of all ages Uses of data: According to NCHS documentation, a variety of users, including federal agencies, research organizations, universities, health- care providers, and educators, use NHANES data. For example, the Food and Drug Administration uses NHANES data to determine whether changes are needed to federal regulations. In addition, use of NHANES data informs key decision making. For example, according to NCHS documentation, NHANES data on lead levels in blood were instrumental in developing the policy to eliminate lead from gasoline and in food and soft drink cans. As part of its broader data-linkage program, NCHS links NHANES survey data to multiple administrative datasets, such as the National Death Index (a centralized index of state death record information) and Medicare and Medicaid claims from the Centers for Medicare and Medicaid Services. The National Death Index linkages give researchers an opportunity to analyze mortality differences among subgroups defined using the survey information. Similarly, the Medicare and Medicaid claim linkages provide an opportunity to examine health conditions, utilization, and costs among subgroups defined using the survey information. Additionally, according to NCHS officials, NCHS is currently conducting a pilot study to link NHANES data on participants from Texas to administrative data on food assistance.

National Health Interview Survey (NHIS)

Purpose: To monitor the health of the U.S. population Sponsoring agency: National Center for Health Statistics (NCHS), Centers for Disease Control and Prevention Annual sample size (estimated): 35,000 householdsAnnual cost to the federal government (estimated): $32.2 millionAnnual burden hours (estimated): 34,977 hours Target population: Nationally-representative sample of households, collecting data on all members of each household Uses of data: According to NCHS documentation, government agencies, policymakers, researchers, and academics use NHIS data for a variety of purposes, such as identifying health problems and evaluating health programs. For example, policymakers used NHIS data to shape the Centers for Disease Control and Prevention’s cervical-cancer screening policy. In addition, other agencies can use the NHIS as a sample frame for their surveys. Lastly, as part of its broader data-linkage program, NCHS links NHIS survey data to multiple administrative datasets, including those it uses for linkages with NHANES data, such as the National Death Index and Medicaid and Medicare claims.

National Survey of College Graduates (NSCG)

NCHS estimates that the total annual burden of the NHIS was 34,977 hours for 2010 and 2011. NCHS estimates that a single respondent who completes all portions of the NHIS for a household can expect a time burden of one hour. Some respondents who complete all portions of the NHIS are asked to take a short reinterview survey, a process that adds 5 minutes to these selected respondents’ burden times. In addition to those who complete all portions of the NHIS, some respondents may only participate in a screener survey and be screened out of the sample, which NCHS estimates takes 5 minutes per respondent. NCHS includes all respondents at these varying levels of participation in its calculation of the annual burden hours.

Cost to the federal government (estimated): $13.3 millionBurden hours per administration (estimated): 34,792 hours Target population: Individuals in the United States who have a bachelor’s degree in science, engineering, or health, and those who have a degree in another field but work in science, engineering, or health occupation.

Uses of data: According to National Science Foundation documentation, information from the NSCG is used by researchers and policymakers. Government agencies use the data to assess available scientific and engineering resources and inform the development of related policies. Additionally, educational institutions use NSCG data to inform the establishment and modification of curricula, and businesses use the data to develop recruitment and compensation policies.

Survey of Income and Program Participation (SIPP)

This estimate is based on the assumption that 83,500 individuals will respond to the survey and each respondent will take 25 minutes to complete it.

The Census Bureau estimates that the production cost for all parts of the SIPP in fiscal year 2011 is $50.1 million. Survey costs vary somewhat from year to year.

Annual burden hours (estimated): 143,303 hours All Target population: Nationally-representative sample of households. household members 15 years old or over are interviewed for the survey.

Uses of data: According to the Census Bureau, SIPP data are used by agencies such as the Department of Health and Human Services and the Department of Agriculture, as well as economic policymakers, Congress, and state and local governments, to plan and evaluate government social-welfare and transfer-payment programs.

The SIPP is limited to the U.S. civilian, noninstitutionalized population, and as a result excludes certain segments of the population, such as active-duty military members living on bases and prisoners.

Appendix III: Selected Statutes Related to Information Collection

Selected statutes that regulate the collection and dissemination of information include the following.

Governmentwide Statutes

The Information Quality Act of 2000 requires, among other things, that the Office of Management and Budget (OMB) develop and issue guidelines that provide policy and procedural guidance for federal agencies for ensuring and maximizing the quality of the information they disseminate. These guidelines include steps designed to assure objectivity and utility of disseminated information. See 44 U.S.C. § 3504(d)(1); OMB guidelines are at http://www.whitehouse.gov/omb/info_quality_iqg_oct2002/.

The Privacy Act of 1974, as amended, and the privacy provisions of the E-Government Act of 2002 specify requirements for the protection of personal privacy by federal agencies. The Privacy Act places limitations on agencies’ collection, disclosure, and use of personal information maintained in systems of records. See 5 U.S.C. §§ 552a and 552a note. The E-Government Act requires agencies to conduct privacy impact assessments that analyze how personal information is collected, stored, shared, and managed in a federal system. See 44 U.S.C. § 3501 note.

The Confidential Information Protection and Statistical Efficiency Act (CIPSEA) of 2002 focuses on confidentiality protection and data sharing. It requires that information acquired by an agency under a pledge of confidentiality and for exclusively statistical purposes be used by the agency only for such purposes and not be disclosed in identifiable form for any other use, except with the informed consent of the respondent. It also authorizes identifiable business records to be shared for statistical purposes among the Bureau of Economic Analysis, Bureau of Labor Statistics, and the Census Bureau. See 44 U.S.C. § 3501 note.

Agency-Specific Statutes

Agency-specific statutes also guide federal data collection and use. For example, the Census Bureau conducts the census and census- related surveys such as the American Community Survey under Title 13 of the U.S. Code, which gives the Census Bureau the authority to request and collect information from individuals but also guarantees the confidentiality of these data and establishes penalties for unlawfully disclosing this information. Unless specifically authorized, these provisions preclude the Census Bureau from sharing identifiable census information with other agencies. See 13 U.S.C. § 9. Title 15 of the U.S. Code permits the Secretary of Commerce to conduct studies on behalf of other agencies and organizations. Identifiable data from surveys conducted under Title 15 authority are subject to the sponsoring agency’s legislation and confidentiality requirements. See 15 U.S.C. § 176a. Statutes and regulations specific to other agencies also affect collection and sharing of data.

Section 6103 of the Internal Revenue Code provides that federal tax information is confidential and may not be disclosed except as specifically authorized by law.

Section 308(d) of the Public Health Service Act requires that identifiable information obtained by the National Center for Health Statistics be used only for the purpose for which it was collected unless consent is obtained for another purpose, and it prohibits the release of identifiable information without consent.

Other legislation such as the Family Educational Rights and Privacy Act, which protects the privacy of student education records, can affect federal data-collection efforts. See 20 U.S.C. § 1232g.

Appendix IV: Printable Interactive Graphic

This table reproduces the information in the interactive figure 3 earlier in this report.

Appendix V: Comments from the Department of Commerce

Appendix VI: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the individuals named above, Tim Bober (Assistant Director), Carl Barden, Russell Burnett, Robert Gebhart, Jill Lacey, Andrea Levine, Jessica Nierenberg, Susan Offutt, Kathleen Padulchick, Tind Shepper Ryen, and Jared Sippel made key contributions to this report.