Information Quality Act: Actions Needed to Improve Transparency and Reporting of Correction Requests
Released: 2016-01-20T00:00:00

Letter

Background

IQA establishes a process that allows the public to help ensure the quality of information disseminated by federal agencies. IQA consists of two major elements. The first element of IQA required OMB to develop and issue government-wide guidelines by the end of fiscal year 2001. These guidelines were to provide policies and procedures for federal agencies to use for “ensuring and maximizing quality, objectivity, utility, and integrity of information (including statistical information),” that they disseminate. The second element required covered federal agencies to develop IQA guidelines by the end of fiscal year 2002. These guidelines were to establish administrative mechanisms allowing “affected persons to seek and obtain correction of information maintained and disseminated” by the agencies. The guidelines were to also require agencies to periodically report to the Director of OMB on the number and nature of IQA complaints and how such complaints were handled.

IQA builds on previous federal efforts to improve the quality of information, including OMB Circular A-130 and the Paperwork Reduction Act of 1980, as amended. OMB Circular A-130 establishes a policy for the management of federal information resources. Two of the purposes of the Paperwork Reduction Act were to improve quality and use of federal information and provide for the dissemination of public information in a manner that promotes the utility of the information to the public and makes effective use of information technology.

OMB’s Office of Information and Regulatory Affairs (OIRA) develops and oversees the implementation of government-wide policies in the areas of information technology, privacy, and statistics. In this capacity, OIRA developed the government-wide IQA guidelines and helped agencies to meet the Act’s requirement that the agencies develop their own guidelines. OMB issued guidance to agencies to clarify how agencies were to satisfy the law and otherwise implement IQA. The guidance required agencies to develop and post IQA guidelines and related information on their websites. An October 2002 OMB memorandum describing the implementation of IQA guidelines noted that it represented the first time that the executive branch has developed a government-wide set of information quality guidelines, including agency-specific guidelines tailored to each agency’s unique programs and information. Agencies’ guidelines, which were to follow OMB’s model, were to include administrative mechanisms that allow “affected persons” to request correction of information that they did not consider correct.

We reported in August 2006 that expanded oversight and clearer guidance by OMB could improve agencies’ implementation of the Act. We found that OMB had issued government-wide guidelines that were the basis for other agencies’ own IQA guidelines. We also reported that OMB required agencies to post guidelines and other IQA information to their websites and required agencies to provide information to OMB on the number and nature of correction requests they received and how such correction requests were resolved. We found that 14 of the 15 cabinet agencies, the Environmental Protection Agency (EPA), and 4 other independent agencies we reviewed had developed IQA guidelines and posted them on their websites. Of these 19 cabinet and independent agencies with guidelines, we found that 4 had information quality links on their home pages while other agencies’ IQA information was difficult to locate online. Moreover, 44 of 86 additional independent agencies that we examined had not posted their guidelines and may not have had them in place at the time. Consequently, users of information from those agencies may not have known whether agencies had guidelines or how to request correction of agency information. OMB also had not clarified guidance to agencies about posting IQA-related information, including guidelines to make that information more accessible.

We also found in our 2006 report that in fiscal year 2003, the Federal Emergency Management Agency and two other agencies used IQA to address flood insurance rate maps, website addresses, photo captions, and other administrative matters. However, in fiscal year 2004, these agencies changed their classification of these requests from being IQA requests. Instead they processed them using other administrative processes that were in place prior to IQA implementation. As a result, we found that the total number of all IQA requests dropped from more than 24,000 in fiscal year 2003 to 62 in fiscal year 2004. We recommended that OMB (1) identify agencies without IQA guidelines and work with them to develop and implement IQA requirements and (2) clarify guidance to agencies on improving the public’s access to online IQA information. In response to our report, OMB stated it would work with agencies as they develop and implement information quality measures and would also continue to work with agencies to improve their dissemination of IQA information. Further, in December 2009, OMB, in an executive memorandum to heads of executive departments and agencies, issued an Open Government Directive (1) establishing deadlines for action that, among other things, encouraged agencies to advance their open government initiatives (including IQA) ahead of those deadlines and (2) calling for each agency to take prompt steps to expand access to information by making it available online in open formats.

Most Agencies in Our Review Reported Receiving Relatively Few IQA Correction Requests; the Majority of Requests Questioned Data Use and Resulted in No Corrections Agencies’ Websites Show a Total of 87 IQA Correction Requests from Fiscal Years 2010 through 2014

According to IQA information posted on the 30 agency websites in our review, 16 agencies reported receiving 87 IQA correction requests from fiscal years 2010 through 2014 (see table 1). The other 14 agencies in our review did not post any IQA correction requests during the period. Agencies reported receiving the highest number of correction requests (26) in fiscal year 2010, with the lowest number (13) coming in fiscal year 2014. Several agencies, including the Departments of Education, Housing and Urban Development, and Labor, the Federal Reserve Board, and the Office of Science and Technology Policy, reported receiving 1 correction request during the 5-year period.

Three agencies— the Departments of Health and Human Services (HHS) and Interior (Interior) and EPA—received 70 percent (61 of 87) of the correction requests during fiscal years 2010 through 2014. These three agencies were also the only ones that reported receiving IQA correction requests during each of the 5 fiscal years. For the entire period, Interior received the highest number of correction requests (26), followed by EPA (21), and HHS (14).

IQA officials at several agencies told us that they receive relatively few IQA requests and provided a number of reasons. For example, EPA officials stated that the quality of data EPA disseminates is currently more robust due to the consideration of the diversity of viewpoints provided by the public and EPA’s opportunity to review pertinent information that may not have been previously considered during the pre-dissemination process. Department of Commerce (Commerce) officials attributed their agency’s low number of IQA requests to the fact that the agency has few highly influential scientific assessment projects and that most of its research is relatively noncontroversial, with the exception of research related to climate change. HHS officials said that it is not surprising that IQA administrative corrective mechanisms are not resulting in a large number of IQA correction requests because many correction requests are for minor edits to agency information. A former OIRA administrator opined that when several federal courts held that the IQA is not subject to judicial review, most of the momentum behind IQA was lost, and that, as a result of these rulings, outside parties do not submit very many IQA correction requests.

In August 2004, the OIRA Administrator issued a memorandum to the President’s Management Council directing that agencies post all information quality correspondence, including a copy of each correction request, the agency’s formal response(s), and any communications regarding appeals on agency web pages to increase the transparency of the process. The memorandum also directed agencies to provide a few sentences describing the request and any subsequent responses. Finally, the memorandum stated that agencies also needed to establish processes for updating their information quality web pages regularly. In addition to posting copies of the IQA correction requests on their websites, agencies are required to report the number and nature of correction requests the agency receives to the Director of OMB and how such requests were resolved. OMB has provided a summary of this agency-reported IQA data since its implementation in annual reports to Congress since 2003.

We found discrepancies between the IQA data we found on agency websites and the IQA data reported to OMB by agencies. Eight agencies who reported receiving IQA correction requests did not post on their website the same number of IQA correction requests that they reported to OMB. In most instances where we identified discrepancies, the number of IQA correction requests agencies posted on their websites was lower than the number of IQA correction requests they reported to OMB. Table 2 provides specific numbers of discrepancies in IQA correction requests received for fiscal years 2010 through 2014.

One CFO Act agency, the Department of Transportation, reported to OMB that it had received an IQA correction request, but did not post the IQA correction request on its website as of November 2015. As stated earlier, OMB guidance requires agencies to post correction requests and agency responses on their websites. OMB staff told us that they issue an annual data call to agencies requesting information on IQA correction requests received. According to OMB, agencies are expected to accurately report their IQA activities, including the number of requests received. OMB staff stated that if there are discrepancies between what OMB received from agencies and what the agencies post on their websites, then there is a miscount or a disconnect on the agency side. Although OMB’s guidance is not prescriptive on the time frames for agencies to post this information, it states that agencies need to establish “processes for updating their information quality web pages on a regular basis.” OMB staff told us that some agencies posted correction requests and responses online soon after sending out the agency responses. For example, HHS officials told us that they post correction requests soon after they are received and do not wait until a response is prepared. According to OMB staff, other agencies waited until the end of the fiscal year to post all relevant documents at the same time. In addition, they told us that agencies often have changes in the staff assigned to report IQA data to OMB that may contribute to late postings, and although the data are eventually posted online, they are sometimes provided months after the request was received and responded to.

Agency officials from the six agencies that we selected for further review offered various explanations concerning their data discrepancies, including the time frames for online postings of IQA data. For example, officials from the Department of Agriculture (USDA) stated that the agency does not specify time frames for posting correction requests. However, USDA officials stated that, in response to our inquiries, moving forward the agency will require its component agencies and staff offices to post all correction requests and their responses to their component agency’s website no later than 60 days after the correction requests are received. USDA officials stated that each component agency maintains its own website and updates it accordingly. Since we initially contacted USDA concerning the data discrepancies, USDA officials have informed us that the Food and Nutrition Service and Rural Development component agencies have updated their websites to reflect the number of correction requests received. In addition, the officials stated that the Forest Service, the Office of the Inspector General, and the Animal and Plant Health Inspection Service are in the process of making the necessary updates to their websites.

Officials from EPA stated that their discrepancy we identified was a result of a joint correction request sent to both OMB and EPA where OMB served as the lead agency. Thus, OMB posted the correction request on its website rather than EPA, but EPA included the correction request in its total IQA request number for fiscal year 2013 to OMB. Interior officials told us that they have not designated specific time frames for posting correction requests online. They explained, however, that one of their component agencies recently split into two separate agencies and as a result, one of these agencies is in the process of developing its own information quality program. According to agency officials, this component agency received and responded to a 2014 IQA correction request, but as of October 2015 had not posted the information on its website. Interior officials stated they expected the data discrepancy issue we identified to be resolved by the end of this fiscal year. OMB staff agreed that agencies sometimes have challenges in accurately and timely reporting and posting IQA corrections requests received and that they are communicating with agencies to address any discrepancies.

A 2009 OMB memorandum on open government states that, the “timely publication of information is an essential component of transparency.” The memorandum adds that “delays should not be viewed as an inevitable and insurmountable consequence of high demand.” However, the memo does not provide guidance on what is considered timely publication. We found that 3 of the 9 agencies that reported fiscal year 2014 correction requests to OMB had not posted IQA correction requests and responses a year or more after the end of the fiscal year. Timely reporting of IQA data would increase the transparency of the process and allow the public to view all current correction requests, agency responses to those requests, and any appeals. Doing so would also allow the public to track the status of correction requests that may be of particular interest.

Sources of IQA Correction Requests Varied

In our review of the written correction requests received by the agencies, we found that most requesters who submitted IQA correction requests self-identified as part of the submission process. As shown in table 3, we found that during fiscal years 2010 through 2014, 58 percent (50 of 87) of the correction requests originated from trade associations and advocacy organizations. Trade associations that submitted correction requests represented several different types of industries including, for example, the Western Energy Alliance which represents more than 450 companies engaged in exploration and production of oil and natural gas in the West, and the Pacific Coast Shellfish Growers Association whose membership is composed of shellfish growers in California, Oregon, Washington, Alaska and Hawaii. Advocacy organizations represented several different interests, including the San Juan Citizens Alliance, which is concerned with public land issues, and the Washington Area Bicyclist Association, whose mission is to create a healthy, more livable region by, among other things, promoting bicycling for fun, fitness, and affordable transportation. Private citizens submitted the next largest number of correction requests at 18 percent (16 of 87). We found that each of the 4 IQA correction requests submitted to the Federal Communications Commission originated from private citizens. Businesses, such as electricity producer PacifiCorp, submitted 15 percent (13) of the IQA correction requests. Local governments, such as California’s County of Siskiyou Board of Supervisors submitted 7 percent (6) of the correction requests. Each of the 6 IQA correction requests submitted by local governments was directed to Interior.

There were 68 different requesters among the 87 IQA correction requests received during fiscal years 2010 through 2014. Although the majority of requesters submitted 1 request, several submitted more. Of those requesters submitting multiple requests, 6 submitted requests to more than one agency. For example, Public Employees for Environmental Responsibility, an advocacy organization, submitted 6 correction requests in total: 2 to EPA, 1 to Commerce, 1 to the Consumer Product Safety Commission (CPSC), 1 to the General Services Administration, and 1 to Interior during fiscal years 2010 through 2014.

The Majority of IQA Correction Requests Questioned Agencies’ Use of Data

We analyzed each of the 87 IQA correction requests posted on agencies’ websites and categorized the requests into two categories—data and administrative. The majority of correction requests received by the 16 agencies during fiscal years 2010 through 2014 (66 of 87 requests, or about 76 percent) questioned either agencies’ use of underlying data or agencies’ interpretation of the data. The following IQA requests received by agencies from fiscal years 2010 through 2014 illustrate the diversity of IQA correction requests involving data.

On November 12, 2013, an advocacy organization stated that CPSC disseminated a product recall announcement based on inaccurate data; specifically claims of defects in design, warnings, and instructions. Among other things, the requester asked that CPSC disclose the statistical and scientific metrics used to determine that the subject posed “a very serious hazard.” On March 13, 2014, CPSC stated that the nature of the correction request was the subject of an ongoing adjudicative proceeding. Thus, CPSC made no corrections.

On June 11, 2010, a trade association sent a correction request to both EPA and the Department of Housing and Urban Development (HUD) on, among other things the accuracy of data used in a public service advertising on childhood lead poisoning prevention. It requested that both agencies withdraw their participation in and sponsorship of the advertisements. On December 30, 2011, EPA and HUD issued a joint response letter stating that the quality of the information included in the childhood lead prevention advertisements was thoroughly reviewed. Thus, neither agency made corrections.

We found that some IQA correction requests (18 of 87 or 21 percent) were administrative in nature. Examples of these correction requests include, among other things, typographical changes or other text revisions to update agency documents and websites.

On December 23, 2010, a private citizen submitted a correction request identifying patent images that he believed to be incorrectly labeled with another patent number in an online database. He requested that the U.S. Patent and Trademark Office within the Department of Commerce (Commerce) correct the images. On January 6, 2011, that office stated the requested correction had been made in full, and that the correct patent had been rescanned and reloaded to the database.

On November 14, 2011, a business submitted a correction that identified two errors—a typographical error and the omission of information to a Final Rule published in the Federal Register—and requested that EPA make corrections to both. On February 14, 2012 EPA agreed with the typographical error and stated that a data table was inadvertently removed from the published information. EPA stated that it was preparing a regulatory fix intended to reinstate the portions of the table that were inadvertently removed from the final rule.

Agencies Determined No Corrections Were Warranted for the Majority of IQA Correction Requests

Of the 87 IQA correction requests agencies received, agencies determined in 59 cases (68 percent) that the request did not warrant any change to the original document or data in question (see table 4). Agencies made full corrections in 11 cases and made partial corrections for 15 of the IQA correction requests received. Two correction requests were still pending as of November 2015.

The IQA correction mechanism includes procedures for requesters to appeal initial agency decisions. During fiscal years 2010 through 2014, requesters appealed agency decisions in 19 IQA cases. IQA guidelines allow requestors to file for reconsideration if they disagree with an agency’s initial response. Of the 19 appealed cases, agencies made no corrections to 15, rejected 1 because the appeal was not submitted within the specified time frame, dismissed 1 as it was withdrawn by the requester, and had not made final decisions in the last 2 cases as of November 2015.

Processes Other than IQA are Available to the Public to Request Corrections of Agency Information

IQA is one of several processes available to the public for requesting corrections of agency information. In addition to IQA, other administrative mechanisms for correcting information available to the public include notice-and-comment rulemaking and peer reviews. We previously reported in 2006 that some agencies had the flexibility to respond to correction requests through various processes, because those processes were in place prior to IQA. For example, we reported that the Federal Emergency Management Agency no longer classified requests to correct flood insurance rate maps as IQA requests. Instead, the agency addressed flood insurance rate map correction requests by using a correction process it had implemented prior to the enactment of IQA.

In this review, we found that one-fourth (15 of 59) of the IQA correction requests that resulted in no corrections were processed through an administrative mechanism other than the dedicated IQA request for correction process. According to OMB staff, agencies may respond to correction requests through the applicable administrative process. For example, agencies may process correction requests using notice-and- comment for rulemaking under the Administrative Procedure Act in instances where the request concerns a proposed rule and the comment process is still open. Processing such requests under IQA, as we reported in 2006, could impact rulemaking outside of the rulemaking process by affecting when or if an agency initiates a rulemaking. The following is an example of an agency response to a correction request submitted under IQA that the agency determined should be addressed through the rulemaking process:

On July 1, 2010, a non-profit organization submitted a request to EPA to “rescind and correct online and printed information regarding alleged greenhouse gas emissions reductions resulting from ‘beneficial use’ of coal combustion waste products.” On February 16, 2011, EPA responded that many of the specific documents in question served as background technical support materials for EPA’s proposed rulemaking to address the risks from the disposal of coal combustion residuals generated by electric utilities and independent power producers. As a result, the agency would address the issues of the correction request through the rulemaking process for the rule.

The peer review process allows the public an opportunity to provide comments and to question an agency’s use of data before it actually disseminates the information. The following is an example of an agency response to a correction request submitted under IQA that EPA determined should be addressed through public comments during the peer review process.

On August 20, 2010, EPA received a correction request from a private citizen requesting EPA to, among other things, correct information used to develop the Draft Benthic Total Maximum Daily Load Development for Accotink Creek, Virginia. On November 15, 2010, EPA responded that the public comment response process would be used to address the concerns outlined in the correction request. EPA stated that all public comments would be considered during the revision of the draft document. EPA also stated that to “avoid duplicate actions that would interfere with the ongoing Total Maximum Daily Load Development process, we will not use the EPA Information Quality Guidelines Request for Correction process to respond” to the correction request.

The public may not be aware of the different administrative processes agencies have available to address correction requests submitted under IQA. As a result, agencies’ IQA staff may be tasked with responding to a number of correction requests outside of the dedicated IQA request for correction process. Including explanations and links on agencies’ IQA websites to other available correction processes that might be more appropriate to the public’s needs could help increase efficiencies for all available information correction processes. Although OMB staff told us that agencies should, in their response to public correction requests, state whether those agencies plan to address the requests through other administrative processes, current OMB IQA guidance does not address this issue. However, we found that at least one agency has included in its online IQA guidance information for submitting correction requests outside of IQA. EPA included additional information on its IQA web page that informs the public on how to report and correct EPA website data errors as well as how to seek correction on information for which EPA has sought public comment.

Agencies cited other reasons for not addressing a number of correction requests submitted under IQA. These included requests related to cases under litigation, requests too broad in nature (not specific), and requests the agencies deemed to lack merit. In addition, agencies did not recognize correction requests where the data in question were contained in a document not subject to IQA (such as a press release or a document not created by the agency). Specific examples follow.

In 2011, a private citizen requested the National Oceanic and Atmospheric Administration (NOAA), which operates within Commerce, to modify information about the location where Tropical Storm Kirsten made landfall in 1966 in Mexico. NOAA responded that the information in question was not subject to the requirements of IQA as the data were considered to be archival (data disseminated by NOAA before October 1, 2002, are considered to be archival information).

On March 30, 2011, an advocacy organization submitted a request to the National Park Service, which operates within Interior, to correct information that the requestor deemed as “unfounded scientific conclusions” in a report on allegations of scientific misconduct at Point Reyes National Seashore. On June 6, 2011, Interior responded that the document in question was a report of an investigation undertaken by the Office of the Solicitor. The investigation looked to resolve allegations of scientific misconduct on the part of employees. Also, the “report was generated as part of the adjudicative process of this personnel matter; it is not subject to review under the IQA.”

OMB and Agencies’ IQA Guidelines and Related Information Is Generally Available Online, but Usability Varied

OMB Makes IQA Information, Including Agency-Reported Data, Publicly Available; However Data Are Not Centrally Located

OMB staff told us they rely heavily on their own website to disseminate IQA, OMB-specific, and government-wide guidance. OMB’s information quality website includes guidelines from OMB that describe its policy for ensuring the quality of information that it disseminates to the public. The guidelines also establish the administrative procedure by which an affected person may obtain correction of information disseminated by OMB. In addition, OMB includes links the public and other interested parties can use to locate individual agency information quality guidelines, government-wide information quality guidelines, and OMB’s annual reports to Congress. OMB’s reports to Congress, included on a separate OMB web page from IQA guidelines, include brief updates on agency reporting under the government-wide information quality guidelines.

As we previously stated, agencies are required by IQA to report to OMB annually on the number and type of correction requests received, as well as their respective responses. Although not required, since 2003 OMB has published agency-reported IQA data from the previous fiscal year in an annual report to Congress. OMB also makes this information available on its website but the data are dispersed across multiple web pages which could make the information hard to find and could contribute to user confusion. For example, OMB provides links to the annual reports on its website where the public and interested parties may access the information (see figure 1). However, there is no central location on OMB’s website where the IQA data are located, for example, in a table or some other format by year or agency. Instead, interested parties would need to go to each separate annual report link, search for IQA data, collect the data, and create their own table to review IQA data government-wide from year to year.

Enabling the public to better access information is one of the principles of the President’s digital government strategy. According to the strategy, the federal government must fundamentally shift how it thinks about digital information. To drive this shift, agencies must, among other things, be customer-centric to focus on customer needs. This means that quality information should be accessible, current, and accurate at any time. Federal digital services guidelines direct agencies to publish digital information so that it is easy to find and access. These guidelines are aimed at helping federal agencies improve their communications and interactions with customers through websites. Although OMB has made government-wide IQA data available in its reports to Congress, finding and compiling such information may take several steps, potentially making it more difficult to access and find, thus hindering transparency. OMB officials acknowledged that consolidating and centralizing IQA information on OMB’s website could improve transparency and access to its IQA data.

Agencies Posted Required IQA Guidelines and Data Online as of November 2015 but the Usability Could Be Improved

In addition to posting correction requests and agency responses on agency websites, agencies are required by IQA to post their IQA guidelines and administrative mechanisms by which affected persons could petition for correction of inaccurate agency information. Twenty- eight of 30 agencies posted the required IQA documents online as of November 2015. However, the Department of Defense did not include administrative mechanisms on its website. In addition, we were unable to find the Federal Housing Finance Agency’s IQA guidelines anywhere on its website. OMB concurred with our review of these agencies’ IQA information and told us it would work with the agencies to improve the information provided on their websites, but as of December 2015, they had not completed that process. Until that step occurs, the public may be unaware of the steps the agencies would take upon receiving a correction request, or even how to submit a correction request.

In addition to the required IQA information, some agencies’ websites included additional features that reflect customer-centric leading web practices identified by the President’s digital government strategy such as posting IQA information on a single website to ease accessibility and identifying points of contact online. For example, the Department of Labor’s website includes points of contact at its 21 component agencies.

Such information enables the interested public with questions regarding IQA to more easily identify agency officials (see figure 2).

OMB’s guidance on posting IQA correction requests states that agencies need to establish processes for updating their information quality web pages on a regular basis but does not define regular basis. We identified agency websites where information was outdated or web links were broken. Specifically, 9 of 30 agencies posted either outdated information or included broken hyperlinks (see figures 3 and 4). Consequently, the public may be unable to access these agencies’ IQA guidelines and correction requests. Ensuring that online content is accurate is one of the guidelines for federal digital services. Easy access to current guidance could also facilitate opportunities for affected parties and stakeholders to provide feedback on those documents.

We identified five agencies that did not include on their websites any information about IQA correction requests. As a result, it is not clear by reviewing these agencies’ websites whether or not the agencies had received such requests during fiscal years 2010 through 2014. Specifically, we could not identify any language stating whether or not the Departments of Energy, Homeland Security, Justice, and Transportation, and the Office of Personnel Management had received correction requests as of November 2015. In addition, we found that as of November 2015:

The Consumer Product Safety Commission’s website included links to the IQA correction requests the agency had received. However, there was no text indicating whether or not the agency had received IQA corrections requests for years where no correction requests were posted or whether that information was simply missing.

The Department of Agriculture’s website included links to IQA data reports for fiscal years 2010 through 2013, but had no information regarding fiscal year 2014.

The Department of Housing and Urban Development’s website did not include IQA data reports for fiscal years 2012 through 2014.

The Federal Housing Finance Agency’s website did not include IQA information for fiscal year 2014.

As noted earlier, OMB’s guidance states that agencies also need to establish processes for regularly updating their information quality websites. However OMB staff told us that if an agency has not received any IQA requests in a given fiscal year, they are not required to report that information on their websites. Without that acknowledgement however, it may be unclear to the public whether an agency has received IQA correction requests but has not posted them or whether the agency has in fact not received any requests. OMB staff agreed that clearly stating whether or not agencies had received IQA correction requests could improve the transparency of IQA. Even when agencies posted IQA information on their website as OMB required, such information is sometimes outdated making it difficult for users to know whether agencies have received correction requests or how to request correction of agency information. OMB staff acknowledged that additional OMB guidance that specifies time frames for agencies to post information on IQA requests received, requires explanations and links to other agency information correction processes, and provides suggestions for improving the usability of agency websites would be useful.

Selected Agencies Have Processes in Place to Implement IQA and Officials Reported Challenges and Benefits

Agency officials at six selected agencies—the Departments of Agriculture (USDA), Commerce, Health and Human Services (HHS), Interior, and Transportation, and the Environmental Protection Agency (EPA)—took a range of actions as part of their efforts to implement the IQA correction process and to better track and address correction requests received, such as the following examples.

According to EPA officials, EPA’s centralized IQA process has provided greater oversight on correction requests from receipt to final response. EPA has developed internal process maps that outline the steps needed to address correction requests. Once EPA receives a correction request, EPA officials enter the request into a tracking database. Then, an acknowledgement receipt is dispersed. EPA officials then identify who within the agency is responsible for the information in question, and forward the request to the appropriate program office or region that schedules scoping meetings to review the request and draft a response. In the meantime, EPA notifies OMB of the correction request.

The Department of Health and Human Services (HHS) also has a centralized IQA correction process. The agency’s Office of the Assistant Secretary for Planning and Evaluation in the Office of the Secretary manages and coordinates the IQA process and administers the HHS information quality website and is the agency’s point of contact with OMB. The HHS component or office within HHS that originated the challenged information is responsible for developing and sending the agency’s response.

Interior’s IQA correction process is decentralized. Within each of Interior’s component agencies, Bureau Information Quality Coordinators address IQA correction requests and coordinate with Interior’s Information Quality Coordinator on the response.

Officials from 3 of the 6 selected agencies also reported challenges in implementing IQA. Department of Transportation officials told us it has been a challenge to retain IQA institutional knowledge amidst staff turnover. EPA and Interior officials both stated that allocating the necessary time to properly respond to IQA correction requests was challenging. For example, EPA officials said that the amount of time it takes to respond to a correction request can take an appreciable part of a full-time employee’s efforts during busy periods. EPA officials said additional review time and attention are required because responses to corrections must be reviewed through EPA’s internal processes for concurrence, as well as with OMB. Interior officials also told us they spend a considerable amount of time addressing the often complex and/or lengthy IQA requests as well as obtaining the necessary reviews and concurrence of the agency response.

We found that agency responses to IQA requests and an appeal have taken 2 years or longer to resolve. Although both EPA and Interior officials cited time spent in addressing correction requests to be a challenge, neither agency was able to provide estimates of agency or employee hours spent in the process. Further, none of our selected agencies had information about the actual workload or the number of staff days for responding to IQA correction requests. As a result, the impact of the IQA correction process on the selected agencies could not be accurately measured because the agencies do not have mechanisms in place to track the effects of implementing IQA. We previously reported that agency IQA officials believe addressing IQA requests is considered to be part of their agencies’ day-to-day business, and because of the multifaceted nature of some requests, allocating time and resources on specific issues or linking work exclusively to IQA requests would be difficult.

According to OMB staff, there is not a specific amount of time that is considered too long for agency responses to correction requests. They explained that IQA correction requests may take a long time for some agencies due to the extensive review that is required to make a final agency decision. OMB staff stated that they did not want to be prescriptive in IQA guidance by adding administrative time requirements to an agency specific process. The officials added that taking a long time to respond to an IQA correction request was not necessarily a bad thing. It may indicate an extensive and comprehensive review by the agency and discussion of the information in question.

Officials at our selected agencies told us they believe IQA has improved the quality of data disseminated by their agencies. For example, EPA officials told us that the quality of data disseminated by EPA is more robust due to the consideration of the diversity of viewpoints provided by the public, and the agency’s opportunity to review pertinent information that may not have been obtained by EPA. Interior officials reported to us that the IQA and peer review standards have greatly assisted in the dissemination of quality information. They stated that their guidelines give “teeth” to the objectives and requirements of quality information. According to HHS officials, the IQA process has proved to be a useful mechanism for the public to raise issues of concern to federal agencies that publicly disseminate information. OMB staff told us that the IQA process has improved agency information quality policies even though the correction request metrics may not show it. They explained that while it’s important for IQA correction numbers to be seen by the public, it is also important that the public is aware that these numbers are only a small piece of the benefits of IQA. IQA guidelines and peer reviews are all about pre-dissemination review, transparency, and ensuring that only information with good quality is released by agencies.

Conclusions

IQA allows businesses, trade associations, advocacy organizations, the public, and others to submit requests to agencies to make corrections to agency disseminated information. Some of the 30 agencies in our review reported receiving relatively few IQA requests from fiscal years 2010 through 2014. Agencies determined that the majority of correction requests received did not warrant any changes. Processes other than IQA are available to request corrections of agency information and agencies addressed a number of correction requests through administrative mechanisms other than the dedicated IQA request for correction process.

Agencies in our review have developed their own guidelines and administrative mechanisms for implementing IQA. OMB and agencies rely on their websites to disseminate guidance and also provide information regarding results of correction requests. However, we found that OMB had not consolidated all IQA data in one centralized location on its website. We also found instances where IQA required information was missing from agency websites or where information was outdated or incomplete. To be effective, guidance documents should be accessible to their intended audiences and corrective processes should be transparent. This is consistent with guidelines for federal digital services. OMB has the opportunity to build on its efforts to improve the transparency of the IQA process. For example, by consolidating summaries of agency IQA information, working with agencies to ensure all IQA requirements are met, and providing additional guidance about posting accessible, user- oriented information on agency websites, OMB could help increase the public’s access to and confidence in that information, thereby helping to further the goal of disseminating quality information.

Recommendations for Executive Action

To better ensure agencies fulfill their requirements, including implementing IQA guidelines and helping to promote easier public access to IQA information on agency websites, we recommend that the Director of OMB take the following actions:

Consolidate and centralize on OMB’s IQA guidance website a government-wide summary of requests for correction submitted under the IQA.

Work with the Department of Defense and the Federal Housing Finance Agency to help ensure that they post their IQA administrative mechanisms and IQA guidance online.

Provide additional guidance for agencies to help improve the transparency and usability of their IQA websites to help ensure the public can easily find and access online information about agency IQA implementation. Such guidance should include specific time frames for agencies to post information on the IQA correction requests they have received, including making it clear when agencies have not received IQA requests; instructions for agencies to include a statement on their IQA websites that the agencies may address correction requests through other administrative processes; instructions for agencies to include, when responding to correction requests, whether those agencies plan to address the request through another administrative processes, and if so, which process they will use; and suggestions for improving usability of agencies’ websites including fixing broken links.

Agency Comments and Our Evaluation

We provided a draft of this report to the Director of the Office of Management and Budget. In oral comments received on December 1, 2015, OMB staff discussed our findings, conclusions, and draft recommendations. They provided technical comments, which are incorporated into the report where appropriate. In response to this discussion, we made minor revisions to the draft and recommendation language to more accurately reflect the role of agencies in responding to correction requests along with OMB’s role in overseeing these activities. The OMB staff stated they agreed with our modified recommendations.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Director of OMB and other interested parties. In addition, the report will be available at no charge on the GAO website at www.gao.gov.

If you or your staff have any questions about this report, please contact me at 202-512-2757 or GoldenkoffR@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of this study were to (1) identify the number, source, and final disposition of IQA correction requests received by the 24 Chief Financial Officer (CFO) Act and other agencies for fiscal years 2010 through 2014; (2) assess the extent to which the 24 CFO Act and other agencies that received correction requests made IQA information publicly available; and (3) identify how selected agencies have implemented IQA.

To address the first objective, we searched the websites of the 24 Chief Financial Officer (CFO) Act agencies and identified 10 of these that had posted correction requests and responses online for fiscal years 2010 through 2014. We selected 2014 as an ending point for a 5-year analysis because 2014 was the most recent, complete fiscal year of data available. To identify other agencies that had received correction requests during the same time frame, we reviewed the Office of Management and Budget’s (OMB) annual reports to Congress for fiscal year 2010 through fiscal year 2013 and identified those agencies outside of the 24 CFO Act agencies that reported receiving Information Quality Act (IQA) correction requests. OMB provided us with agency-reported data for fiscal year 2014 because the report to Congress had not yet been issued. From this, we identified an additional 6 non-CFO Act agencies that posted IQA correction requests and responses on their websites—the Consumer Product Safety Commission, Federal Communications Commission, Federal Housing Finance Agency, Federal Reserve Board, Office of Management and Budget, and the Office of Science and Technology Policy. We reviewed relevant OMB and agency documents, including IQA guidelines and agencies’ annual reports to OMB, examined requests and appeals to correct agency information, and reviewed OMB’s and agencies’ websites. To supplement the documentary evidence obtained, we interviewed agency officials responsible for IQA in their respective agencies. We also interviewed current and former OMB staff to provide additional context on IQA. During the course of our review, we compared agency IQA data posted on their websites with IQA data agencies reported to OMB and identified discrepancies. We discussed the discrepancies with OMB staff and agency officials and included their responses within the report. We determined that OMB and agency data were sufficiently reliable to provide a general indication of the numbers of correction requests received. Although agencies have other processes to correct agency disseminated information, we evaluated only information related to the IQA correction mechanism.

We assessed relevant agency IQA documents—including guidelines, requests and appeals, agency decisions, and related documents—found on the 16 agency websites that posted correction requests during our identified time frame. To supplement and verify the accuracy and completeness of this information, we interviewed OMB IQA staff. Moreover, to better understand specific aspects of IQA requests and how agencies addressed them, as well as to illustrate specific points, we reviewed in detail all of the correction requests posted on agency websites to the extent such information was available online. Two analysts independently assessed each agency’s correction request and final agency response to determine requester type, request category, agency response and justification for response, and resolved all discrepancies. To categorize the sources of the requests by type of entity, such as business, trade association, or advocacy organization, we relied on information and descriptions the requester provided in the correction requests. Specifically, the majority of requesters self-identified as one of the following types of requesters—trade association/advocacy organization, business, private citizen, local government, or anonymous— in their correction requests to the agencies. However, when such information was not available, we searched the requester’s name online and used the descriptions found therein to make our determination as to the type of entity. To determine the final disposition of IQA requests and any appeals, we reviewed related agency documents, including interim agency correspondence, to determine whether or not the agency committed to make a correction(s) in response to the request. We determined a correction was a partial correction if the agency made at least one change based on the request, for example adding clarifying language or additional references.

To address the second objective, we conducted an analysis of the 24 CFO Act agencies’ websites, as well as the six other agencies identified in objective one as having received IQA correction requests during our selected timeframe, using internal site search engines and search terms, such as “information quality,” “correction request,” and “IQA guidelines,” to determine whether they had IQA guidelines and other IQA information online. We identified and used IQA search terms and steps to review and find information on agency publicly available web pages consistent with best practices guidance for search engine optimization from digitalgov.gov’s website. We also used OMB’s Open Government Directive in assessing IQA guidance documents. We compared the information found on the websites to IQA requirements outlined in OMB guidance to agencies on posting IQA documents. We also reviewed other OMB and relevant government guidance on design features to make government-wide information and data accessible. When we found instances where agencies had not posted the required guidelines or administrative mechanisms, we contacted OMB staff for verification.

To identify IQA processes and challenges agencies face in implementing IQA, we selected a non-generalizable sample of six agencies -– the Departments of Agriculture, Commerce, Health and Human Services, Interior, and Transportation, and the Environmental Protection Agency -– to obtain illustrative examples of how they approached and implemented IQA. We selected these agencies based in part on the number (both high and low to include a range) of IQA correction requests the agencies had received from fiscal years 2010 through 2014. We also included one agency (Department of Transportation) based on the relatively high number of peer reviews conducted during the same time frame. We interviewed OMB and agency officials responsible for addressing IQA correction requests to gather their perceptions on the overall IQA process.

We conducted this performance audit from November 2014 to December 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.

Appendix II: Organizations That Filed Information Quality Act Correction Requests during Fiscal Years 2010 through 2014

Appendix II: Organizations That Filed Information Quality Act Correction Requests during Fiscal Years 2010 through 2014 Federal agency receiving request and filer Consumer Product Safety Commission Public Employees for Environmental Responsibility Public Employees for Environmental Responsibility The Association of Proprietary Colleges Department of Health and Human Services Capital Strategy Consultants, Inc.

Center for Regulatory Effectiveness (2)

International Premium Cigar and Pipe Retailers Association Styrene Information and Research Center (2)

Federal agency receiving request and filer Pacific Coast Shellfish Growers Association Pavement Coatings Technology Council (2)

United States Association of Reptile Keepers and Pet Industry Joint Advisory Council Western Energy Alliance (2)

Environmental Protection Agency American Coatings Association American Chemistry Council (2)

Artisan EHS Consulting, LLC The Competitive Enterprise Institute and ActionAid USA Halogenated Solvents Industry Alliance, Inc.

International Platinum Group Metals Association Organic Arsenical Products Task Force and Wood Preservative Science Council Public Employees for Environmental Responsibility (2)

U.S. Chamber of Commerce (2)

Walter Coke, Inc.

W.R. Grace & Co. Conn.

Appendix III: Contacts and Staff Acknowledgments

GAO Contact

Robert Goldenkoff, (202) 512-2757 or goldenkoffr@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Clifton G. Douglas, Jr. (Assistant Director), Dewi Djunaidy (Analyst-in-Charge), Joseph Fread, Lisette Baylor, Michele Fejfar, Ellen Grady, Farrah Graham, Andrea Levine, and Stewart Small made key contributions to this report.