Scientific Integrity: EPA's Efforts to Enhance the Credibility and Transparency of Its Scientific Processes
Released: 2009-06-09T00:00:00

Letter

EPA Reforms Have the Potential to Significantly Improve IRIS, but EPA Could Clarify Some Issues

In March 2008, we reported that the IRIS database—a critical component of EPA’s capacity to support scientifically sound risk management decisions, policies, and regulations—was at serious risk of becoming obsolete because the agency had not been able to complete timely, transparent, and credible chemical assessments or decrease its backlog of ongoing assessments. In addition, assessment process changes EPA had recently made, as well as other changes EPA was considering at the time of our review, would have further reduced the credibility, transparency, and timeliness of IRIS assessments. Among other things, we concluded the following: EPA’s efforts to finalize IRIS assessments have been impeded by a combination of factors. These factors include (1) the Office of Management and Budget’s (OMB) requiring two additional reviews of IRIS assessments by OMB and other federal agencies with an interest in the assessments, such as the Department of Defense, and (2) EPA management decisions, such as delaying some assessments to await the results of new research.

The two new OMB/interagency reviews of draft assessments involve other federal agencies in EPA’s IRIS assessment process in a manner that limits the credibility of IRIS assessments and hinders EPA’s ability to manage them. For example, some of the agencies participating in these reviews could face increased cleanup costs and other legal liabilities if EPA issued an IRIS assessment for a chemical that resulted in a decision to regulate the chemical to protect the public. Moreover, the input these agencies provide to EPA is treated as “deliberative” and is not released to the public. Regarding EPA’s ability to manage IRIS assessments, without communicating its rationale for doing so, OMB required EPA to terminate five assessments that for the first time addressed acute, rather than chronic, exposure—even though EPA had initiated this type of assessment to help it implement the Clean Air Act.

The changes to the IRIS assessment process that EPA was considering, but had not yet issued at the time of our 2008 review, would have added to the already unacceptable level of delays in completing IRIS assessments and would have further limited the credibility of the assessments. For example, the changes would have allowed potentially affected federal agencies to have assessments suspended for up to 18 months to conduct additional research. As we reported in 2008, even one delay can have a domino effect, requiring the assessment process to essentially be repeated to incorporate changing science.

In April 2008, EPA issued a revised IRIS assessment process. The process was largely the same as the draft process we had evaluated during our review and did not respond to the recommendations in our March 2008 report. Moreover, some key changes were likely to further exacerbate the productivity and credibility concerns we initially identified. For example, EPA’s revised process formally defined comments on IRIS assessments from OMB and other federal agencies as “deliberative” and excluded them from the public record. As we stated in our report, it is critical that input from all parties—particularly agencies that may be affected by the outcome of IRIS assessments—be publicly available. In addition, we concluded that the estimated time frames under the revised process, especially for chemicals of key concern, would likely perpetuate the cycle of delays to which the majority of ongoing assessments have been subject. Instead of streamlining the process, as we had recommended, EPA institutionalized a process that from the outset was estimated to take 6 to 8 years for some widely used chemicals that are likely to cause cancer or other serious health effects. This was particularly problematic because of the substantial rework such cases often require to take into account changing science and methodologies.

Largely as a result of EPA’s lack of responsiveness, we added transforming EPA’s processes for assessing and controlling toxic chemicals as a high- risk area in our January 2009 biennial status report on governmentwide high-risk areas requiring increased attention by executive agencies and Congress. Taking positive action, on May 21, 2009, EPA issued a new IRIS assessment process, effective immediately. In a memorandum announcing the reforms to the IRIS assessment process, the EPA Administrator echoed our prior findings that the April 2008 changes to the process reduced the transparency, timeliness, and scientific integrity of the IRIS process. She noted that the President’s recent emphasis on the importance of transparency and scientific integrity in government decision making compelled a rethinking of the IRIS process. If effectively implemented, the new process would be largely responsive to the recommendations outlined in our March 2008 report.

First, the new process and the memorandum announcing it indicate that the IRIS assessment process will be entirely managed by EPA, including the interagency consultations (formerly called OMB/interagency reviews). Under EPA’s prior process, these two interagency reviews were required and managed by OMB—and EPA was not allowed to proceed with assessments at various stages until OMB notified EPA that it had sufficiently responded to comments from OMB and other agencies. The independence restored to EPA under the new process is critical in ensuring that EPA has the ability to develop transparent, credible IRIS chemical assessments that the agency and other IRIS users, such as state and local environmental agencies, need to develop adequate protections for human health and the environment.

Second, the new process addresses a key transparency concern highlighted in our 2008 report and testimonies. As we recommended, it expressly requires that all written comments on draft IRIS assessments provided during the interagency consultation process by other federal agencies and White House offices be part of the public record.

Third, the new process streamlines the previous one by consolidating and eliminating some steps. Importantly, EPA eliminated the step under which other federal agencies could have IRIS assessments suspended in order to conduct additional research, thus returning to EPA’s practice in the 1990s of developing assessments on the basis of the best available science. As we highlighted in our report, as a general rule, requiring that IRIS assessments be based on the best science available at the time of the assessment is a standard that best supports the goal of completing assessments within reasonable time periods and minimizing the need to conduct significant levels of rework.

Fourth, as outlined in the EPA Administrator’s memorandum announcing the new IRIS process, the President’s fiscal year 2010 budget request includes an additional $5 million and 10 full-time-equivalent staff positions for the IRIS program, which is responsive to our recommendation to assess the level of resources that should be dedicated in order to meet user needs and maintain a viable IRIS database.

We are encouraged by the efforts EPA has made to adopt most of our recommendations, including those addressing transparency practices and streamlining the lengthy IRIS assessment process. The changes outlined above reflect a significant redirection of the IRIS process that, if implemented effectively, can help EPA restore the integrity and productivity of this important program. Nevertheless, on the basis of our preliminary review of the new IRIS assessment process, we have some initial questions that EPA may wish to consider as it implements its new process. For example, regarding integrity and transparency, it is not clear whether any significant agreements reached among the federal agencies during interagency consultation meetings will be documented in the public record, since the new policy specifies only that written comments provided by other federal agencies will become part of the public record; and why comments from other federal agencies cannot be solicited at the same time the initial draft is sent to independent peer reviewers and public comments are solicited. This change would enhance transparency and would further reduce overall assessment time frames. Specifically, the public and peer reviewers could have greater assurance that the draft had not been inappropriately biased by policy considerations of other agencies, including those that may be affected by the outcome, such as the Department of Defense and the Department of Energy.

In addition, the new assessment process states that “White House offices” will be involved in the interagency consultation process but does not indicate which offices. Given that (1) EPA will be performing the coordinating role that OMB exercised under the prior process and (2) the purpose of these consultations is to obtain scientific feedback, it is unclear whether OMB will continue to be involved in the interagency consultation process.

EPA Has Improved the Policies and Procedures of Its Science Advisory Board, but Their Wider Use by Other EPA Scientific Advisory Committees Could Enhance EPA’s Scientific Integrity

Independent, expert peer review of EPA’s scientific and regulatory products, such as risk assessments and proposed rules, is integral to the agency’s ability to effectively protect public health and the environment. Specifically, using peer review, EPA seeks to enhance the quality and credibility of the agency’s highly specialized products. One of the several ways EPA obtains expert peer review is from advice and recommendations it requests of its 24 federal advisory committees comprising independent experts. For example, since its inception in 1978, one of EPA’s largest and most prominent federal advisory committees— the EPA Science Advisory Board—has convened hundreds of peer review panels to assess the scientific and technical rationales underlying a wide range of current or proposed EPA regulations and policies. The IRIS program uses Science Advisory Board panels to peer review some of its particularly complex chemical assessments, and the Board is currently expanding a panel that will review existing IRIS assessment values established more than 10 years ago. Federal advisory committees such as the Science Advisory Board are subject to the requirements of the Federal Advisory Committee Act (FACA), which include broad requirements for balance, independence, and transparency.

To be effective, peer review panels must be—and also be perceived to be—free of any significant conflict of interest and uncompromised by bias. Peer review panels should also be properly balanced, allowing for a spectrum of views and appropriate expertise.

These standards, reflected in the act, are important because the work of fully competent peer review panels can be undermined by allegations of conflict of interest and bias.

In 2001, we reported on limitations in the policies and procedures developed by EPA’s Science Advisory Board to ensure that its panels’ peer reviewers are independent and that a balance of viewpoints is represented on each panel. These limitations could reduce the effectiveness of the Board overall by contributing to its being perceived as biased and could inadvertently expose some panelists to violations of federal conflict-of- interest laws. Demonstrating a strong commitment to the integrity of its peer reviews, EPA took a number of actions to implement our report’s recommendations, including establishing a standard process for Science Advisory Board panel formation that includes a requirement to document decisions about conflicts of interest and balance of viewpoints and expertise in forming each panel, as well as prospective panelists’ responses to several standardized questions aimed at assessing impartiality; developing a new confidential financial disclosure form designed to capture needed information to evaluate potential conflicts of interest; allowing the public to review a “short list” of candidates selected for a specific Science Advisory Board panel and to comment on the appropriateness of including any of these candidates on the panel; and developing CD-based conflict-of-interest training for Science Advisory Board panelists.

In 2004, we reported on the policies and procedures at nine federal departments and agencies, including EPA, that extensively use federal advisory committees. We also identified practices that promote independence and balance used by the National Academies and the EPA Science Advisory Board. Regarding the latter issue, we concluded that the National Academies and the EPA Science Advisory Board have developed clear processes that, if effectively implemented, can provide these organizations with an assurance that relevant conflicts of interest are identified and addressed—and that committees are appropriately balanced in terms of points of view. Specifically, we found that the processes used by the National Academies and EPA’s Science Advisory Board clearly and consistently identify the information they deem necessary to assess candidates for independence and to balance committees, explain to the candidates why the required information is important to protect the integrity of the committee’s work, request public comment on proposed committee membership, and require evaluation of the overall balance of committees before committees are finalized.

Regarding the federal advisory committee policies and procedures at nine departments and agencies, in 2004 we found that the Departments of Agriculture, Energy, and the Interior had a long-standing practice of appointing most or all members of their federal advisory committees as “representatives”—expected to reflect the views of the entity or group they are representing and not subject to conflict-of-interest reviews—even when the departments called upon the members to provide advice on behalf of the government on the basis of their best judgment and thus should have appointed them as special government employees. That is, members of federal advisory committees that are providing advice on behalf of the government should be appointed as “special government employees”—short-term or intermittent employees subject, with some important modifications, to the conflict-of-interest requirements applicable to other federal employees. We also reported that representative appointments are generally not appropriate for scientific and technical advisory committees, which typically provide advice on behalf of the government. We made recommendations to the two agencies responsible for overseeing aspects of federal advisory committees to, among other things, provide additional guidance to federal agencies on the appropriate use of representative appointments. In response, these agencies issued such guidance in 2004 and 2005. (See appendix I for additional information on our 2004 federal advisory committee recommendations.)

The two scientific EPA federal advisory committees we assessed in our 2004 report appropriately appointed their members as special government employees. We note that 16 of the 24 EPA federal advisory committees currently use representative appointments, according to the government’s database of federal advisory committee information. While EPA may be appropriately seeking stakeholder advice from some of these advisory committees, a number of its committees focus on scientific and technical questions for which EPA is likely to be seeking advice on behalf of the government on the basis of committee members’ best judgment, rather than stakeholder advice. EPA’s scientific and technical committees using representative appointments include the National Advisory Committee for Acute Exposure Guideline Levels for Hazardous Substances, the Coastal Elevations and Sea Level Rise Advisory Committee, the Environmental Laboratory Advisory Board, and the Children’s Health Protection Advisory Committee. In reviewing information about EPA’s committees, we found that descriptions of the objectives and scope of committee activities for EPA committees using representative appointments are similar to such descriptions for EPA committees using special government employees, such as the Science Advisory Board; the Federal Insecticide, Fungicide, and Rodenticide Science Advisory Panel; the National Drinking Water Advisory Council; and the Human Studies Review Board.

As EPA moves forward with actions to enhance its scientific integrity, it will be appropriate for the agency to review its federal advisory committee appointments, especially those for which it appoints members as representatives, to help ensure that committee work is not jeopardized by allegations of conflict of interest or bias. As discussed earlier, committee members appointed as representatives are not evaluated for potential conflicts of interest. If some EPA committee members are inappropriately appointed as representatives, EPA cannot be assured that any real or perceived conflicts of interest of their committee members who provided advice on behalf of the government were identified and appropriately mitigated. Further, allegations that the members had conflicts of interest could call into question the independence of the committee and jeopardize the credibility of the committee’s work.

Advisory committee charters generally expire at the end of 2 years unless renewed by the agency or Congress. The EPA committees with representative members discussed earlier have charters expiring in 2009 and 2010. As it reviews its policies and procedures to ensure scientific integrity, EPA could either comprehensively review the appointments of its 16 committees with representative members or, alternatively, review them as the charters are renewed. We note that EPA has in-house expertise in managing federal advisory committees composed of special government employees—for example, the staff who administer and coordinate Science Advisory Board committees—and thus should be well positioned to address this issue.

In conclusion, EPA’s most recent changes to the IRIS assessment process, if effectively implemented, would represent a significant improvement over the process put in place in 2008. Among other things, the reforms appropriately restore EPA’s control of the IRIS process and increase the transparency of the process. In addition, EPA was responsive to our 2001 recommendations for improving the independence and balance of committees convened by EPA’s Science Advisory Board by developing policies and procedures that represent best practices. As a result, if these policies and procedures are implemented effectively, EPA can have an assurance that its Science Advisory Board panels are independent and balanced as a whole. However, a number of EPA’s other federal advisory committees do not appear to have benefited from the steps the Science Advisory Board has taken to enhance the integrity and transparency of its committees. As EPA takes additional steps to comply with the President’s March 9, 2009, memorandum on scientific integrity, we believe that EPA’s scientific processes could be further enhanced by considering our questions about some aspects of the IRIS assessment process and reviewing its federal advisory committee appointments.

Madam Chairman, this concludes my prepared statement. I would be happy to respond to any questions that you or other Members of the Committee may have at this time.

GAO Contacts and Staff Acknowledgments

For further information about this testimony, please contact John B. Stephenson at (202) 512-3841 or stephensonj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Contributors to this testimony include Christine Fishkin (Assistant Director), Laura Gatz, Richard P. Johnson, Summer Lingard, Nancy Crothers, Antoinette Capaccio, and Carol Kolarik.

Appendix I: Information on GAO’s 2004 Federal Advisory Committee Recommendations

Following are highlights of the recommendations in our 2004 report, Federal Advisory Committees: Additional Guidance Could Help Agencies Better Ensure Independence and Balance, to the General Services Administration (GSA) and the Office of Government Ethics (OGE). These agencies oversee aspects of federal advisory committees. Specifically, GSA develops guidance on establishing and managing Federal Advisory Committee Act (FACA) committees, and OGE develops regulations and guidance for statutory conflict-of-interest provisions that apply to special government employees.

Our 2004 report contained recommendations to GSA and OGE to, among other things, provide additional guidance to federal agencies on the appropriate use of representative appointments. Specifically, we recommended that guidance from OGE to agencies be improved to better ensure that members appointed to committees as representatives were, in fact, representing a recognizable group or entity. OGE agreed that some agencies may have been inappropriately identifying certain advisory committee members as representatives instead of special government employees and issued guidance documents in July 2004 and August 2005 that clarified the distinction between special government employees and representative members. In particular, as we recommended, OGE’s clarifications included that (1) members should not be appointed as representatives purely on the basis of their expertise and (2) appointments as representatives are limited to circumstances in which the members are speaking as stakeholders for the entities for groups they represent.

We also recommended that OGE and GSA modify their FACA training materials to incorporate the changes in guidance regarding the appointment process, which they have done. In addition, we recommended that GSA expand its FACA database to identify each committee member’s appointment category and, for representative members, the entity or group represented. GSA quickly implemented this recommendation and now has data on appointments beginning in 2005. Finally, we recommended that OGE and GSA direct agencies to review their appointments of representative and special government employee committee members to make sure they are appropriate. OGE’s 2004 and 2005 guidance documents addressed this issue by, among other things, recommending that agency ethics officials periodically review appointment designations to ensure they are proper.

Related GAO Products

High-Risk Series, An Update. GAO-09-271. Washington, D.C.: January. 2009.

EPA Science: New Assessment Process Further Limits the Credibility and Timeliness of EPA’s Assessments of Toxic Chemicals. GAO-08-1168T. Washington, D.C.: September 18, 2008.

Environmental Health: EPA Efforts to Address Children’s Health Issues Need Greater Focus, Direction, and Top-Level Commitment. GAO-08-1155T. Washington, D.C.: September 16, 2008.

Chemical Assessments: EPA’s New Assessment Process Will Further Limit the Productivity and Credibility of Its Integrated Risk Information System. GAO-08-810T. Washington, D.C.: May 21, 2008.

Toxic Chemicals: EPA’s New Assessment Process Will Increase Challenges EPA Faces in Evaluating and Regulating Chemicals. GAO-08-743T. Washington, D.C.: April 29, 2008.

Federal Advisory Committee Act: Issues Related to the Independence and Balance of Advisory Committees. GAO-08-611T. Washington, D.C.: April 2, 2008.

Chemical Assessments: Low Productivity and New Interagency Review Process Limit the Usefulness and Credibility of EPA’s Integrated Risk Information System. GAO-08-440. Washington, D.C.: March 7, 2008.

Federal Advisory Committees: Additional Guidance Could Help Agencies Better Ensure Independence and Balance. GAO-04-328. (Washington, D.C.: April 16, 2004.

EPA’s Science Advisory Board Panels: Improved Policies and Procedures Needed to Ensure Independence and Balance. GAO-01-536. Washington, D.C.: June 12, 2001.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.