EPA's Science Advisory Board Panels: Improved Policies and Procedures Needed to Ensure Independence and Balance
Published: 2001-06-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This report reviews whether the policies and procedures of the Environmental Protection Agency's Science Advisory Board ensure that (1) its peer review panelists are independent and the panels are properly balanced and (2) the public is sufficiently informed about the points of view represented on the panels.
section_title: What GAO Found
paragraphs: GAO found that the policies and procedures used by the staff office to ensure the independence of the Board's peer reviewers and the balancing of viewpoints have limitations that reduce their effectiveness. The staff office has not systematically requested information that is needed to assess the independence and overall balance of viewpoints represented on the panel--such as previous public positions the panelists have taken on the matter being reviewed--until the first meeting, when the panelists have already been chosen. Furthermore, conflicts of interest may not be identified and mitigated in a timely manner. GAO also found that the staff office's policies and procedures for providing the public with information on the backgrounds of the Board's peer review panelists do not adequately inform the public about the points of view represented on the panels.

section_title: Letter
paragraphs: Since its inception in 1978, the Environmental Protection Agency’s (EPA) Science Advisory Board has reviewed key scientific studies and methodologies used by the agency in formulating rules to protect the environment and public health. The Board comprises more than 100 nongovernment experts and provides technical advice directly to the EPA Administrator primarily on the basis of its peer reviews—that is, critical evaluations by panels of independent experts. Through such peer reviews, EPA and the Board seek to enhance the quality and credibility of the agency’s highly specialized products. To be effective, peer review panels must be—and also be perceived to be—free of any significant conflict of interest and uncompromised by bias. Peer review panels should also be properly balanced, allowing for a spectrum of views and appropriate expertise.
You expressed concern about potential conflicts of interest on panels convened by the Board to assess health risk assessment issues. As agreed with your office, we determined whether the policies and procedures of the Board are adequate to ensure that (1) its peer review panelists are independent and the panels are properly balanced and (2) the public is sufficiently informed about the points of view represented on the panels. These policies and procedures are the responsibility of EPA’s Science Advisory Board Staff Office, which reports to the Administrator and provides logistical and administrative support to the Science Advisory Board under the leadership of a staff director.
Federal requirements relevant to independence and balance include (1) the criminal financial conflict-of-interest statute (18 U.S.C. 208) and related U.S. Office of Government Ethics (OGE) regulations that prohibit federal employees from participating in a “particular matter” in which they have a financial interest and (2) the Federal Advisory Committee Act, which requires, among other things, balancing viewpoints represented on government committees and providing information about committee activities to the public. The financial conflict-of-interest statute is applicable to the Board’s panelists because they are hired as special government employees.
To determine how the Board addresses these requirements, we reviewed the written policies and procedures developed by the staff office and also examined their application to four peer review panels you asked us to examine—three on EPA’s draft revised guidelines for assessing cancer health risks and one on a health risk assessment of the chemical 1,3-Butadiene. According to the director of the staff office, these panels are generally representative of Board peer review panels both in subject matter—they address a significant risk assessment methodology and a health risk assessment—and in the processes used in establishing and conducting them. We reviewed the confidential financial disclosure forms of the panelists, along with other information, and discussed with staff office officials the office’s policies and procedures generally as well as how they were applied to the four panels. We did not, however, make independent judgments on whether conflicts of interest existed for members of the four panels or assess whether these panels were properly balanced.
subsections: section_title: Background
paragraphs: The Science Advisory Board, which comprises more than 100 nongovernment technical experts, was established in 1978 by the Congress to provide independent scientific and engineering advice to the EPA Administrator on the technical bases for EPA regulations. The Board’s work is coordinated and administered by 19 EPA employees assigned to the Science Advisory Board Staff Office, which is under the leadership of a staff director. The Board and its staff office both report directly to the Administrator. The Board often convenes peer review panels to assess the scientific and technical rationales underlying current or proposed EPA regulations and policies. The Board convenes these panels in public meetings, and the panels’ recommendations are provided to the Administrator in published reports. In fiscal year 2000, the Board conducted 54 public meetings and issued 37 reports on a wide range of scientific issues.
Board members, who are appointed by the Administrator, have expertise in various technical disciplines and retain their primary employment in academia, industry, state government, and environmental organizations while serving on the Board. The appointments are for 2 years, generally renewable for not more than two additional terms. The Board members serve on 1 or 2 of the Board’s 10 permanent committees, such as the Drinking Water Committee and the Health Effects Committee. As members of these committees, Board members may be assigned to a number of peer review panels depending upon, among other things, their availability. The Board also has some 300 consultants, including scientists, social scientists, engineers, and economists, appointed by the staff director to provide the additional expertise needed for specific peer review panels. In this report, we refer to the members and consultants who serve on the peer review panels as panelists.
In general, a topic for Board peer review is assigned to one of the permanent committees by the Board’s executive committee. The chair of the permanent committee and a staff office employee have primary responsibility for forming a review panel for the topic. The panel may include some or all members of the committee and often some consultants. Often the staff member solicits suggestions for panel members from other EPA staff and sources outside EPA, such as industry groups, environmental organizations, and other experts. The staff director makes the final decision on appointment of panelists in consultation with the staff member and the committee chair.
The policies and procedures governing conflicts of interest, balanced viewpoints on panels, and public disclosures are included in a staff office document, “Guidelines for Service on the Science Advisory Board.” This six-page document covers the selection process and criteria for members and consultants, time limits on service, the panel selection process, and guidance on conflicts of interest and the public disclosure session at Board meetings. In addition, a November 1999 handbook prepared by the staff office for panelists includes information on how the staff evaluate conflicts of interest and provides copies of EPA ethics advisories that address conflicts of interest and other ethics requirements for federal employees.
As a federal advisory committee, the Board must comply with the Federal Advisory Committee Act and related regulations. The act was passed, in part, out of concern that certain special interests had too much influence over federal agency decisionmakers. Among other things, this law requires that committees, such as the Board’s peer review panels, “be fairly balanced in terms of the points of view represented and the functions to be performed.” The advice of such committees should reflect their independent judgment, without improper influences from special interests. The balance requirement thus recognizes that it may be necessary or desirable to include experts on the panel who represent a particular viewpoint in order to conduct an effective and thorough review of an issue, but that as a whole the panel is to be balanced. As required by the act, the Board has an approved charter, announces its meetings in the Federal Register, and provides opportunities for public comment on issues before the Board.
The criminal financial conflict-of-interest statute is applicable to Board panelists because they are hired as “special government employees” in light of their intermittent service to the Board. With some important modifications, special government employees are subject to the ethics requirements applicable to federal employees, including the conflict-of- interest requirements. The financial conflict-of-interest statute prohibits federal employees from acting personally and substantially in any “particular matter” that has a direct and predictable effect on their financial interests; those of their spouse or children; or those of organizations with which they have certain associations, including employers (see app. I). However, under an exemption in the regulations, special government employees who serve on a federal advisory committee (such as the Board) are allowed to participate in particular matters that have a direct and predictable effect on their employer’s financial interests if the employer is affected as part of a discrete and identifiable class—that is, is not singularly affected. This exemption is limited, however, and does not include the financial and other interests of the panelists in the particular matter, such as stock ownership in the employer. If a financial interest is not covered under this or other exemptions, waivers of the conflict-of-interest provisions are allowed upon a determination by the appointing official that the need for the special government employee’s services outweighs the potential for a conflict of interest.
The staff office recognizes that the Board’s peer review activities can be harmed not only by financial conflicts of interest but also by the appearance of bias. That is, some panelists may be unable or potentially unable to render impartial assistance or advice because of activities, interests, or relationships with other people, such as a spouse.  For example, a peer review panelist may be so closely aligned with a point of view or an organization that his or her ability to provide objective and independent advice is impaired or appears to be impaired. The staff office’s written policies and procedures use varying terms when discussing circumstances in which a person’s impartiality may be called into question, such as “apparent conflict of interest” and “perceived conflict of interest.”
section_title: Procedures Do Not Adequately Ensure Independence and Balance
paragraphs: The staff office’s policies and procedures are intended to ensure that the Board’s panelists are independent and that panels as a whole are balanced in their viewpoints and expertise. However, because of shortcomings in the policies and procedures, the Board does not have adequate assurance that this is actually the case. Specifically, the staff office’s policies and procedures do not provide for   determining at the time a panel is being developed and convened what financial conflict-of-interest requirements, if any, the panel is subject to; routinely ensuring that potential panelists’ financial disclosure forms contain sufficient information to evaluate potential conflicts of interest;   systematically requesting other information pertinent to assessing independence and overall balance of the panel, such as previous public positions that panelists have taken on the matter being reviewed, before the panel members are appointed; and   adequately training prospective panel members on the financial conflict- of-interest requirements and other issues that could raise questions about their impartiality.
Further, while one of the goals of the staff office is to ensure that Board panels are properly balanced, it has no systematic way of doing so. These limitations are discussed in the following sections.
subsections: section_title: Relevant Conflict-of- Interest Requirements Have Not Been Identified
paragraphs: Individual Board peer review panels are subject to different requirements under the financial conflict-of-interest provisions depending upon the subject matter being reviewed. If the panel is reviewing a matter that is “broadly applicable,” the panel is not subject to the statutory financial conflict-of-interest requirements. For the requirements to apply, the panel must be reviewing a “particular matter”—one involving a deliberation, decision, or action that is focused on the interest of specific people or an identifiable class of people. There are two types of particular matters: (1) particular matters of general applicability, such as those involving general regulations, policies, or standards that distinctly affect a particular industry or other class of people or entities and (2) particular matters involving specific parties, such as those involving contracts and grants that affect specific individuals or entities. The specific requirements for panelists vary depending on which type of particular matter the panel is to review. However, in many cases the line between a broadly applicable matter and a particular matter of general applicability is not clear. As shown in appendix I, the conflict-of-interest provisions are complex and subject to interpretation.
In the past, the staff office concluded that almost all of the Board’s reviews were broadly applicable and did not involve particular matters. The staff office believed, therefore, that the panelists serving on the review panels generally were not subject to the financial conflict-of-interest requirements. This view is reflected in the staff office’s “Guidelines for Service on the Science Advisory Board,” which states that the Board generally does not get involved in “particular party matters,” and thus legal conflicts of interest are rare.
While this view appears to be reasonable for panels that review matters of such broad applicability as risk assessment methodologies, the Board may review more particular matters and be subject to the conflict-of-interest requirements in more cases than it has considered. As a result of our review, the staff director has reexamined the conflict-of-interest provisions and now believes that more of the Board’s reviews—including the review of EPA’s risk assessment of 1,3-Butadiene—are likely subject to the conflict-of-interest requirements than previously thought.
Because the staff office did not believe the 1,3-Butadiene risk assessment review was a particular matter at the time it developed and convened the panel, it did not identify potential conflicts of interest that should have been examined. For example, the staff office did not identify the manufacturers and distributors of this chemical or carefully review the information provided by the panelists to ensure that potential conflicts of interest were identified and mitigated, as appropriate. Although two panelists owned stock in companies that manufacture 1,3-Butadiene, distribute it, or both, the staff office did not obtain information from these panelists about their investments to determine whether they would need a waiver in order to participate on the panel. (See app. II for more information on the 1,3-Butadiene panel.)
In our view, the uncertainty about which legal requirements a panel may be subject to suggests it would be better to err on the side of caution to protect the Board and its members from embarrassment or potential legal problems. Further, even when the staff office determines that a panel is to review a matter that is not subject to the financial conflict-of-interest requirements, determining whether any of the panelists have interests or relationships that could raise questions about their impartiality would help the staff office assess the interests and also help ensure that the panel is balanced as a whole. To do this, the staff office would have to start identifying the industries and companies that have an interest in the matters being examined.
section_title: Use of Financial Disclosure Forms Is Problematic
paragraphs: To identify potential financial conflicts of interest, the staff office relies primarily on a confidential financial disclosure form that panelists must complete before serving on a panel. Prepared by OGE for executive branch employees, the confidential disclosure form, OGE form 450, requires information on assets and income, liabilities, and outside positions during the prior year and on existing employment agreements or arrangements, such as promises of future employment and leaves of absence. The form requests information on the sources of income and the identification of assets but does not ask for the related dollar amounts. We found that the staff office’s policies and procedures do not provide for (1) routinely ensuring that forms are complete or following up on information disclosed and (2) reviewing the forms promptly to make them most useful for particular panels.
subsections: section_title: Follow-up on Financial Information Is Inadequate
paragraphs: For the four panels we reviewed, some panelists did not identify all income sources or provide complete information on the confidential financial disclosure form—identifying a source of income, for example, only as fees from “various clients” or excluding a primary employer entirely. According to a deputy ethics officer in EPA’s Office of General Counsel who reviews these forms for EPA employees, respondents often make errors in completing the confidential financial disclosure form. For example, he said that he often must speak with individuals who have completed these forms because they are not completed properly. Similarly, OGE guidance on reviewing the disclosure forms directs appropriate follow-up with individuals who prepare the forms and documentation of the responses on the forms. However, the staff office has not taken steps to routinely ensure that all of the required disclosure information is provided.
In reviewing the disclosure forms provided by members of the four panels included in our study, we found that the information—while correctly provided—could raise questions or issues that may be relevant in determining potential conflicts of interest or in assessing impartiality and balance among panelists. For example, some panelists reported fees from what appear to be law firms as well as from companies, including chemical companies. In some of these cases, the reported legal and consulting work was extensive. While the staff director told us, for example, that it is relevant for the staff office to know whether a panelist has provided expert legal testimony—and on behalf of whom—the staff office has neither taken steps to clarify information reported on disclosure forms that suggests such activity nor asked for information on the type of work performed for an affected industry. We believe the staff office would be better able to assess panelists’ impartiality and ensure that panels are properly balanced if it had an understanding of the work performed by the panelists for law firms and industry, particularly for chemical companies. Staff office officials, including the staff director, told us that from time to time they have asked some panelists for additional information, but they acknowledged that this has occurred infrequently and that they have not required this information to be noted in the panelists’ records.
A case in point involves a panelist participating in a review of EPA’s draft revised guidelines for assessing the health risks of carcinogens who had identified a substantial number of consulting fees from companies and law firms, including a tobacco company and a research organization funded by the tobacco industry. (See app. III for more information on the three panels on guidelines for assessing the health risks of carcinogens.) While staff office officials said they were aware of the panelist’s relationships with the tobacco industry, they acknowledged that they did not obtain information about the numerous consulting fees reported. In balancing the panel, the staff office judged this individual to be in the “broad middle” —as opposed to reflecting either an environmental or industry perspective—on the basis of both the panelist’s academic credentials and the staff’s observations at previous panel meetings. While the staff office’s assessment of this panelist may be accurate, it would be more credible if it were based on more complete information about the panelist’s extensive industry and legal consulting work. Furthermore, when this panelist first joined the Board, the staff office’s evaluation had to be performed without the benefit of personal observations.
After reviewing the disclosure forms, the staff office does not notify the panelists whether specific items they reported on their financial disclosure forms could potentially present a conflict of interest in some cases. In contrast, EPA’s Office of General Counsel sends what it terms a “cautionary letter” to employees identifying any item or items reported on the confidential financial disclosure form that could present a conflict of interest in the event the employee was called upon to participate in a “particular matter.” The deputy ethics official in EPA’s Office of General Counsel told us that one benefit of the cautionary letter is that it reminds employees of their responsibility to consider and identify potential conflicts of interest that may arise in the course of their work.
section_title: Reviews Can Be Untimely
paragraphs: Compounding the reliance on the financial disclosure form to identify potential conflicts of interest, the staff office does not necessarily review the financial disclosure forms in the context of forming a specific peer review panel. This situation occurs primarily with Board members, who are required to complete the disclosure forms at the start of each fiscal year. When Board members send in their forms, the specific panels the individual members will serve on throughout the year are not yet determined. In contrast, consultants assigned to a specific peer review panel generally are required to complete the disclosure forms when they are selected for that panel. While a staff office official said that the disclosure forms of Board members may be reexamined when the member is assigned to a specific panel, this step is not taken systematically, and the staff office’s policies and procedures do not require that the forms be reviewed within the context of each peer review panel.
In addition, some Board members do not return their forms at the beginning of the fiscal year as requested but instead submit them shortly before or at the first meeting of a panel. Moreover, the staff office may review some consultants’ forms too late to affect the composition of a panel. For example, on the basis of the 47 financial disclosure forms we reviewed for the four panels, we found seven instances in which panelists submitted their forms either within 2 days of the first panel meeting or after the meeting; eleven cases in which the staff office reviewed the forms after the first panel meeting; and three cases in which the staff office never reviewed the forms. This last case involved panelists who had also served on another EPA advisory committee. In such cases, the staff office relies on the previous advisory committee’s conflict-of-interest review rather than reviewing the individual’s disclosure forms. Moreover, we found that one of the three financial disclosure forms was not reviewed by either the other EPA advisory committee or the staff office. As a result of these various review shortcomings, the staff office could allow individuals who have conflicts of interest to serve as panelists, without taking appropriate steps to mitigate the conflicts or granting a waiver.
section_title: Additional Relevant Information Is Not Systematically Obtained Before Appointing Panelists
paragraphs: Beyond the financial disclosure form, the staff office does not systematically request certain information that is pertinent to assessing both the impartiality of the panelists and the overall balance of viewpoints represented on the panels until a selected panel is conducting its first meeting.
The primary source of background information that the staff office has used—the confidential financial disclosure form—is limited in focus. While the information it elicits covering the prior year is pertinent to assessing financial conflicts of interest, it does not identify other information relevant to assessing impartiality and balancing peer review panels. That is, the confidential financial disclosure form does not request information on the panelists’ backgrounds in the subject matter the panel will review, such as research conducted and previous public statements or positions on the matter being reviewed, or on the interests of the panelists’ employers or clients in the matter.
This information is not systematically requested until the panelists have been selected and are participating in the first meeting of the peer review panel—the public disclosure session. At that time, the panelists are asked, but not required, to disclose to the public and the other panelists additional information regarding their backgrounds, including research conducted and previous public statements or positions on the matter being reviewed, participation in legal proceedings on the matter, interests of their employers or clients in the matter, and prior or current research grants that could be affected by the matter. Panelists are also asked to provide a general description of any other financial interest in the matter, such as having investments that might be directly affected by the matter. However, the staff’s guidance on public disclosure states that panelists are not obligated to reveal information contained in their confidential disclosure form that would otherwise remain confidential.
The staff director told us that generally the staff is already aware of the information that the panelists report at the first meeting of the panel as a result of ongoing communications between the staff and the panelists, and in some cases, information from the organization that recommended the panelists to the staff office.  Moreover, according to the handbook the staff office developed for Board panelists, the confidential financial disclosure forms; annual conflict-of-interest training; “regular” briefings on conflict- of-interest issues; and “regular, informed” contact with the panelists “normally identify actual, as well as perceived, conflict-of-interest issues” before the panel is convened in a public meeting.  Nevertheless, the staff office director acknowledged that the staff office policies and procedures do not include a requirement that the staff discuss with the panelists, in advance, the specific information the panelists are asked to provide at the first meeting of the panel. Further, the handbook for panelists does not include such discussions among the duties the staff perform to support the peer review panels. While the staff director said the office’s goal was to avoid any surprises when disclosures are made at the first meeting, he acknowledged that the staff do not always know what the panelists will say.
The public disclosure session can elicit informative, detailed statements about panelists’ backgrounds not previously brought out in financial disclosure forms, resumes, or information from the organizations that nominated the panelists. Examples of disclosures made at Board meetings by panelists of the type of information that is not requested on the confidential disclosure form follow:   A panelist on the 1,3-Butadiene peer review panel reported having worked on a legal case involving 1,3-Butadiene for a manufacturer of this chemical and also having a leadership role on a major industry-funded study on 1,3-Butadiene.
Several other 1,3-Butadiene panelists identified completed or ongoing research studies on the chemical that they had conducted, some of which were sponsored by organizations that receive funding from chemical companies such as the Chemical Manufacturers Association.
A panelist on the revised cancer risk guidelines panel reported a prior long-term affiliation with a chemical industry organization that had commented to EPA on its revised guidelines.
In our view, while the above disclosures may not represent activities or affiliations that would necessarily preclude any of these individuals from participating on the peer review panel, they do represent information that should be evaluated by the staff office before finalizing its selection of panelists. Because the Board’s policies and procedures do not require the staff office to obtain and document such background information earlier, it is not possible to determine the extent to which the staff were aware of or considered the above information before the first panel meeting. The staff files for the panels we reviewed did not contain any information about these relationships and activities. In contrast, the policies and procedures of the National Academies provide that peer review panel selections will not be finalized until officials have reviewed and evaluated written information on potential sources of bias and conflicts of interest. The Academies send potential panelists a form requesting relevant information about organizational affiliations, financial interests, research support, government service, and public statements and positions.
According to EPA’s “Science Policy Council Handbook on Peer Review,” the issue of conflicts of interest should be addressed with prospective panelists before finalizing the panel membership. However, the staff office’s policies and procedures do not require that potential conflicts of interest be discussed directly with each panelist. As a result, peer review panelists generally are not asked directly whether there is anything in their backgrounds that could present a conflict of interest. This omission is especially significant in light of the limitations of the disclosure forms the staff office relies on to identify conflicts of interest and assess impartiality. It is also important in that panelists may choose the extent to which they participate in the voluntary disclosure process conducted at the first panel meeting. “If there are other circumstances in your background or present connections that in your opinion might reasonably be construed as unduly affecting your judgment in matters within the assigned task of the group to which you have been appointed or which might be reasonably viewed as creating an actual or potential bias or conflict of interest or the appearance of a bias or conflict of interest, please describe them briefly.”
section_title: Conflict-of-Interest Training Is Inadequate
paragraphs: The staff office’s annual training program for Board panelists that covers conflicts of interest consists primarily of a 1-hour study of ethics materials that the panelists complete independently. However, the lengthy materials sent to the panelists do not explain clearly how the effectiveness of the Board can be diminished if they are found to have conflicts of interest or other issues that raise questions about their impartiality that have not been identified and appropriately mitigated. Neither do the training materials clearly identify the financial conflict-of-interest requirements that the panelists are subject to, depending upon the review panel. As a result, the panelists may not provide the staff office with all of the information it needs to correctly identify potential conflicts of interest, assess impartiality, and properly balance the panel.
The training materials cover a broad range of complex ethics issues that apply to regular government employees as well as to special government employees. However, most of the information is targeted to regular government employees. In a number of cases, the requirements are different for special government employees. Panelists are left to interpret complex legal information on their own and take the initiative to call the staff office with any questions. We note that the fiscal year 2001 training materials did not include a February 2000 document prepared by OGE entitled, “Summary of Ethical Requirements Applicable to Special Government Employees,” which could be helpful to the panelists. While the staff office would still need to summarize the key points of this legal document, it is superior to the information currently provided to the panelists because it focuses exclusively on special government employees.
In contrast to the materials the staff office provides to its panelists, the National Academies provide their panelists with a succinct, straightforward paper on potential conflicts of interest and bias. Similarly, Board panelists would be better informed and potentially have an increased incentive to provide timely information to the staff office if they received clear, targeted information on the conflict-of-interest requirements to which they are subject. In addition, the independent study of ethics that the staff office requires of its panelists would be enhanced by periodic briefings on conflict-of-interest issues. While the policies and procedures of the staff office state that it is to deliver briefings on conflict- of-interest issues to the panelists on a regular basis, we found that it had conducted such briefings for only 2 of its 10 committees between October 1998 and March 2001.
section_title: No Systematic Process Exists for Balancing Panels
paragraphs: Although the staff office aims to ensure that the Board’s panels are properly balanced, it has no systematic way of doing so. The staff office’s goal is to generally select panelists representing the “broad middle” spectrum of opinion on the technical issue under discussion. When developing a panel, the staff director and a designated staff member review a list of potential panelists to select a panel with the proper balance of viewpoints and expertise. The selection process is not structured in some cases. However, in some cases, the staff office does use a structured process, ranking the candidates along a numeric scale of 1 to 10, according to whether their perspectives generally reflect the broad middle spectrum of opinion on the technical issue or are more aligned with industry interests or environmental interests. The staff director acknowledges that this judgment is a subjective one, but it may be more subjective than it needs to be. That is, this decision may be made before the staff office reviews the prospective panelist’s financial disclosure forms or discusses with him or her the background information that would provide an indication of the panelist’s perspective.
The staff director told us that, in retrospect, there are a few cases among the four panels we reviewed in which he might have judged the viewpoints of the panelists differently if the staff office had known certain information about them earlier. However, the staff office does not have established procedures for ensuring that (1) relevant information about the panelists’ backgrounds is known before panels are finalized and (2) evaluation of the overall balance of panelists takes place once the staff office has determined who is available to serve on a specific panel. In contrast to the Board, the policies and procedures of the National Academies not only require obtaining relevant background information on panelists before making panel selections, they also require posting information on its Web page about its proposed panels and soliciting comments on potential appointments to committees or panels. According to an official of the National Academies, this process has proved beneficial to them in selecting balanced peer review panels. Similarly, the staff office could request input from the public about potential panelists and overall balance once it has developed a proposed panel.
section_title: Limited Information Is Available to the Public on the Board’s Panelists
paragraphs: The staff office’s policies and procedures for providing the public with information on the backgrounds of its peer review panelists are not sufficient to ensure that the public is adequately informed about the points of view represented on the panels. Specifically, the staff office (1) relies on panelists’ voluntary disclosures of some information that could help inform the public and (2) does not consistently record and make available to the public at large the information that panelists voluntarily disclose.
To provide the public with information about its peer review panelists, the staff office distributes rosters identifying the names of the panelists; their primary employment; and, in some cases, panelists’ titles and requests that panelists discuss their background associated with the issue at hand at the first public meeting. This disclosure process—which is voluntary—is intended to allow the public and other panelists to consider the perspectives or potential bias, if any, that the panelists bring to the issue. This approach is consistent with the Federal Advisory Committee Act, which recognizes the benefits of openness in safeguarding the public interest. The staff office requests that the panelists discuss relevant research they have conducted; previous public statements or positions they have taken; the interests of their employers; their own financial interests in the matter; and research grants from parties—including EPA—that would be affected by the matter. However, this voluntary process does not consistently provide the public with sufficient information to understand the points of view represented on the panel.
Specifically, while the Board does provide minutes of its meetings as required by the Federal Advisory Committee Act, in many cases, disclosures made by the panelists at the meeting are not recorded in the minutes. For example, the minutes of one of the panels on EPA’s revised cancer risk guidelines panel reflect that the recorder left the room during the public disclosures; thus, the comments, if any, of five of the panelists are not available. In another case that is more typical of meeting minutes prepared by the staff office, the minutes state only that no conflicts of interest were identified during the disclosures and do not provide any information on the disclosures that were made. As a result, members of the public not present at the meeting have limited opportunities to obtain information the panelists provided during the public disclosure session.
In addition, our review of the minutes of the meetings of the four peer review panels we examined indicated that panelists may not provide information during the voluntary public discussion. For example, the meeting minutes for the 1,3-Butadiene panel include public disclosure comments for only 10 of the 15 panelists reported as being in attendance. Also, some panelists gave disclosures that were too vague for the public to get an accurate picture of the panelists’ points of view related to the issue under review. For example, a panelist may report publishing articles relating to the subject of the panel but not elaborate on any viewpoints or positions taken in the articles, report conducting research on the matter but not identify the sources of funding for the research, or disclose public pronouncements without clarifying their relationship to the matter at hand. According to an EPA Office of General Counsel deputy ethics official, panelists make general or vague statements because they are often unsure of what to say. As a result of panelists’ uncertainty, their comments may either leave the public wondering about the panelists’ impartiality or be interpreted incorrectly as indicating bias.
This same Office of General Counsel deputy ethics official has expressed concerns about the public disclosure process. The official believes that the disclosures may confuse the public more than enlighten it and has discussed these concerns with staff office officials. The staff director acknowledged that the current approach has flaws and said that changes are being considered. Specifically, the staff office has considered having a staff official introduce the panelists at the first meeting and provide background about each individual to the public. As of March 2001, the Board had not decided what changes, if any, it will make to this process.
section_title: Conclusions
paragraphs: EPA’s Science Advisory Board panels are an important element in ensuring that sound scientific and technical information is available to the agency’s decisionmakers. The regulatory process benefits from the scientific and technical knowledge, expertise, and competencies of panel members. However, the work of fully competent peer review panels can be undermined by allegations of conflict of interest and bias. Therefore, the best interests of the Board are served by effective policies and procedures regarding potential conflicts of interest, impartiality, and panel balance.
The policies and procedures used by the staff office to date have limitations. Specifically, because the conflict-of-interest requirements to which the panels may be subject are not identified at the outset, potential conflicts of interest are not identified and mitigated in a timely manner in some cases. In addition, because relevant background information is not consistently available at the time of the selection process, it is difficult for the staff office to have assurance that it has an accurate sense of the viewpoints of the candidates and that the panel is balanced in terms of viewpoints and expertise. Similarly, Board panelists would be better informed and potentially have an increased incentive to provide timely information to the staff office if they received clear, targeted information on the conflict-of-interest requirements to which they are subject. Moreover, the staff office’s varying approaches to balancing panelists in terms of expertise and viewpoints hamper its ability to consistently ensure the appropriate balance is obtained on its panels. Guidelines that specify the type of information needed about a prospective panelist and about the entire panel before final decisions are made would help the staff office ensure appropriate panel balance is achieved. The staff office could also expand its outreach to help it ensure that its panels are balanced.
Implementation shortcomings, such as not recording the background information panelists discuss during the voluntary disclosure process, hinder the effectiveness of the staff office’s policies and procedures aimed at providing the public with information about the viewpoints and expertise represented on the Board’s peer review panels. As a result, the public is not provided with consistent, relevant background information about the panelists that would enable interested parties to evaluate the viewpoints and expertise of the panelists.
section_title: Recommendations for Executive Action
paragraphs: To better ensure that peer review panels are independent and reflect an appropriate balance of viewpoints and expertise, we recommend that the EPA Administrator direct the Science Advisory Board to develop policies and procedures that better identify and mitigate potential conflicts of interest and support the development of balanced panels. Such changes should include the following:   Determine whether each panel will be reviewing a “particular matter” before selecting the panel in order to identify the financial conflict-of- interest requirements, if any, to which the panelists will be subject.
Obtain and evaluate relevant background information on peer review panel candidates before appointing panel members. The evaluation should include explicitly discussing with potential panelists (1) items not adequately reported on the confidential financial disclosure form as well as items reported that could present conflicts of interest; (2) other information relevant to assessing impartiality, such as research conducted and previous public statements or positions on the matter being reviewed, interest of the employer or clients in the matter, participation in legal proceedings, work for chemical companies or other affected industries, and prior or current research grants that could be affected by the matter; and (3) whether they have any potential conflicts of interest related to the specific panel being established. Further, pertinent information obtained from discussions with panelists should be documented.
Develop training that clearly identifies for the panelists the conflict-of- interest requirements that apply to them, addresses impartiality, and identifies the background information the staff office needs from the panelists to assess (1) the appropriateness of their participation on specific panels and (2) the balance of viewpoints and expertise on the panels themselves.
Develop criteria for and guidance on the process to be used to achieve the proper balance of viewpoints and expertise on peer review panels.
We further recommend that the EPA Administrator direct the Science Advisory Board to (1) provide consistent, relevant information to the public about panelists to enable the public to sufficiently understand the points of view represented on a panel and (2) ensure that this information is properly recorded in the meeting minutes. In addition, we recommend that the Board consider allowing the public the opportunity to comment on proposed panels before final selection decisions are made.
section_title: Agency Comments and Our Response
paragraphs: We provided a draft copy of this report to EPA for its review and comment.  In commenting on the draft, the staff director of the Science Advisory Board Staff Office generally agreed with the report’s findings and recommendations.  He said the staff office is planning actions in response to the report that will improve its operations.  The staff director also provided us with an outline of the policies and procedures that the office is planning to implement to address our recommendations, as well as other actions the staff office believes are warranted in light of the information provided in our report, such as reviewing and improving its record-keeping systems.
section_title: Scope and Methodology
paragraphs: To determine whether the Board’s policies and procedures are adequate to ensure panel independence and balance and to provide sufficient information to the public, we analyzed the criminal financial conflict-of- interest statute (18 U.S.C. 208) and OGE regulations regarding conflicts of interest. The financial conflict-of-interest provisions are particularly relevant to the Board’s peer review panelists because, as is typical of special government employees who retain their nonfederal primary employment, the panelists often have substantial outside activities and financial interests. We also analyzed Federal Advisory Committee Act requirements, including those on balancing the viewpoints represented on government committees and providing information about committee activities to the public. We reviewed the staff office’s written policies and procedures and their application to three peer review panels that reviewed EPA’s draft revised guidelines for assessing cancer health risks and one that reviewed EPA’s health risk assessment of the chemical 1,3-Butadiene.
Two of the peer reviews were conducted by the Environmental Health Committee, which has focused on risk assessment matters, and two other subcommittees of the Board conducted the other two reviews. According to the staff director, these panels were generally representative of Board peer review panels both in subject matter—they addressed a significant risk assessment methodology and a health risk assessment—and in the processes used in establishing and conducting them. We reviewed the confidential financial disclosure forms of the panelists, along with other information, and discussed with staff office officials the office’s policies and procedures generally as well as how they were applied to the four panels. We did not, however, make independent judgments on whether conflicts of interest existed for members of the four panels or assess whether these panels were properly balanced. We also interviewed cognizant officials from the Science Advisory Board’s Staff Office; EPA’s Office of General Counsel; OGE; and other scientific organizations, such as the National Academies, to identify policies and procedures that might better support the Board’s efforts to develop balanced, independent peer review panels. We conducted our work from August 2000 through June 2001 in accordance with generally accepted government auditing standards.
As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter.  At that time we will send copies of this report to the EPA Administrator, appropriate congressional committees, and other interested parties. We will also make copies available on request.
Please call me or Christine Fishkin at (202) 512-3841 if you or your staff have any questions about this report. Other key contributors to this report include Greg Carroll, Bruce Skud, and Amy Webbink.
section_title: Appendix I: Financial Conflict-of-Interest Provisions
paragraphs: The financial conflict-of-interest statute (18 U.S.C. 208) prohibits federal employees, including special government employees (SGE), from participating personally and substantially in a particular matter in which they have a financial interest. A particular matter is one that involves deliberation, decision, or action that is focused on the interests of specific people or a discrete and identifiable class of people. There are two types of particular matters: (1) particular matters of general applicability, such as general regulations, policies, or standards that distinctly affect a particular industry or other class of people or entities and (2) particular matters involving specific parties, such as contracts and grants that affect specific individuals or entities.
If a federal employee’s spouse, minor child, or general partner has a financial interest in the particular matter, that interest is treated as the employee’s interest and therefore disqualifies the employee from participating. In general, the federal employee also may not participate in a particular matter if he or she serves as an officer, director, trustee, general partner, employee, or prospective employee of an organization that has a financial interest in that matter.
U.S. Office of Government Ethics (OGE) regulations provide a number of exemptions for financial interests determined to be too remote or inconsequential to affect the integrity of the federal employee’s services. For example, an employee may participate in a “particular matter of general applicability” if the financial interest arises from ownership of publicly traded stock issued by one or more entities affected by the matter, but the market value of the stock must not exceed $25,000 for any one entity or $50,000 for all affected entities. In addition, some employee benefit plans, diversified mutual funds, and certain government securities are exempt.
The OGE regulations provide a financial interest exemption to SGEs serving on an advisory committee that other federal employees do not receive. An SGE, such as a member of the Science Advisory Board, may participate in a “particular matter of general applicability” on an advisory committee when the disqualifying financial interest arises from the SGE’s principal employment. The employer’s, or prospective employer’s, financial interest is not treated as the SGE’s financial interest. This exemption, however, does not cover the SGE’s ownership of stock in the employer or consulting relationships.
Even if SGEs have a financial interest that would be affected by the activities of the advisory committee, they may receive a waiver of the conflict if the official responsible for appointing them certifies, after reviewing the financial disclosure report, that the need for the SGEs’ services outweighs the potential for a conflict of interest.
Figure 1 identifies questions that should be addressed in determining whether an SGE has a financial conflict of interest.
section_title: Appendix II: The 1,3-Butadiene Health Risk Assessment Panel
paragraphs: The Environmental Protection Agency’s (EPA) Office of Research and Development asked the Science Advisory Board to review its draft health risk assessment of 1,3-Butadiene, a synthetic chemical compound used principally in the manufacture of synthetic rubber, nylon, and latex paint. According to Chemcyclopedia, The Buyer’s Guide of Commercially Available Chemicals, as of January 2001, 11 companies manufacture 1,3-Butadiene, distribute it, or both. 1,3-Butadiene is used extensively in the production of synthetic rubber for vehicle tires, plastics for the automobile industry, and nylon fibers and plastics. Rubber and tire workers can be exposed to 1,3-Butadiene through accidental releases, and 1,3-Butadiene derived from motor vehicle combustion can also be found in outdoor (ambient) air. This appendix includes an overview of the peer review panel on 1,3-Butadiene, followed by information about the panelists.
subsections: section_title: Background
paragraphs: EPA’s Office of Research and Development asked the Board to review the health risk assessment on 1,3-Butadiene for technical quality, comprehensiveness, and clarity. The Board was asked to address specific questions, including the following: (1) Does the science support the classification of 1,3-Butadiene as a “known” human carcinogen? (2) Are the approaches taken to characterize plausible cancer risk reasonable given the science? (3) Are the conclusions and quantitative estimations for reproductive/developmental effects adequately supported?
A 15-member peer review panel met April 30 through May 1, 1998, to conduct the review. A key consideration was whether the panelists would recommend classifying 1,3-Butadiene as a known human carcinogen or as a likely carcinogen. The classification will affect future regulatory actions regarding this chemical and represents one of the areas of controversy in the proposed revised cancer health risk assessment guidelines also reviewed by the Board (see app. III). The Board issued its final report on the draft health risk assessment of 1,3-Butadiene on November 19, 1998.The Board’s 1,3-Butadiene panel did not support classifying 1,3-Butadiene as a known human carcinogen as had been recommended in EPA’s draft health risk assessment. Although the panel did not reach consensus position on this issue, a majority of the panelists recommended that 1,3-Butadiene be classified as a “probable” human carcinogen.
This appendix contains information on the panel that reviewed EPA’s draft health risk assessment of 1,3 Butadiene. For this panel, we identify the panelists, their primary employment, their job titles (when available), and other pertinent information the panelists provided about themselves (1) on the confidential financial disclosure forms and (2) at the voluntary public disclosure session. We also present the staff director’s assessments of the viewpoints of the panelists.
section_title: 1,3-Butadiene Peer Review Panelists
paragraphs: The following information on the panelists is based on the roster of panelists that was provided to the public at the first meeting of the panel:   10 of the 1,3-Butadiene panelists were professors, medical directors, or both at academic or medical institutions;   4 worked for companies or industry-affiliated research organizations; and   1 worked for a state environmental protection agency. (See table 1.)
subsections: section_title: Other Pertinent Information About the Panelists
paragraphs: The staff office obtained other pertinent information from the panelists relevant to identifying potential conflicts of interest, assessing impartiality, and achieving a properly balanced peer review panel. Some of the following information was obtained in advance of the panel’s first meeting from financial disclosure forms, and the rest of the information was obtained before the voluntary public disclosure session at the first panel meeting, during it, or both.
subsections: section_title: Information From Confidential Financial Disclosure Forms
paragraphs: The staff office had access to information from the confidential financial disclosure statements that most of the panelists provided to the staff office before the first meeting. On the basis of our review of the minutes of the meeting, at the public disclosure session the panelists did not volunteer information about the following items that could be relevant to understanding their perspectives.
One panelist was also an official of an environmental health organization.   During the prior year, one panelist received fees from 60 companies and organizations, including chemical companies and law firms, and research funding from several entities, including a chemical company.
During the prior year, one panelist received fees from about 23 companies and organizations, including chemical companies and law firms.
One panelist reported fees for serving as an expert legal witness, and several panelists reported fees for legal consulting work during the prior year.
One panelist worked for a company that manufactures and distributes 1,3-Butadiene.
Two panelists owned stock in companies that manufacture 1,3-Butadiene, distribute it, or both.
Regarding the last item—ownership of stock in companies that manufacture or distribute 1-3,Butadiene—the staff office would have known that two panelists owned such stock if it had identified the companies that manufacture or distribute this chemical. This information is relevant because federal conflict-of-interest provisions prohibit stock ownership in an affected company above certain dollar thresholds. If thresholds are exceeded, a panelist can still participate in the panel, but a waiver concluding that the need for the panelist’s service outweighs the potential conflict of interest is required. However, as discussed earlier, the staff office did not identify the manufacturers and distributors of 1,3-Butadiene before or after the panel was convened. As a result, the staff office did not obtain information from these panelists about their investments to determine whether they would need a waiver in order to participate in the panel.
section_title: Information From Public Disclosure Session
paragraphs: The meeting minutes provide public disclosure comments for 10 of the 15 panelists reported as being in attendance. On the basis of these minutes, it appears that five panelists, including the chair, may not have provided additional background information at the voluntary disclosure session. However, it could also be that the recorder was not available during these disclosures, a situation reported in the minutes of another panel we reviewed. In any event, the public record of the 1,3-Butadiene panel includes the following background information from 10 of the 15 panelists, which we are presenting as it appeared in the minutes with only minor edits.
Dr. Albertini reported (1) developing assays that were used to study 1,3-Butadiene; (2) being asked by the International Life Sciences Institute to write a test case using gene toxicity data under EPA’s guidelines; (3) being asked to comment on the Canadian 1,3-Butadiene document; (4) consulting for the Chemical Industry Institute of Toxicology; and (5) working with the Czechoslovakian government to study 1,3-Butadiene, a study being funded by the Chemical Manufacturers Association and the International Institute of Synthetic Rubber Workers.
Dr. Doull reported involvement with 1,3-Butadiene as a participant on the American Conference of Governmental Industrial Hygienists Threshold Limit Value Committee.
Dr. Faustman reported receiving a grant from EPA to look at dose- response assessment methods for noncancer endpoints; variations of these published methods were used for 1,3-Butadiene.
Dr. Kelsey reported (1) receiving a grant to study 1,3-Butadiene from the National Institute for Occupational Safety and Health and publishing the findings and (2) collaborating on an NIH-funded grant to look at pharmacokinetics and low exposure to humans with principal investigator Tom Smith.
Dr. Lewis reported working on a legal case involving butadiene for Exxonand serving as the Chairman of the Epidemiology Steering Committee for the Styrene-Butadiene Rubber Worker Mortality Study for the International Institute of Synthetic Rubber Producers.
Dr. MacGregor reported (1) working until 1992 for Chevron Corporation, a company that manufactures petrochemical products containing 1,3-Butadiene, and (2) that her spouse had published several mutagenicity studies on 1,3-Butadiene.
Dr. Medinsky reported working for the Chemical Industry Institute of Toxicology which receives dues from chemical companies, and publishing papers on pharmacokinetics pertaining to 1,3-Butadiene that were funded, in part, by the Institute.
Dr. Parkinson reported being a consultant representing rubber workers and having discussed a joint position on 1,3-Butadiene with Dr. Matanoski in the past.
Dr. Swenberg reported that he was completing a study on 1,3-Butadiene in China, being a reviewer for the Canadian health assessment of 1,3-Butadiene, having received funding from the Health Effects Institute, and being a subcontractor on a 1,3-Butadiene study for another member of the panel (Dr. Albertini).
Dr. Zeise reported peer reviewing a study on 1,3-Butadiene as a subcommittee member of the NTP Board of Scientific Counselors, involvement in time-dependent analyses of 1,3-Butadiene rodent studies, and working for a state agency that has evaluated the risks of 1,3-Butadiene.
section_title: The Staff Office’s Views on the Panel
paragraphs: The staff office does not maintain documentation on how it assesses its panels for balance in points of view and expertise represented. As a result, we could not determine how, and to what extent, the staff office considered background information about the panelists in selecting this panel and ensuring that it was appropriately balanced. The Board’s goal is generally to select members from the broad middle spectrum of opinion on the technical issue. This is a subjective assessment based, in part, on the information panelists provide on their confidential financial disclosure forms. The staff office may also receive information about the panelists from the organizations that recommend individuals to serve as panelists, such as the Natural Resources Defense Council and the American Industrial Health Council. We note that the staff office’s assessments may be more subjective than necessary because staff office officials may make decisions before reviewing the prospective panelists’ disclosure forms or without having discussed all relevant background information that would provide a good indication of the panelists’ impartiality.
We discussed the composition of this panel with staff office personnel, including the staff director, to understand how they viewed this panel vis-à-vis their stated policy of generally selecting members and consultants from the broad middle spectrum of opinion on the technical issues being reviewed. Since no documentation was preserved, the staff director attempted to recreate how the panel was balanced. To categorize the potential viewpoints of panelists, the staff director told us that in some cases they use a scale of 1 to 10 representing a continuum of viewpoints from an environmental perspective to an industry perspective. On this continuum, 1 to 3 reflects an environmental perspective, 4 to 6 the middle spectrum, and 7 to 10 an industry perspective. The staff director said that when the 1,3-Butadiene panel was formed he believed he viewed six of the panelists as reflecting the broad middle perspective, three as reflecting an environmental perspective, and six reflecting an industry perspective.
The staff director’s recollections indicate that the other affiliations and activities of the panelists are often as relevant to assessing the points of view panelists may be expected to bring to peer review panels as is the information on their primary employment. For example, for the 1,3-Butadiene panel, the staff director viewed two panelists who worked for academic or medical institutions as reflecting an industry perspective and two others as reflecting an environmental perspective.
When discussing his views on the panelists, the staff director agreed with us that in a few cases, additional information would have been helpful in better assessing the points of view the panelists represented. For example, one panelist served as an expert legal witness and several others reported legal consulting fees, but the staff office did not have information on the clients or the scientific views presented in these instances. While the staff director characterized most of these panelists as in the middle spectrum, additional information could have shifted their scores toward either the environmental or industry viewpoint. In addition, the staff director considered one panelist who worked for an academic or medical institution who also performs extensive consulting for industry (including chemical companies and law firms) as reflecting the middle spectrum. However, the staff director did not have information on the nature of the consulting work. Such information could have provided insight into the viewpoints this panelist might have brought to the panel, and the assessment would have been more credible if it had been based on more complete information about the panelist’s extensive consulting work.
We agree with the staff director that working for industry or environmental organizations, having consulting or research affiliations with industry or environmental organizations, and providing legal testimony for chemical companies or for individuals do not, necessarily, represent conflicts of interest that automatically bar an individual from serving on a peer review panel. Nonetheless, these associations should be assessed to determine the overall balance of viewpoints represented on a panel, identify any potential conflicts of interest, and assess impartiality.
section_title: Appendix III: The Three Panels on EPA’s Guidelines for Assessing the Health Risks of Carcinogens
paragraphs: EPA issued health risk assessment guidelines for carcinogens in 1986, the most widely used and arguably most significant of EPA’s health risk assessment methodologies. The guidelines incorporate some precautionary assumptions aimed at ensuring that the agency does not underestimate health risks. Some experts believe the guidelines are too precautionary while others believe they are not protective enough. Partly because of controversies that proposed changes to the guidelines have engendered, EPA’s efforts to revise and update the 1986 guidelines are not yet completed, although the agency started the process more than 10 years ago. This appendix includes background on the three peer review panels that the Science Advisory Board conducted on EPA’s cancer risk guidelines, followed by information about the three panels and the panelists serving on them.
subsections: section_title: Background
paragraphs: In 1996, EPA first requested a Board peer review of the agency’s April 1996 proposed revisions to its guidelines for assessing the health risks of carcinogens. The Board has convened three different panels consisting of 26 panelists in total. Fifteen of the panelists—or more than half— participated on only one panel, and two of the panelists participated on all three panels. The first peer review panel met in February 1997. In September 1997, this panel recommended that EPA add a number of changes and clarifications to the proposed revisions, including language explicitly addressing the health risks of infants and children. In January 1999, the Board convened a second peer review panel to review the revisions EPA had made in response to the first panel; in July 1999 the second panel reported its evaluation of EPA’s responses. Prior to convening the second panel, the Board had received several criticisms, including that the second panel was not properly balanced because it did not include pediatricians or other experts on the vulnerability and susceptibility of infants and children. The Board subsequently convened a third panel to focus on children’s issues in July 1999. In its September 2000 report, the majority of that panel urged EPA to issue the guidelines promptly and then undertake a program of research and risk assessment improvement to enable EPA to address the childhood susceptibility issue more completely in future guideline revisions.
The staff office believes that these panels were not subject to the financial conflict-of-interest provisions because they addressed a broadly applicable methodology that will be applicable to all of EPA’s cancer risk assessments. Thus, the staff office believes the panelists were not subject to the financial conflict-of-interest requirements that apply only to particular matters that affect a specific party or a group or class of entities. Nonetheless, the staff office’s responsibility for evaluating the impartiality of the panelists and ensuring the overall balance of the panel remains. In this regard, the Natural Resources Defense Council questioned, in a letter to the EPA Administrator, whether it was appropriate for the Board to include industry scientists and consultants who received funding from the chemical industry on a peer review panel whose purpose was to advise EPA on cancer risk assessment guidelines that would directly affect the agency’s future regulation of carcinogens. In our view, this question highlights the importance of ensuring that the peer review panels are properly balanced in terms of points of view and expertise represented.
This appendix contains information on each of the three panels (summarized in table 2) that reviewed EPA’s proposed revised guidelines for assessing cancer health risks. For each panel, we identify the panelists, their primary employment, their job titles (when available), and other pertinent information the panelists provided about themselves (1) on the confidential financial disclosure forms and (2) at the voluntary public disclosure session. We also present the staff director’s assessments of the viewpoints of the panelists.
Because the staff office does not maintain documentation on how it assessed its panels for balance in points of view and expertise represented, we could not determine how, and to what extent, the staff office considered background information about the panelists in selecting them and ensuring that the panels were appropriately balanced. However, we discussed the composition of each of the panels with the staff director to understand how he viewed this panel vis-à-vis the staff office’s stated policy of generally selecting members and consultants from the broad middle spectrum of opinion on the technical issue being reviewed. Since no documentation had been preserved, the staff director attempted to recreate the panel balancing that was conducted in these cases. The staff director ranked the panelists using a scale of 1 to 10 representing a continuum of viewpoints from an environmental perspective to an industry perspective. On this continuum, 1 to 3 reflects an environmental perspective, 4 to 6 the middle spectrum, and 7 to 10 an industry perspective.
section_title: First Peer Review Panel
paragraphs: EPA’s Office of Research and Development asked the Board to review seven issues in the proposed guidelines, including (1) hazard classification descriptors and narratives, (2) information requirements necessary to depart from default assumptions when gaps in data or knowledge were encountered, and (3) dose-response assessment. The 11-member cancer risk panel met in February 1997 and issued its report in September 1997. The report generally commended EPA’s efforts to update its guidelines and identified areas for improvement.
The following information on the panelists is based on the roster of panelists that was provided to the public at the first meeting of the panel:   six of the panelists worked for academic or medical institutions,  three worked for companies or industry-affiliated research organizations,   one worked for a state environmental protection agency, and   one panelist’s employment was not identified.
As shown in table 3, most of the panelists’ job titles were not provided, and neither the primary employment nor the title was provided for one panelist. These omissions limit the ability of the public to assess the panelists.
subsections: section_title: Other Pertinent Information About the Panelists
paragraphs: The staff office obtained other pertinent information from the panelists relevant to reviewing for potential conflicts of interest, impartiality, and achieving a properly balanced peer review panel. Some of the information was obtained from financial disclosure forms that should have been provided to the staff office in advance of the meeting. Other information was obtained before or during the voluntary public disclosure session that the panelists are asked, but not required, to participate in at the first panel meeting.
subsections: section_title: Information From Confidential Financial Disclosure Forms
paragraphs: Much of the additional information the staff office may have obtained about the panelists from confidential financial disclosure forms is not available because the staff office was unable to locate 7 of the 11 forms.Three of the four panelists for whom confidential disclosure forms were found provided the following information that could be relevant to understanding their perspectives.
One panelist was also a board member of an environmental advocacy organization.
One panelist received fees from 19 companies and organizations, including the CMA (likely referring to the Chemical Manufacturers Association), during the previous year.
One panelist was a member of a chemical company committee on biohazards.
Because the public disclosures were not recorded, we could not determine whether any of these panelists provided this information at the meeting.
section_title: Information From Public Disclosure Session
paragraphs: No information that the panelists provided during the public disclosure session at the first meeting of this panel is available because the meeting minutes only summarize the disclosure session: they simply state that the public disclosure discussion was held and that no conflicts of interest were identified.
section_title: The Staff Office’s Views on the First Panel
paragraphs: Retrospectively, the Board’s staff director said he believed that when this panel was formed he viewed two of the panel members as reflecting an environmental perspective, seven a middle perspective, and two an industry perspective. Given these assessments, it is clear that information more than simply the primary employment of the panelists, such as other affiliations and activities of the panelists, is relevant to assessing the points of view they may be expected to bring to the peer review panels. For example, for this first cancer guidelines panel, the staff director viewed one panelist who worked for an academic or medical institution as reflecting an industry perspective and another panelist who worked for an academic or medical institution as reflecting an environmental perspective.
Because most of the confidential disclosure forms were not available and the public disclosures were not recorded, our discussions with staff office officials about the panelists’ backgrounds were necessarily limited. However, we did find that the staff office did not obtain or review the confidential disclosure form for one panelist it classified as representing the middle viewpoint, either at the time the panel was formed or later. Staff office officials do not review disclosure forms of panelists who also serve on other EPA advisory committees, as was the case with this panelist. As a result, the staff office’s assessment did not consider specific employment and background information the panelist had already provided to EPA.
section_title: Second Peer Review Panel
paragraphs: EPA asked the Board to comment on sections of the cancer risk guidelines that had been revised to address the Board’s initial recommendations as well as the public’s comments. Specific issues to be addressed included (1) the adequacy of proposed hazard descriptors, (2) the use of information on the ways an agent produces cancer, and (3) dose-response analysis. A 17-member panel, which met January 20 and 21, 1999, conducted the review. The report, issued in July 1999, addressed each of these topics.
The following information on the panelists is based on the roster of panelists that was provided to the public at the first meeting of the panel:   nine panelists were professors or medical directors of academic or   seven held various positions with companies or industry-affiliated   one worked for a state environmental protection agency.
The panelists’ names, primary employment, and titles were provided to the public at the panel’s initial meeting (see table 4).
subsections: section_title: Other Pertinent Information About the Panelists
paragraphs: The staff office obtained other pertinent information from the panelists relevant to reviewing for potential conflicts of interest, impartiality, and achieving a properly balanced peer review panel. Some of the information was obtained before the panel convened from financial disclosure forms. Other information was obtained before or during the voluntary public disclosure session conducted at the first panel meeting.
subsections: section_title: Information From Confidential Financial Disclosure Forms
paragraphs: The staff office had access to information from confidential financial disclosure statements that most of the panelists provided to the staff office. The following information that could be relevant to understanding five of the panelist’s perspectives was indicated on the forms:   During the prior year, one panelist had received fees from 35 companies, including chemical companies, a research organization funded by the tobacco industry, and law firms. This panelist had also received research funding from a chemical company and a tobacco company.
One panelist had been a consultant to 16 companies, including 4 chemical companies, during the prior year.
One had been a consultant to 11 companies or organizations, including a chemical company, during the prior year.
One owned stock in 11 companies, including 2 chemical companies.   One held stock in a chemical company.
section_title: Information From Public Disclosure Session
paragraphs: The meeting minutes state that the disclosures of 5 of the 17 members of this panel are not reported because the recorder left the room for a time during the disclosure session. The public record of the panel includes the following information from 12 of the panelists, which we are presenting as it appeared in the minutes with only minor edits.
Dr. Bull, senior staff scientist at the Battelle Pacific Northwest National Laboratory, reported (1) having a particular interest in the EPA cancer guidelines in his role as chair of the EPA Science Advisory Board Drinking Water Committee, (2) serving on the International Life Science Institute chloroform steering group, and (3) conducting research related to the EPA cancer guidelines with the EPA and the National Aeronautics and Space Administration.
Dr. Crump, Vice President of KS Crump Group, Inc., reported conducting research on some of the methodology included in the EPA cancer guidelines.
Dr. Dragan, Assistant Professor at Ohio State University, reported conducting carcinogenesis research and serving on the International Life Science Institute subcommittee that had reviewed the cancer guidelines.
Dr. Jones, the Director for Experimental Gynecology-Endocrinology in the Department of Gynecologic Oncology at the MD Anderson Medical Center, reported (1) conducting research that was not directly related to the guidelines and (2) serving as a member of the EPA Endocrine Disrupter Screening and Testing Advisory Committee.
Dr. Lambert, Associate Professor of Pediatrics and Director of Pediatric Pharmacology and Toxicology at the University of Medicine and Dentistry of New Jersey-Robert Wood Johnson Medical School, and Attending Neonatologist at the Robert Wood Johnson University Hospital and St. Peter’s Medical Center, reported (1) working as director for the Center for Child and Reproductive Environmental Health and (2) conducting research on endocrine disruptors and their effect on human development.
Dr. McConnell, President of Toxpath, Inc., reported (1) serving as the chair of EPA’s Scientific Advisory Panel, that will be using the cancer guidelines and (2) conducting research with Dr. James Swenberg on combining tumor and tissues sites, which is referenced throughout EPA’s background material on the cancer guidelines.
Dr. McClellan reported (1) working as president of the Chemical Industry Institute of Toxicology, a not-for-profit research institution that is supported by chemical manufacturers and that receives supplemental funding for research in which cancer is a disease endpoint; (2) serving on the International Life Science Institute’s chloroform steering committee group; (3) making numerous public pronouncements, including congressional testimony; (4) holding advisory roles with several organizations, such as Resources for the Future and the Center for Risk Management; and (5) conducting research on the biological effects of low doses of radon and alternative toxicological hazards.
Dr. Medinsky reported (1) working as a toxicology consultant and (2) conducting physiologically based pharmacokinetic modeling research on hazardous air pollutants, many of which are known human carcinogens.
Dr. Perera reported (1) having published papers in peer reviewed journals and (2) having received funding for research related to the cancer guidelines from the EPA, DOD, NIEHS, and private foundations.
Dr. Teta, Director of Epidemiology, Health Information, Risk Assessment, and TSCA for Union Carbide Corporation, reported (1) having investments in her employer, a producer of chemicals; (2) serving for 10 to 12 years with the American Industrial Health Council, which submitted comments to EPA on the cancer guidelines; (3) serving as chair of the American Industrial Health Council panel and Chemical Manufacturers Association butadiene panel; (4) writing a yet-unpublished paper on the application of the guidelines using ethylene oxide; and (5) providing public comments as an International Life Science Institute member.
Dr. Utell, who works for the University of Rochester, reported (1) the general interest of his employer in the EPA cancer guidelines, (2) involvement with research with the Health Effects Institute and EPA, (3) serving as a member of the Health Effects Institute’s research committee, and (4) serving as a board member of the Annapolis Center.
Dr. Zeise, Chief of the Reproductive and Cancer Hazard Assessment Section for the Office of Environmental Health Hazard Assessment at the California Environmental Protection Agency, reported (1) working extensively in the areas of risk assessment methodologies and cancer risk assessment and (2) serving as an expert witness in several court cases.
section_title: The Staff Office’s Views on the Second Panel
paragraphs: Retrospectively, the staff director said he believed that at the time the second panel was appointed the staff office viewed four of the panel members as reflecting an environmental perspective, nine a middle perspective, and four an industry perspective. In the staff director’s view, panelists from academic or medical institutions do not necessarily represent the broad middle spectrum, and panelists from industry may not reflect an industry perspective. For this panel, the staff director categorized three panelists who worked for industry as having a middle perspective, one panelist who worked for an academic or medical institution as having an industry perspective, and three panelists who worked for academic or medical institutions as reflecting an environmental perspective.
When discussing his views on the panelists, the staff director agreed that, in some cases, additional information would have helped to better assess the points of view the panelists represented. Without that information, the assessments are more subjective than they might have been. For example, the staff director considered one panelist who worked for an academic or medical institution as reflecting the middle spectrum. On the confidential financial disclosure form, however, this panelist reported a substantial number of consulting fees from companies and law firms, including a tobacco company and a research organization funded by the tobacco industry. While staff office officials said they were aware of the panelist’s relationships with the tobacco industry, they acknowledged that they did not obtain information about the numerous consulting fees reported. The assessment of this individual may have been accurate; however, it would have been more credible if it had reflected specific knowledge about the panelist’s extensive industry and legal consulting work. Moreover, staff office officials acknowledged they were not aware another panelist they considered to represent the middle perspective had industry clients because they did not review this individual’s confidential financial disclosure form.
section_title: Third Peer Review Panel
paragraphs: One outstanding issue from the first peer review panel was the recommendation to expand the discussion in the guidelines regarding children. EPA requested that the Board provide comments on further revisions to the guidelines that were intended to address children’s risk. An 11-member panel met on July 27 and 28, 1999, to conduct the review. In the panel’s September 2000 report, a majority of the panelists urged EPA to issue the guidelines promptly and then to undertake a program of research and risk assessment improvement to enable EPA to address the childhood susceptibility issue more completely in future guideline revisions. Several panel members disagreed with the majority, however, asserting that the agency should fully address the childhood susceptibility issue--that is, conduct the research before finalizing the guidelines.
The following information on the panelists is based on the roster of panelists that was provided to the public at the first meeting of the panel: five panelists worked for academic or medical institutions, four panelists worked for companies or industry-affiliated research institutions, and two panelists worked for a state environmental protection or health agency.
The panelists’ names, primary employment, and titles were provided to the public at the panel’s initial meeting (see table 5).
subsections: section_title: Other Pertinent Information About the Panelists
paragraphs: The staff office obtained other pertinent information from the panelists relevant to reviewing for potential conflicts of interest, impartiality, and achieving a properly balanced peer review panel. Some of the information was obtained before the panel convened from financial disclosure forms, and other information was obtained before or during the voluntary public disclosure session at the first panel meeting.
subsections: section_title: Information From Confidential Financial Disclosure Forms
paragraphs: The staff office had access to the following information from confidential financial disclosure statements that the panelists provided to the staff office. The following information that could be relevant to understanding two of the panelist’s perspectives was indicated on the forms:   One panelist was a consultant for the Chemical Manufacturers Association.
One panelist had received consulting fees from 3 organizations, including a chemical industry research organization; received professional fees from 7 law firms; and held stock in about 50 companies, including 2 tobacco companies.
section_title: Information From Public Disclosure Session
paragraphs: Regarding the public disclosure session held at the first meeting of the panel, the minutes state only that none of the panelists identified any possible conflicts of interest. The record does not indicate whether any of the individuals provided additional background information or what they said.
section_title: The Staff Office’s Views on the Third Panel
paragraphs: Retrospectively, the staff director said that at the time the panel was appointed, he believed he viewed two of the panel members as reflecting an environmental perspective, seven a middle perspective, and two an industry perspective. In the staff director’s view, panelists from academic or medical institutions do not necessarily represent the broad middle spectrum, panelists from industry may not reflect an industry perspective, and panelists from environmental or health organizations may not reflect an environmental perspective.  For this panel, the staff director evaluated one panelist who worked for an academic or medical institution as reflecting an environmental perspective, two panelists who worked for companies or industry-affiliated research organizations as reflecting the middle perspective, and one panelist who worked for a state environmental or health agency as reflecting the middle perspective.
As was the case with the other panels, we believe the staff office would have benefited from additional information in assessing the points of view the panelists represented. For example, the assessments would have been more credible if they had reflected specific information about the consulting and professional fees and stocks owned by one of the panelists and the work performed by another panelist for the Chemical Manufacturers Association.