# Presentation: 425267

## Strengthening Science Supporting

**Strengthening Science Supporting**
**Fishery Management**

** ****Standards for Best Available Science**
** ****Implementation of OMB’s Peer Review Bulletin**
** ****Separation between Science & Management**

- Marine Fisheries Advisory Committee
- June 7-10, 2005
- Washington DC

## “Conservation and management measures shall be based upon the best scientific information available.”

**“*****Conservation and management measures shall be based upon the best scientific information available*****.”**

-  **How do we determine and assure the science we use is the **_**best**_**?**
** ****What do we do when there are recognized gaps in the science that is **_**available**_**?**

## Importance of National Standard 2

** ****affirms the role of science as the basis for**
**management decision-making**
** ****has resulted in a set of procedures and guidance for selecting “best” from a number of potential science alternatives (differ regionally)**
** ****stipulates that the lack of perfect science will not be used to delay implementation of required measures, when indicated by the preponderance of available information**
** ****implies commitment to improving science used in decision-making**

## National Research Council Study (NRC 2004)

-  How should adherence to NS-2 be Measured?
-  How and when should it be employed?
-  Should NS-2 be employed to exclude inadequate data or should it be ranked and applied in relation to relevance & rigor?
- Workshop & several studies
-  recommendations & guidance

## Proposed NRC Guidelines for

**Production and Use of**
**Scientific Information in FMPs**

** ****Relevance**
** ****Inclusiveness**
** ****Objectivity**

** ****Transparency &**
**    ****Openness**
** ****Timeliness**
** ****Peer Review**

## OMB Peer Review Bulletin Introduction

- Background
- Basics of Bulletin
- Application
- What’s Covered
- What’s Not
- Peer Review Standards
- Agency Requirements
- Important Dates
- Conclusion

## Background

- Information Quality Act
- Ensure and maximize quality, objectivity, utility, and integrity of information disseminated by NOAA
- Administrative mechanism allowing affected persons to seek and obtain correction of information that does not comply with OMB and NOAA Guidelines
- Report to OMB number and nature of requests received and how they were handled by NOAA

**Notes:**
NOAA’s Information Quality Guidelines were published September 30, 2002, and can be found on NOAA’s home page.  The Guidelines provide guidance and details on the items above.

## Background

- OMB Peer Review Bulletin applies to two types of information products covered by IQA
- Influential scientific information, and
- Highly influential scientific assessments, a subset of influential scientific information

**Notes:**
OMB Peer Review Bulletin was published December 16, 2004

## Basics of Bulletin

- Bulletin establishes:
- Minimum peer review standards
- Transparent process for public disclosure
- Opportunity for public input
- Bulletin issued under the IQA and OMB’s general authorities to oversee the quality of agency information, analyses and regulatory activities

**Notes:**
Standards for when peer review is required and the type of peer review to be considered

Process covers peer review planning, including a web-accessible description (agenda) of the agency’s peer review plans for all information subject to the Bulletin.

OMB cites the IQA, Executive Order 12866, and the President’s Constitutional authority to supervise the unitary Executive Branch as authorities that support the Bulletin.

## What’s Covered

- Influential scientific information
- Scientific information that the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions
- Scientific assessment
- An evaluation of a body of scientific or technical knowledge that typically synthesizes multiple factual inputs, data, models, assumptions, and/or applies best professional judgment to bridge uncertainties in the available information
- Highly influential scientific assessments
- (i) Could have a potential impact of more than $500 million in any year, or
- (ii) Is novel, controversial, or precedent-setting or has significant interagency interest

**Notes:**
“Scientific information” means factual inputs, data, models, analyses, technical information, or scientific assessments based on the behavioral and social sciences, public health and medical sciences, life and earth sciences, engineering, or physical sciences. This includes any communication or representation of knowledge such as facts or data, in any medium or form, including textual, numerical, graphic, cartographic, narrative, or audiovisual forms. This definition includes information that an agency disseminates from a web page, but does not include the provision of hyperlinks to information that others disseminate. This definition does not include opinions, where the agency’s presentation makes clear that what is being offered is someone’s opinion rather than fact or the agency’s views.

“Influential scientific information” means scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions.
“Scientific assessment” means an evaluation of a body of scientific or technical knowledge that typically synthesizes multiple factual inputs, data, models, assumptions, and/or applies best professional judgment to bridge uncertainties in the available information. These assessments include, but are not limited to, state-of-science reports; technology assessments; weight-of-evidence analyses; meta-analyses; health, safety, or ecological risk assessments; toxicological characterizations of substances; integrated assessment models; hazard determinations; or exposure assessments. 

“Highly influential scientific assessment” means scientific assessment that: 
	(i) could have a potential impact of more than $500 million in any year, or 
(ii) is novel, controversial, or precedent-setting or has significant interagency interest. 

## What’s Not

- Exclusions of Section 515 incorporated into Bulletin; examples include:
- Distribution limited to government employees or agency contractors or grantees
- Intra- or inter-agency use or sharing of government information
- Responses to requests for agency records under FOIA, the Privacy Act, the Federal Advisory Committee Act, etc.
- Correspondence with individuals or persons, press releases, archival records, public filings, subpoenas and adjudicative processes.
- Research produced by government-funded scientists if that information does not represent the views of the agency (must include a specific disclaimer)

**Notes:**
	515 exclusions include:
	Excluded from the definition of “dissemination”
Distribution limited to government employees or agency contractors or grantees
Intra- or inter-agency use or sharing of government information
Responses to requests for agency records under the Freedom of Information Act, the Privacy Act, the Federal Advisory Committee Act, the Government Performance and Results Act or similar law. 
Correspondence with individuals or persons, press releases, archival records, public filings, subpoenas and adjudicative processes.
Research produced by government-funded scientists if that information does not represent the views of the agency (must include a specific disclaimer).

Excluded from the definition of “scientific information”
The provision of hyperlinks to information that others disseminate.
Opinions, where the agency’s presentation makes clear that what is being offered is someone’s opinion rather than fact or the agency’s view.

## What’s Not

- Exemptions specific to the Peer Review Bulletin
- Information related to national security or foreign affairs
- Regulatory impact analysis or regulatory flexibility analysis under EO 12866
- Routine statistical information
- Information distributed for peer review in compliance with the Bulletin (must include a specific disclaimer)

**Notes:**
Exemptions from the Peer Review Bulletin (Section IX)
Agencies need not have peer review conducted on information that is:
  related to certain national security, foreign affairs, or negotiations involving international trade or treaties where compliance with this Bulletin would interfere with the need for secrecy or promptness;
  disseminated in the course of an individual agency adjudication or permit proceeding (including a registration, approval, licensing, site-specific determination), unless the agency determines that peer review is practical and appropriate and that the influential dissemination is scientifically or technically novel or likely to have precedent-setting influence on future adjudications and/or permit proceedings;
  a health or safety dissemination where the agency determines that the dissemination is time-sensitive (e.g., findings based primarily on data from a recent clinical trial that was adequately peer reviewed before the trial began;
  an agency regulatory impact analysis or regulatory flexibility analysis subject to interagency review under Executive Order 12866, except for underlying data and analytical models used;
  routine statistical information released by federal statistical agencies (e.g., periodic demographic and economic statistics) and analyses of these data to compute standard indicators and trends (e.g., unemployment and poverty rates);
  accounting, budget, actuarial, and financial information, including that which is generated or used by agencies that focus on interest rates, banking, currency, securities, commodities, futures, or taxes; or
  information disseminated in connection with routine rules that materially alter entitlements, grants, user fees, or loan programs, or the rights and obligations of recipients thereof.
Definitional exclusion unique to the Peer Review Bulletin
Information distributed for peer review in compliance with the Bulletin (must include a specific disclaimer).

## Peer Review Standards:Two Levels

- Standards for influential scientific information
- Standards for highly influential scientific assessments include those above, plus several others

## Peer Review Standards: Influential Scientific Information

- Selection of reviewers
- Expertise and balance
- Conflicts of interest
- Independence
- Choice of peer review mechanism
- Transparency
- Management of peer review process and reviewer selection

## Peer Review Standards:Highly Influential Scientific Assessments

- All standards for influential scientific information, plus
- Selection of reviewers
- Expertise and balance
- Conflicts
- Independence
- Rotation
- Information access
- Opportunity for public participation
- Transparency
- Management of peer review process and reviewer selection

**Notes:**
Selection of reviewers:
Expertise and Balance: Peer reviewers shall be selected based on expertise, experience and skills, including specialists from multiple disciplines, as necessary.
Conflicts: The agency – or the entity selecting the peer reviewers – shall (i) ensure that those reviewers serving as federal employees comply with applicable federal ethics requirements; (ii) in selecting peer reviewers who are not government employees, adopt or adapt the National Academy of Sciences’ policy for committee selection with respect to evaluating the potential for conflicts.
Independence: The agency shall bar participation of scientists employed by the sponsoring agency unless the reviewer is employed only for the purpose of conducting the peer review, with one exception.
Rotation: Agencies shall avoid repeated use of the same reviewer on multiple assessments unless his or her participation is essential and cannot be obtained elsewhere.
Information Access: The agency shall provide the reviewers with sufficient information to enable them to understand the data, analytic procedures, and assumptions used to support the key findings or conclusions of the draft assessment. 
Opportunity for Public Participation: Whenever feasible and appropriate, the agency shall make the draft scientific assessment available to the public for comment at the same time it is submitted for peer review (or during the peer review process) and sponsor a public meeting where oral presentations on scientific issues can be made to the peer reviewers by interested members of the public. 
Transparency: The peer review report shall include the charge to the reviewers and a short paragraph on both the credentials and relevant experiences of each peer reviewer. The agency shall prepare a written response to the peer review report explaining (a) the agency's agreement or disagreement with the views expressed in the report, (b) the actions the agency has undertaken or will undertake in response to the report, and (c) the reasons the agency believes those actions satisfy the key concerns stated in the report (if applicable).
Management of Peer Review Process and Reviewer Selection: The agency may commission independent entities to manage the peer review process, including the selection of peer reviewers, in accordance with this Bulletin. 

## Peer Review Standards:Alternative Procedures

- Agency may:
- rely on the principal findings, conclusions and recommendations of a report produced by the National Academy of Sciences;
- commission the National Academy of Sciences to peer review an agency’s draft scientific information; or
- employ an alternative scientific procedure or process, specifically approved by the Administrator of OIRA in consultation with the Office of Science and Technology Policy (OSTP), that ensures the agency’s scientific information satisfies applicable information quality standards

**Notes:**
OIRA is OMB’s Office of Information and Regulatory Affairs

## NOAA Requirements

- Peer Review Agenda
- Peer Review Plans
- Public Comment
- Annual Reports
- Certification in the Administrative Record
- Populate DOC web site

**Notes:**
Peer Review Agenda: Each agency shall post on its website, and update at least every six months, an agenda of peer review plans. The agenda shall describe all planned and ongoing influential scientific information subject to this Bulletin.  The agency shall provide a link from the agenda to each document that has been made public pursuant to OMB Bulletin. Agencies are encouraged to offer a listserve or similar mechanism to alert interested members of the public when entries are added or updated. 
Peer Review Plans: For each entry on the agenda the agency shall describe the peer review plan. Each peer review plan shall include: (i) a paragraph including the title, subject and purpose of the planned report, as well as an agency contact to whom inquiries may be directed to learn the specifics of the plan; (ii) whether the dissemination is likely to be influential scientific information or a highly influential scientific assessment; (iii) the timing of the review (including deferrals); (iv) whether the review will be conducted through a panel or individual letters (or whether an alternative procedure will be employed); (v) whether there will be opportunities for the public to comment on the work product to be peer reviewed, and if so, how and when these opportunities will be provided; (vi) whether the agency will provide significant and relevant public comments to the peer reviewers before they conduct their review; (vii) the anticipated number of reviewers (3 or fewer; 4-10; or more than 10); (viii) a succinct description of the primary disciplines or expertise needed in the review; (ix) whether reviewers will be selected by the agency or by a designated outside organization; and (x) whether the public, including scientific or professional societies, will be asked to nominate potential peer reviewers. 
Public Comment: Agencies shall establish a mechanism for allowing the public to comment on the adequacy of the peer review plans. Agencies shall consider public comments on peer review plans. 
Annual Reports.  Each agency shall provide to OIRA, by December 15 of each year, a summary of the peer reviews conducted by the agency during the fiscal year. The report should include the following: 1) the number of peer reviews conducted subject to the Bulletin (i.e., for influential scientific information and highly influential scientific assessments); 2) the number of times alternative procedures were invoked; 3) the number of times waivers or deferrals were invoked (and in the case of deferrals, the length of time elapsed between the deferral and the peer review); 4) any decision to appoint a reviewer pursuant to any exception to the applicable independence or conflict of interest standards of the Bulletin, including determinations by the Secretary pursuant to Section III(3)(c); 5) the number of peer review panels that were conducted in public and the number that allowed public comment; 6) the number of public comments provided on the agency’s peer review plans; and 7) the number of peer reviewers that the agency used that were recommended by professional societies.
 Certification in the Administrative Record.  If an agency relies on influential scientific information or a highly influential scientific assessment subject to this Bulletin to support a regulatory action, it shall include in the administrative record for that action a certification explaining how the agency has complied with the requirements of this Bulletin and the applicable information quality guidelines. Relevant materials shall be placed in the administrative record. 
DOC web site:
Must link to www.FirstGov.com
Must be in place for highly influential scientific assessments as of June 16, 2005 and for influential scientific information by December 16, 2005.
The agency shall disseminate its response to the peer review report on the agency's website with related material related to transparency. 
The agency shall disseminate the final peer review report on the agency's website along with all materials related to the peer review (any charge statement, the peer review report, and any agency response). 

## Important Dates

- Bulletin applies to information disseminated on or after June 16, 2005
- Except for information for which the agency has already provided a draft report and an associated charge to peer reviewers
- Section V peer review planning requirements for highly influential scientific assessments apply as of June 16, 2005
- Section V peer review planning requirements for influential scientific information apply as of December 16, 2005
- Annual Reports - December 15 of each year
- Peer Review Agenda on DOC web site – by June 16, 2005

## Conclusion

- Line and Staff Offices must be made aware of requirements
- NOAA must have requirements for highly influential scientific assessments in place by June 16, 2005
- Call for agenda items in mid-March
- NOAA web site must be operational by June 16, 2005
- Compliance with the Bulletin should be addressed early in the development of information products

**Notes:**
If the requirements of the bulletin are triggered for a science information product, need to develop a Peer Review Plan for that product.

## Discussion: Separating Science & Management

- Administratively within NOAA Fisheries Service
- Use of Peer Review Mechanisms by Councils (e.g., SSCs, SSS –SAW/SEDAR/STAR)
- Use of Peer Review Products in Decision Making
- Certification that Management Complies with BAS
- Development of guidelines to formalize science-management relationships