**Guidelines for implementing**

**the office of management and budget's**

**"final information quality bulletin for peer review"**

OctOber 5, 2005

**TABLE OF CONTENTS**

[1. BACKGROUND AND PURPOSE OF THESE PROCEDURES
3](#background-and-purpose-of-these-procedures)

[2. SCOPE AND APPLICABILITY OF PROCEDURES TO DOT ORGANIZATIONS
4](#scope-and-applicability-of-procedures-to-dot-organizations)

[3. KEY PEER REVIEW PRINCIPLES 6](#key-peer-review-principles)

[4. PROCEDURES FOR DETERMINING WHETHER INFORMATION IS COVERED BY THE OMB
BULLETIN
8](#procedures-for-determining-whether-information-is-covered-by-the-omb-bulletin)

[5. PROCEDURES FOR MANAGING PEER REVIEWS OF PLANNED SCIENTIFIC
INFORMATION DISSEMINATIONS
9](#procedures-for-managing-peer-reviews-of-planned-scientific-information-disseminations)

[6. PROCEDURES FOR ESTABLISHING AND MAINTAINING AN ADMINISTRATIVE RECORD
12](#procedures-for-establishing-and-maintaining-an-administrative-record)

[7. PROCEDURES FOR CONDUCTING PEER REVIEWS
14](#procedures-for-conducting-peer-reviews)

[8. PROCEDURES FOR DEVELOPING PEER REVIEW PLANS
19](#procedures-for-developing-peer-review-plans)

[9. PROCEDURES FOR PUBLIC PARTICIPATION IN PEER REVIEWS
21](#procedures-for-public-participation-in-peer-reviews)

[10. PROCEDURES FOR AVOIDING CONFLICTS OF INTEREST
22](#procedures-for-avoiding-conflicts-of-interest)

[11. PROCEDURES FOR ANNUAL REPORTING
24](#procedures-for-annual-reporting)

[12. PROCEDURES FOR PEER REVIEWS SUPPORTING RULEMAKING
25](#procedures-for-peer-reviews-supporting-rulemaking)

[13. BUDGETING FOR PEER REVIEW EXPENSES
26](#budgeting-for-peer-review-expenses)

[14. DEFERRALS AND TO THE PEER REVIEW PROCESS
27](#peer-review-process-deferrals-and-waivers)

[15. EXEMPTIONS TO THE PEER REVIEW PROCESS
28](#exemptions-to-the-peer-review-process)

[APPENDIX A: KEY DEFINITIONS 30](#appendix-a-key-definitions)

[APPENDIX B: LIST OF ACRONYMS AND ABBREVIATIONS
33](#appendix-b-list-of-acronyms-and-abbreviations)

[APPENDIX C: QUESTIONS AND ANSWERS REGARDING COMPLIANCE WITH OMB
BULLETIN
34](#appendix-c-questions-and-answers-regarding-compliance-with-omb-bulletin)

[APPENDIX D: SUMMARY TRACKING MATRIX USING FHWA SUBMISSIONS TO ACHIEVE
COMPLIANCE TEMPLATE
36](#appendix-d-summary-tracking-matrix-using-fhwa-submissions-to-achieve-compliance-template)

[APPENDIX E: SUBMISSION FORM AND PEER REVIEW PLAN
37](#appendix-e-submission-form-and-peer-review-plan)

[APPENDIX F: SAMPLE CONFLICT OF INTEREST FORM
39](#appendix-f-sample-conflict-of-interest-form)

[APPENDIX G: PEER REVIEW CHECKLIST
40](#appendix-g-peer-review-checklist)

# 

# BACKGROUND AND PURPOSE OF THESE PROCEDURES

The following procedures are designed to facilitate the Department of
Transportation's (DOT) implementation of the Office of Management and
Budget's (OMB) Bulletin entitled, " Final Information Quality Bulletin
for Peer Review." This Bulletin establishes government-wide guidance
aimed at enhancing the practice of peer review of government science
documents. Peer review is intended to increase the quality and
credibility of the scientific information generated across the Federal
government.

OMB recognizes that different types of peer review are appropriate for
different types of scientific information. Under the Bulletin, Federal
agencies, including DOT, are granted broad discretion to weigh the
benefits and costs of using a particular peer review mechanism for a
specific information product. The selection of an appropriate peer
review mechanism for scientific information is left to agency
discretion. Various types of information are exempted from the
requirements of the Bulletin, including time-sensitive health and safety
determinations, in order that peer review not unduly delay the release
of urgent findings.

OMB's Bulletin is issued under the Information Quality Act (IQA) and
OMB's general authorities to oversee the quality of Operating
Administration (OA) information, analyses, and regulatory actions. In
the Information Quality Act, Congress directed OMB to issue guidelines
to "provide policy and procedural guidance to Federal agencies for
ensuring and maximizing the quality, objectivity, utility, and integrity
of information, (including statistical information) disseminated by
Federal agencies. Pub. Law. No. 106-554-515(a). The Information Quality
Act was developed as a supplement to the Paperwork Reduction Act, 44
U.S.C.& 3501 [et]{.underline} [seq]{.underline}., which requires OMB,
among other things, to "develop and oversee the implementation of
policies, principles, standards and guidelines to...apply to Federal OA
dissemination of public information."

The DOT has designated the OA Chief Information Officer (CIO) as the
senior official responsible for compliance with Section 515 of the
[Treasury and General Government Appropriations Act for Fiscal Year
2001]{.underline} (PL 106-554). Under this Act, Federal agencies subject
to the [Paperwork Reduction Act]{.underline} (44 U.S.C. Chapter 35)
implemented written guidelines to ensure and maximize the quality,
utility, objectivity, and integrity of the information that they
disseminate. DOT's IQA guidelines apply to a wide variety of substantive
information dissemination activities in order to meet basic information
quality standards set forth by Section 515.

The DOT acknowledges that ensuring the quality of information is an
important management objective along with other Departmental objectives,
such as ensuring the success of OA missions, observing the budget and
resource priorities and restraints, and providing useful information to
the public. The implementation of scientific peer review completes the
DOT's organizational excellence objective by further ensuring quality
scientific information.

# SCOPE AND APPLICABILITY OF PROCEDURES TO DOT ORGANIZATIONS

The procedures provided in this document apply to all operating
administrations and organizations that are planning to disseminate to
the public scientific information after June 16, 2005. *Scientific
information*[^1] means factual inputs, data, models, analyses, technical
information, or scientific assessments related to such disciplines as
the behavioral and social sciences, public health and medical sciences,
life and earth sciences, engineering or physical sciences. This includes
any communication or representation of knowledge such as facts or data,
in any medium or form, including textual, numerical, graphic,
cartographic, narrative, or audiovisual forms. There are two categories
of scientific information that are covered by these procedures:

1.  *Influential scientific information (ISI)* means scientific
    information that DOT can reasonably determine "will have or does
    have a clear and substantial impact on important public policies or
    private sector decisions."[^2]

2.  *Highly influential scientific assessments* *(HISAs)* are those
    "that could have a potential impact of more than \$500 million in
    any one year on either the public or private sector or that the
    dissemination is novel, controversial, precedent-setting, or has
    significant interagency interest."[^3] A scientific assessment is an
    "evaluation of a body a scientific or technical information that
    typically synthesizes multiple factual inputs, data, models,
    assumptions, and/or applies best professional judgment to bridge
    uncertainties in the available information.***"*** Other criteria
    that can be used to determine whether a scientific work product is
    highly influential could include:

    1.  Information focused on significant emerging scientific,
        technical, or regulatory issues.

    2.  Information that involves the application of an innovative
        approach for a previously defined problem/process/methodology.

    3.  Information that satisfies a statutory or other legal mandate
        for peer review.

    4.  Guidance for conducting economic or financial analyses.

    5.  New financial or economic methodologies that serve as the
        primary method or protocol for conducting economic analyses
        within a program or operating administration.

    6.  Unique, novel, or previously un-validated applications of
        economic or financial assessment methodologies.

    7.  Broad scale assessments of regulatory programs, such as those
        required by Congressional-mandates.

    8.  Surveys of customer preferences.

    9.  New national surveys of costs and expenditures for environmental
        protection.

    10. Economic research plans developed to assess and advance the
        state-of-the-art science in economic theory, methodologies, or
        modeling.

    11. New meta-analyses that re-analyze existing published literature
        and supporting data.

    12. New algorithms that will provide the basis of modeling and
        predicting traffic behavior.

    13. Large-scale data sources that have not gone through an
        OMB-approved peer review process in the past.

Examples of highly influential scientific assessments can be found at
[http://www.dot.gov]{.underline} or in Appendices D & E. Examples of
influential scientific information will be available in the Docket
Management System (DMS)[^4] by October, 2005.

#  

# KEY PEER REVIEW PRINCIPLES

There are various ways to conduct peer reviews of scientific information
disseminations. Some may be internal and/or informal while others may be
the formalized and rigorous reviews reserved for highly influential
scientific assessments. The following principles, which are organized
into four categories, are designed to promote the consistency and
adequacy of peer reviews conducted for highly influential scientific
assessments and influential scientific information:

3.  Accountability

    1.  Each organization within DOT will have discretion for
        determining the type of peer review appropriate for influential
        scientific information and adapting procedures for the selection
        of peer reviewers.

    2.  Scientific peer reviews will be coordinated by a supervisor at
        least one level removed from that at which the scientific
        information was produced.

    3.  Selection of peer reviewers is the responsibility of the
        individual managing the peer review.

    4.  Peer review processes and activities should be reviewed on an
        annual basis by each operating administration's chief
        information office.

    5.  Public participation will be encouraged as a key component of
        the peer review process.

4.  Scope

    1.  All influential and highly influential scientific information
        shall be peer reviewed prior to dissemination to the public.

    2.  Peer review planning should take into account both benefit and
        costs.[^5]

5.  Integrity

    1.  Peer reviewers must be selected on the basis of three criteria:
        expertise, balance,[^6] and independence.

    2.  OAs shall ensure that their peer review practices are
        characterized by both scientific integrity and process
        integrity.[^7]

    3.  Reviews shall be conducted by scientific peers as demonstrated
        by their scientific achievements.

    4.  Peer reviewers shall be independent and free of conflicts of
        interest.

    5.  Peer reviewers should not be employed by OA or office producing
        the document.

6.  Transparency

    1.  Responses to public comments on peer review plans and highly
        influential and influential scientific information will be
        provided in writing and provided to the DMS for posting.

    2.  The comments and statements obtained in a review will be made
        available to the investigator or author, and responses by the
        investigator or author about how each comment was addressed are
        to be maintained in project and administrative files for at
        least three years after the completion of a project or the
        release of products, or such other period of time as is judged
        to be reasonable.

    3.  Disclosure and attribution for peer reviews is encouraged within
        legally-specified protections.[^8] Specifically, to the extent
        to which information about a reviewer (name, credential,
        affiliation) will be disclosed along with his/her comments or
        analysis, the agency must comply with the requirements of the
        Privacy Act, 5 U.S.C. 552a, as amended, and OMB Circular A-130,
        Appendix I, 61 Fed. Reg. 6428 (February 20, 1996) to establish
        appropriate routine uses in a published Systems of Records
        Notice.

# PROCEDURES FOR DETERMINING WHETHER INFORMATION IS COVERED BY THE OMB BULLETIN

Effective compliance with OMB Bulletin depends on each OA developing
mechanisms that enable the comprehensive identification of scientific
information disseminations that are either of a highly influential or
influential nature.

7.  DOT OAs shall establish a formal process for reviewing research and
    program activities, projects, and programs to determine if they will
    result in the dissemination of HISAs or ISIs in the "foreseeable"
    future.

8.  DOT OAs shall institute guidance for classifying planned information
    disseminations that includes real world examples of what types of
    information and activities would be properly classified as HISAs or
    ISI.

9.  OA Guidance shall include examples of what types of information
    provided by the OA would be considered exempted or deferred under
    OMB requirements and the rationale for exempting or deferring them.
    These examples may come from prior experience or the experiences of
    other departments and agencies across government.

10. OA Guidance shall include information on how an administrative
    record will be maintained within the operating administration.

11. Each OA shall establish a committee to periodically assess whether
    the scientific information review process is working in a manner
    consistent with the intent of the OMB Guidance and DOT
    implementation procedures.

# PROCEDURES FOR MANAGING PEER REVIEWS OF PLANNED SCIENTIFIC INFORMATION DISSEMINATIONS

Administrators are responsible for ensuring proper implementation of
Departmental guidelines. The Information Quality Peer Review Bulletin
falls under the scope of Section 515/IQA giving the CIO additional
responsibility to post highly influential assessments and influential
scientific information, along with Peer Review Plans, on the Department
of Transportation website: [http://www.dot.gov](http://www.dot.gov/).
The process for identifying, reviewing and submitting scientific
information is the responsibility of each OA. The minimum requirements
for this process are as follows:

12. Each OA will designate a Peer Review Coordinator (PRC) to coordinate
    and monitor peer review activities across the OA. The PRC is the
    main contact for each OA on peer review matters. The
    responsibilities of the PRC include:

    1.  Overseeing the tracking of submissions (See Appendix D for
        template).

    2.  Reporting peer review activities to OA leadership.

    3.  Mediating compliance issues involving implementation of the OMB
        Bulletin.

    4.  Overseeing development of procedures pertaining to the peer
        review administrative record.

    5.  Attending all meetings convened by S-80/C-50[^9] pertaining to
        the implementation of the OMB Peer Review Bulletin.

    6.  Communicating changes and additions in peer review guidance to
        their respective OA management and staff.

    7.  Overseeing the peer review process to ensure effective and
        timely compliance with the OMB Bulletin.

    8.  Identifying and recommending enhancements in operating
        procedures to OA management and S-80/C-50.

    9.  Coordinating with Data Quality Contacts in other OAs.

    10. Provide advice, guidance, and support to the Peer Review Leader.

    11. Work with the OA to identify major scientific and technical work
        products as ISI and HISA peer review candidates for the coming
        year.

    12. Designating (in conjunction with the Project Managers (s) and/or
        the Contracting Officer Technical Representative (COTR) a Peer
        Review Leader (PRL) for each peer review. The responsibilities
        of the PRL include:

        a.  Keeping the PRC informed of the status of the peer review
            project and notifying the PRC that the peer review is
            completed.

        b.  Providing advice, guidance, and support in the preparation,
            implementation, and completion of the peer review.

        c.  Reviewing working and final drafts, statements of work,
            charge, and procurement requests.

        d.  Ensuring that sufficient funds are designated in the
            office's budget request to conduct the peer review; also
            ensuring that adequate resources and or/extramural
            management support are available for the peer review.

        e.  Assisting in establishing a realistic peer review schedule
            of key deadlines.

        f.  Designating the stage(s) of product development where peer
            review is appropriate.

        g.  Ensuring issues and comments raised by the peer reviewer(s)
            are adequately addressed and documented for the record, and
            where appropriate, incorporated into the work product that
            is used as basis for decision-making.

        h.  Identifying and providing a rationale for the administrative
            record and the annual report of each decision not to peer
            review particular information dissemination.

        i.  Ensuring that peer review agenda submissions to the DOT-CIO
            are up-to-dated at least every six months.

        j.  Helping to identify peer reviewers in consultation with
            researchers and COTRs in accordance with guidance provided
            in the OMB Bulletin.

        k.  Advising peer reviewers of their responsibilities.

        l.  Notifying the peer reviewers, information submitters, and
            other interested parties that information regarding a peer
            review is subject to release under the Freedom of
            Information Act (FOIA), unless the peer review information
            meets the criteria for an exemption under the FOIA.

        m.  Providing information to the PRC and others in the
            concurrence chain regarding the peer review process and
            results, including the charge, profile of the peer
            reviewers, the peer review comments, and a proposal on how
            to address the comments***.***

        n.  Submitting the results to the DOT***-***CIO for submission
            in the annual peer review report.

        o.  Maintaining the OA-level peer review record in a manner
            consistent with OMB guidelines and Departmental procedures
            regarding the development and maintenance of the
            administrative record.

        p.  Ensure that all documents provided to Non-Federal peer
            reviewers are screened for information that could be claimed
            as Confidential Business Information (CBI).

13. The procedures for processing peer review submission (agenda) and
    peer review plans are as follows:

    1.  Peer review agenda entries should be tracked. Note: Appendix D
        provides an example of a tracking sheet that can be adapted for
        OA use.

    2.  Peer review plans must include all elements called for in
        Paragraph V. (2) of the OMB peer review Bulletin. See Appendix
        E\-\--Peer Review Plan Template.

    3.  Each DOT organization preparing a peer review agenda entry or
        peer review plan must, after completing internal coordination
        required by its respective operating administration, transmit
        the document to S-80 and C-50 in the Office of the Secretary of
        Transportation for review.

    4.  After all S-80/C-50 comments are addressed by the OA, S-80 will
        post the agenda entry and plan on other DOT peer review web
        page.

    5.  DOT's peer review web page will be updated every six months and
        on an as-needed basis for highly influential scientific
        disseminations.

    6.  The official posting dates for the Peer Review Agenda for highly
        influential information is June 16th.

    7.  The official posting date for the Peer Review Agenda for
        influential scientific information is December 16th.

    8.  Material intended for each June's agenda setting should be
        transmitted to S-80/C-50 by April 15, and for the December
        posting date, by October 15^th^.

    9.  Beginning with the December 2005 agenda, all influential, as
        well as highly influential projects must be reflected in the
        agenda.

# PROCEDURES FOR ESTABLISHING AND MAINTAINING AN ADMINISTRATIVE RECORD

Each OA shall document peer review activities for both influential
scientific information and highly influential scientific assessments
through the DOT Docket Management System that is currently used to
support DOT regulatory and adjudicatory actions.

14. Whenever a project that is likely to produce highly influential or
    influential scientific information becomes active, or when there is
    a document ready for peer review and the peer review process is
    beginning, the OA PRC shall establish an official docket[^10] within
    the DMS. The following types of information will be included in each
    docket:

    1.  Draft work product of highly influential or influential
        scientific information submitted for peer review.

    2.  Materials and information, including the charge, given to the
        peer reviewer(s).

    3.  Written comments, information, and materials received from the
        peer reviewer(s) and the public.

    4.  Information about the peer reviewers (names, affiliations,
        statement concerning potential conflicts and their resolutions).

    5.  Logistical information about the conduct of the peer review
        (times and locations of meetings, etc.).

    6.  A memorandum or written record responding to the peer review and
        public comments specifying acceptance or, where thought
        appropriate, rebuttal and non-acceptance.

    7.  The peer review record is not complete until it contains a copy
        of the final work product that addresses the peer review
        comments.

    8.  If the OA relies on influential scientific information or a
        highly influential scientific assessment subject to the
        requirements of the OMB Bulletin in support of a regulatory
        action, the OA shall include in the administrative record for
        that action documentation that explains how the agency has
        complied with the requirements of the Bulletin and the
        Information Quality Act. Relevant materials will be placed in
        the administrative record.

15. Once a docket is established by the OA through the PRC, it will be
    used to:

    1.  Post results of peer review activities.

    2.  Post public comments on peer review plans.

    3.  Post public comments on proposed disseminations of highly
        influential and influential scientific information.

    4.  Track peer review activities.

    5.  Track OA responses to peer review comments.

16. It is the responsibility of each OA PRC to inform peer reviewers and
    the public how the DMS operates and how it will be used to
    facilitate the peer review process from beginning to end.

# PROCEDURES FOR CONDUCTING PEER REVIEWS

Peer review is used to ensure the quality of published information meets
the standards of scientific and technical community. Peer review
typically evaluates the clarity of hypotheses, the validity or research
design, the quality of data collection procedures, the robustness of the
methods employed, the appropriateness of the methods for the hypotheses
being tested, the extent to which conclusions follow from the analysis,
and the strength and limitations of the overall product.

17. Procedures that apply to peer reviews of both highly influential and
    influential scientific information.

    1.  The first step in the peer review process is to develop the
        "charge" that contains the instructions to the peer reviewers
        regarding the objective of the peer review and the specific
        advice sought. Every peer review must begin with a specific
        written charge. This charge is to be developed prior to
        selecting reviewers.

        a.  Elements of the charge are:

            1.  Brief overview or introduction (describe what work
                product is, how it was developed, how it will be used).

            2.  Brief description or listing of any background materials
                provided to the peer reviewers.

            3.  The issues or questions to be addressed by the peer
                reviewers.

            4.  The due date of reviewer's comments.

            5.  The format of reviewer responses.

            6.  The point of contact in case peer reviewers have
                questions.

        b.  In cases where the OA selects a contractor to perform peer
            review, the OA cannot submit the charge directly to the peer
            reviewers under contracting regulations. The OA can provide
            the contractor with a list of questions that the OA wants
            included in the charge letter. The list of charge questions
            can be incorporated into the Statement of Work by the OA.
            Based on this list, the contactor prepares and submits the
            charge letter to the peer reviewers, after the OA is given
            the opportunity to review the charge letter.

18. Draft influential or highly influential information presented at
    scientific meetings or shared confidentially with colleagues for
    scientific input prior to or in tandem with a formal peer review
    shall include the disclaimer:

"THE FINDINGS AND CONCLUSIONS IN THIS REPORT (PRESENTATION) HAVE NOT
BEEN FORMALLY DISSEMINATED BY (THE OA) AND SHOULD NOT BE CONSTRUED TO
REPRESENT ANY OA \[or DEPARMENTAL\] DETERMINATION OR POLICY."

19. Procedures for the determining the scope of peer reviews for
    influential scientific information[.]{.underline}

    1.  For influential scientific information, the OA determines the
        nature and scope of the peer review.

    2.  Adequate peer reviews could include having the document reviewed
        by a referred scientific journal or a formal letter review by
        three or more reviewers outside the OA, but from another OA
        within DOT. The peer review might also involve convening a
        formal panel comprised of experts from other government
        agencies, universities, or the private sector to discuss and
        review the information.

    3.  For peer reviews of influential scientific information, the OA
        may select reviewers outside the OA but still within the DOT.

    4.  The OA shall disclose the names of the peer reviewers and their
        organizational affiliations in the report.

    5.  Reviewers shall be notified in advance regarding the extent of
        disclosure and attribution planned by the OA.

    6.  The final peer review report shall be posted on the DOT docket
        along with all materials related to the peer review (charge,
        peer review report, and OA response).

    7.  The peer review report shall be discussed in the preamble to any
        related rulemaking and included in the administrative record for
        any related OA action.

20. Procedures for determining the scope of peer reviews of highly
    influential scientific assessments

    1.  The OMB Bulletin establishes a more rigorous set of minimum
        review requirements for scientific information that falls into
        the category of "highly influential scientific assessments."
        These documents are typically highly complex, multidisciplinary,
        and often, they are novel or precedent setting. The peer review
        planning review process should begin at the start of the
        research project.

    2.  Peer reviews of highly influential scientific assessments should
        generally be conducted through a standing panel with a minimum
        of three members.

    3.  Special care must be given to select panel members that have
        requisite expertise to review the various disciplines reflected
        in the highly influential scientific assessment.

    4.  Reviewers must be independent of the DOT.

    5.  Special Government employees, such as academics appointed to
        advisory committees can serve as reviewers.

    6.  In those rare cases where expertise cannot be found outside the
        Department, an OA may use expertise that is found within the
        Department but outside the sponsoring OA.

    7.  The OA must document the reasons for selecting a reviewer from
        within the OA.

    8.  Repeated use of same reviewer (or reviewers) must be avoided
        unless it can be documented that the needed expertise cannot be
        found elsewhere.

    9.  OA shall rotate membership in standing peer review committees
        across the available pool of qualified reviewers to promote
        intellectual diversity.

    10. OAs conducting peer reviews shall provide reviewers with
        sufficient background information, including access to key
        studies, data and models, to perform their role as peer
        reviewers.

    11. Research grants awarded to a potential reviewer do not
        automatically disqualify the individual from conducting peer
        reviews.

    12. The OA must consider whether a potential reviewer's consulting
        and contractual relationships with the OA might raise issues of
        perceived or actual conflicts of interest.

21. Procedures for Preparing the Peer Review Report

    1.  Peer reviewers must prepare a report that includes the charge to
        the peer reviewers, describes the nature and scope of their
        reviews and their findings and conclusions. a verbatim copy of
        each reviewer's comments (with or without specific
        attributions).

    2.  The report should also document the process for selecting
        reviewers and why these reviewers were selected to conduct the
        peer review.

    3.  The OA shall prepare a written response to the peer review
        report explaining (a) the OA's agreement or disagreement with
        the views expressed in the report, (b) the actions the OA has
        undertaken or will undertake in response to the report, and (c)
        the reasons the OA believes those actions satisfy the key
        concerns stated in the report (if applicable).

    4.  The peer review report shall be discussed in the preamble to any
        related rulemaking and included in the administrative record for
        any OA action.

    5.  The DOT is required to disseminate the peer review report and
        the Department's response to the report on the DOT's website,
        including all materials related to the peer review such as the
        "charge" settlement, peer review report, and the OA response to
        the review.

    6.  The peer review report will include information on the slate of
        reviewers and the substance of their comments while avoiding
        specific attributions to particular reviewers.

#####  Responsibilities of Peer Reviewers

Peer reviewers shall be willing participants and have a level of
technical or scientific knowledge commensurate with that of the
scientific/technical dissemination developers. They need to read all
materials, participate fully, and protect confidential information that
arises. Peer reviewers must be fully informed in advance as to all
disclosure requirements of the peer review process.

Peer reviewers are to:

7.  Maintain the confidentiality of all products and deliberations.

8.  Perform reviews in a timely fashion.

9.  Maintain the highest standards of objectivity.

10. Rescue themselves in cases where unexpected conflicts of interest
    arise.

11. To assist OAs conduct peer reviews, the DOT Office of Chief
    Information Office (S-80) and Office of General Counsel (C-50) will
    develop and maintain a list of potential external and internal
    contacts organized in terms of technical expertise. This information
    will be maintained in strict accordance with Federal statutes and
    regulations governing privacy protections. Individuals on this list
    can be employed directly in the peer review process. Alternatively,
    they can identify other technical experts that may be more
    appropriate for a particular peer review project.

```{=html}
<!-- -->
```
22. Alternative Procedures

    1.  In addition to developing a peer review process in accordance
        with OMB requirements and DOT implementing guidance, DOT OAs
        have several alternatives at their disposal:

        a.  Rely on scientific information produced by the National
            Academy of Sciences (NAS).

        b.  Commission the NAS to conduct the peer review.

        c.  Employ an alternative procedure or set of procedures
            approved by the OIRA Administrator in consultation with the
            Office of Science and Technology Policy (OSTP).

# PROCEDURES FOR DEVELOPING PEER REVIEW PLANS

DOT OA's are required to begin a systematic process for peer review
planning for all influential and highly influential information that the
OA plans to disseminate in the "foreseeable" future.

23. A key component of this planning process is a web-accessible listing
    of forthcoming influential and highly influential disseminations
    that together comprise an OA "agenda."

Each agenda item shall include the following items:

1.  A preliminary title for the planned dissemination.

2.  A short paragraph describing the subject and purpose of the
    dissemination.

3.  An OA contact person.

4.  A prediction of whether the information dissemination will be of an
    influential or highly influential nature.

5.  A designation of the type of peer review that will be undertaken.

6.  A draft schedule for the peer review with the notation that the
    schedule is subject to change.

7.  Specific deferrals or waivers that have been applied for.

8.  Any OMB "sanctioned" alternative peer review procedures that will be
    used.

```{=html}
<!-- -->
```
24. Developing the "Charge"

> The "Charge" contains the instructions to the peer reviewers regarding
> the objectives of the peer review process. Each peer review request
> shall contain specific instructions to the peer reviewers regarding
> what is expected in terms of their review such as level of technical
> detail, etc. The "Charge" should be included in the Peer Review Plan.

25. Each Peer Review Plan shall include the following elements:

> (Please note that Appendix E provides a reporting template for the
> following categories of information)

######## Contents

a.  A paragraph including the title, subject, and purpose of the planned
    report.

b.  A statement as to whether the dissemination is likely to be
    influential scientific information or a highly influential
    scientific assessment

c.  Schedule (timing) for peer review including deferrals.

d.  Structure of review (panel or individual letters) or whether
    alternative procedure will be exercised.

e.  The "Charge" to peer reviewers.

f.  Whether there will be opportunities for public to comment on the
    work product to be peer reviewed, and if so, how and when will these
    opportunities will be provided.

g.  Whether OA will provide significant and relevant comments to peer
    reviewers before they conduct their peer review.

h.  Anticipated number of reviewers 3 or fewer, 4-10, or more than 10.

i.  Description of the primary disciplines or expertise needed for
    reviews.

j.  Whether reviewers will be selected by the OA or by designated
    outside organization.

k.  Whether the public, including scientific or professional societies,
    will be asked to nominate reviewers.

l.  OA contact to whom inquiries may be directed to learn the specifics
    of the plan.

m.  Links between plan and documents made public pursuant to the OMB
    Bulletin must be provided on the DOT web site along with public
    comment forms.

```{=html}
<!-- -->
```
1.  Schedule

    a.  For highly influential scientific disseminations, the peer
        review agenda should be updated as soon as a likely
        dissemination timeline is known.

    b.  OAs should update their peer review agenda at least every six
        months for influential scientific information.

    c.  It is strongly recommended that agendas be revised and schedules
        be updated as soon as additional information becomes available.

    d.  OAs shall consider public comments on their peer review plans.

    e.  OAs should provide a "list-serve" or similar mechanism to update
        the public via e-mail every time the peer review agenda has been
        updated.

# PROCEDURES FOR PUBLIC PARTICIPATION IN PEER REVIEWS

The OMB Bulletin requires that OAs shall, whenever feasible, provide for
public participation. Public participation in the peer review process
will vary in accordance with the type of scientific information to be
disseminated.

In some cases, it may be appropriate for the scientific peer review
process to precede an official dissemination so that the public receives
the most scientifically sound product. Other times, it may be
appropriate for an OA to make a draft assessment publicly available
(i.e., a request for comment) at the onset of the peer review process.
If an OA decides to release a product to the public prior to the
completion of the peer review, the following procedures shall be
followed:

26. A process shall be provided for the public to submit either written
    comments or to make an oral presentation before a peer reviewer
    panel, or both.

    1.  The public shall be provided information on how they can submit
        comments through the DOT DMS (See Section 6.0).

27. When written comments from the public are received, the OA shall
    ensure that peer reviewers receive copies of public comments that
    address significant scientific issues.

28. Peer reviewers shall be given sufficient time to fully consider all
    public comments.

29. To avoid undue delay of OA activities, time limits for public
    participation in the peer review process shall be established and
    communicated to the public.

30. OA may publish a "request for comment" or other notice in which
    public comment is solicited prior to a panel of peer reviewers
    commencing it's work.

#  PROCEDURES FOR AVOIDING CONFLICTS OF INTEREST

The NAS defines "conflicts of interest***"*** as any financial or other
interest that conflicts with the service of an individual on the review
panel because it could impair the individual's objectivity or could
create an unfair competitive advantage for a person or organization.
Department OAs will use the following procedures to avoid conflicts of
interest:

31. Reviewers of influential scientific information generally should not
    be employed by the OA or office producing the document.

32. Employees of the DOT, or negotiating, or seeking employment with the
    DOT, are not permitted to be reviewers of highly influential
    scientific assessments. However, special government employees are
    not precluded from participating on peer reviews nor are academics
    appointed to OA Advisory Committees or scientific review groups.

33. OAs shall make an effort to rotate peer responsibilities across the
    available pool of qualified reviewers.

34. OAs will make a special effort to examine potential reviewers'
    financial conflicts, including significant financial investments,
    consulting arrangements, employer affiliations, grants, and
    contracts. Conflicts of interest may be real or apparent.

35. Financial ties are to be scrutinized when the information is
    relevant to regulatory policy.

36. Inquiries into potential conflicts must go beyond financial
    investments and business relationships and shall include work as an
    expert witnesses, family relationships, consulting arrangements,
    honoraria, and sources of grants and contracts.

37. OAs shall consult Federal ethics requirements, applicable standards
    issued by the Office of Government Ethics, and the prevailing
    practices of the NAS.

38. Peer reviewers that are Federal Employees (including special
    government employees) are subject to Federal requirements governing
    conflicts of interest.

39. For non-Federal employees, the DOT shall adopt or adapt NAS policy
    for committee selection with respect to evaluating conflicts of
    interest.[^11]

40. Under some conditions, conflicts are unavoidable. In those special
    circumstances, the DOT OA will publicly disclose those conflicts.

41. The OA shall inform potential peer reviewers of any planned public
    disclosures (comment attributions) at the time they are recruited
    and obtain written permissions.

42. DOT OAs conducting or overseeing peer reviews must ensure that
    proper documentation is received from nominated peer reviewers
    regarding the presence or absence of perceived or actual conflicts
    of interest.

43. Reviewers must certify in writing that they will maintain the
    confidentiality of the review and associated materials and that they
    will not disclose to another individual any matter relating to the
    project.

44. [Conflict of Interest Waivers]{.underline} \-- If no other reviewer
    is available with the expertise necessary to ensure competent
    reviews, a waiver may be granted to allow participation by an
    individual that may have a perceived or actual conflict of interest.

    1.  [Influential Scientific Information.]{.underline} Waivers \-\--
        with documented explanations \-\--may be granted by an OA
        Administrator.

    2.  [Highly Influential scientific Disseminations.]{.underline}
        Waivers \-\-- with documented explanations \-\-- may be granted
        by the Secretary or Deputy Secretary.

#  PROCEDURES FOR ANNUAL REPORTING

The DOT CIO shall prepare an annual report that summarizes key decisions
made pursuant to the OMB Bulletin. The report, which is due December
15^th^ of each year, must include a summary of the peer reviews
conducted by the agency during the fiscal year. The report should
include the following items:

45. Number of peer reviews conducted subject to the Bulletin (i.e., for
    influential scientific information and highly influential scientific
    assessments)

46. Number of times alternative procedures were invoked.

47. Number of times waivers or deferrals that were invoked.

48. The length of time elapsed between the deferral and the peer review.

49. Decisions to appoint reviewers pursuant to any exception to the
    applicable independence or conflict of interest standards of the OMB
    Bulletin.

50. Number of peer review panels that were conducted in public and the
    number that allowed public comment.

51. The number of public comments provided on the OA's peer review
    plans.

52. The numbers of peer reviewers that the OA used that were recommended
    by professional societies.

# 

#  PROCEDURES FOR PEER REVIEWS SUPPORTING RULEMAKING

Although the OMB Bulletin does not require a peer review to be conducted
at a specific time during the regulatory rulemaking process, it does
state, "It is most useful to consult with peers early in the process of
producing information." It also notes that, when the information is a
critical component of rulemaking, it is important to complete the peer
review before the OA announces regulatory options.

OAs may decide that peer review should precede an opportunity for public
comment in situations where parallel "public participation in peer
review is an important aspect of obtaining a high-quality product."

If the OA relies on ISI or HISA that is subject to the OMB peer review
requirements, in support of a rulemaking, the OA shall provide a
certification in the administrative record that explains how the
requirements of the Bulletin were met. Relevant materials will be placed
in the administrative record.

#  BUDGETING FOR PEER REVIEW EXPENSES

The conduct of peer reviews will add administrative costs to scientific
disseminations. These costs of peer review include both the direct costs
of the peer review activity and those indirect costs stemming from the
potential delay in government and private actions that can result from
peer review. On the other hand, peer review, if performed fairly and
rigorously, can build consensus among stakeholders and reduce the
temptation for courts and legislators to second-guess or overturn OA
actions. Moreover, comprehensive stakeholder review can be an effective
mechanism for enhancing that rate at which scientific information and
technologies are deployed into the market place.

53. In all cases, OAs should use informal or formal cost-benefit methods
    to determine the appropriate scope and character of peer reviews. In
    this respect, the OMB Bulletin allows significant discretion
    regarding peer reviews involving influential scientific information.

54. Resources needed to implement the OMB specified peer review process
    are to be requested through the usual Departmental and OA budgetary
    processes. For purposes of budget planning, the costs of peer review
    include:

    1.  Additional FTE costs of staff dedicated to peer review process.

    2.  Contract or other costs associated with the use of outside peer
        reviews.

    3.  Administrative costs associated with conducting reviews
        (including travel expenses).

Note: Expenses pertaining to peer reviews can also be reduced by
effectively using tracking tools available to OAs, including the DMS
discussed Section 6.0. The DMS will enable OAs to track peer review
activities and engage the public without the burden of developing an
electronic peer review tracking system.

###   {#section-3 .unnumbered}

#  PEER REVIEW PROCESS DEFERRALS AND WAIVERS 

The OMB Bulletin allows for a deferral or waiver of peer review
requirements specified in the Bulletin where necessary.

[Influential Scientific Information]{.underline}: The OA Administrators
may waive or defer some or all of the peer review requirements for
influential scientific information if there exists a compelling
rationale. In such cases, the requesting program manager shall issue a
memo through the appropriate Associate Administrator (or the equivalent)
and Chief Counsel to the OA Administrator requesting a deferral or
waiver. This memo will include information on the necessity of the
deferral or waiver. When deferrals are granted by the OA Administrator,
peer reviews should be conducted as soon as practicable thereafter. An
estimated schedule for completing the peer review shall also be attached
to the memo requesting a deferral.

If approved by the OA Administrator, written notification (including the
planned completion schedule) shall be provided to the Director of OMB.

[Highly Influential Scientific Disseminations]{.underline}: For waivers
or deferrals involving highly influential scientific disseminations, the
OA Administrator shall issue a memo through the OST/CIO and OST
Assistant General Counsel for Regulation and Enforcement to the
Secretary (or Deputy Secretary) requesting a deferral or waiver. This
memo will include information on the necessity of the deferral or
waiver. If and when deferrals are granted, peer reviews should be
conducted as soon as practicable thereafter. An estimated schedule for
completing the peer review shall also be attached to the memo requesting
a deferral.

If approved by the Secretary or Deputy Secretary, written notification
(including the planned completion schedule) shall be provided to the
Director of OMB.

Waivers covering conflicts of interest are covered under Sections 10.14.

#  EXEMPTIONS TO THE PEER REVIEW PROCESS

> Types of information specifically exempted from peer review include:

55. Information that is related to national security, foreign affairs or
    negotiations involving international trade or treaties where
    compliance with the Bulletin would interfere with the need for
    secrecy or promptness.

56. Information that is part of individual OA adjudications or permit
    proceedings.

57. Time sensitive health or safety information.

58. Routine statistical information released by Federal statistical OAs.

59. Accounting budget, actuarial, or financial information generated by
    OAs that focuses on interest rates, banking currency, securities,
    commodities, futures, or taxes.

60. Information disseminated in connection with routine rules that
    materially alter entitlement, grants, user fees, or loan programs,
    or the right and obligations of recipients.

61. Licensing, approval, and registration processes for specific product
    development activities as well as site-specific activities.

62. Official dissemination resulting in the course of adjudications and
    permit proceedings, unless the OA decides that peer review is
    practical and appropriate.

63. Research disseminations resulting from DOT contracts or grants to
    universities that are clearly published under the imprimatur of the
    university receiving DOT funding for research.

64. Deferrals for other types of information are expected to be highly
    unusual limited to compelled situations where flexibility is
    necessitated.

65. Deadlines found in consent decrees agreed to by OAs after the
    Bulletin is issued.

66. Regulatory Impact Analysis (RIAs) or regulatory flexibility analysis
    reports because they are already reviewed through an interagency
    review process under E.O.12866 that involves application of the
    principles and methods defined in OMB Circular A-4, except for
    underlying data and analytical models used.

67. Derivative products (such as a product that only summarizes an
    already peer reviewed product or products such as fact sheets).

68. Compendiums of existing models, methods and/or technologies.

69. Minor intermediate products (i.e., a memorandum from a contractor)
    describing methods or results that will be incorporated in a larger
    product that will be peer reviewed.

70. Preliminary or incidental analyses prepared separately from the work
    product ultimately used to support an OA action or decision (i.e.,
    "what-if" analyses).

# APPENDIX A: KEY DEFINITIONS

#### Scientific dissemination

An OA initiated or sponsored distribution of scientific or technical
information to the public.

#### Scientific information

Factual inputs, data, models, analyses, technical information, or
scientific assessments based on the behavioral and social sciences,
public health and medical sciences, life and earth sciences,
engineering, or physical sciences.

#### Information quality act

Refers to Section 515 of Public Law 106-554 (Pub.L. No. 106-554, § 515,
112 Stat. 2763, 2763A-153-154 (2000))

#### Scientific assessment

Refers to an evaluation of a body of scientific or technical knowledge,
which typical synthesizes multiple factual inputs, data, models,
assumptions, and/or applies best professional judgment to bridge
uncertainties in the available information.

#### Highly influential scientific assessment

Highly influential scientific assessments are those "that the agency or
the OIRA Administrator determines could have a potential impact of more
than \$500 million in any one year on either the public or private
sector or that the dissemination is novel, controversial, precedent
setting, or has significant OA interest." A scientific assessment is an
"evaluation of a body of scientific or technical information that
typically synthesizes multiple factual inputs, data, models,
assumptions, and/or applies best professional judgment to bridge
uncertainties in the available information.***"*** Other criteria that
can be used to determine whether a scientific work product is highly
influential could include:

-   Information that focuses on significant emerging scientific
    technical, or regulatory issues.

-   Involves the application of an innovative approach for a previously
    defined problem/process/methodology.

-   Satisfies a statutory or other legal mandate for peer review.

-   Guidance for conducting economic or financial analysis.

-   New financial or economic methodologies that serve as the primary
    method or protocol for conducting economic analyses within a program
    or operating administration.

-   Unique, novel or previously un-validated applications of economic or
    financial methodologies.

-   Broad scale assessments of regulatory programs, such as the required
    by Congressional-mandates.

-   New preference surveys.

-   New national surveys of costs and expenditures for environmental
    protection.

-   Economic research plans developed to assess and advance the
    state-of-the-art science in economic theory.

-   New meta-analysis that re-analyzes existing published literature and
    supporting data.

-   New algorithms that will provide the basis of modeling and
    predicting traffic behavior.

-   Large scale data sources.

#### Influential scientific information

Scientific information the OA can reasonably determine "will have or
does have a clear and substantial impact on important public policies or
private sector decisions."

#### Scientific integrity

In the conte*x*t of peer review, scientific integrity refers to issues
as expertise and balance of the panel members, the identification of the
scientific issues and the clarity of the charge to the panel, the
quality, focus, depth of the discussion of the issues by the panel, the
rational and supportability of the panel's findings, and the accuracy
and clarity of the panel report.

#### Process integrity

The term applies to issues such as transparency and openness, avoidance
of real or perceived conflict of interest, a workable process for public
comment and involvement, and adherence to defined submission and
reporting procedures.

#### Conflict of interest

The National Academy of Sciences defines "conflicts of interest" as a
financial or other interest that conflicts with the service of an
individual on the review panel because it could impair the individual's
objectivity or could create an unfair competitive advantage for a person
or an organization.

#### Docket

A docket is an official public record. DOT publishes and stores on-line
information about proposed and final regulations, copies of public
comments on proposed rules, and related information in the Docket
Management System (DMS). The DOT uses this docketed material when making
regulatory and adjudicatory decisions, and makes docketed material
available for review by interested parties. Specific documents covering
the same issues are stored together in a docket. Therefore, a docket may
contain several documents that are all related to the same subject
matter.

[DOT Docket Management System (DMS)]{.underline}

DMS is an electronic, image-based database in which all DOT docketed
information is stored for easy research, and retrieval.

#### OIRA

Office of Information and Regulatory Affairs (OIRA) is located in the
Office of Management and Budget.

# APPENDIX B: LIST OF ACRONYMS AND ABBREVIATIONS

  ------------------------------- ---------------------------------------
  CIO                             Chief Information Officer

  COTR                            Contracting Officer's Technical
                                  Representative

  DMS                             Docket Management System

  DOT                             Department of Transportation

  IQA                             Information Quality Act (Data Quality
                                  Act)

  ISI                             Influential Scientific Information

  NAS                             National Academy of Sciences

  NIH                             National Institutes of Health

  HISA                            Highly Influential Scientific
                                  Assessments

  OA                              Operating Administration

  OIRA                            Administrator, Office of Information
                                  and Regulatory Affairs, Office of
                                  Management and Budget

  OMB                             Office of Management and Budget

  OSTP                            Office of Science and Technology Policy

  PRC                             Peer Review Coordinator

  PRL                             Peer Review Leader

  RIA                             Regulatory Impact Analysis
  ------------------------------- ---------------------------------------

# APPENDIX C: QUESTIONS AND ANSWERS REGARDING COMPLIANCE WITH OMB BULLETIN

#### What is peer review?

Peer review is intended to uncover any technical problems or unsolved
issues in a scientific work product through the use of technically
competent and independent (objective) experts. Peer review of a major
scientific work product that will have the imprimatur of the Federal
Government needs to be incorporated into the upfront planning of any
action based in the work product. This includes obtaining the proper
resources commitments (reviewers and money) and establishing realistic
schedules.

[What is a peer reviewer?]{.underline}

> Peer reviewers are individuals who have technical expertise in the
> subject matter of the product undergoing peer review. Peer reviewers
> are generally external to the operating administration in cases where
> influential and highly influential information is involved. For highly
> influential information peer reviewers may come from outside the
> Federal Government such as universities. Peer reviewers must be
> objective experts that have not been associated with the generation of
> the specific influential or highly influential work product either
> directly by substantial contribution to its development or indirectly
> by significant consultation during the development of the product.

[How is peer review different from public comment?]{.underline}

Peer review and public comment are often times mutually supportive and
the OMB Bulletin encourages public comment on both peer review plans and
highly influential and influential scientific information when feasible
and appropriate. However, peer review is a rigorous process through
which technical and scientific peers in a particular field (or fields)
collectively work to ensure that the standards of the scientific method
are adhered to during both research and publication phases.

[What is a peer review panel?]{.underline}

> A peer review panel can range from a few individuals to ten or more,
> depending on the nature of the product and the subject of the
> research. Individuals who serve as peer reviewers must always have
> appropriate scientific and technical expertise so that the complete
> range of issues presented in the peer review "charge" can be covered.

[How should peer review records that pertain to a rulemaking action be
handled?]{.underline}

The Peer Review Coordinator should work with the Department's CIO to
ensure that proper docketing requirements are satisfied for a peer
review of a work product supporting a new rule.

[What is the "Charge" to the peer reviewers?]{.underline}

> A key element of any effective peer review process is formulating a
> clear, focused charge that identifies recognized issues and invites
> comments or assistance. This request signals the OA's awareness of
> potential issues and its receptivity to expert recommendations. A
> "Charge" is important for the following reasons:

-   First, it focuses the review by presenting specific questions and
    concerns that the OA expects the peer reviewers to address.

-   Second, it invites general comments on the entire work product. The
    specific and general comments should focus mostly on the scientific
    and technical studies that have been applied in a sound manner.

[When and how often should peer review take place?]{.underline}

As a general rule, it is most usual to consult with peer reviewers early
in the research and development process. On large, complex research
projects it may be prudent to establish a multi-stage peer review
process as part of the overall peer review planning process.

[What content should be in the peer review record?]{.underline}

Items that should be part of the record include:

-   Draft work product of highly influential or influential scientific
    information submitted for peer review.

```{=html}
<!-- -->
```
-   Peer Review Plan

-   Materials and information, including the charge, given to the peer
    reviewers.

-   Peer Selection Process (methods/approaches used)

-   Peer reviewer names and biographies (subject to privacy regulations)
    statement concerning potential conflicts and their resolutions.

-   Peer Report

-   Public comments and responses to public comments

-   A written response to the peer review report explaining the OA's
    agreement or disagreement with peer comments and recommendations,
    the actions that OA has undertaken or will undertake in response to
    peer review comments, and (if applicable) the reasons the OA
    believes those actions satisfy key concerns or recommendations

-   Any deferrals and/or waivers and rationale for granting them

    -   Compliance with Privacy Act, U.S.C. 552A, as amended, and OMB
        Circular A-130, Appendix I, 61 Fed. Reg. 6428 (February 20,
        1996)

# APPENDIX D: SUMMARY TRACKING MATRIX USING FHWA SUBMISSIONS TO ACHIEVE COMPLIANCE TEMPLATE

  ----------------- ------------- ------------- ------------- --------------- ------------- -----------
  Project, products FHWA Office   Highly        Influential   Foreseeable     Completed     Date
  initiatives,                    Influential   Scientific    Dissemination   Peer Review   Submitted
  activities or                   Scientific    Information   Date            Plan Date     to HRTC
  programs                        Assessments                                               
  classified as                                                                             
  scientific                                                                                
  information                                                                               

  Conditions and    Office of     XXX                         January 31,     May 12, 2005  May 12,
  Performance       Legislation                               2007                          2005
  Report Analytical and Strategic                                                           
  Tools             Planning                                                                

  Non-Destructive   Office of     XXX                         2007            Not Ready for 5/12/2005
  Testing and       Bridge                                                    Peer Review   
  Evaluation for    Technology                                                              
  Structural Health                                                                         
  Monitoring                                                                                

  Impacts of        Office of     XXX                         The final       April 1, 2006 May 27,
  Climate           Natural and                               report is                     2005
  Variability and   Human                                     tentatively                   
  Change on         Environment                               scheduled to be               
  Transportation                                              completed in                  
  Systems and                                                 April 2007                    
  Infrastructure --                                                                         
  Gulf Coast Study                                                                          
  ----------------- ------------- ------------- ------------- --------------- ------------- -----------

# APPENDIX E: SUBMISSION FORM AND PEER REVIEW PLAN

EXAMPLE: NON-DESTRUCTIVE TESTING AND EVALUATION

FOR STRUCTURAL HEALTH MONITORING

FHWA OFFICE OF BRIDGE TECHNOLOGY

+---------------+-------------------------+----+---------+-------------+
| Department of | Name of Operating OA    | Na |         |             |
| T             |                         | me |         |             |
| ransportation | *Federal Highway        | of |         |             |
|               | Administration*         | P  |         |             |
|               |                         | ro |         |             |
|               |                         | gr |         |             |
|               |                         | am |         |             |
|               |                         | Of |         |             |
|               |                         | fi |         |             |
|               |                         | ce |         |             |
|               |                         |    |         |             |
|               |                         | Of |         |             |
|               |                         | fi |         |             |
|               |                         | ce |         |             |
|               |                         | of |         |             |
|               |                         | Br |         |             |
|               |                         | id |         |             |
|               |                         | ge |         |             |
|               |                         | Te |         |             |
|               |                         | ch |         |             |
|               |                         | no |         |             |
|               |                         | lo |         |             |
|               |                         | gy |         |             |
+---------------+-------------------------+----+---------+-------------+
| Preliminary   | Non-Destructive Testing |    |         |             |
| Title:        | and Evaluation for      |    |         |             |
|               | Structural Health       |    |         |             |
|               | Monitoring              |    |         |             |
+---------------+-------------------------+----+---------+-------------+
| Subject and   | Include the following   |    |         |             |
| Purpose of    | information:            |    |         |             |
| information   |                         |    |         |             |
| d             | a.  Topic - Bridge      |    |         |             |
| issemination: |     Monitoring          |    |         |             |
|               |                         |    |         |             |
|               | b.  Specific issues --  |    |         |             |
|               |     Health Condition    |    |         |             |
|               |     Monitoring          |    |         |             |
|               |                         |    |         |             |
|               | c.  Broad Goals -- Real |    |         |             |
|               |     time monitoring of  |    |         |             |
|               |     bridge condition,   |    |         |             |
|               |     bridge performance  |    |         |             |
|               |     and safety          |    |         |             |
|               |                         |    |         |             |
|               | d.  Potential Public    |    |         |             |
|               |     Benefits -- Safe    |    |         |             |
|               |     and satisfactory    |    |         |             |
|               |     performance of      |    |         |             |
|               |     bridges             |    |         |             |
+---------------+-------------------------+----+---------+-------------+
| Is this       | Highly Influential      |    | Check   |             |
| information   | Scientific Assessment?  |    | One box |             |
| dissemination |                         |    |         |             |
| likely to be: |                         |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | Influential Scientific  |    |         |             |
|               | Information?            |    |         |             |
+---------------+-------------------------+----+---------+-------------+
| Peer review   | What is the schedule    |    |         |             |
| Plan          | for peer review         |    |         |             |
|               | (including proposed OA  |    |         |             |
|               | deferrals /waivers on   |    |         |             |
|               | peer review             |    |         |             |
|               | requirements)?          |    |         |             |
|               |                         |    |         |             |
|               | *Anticipated in 2007*   |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | Will the review be      |    |         |             |
|               | conducted through a     |    |         |             |
|               | panel, individual       |    |         |             |
|               | letters, or an          |    |         |             |
|               | alternative procedure   |    |         |             |
|               | will be exercised       |    |         |             |
|               | (e.g., such as those    |    |         |             |
|               | used by National        |    |         |             |
|               | Academy of Sciences:    |    |         |             |
|               | see Bulletin, Section   |    |         |             |
|               | IV, Alternative         |    |         |             |
|               | Procedures, pg. 27)?    |    |         |             |
|               |                         |    |         |             |
|               | *Panel, workgroups,     |    |         |             |
|               | technical committees,   |    |         |             |
|               | industry, Government    |    |         |             |
|               | officials.*             |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | Will there be           |    |         |             |
|               | opportunities for the   |    |         |             |
|               | public to comment on    |    |         |             |
|               | the work product to be  |    |         |             |
|               | peer reviewed, and if   |    |         |             |
|               | so, how and when these  |    |         |             |
|               | opportunities will be   |    |         |             |
|               | provided?               |    |         |             |
|               |                         |    |         |             |
|               | *The development        |    |         |             |
|               | process includes        |    |         |             |
|               | representatives from    |    |         |             |
|               | private consultants,    |    |         |             |
|               | industry, and academia  |    |         |             |
|               | and govt officials.*    |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | Will the OA provide     |    |         |             |
|               | significant and         |    |         |             |
|               | relevant public         |    |         |             |
|               | comments to the peer    |    |         |             |
|               | reviewers before they   |    |         |             |
|               | conduct their review?   |    |         |             |
|               |                         |    |         |             |
|               | *Disseminate            |    |         |             |
|               | information/paper       |    |         |             |
|               | s/reports/presentations |    |         |             |
|               | at technical committee  |    |         |             |
|               | meetings, TRB and       |    |         |             |
|               | AASHTO conventions,     |    |         |             |
|               | workgroups*             |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | What is the anticipated |    |         |             |
|               | number of reviewers (3  |    |         |             |
|               | or fewer: 4-10; or more |    |         |             |
|               | than 10)?               |    |         |             |
|               |                         |    |         |             |
|               | *More than 10*          |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | What are the primary    |    |         |             |
|               | disciplines or          |    |         |             |
|               | expertise needed in the |    |         |             |
|               | review?                 |    |         |             |
|               |                         |    |         |             |
|               | *Structural,            |    |         |             |
|               | electrical, and         |    |         |             |
|               | materials engineers*    |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | Will reviewers be       |    |         |             |
|               | selected by the OA or   |    |         |             |
|               | by a designated outside |    |         |             |
|               | organization?           |    |         |             |
|               |                         |    |         |             |
|               | *Both*                  |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               | Will the public,        |    |         |             |
|               | including scientific or |    |         |             |
|               | professional societies, |    |         |             |
|               | be asked to nominate    |    |         |             |
|               | potential peer          |    |         |             |
|               | reviewers?              |    |         |             |
|               |                         |    |         |             |
|               | *All the above*         |    |         |             |
+---------------+-------------------------+----+---------+-------------+
| Peer          | (See 7.1.1)             |    |         |             |
|               |                         |    |         |             |
| Review Charge |                         |    |         |             |
+---------------+-------------------------+----+---------+-------------+
| Contact       | Name                    |    | E-mail: | Phone:      |
| Person for    |                         |    |         |             |
| inquiries     |                         |    |         |             |
+---------------+-------------------------+----+---------+-------------+
|               |                         |    |         |             |
+---------------+-------------------------+----+---------+-------------+

# APPENDIX F: SAMPLE CONFLICT OF INTEREST FORM 

Dear (Peer Reviewer \-- insert name):

You have been requested by (Agency) to serve as a Peer Reviewer
for\_\_\_\_\_\_ \_\_\_\_\_\_\_\_\_\_(*name of
project*)\_\_\_\_\_\_\_\_\_\_\_\_. Your participation in this review
will be greatly appreciated. However, it is possible that your personal
affiliations and involvement in particular activities could pose a
conflict of interest or create the appearance that you lack impartiality
in your involvement for this peer review. Although your involvement in
these activities is not necessarily grounds for exclusion from the peer
review, you should consult the contact named below or other appropriate
official to discuss these matters. Affiliations or activities that could
potentially lead to conflicts of interest might include:

a\) work or arrangements concerning future work in support of industries
or other parties that could potentially be affected by regulatory
developments or other actions based on material presented in the
document (or review materials) that you have been asked to review;

b\) your personal benefit (or benefit of your employer, spouse or
dependent child) from the developments or other actions based on the
document (or review materials) you have been asked to review;

c\) any previous involvement you have had with the development of the
document (or review materials) you have been asked to review,

d\) any financial interest held by you (or your employer, spouse or
dependent child) that could be affected by your participation in this
matter; and

e\) any financial relationship you have or have had with (Agency) such
as research grants or cooperative agreements.

Please contact \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ (name and contact
info for Agency peer review official or primary contractor) to discuss
any potential conflict of interest issues at your earliest convenience,
but no later than \_\_\_\_\_\_\_\_\_.

> \[Be sure to date and sign this inquiry\]

# APPENDIX G: PEER REVIEW CHECKLIST

  ---------- --------------------------------------- ---------------------------
  Number     Check List Question                     Response

  1\.        What is the title of work product?      

  2\.        Have you reviewed DOT Peer Review       \_\_\_yes.\_\_\_\_\_\_no.
             Procedures and the OMB Bulletin         
             covering peer reviews?                  

  3\.        Does the work product constitute *A     \_\_\_yes.\_\_\_\_\_\_no.
             Highly Influential Scientific           
             Dissemination*?                         

  4\.        Does the work product constitute        \_\_\_yes.\_\_\_\_\_\_no.
             *Influential Scientific Information*?   

  5\.        Is a peer review required?              \_\_\_yes.\_\_\_\_\_\_no.

  6\.        Has a peer review plan been developed?  \_\_\_yes.\_\_\_\_\_\_no.

  7\.        How much time will be needed to         
             complete the peer review?               

  8\.        Has your Operating Administration (OA)  \_\_\_yes.\_\_\_\_\_\_no.
             established a concurrence process for   
             peer reviews?                           

  9\.        Has a file for comments and other       \_\_\_yes.\_\_\_\_\_\_no.
             materials been created in the DOT       
             Dockets System (DMS)?                   

  10\.       When will the peer review been          
             completed?                              

  11\.       Are there court-ordered constraints or  \_\_\_yes.\_\_\_\_\_\_no.
             deadlines?                              

  12\.       Has there been a request for a          \_\_\_yes.\_\_\_\_\_\_no.
             deferral, waiver, or exemption?         

  12a.       Is documentation available to support a \_\_\_yes.\_\_\_\_\_\_no.
             deferral, waiver, or exemption?         

  12b.       Have proper approvals been filed for    \_\_\_yes.\_\_\_\_\_\_no.
             deferral, waiver or exemption?          

  12c.       Has deferral been granted?              \_\_\_yes.\_\_\_\_\_\_no.

  13\.       Are you using the DMS to track the peer \_\_\_yes.\_\_\_\_\_\_no.
             review process?                         

  14\.       Has senior OA management been informed  \_\_\_yes.\_\_\_\_\_\_no.
             of progress/problems?                   

  15\.       Who will coordinate peer review on OA   
             side?                                   

  16\.       Who will provide                        
             administrative/logistical support?      

  17\.       Have all conflict of interest forms     \_\_\_yes.\_\_\_\_\_\_no.
             been completed and privacy and other    
             requirements been met?                  

  18\.       What resources will be needed?          

  19\.       Will peer review involve experts from   \_\_\_yes.\_\_\_\_\_\_no.
             outside the OA?                         

  20\.       Will the peer review require experts    \_\_\_yes.\_\_\_\_\_\_no.
             from outside DOT?                       

  21\.       Who will be the *lead* peer reviewer?   \_\_\_yes.\_\_\_\_\_\_no.

  22\.       Who will maintain the administrative    
             record at OA level?                     

  23\.       Where will the administrative record be 
             kept at the OA level?                   

  24\.       What mechanism (e.g., formal standing   
             panel) for peer review will be used?    

  25\.       How long will the administrative record 
             at the OA level be kept?                

  26\.       Will there be a public involvement      \_\_\_yes.\_\_\_\_\_\_no.
             component to the peer review process?   

  27\.       How will comments from the public be    
             integrated with the peer review         
             process?                                

  28\.       Has the "charge" been developed for     \_\_\_yes.\_\_\_\_\_\_no.
             reviewers?                              

  29\.       What is the timing of public            \_\_\_yes.\_\_\_\_\_\_no.
             involvement?                            

  30\.       Have arrangements for interim and final \_\_\_yes.\_\_\_\_\_\_no.
             sign-offs been developed?               

  31\.       Have peer reviewers and the public been \_\_\_yes.\_\_\_\_\_\_no.
             instructed on how to use DMS?           

  32\.       How will the results of the peer review 
             be responded to (or incorporated) in    
             the final work product?                 

  33\.       What is the measure of success for the  
             peer review?                            

  34\.       What "lessons" or "best practices" can  
             be drawn from this peer review          
             experience?                             
  ---------- --------------------------------------- ---------------------------

[^1]: A complete list of definitions is provided in Appendix A and a
    list of Acronyms and Abbreviations is provided in Appendix B.

[^2]: OMB Bulletin titled, "Final Information Quality Bulletin for Peer
    Review." (December 16, 2004), p. 11.

[^3]: OMB Bulletin titled, "Final Information Quality Bulletin for Peer
    Review." (December 16, 2004), p. 23.

[^4]: DMS is an electronic, image-based database in which all DOT
    docketed information is stored for easy research and retrieval. See
    [http://www.dms.dot.gov]{.underline}

[^5]: The costs of peer review include both the direct costs of the peer
    review activity and those stemming from the potential delay in
    government and private actions that can result from peer review. On
    the other hand, peer review, if performed fairly and rigorously, can
    build consensus among stakeholders and reduce the temptation for
    courts and legislators to second-guess or overturn OA actions.
    Moreover, comprehensive stakeholder review can be an effective
    mechanism for enhancing that rate at which scientific information
    and technologies are deployed into the market.

[^6]: Balance refers to ensuring a broad and diverse representation of
    respected perspectives and intellectual traditions within the
    scientific community.

[^7]: Scientific integrity in the context of peer review, refers to the
    expertise and intellectual balance of the panel members; the
    identification of the scientific issues and clarity of the charge to
    the panel; the quality, focus, and depth of the discussion of the
    issues by the panel; the rationale and supportability of the panel's
    findings; and the accuracy and clarity of the panel report. Process
    integrity includes such issues as transparency and openness,
    avoidance of real or perceived conflicts of interest, a workable
    process for public comment and involvement, and adherence to defined
    procedures (OMB Bulletin, pg. 13).

[^8]: The OMB Bulletin strikes a compromise by disclosure of the
    identity of the reviewers, but not the public attribution of
    specific comments to specific peer reviewers.

[^9]: The Office of the Secretary (OST) Chief Information Officer and
    the OST Associate General Counsel are the primary contact
    organizations for Bulletin-related questions.

[^10]: A docket is an official public record. Specific documents
    covering the same issues are stored together in a docket. Therefore,
    a docket may contain several documents that are all related to the
    same subject matter. These two terms, docket and document, are used
    frequently throughout the DOT docket site. A docket is a folder that
    can contain several pages, which are the documents.

[^11]: National Academy of Sciences, "Policy and Procedures on Committee
    Composition and Balance and Conflicts of Interest for Committees
    Used in the Development of Reports," May 2003: Available at:
    http://www.nationalacademies.org/coi/index.html.