Audit of NRC’s

Process for Ensuring Integrity 

in Scientific Research 

OIG-15-A-08 

February 10, 2015 

 

All publicly available OIG reports (including this report) 

are accessible through NRC’s Web site at

http://www.nrc.gov/reading-rm/doc-collections/insp-gen
UNITED STATES

NUCLEAR REGULATORY COMMISSION

WASHINGTON, D.C. 20555-0001

OFFICE OF THE

INSPECTOR GENERAL

February 10, 2015

MEMORANDUM TO: Mark A. Satorius

Executive Director for Operations

FROM: Stephen D. Dingbaum /RA/

Assistant Inspector General for Audits

SUBJECT: AUDIT OF NRC’S PROCESS FOR ENSURING INTEGRITY 

IN SCIENTIFIC RESEARCH (OIG-15-A-08)

Attached is the Office of the Inspector General’s (OIG) audit report titled Audit of NRC’s 

Process for Ensuring Integrity in Scientific Research.

The report presents the results of the subject audit. Following the January 23, 2015, exit 

conference, agency staff indicated that they had no formal comments for inclusion in this 

report.

Please provide information on actions taken or planned on each of the recommendations 

within 30 days of the date of this memorandum. Actions taken or planned are subject to OIG 

followup as stated in Management Directive 6.1.

We appreciate the cooperation extended to us by members of your staff during the audit. If 

you have any questions or comments about our report, please contact me at (301) 415-5915 

or Sherri Miotla, Team Leader, at (301) 415-5914.

Attachment: As stated
Office of the Inspector General

U.S. Nuclear Regulatory Commission

Defense Nuclear Facilities Safety Board

Results in Brief

Audit of NRC’s Process for Ensuring Integrity in 

Scientific Research

What We Found

Although NRC has established an Information Quality Program that contains the 

administrative mechanisms required by the Office of Management and Budget

(OMB), there is room for improvement. NRC’s Information Quality Program is 

ineffective because of a lack of program management and communication 

regarding program requirements to involved staff and management. As a result, 

NRC’s commitment to public participation and transparency is compromised. 

Moreover, NRC is at risk of providing inaccurate information to OMB and 

Congress regarding a Federally mandated program.

What We Recommend

We recommend that the Executive Director for Operations take steps to 

strengthen the agency’s Information Quality Program, including 

designating an Information Quality Program owner and clearly delineating

roles and responsibilities among involved offices; assigning responsibility 

for routinely checking, responding to, and tracking the resolution of 

information correction requests submitted via vehicles identified on NRC’s 

public Web pages regarding Information Quality Guidelines; holding an 

annual meeting with involved NRC staff and management to discuss the 

requirements and responsibilities associated with the identification and 

reporting of Influential Scientific Information and Highly Influential 

Scientific Assessment information products; developing a schedule to 

ensure that the Office of Nuclear Regulatory Research’s internal office 

instructions are regularly reviewed and revised, as appropriate; and 

directing all offices to review and revise office guidance on peer review to 

align with requirements established by OMB and the National Academy of 

Sciences. 

Why We Did This Review

This was a planned audit as 

noted in the 2014 OIG Annual 

Plan. This audit aligned with 

NRC’s identified management 

challenges regarding managing 

regulatory processes to meet a 

changing environment in the 

oversight of nuclear materials 

and facilities. 

The audit objective was to 

determine whether the 

Nuclear Regulatory 

Commission (NRC) has controls 

in place to assure that scientific 

research is objective, credible, 

and transparent. As such, we 

examined NRC’s Information 

Quality Program and peer 

review guidance to determine 

whether NRC had followed 

applicable Federal 

requirements when 

establishing the program and 

instituting guidance. We also 

assessed the oversight of NRC’s 

Information Quality Program to 

determine if there is consistent 

understanding and 

communication of the 

associated roles and

responsibilities assigned to 

involved management and 

staff.

OIG-15-A-08

February 10, 2015
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

ABBREVIATIONS AND ACRONYNMS........................................................ i

I. BACKGROUND ................................................................................1

II. OBJECTIVE......................................................................................5

III. FINDINGS.........................................................................................5

A. NRC’s Information Quality Program Is Ineffective .................5

B. NRC Has Not Adopted OMB Guidelines on Peer Review....16

IV. SUMMARY/CONCLUSION....................................................21

V. CONSOLIDATED LIST OF RECOMMENDATIONS.......................22

VI. AGENCY COMMENTS...................................................................23

APPENDIXES 

A. OBJECTIVE, SCOPE, AND METHODOLODY...............................24

B. RULES, ANALYSES, AND STUDIES REVIEWED BY OIG ..........26

TO REPORT FRAUD, WASTE, OR ABUSE.............................................27

COMMENTS AND SUGGESTIONS..........................................................27

 TABLE OF CONTENTS
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

i

MD Management Directive

NRC U.S. Nuclear Regulatory Commission

OMB Office of Management and Budget

RES Office of Nuclear Regulatory Research 

 ABBREVIATIONS AND ACRONYMS
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

1

Scientific Integrity in the Federal Government 

The Federal Government disseminates a variety of scientific information, 

including statistical information; information about health, safety, and 

environmental risks; and technical information it creates or obtains in the 

course of developing regulations. Scientific information is based on 

scientific research, analyses, and data performed to support the agency’s 

work.

Scientific information is defined as factual inputs, data, models, analyses, 

technical information, or scientific assessments related to such disciplines 

as the behavioral and social sciences, public health and medical sciences, 

life and earth sciences, engineering, or physical sciences. Often, 

regulations are based on scientific, engineering, and economic analyses. 

Therefore, it is crucial that information disseminated by Federal agencies 

be objective, and have utility, quality, and integrity. 

Requirements for Federal Agencies 

To ensure information integrity, Federal agencies are required to adopt 

standards for information quality. These standards are set by the Office of 

Management and Budget (OMB). OMB requires agencies to

• Designate influential1 information.

• Implement an administrative mechanism allowing affected persons to 

seek and obtain correction of information disseminated by the agency.

 1 "Influential," when used in the phrase "influential scientific or statistical information," means the agency 

expects that information in the form of analytical results will likely have an important effect on the 

development of domestic or international government or private sector policies or will likely have 

important consequences for specific technologies, substances, products, or firms.

 I. BACKGROUND
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

2

• Provide a publicly available annual report that provides information on 

information correction requests, designated influential information, and 

peer reviews.

• Follow peer review requirements for designated influential scientific 

information. 

Information Integrity at the U.S. Nuclear Regulatory Commission 

(NRC)

NRC has an Information Quality Program that involves many offices, 

including the Office of Information Services, the Office of Nuclear 

Regulatory Research (RES), and the Office of the Executive Director for 

Operations. These three offices have a central role in implementing 

NRC’s Information Quality Program. Additionally, other offices support the 

Office of Information Services, RES, and the Office of the Executive 

Director for Operations in ensuring information integrity. These offices 

include the Office of Nuclear Reactor Regulation, the Office of Nuclear

Material Safety and Safeguards, the Office of New Reactors, and the 

Office of Nuclear Security and Incident Response.

Areas of NRC Research 

NRC’s regulatory research program addresses issues in nuclear reactors, 

nuclear materials, and radioactive waste. RES is a technical support 

office that supplies technical tools, analytical models, analyses, 

experimental data, and technical guidance to support NRC’s regulatory 

programs and decisions. See Table 1 for examples of NRC research 

areas. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

3

Table 1: Examples of NRC Research Areas

Area of Research Description

Nuclear Reactor Safety 

Research

Research in this area includes reactor fuel behavior, plant aging, 

digital instrument and control, and operational data assessment.

Radiation Protection The radiation protection research program collects, analyzes, and 

disseminates information on occupational exposures reported to 

NRC by licensees.

Radiation Safety Research Research in this area includes a state-of-the-art study on cancer 

risk for populations surrounding NRC-licensed facilities and 

estimates of the offsite radiological health consequences for 

severe nuclear power reactor accidents.

Waste Safety Research Research in this area includes spent nuclear fuel and assessing 

decommissioning and waste management options.

Computer Codes NRC uses computer codes to model and evaluate fuel behavior, 

reactor kinetics, thermal-hydraulic conditions, severe accident 

progression, time-dependent dose for design-basis accidents, 

emergency preparedness and response, health effects, and 

radionuclide transport during various operating and postulated 

accident conditions.

Source: NRC. 

Peer Review 

Peer review is an important procedure used by the scientific community to 

ensure the quality of published information and increase the quality and

credibility of the scientific information generated across the Federal 

Government. Peer review is a form of deliberation involving an exchange 

of judgments about the appropriateness of methods and the strength of 

the author’s inferences. It involves the review of a draft product for quality 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

4

by specialists in the field who were not involved in producing the draft. 

NRC conducts peer review of some of its scientific products. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

5

 III. FINDINGS

The audit objective was to determine whether NRC has controls in place 

to assure that scientific research is objective, credible, and transparent. 

Appendix A contains information on the audit scope and methodology. 

While NRC has controls in place, there is room for improvement. 

Specifically, NRC needs to strengthen its Information Quality Program and 

adopt OMB guidelines on peer review. 

A. NRC’s Information Quality Program Is Ineffective 

Although NRC has established an Information Quality Program that 

contains the administrative mechanisms required by OMB, the program is 

largely ineffective. NRC’s Information Quality Program is ineffective 

because of a lack of program management and communication regarding 

program requirements to involved staff and management. As a result, 

NRC’s commitment to public participation and transparency is 

compromised. Moreover, NRC is at risk of providing inaccurate 

information to OMB and Congress regarding a federally mandated 

program. 

Federal Guidance 

In October 2001, OMB issued the Guidelines for Ensuring and 

What Is Required

 II. OBJECTIVE
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

6

Maximizing the Quality, Objectivity, Utility, and Integrity of Information 

Disseminated by Federal Agencies (2001 OMB Bulletin). These 

guidelines provide policy and procedural guidance to Federal agencies for 

ensuring and maximizing the quality, objectivity, utility, and integrity of 

information they disseminate. Within 1 year after the guidelines’ issuance, 

Federal agencies were required to implement their own guidelines that 

include “administrative mechanisms allowing affected persons to seek and 

obtain correction of information maintained and disseminated by the 

agency.” Pursuant to the guidelines, agencies are also required 

periodically to report to OMB the number and nature of complaints 

received 

regarding the 

accuracy of 

information 

disseminated by 

the agency and 

how the 

complaints were 

handled. 

Agencies are

also required to 

provide an 

annual report on the number of information products that met the criteria 

for Influential Scientific Information or Highly Influential Scientific 

Assessment. Table 2 contains the definitions of Influential Scientific 

Information and Highly Influential Scientific Assessment. Information 

product refers to any book, paper, map, machine-readable material, 

audiovisual production, or other documentary material, regardless of 

physical form or characteristic, an agency disseminates to the public. This 

definition includes any electronic document, CD-ROM, or Web page. 

Source: NRC. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

7

Table 2: Definitions of Influential Scientific Information and Highly 

Influential Scientific Assessment 

Term Definition

Influential Scientific 

Information 

Influential scientific information is information that an agency 

reasonably can determine will have or does have a clear and 

substantial impact on important public policies or the private 

sector. Influential information, as defined in the NRC 

Information Quality Guidelines, is information that forms the 

technical basis for a substantive rulemaking that has 

substantial impact on an industry.

Highly Influential Scientific 

Assessment 

A highly influential scientific assessment is utilized as the basis 

of a rulemaking or regulatory action that NRC determines 

could have a potential impact of more than $500 million in any 

single year on either the public or private sector or that the 

action represents a novel, controversial, or precedent setting 

approach, or has significant interagency interest. These 

assessments are considered a subset of Influential Scientific 

Information.

Source: NRC Management Directive 3.17. 

Agency Guidance

In 2009, NRC implemented Management Directive (MD) 3.17, NRC 

Information Quality Program, which established policy and quality 

standards for information disseminated by the agency and delineated 

organizational roles and responsibilities among offices. In accordance 

with the 2001 OMB Bulletin, MD 3.17 specifically addresses the

administrative process for handling and reporting on information correction 

requests and provides instructions for reviewing information dissemination 

products to determine if they meet the criteria for Influential Scientific 

Information or Highly Influential Scientific Assessment. MD 3.17 provides 

guidance for processing information correction requests to ensure an 

opportunity for public involvement in the agency’s Information Quality 

Program. Specific roles and responsibilities for overseeing the processing 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

8

of information correction requests are identified within MD 3.17 and are 

specific to designated individuals; however, primary accountability for 

ensuring the effectiveness of information correction requests lies with the 

Office of Information Services. It is important that information correction 

requests receive appropriate consideration in a timely manner. For 

example, MD 3.17 states, “The Information Quality Coordinator must 

review the information correction request within five days of receipt, and 

forward it to the responsible NRC office.” MD 3.17 also addresses the 

review of NRC information to identify Influential Scientific Information and 

Highly Influential Scientific Assessments and assigns specific roles and 

responsibilities to the Office of the Executive Director for Operations, the 

Office of Information Services, RES, and other offices. It also identifies the 

criteria for designating information as Influential Scientific Information or a 

Highly Influential Scientific Assessment.

To ensure that the agency’s information products are appropriately 

reviewed and identified as Influential Scientific Information or a Highly 

Influential Scientific Assessment, it is imperative that involved offices fulfill 

their assigned roles and responsibilities. It is also important that the 

criteria associated with Influential Scientific Information and Highly 

Influential Scientific Assessment are consistently understood and applied 

by involved staff and management.

NRC has established an Information Quality Program governed by MD 

3.17. Although the agency’s program meets OMB requirements, it is 

largely ineffective. 

Specifically, the public’s ability to seek correction to agency information or 

access annual reports on the resolution of previously submitted 

information correction requests is limited. Additionally, agency staff and 

management are not consistently reviewing information products against 

the Influential Scientific Information and Highly Influential Scientific 

Assessment criteria. 

What We Found
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

9

Information Correction Requests

As it is currently administered, NRC’s Information Quality Program does 

not ensure effective or efficient processing of information correction 

requests in accordance with OMB and NRC requirements. The 

administrative mechanisms identified by NRC on the public Web page 

titled “How to Submit a Request for Correction” include mail, fax, e-mail, 

and Internet submission. Beginning in August 2014, OIG submitted 

multiple inquiries, following the agency’s instructions using each of the 

identified mechanisms, to test their functionality and determine whether 

the agency followed the 

identified process for 

reviewing each 

submission. Three of the 

four mechanisms were 

functional and the 

inquiries were 

successfully sent; 

however, the fax number 

was invalid. To date,

NRC has not responded 

to any of the three 

successfully sent 

inquiries, thereby significantly exceeding the designated 5-day timeliness 

metric in MD 3.17. Program management agreed that there is no 

consistent monitoring of incoming information correction requests to 

ensure each is acknowledged and reviewed in a timely manner even 

though MD 3.17 assigns this responsibility to an Information Quality 

Coordinator.

To comply with Federal requirements, NRC has developed two public 

Web pages dedicated to making information publicly available on the 

processing of individual information correction requests. The Web page 

titled “Information Quality: Information Quality Correction Request, 

Responses, and Appeals”2 is intended to provide information on each 

information correction request received, the agency's formal response(s), 

and any communications regarding appeals. However, the information 

contained on the Web page is incomplete as there is no reference to years 

 2 http://www.nrc.gov/public-involve/info-quality/req-resp-appls.html

Source: NRC. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

10

2009 through 2014. In fact, this Web page was last reviewed/updated in 

March 2012. The Web page titled “Information Quality: Annual Reports”3

is intended to list annual reports submitted by NRC to OMB regarding 

information correction requests received and processed under the 2001 

OMB Bulletin. The information contained on this Web page is also 

incomplete because there is no annual report listed for 2012.

Essentially, NRC is noncompliant with both OMB and its own 

requirements pertaining to facilitating public participation in the agency’s 

Information Quality Program. In its current state, NRC’s Information 

Quality Program does not provide the public with reliable administrative 

mechanisms to seek correction of information maintained by NRC or 

obtain information on the resolution of all information correction requests 

for a given year. 

Review and Identification of Influential Scientific Information/Highly

Influential Scientific Assessment 

In accordance with OMB requirements to report Influential Scientific 

Information and Highly Influential Scientific Assessments annually, the 

Office of Information Services sends out a data call to involved program

offices to determine if any information products produced each year are

identified as Influential Scientific 

Information or Highly Influential 

Scientific Assessments. Information 

collected through the data call 

should be collated by the Office of 

Information Services and forwarded 

to OMB in a report. Both OMB and 

NRC officials corroborated that 

NRC has never reported producing 

an information product that met the 

criteria of Influential Scientific 

Information or Highly Influential 

Scientific Assessment since the 

OMB requirement was enacted in 

2004. It is common for NRC staff 

and management to automatically 

respond to the annual data call that 

 3 http://www.nrc.gov/public-involve/info quality/annual-reports.html

Source: NRC. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

11

their respective office has not produced any Influential Scientific 

Information or Highly Influential Scientific Assessments without 

consistently reviewing each information product against the identified 

criteria. An Office of Information Services official stated that some NRC 

offices base their response on the prior year’s response, which, to date, 

have always been negative. As such, NRC has consistently reported that 

it does not produce information that is subject to OMB’s Bulletin.

In some instances, staff and management do perform the required 

Influential Scientific Information and Highly Influential Scientific 

Assessment review but may be misapplying the criteria. For example, 

although the criteria are not mutually exclusive, staff and management 

apply them as such. Specifically, some staff and management have 

based their determination that an information product does not meet the 

Highly Influential Scientific Assessment criteria simply based on the fact 

that its impact is not judged to be more than $500 million in any single 

year. Meanwhile, equal consideration is not given to the remaining criteria 

that states, “[T]he action represents a novel, controversial, or precedent 

setting approach, or has significant interagency interest.” 

NRC further defines Influential Scientific Information as any “information 

that forms the technical basis for a substantive rulemaking that has 

substantial impact on an industry.” Given that NRC is a science based 

agency that has initiated several rulemakings and participated in or 

sponsored significant research studies that have impacted its 

stakeholders, it is questionable to think that none of the resulting 

information products met Influential Scientific Information or Highly 

Influential Scientific Assessment criteria. This occurrence has been noted 

both by OMB and by NRC senior management who agree with OIG’s 

assertion that it is likely a result of a misunderstanding and/or 

misapplication of the Influential Scientific Information and Highly Influential 

Scientific Assessment criteria.

NRC’s Information Quality Program does not provide adequate assurance 

that NRC information products are being appropriately reviewed to 

determine if they meet Influential Scientific Information or Highly Influential 

Scientific Assessment criteria. MD 3.17 defines Influential Scientific 

Information and Highly Influential Scientific Assessment and the 

associated criteria. It also assigns involved staff and management 

specific roles and responsibilities for the review and identification of NRC 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

12

information products. However, staff and management do not consistently 

perform the required reviews or correctly apply the associated criteria to 

determine if an NRC information product should be designated and 

reported as Influential Scientific Information or as a Highly Influential 

Scientific Assessment.

Although the agency has met Federal requirements to establish an 

Information Quality Program, it is largely ineffective because of a lack of 

program oversight. OIG concludes that staff and management view the 

program as a formality without a clear purpose. This conclusion has been 

corroborated by involved 

management who have 

referred to the program “as 

decorative,” while the 

associated activities are 

viewed as an 

“administrative 

annoyance,” “not mission 

critical” and “not important.” 

Consequently, information 

correction requests go 

unanswered and information products are not being consistently reviewed 

and identified as Influential Scientific Information or as Highly Influential 

Scientific Assessments, thereby signaling the ineffectiveness of the 

agency’s Information Quality Program. NRC’s Information Quality 

Program is ineffective because of a lack of program management and 

communication to involved staff and management regarding program 

requirements. This is demonstrated by the inherent confusion among staff 

and management in the Office of the Executive Director for Operations, 

the Office of Information Services, and RES regarding aspects of the 

agency’s Information Quality Program, including respective roles and 

responsibilities. For example, staff and management in the Office of 

Information Services stated that it was unclear as to who has responsibility 

for routinely monitoring the receipt of and response to information 

correction requests. There is also confusion among involved offices 

regarding assigned and perceived roles and responsibilities. For example, 

Office of Information Services’ management sees its role as a “liaison” that 

Why This Occurred

Source: NRC. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

13

collects, collates, and disseminates information but does not have a 

responsibility to designate information products as Influential Scientific 

Information or Highly Influential Scientific Assessment. In contrast, RES 

does not see itself as having any “special role in the Information Quality 

Program,” but rather views the Office of Information Services as the lead 

for designating information products as Influential Scientific Information or 

Highly Influential Scientific Assessment. As such, it is evident that the 

staff and management in the involved offices are unclear on their 

respective roles and responsibilities despite the guidance provided in MD

3.17. This includes a shared obligation among staff and management in 

implementing MD 3.17. 

A general lack of communication among staff and management on the 

requirements, assigned roles, and associated responsibilities of the 

Information Quality Program has also contributed to its ineffective 

implementation. This was confirmed when OIG contacted involved staff 

and management to determine (1) if they had evaluated their respective 

information product against OMB and NRC guidance and 

(2) the basis for deciding if it met Influential Scientific Information or Highly 

Influential Scientific Assessment criteria. OIG contacted staff and 

management involved in key NRC rules, research activities, analyses, and 

studies.

4

 Table 3 lists those selected by OIG. Half of the staff and 

management contacted did not evaluate their respective information 

product against OMB/NRC guidance to determine if it met Influential 

Scientific Information or Highly Influential Scientific Assessment criteria. 

Of the information products that staff evaluated, staff determined that none 

met the Influential Scientific Information or Highly Influential Scientific 

Assessment criteria based either on a misapplication or misunderstanding 

of OMB and NRC guidance. For example, reasons for not identifying an 

information product included that it was ordered by the Commission, had a 

“quick turnaround,” or “did not result in a regulatory action.” Furthermore, 

at least one of these products would have reasonably met either the 

Influential Scientific Information or Highly Influential Scientific Assessment 

criteria.

 4 See Appendix B for description of methodology OIG used to select key research activities including 

analyses, reports, and rulemakings.
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

14

Table 3: Sample of NRC Rules, Research Activities, Analyses, and Studies5

Activity Evaluation 

Against OMB & 

NRC Criteria

Criteria Met Agency Reasoning 

2009 Security Rule No No Orders were issued prior to 

OMB Bulletin6; Orders were 

classified 

10 CFR Part 37 Yes No Did not meet monetary threshold 

of Highly Influential Scientific 

Assessment criteria

Continued Storage 

Rule

Yes No Did not meet monetary threshold 

of Highly Influential Scientific 

Assessment criteria

Fatigue 

Management Rule

No N/A Update occurred prior to OMB 

Bulletin7

SOARCA Yes No Did not meet monetary threshold 

of Highly Influential Scientific 

Assessment criteria

Cancer Study Yes No “Study is primarily for public 

confidence reasons and not 

driven by rulemaking”

Fukushima Task 

Force Report

No N/A A “unique product” that was 

Commission ordered; quick 

turnaround; report did not 

“impose regulatory 

requirements” 

GSI-191 No N/A “Study was for public confidence 

reasons and not driven by 

rulemaking”

Source: NRC staff and management. 

 5 For the full title of the rules, research activities, analyses, and studies, see Appendix B. 

6 Although the Orders were published in 2001, prior to the OMB Peer Review Bulletin, the resulting rule 

was published in 2009 and any information products supporting the final rule should have been evaluated 

for Influential Scientific Information or Highly Influential Scientific Assessment. 

7 Although the update was published prior to the OMB Peer Review Bulletin, the final Rule was published 

in 2009, after the OMB Bulletin.
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

15

To improve the quality of Government information, agencies have a 

responsibility to ensure that the information they produce conforms to 

OMB guidance on information quality. This requires that adequate 

systems are in place to foster openness and transparency that facilitate

public participation in the agency’s Information Quality Program.

In its current state, NRC’s Information Quality Program does not meet the 

intent of Federal law and does not fully ensure and maximize the quality, 

objectivity, utility, and integrity of information it disseminates. Stakeholder 

participation allows the public to contribute ideas and expertise, thereby 

further enhancing the quality, objectivity, utility, and integrity of an 

agency’s information products while promoting the concepts of 

transparency and openness. However, the opportunity for the public to 

seek and obtain correction to NRC information is severely limited. As a 

result, an opportunity to maximize the quality, objectivity, utility, and 

integrity of NRC information products through public review is being 

missed. 

Additionally, because the agency’s information products are not being 

consistently reviewed and identified as Influential Scientific Information or 

Highly Influential Scientific Assessments in accordance with Federal 

requirements, NRC may be reporting inaccurate data to OMB. As a result, 

NRC’s regulatory work may not be accurately represented to the public 

and Congress. Consequently, in both instances, stakeholder perception 

of the agency’s commitment to openness and transparency is 

compromised. 

Recommendations

OIG recommends that the Executive Director for Operations

1. Designate an Information Quality Program owner and clearly 

delineate roles and responsibilities among involved offices.

2. Assign responsibility for routinely checking, responding to, and 

tracking the resolution of information correction requests submitted 

Why This Is Important
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

16

via vehicles identified on NRC’s public Web pages regarding 

Information Quality Guidelines.

3. Hold an annual meeting with involved NRC staff and management 

to discuss the requirements and responsibilities associated with the 

identification and reporting of Influential Scientific Information and 

Highly Influential Scientific Assessment information products.

B. NRC Has Not Adopted OMB Guidelines on Peer Review

NRC has not adopted OMB requirements on peer review as its official 

guidelines. The OMB requirements have not been adopted because NRC 

lacks an effective mechanism to ensure that internal office guidance that 

may be impacted by new or revised Federal guidance is regularly 

reviewed to determine if revisions are necessary. As a result, offices 

involved in the peer review process, such as RES, have outdated office 

guidance on the peer review process that is not aligned with OMB 

requirements. Consequently, the potential exists for NRC to 

inappropriately assess the need for and/or conduct a peer review, which 

could subsequently lead to the public questioning the credibility of NRC’s 

research products. 

Federal law requires integrity in the information disseminated by the 

Government. The Information Quality Act of 2001 directed OMB to issue 

Governmentwide guidelines that provide guidance to Federal agencies for 

ensuring and maximizing the quality, objectivity, utility, and integrity of 

information. OMB issued a 2004 Bulletin titled “Final Information Quality 

Bulletin for Peer Review” (2004 OMB Bulletin) that 

1. Defines peer review and explains its importance and purpose. 

2. Requires that important scientific information be peer reviewed, 

What Is Required
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

17

including influential scientific information and highly influential 

scientific assessment.

3. Requires agencies to adopt the committee selection policies of the 

National Academy of Sciences8 when selecting peer reviewers who 

are not Government employees.

One way a Federal agency can ensure the integrity of its information is 

through a peer review. Peer review is an important procedure used to 

ensure that information meets the standards of the scientific and 

technical community. Peer review involves the review of a draft by a 

specialist in the field who was not involved in producing the draft. The 

critique provided by peer review often suggests ways to clarify 

assumptions, findings, and conclusions. Peer review may take a 

variety of forms, depending upon the nature, importance, and 

categorization of the product. 

OMB requires that for peer review of influential scientific information, 

an agency must consider the following: 

1. Individual versus panel review.

2. Timing of peer review.

3. Scope of the review.

4. Selection of the reviewers, including conflicts of interest.

5. Public participation.

6. Disposition of the reviewer comments. 

7. Adequacy of prior peer review. 

OMB requirements for a Highly Influential Scientific Assessment are 

stricter than those for Influential Scientific Information. For example, 

agencies must ensure that reviewers are independent of the agency 

 8 The National Academy of Sciences is a private, non-profit society established by Congress, charged 

with providing independent and objective advice on matters related to science and technology. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

18

sponsoring the review. Additionally, agencies must avoid repeated use of 

the same reviewer on multiple assessments unless his or her participation 

is essential and cannot be obtained elsewhere. 

Moreover, NRC must adopt the National Academy of Sciences policy for 

selection of peer reviewers who are not Government employees, with 

respect to evaluating for the potential of conflicts of interest. For example, 

when selecting a non-Government peer reviewer, NRC must use the 

Academy’s policies for evaluating conflicts arising from investments, 

affiliations, and income. 

NRC has not adopted the OMB requirements as its official guidelines for 

peer review. Specifically, offices that have a central role in the agency’s 

peer review process, such as RES, have outdated office guidance on the 

peer review process. 

RES has internal office guidance on peer review – PRM-010: Peer Review 

of RES Projects (PRM-010). However, PRM-010 is not in full alignment 

with OMB requirements. For example, NRC must adhere to the 2004 

OMB Bulletin in conducting a peer review of Influential Scientific 

Information or a Highly Influential Scientific Assessment, but PRM-010 

does not clearly articulate the criteria associated with identifying Influential 

Scientific Information, or address the specific requirements associated 

with peer reviewing such products. For example, PRM-010 does not 

mention the adoption of the committee selection policies of the National 

Academy of Sciences when selecting peer reviewers who are not 

Government employees. PRM-010 does not provide correct reference to 

MD 3.17, a document that serves as the agency’s primary guidance 

document on Information Quality including review of Influential Scientific 

Information and Highly Influential Scientific Assessment products. 

However, PRM-010 does say that NRC is in the process of adopting 

OMB’s guidelines on peer review as its official guideline for peer review. 

Although NRC has not officially adopted OMB peer review guidelines, 

NRC has a fairly robust peer review program, whereby its products are 

What We Found
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

19

peer reviewed and the agency receives independent feedback in other 

forums. For example, the State-of-the-Art Reactor Consequence 

Analyses (SOARCA), the Spent Fuel Pool Study,

9 and the Cancer Study 

were all peer reviewed. Additionally, NRC receives independent feedback 

in other forums outside the traditional peer review context. For example, 

NRC partners with other countries to exchange information on complex 

computer codes.

10 NRC benefits from these partnerships because the 

countries share their code assessment and experimental results, which

helps NRC's validation of the codes. Also, NRC runs cooperative testing 

with Federal and international partners. For example, NRC ran tests on 

spent fuel pools at a national laboratory with the Committee of the Safety 

of Nuclear Installations and the Nuclear Energy Agency. As a result of 

this cooperative experiment, NRC will benefit from other countries' review 

and analysis of the testing. 

NRC has not adopted OMB’s peer review requirements because there is 

not an effective control to ensure that internal office guidance is reviewed 

upon issuance or revision against Federal requirements. RES requires a 

biennial review of its office instructions, set forth in Office Instruction 

ADM-001, Preparing and Maintaining RES Office Instructions. However, 

the requirement was not followed for updating PRM-010. PRM-010 was 

published in 2007 for interim use and has not been updated. 

Consequently, ADM-001 is not an effective control to ensure that internal 

RES guidance is updated. 

 9 The full title of the study is “Consequence Study of a Beyond-Design-Basis Earthquake Affecting the 

Spent Fuel Pool for A U.S. Mark-I Boiling-Water Reactor.”

10 Partnerships include: (i) CSARP [The Cooperative Severe Accident Research Program], which 

facilitates joint information exchange, periodic analysis, and data for additional code validation and 

verification; and (ii) CAMP [Code Applications and Maintenance Program], which exchanges information 

on thermal-hydraulic safety issues related to reactor and plant systems.

Why This Occurred
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

20

Since RES internal office instruction is not aligned with OMB guidance, the 

potential exists for NRC to inappropriately assess the need for and/or 

conduct a peer review. For example, an NRC product that qualifies as 

Influential Scientific Information or as a Highly Influential Scientific 

Assessment may not be peer reviewed in accordance with required OMB 

standards. Additionally, when NRC selects a peer reviewer who is not a 

Government employee, the required National Academy of Sciences 

selection criteria may not be followed. 

Subsequently, the potential exists for stakeholders to question the 

credibility of information disseminated by NRC. The public must be able 

to have confidence in the Government’s scientific products. This is why 

OMB issued the 2004 Bulletin on peer review – to enhance the quality and 

credibility of the Government’s scientific information. To have public 

confidence in the quality and credibility of its research products, NRC must 

follow OMB requirements. 

Recommendations

OIG recommends that the Executive Director for Operations

4. Develop a schedule to ensure that RES’ internal office instructions

are regularly reviewed and revised, as appropriate. 

5. Direct all offices to review and revise office guidance on peer 

review to align with requirements established by OMB and the 

National Academy of Sciences. 

Why This Is Important
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

21

It is imperative for NRC to have effective controls in place to ensure that 

its information products are objective, credible, and transparent. Not only 

do such controls help ensure that NRC is compliant with Federal 

requirements, but it also strengthens the stakeholder confidence in the 

agency’s ability to regulate in an unbiased, trustworthy, and open manner. 

At the highest level, the public and other stakeholders want and expect 

that there is scientific integrity behind the regulatory decisions the agency 

is making.

 IV. SUMMARY/CONCLUSION
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

22

 V. CONSOLIDATED LIST OF RECOMMENDATIONS

OIG recommends that the Executive Director for Operations

1. Designate an Information Quality Program owner and clearly delineate 

roles and responsibilities among involved offices.

2. Assign responsibility for routinely checking, responding to, and tracking 

the resolution of information correction requests submitted via vehicles 

identified on NRC’s public Web pages regarding Information Quality 

Guidelines.

3. Hold an annual meeting with involved NRC staff and management to 

discuss the requirements and responsibilities associated with the 

identification and reporting of Influential Scientific Information and Highly 

Influential Scientific Assessment information products.

4. Develop a schedule to ensure that RES’ internal office instructions are 

regularly reviewed and revised, as appropriate. 

5. Direct all offices to review and revise office guidance on peer review to 

align with requirements established by OMB and the National Academy of 

Sciences. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

23

An exit conference was held with the agency on January 23, 2015. Prior 

to the exit conference, after reviewing a discussion draft, agency 

management provided comments that have been incorporated into this 

report, as appropriate. As a result, agency management stated their 

general agreement with the findings and recommendations and opted not 

to provide formal comments. 

 V. AGENCY COMMENTS
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

24

Appendix A

Objective

The audit objective was to determine whether NRC has controls in place 

to assure that scientific research is objective, credible, and transparent.

Scope

The scope of the audit focused on identifying and analyzing NRC’s current 

controls to determine if they are aligned with Federal guidance and ensure 

that its scientific research is objective, credible, and transparent. We 

conducted this performance audit at NRC headquarters (Rockville, MD) 

from June 2014 to December 2014. Internal controls related to the audit 

objective were reviewed and analyzed. Throughout the audit, auditors 

were aware of the possibility of fraud, waste, or abuse in the program.

Methodology

OIG reviewed OMB requirements, NRC Management Directives, and NRC 

Office Instructions, including, but not limited to:

• Memorandum for the Heads of Executive Departments and Agencies 

(2009).

• OMB Guidelines for Ensuring and Maximizing Quality, Objectivity, 

Utility, and Integrity (2001).

• OMB’s Final Information Quality Bulletin for Peer Review (2004).

• MD 3.17: NRC’s Information Quality Management Program (2009).

• MD 10.158: NRC Non-Concurrence Process (2014).

• MD 10.159: The NRC Differing Professional Opinions Program.

 OBJECTIVE, SCOPE, AND METHODOLOGY
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

25

• PRM - 001, Rev 0: Process for New Work Requests: Responding to 

Informal Assistance, Research Assistance and User Need Requests

(2009). 

• PRM - 010, Revision 0: Peer Review RES Projects (2007).

• COM - 100: Office Instruction Office of Nuclear Reactor Regulation 

(NRR) interfaces with RES (2006).

• COM - 106: Office of New Reactors (NRO) interfaces with RES Rev. 2 

(2013).

OIG interviewed NRC staff and management to gain an understanding of 

the roles and responsibilities related to assurance that scientific research 

is objective, credible, and transparent. Auditors interviewed staff and 

management from the Office of Information Services, the Office of the 

Executive Director for Operations, the Office of Nuclear Regulatory 

Research, the Office of Nuclear Reactor Regulation, the Office of Nuclear 

Material Safety and Safeguards, the Office of New Reactors, and the 

Advisory Committee on Reactor Safeguards. Additionally, OIG contacted 

representatives from OMB. 

We conducted this performance audit in accordance with generally 

accepted Government auditing standards. Those standards require that 

we plan and perform the audit to obtain sufficient, appropriate evidence to 

provide a reasonable basis for our findings and conclusions based on our 

audit objectives. We believe that the evidence obtained provides a 

reasonable basis for our findings and conclusions based on our audit 

objectives. 

This audit was conducted by Sherri Miotla, Team Leader; Kristen Lipuma, 

Audit Manager; Kevin Nietmann, Senior Technical Advisor; Jaclyn Storch, 

Senior Management Analyst; Ashley Gerwitz, Management Analyst; and 

Anna Boyle, Student Intern. 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

26

Appendix B

OIG analyzed multiple NRC rules, analyses, and studies performed since 

the 2004 OMB Bulletin was issued. These items were selected because 

of significant public interest, interagency cooperation, and potential for 

substantial impact. These items are named in Table 3 and are referred to 

by their commonly known name, rather than their official title. The official 

title of the items is listed below:

• Power Reactor Security Requirements (10 CFR Part 73) [2009 

Security Rule]. 

• NUREG 2155 - Implementation Guidance for Title 10 of the CFR Part 

37, "Physical Protection of Category 1 and Category 2 Quantities of 

Radioactive Material” [10 CFR Part 37].

• Continued Storage of Spent Nuclear Fuel Rule (10 CFR 51.23) 

[Continued Storage Rule]. 

• Final Rulemaking to Amend the Fitness-for-Duty Rule (10 CFR Part 

26) [Fatigue Management Rule].

• SOARCA [State-of-the-art Reactor Consequence Analysis]. 

• Analysis of Cancer Risks in Populations Near Nuclear Facilities: 

Phase I [Cancer Study].

• Recommendations for Enhancing Reactor Safety in the 21st Century: 

The Near-Term Task Force Review of Insights From the Fukushima

Dai-ichi Accident [Fukushima Take Force Report].

• GSI-191: Experimental Studies of Loss-of-Coolant-Accident-Generated 

Debris Accumulation and Head Loss with Emphasis on the Effects of 

Calcium Silicate Insulation [GSI-191]. 

 Rules, Analyses, and Studies Reviewed by OIG 
Audit of NRC’s Process for Ensuring Integrity in Scientific Research

27

 TO REPORT FRAUD, WASTE, OR ABUSE

 COMMENTS AND SUGGESTIONS

Please Contact:

Email: Online Form

Telephone: 1-800-233-3497

TDD 1-800-270-2787

Address: U.S. Nuclear Regulatory Commission

Office of the Inspector General

Hotline Program

Mail Stop O5-E13

11555 Rockville Pike

Rockville, MD 20852

If you wish to provide comments on this report, please email OIG using this link.

In addition, if you have suggestions for future OIG audits, please provide them using 

this link.