OFFICE OF 

THE INSPECTOR GENERAL 

U.S. NUCLEAR 

REGULATORY COMMISSION 

Audit of NRC’s Reactor Program System 

OIG-05-A-11 April 13, 2005 

AUDIT REPORT 

All publicly available OIG reports (including this report) are accessible through 

NRC’s Web site at: 

http:/www.nrc.gov/reading-rm/doc-collections/insp-gen/ 
 April 13, 2005 

MEMORANDUM TO: Luis A. Reyes 

 Executive Director for Operations 

FROM: Stephen D. Dingbaum/RA/ 

 Assistant Inspector General for Audits 

SUBJECT: AUDIT OF NRC’S REACTOR PROGRAM SYSTEM 

 (OIG-05-A-11) 

Attached is the Office of the Inspector General’s (OIG) audit report titled, Audit of 

NRC’s Reactor Program System. 

This audit found that while the implementation of the Reactor Program System 

(RPS) has allowed for a single system for entering inspection information, the 

information is not well protected, is not complete, and is not fully accurate. To 

ensure that the system meets operational requirements, NRC needs to: 

3⁄4 Comply with RPS access control requirements. 

3⁄4 Ensure accurate and timely inspection data. 

3⁄4 Improve management of the system help service. 

3⁄4 Improve the system configuration control process. 

3⁄4 Provide training to system users. 

During an exit conference on March 2, 2005, NRC officials provided informal 

comments concerning the draft audit report. Subsequent to that meeting, OIG 

met with agency senior managers to address issues and comments needing 

further clarification and/or explanation. Comments your office provided at the exit 

meeting and during subsequent discussions have been incorporated, as 

appropriate, in our final report. 

If you have any questions, please call Beth Serepca at 415-5911 or me at 

415-5915. 

Attachment: As stated 
Distribution

 

John T. Larkins, Executive Director, Advisory Committee on Reactor 

 Safeguards/Advisory Committee on Nuclear Waste 

G. Paul Bollwerk, III, Chief Administrative Judge, Atomic Safety and 

 Licensing Board Panel 

Karen D. Cyr, General Counsel 

John F. Cordes, Jr., Director, Office of Commission Appellate Adjudication 

Jesse L. Funches, Chief Financial Officer 

Janice Dunn Lee, Director, Office of International Programs 

William N. Outlaw, Director of Communications 

Eliot B. Brenner, Director, Office of Public Affairs 

Annette Vietti-Cook, Secretary of the Commission 

Luis A. Reyes, Executive Director for Operations 

Jacqueline E. Silber, Deputy Executive Director for Information Services 

 and Administration and Chief Information Officer, OEDO 

William F. Kane, Deputy Executive Director for Reactor and Preparedness 

 Programs, OEDO 

Martin J. Virgilio, Deputy Executive Director for Materials, Research , 

 State and Compliance Programs, OEDO 

William M. Dean, Assistant for Operations, OEDO 

Timothy F. Hagan, Director, Office of Administration 

Frank J. Congel, Director, Office of Enforcement 

Guy P. Caputo, Director, Office of Investigations 

Edward T. Baker, Director, Office of Information Services 

Paul E. Bird, Director, Office of Human Resources 

Corenthis B. Kelley, Director, Office of Small Business and Civil Rights 

Jack R. Strosnider, Director, Office of Nuclear Material Safety and Safeguards 

James E. Dyer, Director, Office of Nuclear Reactor Regulation 

Carl J. Paperiello, Director, Office of Nuclear Regulatory Research 

Paul H. Lohaus, Director, Office of State and Tribal Programs 

Roy P. Zimmerman, Director, Office of Nuclear Security and Incident Response 

Samuel J. Collins, Regional Administrator, Region I 

William D. Travers, Regional Administrator, Region II 

James L. Caldwell, Regional Administrator, Region III 

Bruce S. Mallett, Regional Administrator, Region IV 

Office of Public Affairs, Region I 

Office of Public Affairs, Region II 

Office of Public Affairs, Region IV 
Audit of NRC’s Reactor Program System 

i

EXECUTIVE SUMMARY 

BACKGROUND

The Nuclear Regulatory Commission's (NRC) mission is to regulate 

the Nation's civilian use of byproduct, source, and special nuclear 

materials to ensure adequate protection of public health and safety, 

promote the common defense and security, and protect the 

environment. Fundamental to the regulatory process is NRC’s 

commercial nuclear power plant inspection program, which 

assesses whether plant operations are properly conducted and 

equipment is properly maintained. Inspectors examine licensee 

activity, provide inspection findings to licensee managers, and 

conduct followup inspections to ensure that corrective actions are 

taken. 

The Reactor Program System (RPS) is an information technology 

tool that provides planning, scheduling, and reporting capabilities to 

support the NRC reactor inspection and licensing programs. It is 

used by NRC managers to assess the effectiveness and uniformity 

of the implementation of those programs and related policies. The 

Office of Nuclear Reactor Regulation (NRR) and the regions use 

RPS to schedule their work assignments and to plan and schedule 

licensing activities in NRR and inspection activities at nuclear 

power plants. 

PURPOSE

The objectives of this audit were to determine if RPS (1) provides 

for the availability, confidentiality, and integrity of the data stored in 

the system and (2) meets its required operational capabilities. 

RESULTS IN BRIEF

While the implementation of RPS has allowed for a single system 

for entering inspection information, the information is not well 

protected, is not complete, and is not fully accurate. To ensure that 

the system meets operational requirements, NRC needs to: 

3⁄4 Comply with RPS access control requirements. 

3⁄4 Ensure accurate and timely inspection data. 

3⁄4 Improve management of the system help service. 

3⁄4 Improve the system configuration control process. 

3⁄4 Provide training to system users. 
Audit of NRC’s Reactor Program System 

ii

RECOMMENDATIONS

This report makes 10 recommendations to strengthen protection 

over RPS data and better ensure the system meets its operational 

requirements. A consolidated list of recommendations appears on 

page 19 of this report. 

AGENCY COMMENTS

At an exit conference with agency senior executives held on 

March 2, 2005, NRC officials generally agreed with the report’s 

findings and recommendations. Subsequent to that meeting, Office 

of the Inspector General (OIG) staff met with NRR staff to address 

specific issues and concerns needing further clarification and/or 

explanation. On March 31, 2005, the Executive Director of 

Operations provided a formal response to this report in which he 

agreed with the final version of the report. Appendix B contains a 

copy of the agency’s written comments.
Audit of NRC’s Reactor Program System 

iii

ABBREVIATIONS AND ACRONYMS 

OIG Office of the Inspector General 

NRC Nuclear Regulatory Commission 

NRR Office of Nuclear Reactor Regulation 

RPS Reactor Program System 

SDLCM System Development and Life-Cycle Management 
Audit of NRC’s Reactor Program System 

iv

[Page intentionally left blank.] 
Audit of NRC’s Reactor Program System 

v

TABLE OF CONTENTS 

EXECUTIVE SUMMARY ....................................................................................... i 

ABBREVIATIONS AND ACRONYMS...................................................................iii 

I. BACKGROUND .........................................................................................1 

II. PURPOSE..................................................................................................3 

III. FINDINGS..................................................................................................3 

 A. NON-COMPLIANCE WITH RPS ACCESS CONTROL 

 REQUIREMENTS .................................................................................3 

 B. INSPECTION DATA INACCURATE AND UNTIMELY ....................................7 

 C. RPS HELP (RPSHELP) IS INEFFICIENT ............................................10 

 D. CONFIGURATION CONTROL PROCESS HAS NOT 

 IDENTIFIED SOME CONCERNS ...........................................................12 

 E. TRAINING AND GUIDANCE NOT PROVIDED TO SYSTEM USERS .............15 

IV. CONSOLIDATED LIST OF RECOMMENDATIONS ................................18 

V. AGENCY COMMENTS ............................................................................19 

APENDICES 

 A. SCOPE AND METHODOLOGY .........................................................21 

 B. FORMAL AGENCY COMMENTS.......................................................23 
Audit of NRC’s Reactor Program System 

vi

[Page intentionally left blank.] 
Audit of NRC’s Reactor Program System 

1

I. BACKGROUND 

NRC’s mission is to regulate the Nation's civilian use of byproduct, 

source, and special nuclear materials to ensure adequate 

protection of public health and safety, promote the common 

defense and security, and protect the environment. Fundamental 

to the regulatory process is NRC’s commercial nuclear power plant 

inspection program, which assesses whether plant operations are 

properly conducted and equipment is properly maintained. 

Inspectors examine licensee activity, provide inspection findings to 

licensee managers, and conduct followup inspections to ensure 

that corrective actions are taken. 

RPS is an information technology tool that provides planning, 

scheduling, and reporting capabilities to support the NRC reactor 

inspection and licensing programs. It is used by NRC managers to 

assess the effectiveness and uniformity of the implementation of 

those programs and related policies. NRR and the regions use 

RPS to schedule their work assignments and to plan and schedule 

licensing activities in NRR and inspection activities at nuclear 

power plants. 

The RPS database includes inspection and licensing information, 

plant performance indicators, inspection followup items, and other 

administrative and reactor regulatory data. This information is 

contained in 13 currently active modules,1

 3 of which are 

specifically used to track inspection schedules and results of the 

inspection program. 

Inspection schedules, inspection reports, performance assessment 

letters, plant item matrices, performance indicator data, and 

operator licensing exam schedules are posted on the NRC external 

Web site. This information provided a single location for the public 

to obtain information from RPS and supports the agency’s goal to 

ensure openness in the regulatory process. 

System development was initiated in 1995 when NRR recognized a 

need for regulatory and administrative improvements in the 

inspection program. The system became operational in 1998, 

bringing together functions that were previously performed by 10 

separate mainframe systems that served the reactor inspection 

program in headquarters and the regions. In designing the system, 

NRR intended for NRC employees in headquarters and the regions 

 

1 After audit fieldwork was completed, a 14th module was added to RPS to support the anticipated receipt of 

a construction inspection application related to the building of a new nuclear reactor. 
Audit of NRC’s Reactor Program System 

2

to use an integrated method for planning, scheduling, and reporting 

activities related to reactor inspections. The system was also 

intended to allow data to be entered one time in an effort to reduce 

the data duplication that occurred when the inspection data was 

stored in separate mainframe systems. 

In the years since its implementation, RPS has evolved to meet 

changing agency needs and fulfill inspection program requirements. 

For example, the agency’s implementation of a revised reactor 

oversight process2

 in 2000 necessitated changes in RPS, including 

how and what type of information would be stored in the system. 

Overall, more than 1,400 people use RPS agencywide, although 

not all users have access to all modules. Data is entered into RPS 

primarily by regional staff3 but is analyzed by both regional and 

headquarters employees. 

NRR developed and funded RPS and continues to support and 

maintain the system. To coordinate the needs of NRC regional and 

headquarters users, NRR conducts a meeting every 6 months or on 

an as-needed basis. Participants, referred to as RPS counterparts, 

bring ideas and suggestions for improving the system. Meetings 

are chaired by system managers who are responsible for 

maintaining and updating the system based on user needs. NRR 

modifies RPS in response to changes in agency policies, feedback 

received from users, and information shared during the counterpart 

meetings. NRR issues updated versions of the system when 

appropriate. 

RPS cost $2.7 million to develop and costs $650,000 to maintain 

yearly. According to NRC’s Web site, the agency has saved 

$800,000 each year due to the discontinued use of the 10 

mainframe systems. 

 

2

 OIG has issued two reports assessing components of the revised reactor oversight process: 

OIG-02-A-15, Review of NRC’s Significance Determination Process, and OIG-05-A-06, Audit of NRC’s 

Baseline Inspection Program. OIG plans to conduct a third audit in this area during fiscal year 2005 on the 

performance indicator program. 

3

 NRC regional staff includes senior resident inspectors, resident inspectors, and office assistants at 

licensee sites, and employees working from NRC’s four regional offices. 
Audit of NRC’s Reactor Program System 

3

II. PURPOSE 

The objectives of this audit were to determine if RPS (1) provides 

for the availability, confidentiality, and integrity of the data stored in 

the system and (2) meets its required operational capabilities. 

III. FINDINGS 

While the implementation of RPS has allowed for a single system 

for entering inspection information, improvements are needed in 

RPS to strengthen protection over data stored in the system and 

better ensure the system meets its operational requirements. 

Specifically, NRC needs to: 

3⁄4 Comply with RPS access control requirements. 

3⁄4 Ensure accurate and timely inspection data. 

3⁄4 Improve management of the system help service. 

3⁄4 Improve the system configuration control process. 

3⁄4 Provide training to system users. 

A. NON-COMPLIANCE WITH RPS ACCESS CONTROL REQUIREMENTS

 

RPS user access to inspection data is not restricted to the minimum 

amount necessary because the system offers regional employees 

only one level of access to inspection data (read-and-write). In 

addition, many employees have access to RPS modules they do 

not use. This occurs because (1) there is no written guidance on 

which users should receive what level of access and (2) there is no 

process for removing users from the system after they no longer 

require access. Failure to restrict RPS access increases the risk to 

system data accuracy and security. 

Security Plan Requirements

The RPS security plan acknowledges that individuals authorized to 

have access to information systems potentially impose the greatest 

harm to those systems, both accidentally and intentionally. The 

RPS security plan lists various security controls to prevent and 

detect harm to the system. Such controls include employee 

background checks, the ability to associate users with their system 
Audit of NRC’s Reactor Program System 

4

activity, and the practice of restricting a user’s access to data files 

and the levels of access (e.g., read, write, execute, delete) to the 

minimum amount necessary to perform his or her job. This latter 

practice is known as least privilege. 

Least Privilege Principle Not Followed

The RPS security plan states that the least privilege principle 

should be followed in granting access rights to system users. 

However, this principle is not applied effectively in two ways: (1) 

many users have access to modules they do not use and (2) the 

amount of access that many users have (write access4) to 

inspection data exceeds the actual access needed to perform their 

job functions. 

As noted in the background, RPS contains 13 modules and users 

are granted read-only or read-and-write access to varying numbers 

of these modules based on the nature of their duties. Of the 13 

modules, 3 are used to track inspection schedules and results of 

the inspection program: 

3⁄4 The Inspection Planning module contains information on 

inspection schedules for NRC licensee sites, including reactor 

outages and visits by reactor inspectors. 

3⁄4 The Item Reporting module contains the results of all 

inspections performed at the sites, including information on 

safety issues and inspection followup items. 

3⁄4 The Reports module provides users with more than 100 

standard reports on RPS data including information in the 

Inspection Planning and Item Reporting modules. 

As of August 2004, 738 headquarters and regional employees were 

authorized read access to data in the Inspection Planning module, 

684 were authorized read access to data in the Item Reporting 

module, and 1,382 were authorized read access to data in the 

Reports module. Write access to the Item Reporting and 

Inspection Planning modules was more restricted; on average 178 

employees in each of NRC’s 4 regional offices were authorized 

write access to the inspection data pertaining to their region in 

these modules.5

 

 

4

 Write access is the ability to enter and change system data. 

5

 The Reports module does not offer a write-access option because this module uses existing data in the 

Item Reporting and Inspection Planning modules to create different reports. 
Audit of NRC’s Reactor Program System 

5

OIG further determined that most of the employees in each region 

with write access to inspection data contained in the Inspection 

Planning and Item Reporting modules do not require this access. 

Relatively few of these individuals actually enter RPS inspection 

data in the regions. In one region, 6 employees are responsible for 

entering all of the region’s data into RPS, although 200 employees 

have the capability. All of these employees are based in the 

regional office. In another region, approximately 58 employees 

working at both reactor sites and in the region enter such data, 

while 144 employees have the capability to do so. Similarly, in the 

other two regions, more individuals have access rights to enter 

information in RPS than actually use this capability. 

OIG compared the number of employees authorized any type of 

access to 11 RPS modules6

 with the number of employees who 

actually visited those modules during a 71⁄2-month period during 

2004. Many users with access to RPS modules used those 

modules rarely, if at all. Table 1 presents a comparison of users 

assigned access to each of the 11 RPS modules with the usage 

levels for that 71⁄2-month period. 

 

6

 NRR could provide data concerning only 11 of the 13 modules that were functional during the fieldwork 

stage of this audit. 
Audit of NRC’s Reactor Program System 

6

Table 1. 

Comparison of RPS Users With Access to Each Module to Actual Number 

of Users, as of August 18, 2004

Module Number of 

Employees 

Authorized Access 

Actual Number of 

Users, 1/1/04-

8/18/04 

Percentage of 

Users Who 

Accessed Module 

Inspection 

Planning 

738 510 69.1 

Item Reporting 684 451 65.9 

Reports 1,382 413 29.9 

Inspection 

Procedure 

Authority System 

726 115 15.8 

Inspection 

Planning Cycle 

480 14 2.9 

Inspection Report 

Tracking System 

89 42 47.2 

Time Resource 

Inventory 

Management 

607 453 74.6 

NRC Utilities 33 18 54.6 

Security Access 

Method 

39 27 69.2 

Safety Information 

Management 

System 

15 5 33.3 

Tables 40 31 77.5 

As seen above, for five of the modules, less than half of the 

authorized users visited the modules at any time during the 71⁄2 

months. One module in particular had especially low use with only 

2.9 percent of authorized users accessing the module during this 

timeframe. 

Access Not Sufficiently Restricted

Many regional employees who do not need write access to the 

Inspection Planning and Item Reporting modules have it because 

RPS offers them only one level of access (read-and-write). 

Regional employees are granted read access to all inspection data 

in RPS and write access to inspection data pertaining to their 

specific region. 
Audit of NRC’s Reactor Program System 

7

In addition, many employees have access to RPS modules they do 

not use because they were not removed from the system after they 

no longer required access. In headquarters, there is no established 

process for offices to inform the RPS system administrator of the 

need to terminate user access when an employee’s job duties no 

longer require access. In the regional offices, discretion for 

removing access is left to the time and labor coordinator. As a 

result, headquarters does not provide oversight to ensure that this 

process occurs overall. 

Failure to restrict access to RPS modules based on the least 

privilege principle leads to an increased risk to the accuracy and 

integrity of the system data. In addition, by not removing inactive 

users, the system is not in compliance with the principle of least 

privilege. 

Recommendations

OIG recommends that the Executive Director for Operations: 

1. Implement a tiered access level structure that allows users 

access to RPS modules based on the least privilege principle. 

This should include guidance on which users may receive 

what level of access. 

2. Implement a process for removing access rights from inactive 

Reactor Program System users. 

B. INSPECTION DATA INACCURATE AND UNTIMELY

Some RPS inspection data is inaccurate and untimely because the 

regions employ inconsistent quality control processes over RPS 

data and because RPS data is never locked down to prevent 

alterations to the data after it is entered in the system. Inaccurate 

and untimely inspection results can negatively affect NRC 

decisionmaking and public confidence. Although there are multiple 

safety nets in place, including management expertise, to ensure 

public health and safety, inaccurate inspection results could affect 

the agency’s ability to fulfill its mission of ensuring public health and 

safety. 
Audit of NRC’s Reactor Program System 

8

Data Entry Policy

NRC Inspection Manual, Chapter 0306, “Information Technology 

Support for the Reactor Oversight Process,” provides policy and 

guidance on using NRC information management systems that 

support the reactor inspection program. It requires the timely and 

accurate entry of RPS data for all regional reactor inspection 

activities. Furthermore, upon completing an inspection and no later 

than 10 days after the exit meeting with the licensee, the branch 

chief (or designee) responsible for the inspection must update RPS 

with the inspection results. 

Inconsistent and Untimely Sample Data

Despite the above cited requirements, data is not consistently 

accurate or entered into RPS within the 10-day deadline. OIG 

compared data from 32 inspection reports7

 with data contained in 

the RPS Item Reporting module. The intent of this comparison was 

to determine (1) if the sample sizes reported in the inspection 

reports were consistent with the data reported in RPS and (2) 

whether the sample size data had been entered within the required 

timeframe.8 In total, OIG assessed the accuracy of 646 data items 

and found inconsistencies in 13.5 percent of the sample data.9

 

Based on a statistical analysis of this information, as much as 23 

percent of the inspection data could be inconsistent between RPS 

and the inspection reports. In one case, the number of instances a 

procedure was conducted was overstated by more than 100 

percent in RPS. This analysis also identified that 32 percent of the 

data concerning sample size was entered into RPS outside the 

required time period. In one instance, data was entered into RPS 

378 days after the exit meeting with the licensee. 

Inconsistent Quality Control Process

These data inaccuracies and timeliness issues occurred because 

the regions employ inconsistent quality control processes over RPS 

data and because RPS data is never locked down after it is 

entered, allowing it to be modified at any time by anyone with write 

access to the data. 

 

7

 Auditors reviewed a total of 646 data items contained in 32 inspection reports. These reports reflected 

results drawn from four commercial nuclear power plants in each of NRC’s four regions 

8

 Auditors did not attempt to compare data items in cases where sample size, as described in the inspection 

report, was unclear. 

9

 RPS data was only compared to inspection report data for consistency. When discrepancies were found, 

no conclusions were drawn as to whether RPS or the inspection report contained the correct information. 
Audit of NRC’s Reactor Program System 

9

Although NRC Inspection Manual, Chapter 0306 states that the 

branch chief is responsible for quality control of RPS data, there is 

no consistent approach to quality control among the regions and 

approaches vary considerably in rigor and effectiveness. For 

example, one regional branch chief will not transmit an inspection 

report to the licensee until this individual has personally checked it 

against the data in RPS. In a different region, a branch chief relies 

on regional quarterly reviews, which compare the inspection reports 

from that quarter against the data in RPS, to catch RPS data errors. 

Furthermore, the regions employ inconsistent approaches to 

entering information in RPS. At one extreme, a regional branch 

chief stated that to reduce the chances of error, a single staff 

position — the senior project engineer — enters information into the 

system. This process is enforced throughout the region, which 

results in only six people entering all inspection information into the 

system. In a different region, however, the particular inspector who 

led the inspection is responsible for entering information into RPS. 

This example results in approximately 58 people entering 

information into the system. 

Another factor that jeopardizes data accuracy is that RPS data is 

never locked down and subsequently always remains editable. 

This allows information previously verified as correct to be changed 

at any time. One branch chief stated there are occurrences where 

data that had been verified as correct was later changed without 

the branch chief’s knowledge. This branch chief expressed a 

desire to have the capability to lock down data to prevent such 

occurrences. The RPS administrator stated that RPS could be 

designed to allow data to be locked down. 

Impact on NRC Decisionmaking Process

Inaccurate and untimely inspection results can negatively affect 

NRC decisionmaking and public confidence. Although there are 

multiple safety nets in place, including management expertise, to 

ensure public health and safety, inaccurate inspection results could 

affect the agency’s ability to fulfill its mission of ensuring public 

health and safety. NRC uses inspection information to ensure that 

licensees are operating at acceptable safety levels. When this 

information contains errors or is untimely, it does not provide 

managers with an accurate picture of the NRC inspection program 

as to the safety levels at nuclear power plants. In addition, 

inspection information is provided to the public via the agency Web 

site and inaccurate and untimely inspection results can erode 

confidence in NRC information.
Audit of NRC’s Reactor Program System 

10

Recommendations

OIG recommends that the Executive Director for Operations: 

3. Implement a uniform quality control and Reactor Program 

System data entry review process that will ensure data 

accuracy and timeliness. 

4. Develop a process to lock down Reactor Program System 

inspection data fields after the inspection report has been 

issued. 

C. RPS HELP (RPSHELP) IS INEFFICIENT

NRC requires all information technology system administrators to 

provide their users with support using the system. RPS achieves 

this through an e-mail account, referred to as RPSHELP. 

RPSHELP is inefficient and not widely used because the service is 

not well managed. Specifically, there is no formal process for 

handling help requests and there are no performance metrics to 

assess timeliness and user satisfaction. An inefficient help process 

does not assure that users are receiving timely and adequate 

responses to their questions. 

Help Function

RPSHELP is an NRC e-mail account designed to allow RPS users 

to convey concerns and questions regarding the system directly to 

experts who can respond to matters quickly and accurately. 

RPSHELP was created in 1997 to promote quick and efficient use 

of the system by helping users better understand how to use the 

system to meet their specific needs. NRC Inspection Manual

Chapter 0306 informs users that RPSHELP is available and should 

be used to answer questions regarding use of the system. 

No Assignment and Tracking Process

While users are directed to RPSHELP to better understand the 

system, the response process is inefficient. There is no systematic 

approach for assigning incoming queries to individuals for providing 

the responses and there is no formal process for tracking 

RPSHELP requests to completion and for user satisfaction. 
Audit of NRC’s Reactor Program System 

11

When users submit questions via RPSHELP, three headquartersbased employees who are knowledgeable about RPS are tasked 

with responding to these queries. Subsequently, upon receipt of an 

RPSHELP request, one or more of the three responders may opt to 

respond. According to one responder, there are occasions when 

multiple responders will send an answer to one user who requested 

help. In other cases, help requests are overlooked until one of the 

responders realizes that an answer has not been sent. 

In addition, there is no formal tracking of RPSHELP requests to 

completion and for user satisfaction. The system administrator 

estimated that questions received via RPSHELP are typically 

answered within a day or two, but said there is no formal process to 

track responses. In addition, there are no performance goals 

concerning timeliness or effectiveness of RPSHELP responses. 

Formal Process Needed

The RPSHELP process is inefficient because it is not well 

managed. Managers have not instituted a process for determining 

who is responsible for answering questions submitted to 

RPSHELP. Instead, they rely on an informal process that does not 

ensure responses are disseminated. In addition, there is no formal 

process to determine if answers to help requests are returned in an 

adequate amount of time or if responses are useful. 

Having an inefficient RPSHELP process does not assure that users 

are receiving timely and adequate responses. Without 

performance goals and a tracking process to determine whether 

these goals are met, management cannot assess the effectiveness 

or timeliness of the answers provided to RPS users. 

Recommendations

OIG recommends that the Executive Director for Operations: 

5. Formalize a process for handling Reactor Program System 

Help requests. 

6. Create performance metrics to assess timeliness and user 

satisfaction of Reactor Program System Help. 
Audit of NRC’s Reactor Program System 

12

D. CONFIGURATION CONTROL PROCESS HAS NOT IDENTIFIED SOME 

CONCERNS

NRC requires that management of information technology systems 

include configuration control, a process for determining and 

implementing changes to a system. RPS configuration control is 

conducted during counterpart meetings. These meetings have 

been ineffective in identifying some RPS limitations because 

regional representatives: 

3⁄4 Have not received formal guidance on gathering user concerns. 

3⁄4 Have not conducted surveys of users on their needs. 

By failing to raise these issues at counterpart meetings, regional 

representatives prevent RPS from being further developed to meet 

user needs and system objectives. 

Configuration Control Requirements

Managers of all NRC systems are required to follow the System 

Development and Life-Cycle Management (SDLCM) Methodology. 

This methodology defines the life cycle of an information 

technology system and describes the processes for developing, 

enhancing, and maintaining these systems. SDLCM requires the 

implementation of configuration control over information technology 

systems. Configuration control is a process of evaluating, 

approving or disapproving, and monitoring the implementation of 

changes to a system. RPS configuration control is conducted 

during counterpart meetings, where changes to the system are 

discussed, and after the meetings, when NRC makes changes to 

the system based on these discussions. In 1998, RPS managers 

provided regional counterparts with guidance describing their 

responsibilities. This guidance requires counterparts to: 

3⁄4 Obtain input from users concerning problems and potential 

enhancements to RPS. 

3⁄4 Raise regional concerns at counterpart meetings. 

3⁄4 Work with other counterparts to reach consensus on needed 

enhancements to the system. 

During the counterpart meetings, a representative from each region 

meets with the system managers,10 who are also RPSHELP 

responders, and other users based in headquarters to discuss 

regional concerns with the system. 

 

10 RPS system managers are three NRR employees who are responsible for maintaining and updating the 

system. 
Audit of NRC’s Reactor Program System 

13

Counterpart Meetings Ineffective

RPS counterpart meetings have been ineffective in bringing some 

RPS limitations to management attention. Two specific issues that 

users in all four regions raised with OIG have never been 

mentioned at the counterpart meetings. These issues are: 

3⁄4 The scheduling feature in RPS is limited and does not contain 

the level of detail desired by region-based reactor inspectors. 

3⁄4 RPS does not provide a tool for planning and tracking resident 

inspector annual inspection schedules. 

Region-based reactor inspectors travel to commercial nuclear 

power plants to conduct inspections in the areas of engineering, 

maintenance, radiological controls, emergency preparedness, 

security, and operator license requalification. They find the 

scheduling feature in RPS limiting, as it does not contain the 

desired level of detail. The desired level includes the ability to 

schedule and view the inspection schedule for 18 months, 

determine if inspections are fully staffed, and assess whether 

inspections will overlap at the plants. Thus, instead of relying on 

RPS to meet all of their scheduling needs, users have created 

separate scheduling tools that incorporate their needs. 

Resident inspectors, who conduct inspections that account for a 

major part of the inspection program, have created tools to track 

their inspections because RPS does not have this capability. 

Resident inspectors rely on these separate tracking tools to ensure 

inspection requirements are completed. For example, these tools 

track planned inspections throughout the year, inspectors’ plans for 

implementing inspection programs, and inspections that have been 

completed. Of the 47 RPS users interviewed, 34 used separate 

tools to track inspection information. 

RPS managers told OIG that they rely on counterpart 

representatives to bring regional issues to management’s attention. 

One RPS manager stated that they were unaware of the issues 

mentioned above, but agreed that these concerns were important 

and could be addressed by RPS. In addition, many issues were 

brought to an RPS manager’s attention, during a recent regional 

office visit, which had never been mentioned at prior counterpart 

meetings. The manager said that many of these issues could and 

would now be addressed in RPS. 
Audit of NRC’s Reactor Program System 

14

System Limitations Not Recognized

Various concerns have not been raised at counterpart meetings for 

several reasons: 

3⁄4 Counterparts and users have failed to recognize certain issues 

as problems. 

3⁄4 Counterparts have varying subjective perspectives on what 

constitutes an issue that is appropriate for discussion at 

counterpart meetings. 

3⁄4 Counterparts have ineffective methods of reaching out to users 

to learn of their concerns. 

Regional counterparts and system users have not recognized the 

scheduling issues that have led to the creation of additional 

systems as problems. Instead, they have accepted the system as 

is and viewed it as having limited scheduling capabilities. 

Counterparts and users have stated that the scheduling tool issues 

reflect an inherent system weakness. Subsequently, instead of 

bringing these issues to the counterpart meetings to see if the 

underlying issues can be addressed, counterparts accept that users 

rely on workarounds, such as separate tracking tools. 

RPS regional counterparts rely on subjective criteria to determine 

what issues rise to the level of appropriateness for discussion at the 

counterpart meetings. One regional counterpart uses the volume of 

user requests to identify issues for discussion in counterpart 

meetings, while another raises any issue if there is no quick 

workaround to resolve it. 

In addition, none of the regional counterparts proactively search for 

user concerns or issues. Counterparts bring regional concerns to 

the counterpart meetings based solely on their recollection of 

issues users have asked them about during the months preceding 

the counterpart meeting. 

By failing to raise certain concerns at meetings, counterparts 

prevent RPS from being further enhanced to meet user needs. As 

stated in the background section of this report, RPS was intended 

to reduce the number of locations in which inspection data is 

stored; however, the creation of separate tools does not allow RPS 

to meet this fundamental goal. In addition, separate tracking tools 

do not allow NRR managers to have a complete view of the 

inspection program. 
Audit of NRC’s Reactor Program System 

15

Recommendations

OIG recommends that the Executive Director for Operations: 

7. Develop a formal process for Reactor Program System 

regional counterparts to proactively gather user concerns prior 

to counterpart meetings. 

8. Conduct an annual Reactor Program System user survey to 

determine the needs of the users. 

E. TRAINING AND GUIDANCE NOT PROVIDED TO SYSTEM USERS

Inadequate training and outdated guidance have led to ineffective 

use of RPS. This has occurred because managers did not deem 

training and user guidance a priority. As a result, users are 

unaware of system functionality and there is an increased 

possibility of data error. 

Training Requirements

SDLCM requires that training be provided to information system 

users to allow them to learn a system and how it operates. In 

addition, Office of Management and Budget Circular No. A-130 

Appendix III, “Security of Federal Automated Information 

Resources”, which establishes policy for the management of 

Federal information resources, requires that users receive training 

to ensure they understand their roles and responsibilities with a 

system. 

Formal Training and Updated Manuals Needed

RPS training has been inadequate to ensure that users have a 

thorough understanding of the system. Moreover, written 

instructions concerning how to use the system are outdated. 

RPS users have not received formal training on the system since 

initial training was provided to RPS users in 1998. Of 47 users 

interviewed, 24 had never received training. More than 40 percent 

of those interviewed stated that training would be helpful. Many 

users stated that they learned how to use RPS solely from on-thejob training. When users have questions on how the system works, 

they said they ask a more experienced user. 
Audit of NRC’s Reactor Program System 

16

Furthermore, the RPS user manuals containing information on how 

to enter information in the system and generate reports are 

outdated. The system has been updated multiple times, but the 

system manuals were not kept current with the changes. 

Approximately 45 percent of the 47 users interviewed said a quick 

reference guide to the system would be useful. 

Users have not received training and the manuals have not been 

updated because the system administrator has not deemed it a 

priority. RPS managers said they rely on regional counterparts to 

provide training to users, but there has been no oversight to ensure 

that such training has been provided. In addition, regional 

counterparts have not expressed a need for training to the RPS 

administrator. 

Due to a lack of formal training and updated manuals, users are not 

fully aware of RPS capabilities. This lack of understanding has led 

some users to create separate tracking tools for functions that RPS 

already performs. In addition, lack of training and outdated 

manuals can lead to the dissemination of incorrect procedures and 

incorrect information being entered into the system. 

During the course of this audit, RPS managers began developing 

an online user tutorial on how to use main portions of the system. 

However, users will also need training and current guidance in the 

interim before this tutorial is completed. 

Recommendations

OIG recommends that the Executive Director for Operations: 

9. Implement a formal Reactor Program System training program 

that includes periodic refresher training and classes tailored to 

different user responsibilities. 

10. Complete the efforts to provide users with current system 

information in the form of online tutorials. 

Summary

Without improvements to RPS, NRC decisionmakers cannot have a 

complete picture of the nuclear power plant inspection program. To 

support adequate protection of public health and safety, agency 

decisonmakers need accurate information from RPS. At the time of 

this audit, RPS contained information that was inaccurate and 

incomplete. Reliance on faulty data or data that has not been 
Audit of NRC’s Reactor Program System 

17

locked down could result in poor decisionmaking. Improvements in 

RPS will strengthen protection over data stored in the system and 

better ensure that the system meets its operational requirements. 

Specifically, this will allow: 

3⁄4 RPS to be in compliance with system access control 

requirements. 

3⁄4 Accurate and timely information to be used in decisionmaking 

and when informing the public. 

3⁄4 The system help service to be more responsive to user needs. 

3⁄4 The system to meet the needs of the users. 

3⁄4 Users to better understand the system through training and 

guidance. 
Audit of NRC’s Reactor Program System 

18

IV. CONSOLIDATED LIST OF RECOMMENDATIONS 

OIG recommends that the Executive Director for Operations: 

1. Implement a tiered access level structure that allows users 

access to RPS modules based on the least privilege principle. 

This should include guidance on which users may receive what 

level of access. 

2. Implement a process for removing access rights from inactive 

Reactor Program System users. 

3. Implement a uniform quality control and Reactor Program 

System data entry review process that will ensure data accuracy 

and timeliness. 

4. Develop a process to lock down Reactor Program System 

inspection data fields after the inspection report has been 

issued. 

5. Formalize a process for handling Reactor Program System Help 

requests. 

6. Create performance metrics to assess timeliness and user 

satisfaction of Reactor Program System Help. 

7. Develop a formal process for Reactor Program System regional 

counterparts to proactively gather user concerns prior to 

counterpart meetings. 

8. Conduct an annual Reactor Program System user survey to 

determine the needs of the users. 

9. Implement a formal Reactor Program System training program 

that includes periodic refresher training and classes tailored to 

different user responsibilities. 

10. Complete the efforts to provide users with current system 

information in the form of online tutorials. 
Audit of NRC’s Reactor Program System 

19

V. AGENCY COMMENTS 

At an exit conference with agency senior executives held on 

March 2, 2005, NRC officials generally agreed with the report’s 

findings and recommendations. Subsequent to that meeting, OIG 

staff met with NRR staff to address specific issues and concerns 

needing further clarification and/or explanation. On March 31, 

2005, the Executive Director of Operations provided a formal 

response to this report in which he agreed with the final version of 

the report. Appendix B contains a copy of the agency’s written 

comments.
Audit of NRC’s Reactor Program System 

20

[Page intentionally left blank.] 
Audit of NRC’s Reactor Program System 

21

Appendix A 

SCOPE AND METHODOLOGY 

Auditors reviewed RPS to determine if (1) the system provides for 

the availability, confidentiality, and integrity of the data stored in the 

system and (2) RPS meets its required operational capabilities. 

This audit focused on RPS as an information technology system 

with regard to the system modules that handle inspection planning 

and results. 

The OIG audit team reviewed relevant criteria, including 

Management Directive 8.13, Reactor Oversight Process; NRC 

Inspection Manual, Chapter 0306, “Information Technology Support 

for the Reactor Oversight Process”; and Office of Management and 

Budget Circular No. A-130 Appendix III, “Security of Federal 

Automated Information Resources”. The audit team also reviewed 

the RPS Benefit-Cost Analysis and Security Plan. 

Auditors interviewed NRR staff responsible for the RPS system to 

understand the development and management of the system. 

Auditors also interviewed RPS users in all four NRC regions, 

including branch chiefs, senior project engineers, project engineers, 

reactor inspectors, resident inspectors, and resident office 

assistants to determine users interaction and satisfaction with the 

system. 

Auditors compared information from paper inspection reports to the 

corresponding information in the Item Reporting module of RPS in 

order to determine accuracy and timeliness of the data in RPS. 

Auditors reviewed a total of 646 data items contained in 32 

inspection reports. These reports reflected results drawn from four 

commercial nuclear power plants in each of NRC’s four regions. 

This work was conducted from June 2004 through January 2005, in 

accordance with generally accepted Government auditing 

standards and included a review of management controls related to 

audit objectives. The work was conducted by Beth Serepca, Team 

Leader; David Ditto, Senior Management Analyst; and 

Rebecca Underhill, Management Analyst. 
Audit of NRC’s Reactor Program System 

22

[Page intentionally left blank.] 

 
Audit of NRC’s Reactor Program System 

23

Appendix B 

FORMAL AGENCY COMMENTS