AUDIT REPORT

Audit of NRC’s Inspections, Tests, Analyses, and

Acceptance Criteria (ITAAC) Process

OIG-12-A-16 July 12, 2012

All publicly available OIG reports (including this report) are accessible through

NRC’s Web site at: 

http:/www.nrc.gov/reading-rm/doc-collections/insp-gen/
UNITED STATES

NUCLEAR REGULATORY COMMISSION

WASHINGTON, D.C. 20555-0001

OFFICE OF THE

INSPECTOR GENERAL

July 12, 2012

MEMORANDUM TO: R. William Borchardt

Executive Director for Operations

FROM: Stephen D. Dingbaum /RA/

Assistant Inspector General for Audits

SUBJECT: AUDIT OF NRC’S INSPECTIONS, TESTS, ANALYSES,

AND ACCEPTANCE CRITERIA (ITAAC) PROCESS

(OIG-12-A-16)

Attached is the Office of the Inspector General’s (OIG) audit report titled, Audit of NRC’s 

Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process. The audit 

objective was to assess NRC’s regulatory approach, through the ITAAC review process, 

to ensure that new nuclear power plants have been constructed and will be operated in 

conformity with the license, the provisions of the Atomic Energy Act, and the 

Commission’s rules and regulations. 

OIG met with NRC management officials and staff on May 22, 2012, at which time the 

agency provided informal comments to the draft report. OIG subsequently met with 

agency management and staff during a May 31, 2012, exit conference to discuss 

agency informal comments that OIG incorporated into the draft report as appropriate. 

NRC management and staff reviewed the revised draft report and generally agreed with 

the recommendations in this report. Furthermore, the agency opted not to provide 

formal comments for inclusion in this report.

OIG identified opportunities to improve the agency’s ITAAC process to include 

strengthening guidance, enhancing training procedures, improving oversight of 

construction tracking systems, developing a strategy for inspections at modular assembly 

facilities, and improving change management and communication/coordination practices 

between headquarters and Region II. The report contains recommendations intended to 

improve NRC’s ITAAC process.
If you have any questions, please contact me, at 415-5915 or RK Wild, Team Leader, at 

415-5948.

Attachments: As stated

cc: N. Mamish, OEDO

J. Arildsen, OEDO

K. Brock, OEDO

C. Jaegers, OEDO
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

i

EXECUTIVE SUMMARY

BACKGROUND

The next generation of nuclear power plants will be built under combined 

construction permit and operating licenses (COL) that reference designs that are 

certified by the Nuclear Regulatory Commission (NRC). The Office of New 

Reactors (NRO) is the lead organization for licensing new reactors and overseeing 

their construction.

A COL is issued under Title 10, Code of Federal Regulations (10 CFR), Part 52, a

process that combines the construction permit and operating license. NRC, in 

conjunction with industry, designed the Inspections, Tests, Analyses, and 

Acceptance Criteria (ITAAC) process to verify conformance with the COL as 

construction proceeds. ITAAC are a design-specific, pre-approved set of 

performance standards, grouped into families, which the licensee must meet to 

NRC's satisfaction. Families are composed of ITAAC that are related through 

similar construction processes, resulting products, and general inspection 

attributes. 

NRC conducts performance-based inspections throughout the construction period 

on a sample of ITAAC completed by the licensee to verify that they have been 

appropriately completed. Once the licensee determines that the acceptance criteria 

have been met for a particular ITAAC, it informs NRC by submitting an ITAAC 

closure notification to NRC for review. Not less than 180 days before the scheduled 

date for initial fuel load, the Commission will publish a Federal Register Notice of 

intended operation, which provides a 60-day period for the public to request a 

hearing on whether the facility as constructed complies, or on completion will 

comply, with the acceptance criteria in the COL. Pending no outstanding issues, 

the Commission is then expected to make an affirmative finding in accordance with 

10 CFR 52.103(g) that the acceptance criteria in the COL are met to allow the new 

reactor to begin operating. 

ITAAC inspections are performed primarily by the Center for Construction 

Inspection in Region II. Other Region II construction inspectors and NRC 

headquarters technical staff will participate in the inspection and oversight activities 

to better ensure that the facility conforms to the conditions of the COL. 

NRC vendor inspections are also performed as part of the ITAAC inspection 

process, in particular because a key characteristic of the current approach to new 

reactor construction is the use of modular assemblies, which are constructed offsite 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

ii

and shipped to the construction site for installation. Interdependence between the 

ITAAC and vendor inspection programs is an important aspect of NRC’s role in 

assuring that components destined for modular assemblies that will go into new 

reactors are manufactured to appropriate safety and regulatory standards. ITAAC 

inspection results will be recorded in an NRC-created and maintained electronic 

database referred to as the Construction Inspection Program Information 

Management System (CIPIMS). 

OBJECTIVE

The audit objective was to assess NRC’s regulatory approach, through the ITAAC 

review process, to ensure that new nuclear power plants have been constructed 

and will be operated in conformity with the license, the provisions of the Atomic 

Energy Act, and the Commission’s rules and regulations. 

RESULTS IN BRIEF

The agency established NRO in 2006 to oversee the regulatory activities 

associated with new reactor licensing under 10 CFR Part 52. To date, NRO staff 

have taken significant steps to employ a formalized approach for reviewing new 

reactor construction, such as implementing the ITAAC closure process. Staff have 

continued to strengthen the ITAAC closure process by developing and revising 

guidance and inspection procedures, creating a database tracking system, and 

working to identify and remedy issues associated with the ITAAC process. The 

Office of the Inspector General (OIG) identified opportunities to further improve 

aspects of the ITAAC process. The ITAAC process could benefit by NRC taking 

the following actions:

Strengthening guidance would enhance staff understanding of 

ITAAC requirements.

Designing ITAAC training programs through formalized training 

needs assessments would ensure the staff is receiving 

appropriate and useful training. 

Increasing oversight of CIPIMS development would help 

ensure:

 Development costs are tracked.

 Efficient development and implementation of future 

iterations of the system. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

iii

 Formalizing a strategy for inspection of components at modular 

assembly facilities would strengthen the ITAAC inspection 

program.

Improving change management and communication/coordination 

practices between headquarters and Region II. 

RECOMMENDATIONS

This report makes recommendations to improve the agency’s ITAAC process. A 

consolidated list of these recommendations appears in Section IV of this report.

AGENCY COMMENTS

On May 9, 2012, OIG issued the discussion draft of this report to the Executive 

Director for Operations. OIG met with NRC management officials and staff on May 

22, 2012, at which time the agency provided informal comments to the draft report. 

OIG subsequently met with agency management and staff during a May 30, 2012,

exit conference to discuss agency informal comments that OIG incorporated into 

the draft report as appropriate. NRC management and staff reviewed the revised 

draft report and generally agreed with the recommendations in this report. 

Furthermore, the agency opted not to provide formal comments for inclusion in this 

report.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

iv

TABLE OF CONTENTS

EXECUTIVE SUMMARY ............................................................................ i

ABBREVIATIONS AND ACRONYMS........................................................ v

I. BACKGROUND................................................................................. 1

II. OBJECTIVE ...................................................................................... 6

III. FINDINGS.......................................................................................... 6

A. Strengthening Guidance Would Enhance Staff 

Understanding of ITAAC Requirements............................. 7

B. Development and Implementation of CIPIMS Has

Been Delayed .................................................................. 14

C. A Formal Strategy for Inspection of Components at

Modular Assembly Facilities Would Strengthen

the ITAAC Inspection Program ........................................ 23

D. Coordination Between Headquarters and Region II

Could Be Improved.......................................................... 29

IV. CONSOLIDATED LIST OF RECOMMENDATIONS........................ 34

V. AGENCY COMMENTS.................................................................... 35

APPENDICES

A. EXAMPLE OF TIER 1 PROJECT MANAGEMENT 

METHODOLOGY ARTIFACTS................................................. 36

B. EXAMPLES OF ITAAC ............................................................. 37

C. OBJECTIVE, SCOPE, AND METHODOLOGY......................... 38
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

v

ABBREVIATIONS AND ACRONYMS

ASME American Society of Mechanical Engineers

CFR Code of Federal Regulations

CIPIMS Construction Inspection Program Information Management 

System

COL Combined Construction Permit and Operating License

ITAAC Inspections, Tests, Analyses, and Acceptance Criteria

NRC Nuclear Regulatory Commission

NRO Office of New Reactors

OIG Office of the Inspector General

OIS Office of Information Services

OPM Office of Personnel Management
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

1

I. BACKGROUND

The next generation of nuclear power plants will be built under combined 

construction permit and operating licenses (COL) that reference designs 

that are certified by the Nuclear Regulatory Commission (NRC). The Office 

of New Reactors (NRO) is the lead organization for licensing new reactors 

and overseeing their construction. In fiscal year 2012, NRC allocated 924 

full-time equivalents to new reactor activities and expenditures totaling

$265.4 million. 

In contrast to past licensing procedures under Title 10, Code of Federal 

Regulations (10 CFR) Part 50,1a COL is issued under 10 CFR Part 52. 

With Part 52, NRC established a process of combining the construction 

permit and operating license in order to eliminate unnecessary construction

or startup delays caused by preoperational licensing or litigation. 10 CFR 

Part 52 requires resolution of design and siting issues before the start of 

construction under a COL and continued NRC and licensee attention to 

assuring compliance with the COL during construction. The intent behind 

Part 52 is to provide a more efficient and systematic approach to

construction and NRC oversight (see Figure 1).

To achieve this purpose, NRC, in conjunction with industry, designed the 

Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) process to 

verify conformance with the COL as construction proceeds. 

 

1 Under 10 CFR Part 50, new reactor licensing is a two-stage process. Under this process, NRC would 

issue a prospective licensee with a construction permit, allowing them to begin construction immediately. 

During construction, NRC would inspect construction activities and determine whether or not to approve an 

operating license when the reactor was completed.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

2

Figure 1: 10 CFR Part 52 Reactor Licensing Schematic

Early Site 

Permits

Design 

Certifications

Combined 

License 

Review and 

Hearing

Reactor 

Construction

Reactor 

Operation

Verification of Inspection, 

Tests, Analyses, and 

Acceptance Criteria

Source: NRC

Inspections, Tests, Analyses, and Acceptance Criteria 

ITAAC are a design-specific, pre-approved set of performance standards 

that the licensee must meet to NRC's satisfaction.2 Through direct 

inspections and other methods, NRC must confirm that the licensee has 

met these performance standards, as set forth in the COL, before allowing 

the licensee to begin initial plant startup and operation. A typical new 

reactor design, such as the AP1000, has approximately 700 to 1,500

ITAAC that are grouped into families that are prepared and performed by 

the licensee during and after the construction process. The families are 

composed of ITAAC that are related through similar construction 

processes, resulting products, and general inspection attributes. NRC 

conducts performance-based inspections throughout the construction 

period on a sample of ITAAC conducted by the licensee to verify that they 

have been appropriately completed. Once the licensee determines that the 

acceptance criteria have been met for a particular ITAAC, it informs NRC 

 

2

That is, demonstrate that a plant has been constructed as-designed and licensed.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

3

by submitting an ITAAC closure notification for review. Not less than 180 

days before the scheduled date for initial fuel load, the Commission will 

publish a Federal Register Notice of intended operation, which provides a 

60-day period for the public to request a hearing on whether the facility as 

constructed complies, or on completion will comply, with the acceptance 

criteria in the COL. Pending no outstanding issues, the Commission is 

expected to make an affirmative finding in accordance with 10 CFR 

52.103(g) that the acceptance criteria in the COL are met to allow the new 

reactor to begin operating.3The overall licensing/construction process 

under 10 CFR Part 52 is depicted in Figure 2.

Figure 2: 10 CFR Part 52 Process Flow

103(g) finding

Pre-Construction Construction

Part 52 - Fitting the Pieces Together

COL Issued

Mandatory Hearing Optional Hearing 

on ITAAC 

completion

2

Manufacturing 

License 

Standard 

Design 

Approval

Standard 

Design 

Certification

Early Site 

Permit 

LWA

Load Fuel

Verification of 

Construction 

with ITAAC

Combined 

License 

Review, 

Hearing, 

and 

Decision

Public 

Involvement 

Opportunity

Note: LWA is the abbreviation for Limited Work Authorization.

Source: NRC 

 

3

A 10 CFR 52.103(g) finding is made by the Commission after the licensee closes all ITAAC, and the staff 

determines that all acceptance criteria are met. An affirmative finding by the Commission is required to 

authorize fuel load by the licensee. After an affirmative 52.103(g) finding, ITAAC cease to have regulatory

standing and technical specifications take over.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

4

Construction Inspection Program

ITAAC inspections are performed under the construction inspection 

program in accordance with Inspection Manual Chapters 2503,

Construction Inspection Program: Inspections of Inspections, Tests, 

Analyses, and Acceptance Criteria, and 2504, Construction Inspection 

Program—Inspection of Construction and Operational Programs, as well as 

other internal guidance documents, including those referred to by NRC 

staff as Strategy Documents. The construction inspection program was 

developed by the Division of Construction Inspection and Operational 

Programs in NRO and is being implemented primarily by the Center for 

Construction Inspection in Region II. The Region II construction resident 

inspectors are assigned to a new reactor site during the period prior to COL 

issuance to oversee the daily activities of the licensee and its contractors. 

Other Region II construction inspectors and NRC headquarters technical 

staff will participate in the inspection and oversight activities to better 

ensure that the facility conforms to the conditions of the COL. 

Vendor Inspection Program

The NRC vendor inspection program consists of vendor quality assurance 

inspections to ensure that products and services provided by 

manufacturers for United States-based reactors meet regulatory 

requirements. This is particularly important considering that a key 

characteristic of the current approach to new reactor construction is the use 

of modular assemblies, which are constructed offsite and shipped to the 

construction site for installation. Thus, the ITAAC process is not limited to 

onsite inspections of construction, but can include NRC inspections at 

vendor locations in foreign countries as well as at modular assembly 

facilities. Consequently, the interdependence between the ITAAC and 

vendor inspection programs is an important aspect of NRC’s role in 

assuring that components destined for modular assemblies that will go into 

new reactors are manufactured to appropriate safety and regulatory 

standards.4

 

4

For more on the activities of the vendor inspection program, see OIG-10-A-20, Audit of NRC’s Vendor 

Inspection Program, September 28, 2010.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

5

Construction Inspection Program Information Management System

ITAAC inspection results will be recorded in an NRC-created and 

maintained electronic database referred to as the Construction Inspection 

Program Information Management System (CIPIMS). During the previous 

period of nuclear power plant construction, there were a number of 

problems with NRC oversight.5 NRC conducted evaluations subsequent to 

the first generation of nuclear power plant construction which identified the 

need to strengthen NRC oversight of the construction process—in 

particular, the need to enhance construction inspection oversight programs. 

To that end, CIPIMS is intended to help plan inspections, and organize, 

manage, and generate inspection reports on information pertaining to (1) 

the ITAAC description, status, and closeout process; (2) inspection 

observation and tracking; (3) inspection report generation; and (4) 

assessment information. Information contained in CIPIMS will be used by 

headquarters in the review of ITAAC closure notices as well as by the

Center for Construction Inspection to determine inspection completion. 

 

5 During the 1970s and 1980s, NRC and its predecessor, the Atomic Energy Commission, oversaw the 

industry’s construction of the first generation of U.S. nuclear plants. Several of the construction projects

experienced significant problems related to design and construction quality resulting in the cancellation of 

several plants in various stages of construction. See OIG-09-A-17, Audit of NRC’s Oversight of 

Construction at New Nuclear Facilities, September 29, 2009.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

6

II. OBJECTIVE

The audit objective was to assess NRC’s regulatory approach, through the 

ITAAC review process, to ensure that new nuclear power plants have been 

constructed and will be operated in conformity with the license, the 

provisions of the Atomic Energy Act, and the Commission’s rules and 

regulations. Appendix C of this report contains information on the audit 

scope and methodology.

III. FINDINGS

The agency established NRO in 2006 to oversee the regulatory activities 

associated with new reactor licensing under 10 CFR Part 52. To date, 

NRO staff have taken significant steps to employ a formalized approach for 

reviewing new reactor construction, such as implementing the ITAAC 

closure process. Staff have continued to strengthen the ITAAC closure 

process by developing and revising guidance and inspection procedures, 

creating a database tracking system, and working to identify and remedy 

issues associated with the ITAAC process. 

However, OIG has identified opportunities to further improve aspects of the

ITAAC process. The ITAAC process could benefit by NRC taking the 

following actions:

Strengthening guidance would enhance staff understanding 

of ITAAC requirements.

Designing ITAAC training programs through formalized 

training needs assessments would ensure the staff is 

receiving appropriate and useful training. 

Increasing oversight of CIPIMS development would help 

ensure:

 Development costs are tracked.

 Efficient development and implementation of future 

iterations of the system. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

7

 Formalizing a strategy for inspection of components at 

modular assembly facilities would strengthen the ITAAC 

inspection program.

Improving change management and 

communication/coordination practices between headquarters 

and Region II. 

A. STRENGTHENING GUIDANCE WOULD ENHANCE STAFF 

UNDERSTANDING OF ITAAC REQUIREMENTS 

NRC staff have an inconsistent understanding of existing ITAAC guidance 

and procedures. This has occurred because current programmatic 

guidance and procedures lack clarity, and training is improvised. 

Consequently, NRC may not be able to ensure that new nuclear power 

plants have been constructed and will be operated in conformity with the 

license, the provisions of the Atomic Energy Act, and the Commission’s 

rules and regulations. 

Importance of Clear Understanding of Guidance 

NRC’s strategic guidance and procedures emphasize the importance of 

regulatory consistency and staff understanding of processes and 

procedures. NRC’s current Strategic Plan, for example, emphasizes 

regulatory consistency. Regulatory consistency includes the use of 

guidance and procedures to guide implementation of a new program, such 

as ITAAC. Consistent staff understanding of programmatic guidance and 

procedures is important for program success. 

Staff Have an Inconsistent Understanding of ITAAC Guidance and 

Procedures 

NRC staff do not uniformly understand how they should apply select 

guidance documents and procedures that describe the ITAAC inspection 

and closure process. This guidance includes the Strategy Documents6and 

relevant inspection guidance documents to help focus resources and 

identify ITAAC attributes for inspection. NRO also has a draft Office 

Instruction to assist staff during the ITAAC closure review process. Among 

 

6

For a brief description of Strategy Documents, see the text box entitled ―The ITAAC Strategy Documents‖ 

in the next section of this report.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

8

other things, the draft Office Instruction includes identification of the 

relevant program offices as well as a checklist to guide reviewers during 

the notification closure process. Nonetheless, there are varying staff 

perspectives on what constitutes a sufficient review for ITAAC closure. 

Specifically, OIG identified staff difficulties in understanding:

The use of the Strategy Documents,

The procedures for selecting non-targeted ITAAC for 

inspection, and

The ITAAC closure notification review procedures. 

Staff Understanding of 

Strategy Documents Is

Inconsistent 

NRC staff do not uniformly 

understand the purpose 

and application of the 

ITAAC Strategy 

Documents (see text box,

―The ITAAC Strategy 

Documents‖), and have also expressed concerns as to whether the 

documents are kept up-to-date. During discussions with OIG, staff cited 

concerns regarding the purpose and utility of the Strategy Documents. For 

example, some staff stated that the documents were ―high level‖ guidance 

to be used with Inspection Manual Chapters, a characterization at odds 

with their description as pre-inspection planning documents, as articulated 

to 

OIG by regional staff. One NRC manager noted that the process of how to 

use the documents was ―evolving.‖ Further, some staff indicated that some 

of the documents are out of date and may not contain the latest technical 

information. Specifically, some NRO staff who might be expected to 

provide technical input into the Strategy Documents stated that their views 

were not being taken into consideration and it was difficult for them to 

communicate their concerns.

The ITAAC Strategy Documents

The Strategy Documents were developed beginning in 

2006 by NRC staff for use by regional managers and 

inspection staff to prepare inspections of reactors 

under construction. Each Strategy Document is related 

to a specific ITAAC family in a given reactor type, such 

as the AP1000 series. Among other things, these 

documents provide examples of ITAAC families subject 

to inspection, references to appropriate inspection 

guidance documents, and descriptions of relevant 

structures, systems and components related to a given 

ITAAC family.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

9

Staff Inconsistently Understand the Process for Inspection of Non-Targeted 

ITAAC

Region II and NRO staff have differing perspectives on how to inspect nontargeted ITAAC (see text box, ―Targeted and Non-Targeted ITAAC‖). A 

senior Region II staff member said that any ITAAC not meeting the 

selection criteria for targeted ITAAC should be potentially suitable for 

inspection. ITAAC technical consultants have asserted that a structured 

approach should be 

taken to selecting nontargeted ITAAC for 

inspection, ranking the 

below-threshold ITAAC 

in a systematic fashion. 

However, according to 

one senior regional staff 

member, Region II staff 

consider a more random 

approach to ITAAC 

inspection selection to 

be appropriate. For 

example, one staff member expressed concern that the existing, structured 

approach articulated by headquarters does not allow enough latitude on 

the part of inspectors to look at non-targeted ITAAC. The staff member 

also noted that having too rigid a methodological approach could lead to 

the potential for predictability in inspections of non-targeted ITAAC. 

Specifically, employing a rigid methodological approach could allow for 

licensees to pre-determine which non-targeted ITAAC would be inspected 

by NRC. 

Targeted and Non-Targeted ITAAC

As part of the ITAAC inspection process, targeted 

ITAAC are identified through a ranking 

methodology. ITAAC are first reviewed for safety 

significance in conjunction with a probabilistic risk 

assessment. Then, all ITAAC are assigned a 

numerical value based on safety significance and a 

number of other factors, which is their ―value of 

inspection.‖ All ITAAC that meet a certain selection 

criteria are then identified as ―targeted,‖ and will be 

inspected during the construction process. Other 

ITAAC that fall below the threshold are designated 

as ―non-targeted,‖ but could still be subject to some 

level of inspection.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

10

ITAAC Closure Notification Review Procedures Are Not Well Understood

Some NRC staff do not 

fully understand some 

aspects of the ITAAC 

closure notification review 

process (see text box,

―The Closure Review 

Process‖). To facilitate 

this process, NRO has 

developed a draft Office Instruction titled, ―ITAAC Closure Verification 

Process,‖ to provide guidance to staff. However, NRO is presently 

reevaluating the draft Office Instruction based on management and 

regional staff concerns regarding the validity of some of the guidance 

criteria. These concerns include:

The ―2-4 hour‖ time frame cited in the draft guidance for the 

review of closure notices.

The lack of specification in describing the roles and 

responsibilities of the various staff who might be involved in a 

closure review.

Programmatic Guidance Lacks Clarity and Training Is Improvised 

There are two predominant factors contributing to NRC staff confusion 

regarding the ITAAC guidance and procedures: (1) a lack of specificity and 

clarity in the associated program guidance, and (2) the agency’s 

improvised approach to training program staff on the ITAAC process.

Guidance Documents Are Unclear

ITAAC program guidance is unclear, which has led to inconsistent staff

understanding of ITAAC guidance. OIG found some key areas, beginning 

with pre-inspection planning up through the ITAAC closure notification 

reviews, where existing guidance was unclear. Specifically:

Strategy Documents

There is currently no formal guidance regarding the use and maintenance 

of Strategy Documents. Although these documents are a key part of preinspection planning, they are currently monitored and updated by a primary 

point of contact, with input and updates occurring in an informal fashion. In 

The Closure Review Process

When an ITAAC is completed by the licensee, an 

ITAAC Closure Notice is submitted to NRO for a 

review. This review includes (1) verification of 

ITAAC closure notifications, as submitted by the 

licensee, and (2) instructions for reevaluation and 

opening of a closed ITAAC (if needed).
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

11

turn, regional and headquarters staff who could be providing useful input 

are not being consistently included in the revision process.

NRO’s Draft Office Instruction 

Justification for the ―2-4 hour‖ time estimates for closure notice

reviews provided in the draft Office Instruction have not been 

communicated to staff. During several discussions with NRC 

staff there was no consistent understanding for why ―2-4 hours‖

was chosen. 

The draft Office Instruction is ambiguous regarding the specific 

regional and headquarters staff roles and responsibilities 

associated with the closure notice review process. Frequently, 

the guidance simply states for a given office: ―Provides technical 

support for the verification of the closure of ITAAC.‖

Training for ITAAC Staff Is Improvised

NRO provides ITAAC training in an improvised fashion as determined by 

various managers in headquarters and Region II. OIG discussions with 

NRO and Region II management and staff, including NRO’s training 

coordinators, revealed that they have not systematically developed

training in accordance with the training and development policies listed 

on the NRC’s internal training Web site. These policies are based on the 

Office of Personnel Management’s (OPM) training guidance. OPM/agency

policies call for a training needs assessment to help agency and program 

managers identify appropriate subjects and methods for training. 

OPM’s direction for identifying agency training needs consists of a 

three-stage process that begins with an organizational assessment. The 

organizational assessment can be used to determine what skills, 

knowledge, and abilities an agency needs. Agency staff should then 

conduct an occupational assessment to identify the skills, knowledge, and 

abilities required for various staff. In the last stage, individual assessments 

are conducted to evaluate individual employee capabilities and whether or 

not additional training is required in order to do new or different work. 

Implementing a formal approach to training would greatly improve the 

range and consistency of knowledge on the part of NRO and Region II staff 

regarding their roles and responsibilities in the context of the overall ITAAC 

process.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

12

Shortcomings in Existing Guidance Could Hinder Staff Efforts During 

ITAAC Closure Process 

Inconsistent staff knowledge and understanding of the ITAAC inspection 

and closure processes, combined with ad hoc training practices, hinder the 

staff’s ability to identify and conduct relevant ITAAC inspections and 

closure notice reviews. This, in turn, could negatively impact NRC’s ability 

to provide reasonable assurance that a facility has been constructed and 

will be operated in conformity with the license, the provisions of the Atomic 

Energy Act, and applicable Commission rules and regulations. For 

example, a lack of formal guidance related to the Strategy Documents

could result in inconsistent application—or even misinterpretation—of 

existing guidance. Furthermore, as the documents are not formally 

controlled, there is currently no process to ensure that staff are using the 

most up-to-date versions of the Strategy Documents. This could potentially 

lead to inconsistencies in inspections, resulting in possible licensee 

perception of disparate treatment and might also negatively affect the 

Commission’s ability to appropriately make a 10 CFR Part 52.103(g) 

finding.

During the ITAAC closure review process, there is potential for rushed 

closure reviews resulting from misperceptions associated with the length of 

time needed (e.g., 2-4 hours) to complete a review with limited resources. 

Confusion regarding the time needed to conduct reviews could potentially 

contribute to a loss of reasonable assurance that plants will be built 

according to established rules and regulations as a result of rushed or 

inadequate closure reviews.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

13

Recommendations

OIG recommends that the Executive Director for Operations:

1. Develop formal guidelines governing the use of Strategy 

Documents in the context of construction inspection. 

2. Specify procedures for updating Strategy Documents and 

communicating changes in a systematic and coordinated 

fashion.

3. Provide specific guidance for inspection of non-targeted ITAAC 

and clarify the specific roles and responsibilities of various 

stakeholders during ITAAC closure notification review.

4. Develop and deliver training for the ITAAC process based on the 

results of needs assessments. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

14

B. Development and Implementation of CIPIMS Has Been Delayed 

CIPIMS is a necessary tool to document all ITAAC and vendor inspections, 

inform the agency’s ITAAC closure notice review, and support the 

Commission in making informed findings for permitting licensees to load 

fuel into a newly constructed reactor. However, CIPIMS was not available 

when ITAAC-related construction activities were begun at Vogtle, Georgia,

in March 2010.7 While CIPIMS was officially deployed in January, 2012,

just prior to NRC’s approval of the Vogtle COL on February 10, 2012, two 

software updates were already planned through the end of the fiscal year. 

Delays associated with the development and deployment of CIPIMS 

occurred due to insufficient oversight of database development. 

Consequently, NRC has spent approximately $2 million, some of which 

cannot be accurately accounted for, over a period of 5 years without 

developing a fully functional database for the ITAAC closure process. 

Additional delays and inaccurate accounting are likely to continue.

CIPIMS Is Needed To Track ITAAC Inspection Activity 

NRC has been developing CIPIMS to document inspection items and 

report the results of ITAAC-related and vendor inspections. CIPIMS will be 

used to collect inspection data from multiple sources (e.g., inspection and 

construction schedules) and generate reports. It will also provide critical

database support in the planning, execution, reporting, and reviewing of 

ITAAC inspections. In the ITAAC closure notification reviews, CIPIMS will 

be used by the agency as one means to verify that the prescribed 

inspections, tests, and analyses in the ITAAC were performed and that the 

acceptance criteria were satisfied. Thus, CIPIMS plays a critical role in 

NRC providing reasonable assurance of the safety and security of new 

reactors. In particular, the information collected in CIPIMS will be critical to 

NRC’s ability to verify ITAAC closure, which will provide the basis for the 

Commission’s 10 CFR Part 52.103(g) finding. 

 

7

Located near Augusta, Georgia, Southern Nuclear Company has begun construction on two AP1000 

Pressurized Water Reactors that are designated as Vogtle, Units 3 & 4.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

15

CIPIMS Was Not Available When First Needed and Remains Less than

Fully Operational 

CIPIMS was not available when ITAAC-related construction activities were 

initiated at Vogtle in March, 2010. CIPIMS development began in 2006 

with the creation of what the staff refers to as the ―CIPIMS Prototype‖ and 

legacy systems. CIPIMS 2.0 was implemented in January, 2012, nearly 5 

years after initial development efforts commenced. Even after deployment 

of the current version, the agency is planning to implement future versions 

of CIPIMS that include additional system information and system 

capabilities.

Insufficient Oversight of CIPIMS System Development 

NRO has not consistently applied the policies, principles, and best 

practices prescribed and endorsed by Federal guidelines and agency 

policy for systems development and investment management. Specifically, 

NRO did not provide consistent oversight, quality control, and executive 

review of the CIPIMS project to ensure that it has been developed and 

implemented in a well-managed and systematic manner. This lack of 

consistent oversight is exemplified by NRO’s not appointing a single project

manager thoroughly familiar with all aspects of the agency’s Project 

Management Methodology. 

Staff Not Familiar with Aspects of Project Management Methodology

In accordance with Federal and agency regulations, the agency’s 

information technology capital investments are subject to the requirements 

of the agency’s prescribed Project Management Methodology. However, 

OIG noted during the audit that NRC staff involved in the oversight of the 

CIPIMS project were unable to identify the tier designation of the 

information system—a central aspect of the Project Management 

Methodology. This lack of familiarity was evident in the staff’s inability to 

accurately identify the tier of the system and the corresponding artifacts or 

provide artifacts when requested by OIG. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

16

NRC staff were unfamiliar with the tier designation (see text box, ―Project 

Management Methodology Tier Designations‖) of the current CIPIMS 

system. For example, during a demonstration of the CIPIMS 2.0 database, 

OIG inquired of one of the designated project managers what tier the 

system was assigned. 

The project manager 

could not provide a 

definitive answer and 

deferred to the Office of 

Information Services

(OIS) contractor who 

stated that the database 

was categorized as Tier 

1 because CIPIMS 2.0 is

being developed under 

the Enterprise Project 

Management contract. 

OIG noted that the type 

of contract under which 

a system is being 

developed is not a factor in determining the tier designation. 

Consequently, OIG has concluded that because staff is not familiar with the 

tier of the CIPIMS system, the associated required artifacts (see text box,

―Project Management Methodology Artifacts‖) are not likely to be 

consistently addressed. 

To gain a better understanding of how NRO addressed Project 

Management Methodology requirements (artifacts), OIG requested 

pertinent CIPIMS 

project planning and 

management 

documents from NRO 

early in the audit. After 

several weeks, OIG was 

provided with a project 

organizational chart 

(see Figure 3), the 

CIPIMS System 

Requirements

Specifications, and an incomplete project plan. OIG issued a second 

Project Management Methodology Tier 

Designations 

In accordance with agency requirements, 

information technology systems are to be 

designated by the agency as Tier 1, Tier 2, or Tier 3 

based on several factors, including the complexity of 

the system, security requirements, and projected 

development cost. The tier designation is significant 

as it guides the development and implementation of 

the system. Additionally, it determines the 

requirements that must be addressed to ensure that 

the system is appropriately and systematically 

managed. Consequently, it is expected that the 

project manager of an information technology

system should be familiar with the appropriate tier 

designation and the associated requirements.

Project Management Methodology Artifacts 

The tier-specific requirements are referred to as 

―artifacts‖ and include documentation such as a 

project plan, system requirements, and a risk 

assessment. Depending on the tier designation, 

certain artifacts are required while others are 

considered optional. There are a significant number 

of artifacts that are required for a Tier 1 system. 

See Appendix A.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

17

request for information; however, the agency provided no additional 

documentation. Based on the information that was provided, OIG 

determined that the required artifacts for a Tier 1 system were not being 

appropriately maintained by NRO. In fact, NRO had only addressed some 

of the required artifacts and they were not consistently managed. 

Consequently, OIG concluded that the prolonged development and 

unaccounted costs associated with CIPIMS were, in part, a direct result of 

staff’s lack of familiarity with the requirements of the Project Management 

Methodology.

Figure 3: NRO CIPIMS Project Team Organization as of August 3, 2011

Source: NRC8

 

8 CCI (Center for Construction Inspection); CIPB (Construction Inspection Program Branch); DCI (Division 

of Construction Inspection); DCIP (Division of Construction Inspection and Operational Programs); DCP 

(Division of Construction Projects); DNRL (Division of New Reactor Licensing); EPM (Enterprise Project 

Management); NOEP (Organizational Effectiveness and Productivity Branch)
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

18

Lack of Clearly Designated Project Manager

A project manager plays a significant role in overseeing a project and is 

responsible for the planning, controlling, and monitoring of the project. 

However, during CIPIMS development there was not a clearly designated 

single project manager to oversee the CIPIMS project and coordinate staff 

and system end-user input. Specifically, during the course of the audit, 

staff could not consistently identify a single project manager for CIPIMS 

and noted there were ―multiple project managers‖ overseeing the project 

that were located both in Region II and headquarters. Further, not having a 

clearly designated project manager to coordinate input made participating 

in developing the CIPIMS database difficult. This contributed to a general 

confusion within the agency as to who was responsible for overseeing the 

CIPIMS project, as indicated previously in Figure 3. 

Delays and Unaccounted Costs in CIPIMS System Development 

The lack of adequate project oversight has contributed to CIPIMS 

development delays. Moreover, during a prolonged period of time, the 

agency did not consistently or appropriately track and verify the costs 

associated with the development of the database as required by Federal 

and agency guidance. Consequently, NRC has spent approximately $2 

million, with some expenditures lacking in supporting cost details, over a 

period of 5 years. Although inspectors started using CIPIMS at the end of 

January 2012, the database is still being updated and was not yet a fully 

functional database for the ITAAC closure process.

Prolonged Delays

NRC’s CIPIMS database has been in development for a prolonged period 

of time. Specifically, the development of CIPIMS began in 2006 with the 

creation of what the staff refers to as the ―CIPIMS Prototype‖ and legacy 

systems. In January, 2012, CIPIMS 2.0 was implemented nearly 5 years 

after initial development efforts commenced. OIG has recently learned that 

the agency is planning to implement future versions of CIPIMS that include 

additional system information and system capabilities.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

19

Staff involved in the development of CIPIMS attributed the developmental 

delays to a variety of factors involving the coordination of regional and 

headquarters staff. For example, staff noted that poor communication 

between Region II and headquarters, ―chronic IT issues,‖ disagreement 

over system requirements and platforms, and differing expectations all 

contributed to the lengthy delay in developing and implementing a usable 

ITAAC database. Most notably, one staff member stated that at various 

phases of development it appeared that the ―politics and management 

issues within [the Division of Construction Inspection and Operational 

Programs],‖ were impeding the progress of the project. Together, these 

factors effectively retarded the progress of CIPIMS development and 

significantly increased the cost of developing the database. 

Development Costs Tracking Issues 

The agency could not provide sufficient information documenting the 

development costs for the CIPIMS database. The documentation the 

agency provided was incomplete and inconsistent, and was analogous to 

summary data and cost estimates. For example, in some instances the 

agency provided a spreadsheet of approximated costs without any 

supporting documentation, such as invoices or signed contractor 

timesheets. As a result, OIG could not independently verify the CIPIMS 

development costs cited by the agency. The agency’s inability to 

appropriately track CIPIMS development costs is evident in how CIPIMSrelated invoices are processed and reconciled to the provisions of the 

Enterprise Project Management contract. Table 1 illustrates the various 

iterations of the CIPIMS database, including the associated development 

activities and costs. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

20

Table 1: CIPIMS Development Timetable and Estimated Costs

Years CIPIMS System Development History Estimated 

Cost

2007-08 CIPIMS Legacy Developed in NRR;

Based on RPS

Developed under 

Lockheed Martin 

Contract

$275,000

2008-09 CIPIMS Legacy Work continued by NRO

Developed under 

Lockheed Martin 

contract

$415,250

2008-11 CIPIMS 1.0 Development 

& Operations and 

Maintenance

Migrated from RPS 

Legacy System to 

Microsoft application

Attempted 

implementation during 

17-month period

Developed under the 

EPM contract

$942,550

 

2009-11 CIPIMS Lite Region II developed 

stop-gap system based 

on Microsoft AccessTM in 

absence of functioning 

CIPIMS

$0 (Cost 

included as part 

of staff TAC 

code.)

2011-12 CIPIMS 2.0 Major overhaul and 

redesign of system. 

Developed under EPM 

contract

Operational as of 

January 9, 2012, while 

additions and upgrade 

work continues

$394,560 plus 

an added 

$450,000 

(estimate) for 

latest work up to 

present

Total: 

$2,477,360

Source: OIG analysis of NRC data as of January, 2012

Note: EPM is the abbreviation for Enterprise Project Management; RPS is the abbreviation for Reactor 

Program System; NRR is the abbreviation for Office of Nuclear Reactor Regulation
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

21

Invoice Processing Inconsistencies

According to NRO staff, CIPIMS 1.0 and 2.0 were developed under the 

agency’s Enterprise Project Management contract and are currently comanaged by NRO and OIS.

9

 Under this arrangement, NRO provides daily 

oversight of the contractor and manages the development of the CIPIMS 

database while OIS is charged with disbursing payment on behalf of NRO 

for the services rendered by the contractor under the Enterprise Project 

Management contract. According to one staff member, the arrangement 

between NRO and OIS did not allow the NRO project manager to be fully 

involved in reviewing the development costs associated with CIPIMS. For 

example, an NRO staff member with project manager experience noted 

that under the Enterprise Project Management contract only OIS receives 

invoices from the contractor and that these invoices are not routinely 

forwarded to the NRO project manager for review. The NRO staff member

indicated that—although they are responsible for the day-to-day oversight 

of the CIPIMS project, including overseeing the contractor—the entire 

invoicing process is controlled by OIS. The individual characterized this 

arrangement as ―a disaster.‖ OIG has identified in previous audits that cost 

tracking for information technology systems has been a problem within the 

agency.

10

 

 

9 NRC’s Enterprise Project Management tool is a Microsoft product that helps organizations manage 

capital projects by providing a framework for scheduling, tracking, and updating projects that help to 

optimize resources, minimize costs, manage scope, and deliver on time. NRO currently uses Enterprise 

Project Management to electronically support the agency’s Licensing Program Plan through Microsoft 

ProjectTM, Project Web AccessTM, and the SharePointTM tools.

10 See, for example, ―Results of the Audit of the United States Nuclear Regulatory Commission’s Financial 

Statements for Fiscal Years 2005 and 2004,‖ OIG-06-A-01, November 10, 2005.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

22

Lack of Clarity in Enterprise Project Management Contract Task Orders

The Enterprise Project Management contract is broadly written and did not 

always contain language specific to the development of the CIPIMS 

database. Specifically, the initial Enterprise Project Management contract

contained task orders that were generically worded and provided only 

broad categorizations of the types of operations and maintenance activities 

allowable under the contract. A task order did not exist that was specific to 

CIPIMS development, making it unclear which task orders were to be used 

to track the costs associated with development of the database. Contract 

and project manager staff explained that CIPIMS-related work could be 

charged under multiple task orders depending on the type of work being 

performed. OIG questioned whether it was possible for staff to account for 

CIPIMS development costs if multiple task orders were being inconsistently 

utilized. In effect, actual costs could not be tracked and OIG could not 

separate CIPIMS work from that charged to the generalized task orders. 

Shortly after OIG raised this issue, the Enterprise Project Management

contract was modified to include a CIPIMS-specific task order.

Recommendations

OIG recommends the Executive Director for Operations:

5. Designate a specific Project Manager for CIPIMS as required by 

the Project Management Methodology.

6. Develop and maintain project artifacts for CIPIMS as required in 

the Project Management Methodology. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

23

C. A Formal Strategy for Inspection of Components at Modular 

Assembly Facilities Would Strengthen the ITAAC Inspection Program 

The extent to which NRC’s inspection activities for components 

manufactured and assembled offsite are sufficient for ITAAC verification is 

unclear. This is because NRC has not developed a formal strategy for 

evaluating what inspections are necessary at modular assembly facilities 

located away from the plant construction site. Consequently, NRC may not 

be able to provide reasonable assurance that new nuclear plants are 

constructed in accordance with NRC requirements.

Importance of ITAAC Construction Oversight and Inspection 

NRO and Region II’s Center for Construction Inspection construction 

oversight programs are intended to enable the safe and secure 

construction of nuclear reactors in accordance with approved designs and 

safety regulations. NRO’s Division of Construction Inspection and 

Operational Programs, in conjunction with the Center for Construction 

Inspection, are responsible for overseeing inspections of reactors under 

construction as well as for inspecting structures, systems and components 

that are being assembled off-site. An example of such an off-site facility, 

Shaw Modular Solutions in Lake Charles, Louisiana—which is currently 

providing components to a nuclear power plant construction site located 

834 miles away in Georgia—is shown in Figure 4. 

Figure 4. Shaw Modular Solutions Facility, Lake Charles, Louisiana

Source: NRC
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

24

Role of Division of Construction Inspection and Operational Programs

NRO’s Division of Construction Inspection and Operational Programs 

develops policy and provides overall program management and planning 

for the construction inspection program for new commercial nuclear power 

plants. Activities include oversight of licensee performance assessment, 

allegations, and enforcement activities.

Role of Region II Center for Construction Inspection

Region II’s Center for Construction Inspection conducts oversight activities 

designed to ensure the safety of future operations at new nuclear facilities 

by monitoring licensees’ efforts to construct the facilities according to 

approved design criteria, while using appropriate practices and quality 

materials. Guidance developed by Region II also addresses the 

importance of inspecting structures, systems and components that are 

assembled off-site at modular construction facilities. 

Importance of Inspecting Components Assembled Off-Site 

Strategy Documents, developed by Region II, provide detailed guidance to 

construction inspectors for pre-inspection planning activities. These

documents also address the role of vendor inspections. For example, 

Strategy Documents associated with the pipe welding and 

supports/restraint ITAAC specifies that some pipe segments, pipe 

supports, and restraints will be fabricated off-site at modular fabrication or 

vendor fabrication sites.11 NRC Strategy Documents note that there is the 

possibility NRC will be required to inspect some items ―at the manufacturer‖ 

rather than after arrival at the construction site, which would also require 

Region II construction inspectors to coordinate with NRO vendor 

inspectors. As shown in Figure 5, many modular assemblies are complex 

in nature and the need for careful inspection planning is important. 

Therefore, ITAAC inspections should be coordinated with vendor 

inspections connected with modular fabrication facilities.

 

11 Appendix B provides examples of ITAAC.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

25

Figure 5: Installation of Modular Assemblies

Source: NRC

NRC’s Approach to Inspection of Modular Assembly Facilities Not 

Fully Evaluated

There is currently no assurance that NRC’s inspection activities for 

components manufactured and assembled offsite as part of modular 

assemblies are sufficient for ITAAC verification based on NRC’s ability to 

inspect them. OIG reached this conclusion after inquiring of staff the extent 

to which resident inspectors would be needed at modular assembly 

facilities, and how vendor inspections would be factored into the ITAAC 

review process. Although NRC has taken initial steps to identify targeted 

ITAAC that can be inspected at modular assembly facilities, NRC still does 

not have formal documentation that outlines a comprehensive strategy for 

the inspection of modular assembly facilities.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

26

NRC’s Approach to Modular Assembly Inspection Could be 

Strengthened

NRC has not developed a formal strategy for evaluating what inspections 

at modular assembly facilities are necessary to support the ITAAC 

program. That is, staff are making decisions without the benefit of a 

structured and systematic evaluation to determine what systems, structures 

and components assembled or manufactured off-site need to be inspected 

prior to arrival at the construction site in support of the ITAAC closure 

process. 

For example, during the audit, OIG observed NRC management and 

inspection staff at a meeting with vendor staff at Shaw Modular Solutions. 

The site visit was intended, in part, to help NRC staff better understand 

how the modular assembly facility fit into the overall inspection scheme for 

new construction inspections. After the visit, NRC staff still had 

unanswered questions regarding access to vendor data, the off-site 

assembly construction schedules, and even the precise make-up of some 

of the assemblies. Though NRC has taken initial steps to identify ITAAC 

that could be inspected at modular assembly facilities through updated 

procedures in the Strategy Documents, NRC has not developed a formal 

strategy to identify targeted and non-targeted ITAAC that could require 

some degree of inspection prior to arrival at a construction site. 

Role of the Vendor Inspection Program Not Formalized

NRC has not fully determined how the NRO vendor inspection program 

should be integrated into NRC’s overall ITAAC oversight activities. The 

vendor inspection program is an important part of the overall inspection 

process. Senior headquarters and regional staff stated that vendor 

inspections inform ITAAC. OIG noted that NRO has taken preliminary 

action to address the role of vendor inspections with respect to ITAAC 

inspections. Precisely how vendor inspections inform ITAAC inspections is 

not clear, however, and neither senior regional nor headquarters managers 

could articulate how this is being accomplished. OIG also learned that 

Region II construction inspection staff are relying, in part, on NRO vendor 

inspectors to inform Region II of any findings related to ITAAC that they 

identify during vendor and modular facility inspections. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

27

To help understand the role of vendor inspectors in the ITAAC oversight 

process, OIG requested documentation reflecting a staff analysis of the 

role of vendor inspections in ITAAC-related oversight activities. NRC staff 

stated that no such analysis was conducted and, thus, no formal strategy 

for integrating and coordinating vendor inspections has been developed. 

As one senior NRO program staff member told OIG, from the standpoint of 

the vendor inspection program, the ITAAC inspections are considered as a 

"bonus" and not a necessity. 

Figure 6: NRC Vendor Inspection Team at Shaw Modular Solutions 

Facility

Source: NRC

Opportunities To Identify Safety-Significant ITAAC Problems May Be 

Missed 

Without a formal strategy to guide the evaluation of modular components 

and to delineate the role of vendor inspectors in the ITAAC oversight 

process, the agency may be missing opportunities to identify safetysignificant problems. Specifically, opportunities for detecting safetysignificant issues relating to targeted ITAAC during new reactor 

construction may be limited. The role of modular construction and vendor 

inspections is vitally important, in part because vendors—such as Shaw 

Modular Solutions—have already been assembling safety-related 

components destined for use at new reactor construction sites. Region II 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

28

staff acknowledged that some components would have to be inspected at 

Shaw Modular Solutions, or they might not be visible upon arrival onsite. 

For example, certain modular assemblies have welding that, once 

assembled, would be inaccessible to resident inspectors when the module 

arrived on-site, which could prevent an accurate evaluation of a given 

ITAAC. Furthermore, both ITAAC and vendor inspection teams may be 

missing opportunities to identify problems early on and communicate them 

in a timely manner to those resident inspectors who can best make use of 

the information. For example, during meetings at Shaw Modular Solutions, 

licensee representatives revealed that some modular assembly activity was 

going on at sites other than Lake Charles. However, NRC staff were not 

aware of the activity and were not informed through vendor inspections. 

Without the full details of what is being assembled at locations other than 

the construction site, NRC may not be able to reasonably assure that the 

plant is being built in accordance with its license, the provisions of the 

Atomic Energy Act, and the Commission’s rules and regulations.

Recommendations

OIG recommends that the Executive Director for Operations:

7. Determine what systems, structures and components assembled 

or manufactured off-site need to be inspected prior to arrival at 

the construction site in support of the ITAAC closure process. 

8. Develop a formal vendor inspection coordination strategy to 

ensure appropriate inspections of modular systems, structures 

and components assembled or manufactured off-site. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

29

D. Coordination Between Headquarters and Region II Could Be Improved 

There is a lack of sustained coordination both within headquarters and 

between headquarters and Region II for ITAAC program related activities 

and interactions. These problems would have been minimized if NRO and 

Region II had in place formalized change management processes to 

address communications and coordination problems in a changing 

environment. Without a formalized change management process, 

coordination and communication concerns between NRO and Region II will 

continue to proliferate, with the potential to affect the agency’s safety 

mission.

The Importance of Coordination and Communication to Program 

Success 

Key agency guidance documents note the importance of employing 

effective business methods—such as sustained coordination and clear 

communication—to achieve and maintain operational excellence. For 

example, the agency’s current Strategic Plan references coordination and 

communication as effective business methods. Change management, in 

particular, is vital to enhancing an organization’s effectiveness and 

efficiency. Key aspects of change management include realizing the need 

for increased training and systematically documenting new procedures to 

assist staff throughout the transition process. Furthermore, NRO’s internal 

guidance documents note the key role that interdependence and 

coordination play in its ability to effectively attain its mission. For example, 

in NRO Vision of Success, NRO recognizes the importance of 

interdependent relationships within the agency, and the subsequent need 

for clearly delineating roles and responsibilities. Additionally, the Division 

of Construction Inspection and Operational Programs Web page states its 

approach to the oversight of new reactor construction is based on 

coordination between and within headquarters and Region II. 

Management best practices also acknowledge the significant impact 

coordination has on an organization’s ability to attain its goals. According 

to recognized management best practices, ―...the goal of the enterprise 

cannot be successfully obtained without it [coordination].‖ Management 

best practice states that effective coordination includes sound planning, a 

clear delineation of responsibility, and effective communication within an 

organization. Furthermore, recognized management best practices 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

30

emphasize change management as entailing thoughtful planning, sensitive 

implementation of change, and consultation with, and involvement of, the 

people affected by the changes. An organization’s management is 

responsible for facilitating and enabling change by preparing staff to

understand the purpose of the change. This requires management to 

effectively communicate and involve staff early on in the process to 

facilitate a sentiment of trust and openness. Management must clearly 

articulate to affected staff what their role is in facilitating the change, 

including the new roles and responsibilities they will be expected to perform 

and how they will interact with other entities within the organization. 

Furthermore, management is responsible for identifying potential barriers to 

change and developing plans to address them, in addition to providing a 

systematic transition plan and schedule to guide the organizational change.

Lack of Sustained Coordination and Communication Hinders ITAAC 

Process 

During OIG’s review of NRC’s ITAAC oversight process, development of 

CIPIMS, and oversight of modular components constructed and assembled 

offsite, auditors noted a lack of sustained coordination between 

headquarters and Region II staff. Specifically, OIG observed a lack of 

sustained coordination during the development and revision of ITAAC 

guidance documents, the creation of the CIPIMS database, and the 

interaction between the vendor inspection and ITAAC programs.

ITAAC Closure Guidance 

There is inconsistent coordination between headquarters and regional staff 

in the development and revision of ITAAC-related guidance and 

procedures. Specifically, the process by which the ITAAC Strategy 

Documents are revised is informal, occurs on an irregular basis, and does 

not lend itself to broad staff participation. During the audit, OIG learned 

that revisions to the Strategy Documents are managed via a limited-access 

Region II Microsoft SharePointTM site. Furthermore, the revision process is 

not formalized and occurs on a sporadic, ad hoc basis. Staff also 

expressed significant concern that there did not seem to be any formal 

method for submitting changes to the Strategy Documents, which were 

essentially overseen by a single point of contact and not subject to an 

objective vetting process.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

31

Development of CIPIMS Database

The development and implementation of CIPIMS has been lengthy and 

characterized by inadequate coordination at various levels within and 

between headquarters and regional staff. This lack of coordination 

occurred throughout the development of all iterations of CIPIMS, but was 

most apparent in the development of pre-CIPIMS 2.0 variants. Specifically, 

there was a lack of coordination in how the differing priorities, system 

change requests, and user needs were managed during the database’s 

development. 

Differing Priorities Hampered Coordination. Coordination 

between headquarters and Region II during various phases in 

the development of the CIPIMS 1.0 database was hampered, 

according to staff, as a result of the shifting priorities at 

headquarters and Region II that were driven by resource and 

staff availability issues. 

System Change Requests Not Coordinated. The manner in 

which system change requests were managed illustrates a lack 

of coordination during CIPIMS development. Specifically, there 

was no formal process for submitting, evaluating and 

implementing the system change requests proposed by staff. 

User Needs Were Not Sufficiently Coordinated. Both Region II 

and headquarters staff agree that communication was lacking 

and often led to additional confusion. In some cases, 

headquarters and regional staff indicated that they felt their input 

was not being adequately taken into account. 

Management Efforts to Address Coordination and 

Communications Issues. During the course of OIG’s audit, NRO

managers and staff began taking steps to employ a more 

systematic approach toward the development and 

implementation of the CIPIMS 2.0 database. For example, an 

NRO staff member involved with CIPIMS told OIG that a 

formalized change request management system has been 

implemented for CIPIMS 2.0. This includes taking steps to 

ensure that user needs and requirements are identified and met 

through implementing a more formalized communication 

mechanism. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

32

Interaction of the Vendor Inspection and ITAAC Programs

There is limited coordination between the NRO’s vendor inspection 

program and Region II’s Center for Construction Inspection despite their 

interdependent inspection activities. Specifically, communication between 

the organizations is informal and occurs inconsistently. For example, staff 

noted that information sharing primarily occurs via e-mail or word-of-mouth 

on an irregular basis. Additionally, OIG found evidence that although NRC 

is aware of coordination issues, there is not yet in place a formal process 

for how NRC plans to coordinate construction and vendor inspections 

involving modular assembly facilities. For example, ITAAC inspection staff 

have noted their reliance on the results of the vendor inspections to better 

inform ongoing and future ITAAC inspection work. In contrast, an NRO 

senior manager stated that conducting any work involving or relating to 

ITAAC is a ―bonus,‖ thereby implying that the ITAAC process is not 

routinely considered in the conduct of vendor inspections.12

NRO and Region II managers have identified this issue as an impediment 

to the efficacy of both the ITAAC and the vendor inspection programs. As 

a result, according to NRO management, they are taking steps to ensure 

that the coordination between the programs is strengthened by defining a 

formalized means of communication and developing clearly written 

guidance that addresses the expectations regarding interaction, and 

identifies the specific roles and responsibilities of each program. 

Change Management Process Not Employed 

Communication and coordination problems would have been minimized if 

NRO and Region II had in place formalized change management 

processes to mitigate barriers to change in a dynamic environment. A 

senior NRO manager indicated that they were unaware if any change 

management process had been used to facilitate the most recent NRO 

office reorganization performed in late 2011. During discussions with OIG, 

 

12 Nor did it appear that ITAAC were specifically considered during NRC’s vendor inspection of Shaw 

Modular Solutions in November 2011. The resulting inspection report (Inspection Report No. 

99901401/2011-202) does not mention ITAAC. This is due, in part, to the traditional role of vendor 

inspections to review the vendor’s compliance with 10 CFR Part 50, Appendix B ―Quality Assurance 

Criteria for Nuclear Power Plants and Fuel Reprocessing Plants.‖ Given such a restricted inspection 

focus, anything specific that the vendor inspection team might find regarding an individual ITAAC appears 

unlikely and exceptional and, indeed, might legitimately be characterized as a ―bonus.‖
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

33

managers and staff cited instances of communication and coordination 

problems that were exacerbated by staff turnover. The problems included 

communication of inspection-related activity, as well as communication and 

coordination associated with the development of CIPIMS. In addition, as 

changes to project management occurred, they were not always 

communicated or coordinated effectively to staff, leading to what one NRO 

manager characterized as having to start things all over again.

Continued Coordination Problems May Affect Safety Mission 

Given the changing budget and regulatory environment, it is prudent to 

assume NRC may undergo further change. Some of this change may be a 

result of the projected development and licensing of new reactors, including 

small modular and other advanced reactors, ongoing license renewals, as 

well as the natural evolution and revision of the agency’s regulations in 

response to external events. Without an effective change management 

program in place, continued problems with coordination and 

communication can be expected to continue to adversely affect the ITAAC 

process and the degree to which new plants are safely constructed.

Recommendations

OIG recommends that the Executive Director for Operations:

9. Develop a plan to correct the existing communication and 

coordination problems between headquarters and Region II that 

resulted from lack of a change management process for ITAACrelated issues. 

10. Develop and implement a change management process to 

address future change in the ITAAC process that can create 

barriers to effective communication and coordination.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

34

IV. CONSOLIDATED LIST OF RECOMMENDATIONS

OIG recommends that the Executive Director for Operations:

1. Develop formal guidelines governing the use of Strategy Documents 

in the context of construction inspection. 

2. Specify procedures for updating Strategy Documents and 

communicating changes in a systematic and coordinated fashion.

3. Provide specific guidance for inspection of non-targeted ITAAC and 

clarify the specific roles and responsibilities of various stakeholders 

during ITAAC closure notification review.

4. Develop and deliver training for the ITAAC process based on the 

results of needs assessments.

5. Designate a specific project manager for CIPIMS as required by the 

Project Management Methodology.

6. Develop and maintain project artifacts for CIPIMS as required in the 

Project Management Methodology. 

7. Determine what systems, structures and components assembled or 

manufactured off-site need to be inspected prior to arrival at the 

construction site in support of the ITAAC closure process. 

8. Develop a formal vendor inspection coordination strategy to ensure 

appropriate inspections of modular systems, structures and 

components assembled or manufactured off-site.

9. Develop a plan to correct the existing communication and 

coordination problems between headquarters and Region II that 

resulted from lack of a change management process for ITAACrelated issues. 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

35

10. Develop and implement a change management process to address 

future change in the ITAAC process that can create barriers to 

effective communication and coordination.

V. AGENCY COMMENTS

On May 9, 2012, the Office of the Inspector General (OIG) issued the discussion 

draft of this report to the Executive Director for Operations. OIG met with NRC 

management officials and staff on May 22, 2012, at which time the agency 

provided informal comments to the draft report. OIG subsequently met with 

agency management and staff during a May 30, 2012, exit conference to discuss 

agency informal comments that OIG incorporated into the draft report as 

appropriate. NRC management and staff reviewed the revised draft report and 

generally agreed with the recommendations in this report. Furthermore, the 

agency opted not to provide formal comments for inclusion in this report.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

36

Appendix A

Example of Tier 1 Project Management Methodology Artifacts

PMM Project Management Artifacts T1*

Transmittal Memorandum

Vision Document R

Business Case

(The Business Case Package includes the following:

Transmittal Memorandum

Vision Document

Executive Summary

System Requirements Specification

Business Case

Project Management Plan

System Architecture Document and

Security Categorization Package) 

R

Project Management Plan

(Project Schedule, Budget/Cost, Quality Assurance Plan, etc). R

Project Schedule 

Project Schedule Templates R

Software Development Plan Template (if outsourced; includes schedule, 

and other supporting management approaches) R

Risk/Issues List R

Source: NRC *T1 = Tier 1 Project; R = Required by PMM 

Note: OIG requested documentation from NRO staff to provide support that required Tier 1 artifacts had been 

addressed during the development of CIPIMS. To date, OIG has received no documentation that provides sufficient 

evidence that Tier 1 artifacts were appropriately addressed.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

37

Appendix B

Examples of ITAAC 

Source: NRC

This chart contains two examples of ITAAC. The first example (row 1) is a simple ITAAC and 

involves verifying whether the gripper of the fuel handling mechanism functions properly through 

visual inspection. The second example (row 2) is a complex ITAAC and involves examining

welds in components located on the reactors’ pressure boundary to determine conformance to 

ASME code requirements.

Design 

Commitment

Inspection, Test and/or 

Analyses

Acceptance 

Criteria

1. The Remote Manual 

and Fuel Handling 

Mechanism gripper 

assemblies are designed 

to prevent opening while 

the weight of the fuel 

assembly is suspended 

from the gripper.

The Remote Manual and 

Fuel Handling Mechanism 

will be tested by operating 

the open controls of the 

gripper while suspending a 

dummy fuel assembly.

The gripper will not 

open while 

suspending a dummy 

test assembly.

2. Pressure boundary 

welds in components 

identified in Table 2.1.3-1 

as American Society of 

Mechanical Engineers 

(ASME) Code Section III 

meet ASME Code Section 

III. 

Inspection of the as-built 

pressure boundary welds will 

be performed in accordance 

with ASME Code Section III. 

A report exists and 

concludes that ASME 

Code Section III 

requirements are met 

for Non-destructive 

examination of 

pressure boundary 

welds.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

38

Appendix C

OBJECTIVE, SCOPE, AND METHODOLOGY

OBJECTIVE

The audit objective was to assess NRC’s regulatory approach, through the 

ITAAC review process, to ensure that new nuclear power plants have been 

constructed and will be operated in conformity with the license, the 

provisions of the Atomic Energy Act, and the Commission’s rules and 

regulations. 

SCOPE

The audit focused on assessing NRC’s ITAAC inspections and evaluations

associated with the ITAAC closure process. The audit reviewed current 

agency guidance and practices pertinent to ITAAC, and also included a 

review of Federal regulations and agency policies regarding new 

information technology projects.

We conducted this performance audit at NRC headquarters (Rockville, 

Maryland) from April 2011 through January 2012. The audit also included 

site visits to Region II offices in Atlanta, Georgia; the Vogtle Electric 

Generating Plant construction site in Waynesboro, Georgia; and Shaw 

Modular Solutions modular assembly facility in Lake Charles, Louisiana. 

Internal controls related to the audit objective were reviewed and analyzed. 

Throughout the audit, auditors were aware of the possibility or existence of 

fraud, waste, or misuse in the program.

METHODOLOGY

OIG reviewed Federal regulations and agency guidance regarding the 

ITAAC inspection and closure process and new information technology 

projects, including:

10 CFR Part 52 – Licenses, Certifications, and Approvals for 

Nuclear Power Plants.
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

39

Technical Report on the Prioritization of Inspection 

Resources for Inspections, Tests, Analyses, and Acceptance 

Criteria (ITAAC).

Proposed Rule: Requirements for Maintenance of 

Inspections, Tests, Analyses, and Acceptance Criteria (RIN 

3150-AI77).

Inspection Procedure 40600 – Licensee Program for 

Inspections, Tests, Analyses, and Acceptance Criteria 

(ITAAC) Management.

Inspection Manual Chapter 2506 – ―Construction Reactor 

Oversight Process General Guidance and Basis Document.‖

Inspection Manual Chapter 2503 – ―Construction Inspection 

Program: Inspections of Inspections, Tests, Analyses, and 

Acceptance Criteria (ITAAC).‖

AP1000, ITAAC Family 01F, ―Foundations and Building 

Design/Fabrication Requirements Inspection Strategy.”

NEI 08-01 – Industry Guideline for the ITAAC Closure 

Process Under 10 CFR Part 52, Revision 4, Draft E.

Simulated ITAAC Closure and Verification Demonstration 

Final Report.

Office of Management and Budget Circular No. A-130 –

Management of Federal Information Resources.

NRC Management Directive 2.8 – Project Management 

Methodology (PMM).

OIG interviewed NRC managers and staff involved in the ITAAC inspection 

and closure process. These interviews included an NRO Division Director;

NRO and Region II Branch Chiefs; NRO Project Managers; Resident 

Construction Inspectors; and OIS, and Office of the Chief Financial Officer 

personnel. Additionally, OIG interviewed Shaw Modular Solutions and 

Southern Nuclear Company representatives. Overall, OIG conducted 

interviews with the key program and regional representatives necessary to 

obtain their insights into the agency’s planning and management of the 

ITAAC closure process.

We conducted this performance audit in accordance with generally 

accepted Government auditing standards. Those standards require that 

we plan and perform the audit to obtain sufficient, appropriate evidence to 

provide a reasonable basis for our findings and conclusions based on our 

audit objective. We believe that the evidence obtained provides a 
Audit of NRC’s Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC) Process

40

reasonable basis for our findings and conclusions based on our audit 

objective. 

The audit work was conducted by R. K. Wild, Team Leader; Kevin 

Nietmann, Senior Technical Advisor; Jacki Storch, Audit Manager; Timothy 

Wilson, Senior Management Analyst; and Larry J. Weglicki, Senior Auditor.