**NRC INSPECTION MANUAL**	COLP

INSPECTION PROCEDURE 41502

NUCLEAR POWER PLANT SIMULATION FACILITIES

PROGRAM APPLICABILITY:  2504

41502-01	INSPECTION OBJECTIVES

01.01	To verify the plant-referenced simulator is designed and implemented so that it is sufficient in scope and fidelity to be suitable for conduct of the evolutions listed in 10 CFR 55.45(a)(1) through (13), and 55.59(c)(3)(i)(A) through (AA), as applicable to the design of the reference plant and allows for the completion of control manipulations for operator license applicants.

01.02	To verify continued assurance of simulator fidelity is maintained by the facility licensee per 10 CFR 55.46 (d).

01.03	To verify the suitability of the plant-referenced simulator for use, if applicable, by the facility licensee to meet the control manipulation requirements in 10 CFR 55.31(a)(5), and utilizes models relating to nuclear and thermal-hydraulic characteristics that replicate the most recent core load in the nuclear power reference plant for which a license is being sought; and to verify that simulator fidelity has been demonstrated and met per 10 CFR 55.46(c)(2)(i) and (ii).  

01.04	To verify that a simulator facility for a “new reactor” has sufficient fidelity to be designated a ‘plant reference simulator’ and conforms to objectives 1.01 through 1.03 above.  (A ‘new reactor’ for the purposes of this procedure means as a reactor plant under construction with no core operating history.)

41502-02	INSPECTION REQUIREMENTS AND INSPECTION GUIDANCE

02.01	Background.

In March 1987, the Commission amended its regulations requiring all utilization facility licensees to have a simulation facility (either a “plant-referenced simulator” or an “other-than-a-plant-referenced-simulator”) by May 26, 1991, for use in administering NRC operating tests and licensed operator requalification training (52 FR 9460; March 25, 1987).

In October 2001, the NRC amended its regulations to permit facility licensee applicants for operator’s licenses to fulfill a portion of the required experience prerequisites by manipulating a plant-referenced simulator as an alternative to manipulating the controls of the actual nuclear power plant (66 FR 52667; October 17, 2001). 

In March 2011, the staff issued Revision 4 of Regulatory Guide (RG) 1.149, “Nuclear Power Plant Simulation Facilities for Use in Operator Training, License Examinations, and Applicant Experience Requirements.”  The RG describes methods acceptable to the staff for complying with those portions of the Commission’s regulations [10 CFR 55.46] associated with approval or acceptance of a nuclear power plant simulation facility for use in operator and senior operator training and license examination operating tests and for meeting applicant experience requirements.  The RG also explains that NRC accepts and endorses industry consensus standards such as ANSI/ANS-3.5-2009, “Nuclear Power Plant Simulators for Use in Operator Training and Examination.”  

Facility licensees are expected to adhere to a single version of ANS-3.5, Nuclear Power Plant Simulators for Use in Operator Training [and Examination], for simulator technical requirements to ensure the plant-referenced simulator meet the scope and fidelity requirements of 10 CFR 55.46.  All licensees will have committed to a version of ANSI/ANS-3.5 as a way of ensuring the plant-referenced simulator meets the requirements of 10 CFR Part 55.  This commitment is usually contained in the licensee’s plant licensing basis.  

In preparation for the simulation facility inspection, inspectors should review the specific ANSI/ANS 3.5 version [and associated RG] to which the licensee is committed.  These documents should be used in conjunction with the specific guidance and inspection requirements contained in this inspection procedure (IP).

This IP can be used for new or operating reactors.  All or part of this IP may be used as necessary to supplement IP 71111.11, Licensed Operator Requalification Program.  This IP is to be performed by a qualified Operator Licensing Examiner.

02.02	General Inspection Requirements.

	a.	Goals

		1.	The intent of these inspection activities is to verify conformance with the simulator requirements specified in 10 CFR 55.46 and to assess the adequacy of the facility licensee's simulation facility for use in operator training, license examinations, and applicant experience requirements as described in 10 CFR 55.31(a)(5), if applicable, (if used for applicant experience requirements).  

		2.	Additionally, these inspection activities assess the effectiveness of the facility licensee's process for continued assurance of simulator fidelity with regard to performance testing, and identifying, reporting, correcting, and resolving simulator hardware and modeling discrepancies via a simulator corrective action program.

	b.	 Inspection Requirements

		1.	Verify simulator performance testing as defined by the Commission [meaning simulator testing conducted to verify performance as compared to actual or predicted reference plant performance] is accomplishing the following objectives:

(a) 10 CFR 55.46(d) requires licensees conduct simulator performance testing throughout the life of the simulation facility in a manner sufficient to ensure simulator fidelity.  The testing demonstrates the plant-referenced simulator utilizes models relating to nuclear and thermal-hydraulic characteristics that replicate the most recent core load (the most recent core load is only required if the plant referenced simulator is used to meet control manipulation requirements in 10 CFR 55.31(a)(5)); 

(b) The testing demonstrates the plant-referenced simulator performance mimics the reference plant performance (actual or expected), 

(c) All hardware discrepancies and discrepancies identified from scenario validation and performance testing are documented in the simulator corrective action process,

(d) Uncorrected performance test failures are available for NRC review, 

(e) Performance test results are retained for four years after the completion of each performance test or until superseded by updated test results.

		2.	If the licensee is committed to scenario-based testing (SBT), i.e., ANSI/ANS 3.5, 1998 or 2009, review licensee administered scenario-based testing, see Attachment 2, “Plant-referenced Simulator Scenario Based Testing Methodology Checklist”.  The SBT should confirm parameters/alarms/automatic actions to be evaluated are identified, that the SBT is electronically capturing that data and evaluating the data against reference plant performance, and attesting that simulator performance was satisfactory or documenting discrepancies.    

3.	Review simulator physical fidelity (i.e., the degree of similarity between the simulator and the reference plant control room, such as physical location of panels, equipment, instruments, controls, labels, and related form and function), especially regarding recent modifications implemented in the control room.  Simulation scope and fidelity must be sufficient to allow conduct of evolutions in 10 CFR 55.45 and 55.59.

	For operating reactors, if any simulator fidelity inspection issue impacts licensed operator requalification, refer to Inspection Manual Chapter (IMC) 0609, Appendix I, and “Operator Requalification Human Performance SDP” for determining the significance of the issue.

For new reactors, the physical fidelity is confirmed by reviewing the Inspection, Test, Analysis and Acceptance Criteria (ITAAC) as discussed in 02.02.b.6 below and by performing Attachment 1, "Checklist for Evaluating Plant-referenced Simulator Operating under 10 CFR 55.46(c) and (d)," of this inspection procedure. 

		4.	Determine the suitability of the licensee's simulation facility for use in operator training, license examinations as described in 10 CFR 55.46, and applicant experience requirements as described in 10 CFR 55.31(a)(5), if applicable (if used for experience requirements).  This assessment is to be carried out on a sampling basis and is not intended to be an exhaustive inspection of the licensee's simulation facility.  

(a) Refer to 10 CFR 55.46, "Simulation facility," for specific requirements regarding plant-referenced simulators.  Use Attachment 1 of this inspection procedure, "Checklist for Evaluating Plant-referenced Simulator Operating Under 10 CFR 55.46(c) and (d)," to determine, on a sampling basis, if the plant-referenced simulator is acceptable for use in operator examinations, and to satisfy experience requirements.  Assess any negative responses to the questions and if necessary, seek NRR/NRO program office guidance on the issue.  

(b) Verify that the facility licensee has a configuration management control process in place to maintain the fidelity of the plant-referenced simulator.

(c) Assess simulator discrepancies and the safety impact of any negative training caused by these discrepancies.

(d) If the simulator was used for any license candidate(s) to meet the control manipulation requirements in 10 CFR 55.31(a)(5), the determination that the simulator was deemed suitable for the control manipulations per the criteria in 10 CFR 55.46(c)(2)(i) and (ii) should be documented in the examination report. 

(e) New reactor inspections should evaluate the areas listed above as well as verify that discrepancies in the Issue Resolution Verification portion of the ITAAC are assessed per the configuration management control process and entered in the licensee’s simulator corrective action program.  The results from verification and integrated validation activities performed in accordance with the Design Certification and documented is the Licensee’s ITAAC closure letter(s) may be cited as the bases for satisfactorily addressing applicable 10 CFR 55.46 requirements.  

		5.	Review open (uncorrected) and also sample closed simulator discrepancy reports, including simulator modeling and hardware discrepancies and discrepancies identified from scenario validation and performance testing, to assess the effectiveness of the licensee's process for: problem identification and prioritization, reporting, evaluation, schedule for implementing timely corrective actions, and corrective actions (where simulator discrepancies could result in negative training, corrective actions should include training on actual plant behavior).  Verify that the licensee adequately captures simulator problems and that corrective actions are performed, tracked, trended and completed in a timely fashion commensurate with the safety significance of the item.  Implicit in this review is confirmation that the corrective actions taken have adequately addressed the hardware/software issue and did not introduce new errors into the modeling.   

		6.	For New Reactors Only - In addition to 2.02.b.1 through 5 above, verify the simulation facility is a plant reference simulator suitable for operator licensing examinations and the control manipulation requirements in 10 CFR 55.31(a)(5), if applicable, by performing the following: 

(a) If this is the initial simulator inspection for the facility, verify that the ITAAC has been completed through the Integrated System Validation (ISV), if not contact the NRO Program Office (some portions may be able to be performed in parallel as determined by the program office.)  

(b) Confirm physical fidelity by performing Attachment 1, “Checklist for Evaluating Plant-referenced Simulator Operating under 10 CFR 55.45(c) and (d),” of this inspection procedure.

(c) Verify that the facility licensee has a configuration management control process in place to maintain the fidelity of the plant-referenced simulator.  Assess simulator discrepancies and the safety impact of any negative training caused by these discrepancies.  Verify simulator discrepancies are assessed per the configuration management control process and entered into the licensee’s simulator corrective action program

(d) This simulator inspection shall be performed prior to conduct of each operator licensing operating examination to verify the simulator facility is a plant-referenced simulator suitable for operator licensing examinations and the control manipulation requirements of 10 CFR 55.31(a)(5).

02.03	Sample Selection.

a. Obtain a copy of the simulator management and configuration control procedures – i.e., the procedure that governs simulator discrepancies  and performance testing.  Discrepancies should be prioritized, corrected and resolved in accordance with the management and configuration control procedure.

1. Request a list of all open simulator discrepancies– Review for relevancy to 10 CFR 55.31, 55.45 and 55.59 operator actions.

2. Request a list of closed discrepancy reports for the last 12 months and review for the same criteria as a.1 above.		

3. Evaluate summary list of uncorrected simulator discrepancies and list of corrected discrepancies over the last twelve months.  Specifically evaluate discrepancies which directly impact operator immediate actions, abnormal and/or emergency operating procedural actions, and/or plant automatic actions which contributed to unsatisfactory simulator test performance/test results failures. 

b. Request a list of simulator performance tests.  Select simulator performance tests (and associated test results) in each of the following categories: 

Note: ANSI/ANS-3.5-2009 standard is used in the referenced sections below.  If the facility uses an earlier standard then the section number referenced may vary.

1. Steady State/Transient tests - Evaluate at least one simulator steady-state performance test; and two simulator transient performance tests (such as reactor trip from >50% power and loss of all core flow).  Sample size (1/2), ANSI/ANS reference sections 4.1.3.1, 4.4.3.1 

2. Normal evolutions – Evaluate at least one simulator normal evolution test. Sample size (1), ANSI/ANS reference section 4.1.3.2.

3. Malfunction tests – Review at least five malfunction tests.  This review should determine the malfunction cause if not apparent.  At least three should relate to nuclear or thermal-hydraulic operating characteristics and two should relate to extensive logic/interlock performance.  (Suggestions include loss of feed, main turbine trip, loss of off-site power and main steam line malfunctions.)  Note: If scenario based testing is used, the malfunction tests are covered in the SBT.  Sample size (5), ANSI/ANS reference section 4.1.4.

4.  Simulator scenario-based testing, if applicable - Evaluate at least one (two for new reactors if no requalification scenarios are available) NRC initial license examination scenario(s); evaluate at least one licensed operator requalification annual examination scenario (may be N/A for new reactors); and evaluate at least one simulator scenario used for applicant reactivity control manipulation experience, if applicable.  Sample size (1/2 in each area), ANSI/ANS reference section 4.4.3.2.

c. Simulator reactor core performance testing - Evaluate at least two simulator reactor core performance test(s) (if licensee’s core performance testing is all contained in one procedure then evaluate two segments) related to nuclear and thermal-hydraulic performance.  Examples of items to review include heat balance, determination of reactivity coefficients, 1/M plots, and control rod worth using permanently installed instrumentation.  New reactors may be evaluated against predicted values.  Sample size (2), ANSI/ANS reference section 4.4.3.3.

d. Simulator post-event testing – Was conducted when a reference unit event generates relevant data for evaluating simulator performance.  If multiple events have occurred, select at least one.  Sample size (1), ANSI/ANS reference section 4.4.3.4.

41502-03	INSPECTION GUIDANCE 

03.01	Conduct of Performance Testing.

Verify simulator performance testing has been conducted in a manner sufficient that simulator fidelity has been demonstrated so that significant control manipulations are completed without procedural exceptions, simulator performance exceptions, or deviations from the approved training scenario sequence.  Refer to the specific ANSI/ANS 3.5 revision committed to by the licensee (this information is to be provided by the licensee).  

		a.	Evaluate simulator operability testing.  Test results should meet acceptance criteria and confirm any discrepancies are processed in the licensee’s simulator corrective action program.  (See 2.03.b.1 for sample recommendations) 

1.	Confirm simulator operability testing has been conducted once per reference unit fuel cycle by testing: (New reactors may be evaluated using predicted values).  

				(a)	Simulator steady-state performance.

	(b)	Verify the simulator has been operated from cold shutdown to rated full power and back (credit may be taken for training scenarios provided the evolutions are performed with the reference unit procedures and are evaluated and documented).  

				(c)	Simulator transient performance for a benchmark set of transients.

b.	Evaluate malfunction tests.  Verify malfunction testing has been conducted in a manner sufficient that simulator fidelity has been demonstrated so that significant control manipulations are completed without procedural exceptions, simulator performance exceptions, or deviations.  Test results should meet acceptance criteria and confirm any DRs are processed in the licensee’s simulator corrective action program.  Sample per 2.03.b.3.

		c.	Evaluate simulator scenario-based testing, if applicable to licensee. Attachment 2, Plant-Referenced Simulator Scenario Based Testing Methodology Checklist, has the following attributes: (Reg. Guide 1.149 endorses NEI 09-09, Rev. 1).  Sample per 2.03.b.4.

			1.	Review each sampled SBT for attributes in 03.01.c.2 through 7 below and verifies those SBT attributes are documented in each SBT. 

			2.	Confirm the simulator produced the expected reference unit response without significant performance discrepancies, or deviations from the approved scenario sequence.  

			3.	Confirm the simulator allows the use of reference unit procedures without exception.

			4.	Observable changes in simulated parameters correspond in direction to the change expected from actual or best estimate response of the reference unit to the malfunction.

			5.	The simulator did not fail to cause an alarm or automatic action if the reference unit would have caused an alarm or automatic action under identical circumstances.

			6.	The simulator did not cause an alarm or automatic action if the reference unit would not cause an alarm or automatic action under identical circumstances.

			7.	Verify the following attributes are documented in the SBT performance test record:

(a) The initial conditions, description of the scenario, and perturbations used to induce the transient.  

(b) Listing of key parameters checked and assertion that there were no unexpected changes.  

				(c)	Listing of key alarms and key automatic actions occurring and assertion that they would be expected for the scenario.  

				(d)	Assertion that no unexpected alarms and automatic actions occurred.  

8.	Assess the SBT test results.  Test results should meet acceptance criteria and confirm any discrepancies are processed in the licensee’s simulator corrective action program.  

			9.	Verify that any fidelity issues which impact operator actions have been identified by the licensee and the discrepancies have been entered into their corrective action program for correction and/or resolution. 

			10.	Verify licensees SBT affirmations of test results are substantiated.  

		d.	Evaluate sufficiency of simulator reactor core performance testing.  

	Sample per 2.03.c.

			1.	Verify simulator reactor core performance testing has been conducted for the most recent reference unit fuel cycle (if simulator used for meeting applicant experience requirements in 10 CFR 55.31(a)(5)), (new reactors may be evaluated using predicted values.)  Verify simulator core testing is performed in accordance with the reference unit procedures and compared and demonstrated to replicate the response of the reference unit.  

			2.	Verify the simulator reactor core response and performance meets the reference unit procedure(s) acceptance criteria.

			3.	Test results should meet acceptance criteria and confirm any discrepancies are processed in the licensee’s simulator corrective action program.

e. Verify post-event simulator testing (N/A for new reactors).  Sample per 2.03.d.  

1.	Post event simulator testing should be conducted when a reference unit event generates relevant data for evaluating simulator performance.  If multiple events have occurred, select at least one.  This testing:

(a) 	Considered the sequence of events, operator actions, and was performed in accordance with reference unit procedures

(b) Demonstrated that post-event simulator testing was conducted and relevant data compared to ensure that the simulator is capable of reproducing the response of relevant reference unit parameters within the scope of simulation.  

		f.	Verify discrepancies identified from this inspection are entered into the simulator corrective action process.  Follow up with the licensee regarding any simulator discrepancies that prevent overall satisfactory acceptance of a performance test.

03.02	Simulator Performance Testing Acceptance Criteria.

a. Verify the sampled simulator performance tests/test results met the following acceptance criteria (if necessary, refer to the specific ANSI/ANS-3.5 standard applicable to the simulator being inspected for additional acceptance criteria that may apply):

1.	allows the use of applicable reference unit procedures (i.e., startup test procedure, surveillance procedure, or operating procedure) including procedural acceptance criteria.

	b.	Any observable parameter change corresponds in direction to those expected from actual or best estimate response of the reference unit.

	c.	The simulator shall not fail to cause an alarm or automatic action if the reference unit would cause an alarm or automatic action under identical circumstances.

	d.	The simulator shall not cause an alarm or automatic action if the reference unit would not cause an alarm or automatic action under identical circumstances.

03.03	Simulator Performance Record Retention.

Verify the results of simulator performance tests are retained for a minimum of four years after completion of each performance test (four years may not be available for new reactors, in this case test records should go back to testing done since site delivery) or until superseded by updated test results.

41502-04	RESOURCE ESTIMATE

For planning purposes, the direct inspection effort to complete this inspection procedure is estimated to be 40 hours of onsite inspection, consistent with the scope of the planned regional initiative(s) to be performed. 

It is expected that the actual hours required to complete the inspection may vary from the estimate.  More effort may be needed if major modeling changes/enhancements have taken place such as complete primary thermo/hydraulic model replacement or simulator platform rehost.  The inspection hours allocated for the inspection procedure are an estimate for budgeting purposes.  The hours expended during an inspection should be tailored for the facility licensee and accurately recorded.

41502-05	REFERENCES

10 CFR 55, Operators’ Licenses.

Inspection Manual Chapter 2504, “Construction Inspection Program: Inspection of Construction and Operational Programs”. 

Inspection Manual Chapter 2515, Light-Water Reactor Inspection Program -- Operations Phase.

ANSI/ANS 3.5-2009, Nuclear Power Plant Simulators for Use in Operator Training [and Examination] (includes prior versions).  

Regulatory Guide, 1.149, Rev. 4, Nuclear Power Plant simulation Facilities for Use in Operator Training, License Examinations, and Applicant Experience Requirements (includes prior versions).  

41502-06	PROCEDURE COMPLETION

This IP is considered complete when the required number of minimum samples specified in 2.03 above is complete and a determination has been made as to suitability of the plant-referenced simulator is of sufficient scope and fidelity for the purposes of 10 CFR 55.

Note:  Attachments 1 and 2 are for inspector use only and are not part of the inspection 	       report.  

END

****

**ATTACHMENT 1**

**CHECKLIST FOR EVALUATING PLANT-REFERENCED**

**SIMULATORS OPERATING UNDER 10 CFR 55.46(c) AND (d)**

(Circle yes [Y] or no [N].  Answer questions based upon sampling inspection.  It is not intended that these questions be answered on the basis of exhaustive inspection.)

Y / N 1.	Does the plant-referenced simulator demonstrate expected plant response to operator input and to normal, transient, and accident conditions to which the simulator has been designed to respond? [§55.31(a)(5) and §55.46(c)(1)]

Y / N 2.	Is the plant-referenced simulator sufficient in scope and fidelity with the reference plant to allow conduct of the evolutions listed in 10 CFR 55.45(a)(1) through (13),as applicable to the reference plant? [§55.46(c)(1)(i)]

Y / N 3.	Is the plant-referenced simulator sufficient in scope and fidelity with the reference plant to allow conduct of the evolutions listed in 10 CFR 55.59(c)(3)(i)(A) through (AA), as applicable to the reference plant? [§55.46(c)(1)(i)]

Y / N 4.	Is the plant-referenced simulator designed and implemented in a manner that allows for the completion of control manipulations for operator license applicants? [§55.46(c)(1)(ii)]

Y / N 5.	If the plant-referenced simulator is used to meet experience requirements for applicants for operator and senior operator licenses, does the plant referenced simulator utilize models relating to nuclear and thermal-hydraulic characteristics that replicate the most recent core load in the nuclear power reference plant for which a license is being sought?  The phrase "most recent" means the current core or if the reference plant is in a refueling outage, the core just previous to the outage. [§55.31(a)(5)], [§55.46(c)(1), and §55.46(c)(2)(i)]

Y / N 6.	Has the plant-referenced simulator fidelity been demonstrated so that significant control manipulations are completed without procedure exceptions, simulator performance exceptions, or deviation from the approved training scenario sequence? [§55.46(c)(2)(ii)]

Y / N 7.	Has there been any lapse in the facility licensee conducted simulator performance testing throughout the life of the simulation facility? [§55.46(d)(1)]

Y / N 8.	Are the results of performance testing retained for four years after the completion of each performance test or until superseded by updated test results? [§55.46(d)(1)] 71111.11, App C C-2

Y / N 9.	Are modeling and hardware discrepancies and discrepancies identified from scenario validation and from performance testing being corrected? [§55.46(d)(2)]

Y / N 10.	Are results of any uncorrected performance test failures that may exist at the time of the operating test or requalification program inspection available for NRC review? [§55.46(d)(3)]

Y / N 11.	Has simulator fidelity been maintained such that license application, examination, and test integrity are consistent with 10 CFR 55.49 requirements? [§55.46(d)(4)]

****

**ATTACHMENT 2**

**PLANT-REFRENCED SIMULATOR**

**SCENARIO BASED TESTING METHODOLOGY CHECKLIST**

**SBT Number: 		Date Validated:**

Issue Date:  10/16/12	- 14 -	41502

Issue Date:  10/16/12	- 1 -	41502

Attachment 3 -****Revision History for IP 41502

Issue Date:  10/16/12	Att3-1	41502