**NRC INSPECTION MANUAL**	IRIB

INSPECTION PROCEDURE 71152

# PROBLEM IDENTIFICATION AND RESOLUTION (PI&R)

Effective Date: January 1, 2024

PROGRAM APPLICABILITY: IMCs 2515 A, 2201 A

CORNERSTONES:	ALL

INSPECTION BASES:	See IMC 0308, Attachment 2

# SAMPLE REQUIREMENTS:

* Inspection Time spent assessing PI&R as part of the baseline procedure attachments should be charged to the corresponding baseline procedure.

# 71152-01	Inspection Objectives

01.01	To confirm that licensee problem identification and resolution (PI&R) programs are complying with NRC regulations and applicable industry standards.

01.02	To evaluate the effectiveness of the licensee’s PI&R program in identifying, prioritizing, evaluating, and correcting problems.

01.03	To confirm the licensee’s appropriate use of industry and NRC operating experience.

01.04	To evaluate the effectiveness of licensee audits and self-assessments.

01.05	To gauge supplemental response when Reactor Oversight Process (ROP) Action Matrix thresholds are crossed.

01.06	To confirm licensees have established a safety conscious work environment (SCWE).

01.07	To follow-up on corrective actions for selected previously identified compliance issues (e.g., non-cited violations (NCVs)).

01.08	To conduct follow-up of individual issues through a more focused review.

01.09	*To confirm that licensees are identifying and placing potential 10 CFR 21—"Reporting of Defects and Noncompliance” issues into the (CAP) and appropriately evaluating them.* [C3]

01.10	To review the licensee’s trending of long-standing, unresolved problems. [C1]

# 71152-02	general guidance

Licensee PI&R programs include all methods of identifying, prioritizing, evaluating, and correcting deficiencies. These programs commonly include but are not limited to the licensee corrective action and work management programs. However, any other licensee program or process that addresses deficiencies with risk significant systems, structures and components (SSCs), compliance with regulatory requirements (to include security, emergency preparedness and radiation protection), or adherence to licensee commitments and standards is within the scope of the PI&R program. Resident inspector routine PI&R review guidance is contained in Inspection Manual Chapter (IMC) 2515, Appendix D, “Plant Status.”

PI&R inspections should follow a performance-based approach to the maximum extent possible. Evaluate products and results of the licensee’s PI&R program, including the use of operating experience (OpE), assessments, and audits. Focus on the results associated with risksignificant issues across all the cornerstones. For issues that are determined to be performance deficiencies, inspectors should evaluate the causes that relate to cross-cutting aspects for insights on performance. Inspections performed in accordance with this procedure should focus on the identification, prioritization, evaluation, and corrective actions for risksignificant issues; programmatic and procedural elements associated with PI&R should be of concern when they contribute to risk-significant issues.

The intent of this inspection procedure (IP) is to provide insights into licensee performance in the PI&R area based upon a performance-based review of corrective action issues, operating experience, and self-assessments/audits. More detailed programmatic reviews of licensee performance in the PI&R area will be conducted during supplemental inspections if established performance thresholds are crossed.

Notify the Office of Nuclear Reactor Regulation (NRR), Division of Reactor Oversight, Quality Assurance and Vendor Inspection Branch when issues related to potential vendor or supplier deficiencies are reviewed. Include the vendor’s name and provide a brief description of the deficiency and component, as appropriate. [C3]

## 02.01	Sample Selection Guidance

Inspectors should seek the broadest range of examples from all the cornerstones when selecting inspection samples. Any failures, malfunctions, deficiencies, deviations, defective material and equipment, and nonconformances that either have been placed into a licensee’s PI&R program or inspectors believe should be placed into a licensee’s PI&R program in order to comply with any regulatory or applicable industry standards (including self-imposed) are within the scope of this procedure. Inspectors can obtain insights for determining appropriate samples from reviewing previous inspection reports and end of cycle assessments and discussing PI&R program issues with site and regional staff who are familiar with previously inspected areas. Inspectors may also reach out to NRR operating experience (OpE) staff for insights, if needed. Regional offices, as a part of the assessment and inspection planning processes, can also discuss and recommend general areas or specific samples in order to focus inspection resources on areas determined to have actual or potential performance concerns.

Inspectors should use relevant risk insights, such as maintenance rule program basis documents, current licensee risk analysis results or insights, licensee system health reports, and the Plant Risk Information eBook (PRIB) found in the site-specific standardized plant analysis risk (SPAR) model when selecting samples.

Inspectors may consider samples from the sources listed below. The sample selection guidance is intended to ensure the U.S. Nuclear Regulatory Commission (NRC) obtains insights into a licensee’s PI&R program across the cornerstones throughout an assessment cycle.

a. Licensee-identified issues, including issues identified during audits, self-assessments, and licensee event reports (LERs). LER closures are documented under IP 71153, but in-depth reviews of corrective action documents associated with LERs can also be counted as samples under this procedure.

b. Corrective actions associated with NRC documented issues e.g., notice of violation (NOVs), NCVs, findings (FINs). Requirement 03.04.f addresses follow-up of corrective actions associated with greater than green findings or performance indicators (PIs). Inspectors should also review the licensee’s response to a sample of NOVs, NCVs, and FINs in each cornerstone unless none were identified. If a cross-cutting theme has been identified, consider expanding the sample of non-cited violations reviewed to ensure corrective actions address safety culture.

c. NRC-identified issues during baseline, supplemental, and reactive inspections. Review observations from previous PI&R samples in reports and any reactive or supplemental reports during the current and previous assessment periods. Other baseline teams may uncover PI&R issues not within the scope of their inspection and refer the issues to the project branch for follow-up. Discuss such issues with respective NRC inspectors and management as part of inspection preparations.

d. Safety culture assessments. A licensee's evaluation of specific cross-cutting aspects, cross-cutting areas, functional departments, or levels (e.g., supervisors or nonsupervisory workers) may constitute a safety culture assessment review. [C2]

e. Issues identified through alternative avenues, such as employee concerns or similar programs [C2]. Note that some members of the licensee staff may not have authorized access to information about issues that are captured in these programs. Inspectors should protect this information from disclosure to any unauthorized personnel by limiting verbal or written discussions to only those licensee staff that have access rights to the subject records and to inspection team members that have a need-to-know. Inspectors may restrict access to portions of the exit or debrief meetings as appropriate.

f. Issues identified through NRC and industry operating experience exchange mechanisms (e.g., NRC generic communications, nuclear steam system supplier vendor reports, Electric Power Research Institute reports, and operating experience reports from similar facilities, INPO event reports, and NRC Operating experience smart samples).

g. Issues captured in databases maintained by the site’s corporate office. A site’s corporate office may track such issues separately from the site’s PI&R program. Inspectors may choose to view the contents of such databases to ensure that issues and operating experience are communicated to affected sites owned or operated by or associated with the corporate entity. Should an issue be identified on site that could affect a cornerstone or regulatory compliance and that issue is captured in the corporate PI&R program, that issue and the licensee’s handling of it may be reviewed, even though it is a corporate PI&R program issue. A review of corporate corrective action programs can identify important information affecting multiple sites, such as those identified with bio diesel fuel for which the office of NRR issued Information Notice (IN) 2009-02, for example.

h. Cause analyses and corrective action documentation associated with risk significant SSCs or functions. This includes SSCs or functions classified as (a)(1) status in accordance with the Maintenance Rule (10 CFR 50.65) and failures that resulted in operability evaluations. Inspectors should review the licensee’s trending analysis associated with these SSCs or functions to determine whether the licensee’s PI&R program should have enabled the identification and correction of the issue prior to the SSC or function failure and/or obtaining (a)(1) status. Inspectors may refer to IMC 0326 for additional guidance related to operability determinations.

i. Emerging or existing cross-cutting themes. Inspectors should review licensee actions or products associated with the identified theme for effectiveness.

j. Issues identified by safety review committees or other management oversight mechanisms.

k. Issues that challenge operator performance including but not limited to: operator work arounds, Main Control Room deficiencies, operator burdens and challenges, night orders or standing orders, temporary logs, control room or equipment operator logs, and work requests or work orders dealing with long standing issues.

l. Issues that may be age-related (e.g., due to aging effects such as loss of material, loss of preload, or cracking). Plants with renewed licenses have established aging management programs (AMPs) to identify, address, or prevent aging effects prior to loss of intended function for those SSCs within the scope of the AMP. When inspecting degradation or failures that appear to be age-related, inspectors should, in addition to other inspection activities, determine whether the SSC is being managed by an AMP. If so, the inspector should also determine whether the activities in the AMP are adequate to identify the aging effect prior to loss of SSC intended function, and whether the licensee’s corrective actions address the adequacy of the AMP. Consult with the regional license renewal point of contact for support in evaluating the adequacy of the AMP.

m. Fatigue-related issues identified through fitness for duty effectiveness reviews or licensee assessments reports, see 10 CFR 26.717(9). Refer to IP 93002, “Managing Fatigue” for additional guidance.

n. Defects and non-conforming materials, parts, or components may present a substantial safety hazard. Inspectors should inspect defects or non-conforming conditions for compliance with 10 CFR 50, Appendix B, and 10 CFR 21. Specifically, inspectors should verify that licensees are identifying and placing potential 10 CFR 21 issues into the CAP and appropriately evaluating them. Inspectors may refer to IP 36100, “Inspection of 10 CFR Part 21 and Programs for Reporting Defects and Noncompliance,” and IP 43004, “Inspection of Commercial-Grade Dedication Programs,” for additional guidance. [C3]

o. IP 71111.12, “Maintenance Effectiveness,” instructs inspectors to evaluate corrective actions associated with equipment subject to the Maintenance Rule (10 CFR 50.65). This IP also instructs inspectors to consider applicability of 10 CFR 50, Appendix B, Criterion XVI for safety related SSCs, especially when the corrective action-related requirements of 10 CFR 50.65(a)(1) may not be applicable. If inspectors identify potential PI&R program weaknesses during implementation of IP 71111.12 that require additional focus beyond the expectations of IP 71111.12, inspectors may select the issue as a sample for PI&R follow-up.

p. Inspectors can review Institute of Nuclear Power Operations (INPO) findings, recommendations, corrective actions, and operating experience that are documented in the licensee’s PI&R program. Inspectors may refer to the NRC/INPO Memorandum of Agreement, dated December 1, 2022 (ML23026A093), for guidance prior to reviewing any INPO documents. [C1]

02.02	Biennial PI&R Inspection Planning

Consider team composition, background, and experience during inspection planning. PI&R inspections can either benefit from a diverse team make-up or the inclusion of subject matter experts to focus on specific performance issues. Regional or headquarters specialists and subject matter experts (e.g., safety culture assessors, security, emergency preparedness, and radiation protection inspectors) can participate on the team in a full- or part-time capacity to assist team members in sample selection or inspection. The participation of a resident inspector from the site being inspected is considered a best practice.

Selected licensee documents needed to support the inspection may be obtained in advance. Inspectors should obtain and review documents necessary to accomplish the requirements in section 03.04 for the in-office review. Refer to IMC 0620, “Inspection Documents and Records” for more information on requesting documents for inspection preparation. Consider the following when developing information requests:

a. Procedures that govern PI&R, audits and assessments, operating experience, operability determinations, safety culture, employee concerns, work requests, maintenance programs, etc., related to specific samples. 

b. Lists of PI&R documents issued from the time of the last biennial team inspection e.g., work orders, work requests, temporary modifications, calibration failures, condition/problem identification reports, operability evaluations and determinations, operating experience, etc.

c. Specific PI&R documents related to:

1. Risk significant causal evaluations

2. LERs

3. NCVs

4. FINs

5. Cross cutting themes

6. Specific issues identified by the team during inspection planning.

7. Relevant licensee PI&R program assessments, program performance information, metrics, trend reports, and licensee safety culture assessments.

Inspectors should review NRC inspection reports back to and including the last biennial inspection. Observations and inspection results from PI&R samples during the evaluation period should be reviewed for themes or trends to consider during the team inspection and evaluation of the licensee’s PI&R program. The inspection plan should deconflict samples with any previously completed during the cycle. Issues can be reinspected if the scope of the inspection is different and supports objectives (i.e., resident inspectors reviewed the root cause of an issue, but the team is reviewing the completion of the corrective actions).

Review PI&R related observations and end of cycle discussions for any trends or patterns in PI&R program or performance issues that warrant additional sampling to confirm. For example, a series of issues associated with “failure to follow procedures” within one cornerstone may indicate a corrective action performance deficiency within a portion of the licensee’s organization; a series of issues associated with failure to follow procedures in multiple cornerstones may indicate a broader concern. Also, a lack of licensee-identified corrective action issues within a particular organization may be indicative of a problem with the identification threshold. Consider the need to follow up on performance trends documented as a result of the semiannual trend review.

# 71152-03	INSPECTION REQUIREMENTS

PI&R activities are reviewed in four locations within the baseline inspection program: baseline reviews; semiannual trend reviews; follow-up of selected issues; and biennial team inspections.

## 03.01	Baseline PI&R Review

*Conduct a review of licensee PI&R activities during the conduct of baseline inspection procedures to verify that the licensee has identified equipment, human performance, and program issues at an appropriate threshold, entered them into the PI&R program, classified them in accordance with licensee procedures, and has taken appropriate corrective actions.* [C1]

Specific Guidance:

Most of the baseline IPs contain a requirement to inspect PI&R performance within the IP’s subject area. The inspection of PI&R performance as part of baseline IPs is intended to ensure that over the course of an assessment cycle, a sample of PI&R performance in all cornerstones is established. The primary focus of this portion of the PI&R review should be on verifying that licensees are identifying issues at an appropriate threshold and entering them into their PI&R program. However, inspectors are not precluded from review of corrective action documents once they have been dispositioned to identify potential areas for further inspection. Inspectors should consider PI&R insights when selecting baseline inspection samples and may follow up on PI&R issues as part of a baseline inspection procedure’s PI&R review. Inspectors can review the selected samples against the success criteria contained in section 03.04. Inspectors may choose to assess licensee performance against selected success criteria, as necessary, to be most effective.

Inspectors should compare issues identified by the NRC during the conduct of the inspectable area portions of the baseline inspection program IPs with issues identified by the licensee. Additionally, inspectors can follow up on selected issues and operational occurrences to ensure that corrective actions commensurate with the significance of the issues have been identified and implemented by the licensee.

Inspectors should be alert for potential performance deficiencies as may be associated with equipment failures, inadequate maintenance work practices, personnel errors, inadequate risk assessments, management and emergent work control problems, procedure deficiencies, or non-compliances with procedures or regulatory requirements. When inspectors identify such conditions, they should examine the licensee’s PI&R program records or attend licensee PI&R program meetings to verify that the licensee either previously identified and documented the conditions noted by the inspector or acknowledged the inspector’s observations and entered those conditions into the PI&R program.

Documentation of a baseline inspection procedure scope constitutes completion of the baseline PI&R review. There is no requirement to document a separate PI&R observation or sample completed as part of a normal baseline sample.

Issues of concern identified during the Baseline PI&R review may be deferred to an annual follow-up of selected issues or the biennial team inspection.

## 03.02	Semiannual Trend Review

Review licensee PI&R program documents (e.g., issue tracking databases, licensee audits, and self-assessments) to identify potential trends (either NRC- or licensee-identified) that might indicate the existence of a more significant safety issue.

Specific Guidance:

The scope of this review should include repetitive or closely-related issues that may have been documented by the licensee outside the normal corrective action program (CAP), such as: trend reports, metrics, performance indicators, major equipment problem lists, repetitive or rework maintenance lists, departmental problem or challenge lists, issues that challenge operators in performing duties (e.g., workarounds), system health reports, quality assurance audit or surveillance reports, self-assessment reports, maintenance rule assessments, or corrective action backlog lists. [C1]

Consider a review of PI&R documents which have been dispositioned to identify potential adverse trends in SSCs as evidenced by acceptance of long-standing nonconforming or degraded conditions. Such indicators could include “use-as-is” determinations, revision of engineering or operational acceptance criteria, reductions in design or operational margin, and repetitive work orders. Review the selected sample against the success criteria in section 03.04.

Inspectors could also perform a review of findings or events over a period of time for indications of common causes. Inspectors should be careful not to aggregate findings during this review, but to focus on whether findings or events exhibit similar causes and if those causes constitute a separate concern.

Inspectors should consider emerging or existing cross-cutting themes during the semiannual trend review to develop insights into the licensee’s progress in addressing the themes. Inspectors can perform this review by summarizing the results of the licensee’s reviews and comparing those results to those identified by the NRC through the baseline or supplemental inspection program, including issues identified as a result of the routine PI&R review in IMC 2515, Appendix D, “Plant Status.” This information could be incorporated into the assessment under section 03.04. [C1]

Inspectors should document an inspection observation, in accordance with IMC 0611, when a potential adverse trend that might indicate the existence of a more significant safety issue is identified. Observations should include how the potential adverse trend relates to the success criteria in section 03.04 and any licensee actions in response. The level of documentation for the trend review may include trends that do not rise to the level of an inspection finding.

Additional issues of concern identified during the semiannual trend may be deferred to an annual sample or the biennial team inspection.

## 03.03	Annual Follow-Up of Selected Issues

Perform an in-depth review of selected issues to ensure that the licensee has planned or implemented corrective actions commensurate with the significance of the identified issues.

Specific Guidance

Annual samples should be performed by the inspectors most appropriate to the sample. This could be resident inspectors from licensee’s or another site, or regional or headquarters subject matter experts. These samples may be reviewed throughout the annual assessment cycle. Samples should be representative of all the cornerstones. Inspectors may use the guidance contained in section 02.01 as an aid in selecting samples for review.

When evaluating the effectiveness of a licensee’s corrective actions for a particular issue, the potential impact on nuclear safety and risk should be the primary factors in the licensee’s classification and prioritization of corrective actions. Inspectors should review the selected samples against the success criteria in section 03.04.**[C1] Inspectors are not expected to assess all of the success criteria for every issue selected for follow-up. Instead, inspectors may choose to assess licensee performance against selected success criteria, as necessary, to be most effective. Inspectors can also refer to IP 95001 for guidance on assessing licensee evaluations of significant performance issues (i.e., root cause analysis or other causal product).

Following the issuance of an assessment letter identifying a cross-cutting issue (CCI), the licensee’s progress in addressing the issue may be evaluated as an annual sample.

Document observations in accordance with IMC 0611. This documentation standard is different from issues elsewhere in the quarterly integrated inspection reports. Observations should include a description of the scope of the sample, the basis for the selection, and a description of the licensee’s completed or planned corrective actions. The length of the observation may vary depending on the complexity of the issue. Violations and more than minor performance deficiencies should be documented separately in accordance with IMC 0611.

## 03.04	Biennial Team Inspection

Sections 03.04.b through 03.04.e contain success criteria, which demonstrate the attributes of an effective PI&R program. Inspectors are not expected to assess all the criteria for every issue selected for follow-up. Instead, inspectors may use some or all of the success criteria to assess licensee performance, as necessary. These sections also include observations, which provide examples for when success criteria are not met. Observations can include a range of items, such as minor violations, minor performance deficiencies, violations, and findings. Utilizing the pertinent success criteria and observations, inspectors should develop an assessment of the licensee’s PI&R program.

a. Confirm that the licensee’s PI&R program complies with NRC regulations and any self-imposed or other standards necessary for continued participation in the Reactor Oversight Process.

Specific Guidance

Verify the licensee has established PI&R programs intended to meet applicable standards. PI&R programs may be required to comply with the following NRC regulations, industry standards and self-imposed standards:

The following NRC regulations may apply to aspects of the licensee’s PI&R programs under the various cornerstones

10 CFR Part 21, “Reporting of Defects and Noncompliance”

10 CFR Part 26.717, “Fitness-for-duty program performance data”

10 CFR Part 50 Appendix B, “Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants”

10 CFR Part 50 Appendix E, “Emergency Planning and Preparedness for Production and Utilization Facilities”

10 CFR Part 50 Appendix R, “Fire Protection Program for Nuclear Power Facilities Operating Prior to January 1, 1979”

10 CFR Part 50.55a, “Codes and standards”

10 CFR Part 50.65, “Requirements for monitoring the effectiveness of maintenance at nuclear power plants”

10 CFR Part 71, “Packaging and Transportation of Radioactive Material,” Subpart H, “Qualify Assurance”

10 CFR Part 73, Subpart F, “Physical Protection Requirements at Fixed Sites”

10CFR Part 20, “Standards for Protection Against Radiation”

Regulatory Guide 1.33, “Quality Assurance Program Requirements (Operation),” contains guidance and endorsements for ANSI (American National Standards Institute) and ASME (American Society of Mechanical Engineers) standards related to Quality Assurance Programs (i.e., ASME NQA-1, “Quality Assurance Program Requirements for Nuclear Power Plants”).

The following licensee documents may contain additional commitments and/or requirements:

quality assurance manual

emergency plan

radiation protection plan

fire protection plan

security plan

aging management program

Licensees may also commit to industry documents not endorsed by the NRC such as:

NEI 09-07, “Fostering a Healthy Nuclear Safety Culture”

NEI 16-07, “Improving the Effectiveness of Issue Resolution to Enhance Safety and Efficiency”

NEI 18-03, “Operability Determination”

NEI 18-10, “Monitoring the Effectiveness of Nuclear Power Plant Maintenance”.

This requirement ensures the licensee’s PI&R program meets the minimum expectations necessary for continued implementation of the ROP at the licensee. IMC 0308 Attachment 2, “Technical Basis for Inspection Program,” section 03.02 discusses the importance of licensees’ PI&R programs to the ROP, the necessity of inspection of PI&R across the cornerstones, and what is required to determine that a licensee’s PI&R program does not comply with NRC regulations or applicable standards. Document the result of this requirement in the report cover letter in accordance with IMC 0611, Exhibit4.

b. Evaluate the effectiveness of the licensee’s PI&R program.

Specific Guidance

Use risk insights to select issues that have been processed through the licensee’s PI&R program that encompass an evaluation period since the last biennial team inspection [C1]. The biennial inspection team leader should choose as many issues for review as warranted to complement PI&R samples already completed during the evaluation period and ensure a sufficient basis for evaluating the effectiveness of the licensee’s PI&R program.

The samples chosen for review should include a range of issues selected from the list in section 02.01 and meet the requirements of 03.04. For a subset of the samples chosen for review, the scope of the review should be expanded to at least 5 years to detect and evaluate long term trends. Among the samples chosen for this extended review should be those issues whose significance might be age-dependent, such as issues associated with erosion of piping, degradation of safetyrelated raw water systems, boric acid accumulations, aging of electronic components, environmental qualification, etc. This review can be performed by a PI&R program search for those items designated by the team for the 5year review. [C1]

Inspectors may select one or more risk-significant systems on which to focus sample selections. Performing a walkdown of selected systems will provide insight into the adequacy of the licensee’s implementation of all aspects of the PI&R program (identification, evaluation, and corrective action). Team leaders are reminded to ensure adequate coverage of the Emergency Preparedness, Radiation Safety, and Security cornerstones.

By reviewing a sufficient number and breadth of samples conducted both during the assessment period and the team inspection, the team should develop insights into the licensee’s effectiveness at identifying, evaluating, and correcting problems using the PI&R program. Inspectors should compare these results with the licensee’s performance reviews, including reviews of PI&R programs. Inspectors should determine whether licensee reviews are consistent with the NRC review of PI&R issues.

Utilizing the below criteria, the team should develop a clear and concise assessment of the results of their review. This assessment can be supported by observations uncovered during the inspection activities, including those activities from the baseline PI&R reviews, semiannual trend reviews, and annual follow-up of selected issues, conducted since the last biennial assessment of the licensee’s PI&R program. The discussion should be documented in the inspection report for the biennial team inspection. IMC 0611, Appendix D, provides additional specific and unique guidance beyond that contained in IMC 0611 for documenting the biennial PI&R inspections.

1. Identification

Assess the licensee’s ability to identify and enter issues into their PI&R program.

Specific Guidance

From the samples chosen, and a review of PI&R samples completed during the assessment period, assess the licensee’s ability to identify and enter issues into their PI&R program against the success criteria listed below. Utilize the examples and IMC 0611, Appendix D, guidance to document an assessment.

(a) Related Cross-Cutting Aspects:

(1) Identification (P.1): The organization implements a corrective action program with a low threshold for identifying issues. Individuals identify issues completely, accurately, and in a timely manner in accordance with the program.

(2) Trending (P.4): The organization periodically analyzes information from the corrective action program and other assessments in the aggregate to identify programmatic and common cause issues.

(b) Success Criteria

(1) Licensee staff enter conditions into the licensee’s PI&R program at a low threshold.

(2) Deficient conditions associated with safety system performance or regulatory compliance are promptly brought to the attention of main control room operators so operability and functionality can be assessed. When appropriate, technical specifications action statements or compensatory measures are initiated.

(3) Licensee has an effective trending program which uses PI&R program data and other applicable insights (e.g., Maintenance Rule program, system health reports, etc.) to identify low level trends with equipment and human performance. The licensee addresses identified issues prior to the issues becoming more significant problems.

(4) Licensee has no cross-cutting themes or otherwise adverse trends in P.1 (identification) or P.4 (trending) in the previous 12-month period.

(5) For NRC-identified issue(s), the licensee did not miss opportunities to identify the problem(s)

(c) Observations

(1) A failure to identify or enter a significant condition adverse to quality (SCAQ) into the PI&R program.

(2) An adverse trend that affects the success criteria exists (e.g., an actual or potential cross-cutting theme). An adverse trend other than crosscutting issues should be supported by several examples which indicate a performance or programmatic weakness that affects the success criteria.

(3) Repetitive examples of licensee staff being aware of Conditions Adverse to Quality (CAQs) and failing to document them in the PI&R program.

(4) CAQs affecting operability are not promptly brought to the main control room operators so operability can be determined, and appropriate technical specifications entered when appropriate.

(5) Repetitive examples of issues not being documented with enough relevant detail, such that operators cannot make conservative decisions for identified CAQs or condition adverse to regulatory compliance (CARCs).

(6) Multiple NRCidentified trends during the assessment period not previously identified by the licensee.

(7) A greaterthangreen finding during the evaluation period with a documented performance deficiency of failing to identify a significant adverse condition.

(8) A significant programmatic weakness exists which results in widespread failure to enter SCAQ, CAQ, or CARC into the corrective action program.

(9) Identification of negative trends associated with human or equipment performance that can potentially impact nuclear safety.

(10) Area documented as a significant or general weakness during a Supplemental Inspection (IP 95001, 95002, or 95003) or IMC 0350 inspection.

2. Evaluation

Assess the licensee’s ability to evaluate and prioritize issues entered into their PI&R program.

Specific Guidance

From the samples chosen, and a review of PI&R samples completed during the assessment period, assess the licensee’s ability to evaluate and prioritize issues in their PI&R program against the success criteria listed below. Utilize the examples and IMC 0611, Appendix D, guidance to document an assessment.

(a) Related Cross-cutting Aspect:

Evaluation (P.2): The organization thoroughly evaluates issues to ensure that resolutions address causes and extent of conditions commensurate with their safety significance.

(b) Success Criteria

(1) The licensee appropriately prioritizes issues in accordance with the safety, security, or radiological significance so that licensee resources and oversight are assigned commensurate with the actual or potential consequences of the issue in accordance with their PI&R program.

(2) The licensee thoroughly evaluates issues to ensure that resolutions address causes and extent of conditions commensurate with their safety, security, and radiological significance in accordance with their corrective action program procedure and quality assurance plan.

(c) Observations

(1) Inappropriate assessment of the priority of an SCAQ (e.g., wrong significance assigned).

(2) Failure to evaluate the root cause of an identified SCAQ.

(3) Repetitive examples of licensee staff failing to accurately prioritize issues in accordance with their safety or security significance, and as a result CAQs and CARCs are not corrected commensurate with their safety significance.

(4) Repetitive examples of licensee staff failing to perform an adequate evaluation as assigned in their corrective action program which either: (1) does not identify the correct cause, or (2) does not conduct an extent of cause or extent of condition issues (when required) due to lack of rigor.

(5) Repetitive examples of the licensee failing to follow the corrective action program requirements when an evaluation is unable to determine a cause for a more significant event.

(6) A greaterthangreen finding with a cross-cutting aspect of P.2 (evaluation).

(7) A finding of yellow or red significance with a documented performance deficiency of failing to evaluate a significant adverse condition or develop corrective actions to preclude repetition (CAPRs).

(8) A significant programmatic weakness exists which results in widespread failure to adequately evaluate an SCAQ, CAQ, and CARC, and develop corrective actions to correct the conditions.

(9) A repetitive adverse trend in P.2 (evaluation) documented in semiannual trend reviews or consecutive PI&R area inspections and licensee corrective actions have been ineffective based upon follow up inspection.

(10) Area documented as a significant or general weakness during a Supplemental Inspection (IP 95001, 95002, or IP 95003) or IMC 0350 inspection.

3. Corrective Action

Assess the licensee’s ability to determine, track, implement and evaluate timely corrective actions for issues entered into their PI&R program.

Specific Guidance

From the samples chosen, and a review of PI&R samples completed during the assessment period, assess the licensee’s ability to determine, track, implement and evaluate timely corrective action issues into their PI&R program against the success criteria listed below. Utilize the examples and IMC 0611, Appendix D, guidance to document an assessment.

(a) Related Cross-cutting Aspect:

Resolution (P.3): The organization takes effective corrective actions to address issues in a timely manner commensurate with their safety significance.

(b) Success Criteria

(1) The licensee effectively schedules and completes corrective action development commensurate with their safety significance using the work control process.

(2) The licensee appropriately performs an evaluation in the event that corrective actions placed in the work control process are deferred or cancelled. This evaluation would include compensatory actions, bridging strategies, or alternative corrective actions to ensure the CAQ or CARC is corrected commensurate with its risk significance.

(3) The licensee ensures that the final corrective actions completed adequately address the original CAQ or CARC observed.

(4) The licensee tracks CAPRs for SCAQ to completion in the corrective action program.

(5) The licensee conducts effectiveness reviews for CAPRs associated with an SCAQ and develops new corrective actions when appropriate.

(c) Observations

(1) An example of a failure to implement CAPRs in a timely manner and prevent repetition of an identified SCAQ, resulting in a safety- or securitysignificant finding with a documented performance deficiency of failing to correct a significant adverse condition.

(2) Examples where CAPRs for SCAQ are not being tracked or completed under the CAP.

(3) Examples of effectiveness reviews for CAPRs for SCAQ not being completed or actions taken if the review identified an issue.

(4) Repetitive examples of licensee staff failing to schedule corrective action assignments in a timely manner commensurate with the CAQ’s or CARC’s safety or security significance as evidenced by repetitive failures of equipment or corrective actions not accurately being completed due to errors in the work management process.

(5) Repetitive examples of licensee staff failing to complete corrective actions assigned due to work orders or engineering change packages being deferred and the deferrals not being evaluated and results in unnecessary or uncompensated safety or security risk until the underlying CAQ or CARC is corrected.

(6) Repetitive examples of the licensee failing to identify that the actions completed did not actually correct the CAQ or CARC or restore compliance commensurate with its safety or security significance.

(7) A safety- or security-significant finding with a cross-cutting aspect of P.3 (resolution).

(8) A significant programmatic weakness exists which results in widespread failure to correct an SCAQ, CAQ, or CARC commensurate with their safety or security significance.

(9) A repetitive adverse trend in P.3 (resolution) documented by a semiannual trend review or PI&R area inspection, and licensee corrective actions have been ineffective based upon follow up inspection.

(10) Area documented as a significant or general weakness during a Supplemental Inspection (IP 95001, 95002, 95003) or IMC 0350 inspection.

(11) For NRC-identified issue(s), prior attempts by the licensee to remedy the problems were inadequate

c. Confirm the licensee appropriately uses industry and NRC operating experience.

Specific Guidance

Review a risk informed selection of NRC, industry, and corporate (if applicable) operating experience issued or dispositioned during the assessment period. These may include:

NRC Information Notices

NRC Generic Letters

Part 21 Reports

INPO IERs

Corporate or “Fleet” Operating Experience

Review the licensee’s process for receiving and dispositioning operating experience, and how applicable operating experience is determined.

Utilizing the success criteria and observation guidance below, evaluate the licensee’s use of operating experience; document an assessment in accordance with IMC 0611, Appendix D.

1. Related Cross-cutting Aspect:

Operating Experience (P.5): The organization systematically and effectively collects, evaluates, and implements relevant internal and external operating experience in a timely manner.

2. Success Criteria: Licensee implements a process for gathering, evaluating, and entering issues identified at other facilities into their PI&R program such that potential licensee vulnerabilities or weaknesses are addressed and resolved in a timely manner. Applicable operating experience includes NRC Generic Communications, Part 21 reports, industry wide communications, and fleet and owner’s group reports and recommendations.

3. Observations

(a) Programmatic weaknesses (e.g., lack of procedural guidance, unidentified leads for the operating experience program, or failure to follow self-imposed standards for operating experience monitoring and tracking).

(b) Repetitive examples of licensee failure to capture applicable operating experience.

(c) Repetitive examples of licensee failure to screen operating experience as applicable.

(d) Failure to act on applicable Part 21 reports.

(e) Inadequate actions in response to operating experience.

(f) Failure to track completion of actions related to operating experience

d. Evaluate the effectiveness of licensee audits and self-assessments.

Specific Guidance

Review a sample of licensee audits and self-assessments conducted during the period of assessment [C1].

When the licensee has performed an independent safety culture assessment, inspectors should evaluate the licensee’s assessment. If the licensee conducted any periodic selfinitiated assessments of safety culture during the review period, this assessment should be included along with other non-safety culture self-assessments selected to review. If the licensee performed several assessments that collectively addressed safety culture issues, then those assessments combined should be considered as one assessment [C2]. Inspectors should review the adequacy of the licensee’s evaluation and actions to address the issues identified by the safety culture assessment. Not all actions necessarily need to be handled within the licensee’s corrective action program under 10 CFR 50, Appendix B, Criterion XVI. It may be more appropriate for some issues that are not conditions adverse to quality to be tracked to resolution through an alternate licensee program such as an employee concerns program. The inspectors review should focus mainly on the licensee’s response to the assessment results or actions taken to address identified issues instead of the assessment methodology or an evaluation the assessment’s adequacy. Section 03.04.e provides more guidance on reviewing the licensee’s safety culture assessment from the SCWE perspective.

Utilizing the success criteria and observation guidance below, evaluate the licensee’s ability to conduct audits and self-assessments; document an assessment in accordance with IMC 0611, Appendix D.

1. Related Cross-cutting Aspect:

Self-Assessments and Audits (P6): The organization routinely conducts self-critical and objective assessments of its programs and practices.

2. Success Criteria:

(a) Licensee has an effective quality assurance audit and self-assessment program which identifies weaknesses and places those deficiencies and observations into the PI&R Program for resolution.

(b) Licensee’s quality assurance audits are appropriately identifying problems in the 10 CFR Part 50, Appendix B, area the audit is focused on.

(c) Licensee's audits are in accordance with the quality assurance topical report/ Quality Assurance Plan and the associated industry standards that the Quality Assurance Plan commits to.

3. Observations

(a) Examples of audit results not consistent with the inspector’s observations or previous NRC findings and/or observations.

(b) Licensee failure to identify, implement and track corrective actions resulting from audits and assessments.

e. Review issues that pose challenges to the free flow of information for adequate resolution. Employees should feel free to raise safety concerns, both to their management and to the NRC, without fear of retaliation. [C2]

Specific Guidance

To assess the licensee’s environment for raising concerns, and to determine whether impediments exist to the establishment of a SCWE, inspectors should interview a number of licensee personnel and, if applicable, long-term contractors. These interviews should focus on the willingness of these personnel to raise safety concerns to supervisors/management or through the PI&R program, their knowledge of alternative avenues for raising concerns, and whether they have experienced or heard of anything perceived as retaliation for raising concerns.

Inspectors may conduct these interviews by one of several methods: as a supplement to other discussions with personnel about PI&R issues, as standalone interviews with select personnel, or by conducting focus group interviews. Focus group interviews are permissible only when the inspector facilitating the focus group has received training in conducting focus groups; it is strongly preferred that the facilitator be qualified as a Safety Culture Assessor per IMC 1245, Appendix C12.

When conducting interviews with or observing other activities involving licensee personnel and/or long-term contractors (i.e., those who have been working at the site for at least 6 months) during the inspection, inspectors should be sensitive to areas and issues that may represent challenges to the free flow of information, such as areas where employees may be reluctant to raise concerns or report issues in the PI&R program. [C2]

Interviewing long-term contractors would allow inspectors to assess the SCWE of a group of individuals that have worked at the site for extended periods of time and impacted plant operations and safety. Inspectors should also obtain insights about the SCWE during their review of the licensee’s most recent safety culture and other relevant assessments. Inspectors should be sensitive to similarities and differences between the results of their SCWE interviews with plant staff and the results of the licensee’s safety culture and other relevant assessments.

During inspection preparation and performance, readily available indications of licensee SCWE (e.g., licensee SCWE survey results, NRC allegation data, licensee employee concerns program records, Nuclear Safety Culture Monitoring Panel inputs, and resident input) should be reviewed to determine an appropriate scope for assessing the SCWE via on-site interviews and/or focus groups. To the extent practicable, personnel interviewed should be mostly nonsupervisory and should represent a cross-section of the licensee’s organizational departments (e.g., operations, maintenance, engineering, security, etc.). If possible, the experience levels of the personnel should vary; both newer and mid-career individuals should be included. Focus group interviews should similarly cover a cross-section of the licensee’s organizational departments and should include people with a variety of experience levels, where a focus group consists of 8-10 or more individuals. Each focus group should only include personnel at the same supervisory level and may be supplemented by individual discussions with managers or supervisors.

Appendix A to this procedure provides a list of questions that may be used to assess SCWE in interviews or focus groups.

Although the licensee may be implementing an employee concerns or similar program regarding the identification of safety issues, the possibility of existing underlying factors that would produce a "chilling" effect or reluctance to report such issues could exist, and inspectors should be alert for such indications. Such factors could include but not be limited to direct retaliation, inadequate staffing that results in excessive overtime, an unwillingness to raise issues that might result in further increases to an already high workload, or inadequate corrective actions for previously identified issues causing personnel to be reluctant to identify additional related issues.

If inspectors become aware of (1) instances of employees being discouraged from raising safety or regulatory issues within the licensee’s or contractor’s organization or to the NRC, (2) a “chilling” effect, or (3) other general reluctance of employees to raise safety or regulatory issues unrelated to a specific event or incident, they should refer to IP 93100, “Safety Conscious Work Environment Issue of Concern Follow-up,” and consult with regional management to determine appropriate follow-up actions.

Utilizing the success criteria and observation guidance below, evaluate the licensee’s safety conscious work environment, and document an assessment in accordance with IMC 0611, Appendix D.

1. Related Cross-cutting Aspect:

(a) SCWE Policy (S.1): The organization effectively implements a policy that supports individuals’ rights and responsibilities to raise safety concerns, and does not tolerate harassment, intimidation, retaliation, or discrimination for doing so.

(b) Alternate Process for Raising Concerns (S.2): The organization effectively implements a process for raising and resolving concerns that is independent of line management influence. Safety issues may be raised in confidence and are resolved in a timely and effective manner.

(c) Free Flow of Information (S.3): Individuals communicate openly and candidly, both up, down, and across the organization and with oversight, audit, and regulatory organizations.

2. Success Criteria:

(a) The licensee has an established SCWE program verified by a review of programmatic documents.

(b) The licensee monitors for a SCWE through self-assessment using their proceduralized Nuclear Safety Culture Monitoring Programs, employee concerns program (ECP), and site-specific review boards to screen disciplinary actions for potential chilling implications.

(c) The licensee maintains a safety-conscious work environment as evident through discussions and interviews with licensee staff. Site employees appear willing to raise nuclear safety concerns through at least one of the several means available.

3. Observations

(a) Results of SCWE interviews that indicate multiple employees in a work group are hesitant or unwilling to raise concerns to certain managers or at all.

(b) Licensee or third-party safety culture review/assessment identifies work groups of concern with respect to SCWE.

(c) A step-increase in the number of allegations received compared to the previous assessment period. An example could be a step change that results in the total number of allegations being at least double the industry average for that year.

(d) High-volume of allegations where total allegations are substantially higher than the industry average for at least 2 consecutive years. An example could be where the total allegations are greater than 3 times the industry average.

(e) In any 18-month period there is a documented ROP finding with a cross-cutting aspect of SCWE, and the impact on SCWE was not isolated. For the purpose of this criteria “not isolated” means more than one individual is impacted (e.g., multiple individuals, functional groups, shift crews, or levels within the organization are affected).

(f) The licensee has received a chilling effect letter during the assessment period, or one remains open.

(g) The licensee has received correspondence from the NRC that transmitted a SL I, II, or III enforcement action that involved discrimination or a confirmatory order that involved discrimination. The theme applies only to the sites(s) where the discrimination occurred.

(h) Results of a supplemental inspection of IMC 0350 inspection document a SCWE concern.

(i) Results of SCWE interviews (PI&R samples) documented in 3 consecutive reports show no improvement, declining SCWE, or indications that the concern is impacting multiple site organizations and NRC management has concerns about the effectiveness of their corrective actions taken to date and this concern is documented in an Annual Assessment letter.

f. Review corrective actions related to greater-than-green findings and performance indicators that were not completed by the end of the associated supplemental inspection.

Specific Guidance

A review of all completed corrective actions for greaterthangreen findings and performance indicators provides additional assurance that the licensee’s completed corrective actions for risk-significant performance issues are sufficient to address the root and contributing causes and prevent recurrence. Perform follow-up inspection of any Planned (Open) Corrective Actions to Preclude Repetition that were not completed following an IP 95001 or IP 95002 supplemental inspection. Section 11 of IMC 2515, Appendix B, provides additional guidance.

# 71152-04	REFERENCES

Audit of NRC’s Implementation of 10 CFR Part 21, Reporting of Defects and Noncompliance (OIG-11-A-08, March 23, 2011, ML110820426)

IMC 0308 Attachment 2 “Technical Basis for Inspection Program”

IMC 0326, “Operability Determinations”

IMC 0611, “Power Reactor Inspection Reports”

IMC 0620, “Inspection Documents and Records”

IMC 2515, Appendix B, “Supplemental Inspection Program”

IP 36100, “Inspection of 10 CFR Part 21 and Programs for Reporting Defects and Noncompliance”

IP 43004, “Inspection of Commercial-Grade Dedication Programs”

IP 71111.12, “Maintenance Effectiveness”

IP 93002, “Managing Fatigue”

IP 93100, “Safety Conscious Working Environment Issue of Concern Follow-up”

IP 95001, “Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs”

NEI 09-07, “Fostering a Healthy Nuclear Safety Culture”

NEI 16-07, “Improving the Effectiveness of Issue Resolution to Enhance Safety and Efficiency”

NEI 18-03, “Operability Determination”

NEI 18-10, “Monitoring the Effectiveness of Nuclear Power Plant Maintenance”

NRC Enforcement Manual

NRC Enforcement Policy

NRC/INPO Memorandum of Agreement, dated December 1, 2022 (ML23026A093)

Regulatory Guide 1.33, “Quality Assurance Program Requirements (Operation)”

END

List of Appendices: Appendix A: Guidance for Gathering SCWE and PI&R Insights

List of AttachmentsAttachment 1: Revision History for IP 71152

Issue Date: 10/31/23	1	71152

Appendix A: Guidance for Gathering SCWE and PI&R Insights

The following are suggested questions that may be used when discussing PI&R issues with licensee individuals. It is not intended that these questions are asked verbatim, but rather, that they form the basis for gathering insights regarding whether there are impediments to the formation of a SCWE.

In cases where a potential problem with SCWE is identified in response to these questions, inspectors should consult with regional management to determine if inspection resources should be applied using IP 93100, “Safety Conscious Working Environment Issue of Concern Followup” to gain additional SCWE insights.

SUGGESTED QUESTIONS

Problem Identification and Resolution Program (PI&R):

1. How effective is the PI&R program in addressing problems?

2. Do you think it’s worth taking time to place problems found into the PI&R program? Why or why not?

3. When you enter an issue into the process, do you receive any feedback when it’s been discussed or addressed? Are you satisfied with this level of feedback?

4. Are there informal means you would use to address issues found, other than the official PI&R program? If so, please provide some examples.

5. Can anyone at the site (contractor, security officer, etc.) enter an issue into the PI&R program? When someone enters an issue into the PI&R program, does the entry have to be approved by a supervisor? (If yes, does this affect what gets put in the PI&R program?)

Environment for Raising Concerns (SCWE):

1. Do you feel free to raise concerns without fear of retaliation?

2. Are you aware of any situations where any employee or contractor may have been hesitant to raise concerns or feared a negative consequence for raising a concern? What kind of concerns? Can you give some examples?

3. How do you and your colleagues feel about expressing your opinions? How do you think management receives and addresses opinions and viewpoints?

4. In your opinion, if employees don’t receive a response that they are satisfied with, are they able to escalate their concern to a higher level of management? If no, why not? Is escalation of concerns encouraged by management? If so, how?

5. Have there been any issues recently (2 years) that would affect your willingness to raise safety issues or your confidence in the PI&R program? Please provide examples.

6. How do you feel about using ECP? Are you confident about confidentiality?

7. Do you feel free to bring concerns to the NRC without fear of retaliation?

8. How does your management encourage the use of alternate avenues (ECP) for raising safety concerns?

9. Does your management seem to put what you believe to be the appropriate emphasis on safety (nuclear, radiological, and industrial)? Please provide examples.

10. When production goals (schedules) are set, how are they communicated to you? What is management’s reaction when a safety concern is raised that affects the schedule and thus the production goal is not met?

Preventing, Detecting and Mitigating Perceptions of Retaliation (SCWE):

1. Does the station have a policy concerning maintaining a work environment where workers can raise safety concerns without fear of retaliation? What does it say, in general? Would you say that your management is supportive of the policy?

2. Are you aware of any actions taken by your management to prevent and detect retaliation and/or other behaviors that could cause workers to be hesitant to raise safety concerns, that is, behaviors that could cause a chilling effect? If so, please provide examples.

3. Have your perceptions about this issue changed over time particularly over the last 1 to 2 years?

4. Are you aware of any instance in which someone on site has experienced a negative reaction from a supervisor or manager for raising a safety issue? If so, please provide examples.

END

Issue Date: 10/31/23	AppA-1	71152

Attachment 1: Revision History for IP 71152

Issue Date: 10/31/23	Att1-1	71152