INSPECTION MANUAL CHAPTER 0308 ATTACHMENT 3

TECHNICAL BASIS FOR SIGNIFICANCE DETERMINATION PROCESS

Table of Contents0308.03-01	BACKGROUND	10308.03-02	FUNDAMENTAL ATTRIBUTES FOR ALL SDP TOOLS	102.01	 Objectivity	102.02	 Scrutability (Openness)	202.03	 Timeliness	202.04	 Inspection Planning	202.05	 Responsibility for Significance Determinations	302.06	 Independence from Other NRC Processes	30308.03-03	ADDITIONAL APPLICABILITY FOR SDP TOOLS THAT USE PROBABILISTIC RISK METHODS	403.01	 Use of Computer-Based Risk Models	403.02	 Importance of a Critical and Open Deliberative Process Leading to Understanding	403.03	 Risk-Informed SDP Tools - Specific Principles and Attributes	50308.03-04	RISK-INFORMED VERSUS RISK-BASED	70308.03-05	PERFORMANCE DEFICIENCIES AND DEGRADED CONDITIONS	80308.03-06	THE INDEPENDENCE OF INSPECTION FINDINGS	90308.03-07	TREATMENT OF UNCERTAINTY AND RISK-INFORMED DECISION-MAKING	100308.03-08	QUANTITATIVE RISK METRICS OF CORE DAMAGE FREQUENCY AND LARGE EARLY RELEASE FREQUENCY	1108.01	 Technical Basis for CDF and LERF Metrics	1108.02	 Treatment of Degraded Conditions and Initiating Events	110308.03-09	USE OF THE RISK ASSESSMENT STANDARDIZATION PROJECT (RASP) HANDBOOK	130308.03-10	REFERENCES	13Appendices to Attachment 3	14Attachment 1 - Revision History for IMC 0308 Attachment 3	Att1-1

Issue Date:  11/05/21	i	0308 Att 3

# 0308.03-01	BACKGROUND

Commission paper SECY-99-007A, dated March 22, 1999, describes a method for assigning a probabilistic public health and safety risk characterization to licensee performance deficiencies[footnoteRef:2] related to reactor safety.  This risk characterization method was the first of a set of methods and tools developed that became central elements of the Significance Determination Process (SDP) to determine reactor inspection finding significance consistent with the thresholds used for the risk-informed plant Performance Indicators (PIs).  This allowed inspection findings and PIs to be used consistently as inputs to the overall plant performance assessment portion of the Reactor Oversight Process (ROP). [2:  Performance deficiencies that are determined to be of more than minor significance using IMC 0612 Appendix B, “Issue Screening Directions” are referred to as inspection findings.  See Section 5 of this IMC for additional discussion.] 

Subsequently, other SDP tools were developed to characterize the significance of inspection findings associated with emergency preparedness, occupational and public radiation safety, security, fire protection, plant shutdown operations, containment integrity, operator requalification, maintenance rule, B.5.b and use of qualitative measures for decision-making.  These SDP tools either used quantitative risk evaluation methods or were risk-informed through expert judgment of the staff.  The resulting SDP tools were considered acceptable starting points from which to be continuously improved as experience was gained.

The term “SDP” describes an overall process that includes all associated provisions designed to meet ROP founding principles such as objectivity, scrutability, repeatability, and timeliness.  The SDP is implemented using various cornerstone-specific SDP appendices, which may be referred to by their specific names as individual “SDPs.”  A list of the specific SDPs used in the ROP is provided at the end of this document.  A technical basis for each SDP is presented as a separate corresponding appendix to Inspection Manual Chapter (IMC) 0308, Attachment 3, “Technical Basis for Significance Determination Process.”

# 0308.03-02	FUNDAMENTAL ATTRIBUTES FOR ALL SDP TOOLS

The following fundamental attributes apply to all SDPs, across all cornerstones.  All proposed SDP changes should not detract from maintaining and improving these intended attributes.

## 02.01		Objectivity

Each SDP tool should attempt to provide a decision logic or a decision framework that remains relatively constant across applicable inspection findings.  This enhances objectivity by reducing the likelihood that SDP results are influenced by different value judgments held by different individuals.  Where practicable, a probabilistic risk framework is used to add this desired discipline to SDP results.  The test of having achieved such objectivity is when different individuals using a given SDP decision logic or framework arrive at the same result when using the same input conditions and assumptions.  Achieving SDP result consistency and repeatability is the intended outcome of the objectivity attribute.  This attribute can be achieved through peer reviews of SDP assessments to assure consistency in SDP decision-making.

## 02.02		Scrutability (Openness)

The SDP should be capable of providing a clear framework to facilitate a shared understanding of each significance determination and its basis among technically knowledgeable stakeholders (both internal and external).  This shared understanding allows for broad and independent validation of the staff’s objectivity and most directly enhances NRC public credibility.

When a quantitative risk model is used, the greatest challenge to achieving this attribute is to allow stakeholders a means to independently assess SDP result sensitivity to the most influential assumptions, to understand the basis of the assumptions, and to reveal the limitations and uncertainties of the risk model used and how these were considered by the staff in arriving at a final result.  When quantitative risk insights and inputs from other factors considered for decision-making are used, the bases of the significant factors influencing the decision outcome must be clearly documented in detail for scrutability and effective communication of the final risk-informed decision.

Since September 11, 2001, public access to site-specific models for at-power conditions has been restricted.  As such, the ability of the public to engage in open communications about plant-specific probabilistic risk information has been reduced.

## 02.03		Timeliness

The SDP is intended to support timely decisions to assess the risk significance of findings generally within a timeframe consistent with quarterly updates of the Action Matrix (described in IMC 0305, “Operating Reactor Assessment Program”) portion of the performance assessment component of the ROP.  The overall SDP timeliness metric is 255 days from the date of initial identification.  The process milestone for the end of the 255-day timeliness metric is the issuance of the final significance determination letter after timely completion of a public Regulatory Conference or review of a licensee written response.  Additional information on the SDP timeliness metric is described in IMC 0307, Appendix A, “Reactor Oversight Process SelfAssessment Metrics and Data Trending.”  Additional information regarding other process milestones can be found in IMC 0609, Attachment 5, “Inspection Finding Review Board.”

Achieving SDP timeliness using best available information requires that NRC staff effectively receive information from a licensee, starting when a potential finding is identified.  In addition, maintaining public credibility requires timely public notification of the existence of a potentially significant finding and identification of the staff’s preliminary basis for potential significance.  When appropriate, preliminary SDP results should reveal what influential information is needed from the licensee that might change or confirm the preliminary decision.  Because the SDP assesses licensee deficient performance that occurred in the past and is most often immediately corrected, SDP decisions may proceed, particularly in the case of risk-informed SDPs, with a degree of residual uncertainties that may be greater than those uncertainties considered acceptable for NRC licensing decisions which, for example, might affect the risk of the plant throughout its remaining lifetime.

## 02.04		Inspection Planning

The SDPs should inform the inspection activities and improve the effectiveness of the inspectors who directly implement the reactor inspection program.  Through routine use and application of the SDP tools, inspectors are expected to become more aware of findings of greater significance, with a correspondingly higher likelihood of their identification if they exist.  The best means for inspectors, decision-makers, and others to understand plant-specific risk insights, including the reasons for whether a finding is or is not significant, is to understand the SDP tools and regularly discuss them with risk analysts, as needed, for valuable insights.

In addition, the reactor safety SDP should be used to identify appropriate risk-informed inspection samples within appropriate inspection procedures by using the prior risk insights gained from applying the SDP.  Inspectors can develop risk-informed inspection samples by reviewing information in the NRC Standardized Plant Analysis Risk (SPAR) Plant Risk Information e-Book (PRIB), and the SDP Workspace module in the Systems Analysis Programs for Hands-On Integrated Reliability Evaluations (SAPHIRE) code, or through discussions with Senior Reactor Analysts (SRAs).

## 02.05		Responsibility for Significance Determinations

Each SDP result is the sole responsibility of the NRC staff.  The SDP is not a consensus process with a licensee or other parties and no staff/licensee interactions should be construed as a negotiation.  The ROP requires the staff to make decisions using best available information in a timely manner and that the bases of SDP results be clear and publicly available, to the extent practical and permitted by policy (e.g., security issues).  The SDP affords licensees an opportunity to provide available information that may be useful to the staff in arriving at a bestinformed decision within a reasonable time.  The staff is obligated to be clear about the basis for any SDP result and to consider licensee-provided information.  The staff is not obligated to have “proof” of the assumptions made relative to an SDP result basis.  Staff engineering or technical judgment is often required, but should be consistent with similar previous circumstances, as appropriate.  The staff’s technical judgment should be made objective through its use within the appropriate SDP tool used as a decision framework.  However, a licensee may appeal the staff’s decision if the prerequisites of IMC 0609, Attachment 2, “Process for Appealing NRC Characterization of Inspection Findings (SDP Appeal Process)” are met.

## 02.06		Independence from Other NRC Processes

The significance of inspection findings, as characterized by the SDP, is represented by a color scheme (i.e., Green, White, Yellow, Red) that is consistent with that used for the PIs.  The color of an SDP result carries with it an assurance that all of the specific applicable process provisions of the overall SDP have been met.  Other forms of significance determination may not have the same process attributes, definitions, or assurances, and therefore should not be characterized using the SDP color scheme.  Such other forms may include severity levels of traditional enforcement and other agency probabilistic risk evaluation programs (e.g., Accident Sequence Precursor event or condition evaluations).  Keeping the SDP color scheme independent from other forms of significance determination also aids in ensuring clear and consistent public representations that inspection findings with colors are inputs to the ROP assessment of licensee performance.

02.07		External Stakeholder Participation in SDP Development and Changes

The ROP was developed with substantial involvement from both internal and external stakeholders, notably increasing openness and acceptance of the ROP.  In addition, the ROP is an integrated set of tools and processes in which changes to one component may affect other components.  Therefore, changes to the SDP must be carefully considered and, in some cases, it may be beneficial to engage external stakeholders prior to making substantive changes to the SDP or its component tools.  Such engagement is not intended to arrive at consensus, but rather to ensure that the staff has considered possible effects which could occur from a substantive change.  It is permissible to make changes which, in the judgment of the staff, do not require external stakeholder engagement.  For example, changes to SDP guidance documents that are minor or routine in nature, as outlined in SRM-COMSECY-16-0022, “Proposed Criteria for Reactor Oversight Process Changes Requiring Commission Approval and Notification,” would not require external stakeholder engagement.

# 0308.03-03	ADDITIONAL APPLICABILITY FOR SDP TOOLS THAT USE PROBABILISTIC RISK METHODS

## 03.01		Use of Computer-Based Risk Models

Experience with the SRA position since its inception in 1995 has demonstrated that, for experienced senior inspectors, an 18- to 24-month qualification program dedicated to using and understanding risk analysis techniques, is needed.  The program provides adequate skills and sufficient understanding to begin performing independent risk analyses using computer-based models.  Most risk analysts require several years to fully understand the often times subtle assumptions built into these models.

Providing computer-based tools to non-analysts (e.g., inspectors) generally leads to their use as a “black box,” wherein results are relied upon without necessarily understanding their basis.  Normally, only professionally trained risk analysts should use, review, or present the results of computer-based risk models for regulatory decision purposes, and should seek to facilitate decision-maker and other stakeholder understanding of the most influential assumptions on which the result depends, as well as the range and reasons for modeled uncertainties.  This should not, however, be construed as intending to restrict any person’s initiative to seek to understand computer-based risk model results.

Usability and performance of computer-based tools, such as SAPHIRE, have improved since the beginnings of the ROP.  Inspectors and other agency staff have access to training and resources to become more proficient in the area of risk, probabilistic risk assessment, and tools like SAPHIRE.  SRAs and qualified risk analysts are available for questions, guidance, and knowledge management.

## 03.02		Importance of a Critical and Open Deliberative Process Leading to Understanding

The reactor safety SDP is intended to openly reveal the underlying assumptions and logic that form the basis for significance determinations.  Probabilistic risk analyses are built, most often through a multi-disciplinary effort, upon many assumptions regarding a plant’s design and operation.  However, there is little assurance of the appropriateness or adequacy of the particular modeling assumptions that are most influential to a specific SDP result, without the understanding of those who are best able to judge their adequacy.  No probabilistic risk model, no matter how detailed, should automatically be accepted without understanding its influential assumptions, limitations, and uncertainties.  In particular, when differences exist between the results of risk evaluations using different plant risk models, the principal cause(s) of the differences should be reasonably understood before choosing the most appropriate result that reflects the staff’s best understanding of the issue and the relevant probabilistic modeling assumptions.

The risk-informed reactor safety SDP using the site-specific SPAR Model PRIB and SDP Workspace module in the SAPHIRE code provides a probabilistic “thinking framework” that is reasonably consistent at a high functional level with more detailed risk models.  Most importantly, this tool can foster risk communication among inspectors, staff, and management in a way that intends to provide a more widespread and common understanding of the basis for a risk result and therefore enable technically knowledgeable non-analysts to actively participate in formulating its basis.

In addition to its value as a risk estimation and communication tool, use of the site-specific SPAR Model PRIB and SDP Workspace module in SAPHIRE are effective ways for inspectors and other users to gain risk insights.  Risk analysts gain risk insights by creating, modifying, and exercising a risk model to understand the influences of the various assumptions it is built upon.  Historically, risk analysts have had great difficulty communicating risk insights to decisionmakers and inspectors.  This is at least in part because the burden of communication often rested mainly on the risk analyst, and the recipient of this one-way communication was challenged, in the typically short time available, to understand anything other than the face value results.  This “one-way” approach relies heavily on the risk analyst to understand the influential assumptions used for a specific situation being analyzed.  The reactor safety SDP offers the opportunity for inspectors to gain risk insights by processing findings through the reactor safety SDP, even when it appears they initially may be of very low (Green) significance.  Inspectors can gain valuable plant-specific risk insights, just as seeking to understand the technical aspects of an issue through reference to documents such as Technical Specifications and the Updated Final Safety Analysis Report (UFSAR) that provide~~s~~ valuable understanding of a plant’s design basis.

## 03.03		Risk-Informed SDP Tools - Specific Principles and Attributes

The principles upon which the risk-informed SDP tools were developed should continue to be met to ensure the consistency and coherence of all probabilistic SDP approaches.  In addition to the fundamental attributes for all SDP tools as noted above, any new SDP tool or change to an existing SDP tool using probabilistic risk approaches should be checked against each of the additional specific attributes, as discussed below.

	a.	Risk-informed SDP tools are intended to estimate the risk increase above the nominal baseline level of probabilistic risk (i.e., delta Core Damage Frequency (CDF) or delta Large Early Release Frequency (LERF)) for degraded conditions over a specific exposure time.  This attribute is intended to help achieve SDP objectivity.  The use of delta CDF and/or delta LERF as risk metrics as well as the concept of using the incremental conditional core damage probability (ICCDP) for evaluating the significance of degraded conditions and initiating events (IEs) caused by licensee performance deficiencies is discussed further in Section 8 of this IMC.

	b.	No matter how detailed probabilistic risk models become, they remain approximations of risks due to inherent modeling limitations and uncertainties.  In fact, the word “model” itself is used to convey the fact that the interactive physical realities of a nuclear plant’s operation and responses (e.g., failure mechanisms, timing of events, human errors of commission) cannot be specified without uncertainty and incompleteness.  Therefore, such complexities must be treated at a higher level that “models” the physical realities.  The nature of risk models is to use probability distributions to represent some of the inferred uncertainties, and the use of probabilities (and probability distributions) is then a means to relatively weight various elements of a probabilistic risk model.  It is crucial that all SDP tools using probabilistic risk methods be represented as “thinking frameworks” that are designed to enable technically knowledgeable persons to consider all the variables within this framework, and explicitly to either accept or challenge the built-in assumptions.  In short, the greatest value of risk models is that they reveal operational insights.

	c.	Every risk-informed SDP result must be understandable in terms of its influential underlying logic and assumptions.  Making probabilistic risk-informed SDP results scrutable and understandable to technically knowledgeable stakeholders helps to: (1) ensure that invalid assumptions are detected, (2) reveal any limitations of the analyses, and (3) help prevent an analysis from being manipulated to intentionally achieve a particular result.  It is necessary to engage in a deliberative process among knowledgeable stakeholders to examine and either challenge or accept important assumptions within a risk analysis.  Only through an open deliberative process that results in improved understanding can it be assured that our decision results are based on best available information and are not biased inadvertently or manipulated purposefully.

	d.	Screening questions and logic (e.g., Phase 1), for any risk-informed SDP tool, should aim to expeditiously screen findings for which there is high confidence that the significance is of very low safety significance - Green.  All such findings must still be corrected by the licensee.  The staff bears the burden of an appropriate justification for all SDP results determined as greater than Green.

	e.	If applicable, an additional SDP tool (e.g., Phase 2) for any risk-informed SDP should, as much as possible, provide a simplified and conservative risk-informed process that can be implemented by inspectors and be used as a risk communication and inspection planning tool.  The basis for an SDP result does not have to be more extensive or resource intensive than Phase 2 if this basis reflects the staff’s basic understanding of the significance, which may be checked by professional risk analysts using more detailed computer-based risk models.

	f.	A detailed risk evaluation (e.g., Phase 3) was defined to address the expected need to depart from the screening processes (e.g., Phase 1 and 2) in order to effectively characterize the risk significance.  The detailed risk evaluation is performed for a greater than Green finding and should address Phase 2 modeling assumptions that are known to be inaccurate or incomplete for the specific finding under review.  All detailed risk evaluations require the support of qualified risk analysts.

	g.	The resource burden to perform an SDP analysis is normally considered appropriate if it increases stakeholder understanding of the basis for potentially risk significant conditions, especially when an inspection finding is believed to be greater than Green.  However, it is appropriate due to SDP timeliness considerations for the staff to cease further effort to refine or review an analysis, acknowledge the limitations and uncertainties, and proceed to a final determination using best available information and reasonable technical or probabilistic judgments.  When making the decision to continue further review, especially when the additional review will cause an issue to be untimely, it is essential for the analysts and decision-makers to keep in perspective that the purpose of the SDP assessment is to determine what action the staff should take (e.g., supplemental inspection) as a result of the inspection finding.  Experience with the SDP since its inception has shown that the resources expended for additional reviews are often not commensurate with the final risk significance determination of the degraded condition and the additional actions taken by the staff.

	h.	Inspectors and analysts should use their evaluation of the significance of an inspection finding as communication tools at the earliest possible opportunity to discuss the potential significance of the finding with the licensee, and with NRC management.  Inspectors should not request a licensee to perform any specific analysis.

	i.	Inspectors and analysts should question, when appropriate, the relevant and influential assumptions related to any SDP result.  This approach focuses constructive dialogue between the NRC staff and affected licensees on gathering the technical information and making the input and assumption determinations that are a priority to support a final significance determination.  In particular, differences between risk models should be reasonably understood before a final significance determination is made.

	j.	All technical judgments made by the staff within any probabilistic-based SDP tool should have bases that are clearly observable as “reasonable,” as well as reasoned, using best available information, and not purposefully biased in a conservative manner simply because of uncertainties that are applicable in both conservative and non-conservative directions.  This approach ensures that influential assumptions made in the SDP analysis are as realistic as practicable.  This practice requires that staff technical or probabilistic judgments not be “traded off” within a risk model by allowing a conservative bias in one modeling factor simply because another factor is believed to be non-conservatively biased.  In some cases, it may be appropriate to demonstrate that a particular factor does not influence an SDP result by artificially setting it to the most conservative (i.e., greatest risk outcome) value.  In such cases, the purpose for doing this should be clearly documented.

# 0308.03-04	RISK-INFORMED VERSUS RISK-BASED

The reactor safety SDP is considered to be risk-informed,[footnoteRef:3] not risk-based, and supportive of the Commission Policy on Use of Probabilistic Risk Assessment Methods in Nuclear Regulatory Activities (1995).  As defined in SRM SECY-98-144, revision 1, dated March 1, 1999, a “risk-based” approach to regulatory decision-making is one in which such decision-making is solely based on the numerical results of a risk assessment.  Under this definition, the approach taken by the ROP (for both PIs and the SDP, where appropriate) might be considered “risk-based.”  However, the SDP is considered risk-informed by virtue of the expectation that SDP result bases are sufficiently understood by those technically knowledgeable persons (such as inspectors and technical staff) who are best positioned to critically examine the most influential probabilistic and technical assumptions, as well as by the decision-makers.  Conversely, if decisions are made without an understanding appropriate to the objectives of the ROP, they are risk-based. [3:  A risk informed process is an approach to regulatory decision-making that considers both quantitative and qualitative risk insights.] 

As further defined in this SRM, a “risk-informed” approach should consider “other (unspecified) factors.”  Historically such “other factors” included those listed in NRC Regulatory Guide (RG) 1.174 such as maintaining defense-in-depth, compliance with regulations, engineered safety margins, and prevention of over-reliance on human operators for rapid critical decisions.  However, it might be argued that these factors are all already represented, in various ways, in probabilistic risk models.  Other “factors,” such as NRC management assessment of the general quality of licensee programs, had historically involved significant subjectivity into reactor oversight decision-making.  Given the ROP objective to improve objectivity, the risk-informed approach used within the ROP fundamentally views the use of a probabilistic risk framework as a decision framework which may lend greater discipline and objectivity to the ROP decision process and less reliance on subjectivity.

# 0308.03-05	PERFORMANCE DEFICIENCIES AND DEGRADED CONDITIONS

The operation of a nuclear power plant poses risk to the public.  This risk is maintained at an acceptable level to assure public health and safety via compliance with NRC regulations and associated license requirements and implementation of good operating practices.  As such, each reactor unit has a “baseline” CDF and LERF risk.  This “baseline” provides a reference point from which a divergence is measured.  In cases where there is an increase in risk above the baseline, this divergence is described as a degraded condition.  The term “degraded condition” is intended to describe a reduction in the qualification or functionality of a structure, system or component (SSC) associated with the safety or security of the reactor plant, or other attributes related to all cornerstones.  Degraded conditions can be categorized into two ways; those that are caused by deficient licensee performance and those that are caused by random events not associated with deficient licensee performance.  Although both situations can contribute to an increase from the baseline risk, the SDP only focuses on the degraded conditions caused by deficient licensee performance.

The risk-informed inspection program as described in IMC 2515 (and IMC 2201 for Security) and its various attachments, is based on cornerstone-specific inspectable areas.  Over the course of a calendar year, NRC inspection staff selects risk-informed samples from each of the inspectable areas defined in the Baseline Inspection Program.  During the inspection process, a degraded condition may be identified.  If a degraded condition is identified, the staff makes a determination of whether or not a performance deficiency caused the degraded condition.  A performance deficiency can occur independently of whether any regulatory requirement was not met.  Conversely, a regulatory requirement can be violated without any corresponding performance deficiency.  If a performance deficiency exists and it is determined to be of morethan-minor significance, it meets the definition of an inspection finding (see IMC 0611, “Power Reactor Inspection Reports,” for more detail).  All inspection findings must be assessed by the SDP to characterize the safety or security significance.

The ROP guidance requires that the staff clearly articulate the performance deficiency that caused the degraded condition that resulted in the risk increase.  Applied across all cornerstones of safety and security, this serves the following purposes:

a. The basis for every finding is explicitly grounded in deficient licensee performance, and thus, all inspection inputs to the ROP licensee performance assessment process reflect licensee performance issues (there is no ‘credit’ offset for good performance).

b. If the staff cannot identify a licensee performance deficiency when a degraded condition occurs, then this is considered part of the “baseline risk” imposed by a large complex industrial facility, in which failures occasionally occur even though all regulatory expectations and standards are met.  Such cases should prompt consideration of the adequacy of the applicable regulatory requirements and standards, and be addressed by regulatory processes other than the ROP.  Also, when such degraded conditions involve violations of regulatory requirements, these must be documented in accordance with enforcement policy guidelines, but not treating them as “findings” parallels allowing “enforcement discretion” under traditional enforcement policies.

c. It provides a more objective and understandable basis for the staff to determine that the licensee performance deficiency, as defined, has been or is being addressed by the licensee prior to “closing” greater than Green findings from the Action Matrix.

If a relationship between a degraded condition and a performance deficiency is identified, the inspection staff must describe how the licensee performance deficiency was the proximate cause of the degraded condition.  In other words, the performance deficiency is not the degraded condition itself, it is the proximate cause of the degraded condition.  The determination of cause does not need to be based on a rigorous root cause evaluation (which might take a licensee months to complete), but rather on a reasonable assessment and judgment of the staff.  The term “proximate cause” is intended to describe a cause that was a significant contributor to the occurrence of the degraded condition.  In addition, there could be several additional causal factors that contribute, either in parallel or in series logic, to the occurrence of the degraded condition; however, only a single proximate cause needs to be linked to the performance deficiency.  Once the staff has described how a licensee performance deficiency is the proximate cause of a degraded condition, the SDP, via applicable attachments and appendices, estimates the safety or security significance of the degraded condition.  It is important to ensure the specific wording of the performance deficiency does not restrict the ability to assess all of the risk presented by the issue.

# 0308.03-06	THE INDEPENDENCE OF INSPECTION FINDINGS

Inspection findings are independent entities.  As such, each finding, which has been determined to be the proximate cause of a particular degraded condition, is assessed on its own.  In cases where an inspection finding was the proximate cause of multiple degraded conditions, the collective risk impact of the degraded conditions determines the increase in safety or security significance.  When multiple inspection findings having different proximate causes are determined to be separate and independent, yet cause degraded conditions that overlap in time, the SDP will treat each of them independently.  In other words, if there are two independent findings that are present during the same period of time, one of the degraded conditions is assessed for safety or security significance while the other degraded condition is assumed not to be in effect (i.e., in its nominal or baseline state and vice versa).

As noted in Section 5 of this IMC, the SDP only focuses on assessing the significance of degraded conditions caused by deficient licensee performance, and not degraded conditions caused by equipment out of service for planned maintenance or testing, a random failure or a random initiating event.  As such, when multiple degraded conditions are in effect during the same period of time and a performance deficiency was the proximate cause of only one of the degraded conditions, only the degraded condition caused by the independent performance deficiency is assessed by the SDP.  For example, assume there are three degraded conditions in effect over the same time period.  One degraded condition was caused by a performance deficiency, another was caused by a random failure (i.e., a failure that could not be attributed to deficient performance), and another was the result of a planned test or maintenance activity.  If all three concurrent degraded conditions were assessed collectively, the overall safety or security significance could be very significant.  However, the degraded conditions caused by the random failure and the test and maintenance activity are considered contributors to the baseline risk of the plant since they are not linked to any deficient performance.  In this example, the one degraded condition caused by the performance deficiency is assessed by the SDP as the increase above (i.e., deviation from) the baseline risk.  In this respect, the SDP is quite different from other ROP risk-informed processes (e.g., the reactive inspection program as defined in IMC 0309, “Reactive Inspection Decision Basis for Reactors”), which would assess the significance of all of the degraded conditions during the same period of time regardless of whether or not they were caused by deficient performance.

The Action Matrix, as defined in IMC 0305, “Operating Reactor Assessment Program,” is designed to receive multiple and discrete inputs, to include both performance deficiencies and PIs, and performs the “summing” of risk impacts to inform the degree of regulatory response.  In most cases, it is assumed that the “summing” result of the Action Matrix will produce an appropriate regulatory response.  If the NRC staff and management determine that the regulatory response, based on all of the inputs, is not appropriate, the staff may decide to deviate from the Action Matrix in accordance with the criteria and guidance in IMC 0305.

# 0308.03-07	TREATMENT OF UNCERTAINTY AND RISK-INFORMED DECISION-MAKING

As a tool for making risk-informed decisions in the ROP, the SDP inherently deals with incomplete information (i.e., uncertainty).  In order to make effective decisions, appropriate consideration of uncertainty needs to be applied at all stages of the process.  Consideration of uncertainty was built into the overall framework in three distinct ways.  First, the four significance thresholds of Green, White, Yellow, and Red provide sufficient margin between the threshold boundaries to account for variability in the assumptions used in the evaluation.  Secondly, the staff’s determination of the most appropriate and reasonable assumptions, where they significantly influence the SDP outcome, relies on an understanding of both the technical basis for each assumption and each assumption’s relative influence on the SDP result.  The openness of the SDP is designed to allow people with relevant technical knowledge to understand the basis for risk significance and, as appropriate, participate in formulating an appropriate decision.  Thirdly, the openness of the SDP also encourages an understanding of any known incompleteness in the evaluation.

The Significance and Enforcement Review Panel (SERP), as described in IMC 0609, “Significance Determination Process,” and IMC 0609, Attachment 1, “Significance and Enforcement Review Panel Process,” comprises NRC managers responsible for making riskinformed decisions.  The SERP adheres to an open and deliberative process in which the relevant bases for each assumption and the associated uncertainties are sufficiently understood and vetted.  The process encourages the understanding of insights and perspectives from the licensee and considers both quantitative and qualitative information in making the risk-informed decision.  As part of understanding the uncertainty, it is important that the SERP considers qualitative factors that may impact the risk outcome including both those that increase the risk and those that decrease the risk, when applicable. In addition, the SERP, as an integral part of the SDP, ensures that a timely regulatory decision is made that integrates the best available information during that time frame.

Ultimately, the final significance determination of a licensee performance deficiency is the responsibility of the SERP.  The guidance outlined in IMC 0609, its attachments and appendices, as well as IMC 0308 Attachment 3, and its appendices, serves as a decision framework consistent with the attributes listed in Section 2 of this IMC.  SERP voting members bring diversity of thought and experience to each panel.  It is not expected that each decisionmaker will view the various types of uncertainty the same way.  Similarly, each decision-maker may evaluate the impact of qualitative factors differently.  Deviation from deterministic SDP guidance, such as a flowchart, should be an extremely rare occurrence.  Following the Principles of Good Regulation with the consideration that all final determinations are consensus decisions (see additional guidance in IMC 0609, Attachment 1, “Significance and Enforcement Review Panel Process”), deviations from established SDP guidance should be justified and clearly documented. Care must be taken to ensure the basis for the deviation in both the SERP form (non-public) and inspection report does not establish a new color threshold and cannot be used on its own merits as precedent for future issues.  Otherwise, such a deviation may constitute a threshold change and would require Commission engagement in accordance with SRM-COMSECY-16-022.  Alternatively, an Action Matrix deviation, described in IMC 0305, “Operating Reactor Assessment Program,” Section 11.06 may be more appropriate.  An Action Matrix deviation may be considered for a situation such as a type of finding unanticipated by the SDP that results in an inappropriate level of regulatory attention when entering a specific column of the Action Matrix.  In any case, an ROP Feedback Form should be submitted requesting clarification of the SDP guidance and the IMC revision process should be followed.  

# 0308.03-08	QUANTITATIVE RISK METRICS OF CORE DAMAGE FREQUENCY AND LARGE EARLY RELEASE FREQUENCY

## 08.01		Technical Basis for CDF and LERF Metrics

The CDF and LERF metrics were adopted from RG 1.174 to characterize the safety significance of inspection findings and PIs for use in the NRC’s Assessment Program.  These quantitative risk metrics were chosen to establish risk-informed thresholds for applicable inspection findings and PIs in the reactor cornerstones so that indications of degraded performance could be assessed as equivalent performance metrics.  More discussion on the chosen risk metrics and associated thresholds is provided in IMC 0308, “Reactor Oversight Process Basis Document.”

To determine the significance of inspection findings, the SDP determines the increase in the baseline risk of a facility caused by the performance deficiency.  This baseline risk can be referred to as the annual CDF and LERF because it represents the frequency of an occurrence event of core damage or large early radiological release on a per year basis.

## 08.02		Treatment of Degraded Conditions and Initiating Events

The SDP is designed to estimate the risk increase from a degraded condition.  The degraded condition may be for example the unavailability of equipment or the degradation of safety functions.  For the SDP, the baseline (also referred to as the nominal or annual) CDF takes into account equipment that is removed from service for testing and maintenance at their nominal values.  The additional risk due to deficient licensee performance must be dependent on the performance deficiency and not the particular plant operational configuration during which the issue occurred.  Therefore, if a degraded equipment or function is identified to exist simultaneously with other equipment outages for maintenance or testing, the SDP evaluation will treat these outages as nominal maintenance and test unavailability since they are not associated with the performance deficiency.

In the assessment of a degraded condition, a new risk level results for the length of time (i.e., exposure time) of the degraded condition.  The significance of this degraded condition is determined by the difference between the new annual CDF due to the length of time of the degraded condition and the baseline CDF.  The new annual CDF is a weighted average of the CDF due to the degraded condition and the CDF not due to the degraded condition (i.e., the baseline risk level).  Thus, the new annual CDF is formulated as follows:

CDFnew annual = (CDFdegradation * fraction of year of degradation exposure time) + (CDFbaseline * fraction of year of degradation exposure time)

Once the new annual CDF is determined, the significance of the degraded condition is then the delta CDF, which is formulated as follows:

Delta CDF = (CDFnew annual – CDFbaseline annual) for a defined exposure period.

An equivalent quantitative value, although a different concept with probability versus frequency, would be to calculate the incremental conditional core damage probability (ICCDP).  The ICCDP is determined by quantifying the probability of a core damage during the exposure time of the degraded condition and subtracting the probability of a core damage without the degraded condition over the same exposure time.  The delta CDF or ICCDP are the risk metrics for the SDP to evaluate the significance of inspection findings, and their numerical values are consistent with the risk-informed scale and basis detailed in IMC 0308, “Reactor Oversight Process Basis Document.”

When an IE is caused by deficient licensee performance, the SDP examines the increase in the facility’s baseline risk.  The significance of the degraded condition caused by an IE is assessed by the change of the IE frequency multiplied by the basic event failure probabilities for the mitigating equipment affected by the IE.  Consistent with the Accident Sequence Precursor (ASP) Program practices (reference 13) for IE analysis, the IE frequency is set to 1.0 (because the IE actually occurred).  If any component of a mitigating system failed during the IE occurrence, including operator errors, then the failure probability of the failed equipment is set to “TRUE”.  All other IEs in the risk model are set to zero.  The overall result of this approach is expressed in a Conditional Core Damage Probability (CCDP) estimate.  Since the SDP evaluates the risk increase from a degraded condition, the significance of a performance deficiency causing an IE is determined by using the ICCDP estimate.  The ICCDP estimate is formulated as follows:

ICCDP = CCDP – Baseline core damage probability (CDP)

The baseline CDP is the probability estimate calculated using the nominal IE frequency and nominal failure probabilities of all other components affected by the IE occurrence.  In situations where the nominal IE frequency is greater than 1.0, the CCDP estimate is calculated by adding 1.0 event per year to the nominal frequency.  The net effect of the calculated ICCDP estimate represents the risk increase from a degraded condition caused by an IE.

The ICCDP estimate represents the increase in risk to the plant for the typical 24-hour mission time after the beginning of the IE.  This approach provides a reasonable assessment of the increase in the facility’s baseline risk given the IE occurrence was caused by a performance deficiency.  The assessment approach is applicable to various types of IEs and would include complicated reactor trips such as those involving the unavailability of or inability to recover condenser heat sink, main feedwater, off-site power, and various other support system failure IEs.

# 0308.03-09	USE OF THE RISK ASSESSMENT STANDARDIZATION PROJECT (RASP) HANDBOOK

Specific guidance and best practices in the use of PRA methods to assess the significance of performance deficiencies are provided in the RASP Handbook, Volume 1, Internal Events which can be accessed at this Web link:

https://www.nrc.gov/reactors/operating/oversight/program-documents.html

The RASP Handbook, “Risk Assessment of Operational Events,” is a document of methods and guidance that NRC staff should use to achieve more consistent results when performing risk assessments of operational events and licensee performance issues.  The principal users of the RASP Handbook are SRAs and headquarters risk analysts involved with event and condition assessments.  The RASP Handbook, Volume 1 provides guidance on risk analysis methods such as Common Cause Failure analysis, Human Reliability Analysis, and initiating event analyses.

# 0308.03-10	REFERENCES

1. Commission Policy Statement, “Safety Goals for the Operation of Nuclear Power Plants,” August 21, 1986.

2. Commission Policy Statement, “Use of Probabilistic Risk Assessment Methods in Nuclear Regulatory Activities,” August 16, 1995.

3. IMC 0305, “Operating Reactor Assessment Program,” June 28, 2021.

4. IMC 0307, Appendix A, “Reactor Oversight Process Self-Assessment Metrics and Data Trending,” May 29, 2020.

5. IMC 0308, “Reactor Oversight Process Basis Document,” October 4, 2017.

6. IMC 0309, “Reactive Inspection Decision Basis for Reactors,” October 28, 2011.

7. IMC 0609, “Significance Determination Process,” November 9, 2020.

8. IMC 0609, Attachment 1, “Significance and Enforcement Review Panel (SERP) Process,” August 19, 2021.

9. IMC 0609, Attachment 2, “Process for Appealing NRC Characterization of Inspection Findings (SDP Appeal Process),” January 25, 2021.

10. IMC 0609, Attachment 5, “Inspection Finding Review Board,” December 7, 2018.

11. IMC 0611, “Power Reactor Inspection Reports,” January 7, 2020.

12. IMC 0612, Appendix B, “Issue Screening Directions,” July 23, 2021.

13. NUREG-2122, “Glossary of Risk-Related Terms in Support of Risk-Informed Decisionmaking,” November 2013.

14. RG 1.174, “An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis,” Rev. 3, January 2018.

15. SECY 99-007A, “Recommendations for Reactor Oversight Process Improvements,” March 22, 1999.

16. SECY 00-0049, “Results of the Revised Reactor Oversight Process Pilot Program,” February 24, 2000.SRM SECY-98-144, Rev. 1, “White Paper on Risk-Informed and Performance-Based Regulation,” March 1, 1999.

17. SRM-COMSECY-16-0022, “Proposed Criteria for Reactor Oversight Process Changes Requiring Commission Approval and Notification,” May 12, 2017, ADAMS Accession No. ML17132A359.

18. U.S. Nuclear Regulatory Commission, “Accident Sequence Precursor (ASP) Program: Summary Description (Revision 1),” February 2020.

END

# Appendices to Attachment 3

Appendix A	Technical Basis for the At-Power Significance Determination Process

Appendix B	Technical Basis for Emergency Preparedness Significance Determination Process

Appendix C	Technical Basis for the Occupational Radiation Safety Significance Determination Process

Appendix D	Technical Basis for Public Radiation Safety Significance Determination Process

Appendix E	Technical Basis for the Baseline Security Significance Determination Process

Appendix F	Technical Basis Fire Protection Significance Determination Process (Supplemental Guidance for Implementing IMC 0609, Appendix F) At Power Operations

Appendix G	Technical Basis for Shutdown Operations Significance Determination Process

Appendix H		Containment Integrity Significance Determination Process Technical Basis

Appendix I	Technical Basis for Operator Requalification Human Performance Significance Determination Process

Appendix J	Technical Basis for Steam Generator Tube Integrity Findings

Appendix K	Technical Basis for Maintenance Risk Assessment and Risk Management Significance Determination Process

Appendix L	Technical Basis for the B.5.b Significance Determination Process

Appendix M	Technical Basis for the Significance Determination Process (SDP) Using Qualitative Criteria

Appendix N	Reserved

Issue Date:  11/05/21	15	0308 Att 3

# Attachment 1:  Revision History for IMC 0308 Attachment 3

Issue Date:  11/05/21	Att1-1	0308 Att 3