Number range CHAPTERATTACHMENT 3

SIGNIFICANCE DETERMINATION PROCESS

BASIS DOCUMENT

TABLE OF CONTENTS

1\. BACKGROUND [2](#__RefHeading___Toc198354760)

2\. FUNDAMENTAL ATTRIBUTES FOR ALL SDP TOOLS
[2](#__RefHeading___Toc198354761)

ADDITIONAL FUNDAMENTAL ATTRIBUTES FOR SDP TOOLS USING PROBABILISTIC RISK
METHODS [4](#__RefHeading___Toc198354762)

4\. RISK-INFORMED VERSUS RISK-BASED [8](#__RefHeading___Toc198354763)

5\. PERFORMANCE DEFICIENCY BASIS [8](#__RefHeading___Toc198354764)

6\. TREATMENT OF CONCURRENT DEGRADED CONDITIONS
[10](#__RefHeading___Toc198354765)

7\. TREATMENT OF UNCERTAINTY [11](#__RefHeading___Toc198354766)

8\. USE OF DELTA CDF (AND DELTA LERF) AS THE SDP RISK METRIC
[11](#__RefHeading___Toc198354767)

Attachment 3, Revision History For IMC 0308
Att1-[1](#__RefHeading___Toc198354768)

1\. BACKGROUND[]{#__RefHeading___Toc198354760 .anchor}

Commission paper SECY-99-007A, dated March 22, 1999, described a method
for assigning a probabilistic public health and safety risk
characterization to inspection findings related to reactor safety. This
risk characterization tool was the first of a set of tools that became
central elements of the Significance Determination Process (SDP) to
determine reactor inspection finding significance consistent with the
thresholds used for the risk-informed plant Performance Indicators
(PIs). This allowed inspection findings and PIs to both be used
consistently as inputs to the plant performance assessment portion of
the Reactor Oversight Process (ROP). Subsequently, other SDP tools were
developed to characterize the safety significance of issues associated
with emergency preparedness, occupational and public radiation safety,
physical protection, fire protection, shutdown operations, containment
integrity, operator requalification, and steam generator tube integrity.
These SDP tools either used quantitative risk evaluation methods or were
risk-informed through expert judgement of the staff. The resulting SDP
tools were considered acceptable starting points from which to be
continuously improved as experience was gained.

The term "SDP" should be considered as an overall process description
that includes all associated provisions designed to meet ROP objectives,
such as formal opportunities for licensee input (i.e., Regulatory
Conferences), NRC management review for any significance
characterization of greater than green (i.e., Significance and
Enforcement Review Panel - SERP), and licensee appeal options (i.e.
defined in IMC 0609, Attachment 2). The SDP is implemented using various
cornerstone-specific SDP tools, which may be referred to by their
specific names as individual "SDPs." A list of the specific SDPs used in
the ROP is provided at the end of this document. A technical basis for
each SDP is presented as separate documents within IMC 308.

2\. FUNDAMENTAL ATTRIBUTES FOR ALL SDP
TOOLS[]{#__RefHeading___Toc198354761 .anchor}

The following fundamental attributes apply to all SDPs, across all
cornerstones. All proposed SDP changes should not detract from
maintaining and improving these intended attributes.

A. Objectivity

> Each SDP tool should attempt to provide a decision logic or a decision
> framework that remains constant across applicable findings. This
> enhances objectivity by minimizing the likelihood that SDP results
> will be influenced by different value judgements held by different
> individuals. Where practicable, a probabilistic risk decision
> framework is used to add this desired discipline to SDP results. The
> test of having achieved such objectivity is when different individuals
> using a given SDP decision logic or framework arrive at the same
> result when using the same input conditions and assumptions. Achieving
> SDP result consistency and predictability is the intended outcome of
> the objectivity attribute.

B. Scrutability (openness)

> The SDP should be capable of providing a clear framework to facilitate
> communication of each significance determination and its basis among
> technically knowledgeable stakeholders (both internal and external).
> The objective of such communication is to achieve a common
> understanding, to the extent desired by any interested stakeholder, of
> SDP decision bases. This allows for broad and independent verification
> of the staff's objectivity and most directly enhances NRC public
> credibility. When a quantitative risk model is used, the greatest
> challenge to achieving this attribute is to allow stakeholders a means
> to independently assess SDP result sensitivity to the most influential
> assumptions, to understand the basis of the assumptions, and to reveal
> the limitations and uncertainties of the risk model and how these were
> considered by the staff in arriving at a final result. Given the lack
> of regulatory prescription or approval of the plant-specific
> assumptions used in licensee probabilistic risk models, it is
> essential that this scrutability attribute is effective for those
> stakeholders best positioned to identify logical or factual errors
> that may alter the final results (e.g., NRC inspectors).
>
> Since 9/11, public access to the risk-informed inspection notebooks
> used for the SDP Phase 2 process for at power conditions has been
> restricted. As such, the ability of the public to engage in open
> communications about plant specific probabilistic risk information has
> been reduced.

C. Timeliness

> The SDP is intended to support timely staff, inspection, and
> management responses to identified licensee performance deficiencies
> generally within a timeframe consistent with quarterly updates of the
> Action Matrix portion of the performance assessment component of the
> ROP. The SDP timeliness goal is therefore 90 days from the time the
> inspection finding is documented in an inspection report establishing
> the need for further review to determine significance. However, SDPs
> should also allow for a publically observable and understandable
> licensee/staff dialogue during the staff's decision process,
> consistent with the scrutability attribute noted above. Balancing SDP
> timeliness with public scrutability and obtaining best available
> information requires that all SDPs efficiently focus inspector/staff
> information exchange with a licensee, starting when a finding is
> identified. In addition, maintaining public credibility requires
> timely public notification of the existence of a potentially
> significant finding and identification of the staff's preliminary
> basis for potential significance. When appropriate, preliminary SDP
> results should reveal what influential information is being sought
> that may change the preliminary estimate. Because the SDP assesses
> licensee performance deficiencies that occurred in the past and are
> less likely to occur again, assuming effective corrective actions are
> taken, its bases need not meet the same standard as staff safety bases
> used for licensing or similar regulatory actions. This allows SDP
> decisions to proceed, particularly in the case of risk-informed SDPs,
> with residual uncertainties that may be greater than those permitted
> for NRC staff licensing decisions which, for example, may affect the
> risk profile of a reactor throughout its remaining lifetime. In cases
> where subsequent licensee analyses or other information may
> demonstrate to the staff's satisfaction that certain assumptions or
> judgements used by the staff were previously inaccurate and which
> change the SDP result, the staff should alter any remaining planned
> Action Matrix response as appropriate. If this new information arises
> after all staff response actions have been completed, the staff should
> incorporate any insights gained for future SDP results or
> improvements. When new information becomes available that the staff
> agrees changes the significance of past findings, the licensee should
> be informed of this decision in writing and the Action Matrix input
> should be adjusted accordingly from that point forward.

D. Inspection Planning

> The SDPs should inform the activities and improve the effectiveness of
> the inspectors who directly implement the inspection program. Through
> routine use and application of the SDP tools, inspectors are expected
> to naturally become sensitized and more alert for findings of
> significance, with a correspondingly higher likelihood of their
> identification if they exist. The best means for inspectors,
> decision-makers, and others to understand plant-specific risk
> insights, including why a finding either is or is not significant, is
> to regularly use and understand the SDP tools.

E. Responsibility for Significance Determinations

> Each SDP result is the sole responsibility of the NRC staff. The SDP
> is not a consensus process with a licensee or other parties and no
> staff/licensee interactions should be construed as a negotiation. The
> ROP requires the staff to make decisions using best available
> information in a timely manner and that the bases of SDP results be
> clear and publically available, to the extent practical and permitted
> by policy (e.g., security issues). The SDP affords licensees an
> opportunity to provide available information that may be useful to the
> staff in arriving at a best informed decision within a reasonable
> time. The staff is obligated to be clear about the basis for any SDP
> result and to consider licensee-provided information. The staff is not
> obligated to have "proof" of the assumptions made relative to an SDP
> result basis. Staff engineering or technical judgement is often
> required, but should be consistent with similar previous circumstances
> as appropriate and is intended to be made objective through its use
> within the appropriate SDP tool used as a decision framework. A
> licensee may not appeal the staff's judgements unless the staff did
> not consider relevant licensee input or did not follow established SDP
> process.

F. Independence from Other NRC Processes

> The significance of inspection findings, as characterized by the SDP,
> is represented by a color scheme (i.e. green, white, yellow, red) that
> is consistent with that used for the PIs. The color of an SDP result
> carries with it an assurance that all of the specific applicable
> process provisions of the overall SDP have been met. Other forms of
> significance determination may not have the same process attributes,
> definitions, or assurances, and therefore should not be characterized
> using the SDP color scheme. Such other forms may include severity
> levels of traditional enforcement approaches and other agency
> probabilistic risk evaluation programs (e.g., Accident Sequence
> Precursor event or condition evaluations). Keeping the SDP color
> scheme independent from other forms of significance determination also
> aids in ensuring clear and consistent public representations that
> inspection findings with colors are always negative inputs to the ROP
> assessment of licensee performance.

G. External Stakeholder Participation in SDP Development and Changes

> The ROP was developed with substantial involvement from both internal
> and external stakeholders, notably increasing openness and acceptance
> of the ROP. In addition, the ROP is an integrated set of tools and
> processes in which changes to one component will likely affect other
> components. Because of these factors, changes to the SDP must be
> carefully considered and in some cases it may be beneficial to engage
> external stakeholders prior to making substantive changes to the SDP
> or its component tools. Such engagement is not intended to arrive at
> consensus, but rather to ensure that the staff has considered the
> possible effects which could occur from a substantive change. It is
> permissible to make changes which, in the judgement of the staff, do
> not require external stakeholder engagement.

3.  ADDITIONAL FUNDAMENTAL ATTRIBUTES FOR SDP TOOLS USING PROBABILISTIC
    RISK METHODS[]{#__RefHeading___Toc198354762 .anchor}

```{=html}
<!-- -->
```
A.  Use of Computer-Based Risk Models

    Experience with the Senior Reactor Analyst (SRA) position since its
    inception in 1995 has demonstrated that, for experienced senior
    inspectors, an 18 to 24 month training and qualification period
    dedicated to using and understanding risk analysis techniques is
    needed at a minimum to provide adequate skill and sufficient
    understanding to begin performing independent risk analyses using
    computer-based models. Most risk analysts require several years to
    fully understand the sometimes subtle assumptions built into these
    models. Providing computer-based tools to non-analysts (e.g.,
    inspectors) generally leads to their use as a "black box," wherein
    results are relied upon without necessarily understanding their
    basis. Improved user interfaces may be developed in the future that
    might allow direct inspector use of such computer-based tools, but
    any such development should consider the ROP objectives of
    scrutability and understanding. Normally, only professionally
    trained risk analysts should use, review, or present the results of
    computer-based risk models for regulatory decision purposes, and
    should seek to facilitate decision-maker and other stakeholder
    understanding of the most influential assumptions on which the
    result depends, as well as the range and reasons for modeled
    uncertainties. This should not be construed as intending to restrict
    any person's initiative to seek to understand computer-based risk
    model results.

```{=html}
<!-- -->
```
A.  Importance of a Critical and Open Deliberative Process Leading to
    Understanding

    The reactor safety SDP, like the other SDP tools, is intended to
    openly reveal the underlying assumptions and logic that form the
    basis for significance determinations. Probabilistic risk analysis
    is built, most often through a multi-disciplinary effort, upon many
    assumptions regarding a plant's design and operation. However, there
    is little assurance of the appropriateness or adequacy of the
    particular modeling assumptions that are most influential to a
    specific SDP result, without the understanding of those who are best
    able to judge their adequacy. No probabilistic risk model, no matter
    how detailed, should automatically be accepted without understanding
    its influential assumptions, limitations, and uncertainties. In
    particular, if differences exist between the results of risk
    evaluations using different plant risk models, the principal
    cause(s) of the differences must be understood before choosing the
    most appropriate result that reflects the staff's best understanding
    of the issue and the relevant probabilistic modeling assumptions.
    The risk-informed reactor safety SDP using the risk-informed
    inspection notebooks, as "Phase 2\" of a phased approach, provides
    an intellectually accessible probabilistic "thinking framework" that
    in many cases will be consistent at a high functional level with
    more detailed risk models. Most importantly, this tool should foster
    risk communication among experienced inspectors, staff, and
    management in a way that intends to provide a more widespread and
    common understanding of the basis for a risk result and therefore
    the ability for technically knowledgeable non-analysts to actively
    participate in formulating its basis.

    In addition to its value as a risk estimation and communication
    tool, use of the reactor safety SDP Phase 2 process is an effective
    way for inspectors and other users to gain risk insights. Risk
    analysts gain risk insights by creating, modifying, and exercising a
    risk model to understand the influences of the various assumptions
    it is built upon. Detailed risk models may take months or years for
    a risk analyst to fully understand. Historically, risk analysts have
    had great difficulty communicating risk insights to decision-makers
    and inspectors. This is at least in part because the burden of
    communication often rested mainly on the risk analyst, and the
    recipient of this one-way communication was challenged, in the
    typically short time available, to understand anything other than
    the face value results. This "one-way" approach relies heavily on
    the risk analyst to understand the influential assumptions used for
    a specific situation being analyzed. The reactor safety SDP (using
    the "Phase 2\" plant-specific risk-informed inspection notebooks)
    offers the opportunity for inspectors to gain risk insights in the
    same manner that risk analysts gain them: by manipulating a risk
    model to understand the sensitivity of its underlying influences. By
    processing issues and findings through the reactor safety SDP, even
    when it appears they initially may be of very low ("green")
    significance, inspectors will gain valuable plant-specific risk
    insights, just as seeking to understand the technical aspects of an
    issue through reference to documents such as Technical
    Specifications and the UFSAR provides valuable understanding of a
    plant's design basis. In addition, the reactor safety SDP should be
    used to identify appropriate risk-informed samples within
    appropriate inspection procedures by using the prior risk insights
    gained from applying the SDP or by examining the dominating core
    damage sequences and the potential influence that any identified
    deficiencies might have in affecting multiple functions related by
    specific sequences.

```{=html}
<!-- -->
```
A.  Risk-Informed SDP Tools - Specific Principles and Attributes

    The principles below upon which the risk-informed SDP tools were
    developed should continue to be met to ensure the consistency and
    coherence of all probabilistic SDP approaches. In addition to the
    Fundamental Attributes for all SDP tools, as noted above, any new
    SDP tool or change to an existing SDP tool using probabilistic risk
    approaches should be checked against each of the below additional
    Specific Attributes.

    1\. Risk-informed SDP tools are intended to estimate the actual
    incremental risk increase above the nominal baseline level of
    probabilistic risk (i.e., delta CDF or delta LERF). It does not
    attempt to model the likelihood that a degraded condition might have
    been either better or worse than it actually was. This attribute is
    intended to help achieve SDP objectivity.

    2\. No matter how detailed probabilistic risk models become, they
    will not penetrate to any absolute "true" risk value, due to
    uncertainties and incompleteness. In fact, the word "model" itself
    is used to convey the fact that the interactive physical realities
    of a nuclear plant's operation and responses (e.g., failure
    mechanisms, timing of events, human errors of commission) cannot be
    specified in any absolute sense. Therefore, such complexities must
    be treated at a higher level that "models" the physical realities.
    The nature of risk models is to use probability distributions to
    represent some of the known uncertainties, and the use of numerical
    probabilities (and probability distributions) is then a means to
    relatively "weight" various elements of a probabilistic risk model.
    The results of risk model calculations cannot be fairly equated to
    those of engineering models (e.g., thermal-hydraulics models) that
    can be benchmarked against actual physical experiments. Thus, it is
    crucial that all SDP tools using probabilistic risk methods be
    represented as "thinking frameworks" that are designed to allow
    technically knowledgeable persons intellectual access to manipulate
    the variables within this framework, and explicitly to either accept
    or challenge the built-in assumptions.

    3\. Every risk-informed SDP result must be understandable in terms
    of its influential underlying logic and assumptions. Making
    probabilistic risk-informed SDP results scrutable and understandable
    to technically knowledgeable stakeholders helps to 1) ensure that
    inaccurate assumptions are detected, 2) clearly reveal any
    limitations of the analyses, and 3) help prevent an analysis from
    being manipulated to intentionally achieve a particular result. It
    is necessary to engage in a deliberative process among knowledgeable
    stakeholders to examine and either challenge or accept important
    assumptions within a risk analysis. Only through an open
    deliberative process that results in improved understanding can it
    be assured that our decision results are based on best available
    information and are not biased inadvertently or manipulated
    purposefully.

    4\. Phase 1, for any risk-informed SDP tool, should aim to
    expeditiously screen findings for which there is high confidence
    that the significance is Green. All such findings must still be
    corrected by the licensee. The staff bears the burden of an
    appropriate justification for all SDP results determined as greater
    than Green.

    5\. Phase 2 for any risk-informed SDP should, as much as possible,
    provide a simplified risk-informed process that can be implemented
    by inspectors and be used as a risk communication tool. The public
    basis for an SDP result does not have to be more extensive or
    resource intensive than Phase 2 if this basis reflects the staff's
    basic understanding of the significance, which may be checked by
    professional risk-analysts using more detailed computer-based risk
    models. Even when the reactor safety SDP tool (phase 2) cannot be
    used for a particular reactor safety inspection finding because its
    pre-specified imbedded assumptions are not appropriate to the
    finding, the departure from the publically documented process (i.e.,
    a phase 3 analysis) can often be understood in terms of appropriate
    adjustments that are made to the phase 2 model. Thus, the published
    SDP model may continue to serve a valuable role as a communication
    tool, even for phase 3 analyses.

    6\. Phase 3 was defined to address the expected need to depart from
    the Phase 2 guidance when the Phase 2 modeling assumptions are known
    to be inaccurate or incomplete, and requires professional risk
    analysts to be involved in all such cases. (Note that technical
    analysis to help determine appropriate Phase 2 input assumptions is
    not a Phase 3 analysis unless such analysis cause probabilistic
    assumptions to depart from those in Phase 2.)

    7\. The resource burden of performing an SDP analysis is normally
    considered appropriate if it reasonably increases stakeholder
    understanding of the risk basis for any potentially risk-significant
    condition, especially when a finding is believed to be greater than
    Green. However, it is appropriate due to SDP timeliness
    considerations for the staff to cease further effort to refine or
    review an analysis, acknowledge the limitations and uncertainties,
    and proceed to a final determination based on best available
    information and reasonable technical or probabilistic judgements.
    When making the decision to continue further review, especially when
    the additional review will cause an issue to be untimely, it is
    important for the analysts and decision makers to keep in
    perspective that the purpose of the SDP assessment is to determine
    what action the staff should take (e.g., supplemental inspection) as
    a result of the licensee performance deficiency. Experience with the
    SDP has shown that the resources expended for additional reviews are
    sometimes not commensurate with the final risk significance of the
    deficiency and the additional actions taken by the staff.

    8\. Inspectors should use their evaluation of the significance of a
    finding with the SDPs as communication tools at the earliest
    possible opportunity, to discuss the potential significance of the
    finding with the licensee and with the inspector's management.
    Inspectors should make available to the licensee their own SDP
    evaluation as a starting point for further discussion and to gain
    licensee perspectives. Inspectors should not request a licensee to
    perform any specific analysis.

    9\. Inspectors should question, when appropriate, the relevant and
    influential assumptions related to any SDP result, in part, as a
    means to focus constructive dialogue (both internal to the NRC and
    externally with licensees) on gathering the technical information
    and making the input and assumption determinations that should be
    given priority in coming to a final significance determination. In
    particular, differences between risk models must be reasonably
    understood before a final determination is made.

    10\. All technical judgements made by the staff within any
    probabilistic-based SDP tool should have bases that are clearly
    observable as "reasonable," as well as reasoned, based on best
    available information, and not purposefully biased in a conservative
    manner simply because of uncertainties which are applicable in both
    conservative and non-conservative directions. As a corollary, this
    also requires that staff technical or probabilistic judgements not
    be "traded off" within a risk model by allowing a conservative bias
    in one modeling factor simply because another factor is believed to
    be non-conservatively biased. In some cases it may be appropriate to
    demonstrate that a particular factor does not influence an SDP
    result by artificially setting it to the most conservative (i.e.,
    greatest risk outcome) value. In such cases, the purpose for doing
    this should be clearly documented.

    4\. RISK-INFORMED VERSUS RISK-BASED[]{#__RefHeading___Toc198354763
    .anchor}

    The reactor safety SDP process is considered risk-informed, not
    risk-based, and supportive of the Commission Policy on Use of
    Probabilistic Risk Assessment Methods in Nuclear Regulatory
    Activities (1995). As defined in SRM SECY-98-144 revision 1, dated
    March 1, 1999, a "risk-based" approach to regulatory decision-making
    is one in which such decision-making is solely based on the
    numerical results of a risk assessment. Under this definition, the
    approach taken by the ROP (for both PIs and the SDP, where
    appropriate) might be considered "risk-based." However, the SDP is
    considered risk-informed by virtue of the expectation that SDP
    result bases are sufficiently understood by those technically
    knowledgeable persons (such as inspectors and technical staff) who
    are best positioned to critically examine the most influential
    probabilistic and technical assumptions, as well as by the
    management decision-makers who ultimately make the decisions.
    Conversely, if decisions are made without an understanding
    appropriate to the objectives of the ROP, they are risk-based.

    As further defined in this SRM, a "risk-informed" approach should
    consider "other (unspecified) factors." Historically such "other
    factors" included those listed in Regulatory Guide 1.174 such as
    maintaining defense-in-depth, compliance with regulations,
    engineered safety margins, and prevention of over-reliance on human
    operators for rapid critical decisions. However, these factors are
    all already represented, in various ways, in a probabilistic risk
    model. Other "factors," such as NRC management assessment of the
    general quality of licensee programs, have historically inserted
    significant subjectivity into reactor oversight decision-making.
    Given the ROP objective to improve objectivity, the risk-informed
    approach used within the ROP fundamentally views the use of a
    probabilistic framework as a decision-framework which lends greater
    intellectual discipline and objectivity to the ROP decision process
    and less reliance on subjectivity.

    5\. PERFORMANCE DEFICIENCY BASIS[]{#__RefHeading___Toc198354764
    .anchor}

    All inspection findings must originate with the staff's
    determination that a licensee performance deficiency exists. A
    performance deficiency may exist independently of whether any
    regulatory requirement was violated. Conversely, a regulatory
    requirement may be violated without any corresponding performance
    deficiency. The need to characterize the performance deficiency
    arises from the probabilistic treatment of significance
    determination as discussed further below, but the concept is carried
    throughout all the cornerstone SDPs.

    Probabilistically, a reactor plant creates an incremental risk to
    the public that is maintained very low through compliance with NRC
    regulations, license requirements, and good operating practices. A
    plant's "baseline" risk (using the surrogates of CDF and LERF for
    public risk) is founded upon a risk model of that plant using
    failure probabilities and initiating event frequencies generally
    derived from industry-wide historical experience, sometimes modified
    where appropriate from plant-specific experience (e.g., Loss of
    Offsite Power frequency might be higher than the average for a plant
    subject to hurricanes). The risk-informed SDP uses each plant's
    "baseline" risk as the reference point for estimating the
    incremental increase in risk caused specifically by a degraded
    condition that could increase an initiating event frequency or
    reduce the probability of successful mitigation, and that stemmed
    from a licensee performance deficiency that is subsequently remedied
    (i.e., the degradation was temporary and not expected to recur). The
    ROP guidance requires that the staff clearly articulate the
    performance deficiency that caused the degradation that resulted in
    the risk increase. Applied across all cornerstones of safety, this
    serves the following purposes:

    1\) The basis for every finding is explicitly grounded in deficient
    licensee performance, and thus all inspection inputs to the ROP
    licensee performance assessment process reflect licensee performance
    problems (there is no 'credit' offset for good performance).

    2\) If the staff cannot identify a licensee performance deficiency
    when a degraded condition occurs, then this is considered part of
    the "baseline risk" imposed by a large complex industrial facility,
    in which failures occasionally occur even though all regulatory
    expectations and standards are met. Such cases should prompt
    consideration of the adequacy of the applicable regulatory
    requirements and standards, and be addressed by regulatory processes
    other than the ROP. Also, when such degraded conditions involve
    violations of regulatory requirements, these must be documented in
    accordance with enforcement policy guidelines, but not treating them
    as "findings" parallels allowing "enforcement discretion" under
    traditional enforcement policies.

    3\) It provides a more objective and understandable basis for the
    staff to determine that the licensee performance deficiency, as
    defined, has been or is being addressed by the licensee prior to
    "closing" greater than green findings from the Action Matrix.

    Based on the above logic, the definition of a performance deficiency
    requires the staff to make a reasonable determination that the
    licensee intended to meet some requirement or standard and they did
    not, having had the opportunity to forsee, identify, or correct the
    performance deficiency that led to not meeting the requirement or
    standard. Such a requirement or standard need not be directly
    imposed by the NRC. Licensee good operating practices are expected
    as a means to ensure safety and minimize risk, and may be
    implemented as initiatives that go beyond regulatory requirements
    (e.g., management of shutdown safety by following industry-developed
    guidelines). When such self-imposed standards or requirements are
    not met due to deficient performance, this may cause an estimable
    risk impact and should be the basis for a finding. Note that this
    differs from the pre-ROP inspection program in that findings need
    not only be associated with violations of regulatory requirements,
    which is in keeping with a risk-informed program emphasis.

    Finally, it is important to recognize that discernable risk
    increases come from degraded plant conditions, both material and
    procedure/process in nature, and that the performance deficiency
    should most often be identified as the proximate cause of this
    degradation. In other words, the performance deficiency is not the
    degraded condition itself, it is the proximate cause of the degraded
    condition. This determination of cause does not need to be based on
    a rigorous root-cause evaluation (which might require a licensee
    months to complete), but rather on a reasonable assessment and
    judgement of the staff. For those findings proposed to be of greater
    than green significance, the Significance and Enforcement Review
    Panel (SERP) functions to provide oversight to ensure the staff's
    reasonableness and consistency of identifying licensee performance
    deficiencies and their nexus to the degraded conditions that cause
    the increase in public risk.

    6\. TREATMENT OF CONCURRENT DEGRADED
    CONDITIONS[]{#__RefHeading___Toc198354765 .anchor}

    The use of the ROP Action Matrix demands that its inputs be discrete
    and not duplicative. When separate licensee performance deficiencies
    result in separate degraded conditions that overlap each other in
    time, there could be a synergistic risk impact during the
    overlapping period of time that is greater than the sum of the two
    parts. This possibility motivated the development of a rule to treat
    such conditions. This rule considered the following points.

    The staff is responsible to define licensee performance
    deficiencies. Where the proximate cause of multiple degraded
    conditions is the same, there is likely to be only one finding
    (based on the identified performance deficiency related to the
    proximate cause) and the risk impact of the collective degraded
    conditions (including any overlapping conditions) is then
    appropriately used as the basis for the SDP result. However, this
    concept could be taken to an extreme of defining all licensee
    performance deficiencies as "management weakness" or something
    similarly fundamental. Doing so would then cause all degraded
    conditions to be manifestations of a single and possibly
    never-ending finding, would make unnecessary the need for an Action
    Matrix, and may require the staff to devise a continuous risk meter
    or similar substitute for the Action Matrix. Thus, a "floor" was set
    for the implementation of this concept that is consistent with the
    ROP framework, in that no performance deficiency should be defined
    at a level associated with the ROP cross-cutting issues (i.e., human
    performance, safety-conscious work environment, and problem
    identification and resolution) or more fundamentally. Although
    artificially setting this "floor" may create a philosophical
    inconsistency with use of a probabilistic thinking framework (i.e.,
    if there is really a known common-cause effect taking place, then it
    should be explicitly acknowledged in a probabilistic model), it
    remains necessary for practical reasons as long as the Action Matrix
    continues in its present form. Concerns about possible insufficient
    regulatory responses arising from this approach are also mitigated
    as noted below.

    When multiple findings are judged to be separate and independent of
    each other, yet cause degraded conditions that overlap in time, the
    SDP will treat them independently and thereby allow the Action
    Matrix to receive multiple inputs and perform the "summing" of these
    risk impacts (along with all other cornerstone inspection and PI
    inputs) to inform the degree of regulatory response required for
    that licensee. This approach has the valuable benefit of allowing
    individual SDP risk determinations to proceed independently and
    therefore generally in a more timely fashion. In most cases, it is
    believed that the "summing" result of the Action Matrix will produce
    a regulatory response equal to that of combining the multiple
    findings into a single finding of collective risk significance. The
    staff's use of risk tools in other processes (e.g., Accident
    Sequence Precursor Program) or simply for the purpose of
    understanding collective risk significance may be appropriate, and
    in those cases where it is determined that such understanding should
    alter the agency's response to the specific licensee, the staff may
    invoke a deviation from the Action Matrix by appropriate
    justification and meeting the prescribed process requirements of the
    ROP.

    7\. TREATMENT OF UNCERTAINTY[]{#__RefHeading___Toc198354766 .anchor}

    One of the strengths of probabilistic risk assessments is that
    certain uncertainties can be treated explicitly. The openness of SDP
    deliberation helps to reveal influential assumptions and
    uncertainties. The risk-informed SDP process inherently and
    qualitatively considers uncertainty. The "point-estimate" numerical
    risk values are understood to represent averages/means which are
    central tendency parameters of probability distribution functions
    and as such do not represent any dispersion characteristic (i.e.
    spread) of these functions. In addition, it is recognized that
    limitations exist for any specific risk model, in terms of
    completeness of the model and basis for its various probabilistic
    (e.g., accident sequence logic) and technical (e.g., success
    criteria for mitigation functions) assumptions. Consideration of
    uncertainty was built into the overall risk-informed SDP process
    three distinct ways. First, the decision thresholds (i.e.,
    significance color categories) were chosen to be one
    order-of-magnitude intervals. Thus, a numerical "point-estimate"
    determination that falls anywhere within a particular order of
    magnitude is given the same color characterization in recognition of
    the uncertainty stemming from the known variability of certain input
    data and assumptions. Second, the staff's determination of the most
    appropriate and reasonable assumptions, where they significantly
    influence the SDP result, rests on an understanding of both the
    strength of the basis for each assumption and the assumption's
    relative influence on the SDP result. The openness of the SDP
    process is designed to allow persons with relevant technical
    knowledge to understand the basis for risk significance and, as
    appropriate, participate in formulating the most appropriate
    assumptions. Third, the openness of the SDP process also encourages
    understanding of any known incompleteness of the risk model being
    used. The degree to which any such incompleteness might affect the
    SDP result then becomes a clearly revealed consideration within the
    deliberation process. Overall, an open and deliberative process that
    invites relevant input ensures that the bases upon which assumptions
    are made, and the residual uncertainties inherent in such bases, are
    clearly understood and considered in arriving at a final SDP result.

    8\. USE OF DELTA CDF (AND DELTA LERF) AS THE SDP RISK
    METRIC[]{#__RefHeading___Toc198354767 .anchor}

    The relationship between delta CDF and CCDP (conditional core damage
    probability) is described below. This discussion is also
    correspondingly applicable to the relationship between delta LERF
    and CLERP (conditional large early release probability).

    A.  Different Definitions of CCDP -

        In the below discussion, the use of "CCDP" is primarily with
        reference only to degraded conditions. The term CCDP is also
        commonly used to represent the probability that a core would
        have gone to a damaged state given that (i.e., "conditioned on")
        a specific initiating event occurred and the actual plant
        equipment and operator responses are accounted for. This "event"
        use of CCDP represents the remaining probabilistic "margin"
        (related to defense-in-depth) to core damage at a precise moment
        in time, that of the event itself. This concept of CCDP is
        significantly different from that which refers to a degraded
        condition which may alter the plant's risk over a period of time
        and is related to the plant's ability to mitigate a number of
        different possible initiating events.

    B.  Overview

        The risk-informed SDP tools are designed to estimate the
        increase in annualized CDF risk due to identified deficiencies
        in licensee performance that led to unavailability of equipment
        or safety functions, or to the increase in initiating event
        frequencies. This increase is measured from the normal
        annualized CDF that results from routine plant operation. The
        additional risk contributions caused by deficient licensee
        performance (as characterized by the SDP) are assumed to be
        additive to this normal annualized CDF which already includes
        the risk contribution due to the probabilities of equipment
        failures expected occasionally for industrial facilities of this
        size and complexity.

        Another contribution to normal annualized CDF is caused by
        planned preventive maintenance and testing activities which
        cause the CDF at any particular moment in time to fluctuate
        dependent upon the changes in plant equipment status. The
        additional annualized CDF risk due to deficient licensee
        performance must be dependent only upon the performance issue
        itself and not the particular plant configurations during which
        the issue occurred. Therefore, if a degraded equipment or
        function is identified to exist simultaneously with equipment
        outages for preventive maintenance or testing, the SDP inputs
        cannot include the contribution of the maintenance or testing,
        since this is already included in the normal annualized CDF
        against which the change is being measured. This
        non-consideration of routine maintenance and testing is a
        departure from the historical enforcement practice of including
        the consideration of any additional equipment unavailability
        that made the loss of function due to a deficiency more severe,
        even if the added unavailability were due to routine
        maintenance.

        The SDP actually estimates the CCDP given the degraded condition
        which resulted from the performance deficiency, for the time
        this degradation existed. The nominal CDP, which accounts for
        normal maintenance, during this time, is subtracted from the
        CCDP to obtain the change in CDP due to the degraded condition
        alone (without consideration of any specific maintenance
        configuration that might have existed). This numerical result is
        then normalized by dividing it by 1 year to arrive at a delta
        CDF in units of "per year." If equipment outages due to
        maintenance were included in this delta CDF estimation, the
        result would potentially render results of higher significance.
        This would result in assessments of the risk impact of licensee
        performance that inappropriately would depend as much on the
        licensee's appropriate conduct of on-line maintenance as on the
        licensee's deficient performance.

        The objective of using the SDP is to characterize the
        significance of inspection findings in a manner that is
        comparable to performance indicators for use in the NRC Action
        Matrix. The reactor safety cornerstone performance indicator
        thresholds were developed based on the increase to annualized
        CDF represented by the value of the indicators. Thus, in
        comparing and "adding" the effects of PIs and inspection
        findings within the Action Matrix, it is necessary to use the
        same risk metric.

    C.  Basis in Regulatory Guide 1.174 and Mathematical treatment of
        delta CDF and CCDP -

        In developing the new performance assessment process one of the
        tasks was to establish risk-informed thresholds for PIs and
        corresponding thresholds for inspection findings, so that
        indications of performance degradation obtained from inspection
        findings and from changes in PI values could be put on equal
        footing. The basis documents for establishing risk guidelines
        were Reg Guide 1.174, which bring in the Regulatory Analysis
        Guidelines, and the Safety Goal Policy Statement. The metrics
        that have been adopted in RG 1.174 for the characterization of
        risk are CDF and LERF. These are essentially surrogates for
        health effects, which are the principal metrics in the Safety
        Goal Policy Statement, and, in addition, they are consistent
        with the metrics used in the Regulatory Analysis Guidelines. In
        RG 1.174, acceptance guidelines were established for assessing
        changes to the licensing basis of a plant. Acceptance is
        predicated on increases in CDF and LERF implied by the change to
        the licensing basis being small.

        The philosophy behind the establishment of the thresholds on PIs
        and inspection findings was essentially to assume that an
        increase in PI values or conditions indicated by the finding,
        would, if their root causes were uncorrected, be equivalent to
        accepting a de facto increase in the CDF and LERF metrics. This
        is clearer for the PIs than it is for the inspection findings,
        which may relate to a time-limited deficient condition. For such
        cases, the model used here is that the condition is indicative
        of an underlying performance issue that, if uncorrected, would
        be expected to result in similar occurrences with the same
        frequency.

        Therefore, the challenge is how to calculate the impact of
        changes in PI values and inspection findings on these metrics.
        Since PIs correspond (at least in some approximate sense) to
        parameters of PRA models, it is relatively straightforward to
        make the connection between changes in PI values to changes in
        risk. The thresholds were established by taking a set of PRA
        models, and varying the parameter that corresponded to the PI
        until the change in CDF became 1E-05 or 1E-04/ry, and these
        values were chosen as the thresholds for the white/yellow and
        yellow/red thresholds. Therefore, the risk significance of an
        inspection finding should be measured in the same way. When the
        impact of the finding can be characterized in terms of the
        unavailability of an SSC for some specified duration, then the
        SDP gives an estimate of the change in CDF.

        However, a finding that is associated with one SSC may be
        compounded by another SSC being out for routine maintenance or
        being in a (unrevealed) failed state. Enforcement practices have
        typically escalated enforcement for such coincidences, even
        though, performing maintenance can be a necessary and beneficial
        operating practice. Also, failures will, and should be expected
        to, occur, and if they are unrevealed even though the licensee
        is following acceptable practices, it may penalize the licensee
        for an occurrence beyond their reasonable and expected control.
        The NRC Accident Sequence Precursor process typically includes
        any additional failures or unavailability in its assessment of
        risk significance. Therefore the question has arisen whether
        when applying the SDP, the failed or unavailable SSC that is not
        related to the finding per se should be treated as a failed or
        unavailable SSC.

        In assessing the risk significance of the finding on the same
        basis as the PIs, it has to be remembered that the metric is CDF
        (LERF), and that CDF is an estimate based on a weighted average
        of all possible outcomes. Thus the key issue is how the finding
        is defined. If the finding relates to a specific reason that a
        particular SSC is, or has the potential to be, unavailable (for
        example a failure to follow a particular procedure) then it
        should, for the purposes of the SDP, be irrelevant whether
        another SSC is unavailable because it has been taken out of
        service for routine or scheduled maintenance, unless the cause
        of the reason for the maintenance is also related to deficient
        licensee performance. If the condition of independence holds,
        the finding has only by chance occurred when the second SSC was
        unavailable, and the occurrences are uncorrelated. The chance
        that the events could have occurred simultaneously is accounted
        for in the SDP by using unavailability values for the redundant
        or diverse trains or systems that reflect the probability that
        they are unavailable.

        However, the finding could be related to the component taken out
        of service. For example, if the reason the second component were
        taken out of service at the same time as the first component
        were unavailable is that a required check on operability of the
        first was not performed, the finding relates to the failure of
        the administrative process. The finding would relate to the
        second component, i.e. that which was taken out of service, and
        the failure of process being addressed could have and, given the
        assumption that, if uncorrected, it more than likely could have
        occurred when the first component was available. Therefore, in
        assessing the significance of the failure to follow the
        administrative process in the SDP, the first component should
        not be regarded as having failed. Instead, the SDP would be
        entered with the second component out of service as the
        condition of degradation, and the first component treated as
        being an alternate train. This is a somewhat contrived example
        which raises other questions in that if it were a generic
        degradation of the adherence to administrative procedures rather
        than for a specific procedure this would be impossible to
        analyze with the SDP.

        Including the plant configuration as part of the finding is
        equivalent to risk tracking, as captured by a safety monitor for
        example, and is a record of one sample of all the possible
        outcomes, or a snapshot of the condition of the plant. It is
        however, not directly related to the estimate of the risk impact
        in the sense required for calculating ΔCDF. It is more
        accurately characterized as a measure of the margin to core
        damage, which could be measured as a CCDP (for plant trip
        events), or a conditional core damage frequency (CCDF). There
        are currently no acceptance guidelines for CCDF or CCDP. Note
        that the ICCDP in RG 1.177 could be interpreted as an estimate
        of ΔCDF, in the same way as discussed above. However, we do not
        have a criterion for either CCDP or CCDF.

        Suppose we were to derive appropriate criteria, we would still
        have to answer the question of how to interpret the data to
        generate a useful measure of the margin to core damage. If some
        of the failures that contribute to a high CCDP or CCDF are truly
        random, then does it make sense to penalize the licensee because
        they occur? It could be argued that since failures can and do
        occur, and as long as the indicators associated with those SSCs
        are acceptable, then the licensee could have done little to
        avoid this situation. What makes much more sense is to track the
        residual CCDP or CCDF with respect to the deliberate changes to
        plant configuration, i.e., those unavailabilities over which the
        licensee has direct control.

        A mathematical demonstration that, for a simple unavailability,
        ΔCDP equates to ΔCDF over one year follows below.

        We want to show that, for a given basic event representing
        unavailability of an SSC, that the following holds:

        CDF = \[T~0~/(T~0~ + T~1~)\].CDF~0~ + \[T~1~/(T~0~ +
        T~1~)\].CDF~1~,

        where T~0~ is the time when the SSC is available, and T~1~ the
        time when it is not, CDF~0~ is the CDF evaluated with the SSC
        available (i.e., unavailability = 0), and CDF~1~ is the CDF
        calculated with the SSC unavailable, (i.e., unavailability = 1).
        T~0~ + T~1~ = 1 year.

        That is, the addition of a CDP coming from a particular
        unavailability finding equates to an increase in CDF.

        Note that the cutsets for CDF can be split into two groups,
        those that do not contain the SSC , which therefore corresponds
        to CDF~0~ , and those that contain the SSC. These cutsets would
        typically be of the form x.y.z.U, where U is the unavailability
        of the SSC in question.

        CDF~1~ then is CDF~0~ + Σ(x.y.z.U) with the value of U set to 1.

        Thus the equation above becomes

        CDF = CDF~0~ + \[T~1~/(T~0~ + T~1~)\].Σ(x.y.z.U) with the value
        of U set to 1.

        Now, if we replace the value of U by \[T~1~/(T~0~ + T~1~)\], we
        get the usual CDF expression.

        Thus, while what we are doing in the SDP is evaluating a CDP,
        i.e., an integral under a CDF spike, and translating it to an
        increase in CDF for the year in which the finding occurred.

        END

        Appendices to Attachment 3 (presented as separate documents in
        the Manual Chapter)

        Appendix A Significance Determination of Reactor Inspection
        Findings for At-Power Situations

        Appendix B Emergency Preparedness Significance Determination
        Process

        Appendix C Occupational Radiation Safety Significance
        Determination Process

        Appendix D Public Radiation Safety Significance Determination
        Process

        Appendix E This document has been designated as containing
        "Official Use Only" information and is therefore not available
        to the public.

        Appendix F Fire Protection Significance Determination Process

        Appendix G Shutdown Operations Significance Determination
        Process

        Appendix H Containment Integrity Significance Determination
        Process

        Appendix I Operator Requalification Human Performance
        Significance Determination Process

        Appendix J Steam Generator Tube Integrity Findings Significance
        Determination Process

        Appendix K Maintenance Risk Assessment and Risk Management
        Significance Determination Process

        Attachment 3,

        Revision History For IMC 0308[]{#__RefHeading___Toc198354768
        .anchor}

+--------+-------+-----------------------+-----+--------+--------------+
| Comm   | Issue | Description of Change | Tr  | Tr     | Comment      |
| itment | Date  |                       | ain | aining | Resolution   |
| Tr     |       |                       | ing | Comp   | Accession    |
| acking |       |                       | Nee | letion | Number       |
| Number |       |                       | ded | Date   |              |
+--------+-------+-----------------------+-----+--------+--------------+
| N/A    | 10/   | This IMC has been     | N   | N/A    | N/A          |
|        | 16/06 | revised to            | one |        |              |
|        |       | incorporate comments  |     |        |              |
|        | CN    | from the Commission   |     |        |              |
|        | 0     | in which the term     |     |        |              |
|        | 6-027 | public confidence has |     |        |              |
|        |       | been change to        |     |        |              |
|        |       | openness              |     |        |              |
+--------+-------+-----------------------+-----+--------+--------------+
|        |       |                       |     |        |              |
+--------+-------+-----------------------+-----+--------+--------------+
|        |       |                       |     |        |              |
+--------+-------+-----------------------+-----+--------+--------------+
|        |       |                       |     |        |              |
+--------+-------+-----------------------+-----+--------+--------------+
|        |       |                       |     |        |              |
+--------+-------+-----------------------+-----+--------+--------------+