NOTATION VOTE

RESPONSE SHEET
TO: Annette Vietti-Cook, Secretary
FROM: COMMISSIONER SVINICKI
SUBJECT: SECY-14-0087: QUALITATIVE CONSIDERATION OF

FACTORS IN THE DEVELOPMENT OF REGULATORY
ANALYSES AND BACKFIT ANALYSES

Approved __ XX Disapproved Abstain

Not Participating

COMMENTS: Below ___ Attached _XX__None___

GNATURE

oP 4) M5
DATE

Entered on “STARS” Yes \/ No___



Commissioner Ostendorffs Comments on SECY-14-0087,
“Qualitative Consideration of Factors in the Development of Regulatory Analyses and
Backfit Analyses”

The matter before the Commission is not whether qualitative factors should be considered in
regulatory decision-making. The consideration of qualitative factors has been a part of our
regulatory framework since 1977'. Rather, the matter before the Commission is whether or not
the NRC’s regulatory guidance should be revised to provide additional clarity on the use of
these factors.

In SECY-14-0087 the staff stated that there is “significant precedent for qualitatively considering
factors, both within the NRC and externally,” and the staff concluded that the current regulatory
framework is sound. | agree with the staff's conclusion that the current regulatory framework is
sound and that our historical consideration of qualitative factors to complement quantitative
analyses is well founded. Consistent with the principles of good regulation, | support efforts to
ensure Clarity in the consideration of qualitative factors. As the ultimate decision-makers, it is
essential that the Commission have a clear and complete understanding of all of the factors
used to develop the staffs proposals. Therefore, | approve the staff's recommendation, as
supported by the Advisory Committee for Reactor Safeguards, to update guidance regarding
qualitatively considering factors in regulatory analyses and backfit analyses. My approval is
subject to the following high-level principles:

1. The staff should continue to quantify costs and benefits to the maximum extent possible.

2. The staff should continue to strive to improve its methods for quantitative analyses,
including the treatment of uncertainties.

3. The staff should use the best information available to develop realistic estimates of the
cost of implementing proposed requirements.

4. Consistent with past NRC practice, qualitative considerations should be used to
supplement quantitative analyses only when quantitative analyses are not possible or
practical (i.e., due to lack of methodologies or data).

5. To provide the best available information to support Commission decisions and to
improve transparency, the revised guidance should provide expectations for how the
staff will articulate its rationale for how and when qualitative factors have been
considered in the formulation of staff recommendations.

This approval does not authorize any expansion of the consideration of qualitative factors in
regulatory analyses or backfit analyses. Prior to finalization, the staff should provide a detailed
plan and schedule and the draft revised guidance to the Commission for information.

‘ SECY-77-388A, “Value-Impact Guidelines,” dated December 19, 1977, included instructions for the staff
to consider both qualitative and quantitative factors in assessing the value of proposed alternatives
Guidance to consider qualitative factors was included in the 1983 issuance of NUREG/BR 0058,
“Regulatory Analysis Guidelines of the U.S. Nuclear Regulatory Commission,” and all subsequent
revisions.


Commission.’ These are also cases where the qualitative factor of “improved defense in depth”
became the hazy basis for action (to the detriment of transparent decision making) and where
the Commission appropriately discredited the use of such an ill-defined catchall as a
determinant of outcomes.

As a result of making a priority of activities to improve quantitative analyses, some work on
qualitative factors guidance would inevitably be deferred. This is of lesser concern as the
flexibility available to analysts under current guidance — to adjust and apply the right factors and
the right assessment tools for the specific information available and the specific decision to be
made — has served, and can continue to serve, the agency well, just as long as the agency
principle of producing well-documented and transparent analysis continues to be honored.
Under such principles, | continue to support the use of stylized calculations by NRC analysts as
something appropriate to the agency's needs. Further, | remain skeptical that one set of
guidelines can be developed enveloping all types of analysis. That being said, | agree with the
policy statement of the American Nuclear Society that “perhaps most importantly, qualitative
factors must never be used to impose subjective judgments or allow political factors to override
sound technical evaluations.”

| join Chairman Burns and Commissioner Ostendorf in specifically disapproving any expansion
of the use of qualitative factors (for example, further work on the development and use of a
decision matrix as described in Enclosure 3 of the paper is not approved) and in disapproving
the staff developing any guidance that further expands the use of qualitative factors beyond
what is currently performed by the staff.

| also agree with my colleagues that the staff should continue to strive to improve its methods
for quantitative analyses, including the treatment of uncertainties. The staff should use the best
(i.e., the most authoritative and realistic) information available to develop estimates of the cost
of implementing proposed requirements. Consistent with past practice, qualitative
considerations should be used to supplement quantitative analyses only when quantitative
analyses are not possible or practical. To improve transparency and decision making, any
revised guidance should outline how the staff will articulate its rationale for the selection of
qualitative factors and describe with specificity how these factors were used in the analysis,
including sensitivity analyses.

Any updated guidance resulting from the Commission's decision in this matter should lie before
the Commission, in draft form, for a period no shorter than 10 working days prior to its issuance
by the staff. | join my colleagues in supporting direction to the staff that any revised guidance
should continue to require quantifying costs to the maximum extent and using qualitative factors
to inform decision making only in limited cases.

| appreciate the patience of my colleagues, as | took some time developing this vote. | reviewed
a lot of material in the process. The topic poses many complexities, some quite philosophical
and some which run up against my own bull-headed ideas about liberty and choice. These
issues are best summed up in the following piece, which was included by my reactor advisor in
the three large volumes of information compiled for me as reference material and history on the
use of cost benefit analysis in government decision making. It is not strictly relevant to my vote,
but | include it as dicta, for those interested.

? This is parallel to determinations of adequate protection, which are also the Commission’s domain.

2


Commissioner Svinicki’s Comments on SECY-14-0087
Qualitative Consideration of Factors in the Development of
Regulatory Analyses and Backfit Analyses

| approve the staff's recommendation that it update agency guidance regarding the
consideration of qualitative factors.’ Because the staff's planned set of actions is not clearly laid
out in the paper, my approval is necessarily subject to the terms and limitations contained in this
vote.

The use of qualitative factors in agency analyses is and should continue to be disciplined,
transparently documented, and rare.

As expressly stated in NRC guidance [NUREG/BR-0058, “Regulatory Analysis Guidelines of the
U.S. Nuclear Regulatory Commission,” Revision 4]:

Estimated [costs] and [benefits] should be expressed in monetary terms whenever
possible and expressed in constant dollars from the most recent year for which price
adjustment data are available. Consequences that cannot be expressed in monetary
terms should be described and quantified in appropriate units to the extent possible... .
However, the staff needs to make every reasonable effort to apply alternative tools that
can provide a quantitative perspective and useful trends concerning the value of the
proposed action. Even inexact quantification with large uncertainties is preferable to no
quantification, provided the uncertainties are appropriately considered.

Consistent with this, staff should, as a first priority, dedicate its staff resources and expertise to
improving its capabilities in the conduct of quantitative cost-benefit analyses. Doing so will
signal a clear and unequivocal commitment to implementing the improvements agreed to by
NRC to the U.S. Government Accountability Office (GAO) and the Congress in response to
report GAO-15-98, “Nuclear Regulatory Commission: NRC Needs to Improve its Cost
Estimates by Incorporating More Best Practices.”

In this report, GAO stated its conclusion that:

NRC states that these [cost estimating] analyses ensure that the agency bases its
decision on adequate information, and that the staff arrives at its decisions by following a
systematic and disciplined process that is also open and transparent. However, we
[GAO] found that NRC’s procedures for developing the cost estimates do not fully
incorporate relevant cost estimating best practices. Without procedures that adhere to
cost estimating best practices, NRC does not have assurance that it is producing reliable
cost estimates. Thus, the Commissioners may not have adequate information on which
to base their decisions. . . . [U]nless NRC revises its procedures . . . the process may
continue to yield unreliable cost estimates that could negatively affect a decision-making
process that is intended to ensure public safety.

As examples, | commend to my colleagues the proposed shutdown rule [SECY-97-168; more
distant history] and the proposed filters order [SECY-12-0157; more recent history] as case
studies demonstrating that determinations on the appropriate degree or weight of application of
qualitative factors in regulatory decision making are, and must remain, the province of the

' The staff's insistence on the tortured syntax of “qualitative consideration of factors” is a confounding
obfuscation and should, in this reader's opinion, be dropped from usage entirely. There is no apparent
reason for the NRC to speak differently than the rest of the world


RESPONSE SHEET

TO: Annette Vietti-Cook, Secretary
FROM: COMMISSIONER OSTENDORFF
SUBJECT: SECY-14-0087: QUALITATIVE CONSIDERATION OF

FACTORS IN THE DEVELOPMENT OF REGULATORY
ANALYSES AND BACKFIT ANALYSES

Approved XX_ Disapproved Abstain

Not Participating

COMMENTS: Below___ Attached _XX_None___

Ltt hud,

SIGNATURE 4%

OS rk

DATE

Entered on “STARS” Yes__X No___


ett on UNITED STATES

NUCLEAR REGULATORY COMMISSION
WASHINGTON, D.C. 20555-0001

g
s March 4, 2015
SECRETARY COMMISSION VOTING RECORD

DECISION ITEM: SECY-14-0087

TITLE: QUALITATIVE CONSIDERATION OF FACTORS IN THE
DEVELOPMENT OF REGULATORY ANALYSES AND
BACKFIT ANALYSES

The Commission (with all Commissioners agreeing) approved the subject paper as recorded in
the Staff Requirements Memorandum (SRM) of March 4, 2015.

This Record contains a summary of voting on this matter together with the individual vote
sheets, views and comments of the Commission.

Annette L. Vietti-Cook
Secretary of the Commission

Attachments:
1. Voting Summary
2. Commissioner Vote Sheets

cc: Chairman Burns
Commissioner Svinicki
Commissioner Ostendorff
Commissioner Baran
OGC
EDO
PDR


Commissioner Baran’s Comments on SECY-14-0087, “Qualitative Consideration of
Factors in the Development of Regulatory Analyses and Backfit Analyses”

It is well-established that regulatory analysis “must take into account benefits and costs, both
quantitative and qualitative.”’ Executive Order 12866 explains, “Costs and benefits shall be
understood to include both quantifiable measures (to the fullest extent that these can be usefully
estimated) and qualitative measures of costs and benefits that are difficult to quantify, but
nevertheless essential to consider.”? Office of Management and Budget Circular A-4 reiterates
that “[a] complete regulatory analysis includes a discussion of non-quantified, as well as
quantified benefits and costs.”* As stated in the paper, NRC’s regulatory and backfit analyses
have been consistent with the approach described in Executive Order 12866 and Circular A-4
for decades.

The question before the Commission, therefore, is not whether staff should consider qualitative
factors in its regulatory analysis or should do so with greater frequency, but simply whether the
NRC staff should update the agency’s cost-benefit guidance to assist the staff in articulating
how it uses qualitative factors.

The staff is in the midst of a multi-year, two-phased effort to revise NRC’s cost-benefit analysis
guidance documents. Although the staff believes that “the current framework for the qualitative
consideration of factors is satisfactory,” the staff proposes to update the guidance’s discussion
of the qualitative consideration of factors as part of this broader effort.

| agree with the staff that updating the guidance on qualitative factors could be helpful to NRC’s
analysts and improve the clarity, transparency and consistency of the agency's regulatory and
backfit analyses. | support the staff's recommendation to provide guidance to assist analysts in
describing the qualitative evaluation of factors, the significance of each factor, and how such
factors contribute to the integrated decisionmaking process. The focus of the update should be
on capturing best practices for the qualitative consideration of factors. As the staff explained to
the Advisory Committee on Reactor Safeguards (ACRS) subcommittee, the updated guidance
should provide “a toolkit that is available to the analyst for helping them clarify their thinking with
regard to how they considered qualitative factors.”* It should support regulatory analyses that
clearly present the analyst’s qualitative consideration of factors in a transparent way that
decisionmakers, stakeholders, and the public can understand. The updated guidance should
not be overly complicated or prescriptive in such a way that would hinder decisionmaking.

| also support the ACRS recommendation that NRC staff meet with the Committee periodically
during the development of the updated guidance to enable ACRS to review and, if appropriate,
to comment on the staff's work on portions of the guidance related to the consideration of
qualitative factors.®

Lim decals
an

J Date

‘E.0. 13563 (2011)

2 £0. 12866 (1993).

5 Office of Management and Budget Circular A-4 (2003).

4 Advisory Committee in Reactor Safeguards, Regulatory Policies and Practices Subcommittee (Aug. 19,
2014),

5 ACRS September 11, 2014 letter to Chairman Macfarlane re: SECY-14-0087.


NOTATION VOTE

RESPONSE SHEET

TO: Annette Vietti-Cook, Secretary
FROM: Commissioner Baran
SUBJECT: SECY-14-0087: QUALITATIVE CONSIDERATION OF

FACTORS IN THE DEVELOPMENT OF REGULATORY
ANALYSES AND BACKFIT ANALYSES

Approved x Disapproved Abstain

Not Participating

COMMENTS: Below_____ Attached x None____

LL b-—
SIGNATURE

1213 [14
DATE

Entered on “STARS” Yes ft No__


In her essay entitled, “I'm an adult. Stop nudging me.”, commentator and author Margaret
Wente observes the following.

The idea that public officials have a duty to help you do what's in your own interest has
taken off with a vengeance, thanks in no small part to something known as “nudge”
theory. Nudge theory, which was invented by two guys named Richard Thaler and Cass
Sunstein, is on the face of it quite benign. It recognizes that we are flawed, irrational and
occasionally foolish creatures, who, left to our own devices, cannot be relied on to save
for retirement, eat our vegetables or floss. The idea behind nudge theory — also known
as “soft paternalism” — is to design public policies that make the right choices much
easier.

The most obvious problem with nudge theory . . . is that it divides the world into “we” and
“they.” “We” are the informed, the dispassionate, the rational ones who happen to be in
charge. . .. “They,” the poor schlubs, are myopic, lazy, poorly informed and poorly
controlled. They need saving from themselves.

The other problem is that regulators and governments are people too. They have their
own fallibilities.

A third problem . . . is that “soft paternalism” can morph pretty quickly into “soft
authoritarianism,” exemplified by people who are dogmatic, self-righteous and wrong.

That's the problem in a nutshell. It’s a short step from nudging people to terrorizing them
and pushing them around. Many fields — especially public health — are full of people who
think they have a corner on the truth. These people often bemoan the fact that the
public doesn't trust them. But the reason we don’t trust them is quite often that they are
simply imposing their own preference on the rest of us.

So stop nudging me already. | may be flawed, but | am an adult... . And | don't like
bullies.

istine L. Svinicki SO


Commissioner Burns’ Comments on SECY-14-0087
Qualitative Consideration of Factors in the Development of Regulatory Analyses and
Backfit Analyses

| approve the staff's plans for updating guidance regarding the use of qualitative factors in
regulatory analyses and backfit analyses for nuclear materials users, fuel facilities and nuclear
power plants. Staff has assessed that the current regulatory framework for regulatory analyses
and backfit analyses is sound and is consistent with existing NRC guidance and past
Commission direction. Staff's plan to augment existing guidance on the use of qualitative
evaluation factors could improve transparency, reliability and clarity of NRC decisions.

However, | am sensitive to the perspective that too much reliance by the NRC on qualitative
factors or the use of such considerations without some degree of predictability in their
application has led and could lead in the future to the perception that the NRC is acting
arbitrarily. | agree with the staff when it says that “[reliance on a qualitative evaluation of
factors should only be used after efforts to develop pertinent quantitative data have been
unsuccessful.” To ensure that qualitative considerations are used judiciously, the revised
guidance should continue to encourage quantifying costs to the extent possible and use
qualitative factors to inform decision making in limited cases. The NRC’s regulatory decisions
must be informed by sound regulatory and technical bases. As has been the case since the
1977, Commission direction to consider both qualitative and quantitative factors in assessing
the value of propose alternatives, the use of qualitative factors to augment quantitative analyses
is sometimes necessary to manage uncertainly in the data. | do not approve staff developing
guidance that further expands the use of qualitative factors beyond what is currently performed
by NRC staff.

| appreciate the staff engaging stakeholders and the Advisory Committee on Reactor
Safeguards in putting this paper together. The staff should continue this stakeholder
engagement as it updates the guidance documents. At a minimum, this should include a public
opportunity to comment on proposed revisions to guidance, but the staff should also consider
holding public workshops and other mechanisms to ensure meaningful stakeholder input.

nhs

Stéphen G. Burns
2 December 2014



NOTATION VOTE

RESPONSE SHEET
TO: Annette Vietti-Cook, Secretary
FROM: Commissioner Burns
SUBJECT: SECY-14-0087: QUALITATIVE CONSIDERATION OF

FACTORS IN THE DEVELOPMENT OF REGULATORY
ANALYSES AND BACKFIT ANALYSES

Approved _X Disapproved Abstain

Not Participating

COMMENTS: Below____ Attached _X__None___

SIGNATURE

{2 December 2014

DATE

Entered on “STARS” Yes_x__ No___


VOTING SUMMARY - SECY-14-0087

RECORDED VOTES
APRVD DISAPRVD ABSTAIN PARTICIP. COMMENTS DATE
CHRM. BURNS X X 12/12/14
COMR. SVINICKI X X 1/30/15
COMR. OSTENDORFF X X 10/16/14
COMR. BARAN X xX 12/8/14