TN: DT-06-16

To: NRC Management Directives Custodians

Subject: Transmittal of Management Directive 6.8, “LessonsLearned Program”

Purpose: Directive and Handbook 6.8 are being issued to implement

a new agencywide program that will provide reasonable

assurance that major organizational problems identified

by lessons learned will not recur and that the knowledge

gained from the lessons learned is retained and

disseminated in a manner that will maximize its benefit and

usefulness to the staff.

Office of Origin: Office of the Executive Director for Operations

Contact: John G. Lamb, 301-415-1727

Date Approved: August 1, 2006

Volume: 6 Internal Management

Directive: 6.8 Lessons-Learned Program

Availability: Rules and Directives Branch

Office of Administration

Michael T. Lesar, 301-415-7163

Christy Moore, 301-415-7086

U.S. NUCLEAR REGULATORY COMMISSION

DIRECTIVE TRANSMITTAL

OFFICE OF ADMINISTRATION
Lessons-Learned

Program

Directive

6.8
Volume 6, Internal Management

Lessons-Learned Program

Directive 6.8

Approved: August 1, 2006 iii

Contents

Policy ....................................................... 1

Objectives ................................................... 1

Organizational Responsibilities and

Delegations of Authority ................................. 2

Executive Director for Operations (EDO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Office Directors and Regional Administrators ......................... 2

Applicability ................................................. 3

Handbook .................................................... 3

Definitions ................................................... 3

References .................................................. 5
Approved: August 1, 2006 1

U. S. Nuclear Regulatory Commission

Volume: 6 Internal Management OEDO

Lessons-Learned Program

Directive 6.8

Policy

(6.8-01)

It is the policy of the U. S. Nuclear Regulatory Commission to

continuously self-evaluate and improve agency processes,

procedures, and programs. This directive and handbook provide

guidance to NRC personnel to implement an agencywide program

that will ensure that knowledge gained from significant lessons

learned is retained and disseminated in a manner that will

maximize its benefit to the agency.

Objectives

(6.8-02)

• To establish a formal and rigorous process to ensure correction

of significant agency deficiencies. (021)

• To provide reasonable assurance that major organizational

problems identified by lessons learned will not recur. (022)

• To institutionalize the knowledge gained through the corrective

action processes and develop solutions for long-term

organizational retention. (023) 

• To confirm the long-term effectiveness of corrective actions.

(024)
Volume 6, Internal Management

Lessons-Learned Program

Directive 6.8

2 Approved: August 1, 2006

Organizational Responsibilities and

Delegations of Authority

(6.8-03)

Executive Director for Operations (EDO)

(031)

• Ensures that management officials throughout the agency

understand the value of lessons learned and are responsive to

corrective action recommendations. (a)

• Ensures that lessons-learned recommendations are resolved

and that corrective actions are assigned, tracked, and

completed. (b)

• Charters agency lessons-learned task forces. (c)

• Designates the Lessons-Learned Program Manager (LLPM)

and the members of the Lessons-Learned Oversight Board

(LLOB). (d)

• Assigns and schedules lessons-learned activities to the LLPM,

the LLOB, office directors, and regional administrators. (e)

Office Directors and Regional

Administrators

(032)

• Identify potential lessons learned and make recommendations

regarding the designation as a lessons-learned item. (a)

• Assign staff to conduct and participate in lessons-learned

activities assigned by the EDO. (b)

• Nominate a Senior Executive Service level representative and

an alternate to serve on the LLOB when requested by the

EDO. (c)
Volume 6, Internal Management

Lessons-Learned Program

Directive 6.8

Approved: August 1, 2006 3

Organizational Responsibilities and

Delegations of Authority

(6.8-03) (continued)

Office Directors and Regional

Administrators

(032) (continued)

• Conduct reviews of lessons-learned recommendations, as

assigned by the EDO. (d)

• Complete action plans, root cause analyses, corrective actions,

and effectiveness reviews for lessons-learned

recommendations, as assigned by the EDO. (e)

• Coordinate lessons-learned reviews and corrective action

activities with other potentially affected offices. (f)

• Take prompt corrective actions, as warranted, prior to

completion of the lessons-learned process. (g)

Applicability

(6.8-04)

The policy and guidance in this directive and handbook apply to all

NRC employees.

Handbook

(6.8-05)

Handbook 6.8 provides guidance for implementing the NRC’s

Lessons-Learned Program.

Definitions

(6.8-06)

Configuration management - The maintenance and documentation

of internal processes and procedures designed to ensure that changes
Volume 6, Internal Management

Lessons-Learned Program

Directive 6.8

4 Approved: August 1, 2006

Definitions

(6.8-06) (continued)

made to incorporate corrective actions taken for lessons learned

will not be subsequently altered or removed without adequate

review.

Effectiveness review - An assessment conducted to decide if

completed corrective actions have addressed the identified root

cause of the problem and that the actions have been

institutionalized and remain effective.

Knowledge Management - A continuous, disciplined, and timely

process of identifying, collecting, and using information to better

accomplish the job.

Lessons-learned item - A potential item that the LessonsLearned Oversight Board (LLOB) has determined has met the

lessons-learned threshold criteria.

Lessons-learned threshold - A set of criteria established to

ensure that the rigor and formality of the Lessons-Learned

Program is applied only to those items that are worthy of this level

of resource investment and management review. NRC has other

internal processes and programs that provide mechanisms for

assigning corrective actions for items identified by the staff that do

not reach the threshold of this program. A potential lessonslearned item will be designated a lessons-learned item if the LLOB

determines that it meets the lessons-learned threshold criteria

described in Section II(B) of the handbook.

Potential lessons-learned item - A finding, weakness, or

recommendation identified by an input source in Section II(A) of

the handbook where the resulting corrective action(s) should be

considered for oversight by the Lessons-Learned Program.
Volume 6, Internal Management

Lessons-Learned Program

Directive 6.8

Approved: August 1, 2006 5

References

(6.8-07)

“Degradation of the Davis-Besse Nuclear Power Station Reactor

Pressure Vessel Head Lessons Learned Report,” dated

September 30, 2002.

“Effectiveness Review of Lessons Learned Task Force Reports,”

dated August 2, 2004.

NRC Management Directives

4.4, “Management Controls.”

6.1, “Resolution and Followup of Audit Recommendations.”

8.3, “NRC Incident Investigation Program.”

Staff Requirements Memorandum, “Briefing on Status of DavisBesse Lessons Learned Task Force Recommendations,” dated

December 15, 2004.
Lessons-Learned

Program

Handbook

6.8
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Parts I - VII

Approved: August 1, 2006 iii

Contents

Part I

Introduction ................................................... 1

Program Overview (A) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Program Personnel (B) .......................................... 1

Major Process Steps (C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Part II

Identification and Tracking of Lessons Learned ................. 5

Identification of Potential Lessons-Learned Items (A) ................... 5

Lessons-Learned Threshold (B) ................................... 5

Lessons-Learned Oversight Board (C) .............................. 6

Assignment of Lessons-Learned Items for Corrective Action (D) .......... 6

Timeliness Goals (E) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Part III

Corrective Action Plans ........................................ 8

Corrective Action Plan Development (A) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Corrective Action Plan Acceptance Criteria (B) . . . . . . . . . . . . . . . . . . . . . . . . 9

Corrective Action Plan Review and Approval (C) ...................... 9

Corrective Action Plan Implementation (D) . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Lessons-Learned Item Closeout Package (E) . . . . . . . . . . . . . . . . . . . . . . . . . 10

Corrective Action Plan Completion Acceptance Review (F) . . . . . . . . . . . . . . 11

Lessons-Learned Item Administrative Closeout (G) . . . . . . . . . . . . . . . . . . . . 12

Part IV

Effectiveness Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Conduct of Effectiveness Reviews (A) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Effectiveness Review Report (B) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Part V

Program Oversight and Administration . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Lessons-Learned Program Manager (A) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Lessons-Learned Oversight Board (B) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Parts I - VII

iv Approved: August 1, 2006

Contents (continued)

Part VI

Program Performance Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Performance Reviews (A) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Monthly Status Reports (1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Annual Program Reviews (2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Performance Measures (B) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Part VII

Background and Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Background (A) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Bases (B) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Attributes (C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Exhibits

1 Lessons-Learned Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2 Effectiveness Review Report Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3 Examples of Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4 Desired Attributes of a Lessons-Learned Program . . . . . . . . . . . . . . . . . . . . . 28
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part I

Approved: August 1, 2006 1

Part I

Introduction 

Program Overview (A)

The Lessons-Learned Program (LLP) described in this handbook

is a set of processes, procedures, and oversight that is designed

to collectively ensure that significant agency deficiencies are

identified and corrected in such a way that they do not recur. This

goal is accomplished by using a rigorous process to identify

significant lessons learned, developing detailed corrective action

plans, subjecting those plans to formal review and approval, and

ensuring that the plans have been effective and have not had any

unintended consequences. Each corrective action plan must

include an explanation of what actions will be taken to

institutionalize the knowledge gained through the identification of

a lesson learned to help prevent its recurrence. Because

deficiencies entered into this program are expected to require

significant resources to correct, and because the agency already

has many corrective action mechanisms that function at the office

level and below, a high threshold has been established for entering

deficiencies into the LLP. (1)

The LLP uses two existing agency systems to help fulfill its

objectives: the EDO’s Action Tracking System (EDATS) and the

Agencywide Documents Access and Management System

(ADAMS). EDATS is used to initiate, assign, and track actions.

ADAMS is used to store the products produced as a result of the

taskings. Together the two systems are able to provide a

retrievable record of how each lessons-learned recommendation

entered into the LLP is addressed. (2)

Program Personnel (B)

The EDO will select a Lessons-Learned Program Manager (LLPM)

to administer and implement the LLP. The LLPM will be a member

of the EDO’s staff and will be responsible for the day-to-day
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part I

2 Approved: August 1, 2006

Program Personnel (B) (continued)

implementation of the LLP. A detailed list of the LLPM’s duties and

responsibilities is contained in Part V of this handbook. (1)

The EDO will also designate a Lessons-Learned Oversight Board

(LLOB), composed of senior managers nominated by office

directors and regional administrators, to provide oversight to the

LLP and to provide an independent review and approval of

proposed corrective action plans and their eventual closeout.

Additional duties and responsibilities are contained in Part V of this

handbook. (2)

The LLOB will operate under procedures approved by the EDO,

subject to the specific requirements discussed in this handbook.

(3)

Major Process Steps (C)

The LLP consists of the major steps described below. Figure 1 is

a simplified graphical depiction of these steps: (1) 

• Potential lessons-learned items are identified by internal or

external sources. (a)

• Potential lessons learned are entered into EDATS for tasking

and tracking, and the associated reference documents are

entered into ADAMS. (b)

• The LLOB will make a determination as to which of these

recommendations constitute a lesson learned by applying the

program’s threshold criteria. (c)

• Items determined to be lessons learned will be assigned to a

lead office to create a corrective action plan. Items that did not

meet the threshold will continue to be tasked through existing

processes. (d)

• Upon completion of the corrective action plan, the lead office

will present the plan to the LLOB for review and approval. (e)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part I

Approved: August 1, 2006 3

Major Process Steps (C) (continued)

• Upon completing the implementation of the approved

corrective action plan, the lead office will present a closeout

package to the LLOB for approval. (f)

• Once closeout of the corrective action plan has been approved

by the LLOB, the LLPM will ensure that the effectiveness

review is scheduled by entering the task into EDATS in

accordance with the approved corrective action plan. (g)

• The LLPM will ensure that all related reference documents

have been properly entered into ADAMS so that a retrievable

record exists of the recommended action, the corrective action

plan, and the plan as implemented. Effectiveness reviews are

also entered into ADAMS at the time of their completion. (h)

Additional details regarding each process step are explained in this

handbook. (2)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part I

4 Approved: August 1, 2006

Major Process Steps (C) (continued)

Figure 1
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part II

Approved: August 1, 2006 5

Part II

Identification and Tracking

of Lessons Learned

Identification of Potential

Lessons-Learned Items (A)

Recommendations resulting from the following sources will be

considered potential lessons-learned items: (1)

• Accident Review Groups (a)

• Incident Investigation Teams (IITs) (b)

• Lessons-Learned Task Forces chartered by the EDO (c)

• Office of the Inspector General (OIG) reports (d)

• Government Accountability Office (GAO) reports (e)

The EDO may designate recommendations from other internal and

external sources as potential lessons-learned items, as

appropriate. (2)

Lessons-Learned Threshold (B)

A potential lessons-learned item must meet all of the following

criteria to be designated a lessons-learned item:

• The item has significant organizational, safety, security,

emergency preparedness, or generic implications; (a)

• A need exists to institutionalize corrective action for this item

because the failure to do so would reasonably be expected to

challenge the ability of the agency to meet any of the strategic

outcomes designated in the Strategic Plan,
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part II

6 Approved: August 1, 2006

Lessons-Learned Threshold (B)

(continued)

or 

the corrective action would substantially improve the safety or

security of NRC employees; (b)

• A root cause exists – or can be identified; (c)

• The apparent resolution is actionable. (d)

Lessons-Learned Oversight Board (C)

The Lessons-Learned Oversight Board (LLOB) is responsible for

ensuring that the criteria listed above are appropriately applied to

all potential lessons-learned items. (1)

The EDO will create and maintain procedures that govern the

makeup, selection, and conduct of the LLOB. The LLOB

should include at least one Senior Executive Service (SES)

representative from each major program office. The EDO may also

designate representatives from other offices as members of the

LLOB. The EDO is provided the flexibility to modify the LLOB

charter and procedures to ensure its continued effectiveness and

efficiency as experience is gained with the Lessons-Learned

Program. (2)

The LLOB member from the lead office will not participate in the

review and approval of corrective action plans, and completed

action plans, from their office. (3)

Assignment of Lessons-Learned

Items for Corrective Action (D)

The EDO will assign a lead office for each lessons-learned item on

the basis of a recommendation from the LLOB.
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part II

Approved: August 1, 2006 7

Timeliness Goals (E)

The EDO will establish timeliness goals to – 

• Ensure that each potential lessons-learned item is promptly

considered to determine if it should be processed as a lessonslearned item. (a)

• Ensure that a lead office is promptly identified and tasked for

each lessons-learned item. (b)

• Ensure that the lead office completes review and develops

corrective action plans within an appropriate time frame. (c)

• Ensure that corrective actions are promptly implemented once

approved. (d)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part III

8 Approved: August 1, 2006

Part III

Corrective Action Plans

Corrective Action Plan Development (A)

The lead office assigned the responsibility for the lessons-learned

item will develop a corrective action plan within the schedule

established by the EDO. (1)

The corrective action plan should, at a minimum, include the

following: (2)

• A summary statement describing the lessons-learned item. (a)

• The results of the root cause analysis. (The root cause analysis

will be completed by an individual trained in a root cause

analysis methodology and a subject matter expert in the

area(s). The root cause analysis should be conducted with the

rigor commensurate with the complexity of the issue.) (b)

• Proposed corrective actions to address the root cause(s). (c)

• Any immediate corrective actions, or compensatory measures,

taken or in progress. (d)

• The results of any extent of condition review and corrective

actions taken or planned to prevent recurrence, with

milestones for completion. This discussion should also state

whether the issue is recurring or repetitive. (e)

• The configuration management plan. (f)

• An estimate of the resources required to implement the plan

and the expected impact on any other work if the plan is

implemented as proposed. (g)

• A communication plan. (h)

• A preliminary effectiveness review plan and schedule. (i)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part III

Approved: August 1, 2006 9

Corrective Action Plan Acceptance Criteria (B)

The Lessons-Learned Oversight Board (LLOB) will use

acceptance criteria to ensure that the corrective action plan is

complete and addresses the lessons-learned item. At a minimum,

the following will be included in the criteria:

• The root cause analysis and associated conclusions are

scrutable and supported by the facts. (a)

• The proposed corrective actions address the root cause(s) and

appropriately consider the results of the extent of condition

review. (b)

• The configuration management plan will ensure that the

proposed corrective actions will be sufficiently institutionalized.

These could include approaches such as bases documents,

generic communications, training programs, and qualification

programs. (c)

• The basis for the resource estimate is reasonable. The

schedule is resource loaded using established agency

processes. (d)

• The communication plan will ensure a broad sharing of the

knowledge gained from the lessons learned with stakeholders

and the public. (e)

• The preliminary effectiveness review plan identifies specific

success criteria, the time when the review will be conducted,

the need for an independent review, and a general description

of the methodology that will be used. (f)

Corrective Action Plan Review and

Approval (C)

The LLOB will review and approve the corrective action plan. The

LLOB may require the lead office to modify the proposed

corrective action plan so that it meets the corrective action plan

acceptance criteria. In such cases, the LLOB will give the lead
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part III

10 Approved: August 1, 2006

Corrective Action Plan Review and

Approval (C) (continued)

office a schedule for submitting the modified corrective action plan.

(1)

If the LLOB and the lead office are unable to agree on an

appropriate action plan, the EDO will be consulted. (2)

The LLOB will determine which completed corrective action plans

require a corrective action plan completion acceptance review by

the LLOB. If not designated as requiring a review by the LLOB, the

acceptance reviews will be completed by the Lessons-Learned

Program Manager (LLPM). (3)

Corrective Action Plan Implementation (D)

The lead office will manage implementation of the approved

corrective action plan according to the schedule identified in the

plan. The lead office will inform the LLPM of any schedule

changes and resource issues. (1)

The OEDO Assistant for Operations (AO) will review any proposed

changes to the corrective action plans and approve minor

adjustments to the plan. The AO will forward major changes to the

LLOB for approval. (2)

Lessons-Learned Item Closeout

Package (E)

A memorandum from the lead office to the LLOB is required for the

closeout of the lessons-learned item. This memorandum should

include— (1)

• A summary of the lessons learned and corrective actions,

using the template shown in Exhibit 1. (a)

• A listing of the agency procedures, processes, and documents

changed or created and the associated Agencywide Documents
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part III

Approved: August 1, 2006 11

Lessons-Learned Item Closeout

Package (E) (continued)

Access and Management System (ADAMS) Accession

Numbers. (b)

• Documentation that demonstrates implementation of the

configuration management plan. (c)

• Documentation that demonstrates implementation of the

communication plan. (d)

• Final effectiveness review plan. (e)

• Final resource expenditure data. (f)

• Recommendations or suggestions to improve the LessonsLearned Program (LLP). (g)

Access to lessons-learned information should be unlimited for

NRC internal user communities and limited for external

communities by sensitive information requirements. (2)

Corrective Action Plan Completion

Acceptance Review (F)

The LLOB will determine if the corrective action closeout package

for the lessons-learned item is acceptable. The LLOB must

consider whether—

• The corrective actions have been completed and

institutionalized. (a)

• The corrective actions have addressed the root cause of the

problem. (b)

• The extent of the condition was considered and addressed, as

appropriate. (c)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part III

12 Approved: August 1, 2006

Corrective Action Plan Completion

Acceptance Review (F) (continued)

• The communication plan was completed. (d)

• The effectiveness review is planned and scheduled. (e)

Lessons-Learned Item Administrative

Closeout (G)

The LLPM will ensure that before the closeout of the lessonslearned item in the EDO action tracking system that— (1)

• The lead office has entered into ADAMS the documentation

necessary to close the lessons-learned item. (a)

• The effectiveness review has been scheduled. The LLPM will

assign an EDO Action Tracking Item to track the completion of

the effectiveness review. (b)

• Resource expenditure data have been documented. (c)

The LLPM will address any recommendations for improving the

LLP. (2)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part IV

Approved: August 1, 2006 13

Part IV

Effectiveness Reviews

Conduct of Effectiveness Reviews (A)

Following implementation of the corrective actions for the lessonslearned items, the lead office will review the effectiveness of the

corrective actions to confirm that the completed actions have

addressed the root cause(s) of the problem. These reviews will

normally include the following: (1)

• Assessment of whether the problem has recurred. (a)

• Determination of whether the completed corrective actions are

in place and will remain in place (i.e., institutionalized). (b)

• Assessment of whether the corrective actions have been

effective at addressing the root cause of the original problem.

If the corrective actions have not been effective, a

determination of what further action is needed to ensure the

problem does not recur. (c)

• Assessment of whether the corrective actions have introduced

a new problem. (d)

The lead office should consider the following elements when

developing an effectiveness review plan: (2)

• Independence of the reviewer. (a)

• Knowledge of the reviewer. (b)

• Method of review (e.g., audit, survey, self-assessment,

metrics). (c)

• Success criteria. (d)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part IV

14 Approved: August 1, 2006

Conduct of Effectiveness Reviews (A)

(continued)

• Time elapsed between completion of corrective actions and the

review (i.e., sufficient experience has been gained following

completion of the corrective actions to evaluate the

effectiveness of the completed actions). (e)

Effectiveness Review Report (B)

The lead office reviewer will document the effectiveness review in

a report submitted to the Lessons-Learned Program Manager. If

the reviewer determines that the corrective actions are ineffective,

the report should include the actions the lead office will implement

to correct the identified issues. The report should also state

whether another effectiveness review should be conducted in the

future. An Effectiveness Review Report Template is provided in

Exhibit 2.
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part V

Approved: August 1, 2006 15

Part V

Program Oversight and Administration

Lessons-Learned Program

Manager (A)

The EDO will assign a Lessons-Learned Program Manager

(LLPM) to administer and implement the Lessons-Learned

Program (LLP). The duties of the LLPM will include the following:

• Forwarding to the Lessons-Learned Oversight Board (LLOB),

in a timely manner, potential lessons-learned items received

from the input sources. (a)

• Maintaining a broad awareness of NRC activities in order to

make sound recommendations to the EDO for including

appropriate potential lessons-learned items. (b)

• Ensuring that lessons-learned items are assigned to a lead

office for action according to program requirements. (c)

• Supporting the review and approval activities of the LLOB. (d)

• Tracking and coordinating the status and documentation of

lessons-learned items. (e)

• Performing or coordinating the LLP activities according to NRC

policies and the requirements of this management directive. (f)

• Maintaining LLP documents and implementing procedures. (g)

• Performing preliminary reviews of proposed corrective action

plans and forwarding these reviews to the LLOB for its

consideration. (h)

• Performing acceptance reviews of completed corrective action

plans. (i)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part V

16 Approved: August 1, 2006

Lessons-Learned Program

Manager (A) (continued)

• Independently assessing the results of effectiveness reviews

for the LLOB. (j)

• Preparing monthly status reports for the EDO on the status of

open lessons learned, including any approved changes to the

corrective action plans. (k)

• Conducting an annual review of the LLP to evaluate the

effectiveness of the program, disposition of recommendations

for improving the program, and recommend process changes.

(l)

• Drafting Lessons-Learned Task Force charters, as assigned by

the EDO. (m)

• Managing effectiveness reviews and information-gathering

activities for legacy lessons-learned items conducted to

support the program. (n)

• Identifying future needs and managing additional systems

development that may be needed to ensure that the program

remains efficient and effective. (o)

• Maintaining the bases (i.e., supporting) documents for the LLP.

(p)

Lessons-Learned Oversight Board (B)

An LLOB will be assigned by the EDO to provide program

oversight and approve corrective action plans and closeout. The

LLOB responsibilities will include the following:

• Reviewing potential lessons-learned items. (a)

• Defining the scope of lessons-learned recommendations. (b)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part V

Approved: August 1, 2006 17

Lessons-Learned Oversight Board (B)

(continued)

• Determining whether potential lessons-learned items meet the

program threshold. (c)

• Reviewing and approving corrective action plans. (d)

• Reviewing changes to approved corrective action plans. (e)

• Approving the closeout of completed corrective action plans. (f)

• Providing recommendations to the EDO for improvements in

the LLP. (g)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part VI

18 Approved: August 1, 2006

Part VI

Program Performance Monitoring

Performance Reviews (A)

Monthly Status Reports (1)

The Lessons-Learned Program Manager (LLPM) will prepare a

monthly report on the status of open lessons-learned items for the

EDO. The report will include any approved changes to the

corrective action plans and identify any overdue corrective actions.

Annual Program Reviews (2)

The LLPM will conduct an annual review of the Lessons-Learned

Program to evaluate the implementation of the program, identify

program successes, and recommend process improvements. (a)

The annual program review report will be provided to the EDO and

the Commission. (b)

Performance Measures (B)

The LLPM will monitor the performance of the program using

performance measures. (1)

Examples of useful performance measures that should be

considered in the monitoring of the program are included in

Exhibit 3. (2)

The EDO will determine the appropriate program performance

metrics that should be included in office-level operating plans. (3)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part VII

Approved: August 1, 2006 19

Part VII

Background and Bases

Background (A)

NRC has conducted many lessons-learned reviews to assess its

internal programs and processes because of significant plant

events or organizational failures. Consistent with this practice, the

EDO directed the formation of an NRC task force in response to

the issues associated with the extensive degradation of the reactor

pressure vessel head at the Davis-Besse Nuclear Power Station

(DBNPS). The degraded reactor pressure vessel head was

identified by the licensee for DBNPS on March 5, 2002. On

September 30, 2002, the Davis-Besse Lessons-Learned Task

Force (DBLLTF) completed its evaluation of the NRC’s regulatory

processes related to the Davis-Besse event. Appendix F of the

DBLLTF report noted that issues in several previous lessonslearned reports were similar to issues identified in the DBLLTF

review. The task force recommended that NRC conduct an

effectiveness review of the actions taken in response to past

lessons-learned reviews. (1)

The Office of Nuclear Reactor Regulation (NRR) established an

Effectiveness Review Lessons-Learned Task Force (ERLLTF) to

determine whether recommendations from previous lessons

learned had been adequately implemented. The ERLLTF was also

tasked with identifying root and contributing causes for any

corrective actions that had not been effectively implemented, and

to make recommendations to prevent recurrence. In its report of

August 2, 2004, the ERLLTF found that some corrective action

issues that occurred prior to the Davis-Besse event had not been

effective and identified four root causes: (1) the lack of a corrective

action program, (2) the lack of effectiveness reviews, (3) the lack

of a centralized tracking system, and (4) weaknesses in the line

organizations’ closeout practices. (2)

Some of the specific problems that led to recurrence were (1)

subsequent reversal of corrective actions that had initially been
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part VII

20 Approved: August 1, 2006

Background (A) (continued)

effective, (2) corrective actions that were partially addressed or

completed, (3) corrective actions that failed to address the

weakness, (4) recommendations that did not result in measurable

action, and (5) corrective actions that were closed out before the

work was completed. On the basis of these findings, the ERLLTF

recommended that the staff develop an agencywide corrective

action program that focuses on the corrective actions from major

lessons-learned reports and high-priority commitments. The report

also suggested that corrective actions from recommendations

identified by the Office of the Inspector General, the Advisory

Committee on Reactor Safeguards, and the Government

Accountability Office be considered for the program. (3)

At a briefing for the Commission on December 8, 2004, on the

status of the DBLLTF recommendations, the staff described a

proposal to develop and implement a program for institutionalizing

agency lessons learned. In Staff Requirements Memorandum

M041208B dated December 15, 2004, the Commission endorsed

the proposal to establish the program. (4)

On the basis of the ERLLTF findings, the EDO chartered a team

on January 24, 2005, to (1) develop a process, program, or system

that will provide reasonable assurance that for the lessons learned

from major organizational failures, the problems noted above will

not recur and (2) ensure by whatever means necessary that the

knowledge gained from future lessons learned is retained and

disseminated in a manner that will maximize its benefit and

usefulness to the agency over time. (5)

Bases (B)

The use of lessons-learned information is a principal component

of an organizational culture committed to continuous improvement.

The methods to instill lessons learned as a part of an

organizational culture can vary. The nature of the work and

the complexity of the organization are prime determinants of

cultural and infrastructure support for lessons learned. Cultural
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part VII

Approved: August 1, 2006 21

Bases (B) (continued)

methods often include setting expectations, providing support and

incentives, monitoring and feedback, and continuous

improvement. Infrastructure mechanisms typically include clear

definition of resources, processes, and procedures by which

personnel are supported to identify, share, and use lessons

learned. The infrastructure mechanisms are often called LessonsLearned Programs (LLPs). (1) 

An ideal LLP includes four basic processes. The first is a rigorous

administrative process that includes identification, documentation,

assigning and tracking, and validation of a lesson learned. The

second is a formal corrective action process that includes

evaluation of root causes, identification of actions that will be taken

because of the lessons learned, allocating resources to complete

the actions, and followup to ensure that appropriate actions were

taken. The third is a configuration management process that

provides assurance that the changes made to incorporate

corrective actions taken for lessons learned will not be

subsequently altered or removed without adequate review. The

fourth is a Knowledge Management process that disseminates the

lessons learned to appropriate personnel and ensures that a

library of historical lessons-learned information is maintained. In

addition to these four basic elements, LLPs should include a

process for evaluating and measuring the effectiveness of the LLP

such that the costs and benefits of lessons learned can be

assessed on periodic basis. (2)

For the Knowledge Management element to be effective, all

individuals in NRC should have ready access to lessons-learned

information. Access mechanisms should include a variety of

communication media. The lessons-learned access media should

be responsive and customized to match user needs (types of

information, level of detail, sorting tools, and presentation and

display options). Both push and pull information technology should

be used. Simple search mechanisms should be available. Access

to lessons-learned information should be unlimited for NRC

internal user communities and limited for external communities by

sensitive information requirements. (3)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part VII

22 Approved: August 1, 2006

Bases (B) (continued)

It is expected that the development, communication, and use of

lessons-learned information should be part of everyone’s job. The

EDO will establish and maintain the lessons-learned infrastructure

that supports the LLP. It is expected that the program offices will

work with the EDO to ensure corrective actions taken to address

lessons learned are institutionalized, and that they will seek

continuous improvement of the LLP. (4)

Attributes (C)

In development of the LLP, the staff collected insights and best

practices from other LLPs and corrective action programs. The

staff conducted benchmarking visits with other Government

agencies, external organizations, and licensees. The staff used the

insights collected in the benchmarking activities, along with the

insights gained by the staff from previous lessons-learned efforts,

inspection of licensee corrective action programs, and experiences

with the internal corrective action process, to develop a list of

desired attributes for the program. These attributes, and the

recommendations provided by the staff in the ERLLTF previously

discussed, were used to develop and shape the program

described in this handbook. A complete listing of the desired

attributes developed by the staff is included in Exhibit 4. (1)

One of the desired attributes noted during the benchmarking

activities was that the program should contain performance

measures and metrics. The staff evaluated the measures and

metrics used by other organizations and developed a list of

example metrics that could be used to monitor the program. These

metrics are listed in Exhibit 3. (2)

The LLP will apply a high threshold so that only the most important

issues will be entered into the program. The program will consider

recommendations from Accident Review Groups, Incident

Investigation Teams, EDO-chartered Lessons-Learned Task

Forces, Office of the Inspector General reports, and Government

Accountability Office reports as potential lessons-learned items. In

addition, the EDO has discretion to enter other items into the
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Part VII

Approved: August 1, 2006 23

Attributes (C) (continued)

program. The decision on applying a high threshold was based on

a desire to maximize the impact of the agency resources

expended, to ensure the specific problems identified by the

DBLLTF and the ERLLTF are addressed, and to keep senior

management focused on the most important lessons learned. (3)
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Exhibits

24 Approved: August 1, 2006

Exhibit 1

Lessons-Learned Template

Title: Assign a short title that describes the event or

condition of interest.

Summary: Give a brief one-paragraph summary of the

event or condition (for the summary page on

the Web).

Date: List the date of the event or report that

initiated the agency response.

Description/Discussion: Include a general discussion of the event, a

summary of the major issues and lessons,

and any discussion needed to understand the

issues and help put them in context. The

executive summary of a report could be used

for this field. 

Lessons Learned/Findings: List each lesson learned, or finding, from the

event or condition and include a general

discussion of any amplifying details or

needed clarification.

Root Causes/Contributors: If any root causes or contributing causes

were identified as part of the review, include

those items here. If a root cause was not

established (e.g., legacy items), put “Not

Evaluated.” Indicate if this issue is a recurring

historical lesson learned.

Recommendations/Corrective Actions: Include a tabular listing of each

recommendation and the corrective actions

taken in response to those recommendations.

If the recommendation was not implemented,

or no action was taken, describe the

rationale for not taking
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Exhibits

Approved: August 1, 2006 25

Exhibit 1 (continued)

the action. Also include references that

document completion of the corrective

actions.

Status/Effectiveness Review: Provide a summary of the results of the

effectiveness review.

Primary References: Provide a listing of each reference directly

related to the analyses or investigation of the

event, lessons learned, the recommendations,

and followup correspondence on the corrective

actions.

Related References: Provide a listing of any references that may

be related to the topics at hand or that

provide a basis for the corrective action

decisions.

Keywords: List keywords that will assist the staff in

locating lessons learned about specific

subjects. Limit the list to the primary

keywords.

Programs/Processes: List any agency programs and processes,

and the governing documents and internal

procedures, that include corrective actions

taken in response to this event or condition.
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Exhibits

26 Approved: August 1, 2006

Exhibit 2

Effectiveness Review Report Template

Introduction/Background Discuss the purpose of the effectiveness

review and any background information

needed to understand the need for the

review.

Scope of Effectiveness Review: Discuss the specific corrective actions or

recommendations that are being evaluated.

Methodology/Success Criteria: Discuss the general approach used to

conduct the effectiveness review, the

rationale for using the approach, and the

success criteria established for the review.

Results of Effectiveness Review: Provide the results of the effectiveness

review, including any relevant data or

document references.

Effectiveness Review Conclusion: Provide the overall conclusions of the review.

Recommendations: Provide recommendations for closeout of the

action, followup corrective actions, or further

effectiveness reviews.
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Exhibits

Approved: August 1, 2006 27

Exhibit 3

Examples of Performance Measures

Timeliness of corrective action completion (within original schedule) (process

effectiveness/efficiency)

Number of references in documents derived from lessons learned (value-added

success)

Success rate/failure rate of effectiveness reviews (effectiveness)

Recurrence of issues for which previous lessons-learned corrective actions have been

completed (effectiveness)

Resource expenditures (efficiency)

Number of visits to the Lessons-Learned Web Page (outreach success)

Number of searches on the Lessons-Learned Web Page (outreach/value-added

success)

Timeliness of process steps (efficiency)

• Time to designate as a lessons-learned item

• Time to submit action plan to the Lessons-Learned Oversight Board (LLOB)

• Timeliness of original action plan schedules

Quality of action plans (quality)

• Percentage of action plans requiring revision/rework

• Results of effectiveness reviews

• Accuracy of resource estimates
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Exhibits

28 Approved: August 1, 2006

Exhibit 4

Desired Attributes of a Lessons-Learned Program

Management Support

A program that—

• Possesses management commitment, support, and involvement 

• Contains sufficient resources to implement and maintain the program

• Contains sufficient resources to implement corrective actions

• Clearly defines its ownership

• Is accountable for correction of issues

Formal and Rigorous Program

A program that—

• Is formally defined in a management directive

• Addresses high-level, multi-office, or agency-level findings

• Ensures timely resolution of issues

• Incorporates an agency-level tracking system (centralized)

• Tracks issues to full completion

• Possesses clear thresholds and criteria for what actions will be tracked

• Uses management review boards to approve entry of new findings and proposed

corrective actions

• Uses a priority and classification scheme and links planned actions to the budget

• Uses root cause evaluations

• Completely and accurately identifies the problem

• Contains corrective actions that address the root cause of the problem and

prevent recurrence

• Considers the extent of condition/program implications

• Includes effectiveness reviews of corrective actions conducted, where

appropriate

• Includes formal closeout review and approval of actions by a management

review board

• Implements processes that ensure corrective actions are not removed without

review

• Ensures the quality of information entered into the system
Volume 6, Internal Management

Lessons-Learned Program

Handbook 6.8 Exhibits

Approved: August 1, 2006 29

Exhibit 4 (continued)

Program Assessment and Review

A program that—

• Contains performance measures and metrics

• Periodically assesses program effectiveness

• Periodically reviews system information for relevancy

Integration With Other Systems and Processes

A program that—

• Minimizes duplication of existing tracking systems (and maximizes use of current

systems)

• Includes a process to make changes and revisit previous corrective actions

• Is integrated with other agency Knowledge Management initiatives

• Is integrated with other existing agency processes and programs

Communication to Staff and Stakeholders

A program that—

• Clearly communicates to the staff and appropriate stakeholders the agency

actions taken

 • Places retrievable history and status in a centralized system available to all staff

• Ensures that a basis for process or procedural changes is maintained

• Conducts initial training of staff for program rollout and ensures continuing online

training

• Uses storytelling in a lessons-learned information system