# Presentation: 130128

## Operating Experience:Feedback from the Complex

**Teresa Cochran**
**Oak Ridge National Laboratory**
**cochranti@ornl.gov**

**Integrated Safety Management Champions Workshop****Upton, New York****November 29, 2007**

## Implementation of DOE Order 210.2 DOE Corporate Operating Experience Program

- Gap Analysis
- Implementation Plan
- Procedural Revisions
- Implementation with Management Systems (Work Control, Integrated Safety Management, Training, etc.)
- Senior Management involvement

***We have been busy...***

**Notes:**
What sites activities occurred to implement new DOE O 210.2

## Sites’ Lessons Learned ProgramImplementation Activities

- Identify departmental/division LL coordinators, subject matter experts
- Assign LL responsibilities
- Promote sharing operating experiences

## Sites’ Lessons Learned ProgramImplementation Activities (cont.)

- Streamlining LL creation, review, and dissemination processes
- Tracking/sharing external LL
- Trending data streams for lessons
- Expanding LL product output and target audience
- LL database development
- LL web portal enhancements

## Sites’ Lessons Learned ProgramImplementation Activities (cont.)

- Gathering data for metrics for LLP
- Aggressive approach to obtaining feedback
- Quantitative vs. Qualitative metrics

**Notes:**
DOE says to “Establish metrics to measure program performance and evaluate the effectiveness of actions implemented from LL”.
Feedback obtained from surveys and questionaires mostly, but have to include all correspondences.
Quantitative metrics are the “Bean-counting”. Qualitative metrics is feedback.

## Assessments:What We Have Learned

*“**Site has established a robust lessons learned program.   A ** proactive, engaged site coordinator and over 40 designated ** Department level coordinators, required to subscribe to the DOE ** listserver, employ a “Team” approach for sharing lessons ** learned.”*
*“**An accessible, user-friendly website with lessons from 1999 ** forward, searchable by DOE functional categories, priority ** descriptors, ISM core function, hazard type, date, or keyword/ ** text is available to potential users.” *
*“**Site has established a structured, documented feedback ** process on actions taken, which is actively monitored and ** maintained by the site coordinator.” *

*Best Practices:*

## Assessments:What We Have Learned

*“**Quarterly reports are compiled and forwarded to management ** on lessons learned status, actions taken, ongoing initiatives, ** etc.” *
*“**The Lessons Learned process is well-designed to analyze and ** categorize lessons learned and makes good use of a wide range ** of qualified subject matter experts to determine the appropriate  ** target distribution groups.”*
*“**The interest and use of Lessons Learned has continued to ** increase at the site.”*
*“**Lessons Learned evaluation and distribution is very well ** managed and effective.”*

*Best Practices (cont.):*

## Assessments:What We Have Learned

*“**The procedure does not reflect the requirements of DOE Order  210.2., DOE Corporate Operating Experience Program.  The current  description of the lessons learned process in the draft policy is insufficient to describe: 1) the roles, responsibilities, and authorities for the lessons learned program and 2) does not address the responsibility to perform oversight of contractor lessons learned programs.”*
* *
*“**The LL procedure does not define the process for LL validation.”*
*“**Site planning procedures do not clearly specify that LL searches and reviews are a necessary part of the work planning process.”*

- Opportunities for Improvement: (Procedures)

## Assessments:What We Have Learned

*“**No formal process or documentation for review and application ** of lessons learned in work planning processes.”*
*“**Improvements are needed in the communication of operating ** experience into Lessons Learned.” *
*“...**work planning procedures do not clearly specify that LL ** searches and reviews are a necessary part of the work planning ** process.”*

- Opportunities for Improvement: (Work Control/                                                      Work Planning)

## Assessments:What We Have Learned

*“**There is no separate search of or subscription to product recall ** information such as the Consumer Product Safety Commission.”*
*“**The process and documentation of the review and response to ** externally generated lessons were insufficient.” *
*“**The site procedure does not require that externally generated lessons be formally screened by program owners or subject matter experts to identify applicability and needed actions at the institutional/program level.  The site coordinator performs a high level screening for applicability, but relies upon the department level coordinators to screen external lessons and take appropriate actions.  There is no formal documentation or feedback from department coordinators demonstrating that these lessons are being consistently and effectively screened and applied.   Annual program self-assessments have not evaluated performance of this activity.” *

- Opportunities for Improvement: (External LL)

## Assessments:What We Have Learned

*“**The current method of gathering feedback on the use of LLs is not fully effective and is not meeting its intended function. Current metrics are not adequately effective in gauging program performance.”*
*“**Improvements in lessons learned need additional maturation in the area of trending and further development in the collection of feedback.”*
*“**Current metrics are not fully effective in gauging program ** performance.”*

- Opportunities for Improvement: (Metrics/Feedback)

**Notes:**
Discussion for bullet # 3:  Assessor stated that while there was ample user feedback being collected and analyzed, feedback is not adequately captured on specific cases where LL sharing was beneficial to the user, for example an application of the LL resulted in process changes that made operations safer. Gathering of such info in addition to the general use statistics would provide a more comprehensive view of the effectiveness of the LLP.

## Lessons Learned?

- Obtaining metrics to meet theDOE O 210.2 requirements is easier than *establishing metrics to measure program performance and evaluate the effectiveness of actions implemented from lessons learned.*
- “Other than metrics based solely on unit count which provide no added value, the development of valid metrics in this area has eluded the DOE/NNSA complex in different venues for the last five years and continues to do so today.  The next LL/OPEX metric, if valid, will be the first.”

## Lessons Learned:  It’s All About Change

**Brookhaven National Laboratory**
**Quality Management Office **
**E. Anthony Sierra**
**esierra@bnl.gov**
**(631) 344-4080**

**Integrated Safety Management Champions Workshop****Upton, New York****November 29, 2007**

## Point to Ponder

**“****The invention of a mechanism for the systematic publication of fragments of scientific work may well have been the key event in the history of modern science.”**
**	****	****	****	****- John Ziman**

## Types of Change

**“****First and Second-Order Change**
- Incremental Change (fine tunes the system)
- Deep Change (dramatic shift in direction)
**Single and Double-loop Learning**
- A Current set of strategies
- Expands organizations view of the world while adding new strategies to repertoire
**Difficulty of Second-Order Change**
- Solutions to most recurring modern-day problems require a second-order perspective.

## Second-Order Change & Leadership 

**Knowledge**
**Optimizer**
**Intellectual Stimulation**
**Change Agent**
**Monitoring/Evaluating**
**Flexibility**
**Ideals/Beliefs**

## Innovation May Negatively Affect:

**Culture**
**Communication**
**Order**
**Input”**

**	****	****	**** ****	**** **** “****School Leadership that Works: From Research to Results”                                     R. Marzano, T. Walters, B. McNulty, 2005**

## The Law of Initiative Fatigue

**The law states:**  When resources of time, money, and emotional energy are held constant while the number of old, continuing, and new initiatives rises, a _dramatic decline_ in organizational _effectiveness_** **is inevitable.

**- “The Learning Leader”           ****	****	****	****                                     Dr. Douglas Reeves, 2006**

## The only response to the Law of Initiative Fatigue is a garden party.

Leaders must be ever vigilant for persistent weeds with deep roots in the Laboratory garden.

			                              - Adapted from: “The Learning Leader”           			                     Dr. Douglas Reeves, 2006

**Leaders must be ever vigilant for persistent weeds with deep roots in the Laboratory garden.**
**	****	****	****                              ****- Adapted from: “The Learning Leader”           ****	****	****	****                     Dr. Douglas Reeves, 2006**

**Initiative Fatigue: The Fix**

## Personal Change

**“****Loss of Control**
**Loss of Predictability**
**The Dentist**
**The Candy Machine”**

**                   ****- Mark P. Steinberg**
**                   ****SteinbergPresentations.com**

## Slide 21

## Slide 22

## Suggestions For Positive Change

**“****Ask an unscripted question**
**Don’t complain**
**Count something**
**Write something**
**Change”**
**	****	****	****“****better:**
**	****	****	****A Surgeon’s Notes on Performance”**
**	****	****	****By Dr. Atul Gawande, 2007**