# Presentation: 517403

## Knowledge       				 Risk			  People

**Knowledge       ****	****	****	****	**** Risk****	****	****	****  People**

- Building the Goddard Learning Organization
- Dr. Edward W. Rogers
- Office of Mission Success
- November 1, 2005

**Notes:**
Knowledge and Risk, held in the balance by the decisions of people
How well can they apply their knowledge to address risk

## Knowledge Management Office at GSFC 

- Ed Rogers, BS - Ohio State; MIB - Univ. S. Carolina, Ph.D. - Cornell
- Consultant, International Relief Projects, Professor of Business
- Started at NASA in May 2003
- Task Challenge
- Understand and describe the KM and learning organization problems in an actionable way
- Develop a coherent and coordinated approach to help make GSFC a better learning organization
- Develop and implement practices that can be readily adopted across the center to improve our learning and knowledge management
- Help smart people work together for mission success
- Creating, sharing and applying our best collective knowledge
- First Priorities
- Understand NASA and Goddard
- Gain credibility for the KM function
- Develop a plan that people understand and support

## Not a Learning Organization

- “Shuttle management declined to have the crew inspect the Orbiter for damage, declined to request on-orbit imaging, and ultimately discounted the possibility of a burn-through.”
- “The Board views the failure to do so as an illustration of the lack of institutional memory in the Space Shuttle Program that supports the Board’s claim... that NASA is not functioning as a learning organization.”
- 	CAIB Report (2003) Section 6.1, Page 127

**Notes:**
Claim made by the CAIB report. Widely cited. Little understood.
Not a LO because of a failure to act. 
Why did the org fail to act?

## Unintended Consequences

- “NASA’s culture of bureaucratic accountability emphasized chain of command, procedure, following the rules, and going by the book. While rules and procedures were essential for coordination, they had an unintended but negative effect. Allegiance to hierarchy and procedure had replaced deference to NASA engineers’ technical expertise.”
-  		CAIB Report Vol 1, Section 8.5, Page 200

## Accepting Risk

- “When a program agrees to spend less money or accelerate a schedule beyond what the engineers and program managers think is reasonable, a small amount of overall risk is added. These little pieces of risk add up until managers are no longer aware of the total program risk, and are, in fact, gambling.”
-          CAIB Report Vol 1, Section 6.2, Page 139

## Blocked Communication

- “The organizational structure and hierarchy blocked effective communication of technical problems. Signals were overlooked, people were silenced, and useful information and dissenting views on technical issues did not surface at higher levels. What was communicated to parts of the organization was that O-ring erosion and foam debris were not problems.”
- 		CAIB Report Vol 1, Section 8.5, Page 201

## Lacks Checks & Balances

**Lacks Checks & Balances**

- “The Board concludes that NASA’s current organization does not provide effective checks and balances, does not have an independent safety program, and has not demonstrated the characteristics of a learning organization.”
- 		     CAIB Report Vol 1, Synopsis, Page 12

## The Organization accepts unintended consequences
	Changes in classification of foam anomalies improved schedule but were detrimental to safety. 
The Organization stumbles over itself
	Engineering opinion was controlled by stifling demand for rule adherence to the point where no images were obtained of the orbiter. 
The Organization lacks capability for error correction
	Safety organization failed to operate as an error correction mechanism.

- 	*Changes in classification of foam anomalies improved schedule but were detrimental to safety. *
- The Organization stumbles over itself
- 	*Engineering opinion was controlled by stifling demand for rule adherence to the point where no images were obtained of the orbiter. *
- The Organization lacks capability for error correction
*	**Safety organization failed to operate as an error correction mechanism.*

**Not Functioning as a Learning Organization? **

## Where Does KM Fit in the Big Scheme?

**Notes:**
Why is KM such a lofty topic if it is so integral a part of everyday work?

## The Knowledge Production Function

**Notes:**
Need to understand the mechanism of action that connects KM to organizational output. What are the leverage points?

## How We Accomplished So Much

**How We Accomplished So Much**

## The KM Problem at the Project Level 

- Not Reliable
- Designer dependent outcomes (team make up determines team outcome as much as team function or structure)
- Organizational communication processes introduce risk to system (redundancy, reliability delusions, stress points)
- Knowledge loops are longer than operational throughput cycle time (knowledge is not timely in application)
- Not Sustainable
- Social networks are decaying faster than they are being reproduced
- Knowledge sharing legacy systems are not built around today’s workplace structures
- Mentors have a time-space gap with Mentees for effectively sharing knowledge

**Notes:**
What we do for a business is manage projects, from start to finish. That is where the KM meets the business value chain. KM must improve the way we do projects to be meaningful to GSFC.

## Knowledge Management Office in 170

**Knowledge Management Office in 170**

- “Enhances Center performance as a learning organization through leadership of the knowledge management function including lessons learned, knowledge sharing and training initiatives.”

- Learning

- Sharing

- Training

## Goddard’s Learning Plan

-        **Goals of Learning Plan**
- Build a Learning Organizational Culture
- Manage Knowledge Assets Efficiently
- Facilitate Effective Knowledge Application
-        **Learning Practices**
- Pause and Learn
- Sharing Workshops
- Case Studies
- Lessons Learned
- Training & Development
- Design Rules

- “The Goddard Plan is designed to overcome the previous Agency focus on IT as a KM driver with its over-emphasis on ***capturing*** ***knowledge*** ***from*** ***workers*** for the organization and instead focuses on facilitating ***knowledge*** ***sharing*** ***among*** ***workers***.” 				p5 of draft Goddard Learning Plan

**Notes:**
No  mention of knowledge capture
Capture is not a prerequisite to sharing and application, actually it is an obstacle

## Open Loop Lessons LearnedTypical IT Tools Driven Approach

- Focus is on
- Deploying the
- LL Tool Set

- Capture is the Key Word

**Notes:**
CAPTURE is part of all KM systems designed by IT people because otherwise they have no role in KM
BUT KM is not fundamentally an IT problem. In this case, the system actually is a barrier to learning.
It does not bring people together but isolates them (anonymous to anonymous reporting)
It provides little benefit to the contributor.
It generates no interaction coefficient for learning.
Learning is represented as a ‘Dumb’ system.

Closing the Learning Loop
Lessons Learned from Individuals to the Agency LLIS and back to Individuals is:
Too open ended (not everything makes it in)
Too long (many people wait until mission end)
Too unreliable (material not always validated)
Devoid of context (not easy to assimilate)
Not part of project management culture
Viewed as largely a ‘check box’ activity

## Local Loop Learning ProcessPeople Process Driven Approach

- Focus is on
- Learning in the
- Work Group

- Share is the Key Word

**Notes:**
Learning takes place among people at the work level.
KM tools facilitate sharing, not extracting
Lessons are examined in the course of work (not separate from it).
Center provides means to close loop with training and policy.
LLIS serves as a system repository only. 

Learning Takes Place at the Work Site
Within Project Teams (the doers are the owners)
Within Project Schedules (part of project lifecycle)
Lessons are Shared within the Center
Project, Branch, or Program Level of Ownership
After Action Reviews and Lessons Learned
Workshops and Cases
Technical seminars and Mini Courses
The Owners Share the Lessons & Insights with Center

## Goddard KM Architecture

## Goddard’s Six Key Learning Practices

**Notes:**
The six key practices

## The Need to Pause, Reflect, and Learn

- Lessons Learned are nothing more than a collection of our reflections on our experiences: If we don’t stop and reflect: we generally don’t learn much.
- Reflection helps us overcome near-miss-bias (NMB)
- Must learn from what we did right
- The time to learn is right after the experience
- Conducted a year long pilot adapting the After Action Review Concept to NASA
- Held numerous facilitated sessions
- Worked with project teams to debrief, collect insights & share
- Produced a white paper on the Pause & Learn Concept
- Available from www.missionsuccess.gsfc.nasa.gov

## Pause And Learn Sessions (PAL)

- Produced a white paper describing how we will adapt the process for use inside NASA.
- Obtained some seed money to train facilitators
- Piloted with several projects
- GOES/POES
- ST-5
- SWIFT
- Rolling out to other projects
- JWST
- Showing how process is:
- Immediately useful to the team
- Helps complete other requirements for LL
- Established process for holding center wide sharing within 60 days post-launch for every mission

## Knowledge Sharing Workshops

| Date | Topic | Target Audience | Panel Members | # |
| --- | --- | --- | --- | --- |
| 12/2/03 | Jerry Madden's Proj Mgmt Principles | Project Managers | M.Davis | 74 |
| 2/6/04 | WIRE Case Study | Project Mgrs & Subsys Leads | J. Watzin    D. Everett      J. Hrastar | 30 |
| 3/16/04 | TIMED Case Study | Project Managers | J. Wolfe     G. Colon     B.Campbell | 25 |
| 4/27/04 | VCL Case   Study | Project Mgrs & Systems Engrs | P. Sabelhaus     R. Dubayah | 35 |
| 6/15/04 | Code T LL Workshop | Program and Project Mgrs | Barrowman Hraster Powers McCarthy Rogers | 5 |
| 7/15/04 | ICESAT   GLAS  | Project Mgrs & Systems Engrs | J. Abshire,      E. Ketcham      C. Krebs     | 35 |
| 9/22/04 | Hitchhiker Lessons | Project Mgrs & Safety Eng | G. Daelemans     M.Wright           J. Harper | 22 |
| 10/26/04 | NOAA-N Prime Mishap | Project Mgrs & Safety Engineers | C. Scolese | 75 |
| 12/9/04 | TDRSS Case Study | Project Mgrs & System Engrs | R. Jenkins Ed Lowe  | 20 |
| 3/22-23/05 | Project Mgmt Challenge Conference  | Project Mgrs & System Engrs | VCL, STEREO, SSPPO, CREAM, NOAA N'  | 100+ |
| 4/7/05 | CONTOUR   | Project Mgrs & System Engrs | With APL and GSFC Ed Reynolds Tom Mangus | 75+ |
| 7/14/05 | ESDIS Case Study | Project Mgrs & Science Leads | D. Perkins  R. Obenschain     J. Dalton    C. Scolese | 60 |
| 8/03/05 | CREAM / EQUUS II | Project Mgrs & System Engrs | Jeff Reddish, John Hickman | 25 |

- Sharing

## Case Studies

| PROJECT | TYPE of CASE | Use To Date | Planned Use |
| --- | --- | --- | --- |
| NOAA N prime | Teaching Case | Discussion by Proj. Mgr done at PMChallenge Conference March 05; KSW held 11/04 | RTMS Workshop |
| VCL | Case & Teaching Case | Used at KSW 4/04
Used at PMChallenge Conference March 05 | RTMS Workshop |
| TIMED | Teaching Case | Used at KSW 3/04 | RTMS Workshop |
| CREAM | Teaching Case | Used at PMChallenge Conference March 05 | Planned for WFF 07/05 |
| GENESIS | Teaching Case | Used at PMChallenge Conference March 05
Used at ST-5 Project PAL Session | RTMS Workshop |
| SSPPO | Teaching Case | Used at KSW 9/04
Used at PMChallenge Conference March 05 | TBD |
| TDRSS | Teaching Case | Used at KSW 12/04 | TBD |
| Comanche | Teaching Case | Used at PMChallenge Conference March 05 | TBD |
| Columbia Rescue | Teaching Case | Not yet used | TBD |
| STEREO | Teaching Case | Used at PMChallenge Conference March 05 | RTMS Workshop |
| AC-67 | Teaching Case | Used at TMT (in 04 and 05) | TMT 06 |
| MAP | Reference CD | Used as example for CRT Presentation 3/05 | KSW Sum 05 |
| EOSDIS | Case Study | KSW scheduled for July 14, 2005 | KSW July 05 |
| SWIFT | Case Study | Not yet released | KSW Fall 05 |
| Calipso | Case Study | Not yet released | KSW Spr 06 |
| ICESat/GLAS | Case Study | Not yet released; KSW held 7/04 | RTMS Workshop |

## How Can We Learn From This?

## Goddard Learns from NOAA N-PRIME

| Sept. 6, 2003 | Mishap Occurs in Sunnyvale, CA |
| --- | --- |
| Sept. 13, 2004 | MIB Report Completed |
| Oct. 26, 2004 | GSFC Center Wide Knowledge Sharing Workshop |
| November 2004 | CAP and Implementation Report Done by Project |
| December 2004 | Draft Lessons Learned Done by Project |
| December 2004 | LL Included in Center Common LL Review  |
| January 2005 | Project LL Reviewed by Center KM Architect |
| February 2005 | ASK Magazine Article by Marty Davis, Program Manager |
| February 2005 | Draft Case Study ready for insertion into training |
| March 2005 | Team Debrief Pause and Learn Session |
| March 2005 | Case Study on team response at PMChallenge Conference |
| June 2005 | NOAA N Launch Successful: Team Reflects on Lessons |
| July 2005 | NOAA N Team Shares LL in Center Wide Workshop |

**Notes:**
What have we done at Goddard since this mishap?

## Learning

## Common Lessons Learned Process

**recommend**

**for **

**case study**

**recommend**

**for Sharing**

**recommend**

**for change**

**recommend**

**for change**

**adapt immediately**

**for teaching**

**recommend lesson **

**be included in **

**Common LL **

**Review**

**Develop Case**
**Study **

**Quarterly**
**Review of**
**Lessons**

**Sharing Workshop, **

**SMS Week, **

**Supervisors Mtg.**

**Annual**
**Review**
**of Common**
**Lessons**

**Design Rule, Directives**
**or GEVS Revision**
**Process**

**Develop**
**Teaching**
**Case/Lesson  **

**Common Lessons **

**Learned Review **

**Panel Meets **

**Annually to **

**Review **

**Cumulative **

**Lessons**

**recommend**

**immediate consideration**

**for Rule, Directives, GEVS Change**

**Report **
**to PMC **

**Industry / Gov**
**Lessons Sources **

**NASA LL, Tech**
**Stds, & Cases **

**RFAs ( IDR)**

**RFAs (EPR)**

**NCR**

**PR/PFR**

**SOAR**

**Proj. MSRs/OSRs**

**TOP 10 Lists**

**RISKS**

**Rule Waiver Requests**

**Safety Reports**
**MIB Reports**

**INPUTS to **
**Process**

**Common Lessons Learned Process**

**Pause And Learn Sessions**

**Notes:**
The completion of the annual process is the report to PMC of recommendations on what action to take with respect to the Common Lessons Learned in Training and Policy.
This directly addresses the goal of the Diaz Report to build a system for making sure knowledge is collected and reapplied and that project managers are taught the lessons from prior projects (Challenger, Columbia etc.)

## GOLD Rule Book Development Process

**GOLD Rule Book Development Process**

## A search on the term “FPGA” will yield:

**GOLD Rule 2.10 – Electronic Design**

**Link to **_**specific**_** NASA LLIS data**

**Link to **_**specific**_** MIL-STD**

**Link to **_**relevant**_** case study**

**Link to klabs.org page on **_**FPGA design**_

**Link to JPL Blue Book **_**FPGA rules**_

**FPGA**

**SMART Search Capability****(under design)**

## The Road to Mission Success

- Workshop Series Objectives
**Engage in dialogue with senior management on “Mission Success: The Way Goddard Does Business”**
**Articulate a clear and consistent statement of the rules, processes, and values that contribute to Goddard’s success with flight missions**
**Identify the array of **
**	****support mechanisms in **
**	****place at Goddard**
**Expand and empower **
**	****the cadre of existing and **
**	****potential leaders and **
**	****managers within Goddard**

- Training

## KM Support Efforts Underway at GSFC

- Digital Asset System
- Video Clip Retrieval (Streamsage)
- How to Get it Guide (Reference Searching)
- Semantic Search Pilot with INXIGHT
- Document Repository Standards Development
- Project Document Preservation
- Anomaly Trending Analysis
- Community Portal Development (Use PBMA as much as Possible)
- Center-Wide Document Management

## Video Query Results

## DAS Search Interface

**DAS Search Interface**

**DAS Results Page**

**DAS Metadata Record**

## Lessons Building Learning in the Army 

- The knowledge of the Army profession resides primarily in the minds of its members.
- Connecting members allows the knowledge of the profession to flow from those who know to those who need to know, from those with specific experience to those who need that experience right now.
***Person-to-person connections and conversation allow context and trust to emerge and additional knowledge to flow***.
- Relationships, trust, and a sense of professional community are critical factors that set the conditions for effective connections and convesations.
- From _Company Command_ by Nancy Dixon, et.al. (2005). Center for Advancement of Leader Development and Organizational Learning. p21.

## Navy Knowledge Management Strategy

- “Knowledge Management in the DON is a centralized vision executed through decentralized implementation.”
- “...encourage commands to implement KM programs, structures, pilots, and methodologies as part of process improvement efforts.”
- “The emphasis should be on KM’s applicability to decision superiority, improved organizational performance, and individual task accomplishment.”
- “One of the basic tenets of KM is that ***it can be accomplished without the use of sophisticated IT systems***. However, technology is an enabler...”

- Knowledge Management Strategy Memo DON Oct. 20, 2005

## Why Knowledge Sharing Efforts Fail

- Knowledge management efforts mostly emphasize technology and the transfer of codified knowledge,
- Knowledge management tends to treat knowledge as a tangible thing, as a stock or quantity, and therefore separates knowledge as something from the use of that thing,
- Formal systems can’t easily store or transfer tacit knowledge,
***The people responsible for transferring and implementing knowledge management frequently don’t understand the actual work being documented***,
- Knowledge management tends to focus on specific practices and ignore the importance of philosophy.
- From _The Knowing-Doing Gap: How smart companies turn knowledge into action_ by Jeffrey Pfeffer and Robert Sutton. (1999). Harvard Business School Press. Page 22.

## Lessons Learned About Lessons Learned

- “A second generation KM Architecture must show how learning will occur across the organization to produce a continuous knowledge supply, not just how current knowledge will be efficiently harvested with no thought to replenishment. Sustainment must be part of the design if the results are to last longer than the current version of KM software deployed. All three phases of the knowledge life cycle must be supported: knowledge production, knowledge diffusion and knowledge use. ***As smart as a KM system may be, it will never be smart enough to fool the people expected to use it***.”
- McElroy, M.W. (1999). Double-Loop Knowledge Management, MacroInnovation Inc. Available from www.macroinnovation.com