# Presentation: 867265

## The Search for Knowledge

- 19-Jun-03

**Paul Brandinger:  301-286-4571; Paul.Brandinger@gsfc.nasa.gov**

**Notes:**
Thanks for the introduction and I appreciate the opportunity to speak with your group today
I work in the EOS Program Office on a variety of technical, management and organizational aspects
One activity I am involved with is a Pilot project to improve the process for sharing lessons learned
My effort is coordinated with Center and Agency initiatives associated with Knowledge Management
The Search for Knowledge is a need as old as inquisitive human nature and every day each of us pursues this search
People and organizations have the need to find, organize and share knowledge to solve problems and learn from history

## NASA’s Search for Knowledge

- Mission:  	To understand and protect our home planet		To explore the Universe and search for life		To inspire the next generation of explorers		 . . . as only NASA can

**NASA’s Search for Knowledge**

**Notes:**
Agency challenges of dealing with
Earth Science, Space Science, Human Spaceflight
As only NASA can
Requires world-class skills
Columbia, Challenger, Mars mission experiences
Need to learn from lessons of the past
President’s Management Agenda
”Knowledge management systems are just one part of an effective strategy that will help generate, capture, and disseminate knowledge and information that is relevant to the organization’s mission.”

## Typical Questions Asked by Projects/People

- What are the standard/best practices for a particular function?
- How did another Project/person do this?
- What is an example of a success in this area?
- What difficulties were encountered and how were they resolved?
- Has this problem been seen before?
- Who has experience in dealing with this area?
- What information resources exist?
- How can this be communicated to others?

**Notes:**
Projects are focused on meeting performance, schedule and cost objectives, People are given tasks and they are driven to answer specific questions
The priorities and needs of Projects and people are different even though they are addressing similar questions
Over my career whether managing a satellite Project, and functional organization or dealing with international partners I find myself asking these questions every day. We each have developed our own methods to answer such questions but the challenges are increasing 

## What is Knowledge?

- Data or facts are elements that can be processed and used for analysis
- Information places data in context and provides meaning
- Knowledge is created by people based on education, experience, thinking and decision-making; results in expertise, intelligence and wisdom
- Explicit knowledge – gained from performing a frequent and repeated task that can be documented and replicated (e.g. lesson learned)
- Tacit knowledge – know-how and insight from non-routine tasks not formally captured/documented
- Lessons learned = singles, doubles, triples, home runs and strikeouts
- Knowledge begins and ends with people

**Notes:**
It is no longer sufficient or satisfactory to complete activities just meeting performance, schedule and cost objectives if the insight gained from this effort does not provide future benefit to both the people and the organization.
To place men on Mars will require recreating much of the knowledge gained during the Apollo program since most of the people and their experience have left NASA and the contractor workforce.
Today’s discussion is sometimes referred to as KM-developed by academics and CIOs, I prefer lessons learned, as defined in a broad sense, to mean not just world-class insights.  These aspects apply to not just PMs, SE, but to resource analysts, admin people etc.

## What is the Need?

- Prevent problems from occurring/recurring
- Learn and re-learn from our past
- Exploit collective experience
- Take advantage of opportunities
- Add to the organization’s knowledge and competency base
- Address new and interesting challenges
- Working more effectively
- Get the right information to the right people at the right time
- Share knowledge across Projects

**Notes:**
Benchmarking of Government and commercial organizations indicates that this is a pervasive challenge, there is no one solution in-hand
Historical view has been that knowledge is power and sharing knowledge was giving up a competitive advantage
In today’s Web-connected society, sharing knowledge is the way to capture customers, retain the best employees and be recognized as a world-class organization
Increasing rates of change and innovation require increasing efficiency for new opportunities, those who continue to repeat errors of the past will not be rewarded

## Difficulties are Caused by:

- Loss of personnel
- Lessons and experience are not captured
- Information overload
- Rapidly evolving technology
- Competitive challenges
- Culture does not encourage sharing

**Notes:**
“The government is experiencing a ‘human capital’ crisis.  Half of its workers will be eligible to retire in the next few years, the average employee age is 46, and many agencies suffer from imbalances of available talent and needed skills.” 
Moore’s law; as highlighted by increasing computer power and communications bandwidth demonstrates that our ability to create and disseminate knowledge exceeds our capability to efficiently share/process it
If what you want from Google is on hit #900 you will never find it; we are developing solutions to this problem
Smaller spacecraft have increasing complexity, how many people remember why the design decisions on ESDIS were made?  As system upgrades become more frequent who will be around to explain the rationale.  The preference is to toss out old systems for new, but that is not always economically viable

## Why is Capturing and Sharing ofLessons Learned Difficult?

- Organization structures promote **“silo” thinking** where each area focuses on maximizing their own accomplishment and rewards, as epitomized by the “not-invented-here” syndrome.
- Science/engineering cultures generally value **personal technical expertise** and knowledge creation **over knowledge sharing**.
**Lack of contact**, relationships, and common perspectives among people who don’t work side-by-side.
- An over-reliance on transmitting **“explicit” rather than “tacit” information**.  Most of the important information people need to implement a practice cannot be codified or written down-it has to be shown to them or it requires dialogue and interactive problem solving.  Just creating databases will not cause change to happen.  It is important to capture tacit knowledge including; know-how, judgment, intuition and little tricks.
**Not allowing or rewarding people for taking the time** to learn, share and help each other outside of their own work-area.

**Notes:**
We have developed Project-oriented organizations that emphasize team accomplishment along with individual expertise
The difficulty is that sharing of insight across Projects occurs based on particular individuals; who has the specific experience and what particular Project did someone work on
People and Projects have not been motivated or encouraged to more broadly disseminate their insights
When a Project is completed, people consider that their priority is to bring their skills with them rather than spending the time or effort to allowing others an opportunity to learn from them
This creates situations where a few highly skilled individuals are overworked along with others that do not have the knowledge to perform more than a few specific tasks
People don’t write things down, they want to be asked/hired
When a mission launches developers move on to the next Project

## Knowledge Sources

- Grey-beards and retirees
- Management and staff
- Ongoing and past Projects
- Program and functional offices
- Libraries and personal files
- Contractors and consultants
- Stakeholders/Customers
- Review Boards

**Notes:**
All around us, people are where the knowledge resides, over time we develop our network and the go-to people who can help answer questions
Internal and external resources can provide much useful insight and perspective
People everyday are looking for information and points of contact from these resources that can help them solve current problems
The answers are in the heads, hard disks and 3-ring binders of people who have been around, make sure to get the boxes before they are tossed after a move
In the EOS Pilot we are working to aggregate and disseminate lessons from these knowledge holders

## Where are the Lessonsthat have been Learned?

- Numerous existing resources contain lessons
- Problem, failure, incident and anomaly reports
- Weekly activity reports and monthly status reviews
- Independent review team reports
- Top Ten lists
- Requests for actions
- Project risks
- Award fee letters and Quarterly Executive Dialogs
- Final reports, after-action reviews and case studies
- Direct extraction from these sources will ease data entry efforts
- Staff has expressed interest in gaining access to this data
- Facilitation from various organizations will be required

**Notes:**
Numerous existing repositories hold much historical experience.
Each of these areas representatives a vast wealth of historical experience that, if more easily accessible, would directly assist people and Projects in dealing with day-to-day challenges
The challenge is to harvest lessons from these sources and make it conveniently available thereby allowing stale archives to become actively usable and add value in building the organization’s knowledge base rather than being lost to file buried in some indeterminable location

## Knowledge Sharing Process

**Apply**

**Feedback**

**Capture**

**Disseminate**

**Analyze**

**Identify**

**Target**

**Organize**

**Notes:**
Previous views of how to deal with lessons learned or best practices deal with capturing and organizing information in databases and forgetting about it, even libraries have not been user-friendly
The objective is to get the right information to the right people when they need it by developing a plan for disseminating knowledge and determining the value in applying this knowledge to current activities
We also need to add value and push our  targeted information not a firehose of raw data

## Barriers to Capturing and Sharing Knowledge

- Cultural history and management practices are ingrained
- Internal priorities, interests and separate systems (stovepipes)
- Sensitivity of information limits access
- Staff
- Pride, professionalism
- Emphasis on individuals/Projects not knowledge sharing
- Difficulty in finding time
- Management has not provided scope or support for knowledge sharing

**Notes:**
Organizations have rewarded the successes of Projects and individuals, this has given rise to situations where Project/staff interests are not motivated to support the needs of other Projects or people in the organization
This philosophy, of maintaining competitive advantage, actually prevents organizations from gaining benefit of compartmentalized internal capabilities.
Furthermore, organizations fear of competition has prevented sharing of best practices, or failures, when in fact the unique corporate advantage is with their people and products, Customers are encouraged by sharing of experience and information as shown by experience with CMMI (capability maturity model – integrated) Level 5 for SE and S/W development

## Solutions

**Solutions**

**Notes:**
Starts and ends with people
Multi-faceted challenges
Incentives and motivation are key drivers
Sharing within and between teams

## Historical Emphasis

- Project Management based on having the right people
- IT provided databases, Web portals and associated tools
- Engineering orientation toward explicit information transfer
- Science community driven by personal interaction
- Quality Assurance focus on process improvement
- Library priority on content organization and services
- HR experience with mentoring, retiree outreach and incentives
- Public Affairs charter to provide outreach communications

**Notes:**
Separate parts of the organization are each addressing various aspects and approach to deal with sharing and management of knowledge
A current need is to integrate these disparate elements to create value for Project teams and individuals
The EOS Pilot is working with each of these Directorates throughout the Center to demonstrate meaningful improvements
People and Projects are trying to solve problems every day and they do not want to go to multiple sources for pieces of solutions

## Techniques for Improvingthe Sharing of Knowledge

- Discussions at regular staff/status meetings, weekly informal exchanges and special lessons-learned reviews based on story-telling
- Lessons-learned TV (Webcasts), formal seminars, mini-courses, brown-bag lunches and workshops
- Incorporation within orientation and training materials
- Targeted email and announcements including special alerts
- Common-access Web sites based on Communities of Practice providing aggregation of Project/Center libraries, and related reference resources
- Corporate publications and in-reach activities-tours/lectures
- Guest speakers during kick-off and other meetings
- People-to-people techniques including; mentoring, apprenticeships, retiree outreach, personnel locators, online chat and question-answer services

**Notes:**
Based on feedback from focus groups, surveys and discussions at Project staff meetings the emphasis is:
On people; their needs, interests and work processes
Addressing Project priorities; adding value
Integrating solutions across the organization
Helping people find information and people
Aggregating and organize from multiple sources
Pushing content out to the community rather than having them search for it
In the backup material as part of the handout there are some specific examples of how this is being done

## Incentives, Rewards and Recognition

**Time to use and create knowledge** has to be recognized and rewarded; if participants feel that they have to “steal” time from the “real” work to do this, they won't.
- Using the knowledge system has to be **self-rewarding to the consumer**; users have to get something out of it, knowledge they need or a sense of status and recognition.
- Recognition lies in **being perceived as an expert** by employees and management. Ensure that an internal expert's name is attached to documents, guidelines, and presentations they created.
**Encourage sharing, transferring, using knowledge and best practices**; by celebrating best-practice success stories and propagating tales of significant savings and important contributions.
**Recognize all parties or units involved**, both those who share knowledge and those who receive knowledge.  Both ends need to feel rewarded to achieve the desired results.
- A **“standardized” system** for rewards may help institutionalize the practice into the common culture.

**Notes:**
Studies have shown that sharing of knowledge will not be embraced unless incentives are part of the process.
Staff does not have the time, scope for priority to perform these activities unless management recognizes and rewards their knowledge-sharing efforts.

## Slide 16

**Notes:**
Directorate and Program Office management have demonstrated their endorsement and support for encouraging sharing of lessons learned
This element is a necessary but not sufficient condition for success in knowledge sharing

## Knowledge Sharing Effectiveness Drivers

- Ability to obtain and sustain a level of management support
- Endorsement
- Encouragement
- Motivation
- Approaches that provide individual reinforcement for participation
- Project scope for effort
- Peer support
- Incentives, rewards and recognition
- Demonstration of value-adds to people and projects
- Less time to find information and people
- Ability to exchange best practices, insights and experience
- Greater awareness of and access to other activities

**Notes:**
The following has become clear:
Staff are interested and want to share – everyone asks the same question, what’s in it for them
Nobody has the time or scope to do this
Project Managers have no requirement to share, except to meet individual Project objectives
There needs to be specific emphasis and encouragement to make this sharing happen
Organizations need to be actively committed if any changes/progress are to be expected

## How Can Success be Measured?

- Recognition that the knowledge sharing process offers value
- Active participation in the process at all levels
- Improved communications between those who have knowledge to share and those having the interest or need to learn from these lessons
- Better access to sources of information and people
- Support for extension/expansion of knowledge sharing activities
- Validation of performance metrics

**Notes:**
Measures can be made of:
Process performance
Value of lessons learned that are applied
Qualitative aspects
Think of the challenge that GIDEP has in determining cost avoidance, it depends on numerous assumptions yet no one would argue that this type of service does not provide tangible benefits

## What Have we Learned so Far?

- One size does not fit all; multiples process and techniques are required
- Databases/portals do not solve problems for Projects and people
- Staff interest alone cannot support active involvement
- Value-adds to existing work process will drive participation
- Project, staff and organization needs, interests and perspectives result in different priorities for the same activities

**Notes:**
The organization, Projects and staff have made their needs and interests very clear
They each have a need to share knowledge (7120.5B mandates this)
They want to share knowledge to add value and save time
They do not have the procedures, infrastructure and consistent management support yet

## What is Needed for Knowledge Sharing to Succeed?

- Management endorsement and involvement is essential
- Organizational provision of infrastructure and resources
- Process guidelines and assigned responsibilities
- Time to change the culture

**Notes:**
Management needs to walk the walk
Project management needs to be directly incentivized and motivated to add this to their responsibilities
Staff needs to be provided with time and resources to perform these additional tasks
This is like risk management, a good idea, but will take motivation, reinforcement and time to become an integral part of the system

## Slide 21

## Background Material

- 19-May-03

**Paul Brandinger:  301-286-4571; Paul.Brandinger@gsfc.nasa.gov**

**Notes:**
Why Does NASA need Lessons Learned?/What are Lessons Learned?
Typical Questions asked by Projects
Project Staff and Organization Priorities are Different
Project Document Organization
Lessons Extraction/Capture Timing and Process
Why is Capturing and Sharing of Lessons Learned Difficult?
Best Practices Through Lessons Learned
Incentives, Rewards and Recognition/Peer Awards
How to Develop an Effective Process
Focus Group summary; Success Factors
Summary of Survey Leading Indicators/EOS Project Staff Feedback
Workshops/Mini-Courses/Lessons Learned TV/Webcast Viewers
Home Page/CoP Design
Lessons Learned Process
Assessment/Metrics
References

## Why Does NASA Need Lessons Learned?

- Large number of high-value projects that can benefit from current and prior experience
- Many useful insights gained during project development are not captured, retained or transferred to other activities
- Startup of new projects consistently relearn lessons
- Opportunity to save time in locating people and information
- Encourage the capture of process knowledge
- Understand how to prevent mistakes from recurring
- Provide recognition and encouragement for successes
- Retain the experience of individuals that will retire
- Support the injection of KM efforts and best practices

## What are Lessons Learned?

- Stories about successes and failures
- Insights and experience of interest and value to others
- Examples of novel and innovative developments
- Suggestions on how processes can be improved
- Harvesting knowledge from those who have acquired it and sharing with others to allow reuse

## Project, Staff and OrganizationPriorities are Different

## Project Document Organization

## Capture Timing and Process

## Best Practices Through Lessons Learned

**IMPLEMENTATION**
**“****an idea has no value – **
**until it has been implemented !”**

**ASSESS &**
**EVALUATE**

**APPROVE & **
**DISTRIBUTE**

**COLLECT**

**MANAGEMENT**
**REVIEW OF**
**RESULTS**

**FEEDBACK**

**PROVEN VALUED PRACTICES**

**BEST PRACTICE DEVELOPMENT PROCESS**
**WITH PRESCRIBED ROLES & RESPONSIBILITIES**

**Lessons**
**Learned**

**Dreams**
**Nightmares**

**Reviews**

**CPI**

**Site**
**Visits**

**Task**

**MANY SOURCES OF KNOWLEDGE**

**TQM**

**Workshops**

**Lessons**
**Learned**

**Reviews**

**Site**
**Visits**

**Task**

**Dreams**
**Nightmares**

**CPI**

## How to Develop an Effective Process

- Start with consideration of individual and Project needs
- Develop a people-centric approach
- Listen to what is wanted
- Facilitate what people are already trying to do
- Try not change people’s habits
- Make it easy to use, leverage the experience of others and add high value
- One-size will not fit all, develop multiple techniques and adapt over time
- Engage the community
- Involve different groups within the organization that have something to offer
- Find ways to create positive interest from the bottom up
- Develop a holistic approach rather than a narrowly-defined solution
- Build-in measurement methods during the design

## Focus Group Summary; Success Factors

- Make it interesting
- Keep it simple
- Provide the ability to find what is wanted (direct/indirect) with summaries
- Offer links to related content
- Deliver quality information (reviewed)
- Ensure timeliness/currency of content
- Encourage capture and addition of content
- One size does not fit all (allow alternate pathways/look and feel)

## Summary of Survey Leading Indicators

- The majority of EOS Project staff consider that a lessons learned system _definitely_ could help them:
- Gain benefit from prior experience (79%),
- Share what they have learned with others (66%),
- Find information more quickly and identify people in GSFC with relevant skills (~50%).
- 79% of staff would _definitely_ prefer to obtain lessons learned through searching an online database.
- Staff would _definitely_ find specific lessons learned occurrence reports to be most useful (67%).
- Most staff would _definitely_ find Project documentation and identifying GSFC points of contact with specific skills to be useful (~2/3).
- More than 2/3 of the staff would _definitely_ be encouraged to use a lessons learned process to improve their job performance.

## EOS Project Staff Feedback

- Discussions were held at each Project’s staff meeting
- Staff offered mixed responses
- Projects in formulation were generally receptive and supportive of lessons learned activities
- Projects in operations have immediate needs for lessons-learned information
- Projects in implementation have standard practices in place and based on schedule as well as cost constraints are less supportive of changes to existing processes
- Staff considers that the concept of sharing lessons learned makes sense; however everyone wants to know what value they can obtain from this process
- The time and effort to perform sharing of lessons learned is not within the current scope of Project activities
- Project Managers need to actively support these efforts by allocating resources
- Project staff should understand what priority management attaches to this process

## 1st GPM Lessons Learned Workshop; In House Development

*Come out and see center experts share their thoughts and feelings concerning in-house builds*

- Wednesday, November 13, 2002
- 9:00 a.m. – 4:00 p.m.
- Building 32
- Rooms E103 & E109

- Please contact Leslie Cusick at x6-9094 or leslie.cusick@gsfc.nasa.gov if you plan on attending the first of this series
- Also contact Leslie If you would like to suggest topics or speakers for future sessions

## Earth Science Enterprise

- Science forSociety ————*December 13, 2002*

***“******Accelerating the realization of economic and societal benefits ***
***from Earth science, information, and technology ...”***

**BROWN BAG LUNCH**

## Global Carbon Cycle Plan (GCCP) BROWN BAG LUNCH

**Project Formulation Manager:  Dr. Jan Gervin**
**Project Manager:  Dr. Chuck McClain**
**Project Scientist:  Dr. Scott Denning**
**Mission Systems Engineer:Jaime Esper**
**Study Scientists:  Drs. Jim Abshire, Jim Collatz, Watson Gregg, Forrest Hall, James Hansen, Randy Kawa, Bob Knox, Antonio Mannino, Betsy Middleton,  and Compton Tucker**
** **

**January 10, 2003**

## GSFC Technology Overview Presentation 

**GSFC Technology Overview ****Presentation **

- Robert M Savage
- Flight Programs Technologist
- NASA Goddard Space Flight Center
- February 14, 2003

**EOS BROWN BAG LUNCH**

## Aerosol Mission Development

**Aerosol Mission Development**

- Mark Schoeberl
- NASA/GSFC

**Acknowledgement: Yoram Kaufman**

**March 14, 2003**

**BROWN BAG LUNCH**

## Slide 38

## Lessons Learned TV

- Over 200 items distributed to date
- Mix of mission, engineering, technology and science
- Interviews/presentations with senior staff and managers
- “New” programming scheduled for Webcast through July
- Archive of Webcasts, along with presentations where available, is accessible online

## Lessons Learned TV Webcast Viewers

**Lessons Learned TV Webcast Viewers**

- GPM Lessons Learned Workshop

- Ron Birk Mini-Course

- Mike Bay; MAP Reliability

- Jerry Madden
- GRO Lessons Learned

- Marty Davis
- GOES Lessons Learned

- Jerry Madden
- Profiles of a Project Manager

- Diane Evans; SAR Remote Sensing
- Mitch Ruda Opto-mechanical Alignment

- Rich Schnurr
- Space INTERNET

- GPM; The Promise of Rain

- Gary Martin
- NASA’ Space Architect

- Orlando Figueroa
- Engineering Excellence

## EOS Lessons Learned Process Flow

- Archive

## Community of Practice Portal Layout

## Assessment/Metricsfor the Lessons Learned Pilot

**Measurement objectives**
**Performance of process** elements
**Impact of lessons learned** by identifying improvements to projects and processes
**Qualitative aspects to staff**
**Use of various media** (resources/tools people use to obtain and share information)
- Portal access, online communities, database; entries; hits; number of distinct users
- Mentoring and storytelling, anecdotal discussions, experience from retired employees
**Application of lessons learned**; number and magnitude of benefit
**Value to projects** (performance, cost, schedule), improvement in risk and safety
**Benefit to individuals**; training, recognition, job satisfaction, networking
**Knowledge sharing**; reduced time to find information and people
- Use of systems providing extended access and communities of practice
- Access to resources and implementation of prior experiences
**Periodically survey** results to evaluate interest, effectiveness and suggestions
- Conclusion; **need to design-in measurement techniques** for the entire process

## Process Guidelines for Knowledge Capture and Sharing

- Define specific goals linked with the bottom line.
*It Does Not Matter How *_*Good*_* You Are if You Achieve the Wrong Objective*
- 2.  Access a reservoir of experience to profit from past success/failure.
*Experience Is a Great Teacher...but the “Tuition” is Expensive*
- 3.  Ensure that actions are directed at meeting goals.
*Good Intentions Do Not Guarantee Good Results *
- 4.  Accurately monitor performance.
*What Gets Measured Gets Managed*
- 5.  “Remember” how and why decisions were made.
*To Do It Better Next Time, You Must Understand “Why”*
- 6.  Improve an accessible reservoir of experience.
*Having Data Does Not Mean That It Is Accessible*

## Learn to Fly; Leveraging What We Have Learned

- Professor John Henderson from Boston University once told a powerful story about the US Army to a gathering of senior BP managers. Before he started; he asked whether BP had any formal approach capturing strategic knowledge. The chief engineer raised and described a database of 'project lessons learned'. John acknowledged this, and went on to tell this story:
- I interviewed a colonel. Now, this colonel was a colonel in one of more elite groups in the US Army. He got a call on Saturday morning at 8 o'clock reminding him that a hurricane had just hit. He was told that because the current administration had very strong ties to that particular part of the country that they did not believe that this should be left to the reserve group because they wanted "no screw-ups".
- So the orders to the colonel were very clear: go down there, provide any support necessary to the people after this hurricane and don't screw up. Clear orders. The army calls it intent - strategic intent.
- The strategic intent was clear.
- 'This particular colonel had never actually commanded any type of civilian-related activity. He 'd always been right on the front lines in hot action. It turns out as part of the executive education in the army; he had been exposed to the "Centre for Army Lessons Learned" as part of their executive education process.
- So he got on his laptop computer he dialed into Army net, hooked into the Centre for Army Lessons Learned and asked the following question - he actually showed me the type:
- “What does the Army know about hurricane clean-up?"

## Learn to Fly; cont’d

- Within four hours he had:
- A profile of the deployment of troops in the last three hurricanes that occurred in North America that the army was sent to provide support and clean-up including types of staff, types of skills, numbers of skills.
- A pro-forma budget- both what budget was required and what the actual budget was and where the cost overruns were.
- The ten questions that you will be asked by CNN in the first 30 minutes on your arrival.
- A list of every state agency and federal agency that had to be contacted and coordinated with and the name of the person that he had to contact with, and the army liaison person who was currently working with that group some place in North America.
- And he had established an advisory team of the three commanders, who agreed to be his advisory group in this command structure, four hours later.
- Then John Henderson asked: 'Is this of relevance to British Petroleum?' A pregnant pause followed, after which the chief engineer again raised his hand.
- 'You asked whether we had a formal approach to capturing knowledge? Well, we have nothing, nothing at all that is anything like that!'
- (provided by Mikael Lindvall from the book Learning to Fly by BP)

## References

- EOS Pilot for Lessons Learned:  http://eos.gsfc.nasa.gov/eos-ll/
- NASA Knowledge Management:  http://km.nasa.gov/index.html
- KM in the U.S. government:  http://km.gov/
- KM in the Nation’s Capital:  http://www.icasit.org/km/kmrt/kmregional.htm
- American Productivity and Quality Center:  http://www.apqc.org
- Communities of Practice:  http://www.ewenger.com/ewbooks.html
- Incentives:  http://in3.dem.ist.utl.pt/downloads/cur2000/papers/S28P04.PDF
- Metrics:  http://www.metakm.com/article.php?sid=269
- CSC Level 5:  http://sel.gsfc.nasa.gov/website/sew/2001/F.McGarrySession5.pdf
- COTS S/W Lessons Learned:  _[http://www.stsc.hill.af.mil/CrossTalk/2002/09/rus.pdf](http://www.stsc.hill.af.mil/CrossTalk/2002/09/rus.pdf)_
- Fraunhofer software:  http://www.iese.fhg.de/pdf_files/iese-018_01.pdf