# Presentation: 016425

## EOS Lessons-Learned

**Pilot Plan**

**Preliminary D R A F T**

**Paul Brandinger:  301-286-4571; pbrandin@pop400.gsfc.nasa.gov**

## Agenda

- Objectives for this meeting
- EOS Lessons Learned Pilot overview
- Charter for this Working Group
- Lessons learned considerations and proposed metrics
- Roles and responsibilities for the Lessons Learned Pilot
- Pilot activities and tasks
- Backup material

## Meeting Objectives

- Review and concur with the EOS Lessons Learned Pilot overview
- Approve the charter for EOS Lessons Learned Working group
- Consider incentives, rewards and recognition for the pilot
- Approve Working Group roles and responsibilities
- Concur with the plan of activities for the EOS Lessons Learned pilot
- Setup working subgroups
- Provide feedback regarding implementation aspects of the Pilot
- Identify actions to be addressed prior to the next meeting of this Working Group
- Agree on the date/time for next meeting of this Working Group

## EOS Lessons-Learned Pilot Overview

- Objective:  Develop an improved process for sharing lessons learned
- Approach:
- Obtain Directorate and Center support for a one year pilot effort
- Develop and implement a plan in conjunction with Directorate representatives
- Provide monthly status reviews on pilot progress
- Report on results after one year of pilot activities
- Propose next steps for follow-on to the lessons-learned pilot
- Success criteria
- Recognition of value that the lessons learning process can offer
- Active participation in the lessons learning process at all levels
- Facilitating access to sources of information; increasing awareness and usage
- Improved communications between those who have lessons to share and those having the interest or need to learn from these lessons
- Support for extension and expansion of lessons learning activities
- Validation of performance measurements for the lessons learning process

## EOS Lessons Learned Working Group Charter

- Review and concur with the EOS Lessons Learned pilot plan
- Meet monthly to review progress on the pilot (may be part of the KMWG)
- Determine where changes to pilot activities are appropriate
- Provide active direct support to pilot activities in the areas of;
- Human resources, Library, IT and PAO-related aspects,
- Facilitating tie-in with other Center KM pilots,
- Developing and applying performance metrics,
- Software prototyping and integration of applicable tools, and
- Reviewing, validating and verifying lessons learned

## Why is Knowledge Sharing of Lessons Learned Difficult

- Organization structures promote **“silo” thinking** where each area focuses on maximizing their own accomplishment and rewards, as epitomized by the “not-invented-here” syndrome.
- Science/engineering cultures generally value **personal technical expertise** and knowledge creation **over knowledge sharing**.
**Lack of contact**, relationships, and common perspectives among people who don’t work side-by-side.
- An over-reliance on transmitting **“explicit” rather than “tacit” information**.  Most of the important information people need to implement a practice cannot be codified or written down-it has to be shown to them or it requires dialogue and interactive problem solving.  Just creating databases will not cause change to happen.  It is important to capture tacit knowledge including; know-how, judgment, intuition and little tricks.
**Not allowing or rewarding people for taking the time** to learn, share and help each other outside of their own work-area.

## Methods to Improve Communication of Lessons Learned

- Weekly informal exchanges and special lessons learned reviews (Projects)
- Formal seminars, mini-courses and brown-bag lunches (HR/PAO support)
- Incorporation within orientation and training materials (HR support)
- Targeted email and announcements including special alerts (IT support)
- Common-access Web sites; communities of practice (IT support)
- Center publications and in-reach activities-tours/lectures (PAO support)
- Guest speakers during kick-off and other meetings (HR/PAO support)
- Workshops to include lessons-learned case studies (HR support)
- Add as an agenda item to regular staff/status meetings (Projects)

## Incentives, Rewards and Recognition

**Time to use and create knowledge** has to be recognized and rewarded; if participants feel that they have to “steal” time from the “real” work to do this, they won't.
- Using the knowledge system has to be **self-rewarding to the consumer**; users have to get something out of it, knowledge they need or a sense of status and recognition.
- Recognition lies in **being perceived as an expert** by employees and management. Ensure that an internal expert's name is attached to documents, guidelines, and presentations they created.
**Encourage sharing, transferring, using knowledge and best practices**; by celebrating best-practice success stories and propagating tales of significant savings and important contributions.
**Recognize all parties or units involved**, both those who share knowledge and those who receive knowledge.  Both ends need to feel rewarded to achieve the desired results.
- A **“standardized” system** for rewards may help institutionalize the practice into the common culture.

## Assessment/Metrics for the Lessons Learned Pilot

- Measurement objectives
- Performance of process elements
- Impact of lessons learned by identifying improvements to projects and processes
- Qualitative aspects to staff
- Use of various media (resources/tools people use to obtain and share information)
- Portal access, online communities, database; entries; hits; number of distinct users
- Mentoring and storytelling, anecdotal discussions, experience from retired employees
- Application of lessons learned; number and magnitude of benefit
- Value to projects (performance, cost, schedule), improvement in risk and safety
- Benefit to individuals; training, recognition, job satisfaction, networking
- Knowledge sharing; reduced time to find information and people
- Use of systems providing extended access and communities of practice
- Access to resources and implementation of prior experiences
- Periodically survey results to evaluate interest, effectiveness and suggestions
- Conclusion; need to design-in measurement techniques for the entire process

## Potential Assessment Criteria (Metrics)

- Contribution to Agency vision
- Stakeholder benefit
- Outreach opportunity
- Workforce/individual advantage
- Demonstration of success and failure
- Novelty and innovation
- Process enhancement
- Breadth of applicability
- Performance gain
- Schedule improvement
- Cost reduction
- Quality value, risk-reduction, safety-related
- Use of prior lesson learned/knowledge exchange

## Pilot Roles and Responsibilities

- _Function_						_Responsibility_
- Identify appropriate lessons to be shared and metrics	EOS/LLWG
- Capture knowledge describing lessons			Projects/Individuals
- Organize the lessons; categorize, rank			EOS
- Validate and verify applicability of lessons		EOS/LLWG
- Analyze root causes/outcomes from lessons learned	LLWG/EOS
- Develop an action plan to apply lessons learned		EOS/LLWG
- Disseminate the lessons learned and monitor results	EOS
- Track, assess and report on performance measurement	EOS
- Provide feedback on value of lessons learned		Projects/Individuals

## EOS Lessons Learned Pilot Project - Formulation

- 	_Topic_					_Metric/POC_	_Schedule_
- Develop formulation and implementation plan	Document/PB	2 weeks > WG meeting
- Application of KM approaches		Document/PB	2 weeks > WG meeting
- Define Pilot support elements
- Human resources tie-in		Agreement/DW	mid-August
- Library tie-in			Agreement/JO	mid-August
- IT tie-in				Agreement/SN	mid-August
- PAO tie-in				Agreement/LC	mid-August
- KM Pilot projects tie-in		Agreement/JL	mid-August
- LLIS tie-in				Agreement/AG	mid-August
- Leverage experience from external groups
- HQ/JPL				Meeting/JL		end-August
- DOE lessons lessons process		Meeting//JL	end-August
- Industry best practices			Meeting/JL		end-August
- Other
- Pilot process, staff and organization aspects	Document/PB	end-August

## EOS Lessons Learned Pilot Activities - 1

- 	_Topic_					_Metric_			_Schedule_
- Management and staff orientation		Inform projects/analyze survey	30 days after start
- Roundtable introduction, survey
- Storytelling at staff meetings			Interest/value in stories		Weekly
- Have a 5 minute lesson; may be an invited guest
- Status reports with lessons-learned headlines	Percent reporting with lessons	Weekly
- Everyone should prepare one-pagers
- Structured ongoing mentoring process		Mentor pairs created		Monthly
- Identify interests and resources, use of retirees
- Improve online tools to meet user needs 		Usage			Weekly
- Simple entry, search and presentation
- Facilitate access to knowledge-relevant resources	Usage			Weekly
- Online resumes, communities and libraries
- Develop a rewards and recognition process	Number being recognized	Monthly
- Encouragement for sharing lessons learned
- Reporting on progress/performance		Status briefing/feedback	Monthly
- Develop and track metrics
- Draft GPG for the lessons-learned process	Document completion		Within 1 year
- Develop a Center-wide framework

## EOS Lessons Learned Pilot Activities - 2

- 	_Topic_					_Metric_			_Schedule_
- Identify appropriate lessons to be shared		Document and distribute	30 days after start
- Develop guidelines and taxonomy
- Capture lessons (online, written, verbal, video)	Number of lessons/media	Weekly
- Prepare process for collecting lessons
- Organize lessons; categorize/rank		Number of lessons/category	Weekly
- Consolidate and store information
- Validate and verify applicability of lessons		Percentage of usable lessons	Monthly
- Working group review of lessons
- Analyze root causes and outcomes		Number of lessons analyzed	Monthly
- Subject matter expert review
- Develop action plan to apply lessons-learned	Number of lessons/targets	Monthly
- Implementation review
- Disseminate lessons-learned			Lessons to interested parties	Monthly
- Identify and track lesson usage
- Monitor results				Feedback on lesson usage	Monthly
- Develop and track metrics
- Provide feedback on value of lessons learned	Document 			Monthly
- Survey staff and report to working group

## Conclusions/Recommendations of the Working Group

- Concur with the Lessons Learned Pilot plan
- Approve the charter for Working Group activities
- Approve roles and responsibilities for the Working Group
- Organize membership for the following subgroups
- Human resources aspects
- Library aspects
- IT aspects
- PAO-related aspects
- Incentives, rewards and recognition
- Performance measurement and assessment

## Pilot Effectiveness Concerns

- Ability to obtain and sustain a level of management support
- Endorsement
- Encouragement
- Motivation
- Approaches to provide individual reinforcement for participation
- Incentives
- Rewards
- Recognition
- Demonstration of value-adds to people and projects
- Less time to find information and people
- Ability to exchange best practices, insights and experience
- Greater awareness of and access to other activities

## Next Steps

- Enlist and confirm support from Center organizations
- Develop definitized and agreed plan including KM pilot tie-ins
- Obtain EOS Program Management endorsement to proceed
- Prepare processes, procedures and identify resources
- Introduce to Program Office/Project management and staff along with a survey; baseline user views and solicit ideas
- Collect, categorize, review, store and disseminate lessons learned
- Monitor and report on performance
- Obtain management and staff feedback on value

## Backup

## What are Lessons Learned?

- Stories about successes and failures
- Insights and experience gained of interest and value to others
- Novel and innovative developments
- Suggestions on how processes can be improved
- Knowledge sharing; connecting those who have it with those who can benefit from it

## Types of Lessons Learned

- Successes and failures (home-runs, singles, doubles, triples and strike-outs)
- Major impact and influence to future efforts
- Minor benefits of limited scope
- Routine activities that demonstrate incremental changes
- Performance and process-related
- Job satisfaction
- Applicability
- Individual
- Project
- Functional area
- Technical
- Administrative
- Program
- Directorate
- Center
- Agency

## What is Lessons Learned About?

- People (who can benefit from lessons learned)
- Providers/users
- Requesters/responders
- General community
- Practices (where are lessons learned generated)
- Meetings, reviews, workshops
- Documentation
- Person-to-person
- Process (how can lessons learned be shared)
- Electronically; via Web-based portal/database and email
- Facilities-based; using Libraries
- People-oriented; storytelling and mentoring
- Performance (when lessons are shared how is value measured)
- Creation and application
- Benefit to individual, project, Directorate and Center
- Utilization of facilitating processes

## Resources Applicable to the Lessons Learned Pilot

- GSFC/NASA LLIS
- NASA Technical Standards
- Project, Program Office, Directorate and Center libraries
- GPGs, document templates/guidelines, checklists, post-launch reports
- Space vehicle design criteria monographs
- Symposia/colloquia
- External organizations
- Federal Agencies
- FFRDCs
- Universities
- International Agencies
- Domestic and foreign commercial firms

## Barriers to Lessons Learning

- Cultural history and management practices are ingrained
- Internal priorities and interests, separate systems (stovepipes)
- Sensitivity of information limits access
- Projects are self-contained entities; require motivation to share (silo- thinking)
- Staff
- pride, professionalism
- emphasis on individuals/projects not knowledge sharing
- difficulty in finding time
- determining the priority of this effort

## Lessons Learned Success Stories

- Best practices databases
- Devote people to finding, filtering and entering information
- Show people what information is available and how it can be accessed
- Use of directory and pointer systems
- Promotes sharing of practices, knowledge, know-how and experience
- Encourages those who seek help (“Requestors”) to connect with those who can provide assistance (“Offerors”)
- Simple template for entry of leads and retrieval of information
- Source organization/project, description, date entered, who entered data, point of contact
- Google-like search capability

## Outline for Lessons Learned Survey

- How can lessons learned help you?
- In sharing information, to identify a GSFC or NASA-wide point of contact
- Gaining insight from previous projects, by meeting people with similar interests
- By what means do you exchange lessons learned now?
- Via acquaintances and direct knowledge of experience on other projects
- Online search of NASA/public-domain sources
- What tools could help you in sharing lessons?
- Electronic, person-to-person, workshops
- Why would you not use a lessons learned process?
- Not enough time, too difficult
- Not interested, do not see the value
- What could be done to make lessons useful for you?
- For example; make it simple, provide broad access to historical information

## Industry Guidelines for Best Practices

- Applicable for benchmarking lessons learned
- When it:
- Produces superior results (25% higher than normal)
- Is clearly a new or innovative use of manpower or technology
- Is recognized by at three different references as a best practice
- Has received an external award for this practice
- Is deemed so by customers or suppliers
- Is recognized by an industry expert
- Organization’s using it have a patent for the practice
- Leads to exceptional performance

## Proposed Roles and Responsibilities for the Pilot

- EOS Program Office
- Project lead; champion, plan, organize, coordinate, review, and report
- Facilitator; assist in implementing activities within each project
- Lessons capturer (share with CM); obtain information from each project
- Content organizer (share with library) ;categorize, rank, validate, and verify
- Projects (share responsibilities with CM/RM functions)
- Facilitator/point-of-contact; person responsible for working with the lessons learned team and applying/tailoring/implementing activities within the project, identifying project/individual needs, and assisting project staff in obtaining information/using tools
- Lessons capturer; aggregating staff/project information for sharing
- Lessons Learned Working Group
- Review progress of the EOS lessons learned pilot
- Determine if changes to pilot activities are appropriate
- Provide active direct support for; development and application of performance metrics, software prototyping, screening lessons-learned, injecting results from other Center/Agency knowledge management pilots (e.g. myGoddard portal, multimedia asset management, knowledge preservation, etc.), and coordinating HR, Library, IT and PAO-related aspects

## Needs from Human Resources

- Facilitators and in-reach
- Training and learning methods
- Pilot, design, implementation and assessment techniques
- Mentoring integration and outreach to retirees
- Workshop support
- Incentive, recognition and reward programs
- Career development aspects
- Resume access and privacy considerations

## Needs from Library

- Provide inreach on lessons learned resource availability
- Act as a clearninghouse for where, who and how to find answers to questions
- Offer information on services that are or can be provided
- Disseminate updates on additions to holdings targeted for those with particular interests/needs
- Support a central access point for inter-library information exchange with Projects, Program Offices, Directorates, Centers and HQ

## Needs from IT

- Alternative portals for user evaluation
- Lessons learned entry/distribution
- Administrative access for categorization, review, validation and verification
- Searching among the following information sources
- LLIS and related databases
- Project, Directorate and Center libraries
- Code 400/500/900 staff resources
- Retirees/mentors and subject matter experts
- Colloquia, video nuggets, mini-courses
- Case studies
- User profiling to support distribution of relevant lessons learned
- Links and pointers to additional resources
- Process and content information
- Communities of practice
- Performance tracking
- Flow of lessons learned
- Usage and user feedback

## Needs from LLIS

- Tie-in with the NASA Technical Standards site (duplication?)
- Capability to upload or link to different types of source information (documents, video, databases)
- Issues associated with Pilot tie-in
- Opportunity to tailor and test changes to LLIS (i.e. operational system impacts)
- Ability to segregate Pilot information in various categories
- Restricted access to Pilot information
- Flexibility in next-generation software implementation

## Needs from PAO

- Use of outreach expertise to assist with inreach activities
- Ability to make in-house staff aware of outreach efforts
- Provide insight into resources that can assist staff
- Assist with improving in-house communications
- General interest and newsletters
- Highlighting achievements, needs, opportunities
- Building a bridge between in-house capabilities and external interests
- Offer opportunities that are provided via outreach to the GSFC community
- Staff, support contractors and families

## Needs from KM Pilots

- Identify opportunity for tie-in with EOS Lessons Learned Pilot
- Indicate benefit to existing pilot, and value to Lessons Learned Pilot
- Specify what the existing pilot can provide and when this will be available
- Estimate needs to/from Lessons Learned Pilot
- What metrics can be used to gauge performance and success of use associated of this pilot’s capabilities?
- What issues are anticipated in applying this pilot to the Lessons Learned Pilot?