> [Subject]{.underline} [Date/Reference]{.underline} [From]{.underline}
>
> Report on EOS Lessons Learned November 25, 2002 P. E.
> Brandinger![](media/image1.wmf)
>
> Focus Group Meetings PEB.200211.060
>
> [To]{.underline} [cc]{.underline} [Location/Phone]{.underline}
>
> Distribution P. Sabelhaus GSFC 16/226

R. Day 301-286-4571 (phone)

> 301-286-1145 (fax)
>
> pbrandin@pop400.gsfc.nasa.gov (email)
>
> On 21 and 25 November 2002 Focus Groups were held with EOS Project
> representatives to solicit views with regard to improvement of the
> lessons learning process; a list of attendees is contained in
> Attachment 1, presentation material discussed at this meeting is
> provided as separately attached files.
>
> The agenda for these focus groups was based on the following four
> questions:

1.  How is the process for sharing lessons learned performed now by EOS
    Projects?

2.  What comments are there on the NASA Lessons Learned Information
    System (LLIS)?

3.  What are the issues associated with capture and use of lessons
    learned by the Projects?

4.  What suggestions can be offered to improve the process for sharing
    of lessons learned and how would these tie-in with the Project
    life-cycle?

> The following summarizes discussions held in these focus groups
> organized in response to each question:

1.  How is sharing of lessons learned performed now?

    a.  One Project benefited from common spacecraft contractor
        experiences with testing and operations that were provided to a
        second spacecraft in this series.

    b.  One previous Project generated an after-action report but
        Projects generally do not produce a lessons learned document.

    c.  Routine interaction between people on different Projects is how
        lessons are most commonly shared.

    d.  One Project shared lessons learned by developing improved tools
        based on prior experience.

    e.  Lessons are typically shared by word-of-mouth during hallway
        conversations.

    f.  Exchange of experience takes place between Project Managers at
        MSR meetings.

    g.  Lessons are exchanged by coincidence of events.

    h.  Staff finds answers by looking for people previously involved in
        similar Projects.

    i.  Red team and readiness reviews as well as Project-sponsored
        Workshops provide opportunities to share lessons learned from
        experienced staff.

    j.  A Contractor on one Project was required to generate a short
        final report including lessons learned.

    k.  People bring the lessons that they learn from one Project to
        their next assignment.

    l.  Projects gain benefit of lessons learned by hand-picking the
        people with insight and experience.

    m.  Projects can hire, as consultants, retirees to act as mentors.

    n.  A risk management plan has been developed on one Project that
        requires generation of a lesson learned when a risk item is
        closed out.

2.  Comments on the LLIS

    a.  Don't know what it is, never heard of it.

    b.  Don't use it and some people don't want to read documents to
        find lessons learned.

    c.  No particular problem presents itself in the same way that a
        database would help address.

    d.  A lesson was once submitted to the LLIS.

    e.  Contact with people will provide greater value than just
        retrieving information.

    f.  LLIS process was too cumbersome, reviewer feedback to format
        lessons discouraged people from entering more lessons.

    g.  Unless you know what you are looking for it is difficult to find
        relevant information.

3.  Issues associated with capture and use of lessons learned

    a.  People are not funded, provided the time, nor motivated to share
        lessons learned.

    b.  GSFC culture focuses on current Projects, not what was done
        before or how it will be done in the future (except through NPGs
        and GPGs).

    c.  After a Project is completed neither budget nor schedule is
        available to document lessons learned.

    d.  At the end of a Project, people leave or move-on without
        capturing lessons learned.

    e.  As people retire their knowledge and experience is leaving GSFC.

    f.  Staff are not encouraged to bring their files from previous
        Projects to their next Project assignment.

    g.  Most Projects do not have a formalized process or a specific
        approach for handling lessons learned.

    h.  Lessons learned are for the most part not written down by the
        Projects.

    i.  Why is a formal system needed for lessons learned? Project
        Managers are able to create the necessary environment to
        encourage the sharing of corporate experience.

    j.  I have many ideas, where to put these?

    k.  Workshops and mini-courses have limited attendance,
        participation needs to be encouraged.

    l.  People don't have the time to sift through a massive volume of
        information. After-action reports while containing good
        information do not allow people to quickly find lessons without
        going through the entire document.

    m.  Libraries that organize documents in sequential order make
        finding information difficult.

    n.  Obtaining information from databases and libraries is difficult,
        people won't use them.

    o.  NASA and Contractors unwillingness to report mistakes,
        people/organizations don't want to show their weakness. Need to
        make reporting of negative lessons non-threatening.

    p.  Lessons learned has a connotation of negative events, there
        needs to be encouragement for capture of best practices and
        successes.

    q.  Contractors will not share information that could impact a
        competitive advantage.

    r.  Dealing with multiple Contractors before contract award limits
        information exchange.

    s.  NASA's culture does not encourage capturing or sharing
        information, people want to get and hold not give.

    t.  Interaction is not encouraged in the NASA culture, either
        between Projects or individuals.

    u.  Each Project Manager applies their personal approach to dealing
        lessons learned.

    v.  People-to-people contacts should not need to go through Branch
        Heads.

4\. Suggestions for capture and sharing of lessons learned tied-in with
the Project life cycle

a.  An additional (optional) MSR chart on lessons learned could be
    incorporated as a walk-on.

b.  At completion of each Project there should be a lessons learned
    debriefing and a final report.

c.  Providing lists of subject matter experts would be an immediate
    value-add.

d.  Better use needs to be made of the Center's matrix organization to
    share experience.

e.  Weekly functional and cross-Project roundtable meetings should be
    held to increase awareness of ongoing activities and issues.

f.  Staff prior Project experience and their responsibilities on those
    Projects should be captured so that those seeking assistance can
    find people with relevant background.

g.  Project monthly reviews should periodically be attended by AETD
    branch heads.

h.  Project monthly reviews should be attended by senior staff with
    cross-Project experience.

i.  Results of Project reviews should be more widely disseminated.

j.  Retirees are a source of extensive lessons learned and their
    experience can inspire people.

k.  Use of existing communications techniques such as the Code 400
    Critical Path should be used for dissemination of information
    related to sharing of lessons learned.

l.  If you can find what you want, when you need it then people will use
    it.

m.  Whatever system is developed for sharing lessons learned needs to be
    simple to use.

n.  Word-of-mouth is the easiest process for sharing lessons learned.

o.  Sharing lessons learned needs to be part of the normal process and
    not labor intensive.

p.  Sharing of lessons learned should be integrated as part of the
    Project review process.

q.  Lessons should provide sufficient summary level information to allow
    an assessment of interest/relevance without having full content
    presented.

r.  It is necessary to know what lessons are useful to capture and that
    lesson quality is checked.

s.  Linking related lessons together would be beneficial.

t.  Lessons need to be organized by Project phase, the situational
    aspect of lessons will determine their applicability.

u.  People who can see across multiple Projects should prepare lessons
    learned, and outsiders could write more uniform lessons learned and
    streamline the process.

v.  Best practices need to be captured.

w.  Staff needs to be encouraged (pushed) to write things up.

x.  Need to develop a uniform approach to deal with contractors in
    capturing lessons learned.

y.  The lessons people are looking for need to include things they
    haven't thought about and related aspects with the potential to
    cause problems.

z.  There should be greater emphasis on people-to-people processes such
    as informal mentoring and retiree outreach, people like
    show-and-tell (storytelling).

a.  One-to-one techniques are as, if not more, important than
    one-to-many approaches for sharing lessons learned.

b.  There needs to be more encouragement from Center, Program and
    Project management for sharing of lessons learned.

c.  The people-to-people process for sharing lessons learned needs to be
    interesting and should address a diversity of topics (i.e.
    programmatic, management, engineering, resources, etc.).

d.  Building small spacecraft in-house is one of the most effective
    approaches to gain and exchange Project experience.

e.  The process for capturing and sharing lessons learned needs to be
    formalized and mandated.

f.  Center and Program management needs to motivate a culture change for
    knowledge sharing.

g.  Quarterly briefings to Directorate management on the subject of
    lessons learned should be provided.

h.  Participation in sharing lessons learned needs to be continuously
    reinforced at all levels.

i.  At every meeting and review the question should be asked; "What is
    the lesson learned?"

> Distribution:
>
> EOS Lessons Learned Working sub-Group Members
>
> EOS Pilot for Lessons Learned Focus Group Attendees
>
> EOS Project Managers and Deputies
>
> **Attachment 1**
>
> **EOS Pilot Focus Groups**
>
> **21/25 November 2002**

  ---- ------------- ---------- ---------- --------------- --------------------------------------
       **Name**                 **Code**   **Telephone**   **Email**

       Wolford       Don        114        301-286-9236    <Donald.L.Wolford@nasa.gov>

       Ormes         Janet      292        301-286-6728    <Janet.D.Ormes@nasa.gov>

       Hodge         Gail       292        610-789-6769    <gailhodge@aol.com>

       Gallo         Al         304        301-286-3756    <Albert.M.Gallo@nasa.gov>

       Luce          Peg        400        301-286-6527    <Margaret.A.Luce@nasa.gov>

       Mortimer      Katy       400        301-286-7025    Catherine.D.Mortimer.1@gsfc.nasa.gov

       Yancy         Terri      400        301-286-5566    Terri.R.Yancy@nasa.gov

       Margolies     Don        410        301-286-8984    <Donald.L.Margolies@nasa.gov>

       Brandinger    Paul       420        310-286-4571    Paul.Brandinger@gsfc.nasa.gov

       Schiavone     Bill       420        301-286-5417    <William.J.Schiavone@nasa.gov>

       Chandler      Lynn       420        301-286-2806    Lynn.Chandler-1@nasa.gov

       Roman         Juan       443        301-286-4727    <Juan.A.Roman@nasa.gov>

       Butler        Madeline   500        301-286-4806    <Madeline.J.Butler@nasa.gov>

       Guy           Paul       565        301-286-8804    <Paul.D.Guy@nasa.gov>

       Naus          Steve      585        301-286-5640    <Stephan.A.Naus@nasa.gov>

       Hull          Larry      588        301-286-3009    <Larry.G.Hull@nasa.gov>

       Comberiate    Mike       420.1      301-286-9828    <Michael.A.Comberiate@nasa.gov>

       Dacey         Steve      420.1      301-286-4925    Stephen.W.Dacey.1@gsfc.nasa.gov

       Adams         Jim        420.2      301-286-2508    <W.J.Adams@nasa.gov>

       Everett       Dave       420.2      301-286-1596    <David.F.Everett@nasa.gov>

       Sluder        Bill       420.3      301-286-8976    <William.H.Sluder@nasa.gov>

       Miller        Ron        420.3      301-286-6331    <Ronald.A.Miller@nasa.gov>

       Ullman        Richard    423        301-614-5228    <Richard.E.Ullman@nasa.gov>

       Razzaghi      Andrea     424        301-286-1386    <Andrea.I.Razzaghi@nasa.gov>

       Berry         Rick       424        301-286-9620    Richard.P.Berry.1@gsfc.nasa.gov

       Seftas        Randy      428        301-614-5122    <George.R.Seftas@nasa.gov>

       Shimkaveg     Phil       428        301-614-5027    <Philip.M.Shimkaveg@nasa.gov>

       Krimchansky   Sergey     429        301-286-9843    <Sergey.Krimchansky-1@nasa.gov>

       DeVito        Dan        429        301-286-4850    <Daniel.S.Devito@nasa.gov>

       Dew           Howard     586        301-614-5329    <Howard.Dew@nasa.gov>
  ---- ------------- ---------- ---------- --------------- --------------------------------------