# Presentation: 136480

## Mike Rackley - Establishing Effective Processes for Operations Best Practices and Lessons Learned
Operations Best Practices - ‘rules of thumb’ - generally recognized, but not always formally documented
Operations Lessons Learned - experience based - can lead to best practices and serve as aid to future projects
Knowledge Management - capture relevant information for reuse - real-/near real-time
Influencing Variables
S/C design/technology
GS design/technology
Risk tolerance
Organizational culture
Personnel changeover
Mission models/profiles
An effective Anomaly Management process often drives/establishes foundation for above - identify root cause(s), identify/implement corrective action(s), formalize recommendations/lessons learned, and ensure ready access and consistent implementation over time
Independent Quality/Mission Assurance to monitor/enforce process implementation
Can one size fit all? (multiple missions and mission phases)
Each mission wants/needs its own
Wider implementation affords ‘element of scale’ benefits
Well-defined requirements facilitate effective/’universal’ implementation
Clearly defined roles and buy-in of Operations Organization AND independent Mission Assurance Organization critical to effectiveness
Exposition of GSFC implementation/issues - see SOSTC website for full presentation

- Operations Best Practices - ‘rules of thumb’ - generally recognized, but not always formally documented
- Operations Lessons Learned - experience based - can lead to best practices and serve as aid to future projects
- Knowledge Management - capture relevant information for reuse - real-/near real-time
- Influencing Variables
- S/C design/technology
- GS design/technology
- Risk tolerance
- Organizational culture
- Personnel changeover
- Mission models/profiles
- An effective Anomaly Management process often drives/establishes foundation for above - identify root cause(s), identify/implement corrective action(s), formalize recommendations/lessons learned, and ensure ready access and consistent implementation over time
- Independent Quality/Mission Assurance to monitor/enforce process implementation
- Can one size fit all? (multiple missions and mission phases)
- Each mission wants/needs its own
- Wider implementation affords ‘element of scale’ benefits
- Well-defined requirements facilitate effective/’universal’ implementation
- Clearly defined roles and buy-in of Operations Organization AND independent Mission Assurance Organization critical to effectiveness
- Exposition of GSFC implementation/issues - see SOSTC website for full presentation

## David Oberhettinger - NASA Lessons Learned: Moving to a More Formal Process
Repeated mistakes/violations of known best practices identify an opportunity to add more rigor to the NASA lessons learned process
JPL programmatic environment promotes effective BP/LL processes
High-risk, unique missions
Evolving project mix
Eroding institutional knowledge base
Reorientation to a procedure-based design process
Current NASA Lessons Learned System in place since 1992, but not fully adopted with rigor, center-centric
Cost ($, prestige) of ineffective LL system mandates some assurance LL are identified and actually learned
NPR 7120.6, The NASA Lessons Learned Process (recently issued)
Establishes basic NASA requirements for collection, validation, assessment, codification, and infusion of LL
PI - Establishes a Lessons Learned Committee at each Center - adequately funded with appropriate representation
PI - Requires Center-wide infusion of lessons learned recommendations into procedures and training
Lessons Learned Process Issues
Establish criteria for an effective lessons learned process - How to implement requirements?
High-level Center commitment
Center-wide participation in proposing, vetting, disseminating, and using the lessons
Exposition of JPL implementation - see SOSTC website for full presentation

- Repeated mistakes/violations of known best practices identify an opportunity to add more rigor to the NASA lessons learned process
- JPL programmatic environment promotes effective BP/LL processes
- High-risk, unique missions
- Evolving project mix
- Eroding institutional knowledge base
- Reorientation to a procedure-based design process
- Current NASA Lessons Learned System in place since 1992, but not fully adopted with rigor, center-centric
- Cost ($, prestige) of ineffective LL system mandates some assurance LL are identified and _actually_ learned
- NPR 7120.6, *The NASA Lessons Learned Process (*recently issued)
- Establishes basic NASA requirements for collection, validation, assessment, codification, and infusion of LL
- PI - Establishes a Lessons Learned Committee at each Center - adequately funded with appropriate representation
- PI - Requires Center-wide infusion of lessons learned recommendations into procedures and training
- Lessons Learned Process Issues
- Establish criteria for an effective lessons learned process - How to implement requirements?
- High-level Center commitment
- Center-wide participation in proposing, vetting, disseminating, and using the lessons
- Exposition of JPL implementation - see SOSTC website for full presentation

## Irene Tollinger/Keith Shackelford - Anomaly Resolution Research and Tool Development
Background - NASA Ames delegated responsibility for recommending changes in the NASA anomaly identification/resolution process
Goals
Gather and integrate AR process data across NASA
Develop tools to support AR participants based on data-driven requirements
Recommend solutions to NASA management on AR identification/resolutions
Work Plan
Background research
Retrospective interviews
In-situ observations and data collection
Iteratively prototype AR support tools
Collect tool usage metrics for existing processes and the new tool
Research team expertise in human-computer interaction, AR, risk assessment, and failure analysis
Contextual inquiry adapted to capture data in model form for analysis
Numerous AR process gaps found (e.g., unreliable fidelity of data captured, difficult search capability, etc.)
Recommendations based on process gaps and observations of best practices
Proposed tool: Anomaly Consequence Evaluation System (ACES)
Modules designed for specific customer (e.g., console engineer, analysis team, decision maker, internal stakeholder)
Next steps: on-going concept validation, usability testing, add greater functionality, delivery December ‘05
Continue AR best practices research (Workshop 6/28-29), future process development efforts
See SOSTC website for full presentation

- Background - NASA Ames delegated responsibility for recommending changes in the NASA anomaly identification/resolution process
- Goals
- Gather and integrate AR process data across NASA
- Develop tools to support AR participants based on data-driven requirements
- Recommend solutions to NASA management on AR identification/resolutions
- Work Plan
- Background research
- Retrospective interviews
- In-situ observations and data collection
- Iteratively prototype AR support tools
- Collect tool usage metrics for existing processes and the new tool
- Research team expertise in human-computer interaction, AR, risk assessment, and failure analysis
- Contextual inquiry adapted to capture data in model form for analysis
- Numerous AR process gaps found (e.g., unreliable fidelity of data captured, difficult search capability, etc.)
- Recommendations based on process gaps and observations of best practices
- Proposed tool: Anomaly Consequence Evaluation System (ACES)
- Modules designed for specific customer (e.g., console engineer, analysis team, decision maker, internal stakeholder)
- Next steps: on-going concept validation, usability testing, add greater functionality, delivery December ‘05
- Continue AR best practices research (Workshop 6/28-29), future process development efforts
- See SOSTC website for full presentation

## Andres Aparicio- Lessons Learned and Best Practices in Commercial Spacecraft Operations
Basic LL process
Detect
Categorize
Learn
Disseminate
Establishing a best practice
Evaluate current processes to identify candidates for improvements
Test
Train
Implement
Share? - commercial and military environments raise different information sharing challenges
See SOSTC website for full presentation

- Basic LL process
- Detect
- Categorize
- Learn
- Disseminate
- Establishing a best practice
- Evaluate current processes to identify candidates for improvements
- Test
- Train
- Implement
- Share? - commercial and military environments raise different information sharing challenges
- See SOSTC website for full presentation

## Mike Mattis - Satellites and Submarines: Sharing Lessons Learned about Complex System Design and Operations
Similarities between satellites and submarines
Complex systems with critical missions built with specialized parts and processes
Designed to operate in harsh environments with redundant components
Operated by highly skilled personnel to perform highly complex tasks
No 2nd chances once deployed
Operations mistakes can have extreme consequences
Smart people and good processes make the difference for mission success
Design around known problems and limitations
Anticipate issues based on past lessons
Design systems for operations and be operationally responsive
Catch design mistakes and problems
Recognize unusual and abnormal conditions
Take appropriate contingency/recovery actions
Implement innovative operational fixes
Establish a team-oriented program with common goals that focus on overall success, not individual failures
Certified standard processes ensure repeatability and success
Provide technical leadership that goes beyond book learning and focuses on practical applications/processes
Exposition of design and operations fundamentals common to both satellites and submarines
Institutional memory for design and operations cannot be taken for granted
See SOSTC website for full presentation

- Similarities between satellites and submarines
- Complex systems with critical missions built with specialized parts and processes
- Designed to operate in harsh environments with redundant components
- Operated by highly skilled personnel to perform highly complex tasks
- No 2nd chances once deployed
- Operations mistakes can have extreme consequences
- Smart people and good processes make the difference for mission success
- Design around known problems and limitations
- Anticipate issues based on past lessons
- Design systems for operations _and_ be operationally responsive
- Catch design mistakes and problems
- Recognize unusual and abnormal conditions
- Take appropriate contingency/recovery actions
- Implement innovative operational fixes
- Establish a team-oriented program with common goals that focus on overall success, not individual failures
- Certified standard processes ensure repeatability and success
- Provide technical leadership that goes beyond book learning and focuses on practical applications/processes
- Exposition of design and operations fundamentals common to both satellites and submarines
- Institutional memory for design and operations cannot be taken for granted
- See SOSTC website for full presentation

## Jim Van Laak - Lessons Learned Processes for Human Space Flight Programs
LL marginally effective tool for NASA (collected, celebrated, buried)
Exploration Vision represents a new era of opportunity, responsibility, and vulnerability
NASA needs to do a good job - opportunity will not come again
Innovation is facilitated by a solid LL process that is frequently and thoroughly exercised
Why such a big deal? High stakes, arcane and fragile systems, optimized and operated within small margins
LL for Mars exploration will challenge all learning in every field; nothing can be taken for granted
Margins get vanishingly small; risks become increasingly ubiquitous, innocuous, and catastrophic
Why so hard?
Information management - memory, communication, understanding - international, interagency, inter-program
Designs, procedures, processes evolve (both positively and negatively) through a continuous and insidious process
Intuitively minor issues can rapidly become catastrophic
Data, data everywhere, and not a Truth to see
Want to look forward, not backward - but, reliability of new systems hard to predict
What can be done?
Be professional
Resist popular mantras
Seek first-principals understanding - what worked and why, what didn’t work and why
Learn to assimilate and synthesize higher-level truths - distill from past LL
Exercise risk management - use LL to identify risks and mitigation strategies
Everyone acknowledges the value of LL; almost nobody uses it well; the biggest challenges are human-based
See SOSTC website for full presentation

- LL marginally effective tool for NASA (collected, celebrated, buried)
- Exploration Vision represents a new era of opportunity, responsibility, and vulnerability
- NASA needs to do a good job - opportunity will not come again
- Innovation is facilitated by a solid LL process that is frequently and thoroughly exercised
- Why such a big deal? High stakes, arcane and fragile systems, optimized and operated within small margins
- LL for Mars exploration will challenge _all_ learning in _every_ field; nothing can be taken for granted
- Margins get vanishingly small; risks become increasingly ubiquitous, innocuous, and catastrophic
- Why so hard?
- Information management - memory, communication, understanding - international, interagency, inter-program
- Designs, procedures, processes evolve (both positively and negatively) through a continuous and insidious process
- Intuitively minor issues can rapidly become catastrophic
- Data, data everywhere, and not a Truth to see
- Want to look forward, not backward - but, reliability of new systems hard to predict
- What can be done?
- Be professional
- Resist popular mantras
- Seek first-principals understanding - what worked *and why*, what didn’t work *and why*
- Learn to assimilate and synthesize higher-level truths - distill from past LL
- Exercise risk management - use LL to identify risks and mitigation strategies
- Everyone acknowledges the value of LL; almost nobody uses it well; the biggest challenges are human-based
- See SOSTC website for full presentation