# Presentation: 197988

## Engineering Excellence Initiative

- November, 2000
- Presented by: Orlando Figueroa
- Deputy Chief Engineer for Systems Engineering

## Agenda

**Introduction - Engineering Excellence**
**Systems Engineering Working Group**
**Motivators and Problem to be Solved**
**Specific Needs and Available References**
**Rules of Engagement**
**Deliverables**
**Expected Benefits**
**Roles and Responsibilities**
**Schedule**
**Background - Systems Engineering Concepts**
**Summary - Engineering Excellence**

## Goal of Engineering Excellence

***Stimulate and enable the ******development and advancement of a sound engineering capability****** necessary for success in fulfilling the challenging and ambitious goals of the NASA Enterprises***
- Ensure ***continuous improvement*** of the NASA engineering workforce ***through relevant education, training, and work experiences.***
- Ensure ***sound and effective*** discipline and systems ***engineering****.*
- Coordinate with OAST to ensure the development and implementation of the ***Advanced Engineering Environment (AEE)*** to further enable the achievement of Enterprise goals in the** *****Faster, Better, Cheaper***** **environment.** **
- Provide ***value-added cross-Enterprise products and services*** that enable the infusion of technology, knowledge, and capabilities to support innovation in engineering and push the state of the art.
- Increase participation, membership, and ***leadership in recognized*** national and international ***engineering organizations.***

## NASA Integrated Action Team

- Two Important Dimensions Stressed by the NIAT Report Regarding the Revitalization of the
- Engineering Capability

**NASA Integrated Action Team**

- Development and sustainability of a knowledgeable and skilled workforce
- Development, integration and operation of complex space systems and aerospace technologies.

## Development and Advancement of a Sound Engineering Capability

## Engineering Management Council

- Chartered Systems Engineering Working Group to develop, document, and help establish a common framework for the engineering of systems in NASA.
- Common framework for systems engineering.
- Methodology for capability assessments.
- Basis for continuous measurable improvement.

## Motivators 

*As we strive for greater integration, consistency, and sharing of expertise among NASA Centers, industry and academia in collaborative environments, ... ** the Agency could benefit from an appropriate Agency-wide framework for **systems engineering processes. (NIAT-3)*

**Motivators**** **

- Mr. Goldin’s charge to OCE, Center Directors and EMC for greater emphasis on systems engineering
- Recent indicators:
- -	Mars failure reports.
- -	Spear Report on Faster, Better, Cheaper.
- -	IG and other reports
- Code AE Engineering Excellence Initiative
- -	Stimulate and enable the development and advancement of a sound systems engineering capability.
- -	Support development of an Advanced Engineering Environment.

## Problem 

*There is no endorsed Agency-wide guideline with a focus on the technical and engineering aspects of NASA programs and projects in support of 7120.5*

- Agency-wide framework does not exist to guide:
- The technical production of aerospace products and capabilities.
- Oversight of the engineering efforts.
- Education and training

**Problem**** **

## Specific Needs 

- Need: Consistency in basic approach to systems engineering
- 	Currently each Center has its own internal processes for systems engineering.
- Need:	Common framework of recognized best practices that guides the systems engineering of aerospace program and project products and capabilities.
- 	Lack document to support NPG 7120.5 in systems engineering
- Need:	Common systems engineering terminology and  definitions to enhance communication and collaboration among engineering teams across the Agency and with external partners and customers.
- 	Variations in terminology and definitions creates confusion, misunderstanding, and overhead.
- Need: Basis for assessing and continuously improving systems engineering capabilities.
- 	Lack means to establish baseline and guide effective process improvement.

**Specific Needs**** **

## Currently Available 

**Currently Available**** **

- Handbooks
- -	NASA Systems Engineering Handbook (SP 6105) 1995.
- -	NASA Systems Engineering Process for Programs and Projects 		(JSC 49040) 1994.
- -	System Engineering Handbook (MSFC-HDBK1912A) 1994.
- Tools
- -	 SE tools are available, but no common basis for tools is available
- -	Growing diversity of tools and their inability to communicate cited by NAE as major barrier to achieving Advanced Engineering Environment.
- Training
- -	Systems Engineering course for program and project managers is offered as part of the Academy of Program and Project Leadership.
- -	Systems engineering course is offered as part of the NASA Engineering training curriculum (recently revised).

## Rules of Engagement 

**Rules of Engagement**** **

- Useable Agencywide
- Allow flexibility to innovate and be tailored to unique project needs while retaining adequate rigor and traceability
- Establish links between Systems Engineering Framework and NPG 7120.5 processes and requirements to ensure complementarity and consistency
- Don’t start from scratch!
- -	Select appropriate concepts and processes from established national and international standards on systems engineering.
- -	Incorporate Center internal best practices where suitable for use Agency-wide.
- -	Utilize publicly available and accepted assessment models to establish tools for self assessment of capability consistent framework

## Deliverables 

**Deliverables**** **

- A common framework for systems engineering processes and guidelines
- Models for assessing systems engineering capability.
- Recommendation for Systems Engineering training modules

## Expected Benefits 

**Expected Benefits**** **

- Enable and foster excellence in systems engineering capabilities to:
- -	Formulate feasible program and project concepts.
- -	Deliver required products and services to NASA customers.
- -	Make timely acquisition of enabling products and critical technologies.
- -    Reduce risk in system development and deployment
- Enable and foster effective communications and collaboration within NASA and with external partners and customers.
- Enable effective assessment and improvement of systems engineering capabilities.

## Roles and Responsibilities

**Systems Engineering Working Group (SEWG)**
- -	Plan, develop, and execute the project.
- -	Coordinate project products within members’ Centers
**Engineering Management Council (EMC) **
- -	Provide project oversight and approvals.
**Enterprises **
- -	Verify suitability for accomplishing Enterprise missions.
**Centers**
- -	Provide members of the SEWG.
- -	Provide needed support for reviews, beta tests, and assessments.
- -	Verify suitability for accomplishing Center programs and projects.
**Customers** (NASA engineering community, advanced technology teams, payload developers, the Office of the Chief Engineer, Enterprise management, Center management, program and project managers, external partners, the training organization, and assessment personnel)
- -	Product reviews.
**Other Stakeholders** (PMC, PMCWG, SMOs, Software Working Group, ISE)
- -	Liaison and review of products.

## Systems Engineering Working Group Preliminary Schedule

## Systems Engineering and Standards Concepts

## Evolution of System Standards

**Mil-Std**
**499**

**Mil-Std**
**499A**

**Mil-Std**
**499B**

**EIA/IS**
**632**

**IEEE**
**1220**

**ANSI/EIA**
**632**

**IEEE**
**1220**

**ISO/IEC**
**15288**

**SE CMMI**
**SE iCMM**
**ISO/IEC**
**15504**

- Trial Use

**     ****1969              1974          1991-1993          1994             1998           Emerging   **

_**Timeline**_

**EIA/IS**
**731**

- Capability
- Assessment

- Capability
- Assessment

- Interim

- Not
- Released

- Interim

## Scope of System Standards

**MIL-STD-499B **
***Systems Engineering***** **

**ISO/IEC 15288**
***System Life Cycle Processes***

**ANSI/EIA 632 **
***Processes for Engineering Systems***** **

**IEEE 1220 *****Application & Management of the SE Process***

**Level of Detail**

**Breadth of Scope**

## System Standards

- 2.	Usually reflect perceived good practices.
- Sometimes behind state of the art.
- Sometimes lead the state of the art.
- 3.	Useful for benchmarking.
- 4.	Intended to be down-scoped to an organization and tailored for projects.
- 5.	Include:
- Project-centric application context.
- Basic concepts—system, lifecycles, IPTs.
- Processes with full lifecycle perspective and purpose, outcomes/objectives, activities and tasks.

**System Standards**

## Building Block Life Cycle Perspective

***Span of Interest***

***Perform***

***Operational***
***Functions ***

***Examples***

***Perform***

- Not necessarily under project control
- Acquire enabling products “just-in-time” to support System of Interest

- Need to design operational “ilities” into lifecycle products

- GSE
- Major Test Facilities
- Launch
- Operations
- Logistics

*Building Block Life Cycle Perspective*

## Every System Exists In The Context of a Broader System

- A System is a set of interrelated components, which interact internally and externally in an organized fashion toward a common purpose

## Engineering Life Cycle

*Systems are engineered from the top down and bottoms up*

**Pre-system**
**definition **

**System**
**definition **

**Subsystem**
**design **

**Detailed**
**design **

**Integration &**
**test and evaluation**

**Review**

**Review**

**Review**

**Review**

**Decision**
**Milestone**
**Exit**
**Review**

**Build, buy,**
**reuse &**
**T&E**

**System**
**Integration &**
**test and evaluation**

**All**
**Subsystems**

**All**
**Subsystems**

**Notes:**
**Bottom-up products may be analytical models, simulations, or physical products**
**Product may be HW, SW, Humanware, facilities, services**

## Life Cycles

- -	Stages every system progresses through.
- -	Lust to dust view:  conception, definition, realization, operation, disposition.
- 2.	Enterprise Decision-based Life Cycle
- -	Control of system progress though management reviews at established decision milestones.
- -	Incremental, phased approach for risk mitigation.
- -	Example - Old and New (7120.5) NASA acquisition life cycle.
- 3.	Engineering Life Cycle
- -	Used as often as necessary during each decision-based life cycle phase to satisfy decision milestone review entry criteria.
- -	Develop and realize products and services consistent with decision-based phase objectives.

**Life Cycles**

## Using the Engineering Life Cycle

**Approval**

**Evaluation**

**Formulation**

**Implementation**

**Advocacy**

***Investigation***
***Concepts &***
***Enabling***
***Technology***
***Investments***

***System***
***Definition***
***& Design***
***Options***

***Pre-Approval***
***System***
***Design***

***Post-Approval***
***System***
***Design***

***Detailed***
***Sub-System***
***Design***

***Pre-Deployment***
***Fabrication,***
***System I&T***

**NPG 7120.5 PAPAC Process **
**(Enterprise decision-based life cycle)**

**Using the Engineering Life Cycle**

**Reviews for**
**Confirmation of**
**Decisions**

## Framework for Engineering Excellence

- Consistency in systems engineering approach at all levels
- Culture based on rapid prototyping, modeling and simulation capability for development of multiple and diverse designs
- Experienced, well trained engineers in application of process, tools, methodology, and customer relationship/interaction
- Continuous improvement through self assessment at the personal and organizational level

## Survey Information

## Survey Results 

**  ****# of Yes****	****Implication****	****CMMI SE**
**Responses ****	****	****    Level**

- 16
- 10 - 15
- 5 - 9
- < 5

**World Class**
**Improved Chance **
**of Success**
**Lighthouse Living**
**Proactive**
**Firehouse Living**
**Reactive**
**Ad Hoc & Chaotic**
**Rely on Heroes**

- 4 - 5
**Quantitatively**
**Managed/**
**Optimized**
- 3
**Defined**
- 1-2
**Performed/**
**Managed**
- 0
**Incomplete**

## Appendix

- Back Up Charts

## A Capability Survey

- Circle your responses Yes (Y) or No (N) to each of the 16 statements below.
**Base your response on your perception of how systems are engineered within your current project/program or organization, or base your response on prior experience:**
- Y  or  N 	1.	Has a documented and disciplined top-down integrated product 				development process?
- Y  or  N	2.	Consistently meets project performance, schedule, and cost 				requirements?
- Y  or  N	3.	Relies on integrated teamwork rather than individual talent or 				heroic effort to meet product expectations?
- Y  or  N	4.	Meets management priorities such as schedule and cost without 				sacrificing quality standards or practices.
- Y  or  N	5.	Rarely understates the level of technical risk in a project?
- Y  or  N	6.	Develops hardware and software for a system as one integrated 				effort?
- Y  or  N	7.	Utilizes meaningful product metrics for measuring technical 				progress _and_ process metrics for measuring efficiency of doing 				planned work?
- Y  or  N	8.	Has an organization-wide culture that supports integrated 				multidisciplinary teamwork?

## A Capability Survey (Continued)

- Circle your responses Yes (Y) or No (N) to each of the 16 statements below.
**Base your response on your perception of how systems are engineered within your current project/program or organization, or base your response on prior experience:**
- Y  or  N	9.	Has a competitive product development cycle time?
- Y  or  N	10.	Consistently delivers products _and_ services that meet 					stakeholder requirements?
- Y  or  N	11.	Has effective controls for engineering a system to include: 				requirements management, opportunity management, technical 				risk management, configuration management, change 				management, performance based progress measurement (TPM, 			Earned Value, Technical Reviews and Audits), interface 				management, and data management?
- Y  or  N	12.	Provides continuing education and training in the processes for 				engineering systems for all members of the professional staff (to 			include team building training for teams _after_ they are formed)?
- Y  or  N	13.	Maintains a shared database (electronically based) with 				information available to appropriate development teams and 				interdisciplinary team members?

## A Capability Survey (Continued)

- Circle your responses Yes (Y) or No (N) to each of the 16 statements below.
**Base your response on your perception of how systems are engineered within your current project/program or organization, or base your response on prior experience:**
- Y  or  N	14.	Supports the practice of sustained (continuous) measurable 				improvement in all areas of the organization?
- Y  or  N	15.	Invests the necessary resources to develop, procure and 				implement the processes, methods, and automated tools for 				concurrent development of end products and their associated 				enabling products for production, test operations, and logistics 				(including deployment, training, support/maintenance, and 				disposal?
- Y  or  N	16.	Has an appropriate rewards and recognition system for 				encouraging interdisciplinary teamwork as well as individual 				excellence?
- Upon completion count the total number of Yes responses and enter that total in the space below.
- Total Yes responses: ___________.
- Copyright © 1999, SMi

## Assessment Model - CMMI SE

**0**

**1 Performed**

**Achieve**
**	****Implementation Goals**

**Perform Process Activities**

**2 Managed**

**Managed Process **
**	****Plan**
**Managed Process**
**	****Enactment**

**Organizational Policy**
**Process Plan**
**Resources**
**Responsibility**
**Training**
**Perform Managed Process**
**Measurement of Process**
**Process Monitor & Control**
**Manage Configurations**
**Objective Verification**
**Management Review**

**Level**

**Generic Goals**

**Generic Practices**

**Incomplete**

**Assessment Model - CMMI SE**

## Assessment Model - CMMI SE

**Level**

**Generic Goals**

**Generic Practices**

**3 Defined**

**4 Quant-**
**	****itatively**
**	****Managed**

**Tailored**
**	****Process**
**Process **
**	****Improvement**
**	****& Learning**

**Standard Set of Processes**
**Organizational Infrastructure**
**Defined Descriptions**
**Performed Defined Process**
**Work Product Inspection**
**Improvement Measurements**

**5 Optimizing**

**Understand**
**	****Quality,**
**	****Performance,**
**	****& Capability**

**Defined Quality Goals**
**Quantitatively Managed**
**Performed**
**Measured Process Capability**

**Technology**
**	****Improvements**
**Capability**
**	****Improvements**

**Improvement Goals**
**Perform Optimized Process**
**Make Improvements**
**Address Problem Causes**
**Deploy Improvements**
**Measurements**

**Assessment Model - CMMI SE**

## Systems Engineering Proposed  Definition for NASA

*Systems engineering*  is an interdisciplinary approach to the realization of successful systems.  It is the integration of related technical parameters ensuring compatibility of all physical, functional, and program interfaces in a manner that optimizes the total system to meet its goal within resource constraints.
- Specifically, systems engineering is an iterative process consisting of identification and quantification of system goals, creation of alternative system design concepts, performance of design trades, recommendation of the best design based on trade results, implementation of the selected design, verification that the design is properly built and integrated, and post-implementation assessment of how well the system meets (or met) the goals.