Enhancing Mission Success –

A Framework for the Future

A Report by the

NASA Chief Engineer

and the

NASA Integrated Action Team

December 21, 2000
Enhancing Mission Success – A Framework for the Future

ii

A Report by the

NASA Chief Engineer

and the

NASA Integrated Action Team

Enhancing Mission Success – A Framework for the Future
Enhancing Mission Success – A Framework for the Future

iii

Table of Contents

Executive Overview .................................................................................................... 1

Introduction ..................................................................................................... 1

NASA Integrated Action Team (NIAT) Charter and Approach......................... 1

Approach to Readiness of Recent Missions ...................................................... 2

Results of NIAT Assessment ........................................................................... 4

General Context ......................................................................................... 4

Assessment of Faster, Better, Cheaper (FBC) ............................................. 5

Key Themes .............................................................................................. 7

Overview of Actions .................................................................................. 10

Implementation ............................................................................................... 10

Summation ...................................................................................................... 13

NIAT Action Descriptions .......................................................................................... 14

Introduction..................................................................................................... 14

Theme I—Developing and Supporting Exceptional People and Teams ............... 16

NIAT-1: Health and Safety .................................................................... 17

NIAT-2: Development of the Workforce ............................................... 19

NIAT-3: Revitalizing Engineering Capability ........................................ 25

Theme II—Delivering Advanced Technology .................................................... 28

NIAT-4: Balanced Technology Investment Strategy .............................. 30

NIAT-5: Integrated Technology Planning Process ................................. 33

NIAT-6: Technology Development and Infusion ................................... 36

Theme III—Understanding and Controlling Risk ............................................... 40

NIAT-7: Risk Identification, Assessment, and Management .................. 41

NIAT-8: Safety and Mission Assurance ................................................ 44

Theme IV—Ensuring Formulation Rigor and Implementation Discipline ........... 45

NIAT-9: Software Development and Assurance .................................... 46

NIAT-10: Integrated Review Process ...................................................... 48

NIAT-11: Ensuring Adequate Resources ................................................. 52

NIAT-12: Faster, Better, Cheaper ............................................................ 54

NIAT-13: Surveillance ............................................................................ 55

NIAT-14: Verification and Validation ..................................................... 57

NIAT-15: Management Responsibility and Accountability ...................... 58
Enhancing Mission Success – A Framework for the Future

iv

Table of Contents (Continued)

Theme V—Improving Communication .............................................................. 60

NIAT-16: Organizational Communication ............................................... 60

NIAT-17: Knowledge Management ........................................................ 62

Other Recommendations Assessed ..................................................................... 66

Conclusions ........................................................................................................ 67

Appendix A—NIAT Membership ...................................................................... 68

Appendix B—NIAT Actions Versus Report Recommendations ......................... 69

Appendix C—Acronyms..................................................................................... 82
Enhancing Mission Success – A Framework for the Future

1

Executive Overview

Introduction

In March 2000, NASA released a series of reports that were the product of activities chartered

by the Agency in response to failures in the Mars Program, Shuttle wiring problems, and a

generic assessment of NASA’s approach to executing “Faster, Better, Cheaper” projects. The

subject reports are:

• Mars Climate Orbiter (MCO) Mishap Investigation chaired, by Mr. Arthur Stephenson,

Director, Marshall Space Flight Center (MSFC),

• Mars Program Independent Assessment (MPIA), chaired by Mr. A. Thomas Young,

Lockheed Martin (retired),

• NASA Faster, Better, Cheaper (FBC) Task, chaired by Mr. Anthony Spear, Jet Propulsion

Laboratory Mars Pathfinder Project Manager (retired),

• Shuttle Independent Assessment (SIA), chaired by Dr. Henry McDonald, Director, Ames

Research Center (ARC).

Recommendations contained in the reports not only addressed root and contributing causes of

specific failures but also looked beyond those incidents to make broader recommendations to

the Agency on ways it might improve its general approach to executing programs and projects.

The Office of Space Science (OSS) and the Office of Space Flight (OSF) are completing

responses to program specific recommendations of the reports.

NASA Integrated Action Team (NIAT) Charter and Approach

In addition to the program-specific assessments, the NASA Administrator recognized a need to

assess and respond to findings and recommendations that could be more broadly applied to the

wide range of NASA programs and projects. The NASA Chief Engineer was given

responsibility for defining an integrated plan to address those recommendations and to

formulate proactive steps to address opportunities for improvement from an Agency

perspective.

To accomplish this assessment, in March 2000, the Office of the Chief Engineer chartered a

NASA-wide senior team, the NASA Integrated Action Team (NIAT), chaired by Carolyn

Griner, Deputy Director of the MSFC.

The team was comprised of 42 senior-level staff from NASA Headquarters and all Centers

with substantial awareness of Agency program and project management policy as well as

insight to the way in which such programs and projects are being implemented across the

Agency. The collective expertise of this team provided the ability to understand the

recommendations of the reports and to evolve appropriate actions needed to respond to them.
Enhancing Mission Success – A Framework for the Future

2

The NIAT team was chartered to:

• Assess the selected recommendations from the reports,

• Define a specific set of actions focusing on which improvements were of value, with

enough detail on the approach to guide the resultant implementation toward a desired

outcome,

• Identify organizations responsible and accountable for implementation of each action,

• Identify areas requiring further study or management review, and

• Integrate the actions into a logical and meaningful format.

The NIAT evaluated 165 recommendations from the four reports.

Given the high degree of correlation of the recommendations in the various reports, the NIAT

divided the recommendations into four key focus areas: People, Process, Process Execution,

and Advanced Tools and Technology. Actions were defined for each area and then integrated

into five implementation themes:

• Developing and Supporting Exceptional People and Teams

• Delivering Advanced Technology

• Understanding and Controlling Risk

• Ensuring Formulation Rigor and Implementation Discipline

• Improving Communication.

These themes are the tactical and strategic elements of response to the reports and are directly

focused on critical implementation issues.

Approach to Readiness of Recent Missions

Before discussing the NIAT results, it is appropriate to describe how NASA has approached

affirming readiness of missions launched subsequent to the Mars ’98 failures. Outside the

scope of the NIAT activity, the Agency has expended significant effort on a case-by-case basis

to ensure, through independent comprehensive assessment, that residual risks associated with

such missions were fully understood prior to launch. Center Directors and the respective

Enterprises have redoubled efforts to ensure that sound practices were applied, that risks were

fully understood and to ensure that constraints associated with cost, schedule, and scope of the

effort had not combined in a way that may have compromised the probability of mission

success.

The process for ensuring such has been rigorous, well documented, and effectively executed.

For missions launched on Expendable Launch Vehicles (ELV) following the Mars mission

failures, the Directors of the managing Centers in consultation with the Enterprise Associate

Administrators have, for each mission, chartered a group of respected experts, referred to as

“Red Teams,” from outside of that Center. They are charged with evaluating the approaches

used to identify, evaluate, and eliminate or control risks during the design, development,

integration, and test activities, in order to understand the process used and, if necessary, to
Enhancing Mission Success – A Framework for the Future

3

prescribe additional efforts needed to verify readiness for launch with high confidence of

mission success. In some cases, the results of these “Red Team” evaluations have mandated

additional analyses and tests to further mitigate risks. In all cases to date, these additional

efforts have been fully completed prior to a successful launch. The Agency will continue this

practice, as necessary, for future missions. In addition to this expert review, all ELV missions

are subjected to an internal Certification of Flight Readiness (CoFR) review process to ensure

the total vehicle readiness for flight.

For missions associated with human space flight endeavors, the Space Shuttle program has

had, since its beginning, a rigorous CoFR process for each mission. In response to the findings

of the Rodgers Commission following the Challenger accident, this process was significantly

strengthened to provide assurance that risks are understood and accepted prior to each launch.

Safety considerations have not and will not be compromised on human space missions. In

addition to this rigorous approach for each mission, when problems of a general nature arise,

independent assessments by experts from outside the program are conducted followed by

implementation of comprehensive action plans in response to all recommendations. An

example of this approach is the review chaired by Dr. Henry McDonald in response to

problems associated with the Shuttle electrical wiring.

To meet the tremendous challenge of assembling the International Space Station (ISS),

independent readiness reviews, similar to “Red Teams”, are being held for each flight. The

objective of the review is to ensure that assembly missions are safely executed and that the

long-term operability requirements for the ISS are met for each flight. The review team

assesses operability and capability issues by examining the interdependencies across upcoming

flights and mitigation plans designed to address identified shortfalls. The expected outcome is

the identification and understanding of residual risks that exists prior to launch and the

expected effectiveness of risk mitigation activities. To strengthen this effort, the team also

draws information from other reviews such as the CoFR, also used by the ISS, and the

independent Aerospace Safety Advisory Panel (ASAP).

Prior to each launch, both the Space Shuttle and ISS programs conduct Flight Readiness

Reviews (FRR) to determine the readiness of the vehicle, flight crew, mission operations, and

payloads. Responsible organizations certify the completion of all tasks, open paper, and

planned work required to prepare the flight and ground hardware and software, support

facilities, and operations personnel to safely and successfully support a launch and mission.

Readiness for launch is determined through the Senior Management review of necessary data

to ensure satisfactory closeout of all FRR certification requirements, exceptions, and launch

constraints in sufficient detail to provide management with the information needed to certify

launch. The endorsement documentation required at the completion of the FRR becomes the

CoFR for launch.

These proactive management, technical, and operational practices have been used to provide

high confidence that risks have been appropriately mitigated for recently launched missions.

They will continue, as appropriate, until the steps identified later in this report, are fully and

effectively deployed. In the time since the failures of the Mars ’98 missions until the issuance

of this report, all NASA missions have fully met success objectives.
Enhancing Mission Success – A Framework for the Future

4

Results of NIAT Assessment

General Context

The result of the NIAT assessment presents a framework for strengthening the approach used

by NASA to formulate and implement its programs and projects and to improve the supportive

nature of the environment in which they are executed. It represents an adjustment for the

future not a return to the past. Implementation of the NIAT actions provides the opportunity to

ensure application of best practices across the entire spectrum of Agency programs and

projects.

The stimulus for this assessment was principally provided by the failures of the multiple

missions associated with the Mars ’98 program, and the various investigative reports that

followed. The use of these incidents by NASA to trigger a reevaluation of its practices reflects

a dedication to continual improvement through assessment of the lessons learned from the

many recent successes achieved, as well as those from the disappointing failures encountered

by the Mars ‘98 missions.

NASA has celebrated numerous successes over the past decade, many of which served as the

vehicle for innovation in management and engineering practices, aggressive technology

development, and acceptance of prudent risk taking when the potential for significant payoffs

warranted it. Greater than 97 percent of the budgetary investment in flight missions has

resulted in successful outcomes. This highlights the fact that NASA’s general approach to

program and project management is fundamentally sound. However, the problems identified

by the reports remind us that there is room for improvement and that, although the potential for

failure will always be present, we must remain ever vigilant to minimize failures that are

preventable.

As we move forward, acceptance of prudent mission risk that does not compromise safety

considerations must remain the hallmark of our approach. This approach can enhance

performance that achieves challenging mission objectives while vigorously pursuing cost and

schedule improvements. Determination of acceptable risk must result from an objective

assessment on a case-by-case basis by the team in concert with their customers, and with clear

and consistent communication to and acceptance by management and stakeholders. The

degree of acceptable risk will vary greatly as a function of mission-specific considerations such

as complexity, cost, national need, etc. The mission risk profile and the balance of scope and

resources must be continuously evaluated; budgetary reserves and design margins must be

sufficient to support the risk posture. Additional funding is not the only alternative for

maintaining balance. Changes in scope, provided that sufficiently viable mission objectives

remain, must be considered. Sometimes, other alternatives, perhaps even cancellation, will be

the proper solution. Obviously, a human space flight endeavor contrasts greatly in many of

these considerations with a small, single-instrument science mission. Nonetheless, the

underlying approach of defining and agreeing to acceptable risk remains the same for each
Enhancing Mission Success – A Framework for the Future

5

mission, and departure from good planning and sound practice cannot be viewed as acceptable

risk.

The NIAT actions represent a systems solution to continually improve NASA’s ability to

effectively execute its programs and projects. This involves a comprehensive set of practices

that focus on the objectives of well prepared people, sound decisionmaking, and effective

communications. As we move toward the future we must recognize that the practices by which

we achieve those objectives are likely to be dramatically affected by emerging paradigms that

will be enabled by technology developments. Currently we are only at the threshold of

understanding how considerations associated with the bio-, nano- and info- technology triangle

may revolutionize our ability to design, develop, validate, operate, and maintain systems in

ways that are safer and more reliable while being able to autonomously adapt to unanticipated

events. We must be open to and even actively pursue capabilities offered by innovative

applications of such technology in order to more effectively guide programs and projects

toward successful outcomes.

Assessment of Faster, Better, Cheaper

As first introduced in NASA in 1992, FBC reflected a management approach that intended to

stimulate innovative development and application of technology, to streamline policies and

practices, and to energize and challenge a workforce to continue to safely and successfully

undertake bold new missions in an era of diminishing resources. It emphasized the following:

• Distribution of risk by moving from single high-cost, long-development time missions to

multiple low-cost, shorter development time missions, when compatible with objectives,

• Accountability and responsibility for success that is clearly placed with the implementing

teams at NASA’s Centers, as well as within industry and academia,

• Efficiency in process and methodology, and exploitation of new yet mature technology to

enable and enhance new and challenging science and technology programs and projects

consistent with short development cycles.

The above principles were overarching, and were intended to stimulate NASA to strive to be

“Faster, Better, Cheaper”. In some instances, the principles were applied well and resulted in

mission success. Programs such as Explorers, Discovery, Earth Science System Pathfinder,

and New Millennium, were reinvented to capitalize on smaller, less costly, and technologically

challenging approaches to achieve the scientific and technological objectives. In other

instances, notably Mars ’98, Lewis, and Clark, attempts to apply these principles resulted in

mission failure.

FBC promoted prudent risk taking to push the technical and programmatic boundaries.

Process constraints were minimally controlled to stimulate innovation. Resources were highly

constrained and guidance on the boundaries of innovation and risk taking was lacking, thereby

engendering a variety of approaches to adapting to this new paradigm.
Enhancing Mission Success – A Framework for the Future

6

At the same time, other changes occurred that exacerbated the potential difficulties associated

with the adaptation.

The FBC emphasis itself resulted in a threefold increase in the number of programs and

projects, particularly in small science payloads. This required more project managers and

systems engineers. During this same period, NASA reduced its civil service workforce by 24

percent from Fiscal Year (FY) 1993 through FY 1999, causing both a loss in corporate

knowledge and a substantially increased workload on the remaining employees. Losses also

occurred within the vitally important aerospace industry workforce. Technical complexity,

primarily in software utilization was increasing as well. These changes in practice, skills, and

knowledge of the workforce, coupled with the demand for innovation in aerospace science and

technology, particularly the revolution in information technologies, presented a tremendous

challenge to NASA.

Despite these challenges, NASA’s overall mission success rate in the 1990’s remained

impressively high as previously stated. However, the success rate for missions clearly

associated with exploring the boundaries stimulated by the FBC approach was approximately

two out of three. This result and the desire to evaluate the approach being used and to

understand best practices and lessons learned had already engendered the effort by Anthony

Spear. When the Mars ’98 failures occurred, other failure investigations and more formalized

assessment of Agency practices resulted.

The reports reinforce the validity of properly applied principles of the FBC philosophy. They,

of course, criticize the way in which those principles were applied to the Mars ’98 projects and

point out that the dramatically disappointing results provide a case study that must be fully

evaluated for lessons learned regarding current practices and approaches in order to modify

them where necessary.

The Mars ’98 missions were over-constrained by the simultaneous existence of fixed resources,

schedule, and science requirements that were not rebalanced. This forced the acceptance of

significant risk that manifested itself as a lack of rigorous attention to sound process and

practices. The combination of these factors was a significant contributor to the mission

failures.

It is recognized that Agency guidance associated with the application of FBC principles to

actual situations was not sufficiently articulated. As a result, the specific actions delineated in

this report focus on how NASA must approach execution of all programs and projects because

the underlying principles of FBC, when properly applied, have applicability to all that the

Agency does. The governing process by which the Agency guides execution of its programs

and projects does not currently differentiate projects that are FBC and those that are not.

Instead, it relies upon a careful assessment on a case by case basis to establish the risk posture

associated with a particular mission or endeavor. NASA’s work is and will continue to be

inherently high risk. Therefore, the goal is to strive for the reduction of residual risk to that

inherent in the challenge of the science or technology mission without compromising safety

considerations. In this light, we do not see a need for differentiation of FBC projects. Rather,
Enhancing Mission Success – A Framework for the Future

7

we must ensure that there is adequate guidance for decisionmaking and risk management for

all projects.

Note that there is a high degree of correlation between the underlying principles of FBC and

the independently developed five themes. Individual competency, team functionality,

utilization of technology, prudent risk taking, rigor of practice, and management awareness and

consent are all key considerations in getting the job done and are all elements of both FBC and

the five themes. Through the actions of this report, the Agency will improve its approach to

applying the principles of FBC with safety and prudent acceptance of mission risk as key

criteria.

Key Themes

The NIAT themes reflect a framework to safely and successfully execute our missions. To that

end, the actions delineated in this report represent a “required state” for formulation and

implementation of NASA programs and projects. Key aspects of each theme are discussed

below.

Theme I—Developing and Supporting Exceptional People and Teams

NASA has a vision of being a learning organization implemented by continual development of

individuals and teams through experience and training in a supportive environment. The

development and proactive support of our people is essential to the sustainability of our

excellent capability. At the core of this is challenging work that provides opportunity to

develop relevant skills, adequate training, and a safe and healthy work environment. The

success of NASA depends on having a knowledgeable and skilled workforce, supported by

clearly understood processes and methodologies, and armed with tools that leverage emerging

technology to simplify and improve design, development, and verification related engineering

approaches. Most importantly, however, the workload on individuals must be managed to

enable their success.

The actions associated with this theme recognize that meaningful and relevant hands-on work

is the primary way to develop skills and that a well-structured training program is essential

support to those on-the-job-development experiences. They also recognize the importance of a

revitalized engineering capability with emphasis on systems engineering, and finally, they

recognize the importance of laying the groundwork to facilitate balance of the capabilities of

the workforce with the demands of the workload.

Theme II—Delivering Advanced Technology

The imperative for new technology that both meets the needs of current missions and enables

new, yet unforeseen, missions is required to accomplish NASA’s ambitious Strategic Plan and

long term vision. This vision requires a successful and responsive technology pipeline, and
Enhancing Mission Success – A Framework for the Future

8

aggressive pursuit of emerging paradigms in science and technology to solve the engineering

challenges embodied in the vision. Delivering advanced technology has three codependent

elements. First, a balanced technology investment strategy to maintain a pipeline of

demonstrated technologies that meets the needs of near- to mid-term missions, as well as the

revolutionary capabilities needed to enable new classes of missions for the 21st century.

Second, that strategy must be developed and implemented through a well-defined planning

process that identifies new technologies and emerging paradigms, defines opportunities, and

ensures the efficient transition of new technologies into missions. Last, the technology life

cycle, from basic research through application, must be done in a logical and thorough manner

so that it meets the performance requirements and is easily infused into the programs and

projects. Dramatically shorter mission cycles make it imperative that NASA finds ways to

accelerate the introduction of new technologies into missions. Processes for managing the full

life cycle of technology development, maturation, and infusion must enable new levels of

performance and capability. Overlaying all these elements is the issue of stable and adequate

funding that enables a healthy technology pipeline, and the pursuit of “out of the box”

engineering approaches.

Theme III—Understanding and Controlling Risk

NASA is an organization charged with constantly pushing the envelope in science and

technology. By its very nature, this work is unique to every mission and has inherent risks.

The approach must not be risk averse. Prudent risk associated with the application of advanced

technology to enhance performance that enables achievement of challenging science must be

recognized and encouraged. NASA has always recognized risk management as a key factor in

project management. However, the right risks must be identified, assessed, tracked, and

continuously managed by the project team, and must be communicated and agreed to by

management, customers, and stakeholders. Existing tools must be fully used to support this

effort. In the future better tools, methodologies, and techniques, many of which utilize

emerging paradigms enabled by intelligent systems that draw on soft adaptive computing will

permit more comprehensive and objective containment and management of risk. The objective

is to better ensure balance between the scope of the effort and the resources allocated for the

job. Resource reserves and design margins must be adequate to support the risk posture.

While balance must be maintained, this does not imply that adding money is the only way to

maintain appropriate balance. A clear definition of “Acceptable Risk” and project “Success

Criteria” is essential to effective life cycle risk management.

Primary responsibility for effective management of risk rests squarely with the program and

project manager. All members of the team must fully understand their role in identifying and

controlling risk. With the increased emphasis on analysis of risk, NASA’s Safety and Mission

Assurance (SMA) community takes on an increasingly important role in supporting the

program and project managers. SMA provides guidance during the formulation phase for the

identification of risks and development of the risk management plan. During implementation,

SMA provides independent assessment to ensure adequate attention to continued risk

management. The SMA function at NASA must be enhanced through better preparation of

people to support value added participation in appropriate activities of projects.
Enhancing Mission Success – A Framework for the Future

9

Theme IV—Ensuring Formulation Rigor and Implementation Discipline

NASA has developed new policies, processes, and practices for program/project management

in keeping with the principles of FBC. NASA must ensure that these are understood at all

levels of the organization and that they are applied with discipline throughout the life cycle of

programs and projects.

To promote effective execution by the project team, the actions provide more detailed guidance

in the use of the project management process. A comprehensive well-integrated review

program, a risk-based supplier surveillance strategy, and a comprehensive verification and

validation program are essential. The actions also provide for more direct engagement of the

experience and skills of institutional management and independent experts who can help

surface problems and determine solutions as early as possible.

To enable excellence in project management, excellence in engineering practice is a

prerequisite. Improvement of software development processes and standards are a major new

Agencywide thrust that will ensure that NASA operates at a level commensurate with the

demand of increasingly complex software and its interaction with hardware.

The actions are intentionally not overly prescriptive, however. The ability to innovate and

tailor approaches to unique needs must remain an essential part of the team’s ability to meet its

responsibilities and be accountable for its results. Successful performance of the project team

remains the critical aspect of achieving mission success. The assurance of adequate rigor must

be retained, and changes of practice must be consciously considered, have a reasonable

confidence of success, and have management consent. The result of this cultural

transformation will be a workforce with a common lexicon, practices that are second nature,

and an environment in which creativity and innovation can flourish to foster proactive

management of NASA’s programs and projects.

Theme V—Improving Communication

A recurring theme throughout the recommendations was the need to improve communication

at all levels. Failures in communication are an endemic problem that constantly threatens

organizations, and particularly large complex organizations like NASA. NIAT actions focus

on two aspects of communication: a renewed emphasis on the improvement of organizational

communication, and the capture and management of knowledge through information

technology to make critical information readily available.

Efforts to improve communications and the tools that support it must have priority in the

implementation of the actions. Without this emphasis, the understanding of the improvements

will be lost and an opportunity will go unanswered. An organization and management team

that communicates in an open environment and rewards the achievements resulting from that

communication will set the path of success for others to follow.
Enhancing Mission Success – A Framework for the Future

10

Overview of Actions

The specific actions associated with each theme are given in Table 1.

Some actions are characterized as “Major New Improvements”. These represent new

initiatives requiring additional resources and planning. These are long-term and developmental

in nature; such as, tools and approaches beyond current practice. As they mature and become

fully deployed, the results of these initiatives will penetrate all levels of the organization, and

thereby influence the manner in which programs and projects are executed. No immediate

action is required by current programs and projects, except that portion of NIAT-9 associated

with the need to evaluate and, when appropriate, conduct Independent Verification and

Validation (IV&V) of critical mission software. Action related to such IV&V considerations

has been underway well in advance of issuance of this report.

 “Major Improvements Previously in Work” represent initiatives that were started prior to

NIAT or shortly thereafter, based on early NIAT results. Their development does not require

significant new resources beyond that which had already been planned. In the case of those

initiatives started prior to NIAT, the NIAT action either validated the initial scope or provided

refocus to ensure that identified shortcomings were being addressed. These initiatives are

considered major because they address critically important improvements to the Agency

approach to executing its programs and projects. The Health and Safety action (NIAT-1) and

those associated with Delivering Advanced Technology (NIAT-4, 5, & 6) are long term in

nature and do not require any immediate action by current programs and projects. Those

associated with Understanding and Controlling Risk (NIAT-7 & 8) require evaluation by

current projects for compliance with applicable elements of the actions. If needed, adjustments

to current practice on those programs may be required.

Actions in the “Clarification and Verification” category represent straightforward actions that

expand upon or modify existing Agency policy, requirements, and guidance or develop new

ones to address identified shortcomings. This report provides the basis for those changes. The

formal policy and requirements aspects of these actions will be effected in the near term. In

addition, current programs and projects must evaluate their current practices for compliance

with applicable elements of these actions and, if needed, adjust current practice.

The result of these actions will be to move NASA to the “required state,” characterized by the

right policies and practices, the right people with the right skills, supported by the right

technologies.

Implementation

The Agency is firmly resolved to implement the actions contained in this report in a way that

has a positive and lasting effect. In the process, we will enhance both the application of sound

practice by project teams as they execute their activities and the supportive nature of the

infrastructure and management team that must be attentive to the needs of the project team in

order to enable them to be successful. Discussions at meetings of the Agency Senior
Enhancing Mission Success – A Framework for the Future

11

Table 1. Themes and Their Actions.

Actions

Major

New

Improvements

Major

Improvements

Previously

in Work

Clarification

and

Verification

Developing and Supporting Exceptional

People and Teams

NIAT-1 Health and Safety X

NIAT-2 Development of the Workforce X

NIAT-3 Revitalizing Engineering Capability X

Delivering Advanced Technology

NIAT-4 Balanced Technology Investment

Strategy

X

NIAT-5 Integrated Technology Planning

Process

X

NIAT-6 Technology Development and

Infusion

X

Understanding and Controlling Risk

NIAT-7 Risk Identification, Assessment, and

Management

X

NIAT-8 Safety and Mission Assurance X

Ensuring Formulation Rigor and

Implementation Discipline

NIAT-9 Software Development and

Assurance

X

NIAT-10 Integrated Review Process X

NIAT-11 Ensuring Adequate Resources X

NIAT-12 Faster, Better, Cheaper X

NIAT-13 Surveillance X

NIAT-14 Verification and Validation X

NIAT-15 Management Responsibility and

Accountability

X

Improving Communication

NIAT-16 Organizational Communication X

NIAT-17 Knowledge Management X
Enhancing Mission Success – A Framework for the Future

12

Management Council have emphasized the need for Senior Managers to commit to these

objectives.

To that end, the Agency will consciously ensure that appropriate attention is being given to

implementing and measuring the success of these actions through the following approach.

Overall responsibility for ensuring effective implementation of these actions throughout the

Agency rests with the Associate Deputy Administrator (ADA). As the Agency’s Chief

Operating Officer, the ADA chairs the activities of both the Program Management Council and

the Capitol Investment Council (CIC) that integrates all Agency budgetary requirements and

develops recommendations for the Administrator. In addition, he oversees the activity of the

Senior Management Council. He is, therefore, the appropriate person to see that

implementation of these actions is appropriately integrated into the budget plan, that they

proceed in a timely fashion, and that deployment plans and success metrics are defined. The

long-term and development nature of actions identified as “Major New Improvements” will be

phased in a manner consistent with the availability of resources identified by the CIC.

For each theme, a Headquarters office has been designated as a theme champion. These

champions are responsible for ensuring consistency across the action set within the theme.

These champions are the change agents that will guide the cultural change for improved

performance, not just the completion of individual actions.

For each specific action, a lead Headquarters organization has been clearly defined (see Table

2). In addition, supporting individuals or organizations, and a target completion date for the

essential elements of the action have been identified. The lead organization has the

responsibility to involve additional organizations in the disposition of the action as needed. In

addition, the lead organization must also identify intermediate milestones, define the plan for

deployment of the actions, identify the metrics by which the success of that deployment can be

measured, and serve as the advocate to the CIC for necessary funding.

The Office of the Chief Engineer will serve as the Agency’s focal point for tracking these

actions and reporting progress quarterly to the Agency’s Senior Management. Periodic reports

of progress also will be made to the NASA Advisory Council.

The responsibility for evaluating compliance of ongoing programs and projects against the

provisions of applicable actions rests with the respective Enterprise Associate Administrators

and Center Directors who, as a minimum, must evaluate the state of risk associated with such

decisions related to adequacy of current practice. That assessment should consider the level of

risk acceptable for the particular program or project, where the program or project is in its life

cycle and the ability and desirability of making changes. The resources required and the

benefit to the Agency are integral parts of determining and selecting specific solutions.
Enhancing Mission Success – A Framework for the Future

13

Summation

NASA is justifiably proud of its scientific and technological accomplishments, particularly

during the past decade with its significant budgetary and workforce reductions. Nonetheless,

NASA is a learning organization and continually strives to understand the present state of the

Agency in terms of its people, policies, procedures, and capabilities to learn what is needed to

improve in the future. To plan a path for improvement, it is critical to understand and

communicate the required state.

The people of NASA and its partners are the linchpins of our present and future success.

Challenging work, executed in a safe and productive environment, with manageable stress, by

people that are well prepared for and supported in their work, is essential.

Well-defined and executed formulation and implementation is required to deliver safe, quality

products that are responsive to the needs of the customer. Processes must be focused around

sound decisionmaking, driven by thorough understanding and control of risks. Open, effective

communication is essential to allow problems to be found early, at a time when the right people

can more easily become involved and when the resources needed to solve them are less. The

challenge is to allow and encourage innovation within a framework of sound management and

engineering fundamentals.

A critical element is the access to state-of-the-art tools and methodologies. The cutting edge of

space research and technology can only be achieved through advancing the way we do work, in

parallel with the work itself. NASA will be continually evolving as new capabilities and

methodologies are enabled by emerging technology.

Technology to enable revolution is the life’s blood of NASA. It must be cutting edge with a

healthy sustained pipeline that addresses the needs of today and the future. Planning must be

strategic, as well as tactical, and be dynamic in response to needs of the Agency and the

Nation. A sustained investment that makes the planning a reality is imperative. Effective

infusion of the new technologies into NASA, academia, and industry is the engine that enables

global competitiveness and leadership.

Failure provides us the stimulus to scrutinize the practices by which we strive to achieve our

scientific and technological objectives. As we evaluate those practices, we do it with

awareness of and respect for our considerable successes, using them to build the framework to

guide future endeavors. As we go forward, we must continue to pursue bold challenges while

pressing the boundaries on what is needed to accomplish them, and accepting prudent risks as

we go. Along the way, it must be remembered that the inherent uncertainty associated with our

business will occasionally result in failure.

This is the plan to take NASA into the future. Through it, we will further strengthen our ability

to be effective stewards of the public trust. As we travel this road, we will, over the upcoming

months, have increased dialog with our partners in industry and academia. Through this

exchange, we can strengthen the entire community with shared experiences and lessons learned

as we work together to better understand our respective contributions to our mutual success.
Enhancing Mission Success – A Framework for the Future

14

NASA Integrated Action Team (NIAT)

Action Descriptions

Introduction

Detailed descriptions of the NIAT actions within the themes described in the Executive

Overview are contained in the following sections. Each description contains an assessment of

the current state of NASA in that particular area, a figure containing the action and references

to the report recommendations that it addresses, and a detailed explanation of the rationale and

content of the action. Actions contain “minimum elements” that must be considered during

implementation. Table II summarizes all actions and identifies the NASA Headquarters office

responsible for serving as the Theme Champions and the Action Lead.

The NIAT, whose membership is shown in Appendix A, assessed 165 report findings and

developed 17 actions containing minimum elements recommended to the Agency. Report

findings are correlated with the actions and contained in Appendix B.

A list of acronyms is provided in Appendix C.
Enhancing Mission Success – A Framework for the Future

15

Table 2. NIAT Actions.

Theme I—Developing and Supporting Exceptional People and Teams (AE)

➛ NIAT-1 Health and Safety (AM)

➛ NIAT-2 Development of the Workforce (F)

 • NIAT 2.1 Enabling Team Competency

 • NIAT 2.2 Improving Workforce Development Capabilities

➛ NIAT-3 Revitalizing Engineering Capability (AE)

Theme II—Delivering Advanced Technology (R-CT)

➛ NIAT-4 Balanced Technology Investment Strategy (R-CT)

➛ NIAT-5 Integrated Technology Planning Process (R-CT)

➛ NIAT-6 Technology Development and Infusion (R-CT)

Theme III—Understanding and Controlling Risk (Q)

➛ NIAT-7 Risk Identification, Assessment, and Management (AE)

➛ NIAT-8 Safety and Mission Assurance (Q)

Theme IV—Ensuring Formulation Rigor and Implementation Discipline (AE)

➛ NIAT-9 Software Development and Assurance (AE)

➛ NIAT-10 Integrated Review Process (AE)

➛ NIAT-11 Ensuring Adequate Resources (B)

➛ NIAT-12 Faster, Better, Cheaper (AE)

➛ NIAT-13 Surveillance (AE)

➛ NIAT-14 Verification and Validation (AE)

➛ NIAT-15 Management Responsibility and Accountability (AE)

Theme V—Improving Communication (AE)

➛ NIAT-16 Organizational Communication (AE)

➛ NIAT-17 Knowledge Management (AO)

( )- Denotes responsible NASA Headquarters Organization
Enhancing Mission Success – A Framework for the Future

16

Theme I—Developing and Supporting

Exceptional People and Teams

Introduction

The importance that NASA places on the health, safety, and personal and professional

development of its people has always been a hallmark of the Agency. The intimate

relationship of our personnel to the success of NASA is a principle that is woven throughout

the NASA Strategic Plan. In that plan, we clearly state that “our greatest strength is our

workforce.” The Agency must properly invest in the maintenance and professional growth of

this valuable resource—its human capital—through a combination of management approaches.

To support the full utilization of the NASA workforce in achieving NASA’s strategic

outcomes, the workforce must have the tools, skills, knowledge, and experience for optimal

performance. This job performance enhancement is enabled by on-the-job work experience,

supported by developmental assignments and training.

Project teams are comprised of civil service and partner individuals that must have the

technical and interpersonal skills to work together effectively. Three actions describe what is

needed to address critical aspects of developing and supporting exceptional people and teams,

including the provision of a safe and healthy work environment conducive to mission success.

Health and Safety discusses how increased demands on employees have caused significant

stresses on physical and psychological health and increase the potential for safety-related

errors. The greatest factor contributing to this stress is not having enough people with the

proper skills, combined with an increase in workload.

Developing the Workforce discusses how successful projects are based upon a strong

foundation of competent and capable people. NASA must invest in enabling team competency

and improving personnel development capability to ensure an adequate foundation for future

programs and projects. Ensuring that the team has the right people with the right skills at

critical times during the life of the project is essential.

Revitalizing Engineering Capability discusses the need for a comprehensive plan to ensure a

world-class engineering capability that includes the development and application of advanced

engineering tools and capabilities. Much of this effort will focus on strengthening capabilities

in systems engineering. State-of-the-art integrated tools will support geographically dispersed

project teams in an advanced engineering and project management environment. At the same

time, the emergence of future technologies will stimulate improved design methodologies and

enable the development of adaptive systems that are less susceptible to human error and more

tolerant of unanticipated operational perturbations.
Enhancing Mission Success – A Framework for the Future

17

NIAT-1: Health and Safety

Current State

Over the past few years, while the NASA workforce has been reduced, the workload on

employees has not comparably declined. The basic nature of the work of NASA—high

visibility and high risk—can create stress that is further compounded by short deadlines,

increasing hours, and fatigue. Stressful situations at work exact an emotional, physical, and

productivity toll on the performance of NASA’s employees and organizations. They also

create the potential for safety-related errors.

Safety and health have been reaffirmed as NASA’s highest core values, and NASA has

emphasized the importance of every employee’s involvement in the Agency Safety Initiative

established in FY 1999. Promoting and maintaining health is a prerequisite to ensuring safety

and productivity in the unique NASA work environment on the ground, in the air, and in space.

The principal contributor to stress and health issues is too heavy a workload.

In FY 1998, a study was completed using focus groups to identify the possible sources of

stress. The NASA Occupational Health and Safety Executive Board (OHSEB) developed

recommendations for stress reduction. An expanded Employee Assistance Program (EAP) was

established, providing after hours support to personnel. On May 1, 2000, the Administrator

announced the creation of a new office, under the leadership of a Chief Health and Medical

Officer, to increase the Agency’s emphasis on health and safety. A NASA Health Council will

be established to address the Agency’s needs and investments in health, including

strengthening external interfaces with other health agencies.

The reports validated the concerns of NASA and emphasized quality of life issues. It was

recognized that the action required is more than training the workforce on “coping” skills but

must include a focused effort on the part of Agency Senior Management to make decisions that

result in a balanced workload.
Enhancing Mission Success – A Framework for the Future

18

NIAT-1 Action Summary

NASA should provide a physically and psychologically safe and healthy work environment for

all its employees.

Figure 1. Safe and Healthy Work Environment.

The objective of this action is to continue implementation of a series of ongoing and planned

initiatives related to the safety and health of the NASA workforce such as Agency Safety

Initiative and the Occupational Health and Safety Executive Board. The Office of the Chief

Health and Medical Officer (Code AM) will continue to monitor various health indicators

affecting the NASA workforce and recommend necessary mitigation and prevention efforts.

To train supervisors and employees, Web-based training modules will be developed to help

identify and manage stress.

The action requires an assessment of the current workload/workforce balance situation and a

determination of specific remedies to control excessive stress on the workforce.

Recommendation Reference: SIA-4, FBC 35, also noted in other reports

Minimum Elements:

A. Ensure continued implementation of the overarching actions of the Agency Safety

Initiative

B. Implement key recommendations from the NASA Officials-in-Charge, prioritized by

the Occupational Health and Safety Executive Board, for reducing stress

C. Develop training modules and train supervisors and employees to be aware of stress

 levels and mitigation.

D. Assess balance of workload and workforce capability and define an approach for

achieving and maintaining balance.

NIAT-1 Action: Provide a physically and psychologically safe and healthy work

environment for all NASA employees [AM, Q, SMC, F, Center Directors; 6/01].
Enhancing Mission Success – A Framework for the Future

19

NIAT-2: Development of the Workforce

Introduction

Successful programs and projects are based upon a strong foundation of competent and capable

people and teams. NASA has provided outstanding professional development opportunities

that have facilitated the establishment of knowledgeable project teams whose skills have been

developed through hands-on experience supplemented by training. The simultaneous increase

in projects accompanied by a reduction in experienced practitioners demands greater attention

to the process of developing and supporting the workforce.

Agency and Center leadership must ensure that development of people is a primary

responsibility of management and that it cannot be compromised by short-term deadlines and

requirements. Accelerated preparation requires more focused development and training that is

planned between employee and manager. Management must personally support less

experienced project managers during project execution as a real time hands-on learning

experience and mentoring opportunity. Project teams need advanced job aids and tools

accessible through the Web, and real-time performance support.

NIAT-2 is separated into two parts, one addressing the development of the team and the second

aimed at improving the capabilities that enable and support development of individuals in the

workforce.

NIAT-2.1 Enabling Team Competency

Current State

NASA has a standard of professional development called the Project Management

Development Process (PMDP) that includes people in disciplines such as engineering, business

management, safety, and procurement. The process defines the competencies necessary for

success primarily through on-the-job work experiences augmented with training.

Approximately 90 percent of the competency is acquired experientially and the remainder

acquired through a variety of supportive education and training mechanisms tailored to the

needs of the individual. This process imparts the basic knowledge of Agency practices in a

way that enhances the value of on the job development experiences. Individuals can use the

PMDP to identify competency requirements, establish a benchmark of current capability, and

provide guidance for individual development plans.

Teams are successful when they have the right people with the right skills at the right time.

The single, most important aspect of project success is the performance of the project team.

The determination of project team membership is the responsibility of the Center Director,

with key assignments usually being made within the Center. The right staffing of project

teams includes consideration of technical skill, interpersonal skills, and resources.
Enhancing Mission Success – A Framework for the Future

20

The technical roles and responsibilities of the program and project managers is currently

contained in NASA Program and Project Management Processes and Requirements

(NPG 7120.5). The aspect of managing human resources is not addressed. The reports

expressed concern as to the consistency of competency across teams in light of the need to

establish teams that are multiskilled, including systems engineering, operations, and scientific

expertise. A more indepth discussion of the team management role of the project manager as

well as the potential for certification of program and project managers and teams was also

suggested.

NIAT-2.1 Action Summary

Ensure that teams are composed of competent personnel through expansion and disciplined use

of the PMDP, selecting the right team skills for the project’s life cycle, and better defining the

roles and responsibilities of the project manager as leader of the team.

Figure 2. Enabling Team Competency.

Recommendation Reference: MCO-1, 2, 6, 7, 8, 9, 10, 11, 12, 14, 23, 29, 30; FBC-6, 8,

9, 35; SIA-2, 4,MPIA-1, 2, 3, 6, 9, 14, 15, 26, 27

NIAT-2.1 Action: Ensure that teams are composed of competent personnel

through expanded and disciplined use of PMDP, selecting the right team skills for

the project lifecycle, and better definition of the roles and responsibilities of the

Project Manager as leader of the team.. [F, AE, H, PMCWG, Center Directors;

6/01].

Minimum Elements:

A. Designate PMDP as the Agencywide standard for program/project management

competencies.

A.1 Review and upgrade PMDP standards for program/project management and

enhance engineering development standards.

A.2 Ensure a disciplined approach to using PMDP competencies consistently as

considerations for the selection, training, and assessment of key project

personnel

A.3 Analyze and assess the need for Center-specific PMDP competencies.

A.4 Consider incorporating acquisition evaluation criteria for

consistency/equivalency with PMDP competencies.

B. Evaluate the advisability of a formal NASA certification process for Program/Project

Managers. Benchmark with industry and Government organizations that certify

Program/Project Managers to determine options for NASA, considering industry

standards, benefits and detriments to certification.

C. Develop a project lifecycle staffing guide tool, including all team disciplines and

 project timeframes, to assist Project Managers in developing:

• Staffing plans

• Team self-assessment and intact team training

• Conducting gap analyses

• Assigning roles and responsibilities to team members.

D. When practicable, publicize project manager and key positions to attract the

 best-qualified personnel across the Agency and beyond.
Enhancing Mission Success – A Framework for the Future

21

Figure 2. Enabling Team Competency (continued).

NASA will designate the PMDP as the Agencywide standard for program and project

management professional competency and use them as considerations for selection, training,

and assessment of key project personnel. The current process will simultaneously be reviewed

and upgraded. Centers will analyze and assess the benefits of establishing more Centerspecific PMDP competencies that complement and offer more specificity than the Agencylevel process.

The issue of program and project management certification has been raised by several of the

reports. Certification represents a more stringent and rigid application of professional

development standards by requiring formal compliance of standards before an individual could

be selected for a position. NASA recognizes both potential benefits and problems with

certification. To make an informed decision, NASA will conduct a benchmark study with

industry and Government organizations to understand their experiences and assessment of

certification for program and project managers. The benchmark study will address the industry

standards, the pros and cons for certification, and its potential impact to NASA. Options for

addressing the issue of certification will be analyzed and submitted to the NASA Chief

Engineer for consideration. A decision on whether or not to establish certification will be

made based on Senior Management Council review of the findings and recommendations.

There is a risk in skills and staffing that need to be recognized as a project goes through several

transitions in its life cycle. Some of these are the move from formulation into implementation

and another is the transition from development into operations. The inherent risk in these

transitions must be mitigated through the right staffing of the project teams as a function of

project maturity. Specific considerations are needed for systems engineering skills at the

“mission” level, and below, to ensure the “systems” perspective is maintained at all levels

throughout the life cycle. The early and continuous involvement of the scientists or their

representatives is needed so they may participate in trade studies and decisionmaking.

Throughout the life cycle, operations skills are a critical element of the project team as well.

The participation of operations personnel in the formulation of a project is essential for making

the right architecture and concept decisions that consider the operational requirements and

long-term costs. Integrating operations skills early in formulation also helps reinforces the

systems perspective to the mission. As the project matures, the operations team has the depth

of knowledge to execute the mission and address mission contingencies.

To support team formation, NASA will develop a staffing guide as a tool to assist Center

leaders and project managers in developing staffing plans, conducting gap analyses, and

E. Review successful practices to evaluate teams, define motivators for high 

 team performance, and present recommendations for Agencywide implementation.

F. Develop an improved Rewards & Recognition process which emphasizes 

 successful, high-performing teams.

G. Expand NPG 7120.5 to clarify PM roles and responsibilities for selecting and

 phasing the project team, and add project team definition (Appendix B) and 

 human resource management (Chapter 4).
Enhancing Mission Success – A Framework for the Future

22

assigning roles and responsibilities to team members. The staffing guide will identify key

functional areas and expertise needed for a successful team. It is expected that this guide will

serve as a tool for less experienced project managers as well as a staffing validation tool for

any Project Manager. An Internet-based team self-assessment tool and intact team

development support also will be made available to support the development of a cohesive and

superior project team.

NASA will modify the current project management guidelines and policies to better reflect the

central role of building an effective team. Resources and tools that support team performance

will be made more widely available. In addition, the roles and responsibilities of the project

manager will be clarified. It should be noted that an effective partnership between the project

manager and line management is directly related to the potential for success. Line

management support is addressed in NIAT actions 15 and 16.

NIAT-2.2 Improving Workforce Development Capabilities

Current State

The emphasis on project management and engineering training and development has

accelerated significantly over the past decade. The establishment of the NASA Academy of

Program/Project Leadership (APPL) underscored the importance and value in 1997. During

the same time period, NASA established the engineering curriculum to provide similar support

to the engineering community. Integrated directly into the NASA strategic management

process, support is offered for total team and individualized professional development through

training, developmental assignments, university partnerships, and advanced technology tools.

Supervisors need to participate actively in determining the need for employee training to

address particular competencies and to ensure training attendance.

The current environment requires timely development and training to effectively support the

execution of NASA’s complex engineering and project management tasks. The reports

recommended an expansion of APPL and a more effective integration with the engineering

curriculum. Hands-on experience through challenging assignments, as well as training support

at the Center level, is seen as essential.

The reports recommended expanding training and improving training tools and methodologies

through use of information systems technologies. Inclusion of the entire project team in

training and an expert review of the training curriculum and methodology is also necessary.

During the last few years, a sizable portion of corporate knowledge, in the form of experienced

practitioners, have left NASA due to normal attrition and planned buyouts. With the

simultaneous increase in the number of projects, a less experienced workforce is being called

upon to manage projects. These project managers and their teams need the insight and wisdom

from experienced practitioners.
Enhancing Mission Success – A Framework for the Future

23

The report findings indicated a lack of availability of experienced personnel to advise and help

the project team either by line management or mentors. Some pilot mentoring systems are in

place and are proving to be valuable. However, a process is needed to be able to deliver this

expertise on demand.

NIAT-2.2 Action Summary

Improve the hands-on experience, training curriculum, and mentoring for project managers and

engineers.

Figure 3. Improve Workforce Development Capabilities

Since on the job-experience is the primary source of training and development, it is imperative

that line managers support development for their employees. Managers must work with their

employees to determine the job-related knowledge, skills, abilities, and experience required for

the work of their particular organization and their individual development objectives. This

includes the short-term and long-term work assignments and classroom training needs.

Recommendation Reference: MCO-2,3,8; FBC- 1, 2, 3, 4,5, 6,7,8,9,11,29,35;

SIA-2,4; MPIA-1,2,3,6, 9,14,15,26,27

NIAT-2.2 Action: Improve the hands-on experience, training curriculum and

mentoring for project managers and engineers. [F, AE, Center Directors, Project

Managers, PMCWG; 9/01].

Minimum Elements:

A. Ensure line management is providing adequate hands-on experience and other

training to support skills development.

B. Establish improved strategies and resources for training and development and

accelerate required capability in advance of need, as appropriate, to include:

– Alliances with universities, industry and other Government agencies to expand

 project and engineering joint training.

– Just-in-time project team support

– Project management automated/Web-based tools and technologies

– Training tools that support NPG 7120.5.

– Realistic scenario-based virtual training simulations

C. Establish a customer-focused Board of Directors to review project and

 engineering curriculum to assess the adequacy of course content.

D. Require project managers to estimate and plan for required training and

 support resources for intact team training.

E. Broaden existing mentoring programs to provide access to experienced and

successful experts.

– Establish a professional project network (ProNet) of experienced and talented

 practitioners who are willing to mentor at their Centers.

– Expanded NASA Masters in Project Management Forum

– Develop legislative proposal to reduce limitations on compensation of individual

 retirees to facilitate access to lost expertise and provide critical knowledge base

 for mentoring.
Enhancing Mission Success – A Framework for the Future

24

Through alliances with universities, industry, and other Government agencies, NASA will

develop improved workforce development capabilities. These capabilities will enhance

professional readiness and competency of its program/project management and engineering

communities through multiple strategies. This strategic approach must set priorities for

training and development based upon customer-driven requirements and then match the

availability of comprehensive services to those requirements. The enhanced portfolio

emphasizes the need for training that minimizes time away from the job.

Project teams will have more “just-in-time” performance support tailored to meet the particular

needs of the project at their site. Automated and Web-based project management tools will be

expanded to provide closer links between training and practice. Virtual learning tools and

simulations will provide interactive realistic scenarios to “practice” project management

techniques and decisionmaking.

A closer integration between the program/project management and engineering development

activities will ensure that project personnel and engineering practitioners are applying the same

knowledge, concurrently. A customer-focused Board of Directors, supported by NASA senior

managers, will review all courses in the project and engineering curriculum to recommend

improvements. Program and project managers must estimate the resources and training

requirements to support intact team training, and must incorporate these estimates into their

program and project plans.

NASA will expand on existing project and engineering mentoring programs and will form a

professional network of experienced and talented practitioners who are willing and capable of

mentoring at their Center. The current NASA Masters in Project Management Forum will be

used to help establish a “Leaders as Teachers and Mentors” initiative to increase the use of

established project and engineering leaders for both teaching and mentoring. NASA also will

increase its use of retired experts to supplement mentoring. Such mentoring activity will draw

on expert experience regarding the objectives of effective and efficient program and project

execution while recognizing the need to embrace an evolution in sound practice as it becomes

enabled by emerging techniques.

The Internet-centered “Academy Sharing Knowledge” Web site will be used as a support for

facilitating best practices and project knowledge sharing. The current tool, Project Manager

Coach (PM Coach), will be upgraded to provide self-help guidance, self-assessment processes,

tools, and job aids directly to the workforce. Legislative relief is needed to engage

knowledgeable retirees without requiring financial sacrifice on their part.
Enhancing Mission Success – A Framework for the Future

25

NIAT-3: Revitalizing Engineering Capability

Current State

NASA takes great pride in possessing a knowledgeable and skilled engineering workforce

capable of world-class performance in the development, integration, and operation of complex

space systems and aerospace technologies. The sustainability of such capability is absolutely

essential to NASA and the Nation.

Sustaining this end-to-end capability depends on robust hiring practices and development of

new engineers, challenging experiences that include hands-on work, training, tools, mentoring,

and management support. Engineers represent approximately 49 percent of the NASA

workforce. They are the talent pool for providing experienced leaders in the engineering of

complex space and technology systems, and in program, project, and institutional management.

The engineering workforce has decreased significantly over the last 8 years, with very limited

hiring of fresh-outs. As a result, the Agency has lost significant corporate knowledge and

critical skills, while the number of approved projects increased significantly. In addition, the

program level architecture tradeoffs, mission options, and technology planning and execution

needed to support the strategic plans of the Enterprises increased as well, placing even greater

demand for skilled engineers, and in particular, systems engineers. The increased number of

projects amplified the challenges on the systems engineering pool by placing equal demands

for project managers from the same talent pool. Like other Government agencies, NASA faces

an aging population, with a large percentage of the most experienced engineers eligible for

retirement in the very near future.

The demand for experienced engineers is expected to remain high, and the complexity of the

systems needing that expertise is expected to increase as we continue to push the envelope of

science and technology. The development, infusion, and application of advanced tools and

methodologies are essential to achieve the efficiencies necessary to be effective in this

environment.

A third ingredient in the assessment of NASA’s engineering capability is consistency in

process, and execution. Over time, each of the NASA Centers has developed internal

processes for systems engineering that have made them largely successful in their mission.

However, as we strive for greater integration, consistency, and sharing of expertise among

NASA Centers, industry, and academia in collaborative environments, it appears that the

Agency could benefit from appropriate Agencywide standards in the systems engineering

process.

NASA is committed to the revitalization and sustainability of its engineering capability. With

the support of the Administration and Congress, NASA has started to fill critical engineering

and other skills essential to health and safety of the Shuttle and ISS programs. In February

2000, the NASA Administrator created the position of Deputy Chief Engineer for Systems

Engineering. This position was established to develop the vision, objectives, and strategies for
Enhancing Mission Success – A Framework for the Future

26

the development and maintenance of the Agency’s world-class engineering capability in the

Agency.

NIAT-3 Action Summary

The purpose of this action is to establish the framework for a comprehensive assessment and

definition of strategies to revitalize the engineering capability of the Agency in the

formulation, implementation, and operation of space systems. This includes people, tools, and

facilities. The assessment is intended to address adequacy of processes, the rigor of their

execution, training and development of the workforce including hands-on experience,

advanced tools and methodologies, and the use of metrics for continuous improvement.

Advances in tools and methods are intended to enable rapid access of knowledge to perform

engineering analyses, assess risk, and support decisions, including the cost, schedule,

performance, and safety of increasingly complex systems. These ingredients are key to

sustained excellence in the engineering capability within NASA.

Figure 4. Revitalizing Engineering Capability.

Recommendation Reference: MPIA-7,11; MCO-21,25,27,71; FBC-19,36,43

Minimum Elements:

A. Determine additional requirements to sustain a world-class capability, and

 develop options to address gaps between present state and needs.

B. Enhance education and training for engineering capability including real-world

 hardware experience and partnering with academia to develop curriculum such as

 systems engineering, advanced engineering environment, risk assessment, and

 cause-and-effect analysis tools and methods.

C. Review policy regarding the appropriate level of work to be performed in-house.

D. Develop Agency-wide process standards, requirements and guidelines for the

 effective implementation of systems engineering in programs and projects.

E. Develop near-term engineering and program/project management environment

 requirements:

 • Assess current Agency investments in legacy systems, tools, and infrastructure

 • Determine interim deliverables provided by the Intelligent Synthesis Environment

 • Determine “capability gaps” and any areas of duplication across Centers.

F. Prioritize and fund an appropriate set of well-defined, near-term engineering and

 program/project management capabilities, and tools that draw on capabilities

 enabled by emerging technologies, particularly intelligent systems

G. Develop an improved tactical plan for infusing/deploying technologies, tools, and

 processes into Center infrastructure to address:

 G.1 Assure integration, interoperability, and efficiency in an improved

 standards-based, open, secure architecture for common infrastructure.

NIAT-3 Action: Develop and implement a comprehensive plan to ensure

excellence in Agency engineering capability, including hands-on work experience,

and the development and application of advanced engineering tools and

methodologies in an advanced engineering and program/project management

environment [AE, EMC, F, Center Directors, Q; 6/01]
Enhancing Mission Success – A Framework for the Future

27

Figure 4. Revitalizing Engineering Capability (continued).

Minimum Elements (Continued):

 G.2 Develop tools and databases to support rapid design and transition to 

 implementation.

 Evaluate feasibility of the recommended common, multi-Agency Internet

 Store for high-quality, modular, advanced components supplied by multiple

 vendors.

 G.3 Planning and timing considerations for transitioning new capabilities into

 Center infrastructure in a way that optimizes legacy systems and advanced

 tool integration. Transition plan should address resource planning, security,

 operations, and maintenance.

G.4 Cultural changes needed to transition to new user environment, including

 training.

G.5 Improved approaches for managing configuration and information across

 Centers and programs.

H. Establish improved process to guide and evaluate integration and use of near-

 term and advanced capabilities and tools

 H.1 Establish Customer-Based Steering Group(s) to provide guidance and

 coordination on near-term elements

 • Engineering Management Council for engineering-related environment

 capabilities

 • SMO for program/project management-related environment capabilities

 • Center institutional operations representatives for infrastructure-related

 capabilities.

 H.2 Establish metrics to evaluate the use and effectiveness of evolving suite of

 advanced engineering and program/project management environment

 capabilities.
Enhancing Mission Success – A Framework for the Future

28

Theme II—Delivering Advanced Technology

Introduction

NASA defines “technology” as the practical application of knowledge to create the capability

to do something entirely new or in an entirely new way. This can be contrasted to “scientific

research” which encompasses the discovery of new knowledge from which new technology is

derived, and “engineering” which uses technology derived from this knowledge to solve

specific problems.

In meeting NASA’s imperative of working at the cutting edge, we depend on the use of

advanced technologies to enable our missions. Highly intelligent, miniature, adaptable, self

healing systems are being enabled by leveraging on discoveries in fundamental sciences such

as biology, molecular science, and others to solve present day engineering problems, thus

creating new ways to address the challenges of the future.

In recent years, NASA has undertaken sweeping changes to strengthen and highlight the role of

advanced technology. In the past, NASA based mission selections on the desirability of

proposed scientific objectives with insufficient regard for technology readiness or alternative

mission concepts (i.e., technologies enabled by mission opportunities). A new paradigm has

emerged where technology investments for generic classes of very challenging missions are

made in advance, and where specific missions are not approved for development until the

enabling technologies have matured (i.e., mission opportunities enabled by technologies).

Developing advanced technologies before mission selection reduces the schedule and cost

uncertainty associated with incorporating them into the mission, as well as significantly

reducing the final mission development times.

NASA has also implemented new organizational structures and approaches to ensure that

technology investments are closely aligned with mission needs. In February 2000, the NASA

Chief Technologist and supporting office were merged within one of NASA’s four Strategic

Enterprises, the Office of Aerospace Technology, to better focus the Agency’s strategy for

maintaining its long-term technology base. The NASA Chief Technologist retains

responsibility for serving as the Administrator’s principal advisor on Agencywide technology

issues, while also becoming the Associate Administrator for Aerospace Technology. This

merger centralized planning and execution of Agency-level technology within one organization

while still allowing Enterprise-specific mission technology development by each of the four

respective Enterprises. NASA has also created the Technology Leadership Council,

representing each of the Enterprises, Centers, and other key senior officials, to set the Agency’s

technology strategy, address critical issues, and formulate and advance NASA’s vision for

technology.
Enhancing Mission Success – A Framework for the Future

29

Delivering the right technologies to meet today’s mission needs while maintaining a healthy,

robust pipeline of new technologies, and pursuing new and emerging paradigms in science and

engineering to enable the missions of the future is a complex management challenge. While a

number of initiatives are underway to address this challenge, NASA will improve how it plans

and manages the delivery of advanced technologies in three areas:

Balanced Technology Investment Strategy—A balanced investment strategy is crucial to

maintaining a pipeline of demonstrated technologies that meets the needs of near- to mid-term

missions, as well as the revolutionary capabilities needed to enable new classes of missions for

the 21st century. A healthy technology pipeline is one that results in a balanced portfolio of

technologies in various stages of development: advanced concepts, technologies in

development, technologies that are flight qualified or tested, and technologies that have been

proven or demonstrated in missions. Long-term and sustained internal investments are also

needed to provide the necessary expertise and capabilities necessary in technology areas that

are strategically critical to NASA, and to stimulate and enable new engineering approaches to

address the unique challenges of space exploration.

Integrated Technology Planning Process—Identifying new technologies to invest in, defining

new mission opportunities, and ensuring the efficient transition of new technologies into

missions is a complex and iterative process. Responsibility for technology planning is shared

among a wide range of Agency, Enterprise, and program players, each representing unique

requirements, opportunities, time horizons, and risk perspectives. Improvements are needed to

ensure an integrated, coordinated technology planning process that is responsive to customer

needs, maximizes synergy among the Enterprise technology goals, and minimizes duplication

of efforts. In addition, rapid advances and potential applications of new and emerging

information technology, biology, and molecular science offer promising capabilities for

NASA’s missions. To take advantage of those capabilities, NASA needs specific plans for

integrating NASA’s information technology programs not only for enabling advanced

engineering environments, but to address engineering design issues. Likewise, adequate

planning and investment are needed to introduce biology and molecular science into the

engineering mainstream.

Technology Development and Infusion—The technology life cycle begins with innovative

ideas vetted through activities such as technology workshops, advanced concept studies, and

mission studies. The technology life cycle ends with successful acceptance and use to

accomplish a NASA mission. Dramatically shorter mission cycles and the growing frequency

of mission launches makes it imperative that NASA find ways to accelerate the introduction of

new technologies into missions. This imperative also applies to large programs that must

address needs for critical technology early in the program life as well as needs for nextgeneration technologies in later program phases. Current processes for managing the full life

cycle of technology development, maturation, and infusion need to be improved to enable the

new levels of performance and capability required by current and planned missions.

Improvements are needed to identify, develop, and infuse promising new technologies more

effectively, reinforce collaborative relationships between technologists and project managers,

and establish new forums for communicating information about technology initiatives.
Enhancing Mission Success – A Framework for the Future

30

The following three actions will be taken to address each of these opportunities for

improvement. The plans should, however, be viewed as a holistic set of related actions that are

tightly coupled. Integration, collaboration, coordination, proactive customer involvement, and

balance are consistent principles throughout the plans. Demonstrating these principles is

integral to the Agency’s ability to improve strategic planning and implementation of

technology programs aimed at meeting the needs of tomorrow’s missions.

NIAT-4: Balanced Technology Investment Strategy

Current State

NASA invests approximately $2 billion annually in technology and technology-related

investments to support space and aeronautics program areas, including long-term strategic

technology efforts that will have broad impact across the spectrum of NASA activities.

The budget formulation process and the investment decisions and trades that are part of that

dynamic process are inherently iterative and complex. Within this environment, investment

decisions must result in a technology investment profile that delivers the technologies needed

to meet near- and long-term mission needs. In addition, adequate investments in infrastructure

must be sustained over time to ensure critically needed technology expertise and capabilities

are available when needed. Investment decisions must balance a number of competing

demands and requirements, including the following long-term implications on NASA’s

technology capability:

• Investment decisions must balance near-term mission requirements for technology and

need for evolutionary improvements with the need for longer-term strategic technology

investments that may be more revolutionary in nature.

• Investment decisions must balance among alternative providers of technology concepts and

solutions. In many cases, industry, academia, and other Government agencies share

technology challenges similar to NASA. NASA’s approach for investment decisions is

“buy when feasible, build when necessary.” Capabilities that are available within NASA,

industry, academia, and other Government entities must be preserved and used, where

appropriate, to meet national technology requirements.

• Investment decisions must consider the long-term implications on NASA’s capability to

meet technology requirements. NASA must provide sufficient internal investments to

sustain the core technology competencies and necessary expertise to provide leadership in

areas that are strategically critical to the future of the Agency.

• Investment decisions must factor in the current maturity of technologies, what investments

are required to mature a technology, and its expected maturation life cycle. The objective

is to ensure a consistent pipeline of technology with no unplanned gaps or constrictions

(e.g., appropriate level and type of advanced concept studies, technologies in development,

demonstrated/tested technologies, flight-proven technologies, etc.) to assure that the right

technology is available when it is needed by missions.
Enhancing Mission Success – A Framework for the Future

31

• Investment decisions must consider the perspectives of a wide range of customers and

interests. The technology user must be involved as investment decisions are made to

ensure the technologies selected for development and maturation are aligned with mission

needs.

• Investment strategies must include the pursuit of new paradigms in biology, molecular

science, information technologies and others, to stimulate revolutionary approaches to

address the engineering problems of today and the future.

How are decisions for technology investments made? NASA has established organizational

entities and defined processes for planning and formulating budgets and making investment

decisions (reference the NASA Strategic Management Handbook, NPG 1000.2, February

2000). Agency, Enterprise, and program strategic directions, requirements, priorities for

decisionmaking, and resource constraints are established annually as part of the budget

formulation process to guide investment decisions. Specific to the Agency’s technology

investments, the NASA Technology Leadership Council, chaired by the Chief Technologist, is

responsible for formulating and advancing NASA’s vision for technology, establishing

technology priorities, and recommending an Agencywide investment strategy to the CIC. The

CIC is responsible for integrating and balancing among Enterprise program investments,

crosscutting technology investments, and institutional investments.

Both the Faster, Better, Cheaper and Mars Climate Orbiter reports point out the criticality of

adequate and stable technology funding that enables the availability of more technologies to

reduce risk or reduce mission constraints. Technology funding needs to:

• Balance research and advanced technology with focused technology, and provide “seed

corn” to trigger revolutionary approaches,

• Place higher priority on funding and support University research and advanced

development,

• Balance competition of technology development along with sustainable technology

development at NASA Centers, and

• Address broad Agency needs and reduce risks (increased Technology Readiness Level

[TRL]) prior to project initiation, including flight opportunities such as the New

Millennium program.
Enhancing Mission Success – A Framework for the Future

32

NIAT-4 Action Summary

NASA should invigorate the Agency’s technology investment to ensure a balanced portfolio of

evolutionary and revolutionary technology.

Figure 5. Balanced Technology Strategy.

The objective of the action is to undertake a thorough and explicit review of the Agency’s

technology portfolio and formulate an invigorated strategy that balances investments from a

variety of perspectives. Enterprises will review near-, mid-, and long-term mission needs and

identify the investments required to deliver technologies that will reduce life cycle cost and

time, and improve safety and mission return. The NASA Chief Technologist will also review

near-, mid-, and long-term needs for cross-Enterprise technology (i.e., not Enterprise-specific)

and identify the investments required to deliver the revolutionary technologies needed to

achieve NASA’s strategic goals. These reviews will also determine whether NASA has an

optimal distribution of investments in technology expertise and capabilities in the private

sector, universities, other Government agencies, and NASA. In addition, NASA will complete

the assessment of Center capabilities that support critical technologies and identify the level of

long-term sustained funding necessary to support these areas. Based on the results of each of

these assessments, NASA will develop an optimum and balanced technology investment

strategy to be advocated as part of the FY 2003 budget process.

Recommendation Reference: FBC-31, 32, 36, 37, 38, 39, 40, 41, 45, 46;

MCO-64,67,70

Minimum Elements:

A. Assess Enterprise and mission-specific requirements and investments, including

near-, mid-, and long-term balance, as well as internal/external participation

balance.

B. Assess Agencywide (cross-Enterprise) requirements and investments, including

near-, mid-, and long-term balance, as well as internal/external participation

balance.

C. Complete the ongoing Technology Assessment, including human, physical, and

financial resource model considerations.

D. Develop and implement the new Agency technology investment strategy that

delineates optimum near-, mid-, and long-term, and internal/external participation

balance.

E. Create a science and engineering research network for the pursuit of nontraditional engineering solutions based on emerging paradigms in biology,

molecular science, information technology and others.

NIAT-4 Action: Invigorate the Agency’s current technology investment to ensure

that a balanced portfolio of existing, new, and emerging technology is available

[R-CT, EAA, Center Directors, B; 4/01(Development Complete)].
Enhancing Mission Success – A Framework for the Future

33

NIAT-5: Integrated Technology Planning Process

Current State

NASA’s technology planning process is multifaceted and involves a wide range of

organizations that are focused on delivering the technology needed to meet NASA’s aggressive

strategic goals. First, each Enterprise works with its respective customer and user community

to translate broad strategic goals into Enterprise missions and their associated technology

challenges. Enterprises then work with Centers to define and develop specific technology

programs to meet the needs and technical challenges of the planned missions. This integration

of technology within the Enterprise ensures that technology programs are closely aligned with

Enterprise mission goals, frequently referred to as “mission pull.” In addition, Enterprises are

responsible for program formulation and funding for technology activities. This ensures that

technology considerations are closely coupled with mission decisions, technologies are

relevant to Enterprise needs, and mechanisms are provided to transfer successful maturing

technologies into operational systems.

To ensure that NASA’s technology investments do not evolve only toward near-term goals and

incremental improvements, the NASA Chief Technologist addresses common technology

requirements across Enterprises and commercial interests and identifies technology areas that

may revolutionize the way missions are conducted. These strategic types of technologies are

referred to as “technology push” investments. Today, the “technology push” areas that are

strategically important to achieving ambitious future NASA mission areas are advanced

miniaturization, intelligent systems, compact sensors and instruments, self-sustaining human

support, deep space systems, and intelligent synthesis environments. Once Enterprise-specific

and cross-Enterprise plans are prepared, they are reviewed by the Chief Technologist and the

Technology Leadership Council, who then develop recommendations for budget and technical

priorities as part of the annual budget cycle. Decisions and final plans that result from this

process are documented in the NASA Technology Plan.

NASA’s recent changes to technology planning and management processes are a significant

improvement over historical approaches, but more improvements are needed. A shared vision

for technology with clear and complementary roles and responsibilities that are well defined

and supported is crucial to an integrated technology plan. Closer collaboration and more

coordinated planning are needed to ensure that cross-Enterprise and Enterprise technology

requirements, priorities, and investments will deliver the technologies needed to meet mission

needs. Processes to transition promising technology from basic research to deployment in

missions need to be well understood and broadly communicated. Finally, better ways to

predict and evaluate the effectiveness of NASA technology investments are needed.
Enhancing Mission Success – A Framework for the Future

34

NASA’s bold missions in space exploration and aeronautics will require advances in many

areas of science and technology. An area that offers exciting promise is the rapid advances and

potential applications of new information technology. NASA is significantly investing in

information technology research to improve the Agency’s core capabilities and to deliver

promising new capabilities into programs and missions. Examples include:

• Intelligent Systems (IS) Program to provide critical new capabilities in automated

reasoning, human-centered computing, intelligent data understanding, and revolutionary

computing, and

• Intelligent Synthesis Environment (ISE) Program to research, develop, and implement tools

and processes for revolutionary changes in engineering practice and science.

We will also be examining other programs to achieve ultrahigh levels of safety and mission

success by fundamentally advancing NASA’s system life cycle approach through infusion of

advanced information technology in intelligent risk management, knowledge synthesis, and

intelligent mitigation and self-healing technologies.

NASA needs to improve the planning, implementation, and application of the exciting promise

that these new research areas have in the near and far term. Integrated planning and user-based

pilot demonstrations for these important new technologies will enable NASA to exploit these

advances and execute our missions with greater safety, performance, and robustness.

Report findings stress that integrated technology planning must balance evolutionary and

revolutionary capability development and ensure early and continuous involvement of the enduser. Cost and risk must be reduced through aggressive integration of leading-edge

technologies. Program management needs to review specific future mission needs and

technology requirements early in the program development. Technology needs that reduce risk

need to be expediently funded and more fully developed prior to project initiation. A better

integrated technology (development) plan is needed as is an Integrated Information

Technology Plan and Program that encompasses ISE, IS, and related Information Technology

(IT) activities.
Enhancing Mission Success – A Framework for the Future

35

NIAT-5 Action Summary

NASA should strengthen its technology planning process and update the NASA Technology

Plan.

Figure 6. Integrated Technology Process.

The objective of the action is to position NASA to more aggressively integrate leading-edge

technology into missions through improvements to the technology planning process. First,

NASA will improve the management approach and mechanisms for conducting technology

planning within the Agency. NASA will focus on a shared vision for technology development,

and clearly define roles and responsibilities for integrating technology plans at all levels. The

Chief Technologist and Technology Leadership Council will establish criteria for identifying

and prioritizing cross-Enterprise technologies that are relevant to Enterprise strategic goals. In

addition, NASA will improve its process for transferring technologies with broad Agency

application to missions. The Agency will implement process improvements to help bridge the

gap from technology research to mission use within allowable risk constraints. Finally, NASA

will continue to refine its method for measuring the effectiveness of technology investments

using metrics and other evaluative processes.

Based on these improvements and others, the NASA Technology Plan will be updated and

improved to provide a more robust tool to manage and communicate technology plans and

processes within NASA and with our external customers and partners. The updated NASA

Technology Plan will reflect the clarified roles, responsibilities, and process improvements

Recommendation Reference: FBC-42, 46; MCO-66, 69, 72, 74, 75, 76, 77

Minimum Elements:

A. Clarify roles and responsibilities for defining cross-Enterprise needs and plans

(e.g., IS, ISE, HPCC, 632, etc.) and Enterprise-specific requirements, and how

they are integrated.

B. Establish criteria for identifying and prioritizing cross-Enterprise technologies that

are relevant to future Enterprise needs.

C. Refine processes to “bridge the gap” between technology research and low-risk

technology infusion into missions.

D. Improve processes and metrics for evaluating the effectiveness of NASA

technology investments.

E. Update NASA Technology Plan to capture process improvements:

E.1 Clarify Technology Roles & Responsibilities and include Multi-Year

Investment Profiles.

E.2 Develop clearer Enterprise and cross-Enterprise technology insertion

 roadmaps to identify multi-year maturation path and projected mission

insertion points.

NIAT-5 Action: Strengthen NASA’s technology planning process and update the

NASA Technology Plan [R-CT, TLC; 5/01].
Enhancing Mission Success – A Framework for the Future

36

discussed above. In addition, multi-year investment profiles, technology maturation paths,

projected mission insertion points, and anticipated benefits of the technology will be included

to provide a long-term perspective of investments and timeframes when technologies are

expected to be available for missions. The plan will clarify the objectives of the ISE, IS, and

other potential programs and describe how they are integrated to yield high levels of safety and

mission success through the infusion of advanced information technologies in the near term

and beyond. In addition, NASA’s current plans for other leading-edge technology areas and

how they are being applied will be described. These include soft-computing technologies,

autonomous operations and avionics technologies, and technologies which lead to

improvement in aircraft, spacecraft, or aerospace vehicle design.

NIAT-6: Technology Development and Infusion

Current State

The development and infusion of new technologies is a crucial element of to the Faster, Better,

Cheaper philosophy. A healthy technology pipeline is a continuum of activity that stretches

from basic research through application of technology in flight. Dramatically shorter mission

cycles and the growing frequency of mission launches makes it imperative that NASA find

ways to accelerate the introduction of new technologies into missions. Current processes for

managing the full life cycle of technology development, maturation, and infusion need to be

improved to enable the new levels of performance and capabilities required by current and

planned missions.

Technology is a complex and iterative activity that involves a continuous evolution and

refinement of what is available, what is needed, and what could be available in the future.

Processes and approaches for managing long-term research activities are quite different from

those applied to mission development and operations programs; yet, elements of customer

involvement, reviews, and oversight are common to the ultimate success of both.

In between technology concept and infusion, the technology must be matured—a process of

testing and analysis that progressively reduces the programmatic risk of selecting that

technology for an application and increases the readiness of that technology for use in a

mission. Historically, “bridging the gap” between technology development and acceptance

into mission applications has been the most difficult step in the technology development

process. In addition, the length of time it takes to successfully mature technology varies,

depending on the nature of the capability. For example, history has shown that it has taken

decades to bring a new flight vehicle from the concept stage through development, design,

manufacturing, and certification of the new aircraft. Yet, the life cycle of computer hardware

or software may only be 3 to 4 years (and shrinking each day) and only have a useful life of 18

months. This rapidly changing state of technology complicates the timely delivery of new

capabilities.
Enhancing Mission Success – A Framework for the Future

37

New technologies provide the innovations to find “win-win” solutions in the Faster, Better,

Cheaper trade space. It has been said that technology development and infusion is a contact

sport between technologists, who are more experimental by nature, and project managers, who

face the reality of cost, schedule and performance challenges. Close collaboration and ongoing

dialogue between the technologists and project managers throughout both the mission life

cycle and the technology development cycle is critical for successful infusion of technology.

There are two aspects to maximizing common understanding and supporting a team approach

in identifying technology opportunities, needs, and solutions:

(1) Better information about technology requirements, investments, capabilities, and

applications across the Agency that is comprehensive, integrated, and readily accessible.

Better information will expose and educate program/project managers and engineers about

available and future technology products and capabilities. NASA has recently launched a new

NASA Technology Portal located at http://nasatechnology.nasa.gov/portal_main.cfm that

provides a common entry point to a wide range of information on new technologies, and the

Agencywide inventory of technology initiatives, and includes updates on technology news and

events. This portal is an excellent step forward in providing an easy integrated access to

needed technology information, but more is needed.

(2) Better guidance and requirements are needed for program/project managers on how to

identify, evaluate and assess, select, develop where necessary, and infuse technology within

their missions. The current set of guidance and requirements provided in NPG 7120.5 needs to

be enhanced to be more specific on technology activities and their value. Better training for

program/project managers to reflect improved guidance is also needed to heighten the

awareness of benefits of new technologies and what program/project managers must do to

incorporate them into missions.

The reports found that the end-to-end process for identifying, developing, demonstrating/

qualifying, and employing technologies in NASA missions must be improved. This can be

accomplished through establishing technology requirements early, holding yearly “out-of-thebox” workshops, and engaging project and technologist teams early to facilitate acceptance.

Technology needs should be expediently funded and matured compatible with project needs.
Enhancing Mission Success – A Framework for the Future

38

NIAT-6 Action Summary

Increase the speed and effectiveness of developing and infusing leading-edge technologies into

missions.

Figure 7. Technology Development and Infusion.

Recommendation Reference: MCO-66, 67, 69, 72; SIA-26, FBC-40

Minimum Elements:

A. Clarify processes and requirements for managing long-term technology research

 and development programs/projects, to include the following points:

• Customer focus and involvement to keep aligned with, and relevant to,

Enterprises, programs, and projects;

• Internal and external quality reviews to ensure technical excellence and

feasibility;

• Lifecycle systems analysis to assess impacts and payoffs;

• Technology testbeds to support the “build a little, test a little” approach;

• Ways to “bridge the gap” between technology research and infusion;

• Maturity growth including ground qualification/space demonstration to facilitate

infusion;

• Documentation and oversight requirements to add rigor without stifling

innovation;and

• Capturing technology infusion lessons learned.

B. Establish a comprehensive environment and suite of tools to provide information

 about available technologies, opportunities, and needs.

B.1 Assess the use and utility of the NASA Technology Inventory database and

 other technology databases and sources (both internal and external to NASA)

 to provide information to program/project managers and systems engineers.

B.2 Establish a Web-based, interactive environment and suite of tools for

 program/project managers, engineers, and technologists to provide easy

 access to technology-related information, including:

 • Upcoming advanced and “out-of-the-box” technology workshops,

 conferences, and seminars;

 • Technology-related training and short courses;

 • Technology product availability and source/contact information, both 

 internal and external to NASA;

 • Potential technology applications and benefits; and

 • Advanced concept mission study results.

C. Clarify NPG 7120.5 guidance and requirements to identify, evaluate/assess, select,

 develop, and infuse technology, to include:

• Processes and requirements for assessing current/projected technology needs

beginning at the earliest conceptual planning phases to ensure technology needs

are met prior to infusion.

• Tools, resources, and processes for identifying and tracking potential/available

technologies.

• Guidance on conducting technology assessments at key program/project phases

to evaluate feasibility; readiness; and lifecycle costs, risks, and benefits.

NIAT-6 Action: Increase the speed and effectiveness of developing and infusing

leading-edge technologies into missions [R-CT, AE, PMCWG, FT; 6/01].
Enhancing Mission Success – A Framework for the Future

39

Figure 7. Technology Development and Infusion (continued).

Three primary steps will be taken to increase the speed and effectiveness of developing and

infusing leading-edge technologies into missions.

The first step addresses activities to clarify processes and requirements for managing long-term

technology research. Steps will be taken to improve the way NASA manages these programs

in a way that maintains customer focus, keeps alignment with missions objectives, and

provides improved rigor and efficiency without stifling the creativity and innovation so integral

for longer term research initiatives.

The second step includes activities to establish a comprehensive environment and suite of tools

to provide information about available technologies, opportunities, needs, and significant

events. NASA has a Technology Inventory database that describes technology tasks and how

they are related to mission needs, and includes an indication of technology maturity and other

key information. In addition, other technology databases exist, within NASA and

commercially, as sources of technology information targeted to certain needs or locations.

NASA will evaluate how these information sources and tools may be integrated and used more

effectively to provide robust information about technology initiatives vital to program

managers, technologists, and engineers. In addition, NASA will enhance the new NASA

Technology Portal to include a content-rich, interactive environment and suite of tools for

communities of interest on technology topics of key importance. For example, information on

upcoming workshops, training sessions, product availability, and study results will be included

and serve as a valuable resource to NASA missions and programs.

The third step addresses improvements that are needed to NASA’s current guidance to program

and project managers on the technology development and infusion process. Current guidance

addresses the process of technology assessment and selection during the formulation phase of a

program/project. Guidance on new tools and resources to support program/project managers

on technology matters will be expanded. Roles and processes for how program/project

managers identify technology requirements, how technology development programs are

chosen, and how technology needs and requirements are fed back into the Agency’s overall

technology planning process also will be included. Training material will be added to the

APPL curriculum on technology processes, requirements, and resources. This training will

enhance a manager’s ability to make the decisions for optimal application of new technology

for cost and risk reduction and maximum return to their project.

Minimum Elements (Continued):

• Strengthen requirements to reflect recurring nature of technology assessment

 processes.

• Guidance on how program/project manager provides feedback on program/project

 technology needs/plans into Agency technology planning processes.

D. Develop APPL curriculum that incorporates new and clarified guidance on

 technology processes and requirements.
Enhancing Mission Success – A Framework for the Future

40

Theme III—Understanding and Controlling Risk

Introduction

One of the critical elements for success in the Faster, Better, Cheaper approach is the project

manager’s ability to understand and control risk. In successful efforts, the project managers

identify and accept (or mitigate) risks early in the project formulation and control the risks

throughout the mission life cycle. They define and understand the criteria for mission success

at the beginning of the effort.

NASA has always recognized risk management as a key factor in project management.

Historically, effective risk management application was dependent on the specific background

and experience of the project manager. With the downsizing in the late 1990’s, much of this

experience left the Agency, and with it, a loss of corporate knowledge on risk management.

In April 1998, NPG 7120.5A, “NASA Program and Project Management Processes and

Requirements,” established a disciplined, defined risk management process. This NPG

requires all projects to develop a risk management plan that describes the process for risk

identification, analysis, mitigation, tracking, and communication. In addition, projects must

present their critical risk list during the formulation phase for Governing Program Management

Council (GPMC) review. During the implementation phase, the knowledge and understanding

of that accepted risk is carried into implementation. Where new risks arise, they are

incorporated into the risk management process and reviewed by the GPMC.

Primary responsibility for effective management of risk rests squarely with the program and

project manager. All members of the team must fully understand their role in identifying and

controlling risk. With the increased emphasis on risk management, NASA’s Safety and

Mission Assurance (SMA) community takes on an important role in supporting the program

and project managers. During the formulation phase, SMA provides guidance and assistance

to the project team in the identification of risks and the development of the risk management

plan. Once a project is underway, SMA provides independent assessment to ensure decisions

and the decision process pay sufficient attention to risk. In addition, SMA assists the projects

in using specialized tools, such as Fault Tree Analysis, Failure Modes and Effects Analysis,

and Probabilistic Risk Assessment to identify and mitigate potential risk to success.

In the area of “Understanding and Control of Risk,” the reports determined that improvements

were needed in risk identification, assessment, and management; the definitions of acceptable

risk and success criteria; and safety and mission assurance.

The following action summarizes the improved capabilities and performance needed in these

areas.
Enhancing Mission Success – A Framework for the Future

41

NIAT-7: Risk Identification, Assessment, and Management

Current State

Although risk management guidance has been developed and requirements for risk

management plans currently exist, the implementation is not uniform across the Agency.

NIAT-7 Action Summary

Improve and enhance the ability to identify, assess, mitigate, and track risk through the

definition of success criteria and acceptable risk, utilization of tools and proper policy and

guidance. Integral to this action is establishing an accurate, agreed-upon understanding of

acceptable risk in the context of well-defined success criteria.

This action has three main objectives, each a critical part of technical and programmatic risk

management. Understanding risk management requirements and tools focuses on ensuring that

proper policy and guidance is in place and understood and that tools exist to facilitate the risk

identification, assessment, mitigation, and communication process throughout the life of the

project.

To ensure that risk management process is included in acquisition an interim rule change to the

NASA Federal Acquisition Regulation (FAR) Supplement has been put in place. Additional

changes to other acquisition mechanisms such as, Cooperative Agreements, Announcements of

Opportunity, etc. will follow.

The application of new and existing tools for risk assessment will be improved by training the

acquisition community and the program/project teams. In the near term, projects will be

required to use currently available tools such as Fault Tree Analysis and Failure Mode Effects

Analysis (FMEA), and, when appropriate, a more in-depth Probabilistic Risk Assessment to

uncover risks and determine the robustness of risk mitigation. In the long term, intelligent

systems based tools, enabled by emerging technologies, such as soft adaptive computing must

be pursued to develop more rigorous risk assessment and management approaches.

Center Directors will ensure that the programs/projects address risk mitigation plans and

progress at periodic GPMC reviews during the program and project life cycle in order to ensure

that the job of meeting the success criteria is supported by the available resources, and that the

budgetary and schedule reserves, as well as the design margins, are compatible with the risk

posture.

The program/project team must develop clear “Success Criteria” during the formulation phase

and that success criteria must be clearly communicated to all levels of the program and project

organizations to define scope of the effort and to guide risk decisions. Furthermore, success

criteria need to be developed at lower levels; e.g., system, subsystem, and component level to

define trade space and support risk decisions. Success criteria will continue to evolve

throughout the life cycle of the project. They may be modified through discussion and

decisions related to risk, technology, program requirements, etc. as the project matures. It is
Enhancing Mission Success – A Framework for the Future

42

imperative that the understanding of these criteria be communicated up and down the

organization by project and line management to the entire team, including partners or

contractors.

 “Acceptable Risk” is defined in terms of the program/project management decision process. It

is impossible to adequately define “categories” of risk and undesirable to move back to overly

prescriptive rules for those categories. The variety of work within NASA makes this an

impractical approach. This definition ensures that acceptable risk is properly identified by the

program/project, drives the development of the risk management plan and future management

decisions and is well communicated up and down the organization. The proposed definition is

“Acceptable risk is the risk that is understood and agreed to by the program/project, GPMC,

and the customer as sufficient to achieve defined success criteria within an approved level of

resources.” Each program/project must have sufficient information to make an informed

decision on what is acceptable risk. The understanding of risk and risk management is an

essential part of engineering from concept development through design and into operations.

To support engineers and project managers, advanced quantitative tools are needed for risk

analysis. When delivered, these tools will aid in the flow down of risk and support the

assessment of success criteria at all levels.

Figure 8. Risk Identification, Assessment, and Management.

Recommendation Reference: MCO-18, 19, 21, 22, 24, 25, 27, 31, 32, 33, 34, 35,

36, 37, 39; FBC-19, 21, 44; MPIA-4, 5, 8, 11, 13, 19; SIA-9, 10, 12, 15, 16, 18, 22

Minimum Elements:

A. Ensure proper policy is in place and understood

A.1 Use NPG 8705 to better communicate the risk management process.

A.2 Develop an interim change or a Procurement Information Circular to

 explicitly address the AO/NRA process.

A.3 Assess changes to the Grant and Cooperative Agreement Handbook to

 address RM for Cooperative Agreements.

A.4 Implement a Risk-Based Acquisition Management activity for explicit

 safety and mission assurance guidance to RFP development, SOW

 development, source evaluation criteria, pre-award and post-award surveys.

A.5 Strengthen NPG 7120.5 to reflect risk assessment by the entire team during

 formulation.

A.6 Ensure that the GPMC’s address risk mitigation plans and progress at

 program/project reviews.

B. Provide improved guidance on tools and techniques for risk assessment.

B.1 Modify NPG7120.5A to include FMEA, FTA, and PRA to support risk

 assessment.

NIAT-7 Action: Improve and enhance NASA and contractor knowledge and ability to

identify, assess, mitigate, and track risk through the definition of success criteria,

acceptable risk, utilization of existing and new tools, and proper policy and guidance

[AE, Q, H ,PMCWG, FT, Center Directors; 9/01].
Enhancing Mission Success – A Framework for the Future

43

Figure 8. Risk Identification, Assessment, and Management (continued).

Minimum Elements (Continued):

B.2 Develop a project management training module on PRA, FTA, FMEA.

B.3 Continue activity to expand NASA’s peer reviewed “Quantitative Risk 

 Assessment System (QRAS)” tool to include projects beyond the Space

 Shuttle. Evaluate other state-of-the-art tools that may be available.

C. Develop enhanced risk management methodology and tools that enable

 quantitative assessment of risk that can be flowed down and allocated, similar to

 a power or weight budget.

 – Tool should allow assessment of system trade space and support design

 decisions.

 – This allocated risk should be a contractor requirement.

D. Develop an improved methodology to provide enhanced risk tracking that alerts

 all levels of project management of possible problems.

D.1 Expand NPG 8705, current “Risk Profile” description to provide more

 detailed guidance into how to develop and maintain a risk tracking system.

D.2 Ensure risk tracking is in place for programs and projects.

E. Ensure that risk plans include intraprogrammatic considerations (e.g.,

 spacecraft-to-spacecraft dependencies).

F. Define “Success Criteria.” “Success criteria is that portion of the top

level requirements that define what should be achieved to successfully satisfy the

strategic plan objectives addressed by the program, project or technology

demonstration.” Additional considerations include the following:

 – Success criteria are established during the Formulation subprocess to drive

 requirements, define allowable trade space, and guide risk and safety

 decisions. Success criteria are under change control.

 – These criteria flow down from the overall requirements to the system/subsystem

 level to meet the overall objectives.

G. Ensure flow down of success criteria through the systems, system trades, and

decision processes of current programs and projects and communicate the criteria

to all team members.

H. Define “Acceptable Risk:” “Acceptable Risk is the risk that is understood and

 agreed to by the program/project, GPMC, and customer sufficient to achieve

 defined success criteria within an approved level of resources.” Incorporate into

NPG 7120.5.

I. Ensure that approval to initiate implementation includes management

concurrence with a baseline risk management plan that defines the following:

 – General risk mitigation strategies to be employed throughout program/project

 implementation.

 – The initial set of identified risks and the risk mitigation/acceptance plan for each.

 – The process to be used for continual assessment of risk posture.

 – The approach to regularly communicating risk both internally to the project staff

 and throughout the management chain.
Enhancing Mission Success – A Framework for the Future

44

NIAT-8: Safety and Mission Assurance

Current State

Under cost and schedule pressure, involvement of SMA organization is frequently limited to

only specific deliverables (e.g., review of a quality plan). Project managers sometimes do not

plan for adequate SMA coverage, particularly after the formulation subprocess and into

implementation including operations. Risks and margins are also factors that should be

considered in the establishment of the right level of SMA involvement (e.g., surveillance,

lessons learned). In addition, the processes at the Centers for the reporting of incidents,

anomalies, etc. are not implemented uniformly well by all projects. The SMA organizations

need to be engaged in this process to ensure the efficacy of the reporting system and the

follow-up and closure of such events.

SMA organizations have trouble recruiting and retaining a skilled engineering staff due to the

perception that these positions have a limited career path.

NIAT-8 Action Summary

NASA should enhance SMA presence and involvement throughout the program/project life

cycle.

Figure 9. Safety and Mission Assurance.

The assessment of the situation in SMA support by the project managers, in conjunction with

the GPMC, will identify any areas of concern on a project-specific basis. Relative to the

incident reporting processes, Centers should ensure the process is working using SMA as an

assessment agent. The SMA in partnership with the Centers, will define approaches for

enhancing engineering skills of assurance personnel through various means such as education,

recognition, rotational assignments, and improved career paths.

Recommendation Reference: MCO-26; FBC-15; SIA-7, 17, 19

NIAT-8 Action: Improve NASA Safety and Mission Assurance to provide lifecycle rigor

in the formulation and implementation of flight systems [Q, Center Directors; 4/01].

Minimum Elements:

A. Ensure sufficient value added SMA involvement in programs/projects.

B. Define approaches to enhancing engineering skills of in-line assurance

 personnel through education, recognition, rotational assignments, improved career

 paths that enable greater growth, etc.

C. Ensure the SMA involvement in the oversight of the process of reporting incidents,

 anomalies, etc. and their responsibility in following through on these reports,

 ensuring close-out occurs.

D. Field Center ensure adequate training of personnel on problem reporting systems.
Enhancing Mission Success – A Framework for the Future

45

Theme IV—Ensuring Formulation Rigor

and Implementation Discipline

Introduction

To successfully execute our programs and projects, NASA must exercise rigor and discipline

in formulation and implementation. The processes and practices of good program and project

management have long been understood by the Agency, but one of the findings of the reports is

that on some projects those processes or practices have not been followed with sufficient

discipline. A thorough formulation phase, confirmed to be such by rigorous independent

assessment, is essential before initiating implementation of any program or project.

Appropriate use of competition to further stimulate innovation of approach to accomplishing

objectives with reduced, yet realistic resources must be considered.

During the last decade, NASA has been asked to do more with less. With a few exceptions, the

Agency has delivered a greater number of programs and projects at lower cost and in a shorter

development time. These projects have produced many exciting findings and have answered

some of the fundamental questions that send us faring into space. In light of the recent

problems, however, it is appropriate for NASA to revisit policies and procedures for

formulating and implementing projects. Center Directors, Associate Administrators, and other

senior managers must verify that all policy, procedures, and practices used by civil service and

contractor personnel adhere to the fundamentals of sound management and engineering

practice.

Additional clarification is needed to ensure common understanding of the Faster, Better,

Cheaper management philosophy and the roles and responsibilities of the performing

organizations in supporting mission success. This includes ensuring that adequate resources

and margins are available and that sufficient rigor is being applied at all levels of the

organization to deliver successful programs and projects.

The seven actions that follow, outline the steps needed to produce the common understanding

of Agency policy and to ensure that NASA is routinely formulating and implementing its

programs and projects in a rigorous and disciplined manner. The actions are intentionally not

overly prescriptive. The ability to innovate and to tailor approaches to unique needs is

recognized as an essential part of a team’s ability to meet their responsibilities and be

accountable for their results. In so doing, however, changes to practice must be consciously

considered such that reasonable assurance of adequate rigor is retained.
Enhancing Mission Success – A Framework for the Future

46

NIAT-9: Software Development and Assurance

Current State

Over the last decade, NASA has seen a dramatic increase in software needed for our missions

and in particular a significant increase in on-board computational capability. As an example,

Voyager had 3,000 lines of code on board compared to the 160,000 lines of code for Mars

Pathfinder. In the future, we foresee exponential growth in software and software complexity

within NASA as well as industry. For example, industry data indicate a growth in computer

chip complexity by a factor of 2 every 18 months. Today, commercial packages such as

operating systems, enterprise resource management systems, and telecommunications

operations software are over 30 million lines of code.

Software is an increasingly difficult challenge to mission success. In general, the tools we

have for developing and verifying software need improvement to prevent the systemic

management and technical problems that traditionally occur in software development and

verification. An independent study of over 8,000 software projects across all industries

conducted in the mid-1990’s showed that 31 percent of the projects were cancelled during

development; 53 percent had major problems related to cost, delivery time, or performance

capability; and only 16 percent were fully successful. For large software projects, only 8

percent were successful. Even then, there are typically 5 to 10 errors per 1,000 lines of code in

commercial products.

NASA has many examples of excellence in software development process and performance

such as the Space Shuttle software system. However, with the projected growth in software

scope and complexity, NASA must take proactive steps to ensure software quality and

reliability.
Enhancing Mission Success – A Framework for the Future

47

NIAT-9 Action Summary

NASA will improve the quality and reliability of software.

Figure 10. Software Development and Assurance.

Because of our increased reliance on software and its critical role in mission success, NASA

has begun an initiative to improve the quality and safety of software. This initiative will

address process improvement, metrics, enhanced verification and validation, and fundamental

software research applicable to verification and validation of software systems. The core

capability in software Independent Verification and Validation (IV&V) represented by the

NASA Fairmont IV&V Facility is expected to play a pivotal role in the Agency’s thrust to

ensure successful missions.

The action also sets the stage for the Agency to develop necessary requirements and guidance

for the development and implementation of a plan that moves the Agency toward uniformly

achieving a level of capability in critical software systems commensurate with the Carnegie

Mellon/Software Engineering Institute level 3.

Recommendation Reference: MCO-28

NIAT-9 Action: Revolutionize the process of developing and delivering safe, reliable,

quality software, and improve transfer of new software methods and tools into NASA

practice [AE, FT; 6/01].

Minimum Elements:

A. Develop and implement a plan to move all critical software engineering

 development, management and assurance processes to a level commensurate

 with Capability Maturity Model (CMM) level 3.

B. Develop requirement for software-specific training as part of APPL curriculum

commensurate with CMM level 3 goal.

C. Develop Agencywide requirements and guidelines for the development of

 software in Programs and Projects based on widely accepted industry standards.

D. Establish and apply basic metrics to assess performance in the development

 of software.

E. Establish process, criteria, and standards for V&V and IV&V emphasizing end-

 to-end system verification.

F. Emphasize fundamental software research in areas of reliability, error tolerance,

 automated reasoning, software reuse, and emerging paradigms

 (e.g., learning systems, and design for safety).

G. Develop a plan for the implementation of soft-computing methods in Programs

 and Projects.
Enhancing Mission Success – A Framework for the Future

48

NIAT-10: Integrated Review Process

Current State

NPG 7120.5 provides the requirements and process for performing programmatic and technical

reviews. The guidance provides the framework for knowledgeable practitioners to develop an

effective, tailored set of reviews for each program and project. Systems Management Offices

have been implemented at each of the Centers as a resource to assist Center management and

program/project managers.

Inadequate review is a frequently cited theme in the recent mishap reports. The reports suggest

that rigorous discipline must be routinely enforced in the review process. Projects should

conduct a comprehensive program of thorough technical and programmatic reviews. The

choice of experts to participate in reviews is critical to success. Continuity of review panels

throughout the project life cycle is important. Complete, timely, and accurate reporting and

closure of issues is required.

The guiding principles for reviews are articulated below. The major classes of reviews and

how they fit into an integrated and comprehensive continuum of reviews also are defined.

Guiding Principles of Management and Technical Reviews

• Reviews are a resource. They offer an opportunity to add value to the products and to the

sharing of knowledge by inviting outside experts that can provide confirmation of the

approach and/or recommend options.

• Reviews are a tool for communication. They offer an opportunity to organize, assess, and

communicate critical data and information between providers, customers, and stakeholders.

Major Classes of Reviews

The objectives and salient features of the four major review classes are provided to guide

program/project managers in the formulation and implementation of an integrated and

comprehensive continuum of reviews. The term “integrated and comprehensive continuum” is

used to emphasize that there is both a life cycle relationship and a hierarchical relationship to

these reviews. Reviews provide the opportunity to confirm the approach or offer options, if

needed, and communicate progress and risks toward meeting the success criteria. Reviews also

serve the needs of the various levels of the management hierarchy from those of an individual

product lead on a project to those of the NASA Administrator. The products of these reviews

(i.e., assessments, options, recommendations, and decisions) flow upward to subsequent

reviews as appropriate to ensure alignment between providers, customers, and stakeholders,

and ensure proper disposition of issues. It is up to the program or project manager to propose

options to combine reviews to management, customers, and stakeholders, provided that the

objectives of each are met. The goal is to enhance the probability of mission success through

added value and efficiencies. The review program must be tailored to the needs of a program

or project in a way that is compatible with acceptable risk.
Enhancing Mission Success – A Framework for the Future

49

• Product Integrity Reviews are an inclusive term for the variety of surveillance

mechanisms employed by line management organizations to ensure the quality, viability,

and safety of the products they provide. The expectation is that the supervisory chain of all

performing organizations within NASA Centers, contractors, suppliers, universities,

foreign partners, etc. exercises their responsibility for the integrity of the products and

services their employees deliver to a project. The management and technical experts of the

performing organizations conduct the reviews according to their internal processes. The

ultimate customer is the project team member responsible for the delivery and performance

of the product. This product lead is accountable to the project manager for ensuring that

performing organizations understand and fulfill their accountability for the product.

• Engineering Peer Reviews are focused, in-depth technical reviews used to provide

confirmation and offer options by bringing experts in as early as possible. A thoughtfully

formulated, comprehensive set of Engineering Peer Reviews is a cornerstone of a

successful project. The reviews provide a penetrating examination of design, analysis,

manufacturing, integration, test and operational details, drawings, processes, and data.

Engineering Peer Reviews are most frequently applied to subsystem or lower level

development activities. They also are well suited for the evaluation of concepts, designs,

and processes associated with combinations of subsystems and crosscutting functional

subdivisions such as the end-to-end optical path, command and data pipeline, maneuver

planning, or autonomous fault detection/correction system. Engineering Peer Reviews are

most effective when accomplished with a small group of reviewers working around a table

with the developers. Reviewers are experts independent of the project team, including

experts from outside of the performing organization. Reviewers are appointed in

collaboration with the product lead’s line management. The customers are the product

leads and the project manager. They are also accountable for the definition of review

objectives and subsequent communication and closure of issues resulting from the reviews.

• Critical Milestone Reviews are the life cycle series of rigorous system-level technical and

programmatic evaluations conducted at key formulation and implementation milestones.

Key milestones in this context are the major transition points in the project life cycle, such

as the transition from formulation to implementation, and the transition from the assembly

and integration of components to system-level environmental testing. The purpose of a

Critical Milestone Review is to assess the technical and programmatic health of a project or

major element of a project with respect to the success criteria and acceptable risk. The

reviews provide top-down systematic evaluations of the derivation and functional

allocation of requirements, the engineering implementation to address the requirements, the

validation and verification of the requirements, the preparation for mission operations and

data analysis, and the system management processes that tie it all together. Critical

Milestone Reviews should also address the resources (e.g., workforce, budget, schedule)

required to implement the program/project, any associated resource constraints,

issues/risks, and reserves. Reviewers are independent of the project and largely

independent from the performing organization. They are chosen based on their combined

expertise and their ability to assess the implementation of an entire system that employs

numerous engineering and other disciplines. Review teams that provide continuity
Enhancing Mission Success – A Framework for the Future

50

throughout the life cycle of the project is desirable to limit the amount of reeducation that

must be done to get new members knowledgeable.

• Independent Reviews provide senior Agency managers with objective assessments of

program/project planning, resource requirements, status, and risks. An independent review

is generally requested by or on behalf of a Center Director, Enterprise Associate

Administrator, or the NASA Administrator. Reviewers are experts from organizations

outside of the advocacy chain of the program/project being reviewed. To the extent

possible, continuity of review panel membership is maintained throughout the life cycle of

the program/project.
Enhancing Mission Success – A Framework for the Future

51

NIAT-10 Action Summary

Define and rigorously implement a refined and reinvigorated NASA Integrated Review Process

for programs/projects.

Figure 11. Integrated Review Process.

Recommendation Reference: MCO-41, 42, 43, 44, 45, 46, 47, 49, 51, 52, 58, 60,

61, 62, 63; FBC-17, 22, 23; SIA-20; MPIA-17, 21, 22, 24, 25

NIAT-10 Action: Define and rigorously implement a refined and reinvigorated

NASA Integrated Review Process for Programs/Projects, which will:

[AE, PMCWG, Center Directors; 4/01]:

 – Clarify an integrated continuum of reviews and associated rollup of value-

 added review products.

 – Specify required reviews and classes of reviews for tailoring by

 Enterprise/Center/Program/Project including “peer” and “independent”

 reviews.

 – Clarify objectives and expectations for each review/review class

 – Identify the review “customer” and specify participants, roles, and

 responsibilities.

 – Address independence and expertise requirements for reviewers, and the

 continuity and discipline area penetration assignments of reviewers.

Minimum Elements:

A. Revise NPG 7120.5 to guide program/project-defined technical and

 programmatic reviews.

 • Plan and conduct a comprehensive and rigorous review program tailored to

 add value to the program/project.

 • Increase attention to the selection and continuity of review panel members

 • Plan and allocate resources for the review process.

 • Complete timely and accurate reporting and closure of issues.

B. Confirm that all programs/projects currently in formulation and implementation

 have planned and are executing the appropriate technical and programmatic

 reviews.

C. Evaluate and update the technical and programmatic review processes at field

 Centers to ensure that each program/project plans and executes an effective,

 rigorous, and efficient review program, including the following:

 • Ensure that projects plan and conduct a comprehensive review program and

 that individual reviews are thorough.

 • Evaluate the execution of the formulation process including independent

 assessments of resource requirements

 • Ensure thorough engineering reviews by independent peers

 • Ensure that the selection of review panel membership is given a high priority

 and independently assessed.

 • Ensure that resources are planned and allocated for the review process. 
Enhancing Mission Success – A Framework for the Future

52

This action requires the Office of the Chief Engineer and the Program Management Council

Working Group (PMCWG) to refine Agency policy and procedures on reviews to provide

clearer guidance to Agency, Center, and program and project managers.

Center Directors are asked to confirm that current projects are receiving adequate review and

that Center policies and procedures on reviews are adequate to ensure that future programs and

projects plan and execute an effective, rigorous, and efficient review program. An essential

element of project success is the involvement of line management in the review and assessment

of projects during formulation and implementation.

NIAT-11: Ensuring Adequate Resources

Current State

The budget process requires programs and projects to estimate their resource requirements

approximately 18 months prior to the provision of appropriations. This lead time requires

Enterprise or program, project managers to make assumptions about resource requirements

very early in the formulation phase, when the system design and therefore the requirements

are still immature. The uncertainty can be significant, especially for high risk leading edge

endeavors, and thus without a clear understanding and communication of the same, unrealistic

expectations may be set among customers and stake holders. As the program or project

progresses through the formulation and into the implementation phase the uncertainty and the

attendant risk in accomplishing those requirements are reduced as the initial assumptions are

either validated, modified or discarded. Tools and methodologies for better estimation of the

uncertainty in the resource requirements are required. Program and project managers must

plan for and request resources that are adequate and commensurate with the scope, level of

maturity, and understanding of the end to end system.

In the current constrained budgetary environment, it has been difficult for NASA to maintain

robust margins. The Agency has taken a number of steps to get “the most bang for the buck:”

reserves have been reduced, mission costs have been internally “capped,” development

schedules have been fixed, and reserves have sometimes been controlled at a program rather

than project level and sometimes even eliminated altogether. While well intended, and often

effective, such practices cannot be decoupled from the risk management process.

A primary finding of the Mars failure reports was that funding, including reserves, was

inadequate to ensure mission success. The scope of the job and the allocated resources must

be balanced throughout the life cycle. When circumstances warrant, balance must be restored.

It must be recognized that, in such cases, additional funding is not the only alternative.

Modifications in scope, leverage of other resources, or other potential solutions must also be

considered as we strive to identify innovative ways to accomplish challenging mission

objectives. Cancellation must be viewed as one alternative when resource needs are well

beyond current allocations. Indeed, continuing an inadequately funded program or project may

be worse than canceling the effort. Such decisions, when necessary, must be made as early as
Enhancing Mission Success – A Framework for the Future

53

possible, a point that only further emphasizes the need to be continually aware of the balance

between scope and resource.

NIAT-11 Action Summary

Throughout the life cycle NASA should ensure provision of program and project resources,

including margins and reserves, that is compatible with the scope of the job.

Figure 12. Ensuring Adequate Resources.

The objective of the action is to ensure that adequate resources, including sufficient budget and

schedule reserves and design margins to support the risk posture, are included in initial

formulation estimates and maintained through the life cycle of the project or program.

Appropriate utilization of competition must be recognized as a valuable method of effectively

eliciting innovative approaches that require reduced, yet realistic resources to meet challenging

mission needs. Scope must be well delineated at the time initial funding estimates are defined,

and must be effectively controlled throughout execution. It is critical too, that the responsible

project or program manager controls the reserves and margins. This will ensure timely

response to issues. Center Directors will examine the adequacy of the margins and reserves for

the programs and projects assigned to them. Associate Administrators will verify the adequacy

of the margins and reserves and take steps to ensure timely release to the responsible program

and project managers. The Office of the Chief Financial Officer will work with the appropriate

Government entities to better inform stakeholders of the preliminary character of life cycle cost

estimates generated during formulation and ensure realistic expectations of cost and

performance.

Recommendation Reference: MCO-17; MPIA-12

Minimum Elements:

A. Assess the adequacy of current program and project resources for the current scope

of the job.

B. Verify the adequacy of program reserves and ensure reserves flow down to project

managers.

C. In budget documents and external communications that are provided prior to

program/project approval, clarify the preliminary character of lifecycle cost estimates

to facilitate any required revision in cost, schedule, or scope.

NIAT-11 Action: Throughout the life cycle ensure provision of program and project

resources that is compatible with the scope of the job. [B, EAA’s, Center Directors;

4/01].
Enhancing Mission Success – A Framework for the Future

54

NIAT-12: Faster, Better, Cheaper

Current State

In 1992, the NASA Administrator recognized that the Agency had been on a path that focused

most of its resources on major projects and programs that took 7 to 10 years to complete. FBC

was and is intended to stimulate innovation and provide challenge for bold new missions

through application of technology, streamlined practices, managed risk, and accountable

project teams. FBC has promoted taking risk with insufficient guidance as to the boundaries.

Resources have been highly constrained and the impact on resultant risk has not always been

mitigated.

In 1996, the Agency provided the guidance that moved project management from a strict “rulebased” approach to managing projects to a more flexible but structured process approach. This

guidance was codified through the development of the NASA Procedures and Guidelines

7120.5, “NASA Program and Project Management Processes and Requirements”. Several of

the findings of the failure reports identified the lack of a commonly held definition and process

guidance as contributing factors to these early FBC missions.

NIAT-12 Action Summary

Define Faster, Better, Cheaper and appropriately incorporate the definition into NPG 7120.5.

The definition in Figure 13 below has been developed for inclusion in NPG 7120.5. This

clarification will enable better understanding of the intent and environment of program/project

management. Project success will be enabled by good planning and implementation per

NPG 7120.5 and through the application of sound management and engineering practices.
Enhancing Mission Success – A Framework for the Future

55

Figure 13. Faster, Better, Cheaper.

The conclusions of the reports, and this NIAT assessment, are that the FBC principles are valid

if properly applied. They apply in varying degrees to different programs recognizing that the

portfolio of Agency missions covers a broad spectrum of activities for which the level of

acceptable mission risk must be determined by mission unique considerations. The NASA and

Enterprise strategic plans project a vision full of multiple challenging missions and technology

developments that to become a reality in the present environment need sound application of the

Faster, Better, Cheaper philosophy and its principles.

NIAT-13: Surveillance

Current State

The current NASA Procedures and Guidelines for the management of programs and projects

define the responsibilities of management officials in the approval and ongoing surveillance of

programs and projects. Responsibility for program management has been delegated to the

Centers. A hierarchy of Program Management Councils has been established to ensure that

Agency management is regularly engaged in the assessment of program and project status,

issues, and risks. System Management Offices have recently been formed to assist Center

Management in the execution of their surveillance responsibilities.

Recommendation Reference: FBC-20

NIAT-12 Action: Define “Faster, Better, Cheaper” and incorporate into NPG 7120.5 

[AE, PMCWG; 4/01].

Minimum Elements:

Define FBC. “Faster, Better, Cheaper is a management philosophy that promotes 

the acceptance of prudent mission risk while using sound and innovative approaches 

to safely accomplish bold mission objectives (better) while conserving time and money

(faster, cheaper).” 

To accomplish this, NASA:

– Emphasizes that safety of the public, its flight crews, its employees, and its critical 

 assets are of paramount importance.

– Relies upon individual and organizational commitment to responsibility and 

 accountability for doing it right the first time.

– Fosters efficiency in process, and the application of innovative methods 

 and tools to greatly reduce product development cycle time and costs while ensuring

 that the risk is acceptable.

– Invests in an educated and empowered workforce to ensure the application of 

 sound Project Management and Engineering practices.

– Invests in a sound technology program aimed at future needs, and encourages 

 the infusion of those technologies.

− Recognizes that occasional mission failures will still occur.
Enhancing Mission Success – A Framework for the Future

56

The delegation of program management authority, changes in day-to-day relationships with the

contractor community, introduction of Principal Investigator-managed missions, other changes

in project management, and the recent mission failures have raised issues relative to the

adequacy of surveillance activities. The reports found that surveillance was sometimes

inadequate and contributed to a lack of communication of issues among the project team,

management, and contractors.

NIAT-13 Action Summary

Ensure that programs and projects receive adequate surveillance based on the understanding of

risk and the ability of the supplier to deliver the product or service and that there is ample

opportunity for the communication of issues and concerns.

Figure 14. Surveillance.

The action requires that current guidance and requirements be reviewed for adequacy, with

revisions being provided if clarification is required. The first step will entail a review of

NPG 7120.5 to ensure that:

• The hierarchy of management roles and responsibilities is clear,

• Project documentation and processes are structured to provide both planning for

surveillance and mechanisms to communicate project status and risk.

Center Directors also will review their surveillance and reporting requirements and procedures

for ambiguities, conflicts, and possible omissions to confirm that the appropriate reporting and

Recommendation Reference: MCO-16, 40; FBC-18, 30, 33, 34; MPIA-10, 16, 22, 23

Minimum Elements:

A. Ensure that NPG 7120.5 clearly defines the hierarchy of management roles,

surveillance, and planning documentation required relative to project risks.

Incorporate any deficiencies into NPG 7120.5.

B. Review surveillance and reporting requirements and procedures at field Centers for

ambiguities, conflicts, and omissions. Utilize risk to help determine appropriate

surveillance and reporting requirements.

 • Confirm that appropriate surveillance and reporting is applied to projects including

 in-house, Government/contractor partnerships, and contractor advanced

 development activities.

 • Confirm that process and acquisition instruments enable the participation of all

 parties in open and frank communication and effective surveillance.

 • Confirm that appropriate mechanisms exist for program management to

 periodically review project decisions for program or mission-level impact.

 • Confirm that major issues identified in programmatic and technical reviews of

 projects are reported to senior management.

• Provide for the periodic reassessment of the reporting process.

 • Integrate and maximize the efficiency of insight personnel. 

NIAT-13 Action: Ensure that programs and projects receive adequate surveillance

based on risk knowledge and that there is ample opportunity for the communication of

issues and concerns [AE, PMCWG, Center Directors; 6/01].
Enhancing Mission Success – A Framework for the Future

57

surveillance is in place for current programs and projects. These processes should be part of

every Center’s Quality Management System.

Surveillance can be streamlined to be responsive to particular risk or skill issues specific to

each project task or to specific suppliers. Efficiency can be realized by explicitly evaluating

these and other factors to establish the right level of surveillance.

NIAT-14: Verification and Validation

Current State

Verification and validation activities are hierarchical in nature, building from the part level

through the integrated system level. Verification typically includes use of breadboards,

simulators, development hardware and software, and finally the flight articles themselves.

NPG 7120.5 contains basic verification and validation requirements, but provides minimal

detail on “how” or “how much.” The fundamental assumption has been that each project

would establish adequate verification requirements and conduct the testing necessary to ensure

the success of the project, taking into consideration both the risk associated with the project

and the performance and operational requirements. Approval of the plans by higher level

authorities would constitute agreement with the assumptions and verification approach.

However, constrained resources have sometimes caused projects to depart from the

fundamental principle of “test as you fly and fly as you test.”

NIAT-14 Action Summary

Ensure that all projects plan and execute a thorough verification and validation (V&V)

program.

Figure 15. Verification and Validation.

Recommendation Reference: MCO-50, 51, 53, 54, 57, 59; MPIA-20, 28; SIA-21

Minimum Elements:

A. Revise NPG 7120.5 to provide additional guidance concerning V&V planning.

B. Conduct evaluation of V&V plans and execution for projects currently in

implementation.

C. Review/update Center-level procedures and guidelines for V&V planning and

execution.

 • Ensure that procedures and guidelines are sufficiently comprehensive in

 addressing hardware and software at all levels through the integrated system.

 • Confirm that review processes provide for thorough, independent assessment of

 the planning and execution of project V&V.

NIAT-14 Action: Ensure that all projects plan and execute a thorough verification

and validation (V&V) program for hardware, software, and hardware/software

systems [AE, Center Directors, PMCWG; 4/01].
Enhancing Mission Success – A Framework for the Future

58

NASA will revise NPG 7120.5 to provide additional guidance on V&V planning and

execution.

In addition, Center Directors will conduct an evaluation of V&V plans and execution for

projects currently in implementation. Center Directors will also review and update Center

procedures and guidelines for V&V planning and execution. Emphasis should be given to

thorough V&V of hardware, software, and integrated hardware/software systems, including use

of simulations, when appropriate.

NIAT-15: Management Responsibility and Accountability

Current State

NASA deployed an Agencywide NASA Policy Directive (NPD) 7120.4 in November 1996 for

Program/Project Management, and NPG 7120.5 in April 1998 for NASA Program and Project

Management Processes and Requirements. The processes and requirements defined by these

documents are an integral part of the Agency management system established to meet the goals

of the NASA Strategic Plan. This management system provides the overall framework to

govern the formulation, approval, implementation, and evaluation of NASA programs and

projects.

Program/project management responsibility in NASA spans the management chain from the

Administrator through the Associate Deputy Administrator, Associate Administrators, Center

Directors, Program Managers, Project Managers, and other line management officials, each of

whom has a role in ensuring consistency of program and project implementation with Agency

policies and procedures.

The project manager is responsible for project success. In support of the project manager, line

management has a responsibility for the technical accuracy of the project. This responsibility

is exercised through management review (see NIAT-10) and frequent interaction with team

members. Line management must also be knowledgeable of both technical information and

the program/project management process to adequately support and advise their employees to

proactively contribute to project success.

The report findings identify that management has the above responsibilities but that their

implementation has not always been sufficiently rigorous.

NIAT-15 Action Summary

Senior and line management at performing organizations must be knowledgeable of Agency

policies and guidelines and their traceability to Center specific processes, procedures and

policies. They are also responsible for ensuring the application of sound engineering and

management practices. This extends to providing all, but in particular less experienced,
Enhancing Mission Success – A Framework for the Future

59

managers and other leads the support necessary for their development while ensuring that those

practices are followed.

Management is ultimately accountable to implement the Agency’s policies and procedures and

provide guidance to others throughout the organization. They can only meet this responsibility

through more direct involvement in programs and projects. To enhance their performance and

improve support to the project teams, additional in-depth knowledge of the evolving

terminology, methodology, and lessons learned is needed.

Senior and line management must work closely with the Systems Management Offices at each

Center to provide continuous assessment and take proactive steps to support the project

managers and project teams. The Office of the Chief Engineer, as steward of NPG 7120.5, will

monitor the effectiveness of Agency policies and procedures in the support of management.

Figure 16. Management Responsibility and Accountability.

Recommendation Reference: MPIA-1, 3, 14, 15; FBC-10; MCO-14, 38; SIA-1

Minimum Elements:

A. Incorporate roles and responsibilities of Center senior and line management in

NPG 7120.5.

B. Develop focused training materials and workshops/forums for senior and line

management to become familiar with NPG 7120.5 and related documents.

C. Evaluate Center policies and processes to ensure traceability and consistency

with NPG 7120.5 and related documents.

D. Develop and put in place an assessment methodology to ensure that Project

Manager’s and leads have the training and institutional support necessary for their

success.

NIAT-15 Action: Ensure that Center senior and line management understand and

effectively exercise their continual role in supporting Project Managers and Project

Teams [AE, Center Directors, PMCWG, FT; 6/01].
Enhancing Mission Success – A Framework for the Future

60

Theme V—Improving Communication

Introduction

The NASA team is comprised of a diverse group of highly skilled men and women located

throughout the Nation with the charge to accomplish the Agency’s mission, goals, and

objectives. In most cases the team relies on multiple partners, including other Government

agencies, industry, and academia. The Agency must have a culture of teamwork and open

communications. To achieve that environment, we must continue to remove communication

barriers and foster an inclusive environment where open and candid communication is the

norm.

The essential knowledge for success is embedded in the systems and processes used within the

Agency and the skills of NASA employees and partners. This knowledge is what makes

NASA uniquely capable. It is not easy to capture and share information and key lessons across

the Agency. To succeed, NASA must sustain an open learning environment that is facilitated

through an effective communications process. This environment must capture and deliver

lessons learned to effectively facilitate the sharing of information and expertise and enable

knowledge management.

NIAT-16: Organizational Communication

Current State

With the diversity of needs, learning styles, and methods in place, the importance placed by

people on an open communications process is inconsistent across organizations and

organizational levels. NASA has a great challenge since our “can do” culture sometimes

constrains the communications process, especially when issues and problems are involved.

The spirit of “can do” drives employees to investigate all other avenues of solution until they

have been exhausted. By this time, it is sometimes too late to effectively implement corrective

or preventive actions. What enables our high rate of success also establishes barriers to an

open communications process.

Most program and project teams include communication mechanisms within the team structure

as a normal way of doing business. An example of a frequently used mechanism is that of

project reviews. They currently provide a structured forum internal to the team. External to

the team, they provide the project manager the opportunity to promote the understanding of

requirements, to balance and understand risks, and to obtain expert advice on specific issues.

Nevertheless, improvements are needed to strengthen communications both inside and outside

the project structure.

One of the most important enablers to open communication is trust. Team members must feel

free to express concerns without fear and openly communicate potential risks and issues. A

key responsibility of program and project managers is to clearly communicate regularly to their

management, as well as customers and stakeholders and they must ,therefore have that
Enhancing Mission Success – A Framework for the Future

61

competency. In return, their management must be open to discussion and practice effective

communication skills. Management must advocate prudent risk taking that does not

compromise safety considerations as a tenet by which the Agency approaches its missions, and

it must accept the occasional downside outcomes that may result from doing so. Barriers that

can inhibit effective communication, such as lack of effective tools, travel constraints,

organizational and cultural barriers, fear, and lack of trust, must be minimized.

NIAT-16 Action Summary

NASA should continue to remove communication barriers and foster an inclusive environment

where open communication is the norm.

Figure 17. Organizational Communication.

The objective of this action is to establish an initiative similar to the Agency Safety Initiative

where the importance of communication and a culture of trust and openness permeate all facets

of the organization. Multiple processes to get the messages across the organizational structure

need to be explored and encouraged. In addition, avenues to clearly define and communicate

roles and responsibilities to all need to be developed. The Agency should solicit expert advice

in identifying and removing barriers; provide tools, training and education; and the means to

facilitate the communication process.

Recommendation Reference: MCO-4, 5, 8, 12, 13, 14; FBC-12; MPIA-18, 27; SIA-1

Minimum Elements:

A. Emphasize the importance of communication by developing a strategy similar to

the NASA Safety Initiative to permeate the workforce culture.

B. Acquire expert advice and consultation on root causes and potential barriers to

communication throughout the NASA organization.

C. Enhance PMDP competencies to include communication skills and best practices

training.

D. Discuss with teams findings of the NASA Project Team Development study and

utilize self-assessment tool available across the Internet, especially the effective

communications indicator.

E. Consider implementation of feedback mechanisms (such as 360-degree

appraisals) so employees can provide input on leader’s effectiveness in

communication.

F. Hold periodic symposia to openly discuss issues, program status, and to highlight

successful practices.

G. Educate all managers on the importance of totally supporting team members.

H. Expand development and implementation of training process for team facilitation

to build more internal communication facilitators.

NIAT-16 Action: Continue to remove communication barriers and foster an

inclusive environment where open and candid communication are the norm

[AE, F, FT, Center Directors ,Project Managers; 6/01].
Enhancing Mission Success – A Framework for the Future

62

NIAT-17: Knowledge Management

Current State

NASA has long recognized the importance of managing its corporate knowledge. The

processes of generating and communicating NASA’s knowledge are considered core processes

that cut across organizational boundaries. NASA has made strides toward improving the way

in which knowledge is captured, organized, and stored. A NASA Knowledge Management

(KM) Team, chartered by the Chief Information Officer with members from multiple

disciplines, has been formed to develop an approach to KM. NASA has a set of KM activities

led by this team, including some pilot projects. These pilots focus on:

• Increasing access to and archiving of information within communities across the Agency

through customized portals,

• Improving the capture and reuse of lessons learned by augmenting the current Lessons

Learned Information System (developed by NASA’s SMA), and

• Locating experts across NASA.

In addition to the KM team’s efforts, the Academy of Program/Project Leadership (APPL)

has several KM efforts that are in their early stages of implementation. APPL’s Project

Management Coach online site includes guidance, points of contact, samples, templates, and

training information on a variety of topics. APPL also provides a number of mechanisms for

program and project managers to share experiences and knowledge through training classes

and other forums.

Although NASA’s efforts so far are commendable, the Agency must go further. In the current

environment, effective management and sharing of knowledge is more critical than ever. The

experience of prior managers is not uniformly well documented and made available for the

benefit of newer or less experienced program and project managers to effectively utilize in

their situations.

The report findings indicate a lack of access to and process for using lessons learned.

Continuous knowledge capture was cited as necessary to promote communications and

learning. In addition, checklists were recommended to facilitate the management and review

of programs and projects.
Enhancing Mission Success – A Framework for the Future

63

NIAT-17 Action Summary

Promote the continuous capture, dissemination, and utilization of knowledge.

Figure 18. Knowledge Management.

Better alignment of NASA’s knowledge base with the needs of the Agency and its program

and project managers, as well as its technical personnel, is required. NASA must ensure that

its KM tools facilitate the capture of effective and appropriate information that can be reused in

a beneficial way. Examples of the useful application of KM tools include:

• Providing tools and training to make the formal process for reporting problems and

anomalies user friendly,

• Continuing Agency lessons learned symposia and working groups, and

• Maintaining a database of lessons learned and infusing the knowledge gained into the daily

routines of NASA personnel.

The information gained in these KM activities will help NASA to understand how it can more

aggressively proceed with managing and sharing its knowledge among employees and with

external partners. The desired end-state is for NASA to become a true community of learning,

where knowledge is used to help the Agency and its partners learn and progress. In order to

Recommendation Reference: MCO-15, 48, 55, 56, 68, 79; FBC-24, 25, 26, 27, 28, 29

Minimum Elements:

A. Develop, with customers, a new lessons learned tool to effectively capture and

utilize lessons learned including:

• Definition of the information (lessons and the project environment) that is relevant

 to the current environment.

• Determination of how to capture and organize information for ready access

 – Reduce overhead and improve timeliness of posting.

 – Include mechanism for rapidly developing information in a useful and

interesting format.

• Identify external organizations which excel in knowledge management and 

 determine best practices.

• Select a pilot project to test and refine criteria, including a customer assessment

• Address the cultural and process changes needed to encourage and reward

 knowledge sharing.

• Incorporate into relevant training.

B. Capture checklists in LLIS database and APPL online tools to facilitate ready

 reference for project teams and independent reviewers.

NIAT-17 Action: Promote the continuous capture, dissemination and utilization of

knowledge and make checklists available to support project managers. [AO, FT, Q, AE,

Center Directors; 6/01].
Enhancing Mission Success – A Framework for the Future

64

achieve this, NASA must strive to promote a culture of knowledge sharing throughout the

organization.

A primary mechanism for the promotion of knowledge management within the Agency will be

improved lessons learned tools, developed in coordination with users of the system that draw

on improved data base technology using advanced search engines. Three critical aspects are

needed to ensure the success of the improved tool: input of new lessons learned information,

knowledge of the environment and applicability of the lesson, and review of past lessons

learned by programs and projects that will lead to changes in behavior. The system will

capture lessons learned information from programs, projects, and missions, with the goal of

ensuring that NASA does not have to keep “relearning” the lessons of the past. Further, the

lessons learned system and process will be designed to affect changes in day-to-day operations

and mission activities through ease of access and well organized information. For example, the

application of previous lessons learned might lead to changes in current procedures, standards,

and information management techniques in a new mission or project. The key to success in the

application of lessons learned is communicating the lessons to people who will benefit from

them.

To ensure successful deployment of the lessons learned tool, it will be developed first as a pilot

project, allowing system users to provide feedback on the effectiveness of the system and the

knowledge capture process. This pilot will also provide an opportunity to infuse lessons into

everyday processes and systems. It will be important to avoid broadcast transmission of every

lesson learned to every manager; targeting of communications to the appropriate persons will

be crucial.

The lessons learned tool will constitute only one dimension of NASA’s expanded focus on

KM. The Agency will benchmark other organizations that excel in the ability to share internal

knowledge to provide measurable improvements. The Agency’s KM initiative will also

identify best practices with applicability to the NASA environment. To bring about true

change, NASA will promote an atmosphere of knowledge sharing in its formal and informal

cultures and work processes, and reward the open dissemination and sharing of information.

Finally, NASA will ensure that KM concepts in general, and the use of the lessons learned

system in particular, are incorporated into Agency training classes and programs.

The continuous capture and application of project knowledge and lessons learned must become

a core business process within the Agency’s program and project management environment.

Regular input into NASA’s knowledge bases, such as the lessons learned database, should be

emphasized. Programs and projects should implement a “document-as-you-go” philosophy,

promoting continuous knowledge capture for the benefit of current and future missions. More

importantly, program and project managers must regularly utilize the knowledge management

tools to apply previous lessons learned to their own projects. The Agency can provide help for

individuals to understand, learn from, and apply the lessons of others to their own work as part

of a daily routine.
Enhancing Mission Success – A Framework for the Future

65

The NIAT recommends that checklists from the MCO report and others be made available to

management and project teams. If readily available, Center management, project management,

and others can develop project-unique checklists as an aid to internal and external

communication. NIAT does not recommend an Agency-level requirement for the generation

of checklists. The tendency could be to focus on the list exclusively and not be as aware of

new issues as they arise.
Enhancing Mission Success – A Framework for the Future

66

Other Recommendations Assessed

Eleven recommendations considered by NIAT were found to be adequately addressed by

existing NASA policy, policy changes in process, organizational changes already completed,

and budgets already proposed. These recommendations and a description of the NIAT

assessment are provided in Figure 19.

Figure 19. Recommendations Assessed by NIAT Requiring no Additional Action.

Recommendations Requiring No NIAT Action: FBC-13,16, 38, 45; MCO-20, 31,

33, 70; SIA-7, 18, 22,

• FBC 13 Continually evaluate the effectiveness of NASA policies, rules, 

procedures, etc., like what is being accomplished for NPG 7120.5.

 – In ISO formal process now in place at HQ and Centers. Specific issues

 addressed in other NIAT actions.

• FBC 16 Ensure that Programs and Projects develop the cost and schedule

caps commensurate with mission scope by working their estimates from the

bottom up with all members of the team participating, who then own the Project

Plan.

 – Included in proposed changes in NPG 7120.5B 2.1.1.2.c; 3.1.1.2.b; 4.1.1; 4.3

• FBC 38 Form a Technology Office led by a results-oriented Chief Technologist

Officer with as much stature and clout as Enterprises.

 - Effective 02/2000, Chief Technologist was also appointed Associate

 Administrator for Aerospace Technology Enterprise.

• FBC 45 Reduction of launch costs [must be] a national priority to drive FBC.

 - Space Transportation Architecture studies over the past 2 years in close

 concert with the Administration has produced NASA’s Integrated Space

 Transportation Plan; a $4.5 billion multi-year investment submitted in the

 President’s FY 2001 Budget.

• MCO 20 Define program architecture at the beginning of a program by means

 of a thorough mission formulation process.

 – See NPG 7120.5, Paragraph 2.1 and E.3.

• MCO 31 Each mission should maintain a formal record of risk factors to 

mission success in the form of a risk list.

 – See description of the risk management process NPG 7120.5B, 4.2.2.a & b.

• MCO 33 The risk management process should thoroughly address the question

 of “what could go wrong” early in the project.

 – See description of the risk management process in NPG 7120.5B, 4.2.2.a & b.

• MCO 70 The New Millennium Program or its equivalent should be adequately

funded to provide flight testing opportunities.

 • FY01 Budget submission reflects restructured and adequately funded

 New Millennium Program. Program has been realigned to emphasize

 technology flight demonstration requirements over science data

 objectives in project selection.

• SIA 7, 22 Failure analysis and incident investigation should identify root cause

 and not be artificially limited to a subset of possible causes.

 – See NPG 8621.1.g., “Mishap Reporting and Corrective Action”.

• SIA 18 Where redundancy is used to mitigate risk, it should be fully and carefully

 implemented and verified. If it cannot be fully implemented due to design

 constraints, other methods of risk mitigation must be utilized.

 – See description of the risk management process in NPG 7120.5B, 4.2.2
Enhancing Mission Success – A Framework for the Future

67

Conclusions

The NIAT has reviewed and assessed a total of 165 findings of the referenced reports. The

product of that assessment is a set of 17 actions that provide an integrated framework for longterm improvement of the planning and execution of NASA’s programs and projects. These

actions will drive the Agency toward the required state of excellence through a continual

improvement process that engages all levels of NASA. Our challenge is to clearly

communicate not only the action statements, but also the intent behind them. This report

endeavors to do that.

Excellence in communication is essential for the success of this effort, and any other, within

the NASA community. Communication is the key ingredient in the understanding and control

of prudent risk deemed acceptable in this challenging realm that must be at the forefront now

and in the future.

We cannot hope to achieve the ambitious goals of the Strategic Plan without success in our

programs and projects and the engagement of the entire team: NASA, contractors, partners,

academia, customers, and stakeholders. NASA has a history of achievement. Our ability to

continue that legacy is vested in the preparation, support, and performance of an excellent

workforce. We must ensure, through careful planning and implementation, that we create the

open environment for creativity, innovation, and challenge in which people thrive.
Enhancing Mission Success – A Framework for the Future

68

Appendix A—NIAT Membership

Beth McCormick

Olga D. Gonzalez-Sanabria

Howell Hilton

Anngie Johnson

Renee Higgins

Tony Maturo

Joanne Mueller

Ed Hoffman

Michael Greenfield

John Whiteley

Randy Taylor

Bill Piland

Lynn Bailets

Harvey Schabes

Keith Hudkins

Richard Day

Norm Haynes

Lia LaPiana

Gary Krier

Larry Schultz

Paul DeMinco

Wally Sawyer

Mike Conley

Jo Gunderson

Andrea Norris

Ron Johnson

Rob Anderson

Greg Reck

Len Sirota

Jeanne Holm

Chuck Smith

Michael Gilbert

Nicola Muscettola

Gordon Johnston

J.C. Duh

Stephen Prusha

Richard Gilbrech

2 Hq AE

2 Hq AO

1 Hq R

2 Hq F

1 Hq H

1 Hq Q

1 Hq M

2 Hq R

2 Hq S

1 Hq U/Z

3 Hq Y

2 ARC

2 DRFC

2 GRC

2 GSFC

4 JPL

2 JSC

2 KSC

3 LaRC

4 MSFC

1 SSC

Organizations:

People Process

Process

Execution

Advanced Technology

& Tools

Robert McBrayer, Orlando Figueroa, Keith Hudkins, Jo Gunderson,

Marcus Watkins, Ed Hoffman, Nancy Kaplan

Carolyn Griner, Chair

Integration
Enhancing Mission Success – A Framework for the Future

69

Appendix B— NIAT Actions Versus Recommendations

Niat No. Action/Objective Recommend Recommendation TextNIAT-1.1 Provide a physically and psychologically safe FBC-35 Balance Workload (number and/or scope of projects) to as

and healthy work environment for all NASA

SIA-4 Work teams should be supported through improved emplo

effect on health and work, Workload and "overtime" pressrealistic planning and scheduling; a serious effort to prese

NIAT-2.1 Ensure that teams are composed of competent FBC-6 Promote mobilization of key personnel among Centers and 

personnel through expanded and disciplined use of the PMDP, selecting the right team skills forthe project life cycle, and better definition of theroles and responsibilities of the Project Manageras leader of the team.

FBC-8 Assign the HQ Safety and Mission Assurance Office the re

Workshop to effect better NASA teaming arrangement – in

FBC-9 Form and motivate an excellent team, a mix of experience 

enthusiasm and new methods

FBC-35 Balance Workload (number and/or scope of projects) to as

MCO-1 Project Managers selected based on experience gained oMCO-2 Provide on-the-job mentoring for Project Managers from eMCO-6 Establish and fully staff a comprehensive systems enginee

MCO-7 Project Manager to determine and insist on an appropriate 

contractor

MCO-8 Increase inter-center participation and technology sharing 

subsystem levels, focused around Center of Excellence ar

MCO-9 Ensure that teams foster an environment of commitment a

don't fit be replaced

MCO-10 A cohesive team must be developed and involved in the pMCO-11MCO-12MCO-14 Roles, responsibilities and accountabilities must be made 

project and a visible leader appointed over the entire oper

MCO-23 Assign adequate systems engineers not only at the projecMCO-29 Engage operations personnel early in the project, preferabMCO-30 The science representatives must be full members of each 

throughout the life cycle of the mission

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

70

Niat No. Action/Objective Recommend Recommendation TextNIAT-2.1 Ensure that teams are composed of competent MPIA-1 The respective institutions are responsible for assuring tha

personnel through expanded and disciplined use fully trained and/or a mentor is assigned to them of the PMDP, selecting the right team skills forthe project life cycle, and better definition of theroles and responsibilities of the Project Manageras leader of the team.

MPIA-2 Ensure that the team has the right skills, is adequately staMPIA-3 Empower the project manager to access Agencywide insti

enhance mission success.

MPIA-6 The project management should have end-to-end respons

development and operations for management of the total mand management of risk

MPIA-9 The Program Office must have the flexibility to realign and 

and flight project elements to meet the mission success cr

MPIA-14 Institutional management must be accountable for policies 

level of mission success

MPIA-15 Institutional management must assure project implementa

and procedures

MPIA-26 Scientists must participate in all stages of project formulati

science goals are understood and taken fully into account

MPIA-27 Clear lines of responsibility and authority should be establ

SIA-2 Program/projects should assure that critical functional areaSIA-4 Work teams should be supported through improved emplo

effect on health and work, Workload and "overtime" pressurealistic planning and scheduling; a serious effort to preserv

NIAT-2.2 Improve the hands-on experience, training FBC-1 Certify Project Managers, and Teams as to experience an

curriculum and mentoring for project managersand engineers.

FBC-2 Conduct the FBC Training Workshops throughout NASA FBC-3 Expand the role and clout of NASA ‘s Academy of Program FBC-4 Generating training material for FBC Training workshops f

teams which is first subjected to a "dry run" in front of expeeach Center, Industry and Academia

FBC-5 NASA must retain the expertise to do in-house Projects. T

sustaining expertise that is the foundation for space explor

FBC-6 Promote mobilization of key personnel among Centers and FBC-7 Acquire outside help on cultural change, core competency

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

71

Niat No. Action/Objective Recommend Recommendation TextNIAT-2.2 Improve the hands-on experience, training FBC-8 Assign the HQ Safety and Mission Assurance Office the re

curriculum and mentoring for project managers Workshop to effect better NASA teaming arrangement – inand engineers.

FBC-9 Form and motivate an excellent team, a mix of experience 

enthusiasm and new methods

FBC-11 Develop "Badges of Courage" for all ProjectsFBC-29 Continue symposiums on lessons learned, re-engineering

change, Teaming, etc., bringing in experts from within/outs

FBC-35 Balance Workload (number and/or scope of projects) to as

MCO-2 Provide on-the-job mentoring for Project Managers from eMCO-3 Formal certification training program in Project Manageme

Chief Systems Engineer, etc.

MCO-8 Increase inter-center participation and technology sharing 

subsystem levels, focused around Center of Excellence ar

MPIA-1 The respective institutions are responsible for assuring tha

fully trained and/or a mentor is assigned to them

MPIA-2 Ensure that the team has the right skills, is adequately staMPIA-3 Empower the project manager to access Agencywide insti

enhance mission success.

MPIA-6 The project management should have end-to-end respons

development and operations for management of the total mand management of risk

MPIA-9 The Program Office must have the flexibility to realign and to 

and flight project elements to meet the mission success crite

MPIA-14 Institutional management must be accountable for policies 

level of mission success

MPIA-15 Institutional management must assure project implementa

and procedures

MPIA-26 Scientists must participate in all stages of project formulati

science goals are understood and taken fully into account

MPIA-27 Clear lines of responsibility and authority should be establ

SIA-2 Program/projects should assure that critical functional areaSIA-4 Work teams should be supported through improved emplo

effect on health and work, Workload and "overtime" pressrealistic planning and scheduling; a serious effort to prese

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

72

Niat No. Action/Objective Recommend Recommendation TextNIAT-3.1 Develop and implement a comprehensive plan to FBC-19 Establish upfront agreements and maintain them

ensure excellence in Agency engineeringcapability, including hands-on work experience,and the development and application ofadvanced engineering tools and methodologiesin an advanced engineering and program/project management environment

FBC-36 Clearly define and resolve Center Core Competency and C

operations including In-House work policy

FBC-43 A common multi-Agency Internet Store for high quality, mo

by multiple vendors a National Priority

MCO-21 Changes to the baseline should be avoided to the maximuMCO-25 Systems engineer should ensure that project requirements MCO-27 Develop a comprehensive set of mission requirements ear

thorough flowdown of these requirements to the subsystem 

MCO-71 Take advantage of useful Integrated Synthesis Environme

available

MPIA-7 Appropriate levels of systems engineering need to be in pl

implementation phases of all projects

MPIA-11 A clearly defined mission success criteria needs to be esta

NIAT-4.1 Invigorate the Agency's current technology FBC-31 Strengthen NASA HQ Management by assigning a NASA 

investment to ensure that a balanced portfolio of a NASA Center Teaming Office to bring NASA into the 21s existing, new and emerging technology is as follows: Effective NASA HQ relationships with Centers, 

Core Competency & Center of Excellence Issues and Opera

FBC-32 Centers to focus on a few core competencies for which they 

Centers, government agencies, industry and academia for 

FBC-36 Clearly define and resolve Center Core Competency and C

operations including In-House work policy

FBC-37 Place higher priority on funding and supporting University 

and their space flight Missions

FBC-38 Form a Technology Office led by a results-oriented Chief T

much stature/clout as Enterprises

FBC-39 Balance research and advanced technology development 

 Provide "seed corn" for research and advanced developm

FBC-40 Hold yearly "Out of the Box" technology workshopsFBC-41 Balance competition of technology development with placi

NASA Centers of Excellence.

FBC-45 Reduction of Launch costs a National priority to drive FBC

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

73

Niat No. Action/Objective Recommend Recommendation TextNIAT-4.1 Invigorate the Agency's current technology FBC-46 Develop a NASA (Integrated) Information Technology Plan 

investment to ensure that a balanced portfolio of Intelligent Synthesis Environment, Information Technology existing, new and emerging technology is Super Computing Management Office, and all related IT a

MCO-64 Adequate funding for technology development aimed at brMCO-67 Technology needs should be expediently funded and met MCO-70 The New Millennium program or its equivalent should be a

testing opportunities

NIAT-5.1 Strengthen NASA's technology planning FBC-42 Develop a better NASA Integrated Technology (Developm

process and update the NASA Technology Plan.

FBC-46 Develop a NASA (Integrated) Information Technology Plan 

Intelligent Synthesis Environment, Information Technology Super Computing Management Office, and all related IT a

MCO-66 Program management should review specific future missio

requirements early

MCO-69 Technology needs, and risks, should be identified, funded

increasing TRL to appropriate level) prior to project initiatioa project starts, technology insertion should be low risk. (A

MCO-72 Missions should aggressively integrate leading-edge techn

cost and project risk. Specific examples are included reco

MCO-74 Pursue soft-computing software technology, such as neura

 learn and adapt to changes in the environment.

MCO-75 Develop autonomous operations and avionics technology, 

and improving onboard fault detection and recovery.

MCO-76 Pursue development of leading-edge technologies leading t

cost-cutting measures and improve operating capability.

MCO-77 Pursue development of leading-edge technologies leading 

those afforded by advanced propulsion systems and lightw

NIAT-6.1 Increase the speed and effectiveness of FBC-40 Hold yearly "Out of the Box" technology workshops

developing and infusing leading-edgetechnologies into missions.

MCO-66 Program management should review specific future missio

requirements early

MCO-67 Technology needs should be expediently funded and met MCO-69 Technology needs, and risks, should be identified, funded

increasing TRL to appropriate level) prior to project initiatioa project starts, technology insertion should be low risk. (A

MCO-72 Missions should aggressively integrate leading-edge techn

cost and project risk. Specific examples are included reco

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

74

Niat No. Action/Objective Recommend Recommendation TextNIAT-6.1 Increase the speed and effectiveness of SIA-26 Ensure early involvement of the project team with the tech

developing and infusing leading-edge and ownership of new technology by the projects.technologies into missions.

NIAT-7.1 Improve and enhance NASA and contractor FBC-19 Establish upfront agreements and maintain them

knowledge and the ability to identify, assess,mitigate and track risk through the definition ofsuccess criteria, acceptable risk, utilization ofexisting and new tools, and proper policy andguidance.

FBC-21 Define "Allowable Risk" as function of Program/ProjectFBC-44 Develop advanced tools to assess, analyze, manage, and 

MCO-18 All critical flight phases be fully instrumented to support deMCO-19 Establish a concise set of mission-success criteria early in MCO-21 Changes to the baseline should be avoided to the maximuMCO-22 Insertion of a new project definition phase, after the prime 

allows for a thorough reassessment of cost, schedule, con

MCO-24 Continually perform system analyses necessary to explicit

communicate these risks to all segments of the project tea

MCO-25 Systems engineer should ensure that project requirements MCO-27 Develop a comprehensive set of mission requirements ear

thorough flowdown of these requirements to the subsystem 

MCO-31 Each mission should maintain a formal record of risk factors 

risk list

MCO-32 Failure Mode and Effects Analysis, fault Tree Analysis and 

should be used to develop quantitative risk estimates

MCO-33 The risk management process should thoroughly address 

early in the project

MCO-34 Identify and control risk from the start of each project by tr

course of the project

MCO-35 The mission risk profile should become a part of each proj

reviewed at all periodic center and external reviews

MCO-36 A team should be formed to refine the implementation of rMCO-37 Acceptable risk must be defined and quantified, wherever 

 the team and the organization to guide all activities in the 

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

75

Niat No. Action/Objective Recommend Recommendation TextNIAT-7.1 Improve and enhance NASA and contractor MCO-37 Acceptable risk must be defined and quantified, wherever 

knowledge and the ability to identify, assess, the team and the organization to guide all activities in the 

mitigate and track risk through the definition ofsuccess criteria, acceptable risk, utilization ofexisting and new tools, and proper policy andguidance.

MCO-39 Projects and line organizations need to be extremely vigila

First attitude propagates through all levels of the organizat

MPIA-4 Risk must be assessed and accepted by all accountable p

program management, and project management

MPIA-5 All projects should utilize established risk management too

failure effects and criticality analysis

MPIA-8 Each involved organization should establish a policy requi

through mission-critical events

MPIA-11 A clearly defined mission success criteria needs to be estaMPIA-13 Senior management needs to establish that risk associate

innovation is acceptable as is risk associated with pursuing with deviating from sound management and engineering p

MPIA-19 Contractor responsibilities must include formal notification 

deviations from acceptable practice

SIA-9 Quantitative methods of risk assessment (likelihood of failu

SIA-10 Risk assessment matrix and Failure Modes and Effects Ana

flight failure experience, aging and maintenance history, and 

SIA-12 Quantitative methods of risk assessment and safety need t

 to perform trade-off studies on the effect of new technologetc., upon vehicle risk

SIA-15 NASA and NASA contractors should develop a Risk Mana

communicating risk as an integrated effort

SIA-16 The Risk Management Plan would flow SSP expectations 

level engineers and technicians, and provide insight and re

SIA-18 Where redundancy is used to mitigate risk, it should be ful

verified. If it cannot be fully implemented due to design comitigation must be utilized

SIA-22 Failure analysis and incident investigation should identify r

to a sub-set of possible causes

NIAT-8.1 Improve NASA Safety and Mission Assurance FBC-15 Expand Safety and Mission Assurance responsibilities at N

to provide life cycle rigor in the formulation and verifying: Team Certification, Risk Signatures, FBC Perforimplementation of flight systems. Start, Launch Operations, Compliance to FBC Lessons Le

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

76

Niat No. Action/Objective Recommend Recommendation TextNIAT-8.1 Improve NASA Safety and Mission Assurance MCO-26 A strong mission assurance function should be present in 

to provide life cycle rigor in the formulation andimplementation of flight systems.

SIA-7 The Safety & Mission Assurance role should include: man

Prevention/Resolution Teams and in problem categorizatiocatches and dissemination of lessons learned

SIA-17 Quantitative measures of safety (likelihood of error), includ

should be developed, e.g., Occupational Stress Inventory 

SIA-19 NASA Safety and Mission Assurance oversight should be 

NIAT-9.1 Revolutionize the process of developing and MCO-28 The definition of mission-critical software for both ground a

delivering safe, reliable, quality software and software development process to provide a check-and bal

improve transfer of new software methods andtools into NASA practice.

NIAT-10.1 Define and rigorously implement a refined and FBC-17 Establish better reality checks of the feasibility of impleme

reinvigorated NASA Integrated Review Process resources allocatedfor Programs/Projects, which will:– Clarify an integrated continuum of reviewsand associated rollup of value-added reviewproducts– Specify required reviews and classes ofreviews for tailoring byEnterprise/Center/Program/Project including“peer” and "independent” reviews– Clarify objectives and expectations for each

review/review class– Identify the review “customer” and specify

participants, roles, and responsibilities

– Address independence and expertiserequirements for reviewers; continuity anddiscipline area penetration assignments of

FBC-22 Consolidate all Independent Review objectives into one In

Programs and Projects

FBC-23 Ensure Careful upfront peer reviewed planning, design, an

MCO-41 The choice in choosing the right experts to participate in re

as well as the project manager

MCO-42 That a review no longer be checked off until the right peopMCO-43 Peer review results should be presented at all external revMCO-44 Standing external review boards should be appointed for e

continuity and greater familiarity with the subject matter

MCO-45 Rigorous discipline must be enforced in the review proces

skill mix of personnel for all disciplines involved in the subj

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

77

Niat No. Action/Objective Recommend Recommendation TextNIAT-10.1 Define and rigorously implement a refined and MCO-46 The external review process should include the reporting o

reinvigorated NASA Integrated Review Process and other issues.

for Programs/Projects, which will:– Clarify an integrated continuum of reviewsand associated rollup of value-added reviewproducts– Specify required reviews and classes ofreviews for tailoring byEnterprise/Center/Program/Project including“peer” and "independent” reviews– Clarify objectives and expectations for eachreview/review class– Identify the review “customer” and specifyparticipants, roles, and responsibilities– Address independence and expertiserequirements for reviewers; continuity anddiscipline area penetration assignments of

MCO-47 Support from other centers for review teams should be incMCO-49 Funding and schedule allocations for executing reviews an

resulting from reviews should be baselined at the start of t

MCO-51 Final end-to-end verification and validation of all mission-c

performed

MCO-52 All parties should make use of the project's established pro

resolution of all issues raised during reviews. (Per 4/13 Te

MCO-58 Before every launch, a full operations team should be asse

and contingency operational scenarios

MCO-60 A core set of operations personnel should be assigned to ea

MCO-61 A core set of development personnel should be defined foMCO-62 Appoint a deputy project manager for operations at the be

trade studies properly consider the development and oper

MCO-63 A core group of system developers and systems engineer

operations team in developing nominal and contingency poperational

MPIA-17 Ensure that the best people are available and provide con

 team should be comprised of a combination of experience

MPIA-21 Conduct thorough technical and programmatic reviews be

assembly and subsystem, and through the full up system.

MPIA-22 All issues identified in programmatic and technical reviews 

to ensure complete, timely and accurate closure.

MPIA-24 Review teams should strive for continuity through the proje

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

78

Niat No. Action/Objective Recommend Recommendation TextNIAT-10.1 Define and rigorously implement a refined and MPIA-25 Competent and efficient reviews of projects by experts from 

reinvigorated NASA Integrated Review Process implementing institutions should provide overall assessme

for Programs/Projects, which will: evaluation of risks– Clarify an integrated continuum of reviewsand associated rollup of value-added reviewproducts– Specify required reviews and classes ofreviews for tailoring byEnterprise/Center/Program/Project including“peer” and "independent” reviews– Clarify objectives and expectations for eachreview/review class– Identify the review “customer” and specifyparticipants, roles, and responsibilities– Address independence and expertiserequirements for reviewers; continuity anddiscipline area penetration assignments of

SIA-20 An independent review process, utilizing NASA and other 

NIAT-11.1 Ensure provision of adequate program and MCO-17 Programs should be sufficiently funded to ensure mission 

project resources throughout the life cycle.

MPIA-12 Ensure that projects have adequate cost, schedule, and p

 upon baseline to meet the mission success criteria.

NIAT-12.1 Define “Faster, Better, Cheaper” and incorporate FBC-20 Define FBC and the application of it

 into NPG 7120.5

NIAT-13.1 Ensure that programs and projects receive FBC-18 Treat "all" NASA work like Projects, including advanced 

adequate surveillance based on risk knowledge schedule can be produced which targets periodic Peer Rand that there is ample opportunity for the possible need for new direction

communication of issues and concerns.

FBC-30 Reporting requirements and processing and reporting proc

ambiguities, conflicts, and omissions, and the audit or revi

FBC-33 Address impact of erosion of Industry Capability and the b

responsibilities and Oversight while allowing innovation to 

FBC-34 Maintain open communication relative to Risk and Effectiv

sound Technical, Cost, Schedule)

MCO-16 Representative of the program office periodically review alMCO-40 A proper balance of contractor and project oversight by tec

 is required to ensure mission success and to develop a sethe institution

MPIA-10 Ensure senior management is engaged in supporting projeMPIA-16 NASA Headquarters and Program management should ha

objectives, requirements, constraints, and risk assessmen

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

79

Niat No. Action/Objective Recommend Recommendation TextNIAT-13.1 Ensure that programs and projects receive MPIA-22 All issues identified in programmatic and technical reviews 

adequate surveillance based on risk knowledge to ensure complete, timely and accurate closure.and that there is ample opportunity for thecommunication of issues and concerns.

MPIA-23 Ensure that the appropriate level of oversight is employed 

NIAT-14.1 Ensure that all projects plan and execute a MCO-50 From the simplest component or module to the most comp

thorough verification and validation (V&V) validation conducted via simulation or testing of hardware/program for hardware, software and tractability and compliance with mission and derived requihardware/software systems.

MCO-51 Final end-to-end verification and validation of all mission-c

performed

MCO-53 Conduct extensive testing and simulation in conditions and 

conditions as possible

MCO-54 Integrated tests across subsystems should be planned ea

development hardware and simulations

MCO-57 Hardware/software integration tests should be performed u

identify integration issues early in code development

MCO-59 The operation team should use high-fidelity simulators to p

validate all nominal and contingency procedures, assess smission preparedness

MPIA-20 Plan and execute a thorough verification and validation pro

assembly and through the full up system.

MPIA-28 Perform independent verification and validation of critical e

including non-testable conditions (through thorough ana

SIA-21 Testing must be carefully scrutinized to ensure adequate sim

applicability to multiple sub-system, and complete docume

NIAT-15.1 Ensure that Center senior and line management FBC-10 Implement more effective NASA HQ relationships with the 

understand and effectively exercise theircontinual role in supporting Project Managersand Project Teams

MCO-14 Roles, responsibilities and accountabilities must be made 

project and a visible leader appointed over the entire oper

MCO-38 The line organization managers and project managers be 

success or failure of a mission, within their appointed area 

MPIA-1 The respective institutions are responsible for assuring tha

fully trained and/or a mentor is assigned to them

MPIA-3 Empower the project manager to access Agencywide insti

enhance mission success.

MPIA-14 Institutional management must be accountable for policies 

level of mission success

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

80

Niat No. Action/Objective Recommend Recommendation TextNIAT-15.1 Ensure that Center senior and line management MPIA-15 Institutional management must assure project implementa

understand and effectively exercise their and procedures

continual role in supporting Project Managersand Project Teams

SIA-1 Communications between the rank and file work force, sup

should be improved

NIAT-16.1 Continue to remove communication barriers and FBC-12 Foster an environment where open, candid, thorough com

foster an inclusive environment where open and candid communication are the norm.

MCO-4 Ensure that teams maintain full communication with contra

 or geographical barriers

MCO-5 Workers should be trained to detect, broadcast, interpret a

level necessary until resolved

MCO-8 Increase inter-center participation and technology sharing 

subsystem levels, focused around Center of Excellence ar

MCO-12 Project Managers to foster an environment where problemMCO-13 Communications meetings must be regular and frequent, a

entire project team, including contractors and science elem

MCO-14 Roles, responsibilities and accountabilities must be made 

project and a visible leader appointed over the entire oper

MPIA-18 Senior management must be receptive to communications MPIA-27 Clear lines of responsibility and authority should be establ

SIA-1 Communications between the rank and file work force, sup

should be improved

NIAT-17.1 Promote the continuous capture, dissemination FBC-24 Maintain database of lessons learned and communicate 

and utilization of knowledge and make checklists available to support project managers.

FBC-25 Consolidating the findings of this report with the Mars Prog

Investigation Reports, deriving composite FBC Project LesEngagement and Project Implementation check lists

FBC-26 Integrate Rules of Engagement Metric into regular review 

 Projects

FBC-27 Assign JPL the responsibility of conducting a NASA-Wide 

to further evolve re-engineering products. Use the NASA already established.

FBC-28 Keep lessons learned in front at all times and provide menFBC-29 Continue symposiums on lessons learned, re-engineering

change, Teaming, etc., bringing in experts from within/outs

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

81

Niat No. Action/Objective Recommend Recommendation TextNIAT-17.1 Promote the continuous capture, dissemination MCO-15 Team members properly document lessons learned follow

and utilization of knowledge and make checklists available to support project managers.

MCO-48 The projects should implement an "document-as-you-go" p

the project to promote continuous knowledge capture

MCO-55 Every negative response to a checklist question should be 

action items, which have an associated timetable for resol

MCO-56 The checklist composed by the Mars Climate Orbiter Mish

apart of the project management process.

MCO-68 Provide tools and training to make the formal process for r

anomalies user-friendly

MCO-79 The checklist should be maintained, expanded and shared

z-NAR FBC-13 Continually evaluate the effectiveness of NASA policies, ru

accomplished for NASA 7120.5A

FBC-16 Ensure that Programs and Projects develop the cost and s

mission scope by working their estimates from the bottoms participating , who then own the Project Plan

FBC-38 Form a technology office led by a results-oriented Chief Te

stature and clout as Enterprises

FBC-45 Reduction of Launch costs [must be] a National priority to 

MCO-20 Define program architecture at the beginning of a proram b

formulation process.

MCO-31 Each mission should maintain a formal record of risk facto

risk list.

MCO-33 The risk management process should thoroughly address th

early in the project.

MCO-70 The new Millennium Program or its equivalent should be a

testing opportunities

SIA-7 Failure analysis and incident investigation should identify r

to a sub-set of possible causes

SIA-18 Where redundancy is used to mitigate risk, it should be ful

verified. If it cannot be fully implemented due to design comitigation must be utilized.

SIA-22 Failure analysis and incident investigation should identify r

to a sub-set of possible causes

Thursday, October 05, 2000 
Enhancing Mission Success – A Framework for the Future

82

Appendix C—Acronyms

ADA Associate Deputy Administrator

AE Office of the Chief Engineer

AM Office of the Chief Health and Medical Officer

AO Office of the Chief Information Officer

APPL Academy of Program and Project Leadership

ARC Ames Research Center

ASAP Aerospace Safety Advisory Panel

ATT Advanced Tools and Technology Team

B Office of the Chief Financial Officer

C Office of Headquarters Operations

CIC Capital Investment Council

CMM Capability Maturity Model

CoFR Certification of Flight Readiness

DRFC Dryden Flight Research Center

EAA Enterprise Associate Administrator

EAP Employee Assistance Program

ELV Expendable Launch Vehicle

EMC Engineering Management Council

EO Equal Opportunity

F Office of Human Resources and Education

FAR Federal Acquisition Regulation

FBC Faster, Better, Cheaper

FMEA Failure Mode Effects Analysis

FRR Flight Readiness Review

FT Training and Development Division

FTA Fault Tree Analysis

FY Fiscal Year

GPMC Governing Program Management Council

GRC Glenn Research Center

GSFC Goddard Space Flight Center

H Office of Procurement

HPCC High Performance Computing Capability

HQ NASA Headquarters

IDP Individual Development Plan

IS Intelligent Systems

ISO International Standards Organization

ISE Intelligent Synthesis Environment

ISS International Space Station

IT Information Technology

IV&V Independent Verification and Validation

JPL Jet Propulsion Laboratory
Enhancing Mission Success – A Framework for the Future

83

JSC Johnson Space Center

KM Knowledge Management

KSC Kennedy Space Center

LaRC Langley Research Center

LLIS Lessons Learned Information System

M Office of Space Flight

MCO Mars Climate Orbiter

MEDA Maintenance Error Decision Aid

MPIA Mars Program Independent Assessment

MSFC Marshall Space Flight Center

NASA National Aeronautics and Space Administration

NIAT NASA Integrated Action Team

NPD NASA Policy Directive

NPG NASA Procedures and Guidelines

NRA NASA Research Announcement

PM Program/Project Manager

PMCWG Program Management Council Working Group

PMDP Project Management Development Process

PRA Probabilistic Risk Assessment

Q Office of Safety and Mission Assurance

QRAS Quantitative Risk Assessment System

R Office of Aerospace Technology

R-CT Office of Aerospace Technology- Chief Technologist

RFP Request For Proposal

RM Risk Management

S Office of Space Science

SIA Shuttle Independent Assessment

SMA Safety and Mission Assurance

SMO System Management Office

SOW Statement of Work

SSC Stennis Space Center

SSP Space Shuttle Program

TLC Technology Leadership Panel

TRL Technology Readiness Level

U Office of Life and Microgravity Sciences and Applications (now

 Office of Biological and Physical Research)

Y Office of Earth Science

Z Office of Policy and Plans

V&V Verification and Validation