NASA-TM-112368

NASA Fast Track Study

Table of Contents

Executive Summary ................................................................................................................. ii

Survey/Feedback Tearout Sheet .............................................................................................. v

Acknowledgments .................................................................................................................. vi

Introduction/Background/Purpose ......................................................................................... 1

Methodology ........................................................................................................................... 3

Findings .................................................................................................................................. 7

A. Planning .............................................................................................................. 8

B. Teamwork ........................................................................................................ 10

C. Day-to-Day Management .................................................................................. 11

Conclusions ........................................................................................................................... 13

Recommendations ................................................................................................................. 15

Appendices

A.

B.

C.

D.

E.

APM-23 Draft Guide

APM-23 SSG Interview Compilation

SRI Space Interview Compilation

SKI Non-Space Interview Compilation

Supplemental Information

Strate_lic Resources, Inc. (SRI)

i

NASA Fast Track Study

Executive Summary

The NASA Fast Track Study supports the efforts of a Special Study Group (SSG) made up of

members of the Advanced Project Management Class number 23 (APM-23) that met at the

Wallops Island Management Education Center from April 28 - May 8, 1996. Members of the

Class expressed interest to Mr. Vern Weyers in having an input to the NASA Policy Document

(NPD) 7120.4, that will replace NASA Management Institute (NMI) 7120.4, and the NASA

Program/Project Management Guide. The APM-23 SSG was tasked with assisting in

development of NASA policy on managing Fast Track Projects, defined as small projects under

$150 million and completed within three years.

The approach of the APM-23 SSG was to gather data on successful projects working in a

"Better, Faster, Cheaper" environment, within and outside of NASA and develop the Fast Track

Project section of the NASA Program/Project Management Guide. Fourteen interviews and four

other data gathering efforts were conducted by the SSG, and 16 were conducted by Strategic

Resources, Inc. (SKI), including five interviews at the Jet Propulsion Laboratory (JPL) and one at

the Applied Physics Laboratory (APL). The interviews were compiled and analyzed for

techniques and approaches commonly used to meet severe cost and schedule constraints.

Strategic Resources, Inc. (SRI)

ii
NASA Fast Track Study

Findings Summary

From the analysis, the following findings were derived:

Within an improved Product Development Cycle (PDC) process that compresses the time needed

to go from concept to operation, three primary project management applications were discovered

necessary for Fast Track Project success:

1. Thorough planning (including Risk Planning, metrics for management and use of

concurrent engineering).

2. Teamwork (including use of Integrated Product Teams).

3. Minimizing disruptive outside events and their potential for consuming time and resources

(including extra reviews and second guessing team decisions).

This study found that success was not achieved by omitting steps in the Life Cycle Development

(LCD) process, but by innovatively tailoring the process to fit the constraints of the specific

project.

Additionally, related management applications that should be considered for use by each project

manager as applicable: the use of Risk Management; the use of facilitating technology or tools,

such as Rapid Prototyping, Computer Aided Design software, and communication tools; and,

training for specific project management applications, team operations, and available supporting

management information resources.

From these findings, recommendations for Fast Track Projects were formulated:

l. Require and allow time for the actual project manager and team to plan at the front end of

Fast Track Projects (especially entering Phase C/D), to include Risk Management

Planning and Design-to-Cost, with the understanding of the importance of controlling

project technical and programmatic requirements throughout the life cycle.

. Use Program Commitment Agreement concept at all levels of the project as a series of

bilateral agreements, used to ensure common requirement understanding and to control

requirement creep.

3. Use metrics for each project that will measure progress and value.

4. Use team management and cross functional integration as a tool to achieve success.

5. Educate project managers in innovative techniques for managing reviews, documentation,

oversight and risk to minimize disruptions in a schedule constrained project.

Strategic Resources, Inc. (SRI)

iii
NASA Fast Track Study

° Provide NASA Program/Project Managers access to an educational/corporate knowledge

information system addressing issues of Fast Track Project management, available as an

on-line resource to support both project management and career development.

Strategic Resources, Inc. (SRI)

iv

NASA Fast Track Study

Fast Track Study Feedback

Was the information helpful to you?

Why/Why not?

If Yes, how will you use

the information?

[] Yes [] No

Was the arrangement of the

information helpful?

Why/Why not?

[] Yes [] No

Is there more information you

would like to see?

If Yes, please describe it.

[] Yes [] No

What would you like to see in an

information reference library?

I have ideas/experiences to share. Contact met

Name

e-mail

Phone

Office Address

Return form to: Dr. Edward Hoffman

Code FT

NASA Headquarters

300 E Street, SW

Washington, D.C. 20546

Strategic Resources, Inc. (SRI)

v

NASA Fast Track Study

Acknowledgments

The following members of the APM-23 Class contributed significantly to the writing of the draft

Fast Track Guide and to the research and data gathered for this study.

Mr. Greg Stover LaRC

Mr. A1 Motley LaRC

Mr. Jim Stewart DFRC

Ms. Cynthia Peslen GSFC

Mr. Dave Chfistiansen HQ NASA

Mr. Paul Bertsch JSC

Mr. Jack James JSC

Mr. Bill Franklin KSC

Mr. Enoch Moser KSC

Mr. Richard Nelson KSC

Ms. Maynette Smith KSC

Mr. Tom Sutliff LeRC

Mr. Dan Morilak LeRC

Mr. David Mann MSFC

Mr. Lowell Pfimm MSFC

Mr. Fred Sanders MSFC

Mr. David Weeks MSFC

Mr. Mike Potts SSC

(Coordinator)

(APM-23 SSG HomePage WebMaster)

(Not in APM-23)

In addition, the contn_outions of the following are also acknowledged:

Dr. Edward Hoffman

Ms. Loft Lindholm

Mr. Jerry Coady

Mr. Dennis VanLiere

NASA HQ/Code FT

SKI

SKI

SKI

Strategic Resources, Inc. (SRI)

vi

NASA Fast Track Study

Introduction/Background

This study is a result of the thinking, concern and efforts of a number of members of the NASA

Advanced Project Management Class number 23 (APM-23), and the encouragement of Mr. Veto

Weyers and the Program/Project Management Working Group (PPMWG), charged with

developing NASA Policy Directive (NPD) 7120.4. The class convened at NASA's Management

Education Center, Wallops Island, Virginia from April 28 through May 8, 1996. Mr. Weyers, the

speaker for the graduation session as Chair of the PPMWG, discussed the issues surrounding new

approaches to smaller projects. Following Mr. Weyers' presentation, the class expressed their

concern for NASA to provide guidance, not only for smaller projects, but also for the accelerated

project structure needed to meet NASA's "Better, Faster, Cheaper" objectives. Mr. Weyers

challenged the class to provide their issues, concerns and recommendations to the full PPMWG.

The class met with the PPMWG on June 19, 1996, and recommended gathering information and

data from visionary managers within NASA on their experiences and ideas on how to do ''Better,

Faster, Cheaper" projects. The PPMWG chartered the class members as a Special Study Group

(SSG) and tasked them to research Fast Track Projects. The SSG was also tasked to produce a

Guide for Fast Track Projects as part of the NASA program/Project Management Guide. In

addition to looking within NASA, Code FT tasked Strategic Resources, Inc. (SKI) with

supplementing the SSG's findings by performing similar research with industry and non-NASA

government agencies that have successfully addressed compressing the development cycle.

The interest in ''Better, Faster, Cheaper" at NASA grew from studies performed in 1991 and 1992

showing that out of 29 programs evaluated, NASA had experienced approximately 65 percent

cost and schedule overruns, and programs averaged 12 years. NASA Administrator, Mr. Daniel

Goldin, focused on the concept of ''Better, Faster, Cheaper" for NASA in response to declining

federal finding. In light of this, sustaining exploration of the solar system, placing a Space

Station in orbit and developing a follow-on to the Space Shuttle, requires different approaches

throughout NASA than those of the programs referenced in the 1991/92 study.

NASA rewrote its basic program Management policy for Life Cycle Development, NASA

Management Instruction (NMI) 7120.4, in 1993 in response to the study's results. However,

much of the evidence applied only to traditional, large programs. The PPMWG is now rewriting

this document, as NASA Policy Directive (NPD) 7120.4 to apply to the broader range of

programs and projects, including relatively smaller projects defined as ''Better, Faster, Cheaper."

This directive will be accompanied by a new NASA Hand Book (N-HB) 7120.5 that will provide

guidance for Program/Project Management with the draft Fast Track Guide, produced in

conjunction with this study, addressing ''Better, Faster, Cheaper" projects.

Strategic Resources, Inc. (SRI)

1
NASA Fast Track Study

This study documents, summarizes and discusses the methodology, findings, conclusions and

recommendations of individuals within and outside NASA experienced in "Better, Faster,

Cheaper" projects. The findings are summarized below and discussed more thoroughly in the

body of this study. The draft Fast Track Guide is provided as Appendix A_ The compiled

interviews at Appendices B, C, and D, and Supplemental Information provided by interviewees is

at Appendix E.

Strategic Resources, Inc. (SRI)
NASA Fast Track Study

Methodology

The SSG was tasked to produce the Fast Track portion of the NASA Program/Project

Management Guide. The SSG determined that data for this effort should be based on the

experiences of managing projects that may be defined as Fast Track. These projects are usually

constrained by cost and/or schedule.

This study was designed as a compilation of interviews conducted at NASA's Centers or

coordinated by the members of the SSG. NASA/Code FT, responsible for NASA's

Program/Project Management Initiative, tasked SKI to support the SSG by compiling the SSG

interviews, coordinating the editing of the draft Guide, and researching Fast Track type projects in

industry and other government agencies. The objective of both the SSG and SRI research was to

examine a cross section of projects to provide a better understanding of how to manage cost and

schedule constrained projects.

The SSG and SKI developed interview protocols as a common framework for the study, however,

the actual interviews also included information outside the protocol. Although each project

manager interviewed differed with regard to what was important and what to focus on when

managing a Fast Track Project, several common elements emerged.

Strategic Resources, Inc. (SRI)

3
NASA Fast Track Study

NASA Interviews: The NASA interviews occurred at all Centers except Dryden and Ames.

There were a total of 14 project interviews and four related data gathering efforts focused on

defining what is needed for Fast Track Projects. These were compiled into the areas listed on the

following page, to provide a common basis for examining the data from all of the interviews.

Non-NASA Interviews: Identifying potential Non-NASA interviewees required research to

select candidate projects, an interview point of contact, and making appointments for conducting

the interviews. Of note were the Jet Propulsion Laboratory (JPL) personnel, who were very

cooperative, as were those contacted at the Applied Physics Laboratory (APL) at Johns Hopkins

University. These interviews complemented data from the NASA Centers as several of the JPL

and APL projects are managed as '_Better, Faster, Cheaper."

The industry and other government agencies' projects also provided supporting evidence on

managing the development of new products from concept to operation. Although there is not a

one-for-one correspondence with NASA's efforts, many of the problems faced by these project

managers are very similar.

Compilation Format: The compilation format emerged from the common threads revealed in

both the NASA and non-NASA interviews. Each interview was evaluated for applicability to one

of the four areas: Practice Oriented Ideas, Policy Related Comments, Cultural Changes and

Technology/Tools. Points made that were determined as applicable to Fast Track Projects, but

which did not clearly fit into one of the four areas, were listed in Other Observations.

Strategic Resources, Inc. (SRI)
NASA Fast Track Study

Interview Compilation Format

A_ Practice Oriented Ideas

B. Policy Related Comments

C. Cultural Changes

D. Technology/Tools

E. Other Observations

Examples

• Use of Teams

• Use of Risk Management Practices vs. Risk

Avoidance

• Use of Metrics

• Use of Co-location/Partnering

• Reduction in Level of Oversight

• Reduction in Rigidity of Life Cycle Development

Process

• Increased Training Uniformity for New Personnel

• Evaluation of Hierarchy of project Management by

Cost, Risk, Size, etc.

• Reduction in Documentation Requirements

• Location of Program Manager and project Manager

• Use of Cross Functional Integration

• Use of Up-Front Agreements

• More Up-Front Planning

• Use of Concurrent Engineering

• Use of Rapid Prototyping

• Use of CAD/Simulation Software

• Use of Communication Tools

Strategic Resources, Inc. (SRI)

5
NASA Fast Track Study

Analysis, Findings, Conclusions, Recommendations: Once compiled, the interview data was

analyzed for commonality of concepts. This commonality was fitrther organized into areas that

project managers identify as those that must be managed. Conclusions were drawn, and

recommendations provided based on ensuring successful Fast Track Projects. The Draft Fast

Track Guide is supported by these Findings, Conclusions, and Recommendations.

Fast Track Guide Development: The Draft Guide used an evolutionary development process

that included a session at Wallops Island to consolidate ideas in a facilitated, focused

environment. Much discussion and reflection was invested in this Guide through cooperation and

involvement from all NASA Centers.

Strategic Resources, Inc. (SRI)
NASA Fast Track Study

Findings

The interviews and research process identified a wide variety of approaches to managing

development of new products in a cost and schedule constrained environment, but common

elements emerged among the managers interviewed. These included: the primacy of good

planning at the start of a project; the use of teams, such as Integrated Product Teams OPT); and

solid project management that adheres to the plan.

The interviews revealed that success is not achieved by omitting steps found in the conventional

Life Cycle Development Process, but is achieved by examining the entire process to:

a) minimize resource consuming activities that do not add value to the project;

b) move activities forward in the process where possible;

c) compress or shorten process steps; and

d) understand that cost and schedule constraints will affect design decisions.

Success is also achieved in projects by controlling project execution to minimiTe or eliminate

changes to original requirements, objectives and the project plan. This requires taking time to

develop understanding and ownership of the requirements and objectives, to develop a credible,

comprehensive management plan appropriate for the project size and complexity, and to adhere to

the plan.

Teamwork was seen by the project managers interviewed as a value-added approach that requires

an up-front investment over other non-team approaches, but pays dividends during project

execution, where both time is saved and product quality ensured.

The definition of team varies by project, but several notable examples, such as the Joint Direct

Attack Munition (JDAM) Program show that all individuals involved in a project add value to the

project when deliberately made part of the team.

The interviews revealed an array of approaches, even when using common techniques, as well

as varying levels of emphasis for each area. This reinforces the finding that there is no detectable,

one-size fits all way to manage Fast Track Projects, but that success will be achieved by tailoring

the management approach to the uniqueness of the project. One of the hallmarks of the

successful project is the use of innovative cost and schedule controls. Everything in the Life

Cycle Development Process is open for examination by the Fast Track Project Team to define

ways to maintain the schedule and control costs. In the Findings Summary that follows, the

concepts of Planning, Teamwork and Day-to-Day Management are discussed with attention given

to specific techniques or ideas provided from the interviews.

Strategic Resources, Inc. (SRI)
NASA Fast Track Study

1.0

1.1

1.2

1.3

Plannin_

Up Front Planning: Planning is the key to successful management of a Fast Track Project

constrained by cost and schedule. The time taken in the early stages to think through all

of the aspects of the project alleviates the potential for the project manager, in the later

stages, of having to redirect. The project managers who made this point referred to the

conventional attitude toward planning, which is based on the belief that there will be

sufficient time and other resources to react to any issues as they arise and to change the

course of the project as needed.

Resource Availability (i.e., project information library, lessons learned): For many project

managers, Fast Track is a new operational concept requiring training and sharing of

knowledge. While formal classes provide a foundation for Project Management, sources

of data and information on management approaches for Fast Track Projects, available on

demand, are also desired.

Planning Areas

1.3.1 Risk (Identification of Issues/Strategy to Manage): Many saw Risk Management

as a key planning activity. To be able to identify those potentially stressing aspects of a

project, arrive at an assessment of probability of occurrence, and determine the cost

impact to the project if a risk event occurred allows a project team to plan mitigation

strategies. Many felt that conventional projects avoided risk by simply spending more

money. For the cost constrained Fast Track Project, this is not an option and risk must be

aggressively managed in ways other than avoidance.

1.3.2 Design (Design-to-Cost): The need to understand the nature of cost driven'

activities is fundamental. Several project managers mentioned that few seem to

understand the concept of Design-to-Cost (DTC) as it relates to the technical-scientific

environment. Successful application of DTC within NASA will be necessary to support

Fast Track Projects.

1.3.3 Schedule (Design-to-Schedule): Similar to DTC, it is critical for project managers

to understand that in the Fast Track environment, cost and schedule are independent

variables and must be controlled. This means that, while flexl_bility must be available in the

technical design, it should be built into the project plan from the beginning.

Strategic Resources, Inc. (SRI)
NASA Fast Track Study

1.4

1.3.4 Test (Design-to-Test): While not addressed as strongly, many found this to be an

area that can provide both cost and schedule savings. By asking questions about what

needs to be tested during the design phase, testing may be accomplished earlier in the

process, leading to earlier, less costly and less schedule-disruptive testing. Incremental

testing is a key approach allowing users to gain insight into design assumptions and how

they impact the satisfaction of user requirements.

1.3.5 Manufacturing/Production (Design for Manufacturing): Several project managers

indicated that costs could be controlled very effectively by involving those responsible for

manufaetming in the initial design and planning phases. This allows the team to identify

opportunities for design decisions that allow earlier selection of alternatives, thereby

lessening or shortening the manufacturing schedule. Cost controlling efforts in the area of

single process engineering in factories was also revealed as a significant cost saving

methodology.

1.3.6 Decision Points (Reviews, Tailoring of Life Cycle Development Process): An area

discussed by almost all interviewees was the need to manage disruptive, outside

influences. Most elected to incorporate review processes into the project and to not allow

them to be seen as being accomplished for only the benefit of individuals outside of the

project, such as upper-level management. Instead, reviews are conducted as

communication events, or 'peer reviews' to foster team knowledge and to encourage team

synergism. Where interface with mandated reviews occurred, it was managed so that the

oversight panels or board reviews were conducted as part of the planned reviews and

scheduled based on the project's requirements. Minimal time was spent on producing

documentation for reviews as well, using only those management products actually used

by the team

1.3.7 Metrics (Earned Value, Performance Measurement, etc.): All stressed the

importance of being able to know how the project is progressing. This means that for most

project managers, a well thought-out set of project metrics, sometimes unique for each

project team, must be identified during planning. The metrics should answer the questions,

'_What will we have?" and "How will we know it?" They should always be used for high

probability, high impact risk management areas. Many mentioned using Earned Value and

Performance Measurement methods.

Requirements Definition: The understanding and management of requirements in the

constrained environment of Fast Track Projects is absolutely key to success. The project

team must have a common understanding of requirements before planning begins. The

customer must also understand that planning will 'freeze' requirements, as cost and

schedule constraints will not allow course corrections, unless the customer is willing to

pay for the changes and/or accept schedule realignment. The team, including the

customer, should understand that the requirements at the end of the project, should match

the requirements identified at the start of the project.

Strategic Resources, Inc. (SRI)

9
NASA Fast Track Study

1.5 Operations: Operations must not be neglected during the development process. Decisions

made in the design and development phases will have cost impacts on the operations

phase. In addition, several interviewees pointed out that significant schedule and cost

savings are available for most projects when transitioning to operations. Consideration

must be given to the entire life-cycle for a true understanding of potential schedule and

cost savings.

2.0 Teamwork

2.1 Teamwork: Teams and teamwork is a developing area in government project

management. The overall project management structure as a team, and the IPT concepts

have been used on many successful Fast Track Projects. These team concepts are

characterized by openness, concern for each other, and loyalty to team objectives. Less

visible, but equally important, is the team's understanding of the project and its goals.

2.2 Why Teamwork (Rationale): Teamwork was mentioned by many interviewees as one of

the most important areas for project management of cost and schedule constrained

projects. Many also mentioned that all on the team, including the project leadership, must

understand the benefits of teams in order to accept the up front costs associated with team

management versus the more traditional, leader-decision maker management style. Some

of these benefits follow.

2.2.1 Cross-Functional Integration-Functional Synergism: The most obvious benefits

identified are in the area of empowering all project participants with insight and input in

the planning and design phases. The interfaces, interactions and integration activities are

much more effective and efficient. All individuals that may impact the project are

candidates for inclusion on the team, as all must be part of the effort of staying on cost and

schedule. All team members must understand the goals of the project and be encouraged

to provide innovative ideas on meeting those goals. The Fast Track environment cannot be

supported by stove-piped organizations that simply throw the product over the fence to

the next function in the process.

2.2.1.1 Eliminate Problems Early Rather than Late: Many of the savings in cost

and schedule occur because problems are identified early and can be solved before sunk

costs accumulate and schedule redirection is needed.

2.2.1.2 Save Money and Time by Fixing when it's Cheaper/Easier: This approach

allows the project team to make any necessary changes in design before design is frozen or

before bending metal. It is much less expensive to make corrections early in the process.

2.2.1.3 Better Design through User/Other: By involving the user on the team,

insight into user-peculiarities and perceptions may be gained to provide a better product

from the start and prevent the need for late stage corrections.

Strategic Resources, Inc. (SRI)

10
NASA Fast Track Study

2.3

3.2

Making It Work: A team environment is new for many managers, requiring a new set of

skills and abilities. Many project managers emphasized that training to lead teams or be a

team member is absolutely essential.

2.3.1 Culture Change (Badgeless Environment): In the environment characteristic of

many NASA projects, there will be both in-house and contractor/ subcontractor

involvement throughout the life of the project. An environment of trust and openness

must exist between project members. For example, Near Earth Asteroid Rendezvous

(NEAR) Project used the idea of a badgeless environment to describe the working

relationship they created where all team members were encouraged to share openly.

2.3.2 Selecting the Right Members: Many interviewees pointed out that selecting both

government team members and corporate partners able and willing to work in a team

environment is a must. It is very easy to break trust and difficult to get it back.

Procedures to obtain government members and contractor selection must be well thought

out and planned from the beginning.

2.3.3 Cross-Functional Integration (Working Agreements): The idea of working

agreements for lower level teams and for the overall team as a way of focusing everyone

on the goals and boundaries has proven effective.

2.3.4 Communication (Technology/Process): Communication between team members is

a must. The project leadership must do whatever it takes to make this happen. Electronic

forms of communication seem to be effective if supported by existing equipment. Several

interviewees mentioned that the payoff of good communication was so evident that the

purchase of common equipment was a good, essential investment.

2.3.5 Co-Location: Communication on a daily basis is desired for effective crossfunctional integration, and several interviewees indicated that co-location should be a

requirement.

Day-to-Day Management

Day-to-Day Management: Ability to manage to the plan is another key to success. In a

constrained environment, there will be little opporttmity to conduct reactive management

and redirect the project.

Risk Management: Managing risk during the project is essentially staying on top of those

things identified during risk planning through the effective use of metrics. The idea

mentioned by many is being able to see things occurring as they start and initiating

mitigation activities as early as possible. The activities may be tailored to the situation by

the tea_L

Strategic Resources, Inc. (SRI)

11
NASA Fast Track Study

3.3 Oversight: Insight vs. Oversight was an idea put forward by several of the interviewees.

This means that project management must work with oversight boards to create a climate

of confidence in the team and in the project leadership. The project manager/program

manager must strive to develop the same open atmosphere for the team as well.

3.4 Continuous Improvement: Each team member is a potential source of cost and schedule

savings, and project managers must establish an environment conducive to fostering ideas

from all team members.

3.5 Reviews: As mentioned above, reviews in a Fast Track environment should be minimally

disruptive. They should address only open issues and be well planned. The idea

mentioned by Motorola regarding the use of reviews to manage the white spaces between

different activities or different teams within the project seemed to be an effective

approach. The idea of having reviews when the project is ready for them, and not at times

determined artificially outside of the project, was also emphasized.

3.6 Documentation: In keeping with the theme that time is a limited resource on Fast Track

Projects, all potentially time extending activities must be held to a minimum.

Documentation was singled out as a management tool that may needlessly consume time.

Only the documentation planned for use by the team for specific project activities, or

required to proceed to subsequent stages, should be produced.

Strategic Resources, Inc. (SRI)

12
NASA Fast Track Study

Conclusions

Successful project management within the cost and schedule constrained world of "Better, Faster,

Cheaper" requires an experienced project manager and the ability to manage to the plan. There is

tittle, if any time, for major changes in requirements or to the established plan during project

execution in this new focus for space mission science and spacecrat_ development projects.

Managers must clearly understand the structure in the Life-Cycle Development model used by

NASA and how to use this model to create a comprehensive project plan. Planning skills, along

with the ability to communicate the understanding of requirements to the ultimate customer and

project management team, are critical.

Team structure and teamwork are essential to the successful Fast Track Life Cycle Development

process. Teamwork is associated with cross-functional integration and elimination of the stovepiping of disciplines of past approaches. A variety of disciplines must work together to be able to

shorten the development cycle. Teamwork provides the ability to achieve that compression, and

to identify and eliminate problems early, rather than late, in the project development process.

Team leadership and team management are very important for the project manager to be

successful in the Fast Track environment.

Since there is little time to recover from false starts or mid-project redirections in this time

constrained '`Better, Faster, Cheaper" management era, trust becomes very important. For trust

to exist, the fight project management team, whether in-house or contracted, must be selected and

given authority and responsibility for the project. This requires the ability of the team selected to

demonstrate from past experience that the capability to work in this environment exists, and that

the necessary and appropriate management tools are assembled for the challenges of the project.

Working closely with procurement personnel and the ability to manage the contractor selection

effort are absolute requirements.

Risk must be identified and risk mitigation planned, along with a reporting system that allows the

project management team to implement risk management. Oversight must be as least disruptive

as possible. For this reason, documentation and project reviews should only be those that are

used by the team to actively manage the project. Higher level reviews should accommodate the

project team by allowing those required reviews to fit the project team's plans and to present

status reports from the project team's management tools and metrics.

Innovation must be encouraged, both in technical and managerial approaches. For the technical

team, cost and schedule will drive the design. This does not mean that science needs to take a

back seat, but management must recognize early what good enough will be and use better as a

design margin. Simplicity and industry-standard design plus off-the-shelf must be used wherever

possible. Design-to-Cost and Design-to-Schedule must be taught as a way of life for the "Better,

Faster, Cheaper" project teanL

On the managerial side of the project, the entire team must look for opportunities to perform

activities in parallel, to help each other identify opportunities, to begin work as early as possible in

Strategic Resources, Inc. (SRI)

13
NASA Fast Track Study

each functional area, and to identify and minimize disruptive but necessary activities. In other

words, approaches such as concurrent engineering will also become a way of life and must be well

understood. Communication of information is more important than ever and innovation here can

provide significant benefits in time and cost savings as well. For this reason, meeting and review

management are also key skills for the "Better, Faster, Cheaper" project manager.

Strategic Resources, Inc. (SRI)

14
NASA Fast Track Study

.

.

.

4.

5.

o

Recommendations

Require and allow time for the actual project manager and team to plan at the front end of

Fast Track Projects (especially entering Phase C/D), to include Risk Management

Planning and Design-to-Cost, with the understanding of the importance of controlling

project technical and programmatic requirements throughout the life cycle.

Use the Program Commitment Agreement concept at all levels of the project as a series of

bilateral agreements, to ensure common requirement understanding and control

requirement creep.

Use metrics for each project that will measure progress and value.

Use team management and cross functional integration as a tool to achieve success.

Educate project managers in innovative techniques for managing reviews, documentation,

oversight and risk to minimize disruption in schedule constrained projects.

Provide NASA Program/Project Managers access to an educational/corporate knowledge

information system addressing issues of Fast Track Project management, available as an

on-line resource to support both project management and career development.

Strategic Resources, Inc. (SRI)

15

NASA Fast Track Study

Appendix A

NASA Program/Project Management Guide for Fast Track Projects

Provided under separate cover.

Strategic Resources, Inc. (SRI)

A-1

NASA Fast Track Study

Appendix B

APM-23 SSG Interview Compilations

L

IL

Proieet Interviews

1. X-CRV

2. LANDSAT

3. Earth Orbiting Satellite (EOS) (Christopher J. Scolese)

4. Explorers

5. Marshall Space Center (MSC) Interviews

a. Optical Transient Detector

b. AXAF

c. SAIL

d. Transient Pressure Test Article (TPTA)

e. STABLE

6. Advanced Rocket Motor (ASRM)

7. Evolved Expendable Launch Vehicle (EELV) (Stephen C. Nunez)

8. Halogen Occultation Experiment (HALOE)

9. In Space Technology Experiment Program (INSTEP)

(Lenwood G. Clark)

10. Lidar In Space Technology Experiment (LITE) Instntments

Non-Interview Data Gatherin2

1. Goddard Space Flight Center (GSFC) Senior Group Fast Track

Brainstorming Ideas

2. GSFC Junior Group Fast Track Brainstorming Ideas

3. Lewis Research Center (LeRC) Pyramid Team Pare Down

of NI-IB 7120.5

4. LeRC Clean Sheet Team

B-2

B-4

B-7

B-9

B-13

B-21

B-23

B-25

B-28

B-31

B-34

B-37

B-40

B-42

Strategic Resources, Inc. (SRI)

B-1
NASA Fast Track Study

L Interviews

1. X-CRV Prelect Back_ound: The X-CRV is a six-passenger liRing body reentry

vehicle that is to remain docked with the space station. In the event of a life threatening

emergency, the space station crew would board the vehicle and reenter for a parachute landing.

A) Practice Oriented: The X-CRV project will proceed through the development of a

flying prototype using civil servants for the project staff.. Research and development,

experimental, or ill-defined projects are well-suited for development by civil servants.

Operational production, and other well-defined projects are good candidates for total

delegation to contractors. Competitive procurement will be initiated for the production

phase of the X-CRV project.

The X-CRV team is housed in a small building isolated from the main buildings at Johnson

Space Center (JSC) and they use a hands-on hardware approach to development. The

manufacturing team is sitting with the design team and working issues and problems oneon-one on a daily basis. Rapid communication occurs because everyone is collocated.

The project manager must continually communicate the importance of completing the

project as planned

B) Policy Related: Because of the sensitivity of human space flight, NASA people should

negotiate, manage, and make decisions on human space flight projects.

A big project should be built as a set of individual smaller projects working towards

accomplishment of incremental milestones. The key is to set up 'deliverables'

(performance milestones) to show progress. A core set of team members decide how to

divide up the project and build up milestones. This type of plan must be done by the

program or project team

The project philosophy is to "build things small throw them away, and move on" to the

next phase of the project.

Progressive formalization should be used to build the project plan and conduct the project.

Using progressive formalization, you minimize the amount of documentation up front and

put more emphasis on documenting as built.

ISO 9000 is important for production organizations; but, it does not apply for rapid

prototyping, Skunk Works TM type projects. Instead of written specifications and

processes, the project manager ensures the technician and designer work together to build

the project.

Strategic Resources, Inc. (SRI)

B-2
NASA Fast Track Study

C) Cultural Changes: The procurement process tends to treat all procurements the

same. A Fast Track project team should include dedicated procurement and business

personnel. As part of the team, they are dedicated to the project, not the procurement

process. They need to understand that the biggest risk is the failure of the project, not of

the procurement.

The best way to reduce cost is to understand the technical details that are required and

how much it costs to implement them. Once the technical decisions have been made,

'ignore the rules and go get it done".

D) Technology/Tools: All announcement and selection work on the X-CRV project is

done electronically. This minimiTes paperwork and gives competitors instant access to the

data they need, and the project team has instant access to contractor proposals.

Minimize the project plan. The X-CRV project plan is graphically depicted, in five stages

on a poster. The X-CRV original written project plan was a one page document with

about a dozen top level goals. It has evolved to about five pages, but is still minimal.

NASA middle managers are the "keepers of the culture". Most resistance to getting

things done smaller, faster, cheaper has occurred in middle management. The failure of

most projects is usually due to a "gap in leadership." Projects with strong

leaders----succeed; and projects without strong leaders--fail. The project manager must be

totally, personally committed to the success of the project.

E) Other Observations from Interviewee: None

Strategic Resources, Inc. (SRI)

B-3
NASA Fast Track Study

2. LANDSAT Proiect: NASA took LANDSAT over from the Air Force midway between

Pre "hminary Design Review (PDR) and Critical Design Review (CDR). As an existing contract, it

had less flexibility in terms of "new ways of doing business" than a new start program would

have. Nonetheless, significant changes to the management and implementation style of the

LANDSAT project makes it a non-traditional Goddard project.

Three management principles are: communicate, communicate, communicate. Beyond that, the

project manager's most important responsibilities are: (1) technical and programmatic

performance, and (2) staff development.

A) Practice Oriented Ideas: The original 150 CDRL items in LANDSAT were reduced

to five by asking the civil servants and the two contractors to identify the minimum

documented information needed to do the job, dropping all preconceptions for "how we

always did it."

NASA should play the role of referee, not overseer.

LANDSAT has a dedicated person to keep the Risk Management Plan relevant and risk

assessment meetings are held every two weeks among all senior managers.

There must be an Action Plan for every problem. If you've got a problem and no

recommendation, a solution can be found driven solely by cost and schedule constraints,

which you may not like.

Growing people is the most important non-project role of the manager. To avoid burnout,

the project manager must make sure each player:

• Has a job they can complete

• Gets a sense of satisfaction

• Has a means to avoid frustration

• Understands the constraints and limits, knows which are pushbacks

• Owns the problem or task, is part of its definition, isn't a sniper at meetings

• Knows the project manager's constraints and limits - what can be changed and

what can't

A good Peformance Measurement System is a must, but it has to be flexible. It should

rigidly track analysis tasks early in the program and be less rigid during implementation

and test to allow creative solutions for emerging events.

If the reporting format doesn't match your internal tracking format, keep two sets of

books: one to manage from and one to present. Don't get bogged down in awkward or

inappropriate formats. Work from one preferred format and translate to another as

needed.

Strategic Resources, Inc. (SRI)

B-4
NASA Fast Track Study

Cost models are for scholars and research. They are completely relevant to the resource

Analysis Office (RAO), but not applicable at the project level. Comparing project

estimates against the RAO is an excellent sanity check, a way to identify

misunderstandings and shortcomings, and to identify tall pole cost drivers. Detailed

historical tracking of costs from many missions is not relevant to execution of the specific

project.

B) Policy Related: The most significant constructive change management could make is

to hold individuals responsible for their own work. Reward excellence and penalize

stupidity.

Procurement rules can be changed to secure contracts faster with less formal approval at

each evaluation step, while still maintaining fair and open competition.

Teams and people need to be rewarded for completing their jobs.

A whole new Center could result from rotating the directorate-level office staff laterally

among the nine Codes, independent ofbackgrotmd, interest, or skills.

The new project management NMIs don't really apply to LANDSAT because of its midproject transfer to NASA; however, the NMI management principles still apply.

The travel budget must be enough to support site visits.

There is absolutely no chance for a budget increase in LANDSAT. The best way to

handle this is to allocate budget performance to responsible levels, and replace those who

can't perform to budget.

C) Cultural Change: There needs to be fewer "empires" at the Center, making track

teams function-specific with minimal people and fixed project lifetimes.

Independent Annual Reviews (lAP,) can be very productive, depending on who is on the

panel. People proud of their cleverness in finding fault will make you jump through a lot

of hoops with no value added. People who understand the project limits, who are tough

but fair, and who are interested in the mission, will provide a meaningful review.

Generally, the problem with selling new ideas and getting innovations approved is not with

senior management anyway, but with all the layers in between. A clever manager can brief

red and yellow issues with complete disclosure, adhering absolutely to the Monthly Status

report (MSK) style and content, yet never really addressing the project issues that affect

mission success. It is very hard to be forthright in this kind of review environment.

Strategic Resources, Inc. (SRI)

B-5
NASA Fast Track Study

D) TeclmologT/Tools: It would be valuable to set up a structure to capture lessons

learned from all projects to learn what works, what doesn't, and the reasons why. Besides

providing practical advice and insight, this would also present a "report card" on project

managers, so their performance could be seen by the general staff in terms of applied

management abilities, not just word of mouth and formal newsletter articles.

Home pages and electronic data transfer are tools that allow team members to

communicate and update the general knowledge pool quickly and accurately, as long as

they don't take a lot of resources to set up and maintain.

E) Other Observations: In terms of new Fast Track thinking, new approaches and ideas

won't come out of traditional hierarchies and empires. Radically stressing the

organization, maybe even by moving key people across directorate boundaries, could

shake out inefficiencies and result in new ideas.

The lack of constructive stress makes it too easy to rely on old solutions (Le., more

people, more money) and too difficult to recognize the real problems and to see new

solutions. Old solutions are usually not valid in today's environment; new insights can

best be gained from challenging the norms.

If Fast Track means making rapid progress, then the existing obstacles are the

procurement rules, documentation, and contractor interfaces. Here are three ways to Fast

Track:

• Run a minimally-staffed project (i.e., four heads and a fixed-price, hands-off

contract).

Partner with the contractor - exclude government oversight. Rely on sharing

the workload where one side does specific tasks for the other in a role like a

subcontractor, with task definition, schedule, completeness criteria, etc.,

defining the task envelope. Don't allow duplication of work.

Put a small, focused staffon a short (three-year) mission. The project manager

decides how it will be staffed even if the total headoount is predefined by the

Directorate, (Le., the Project Manager decides on the evolving need for project

support, software manager, Deputy Project Manager, Resources (D/PMR),

Integration and Test (I&T) and all other roles).

If the budget doesn't support the mission, make it clear to management what you can do

for the budget - accepting $100 million to push a rope will end up with an stationary rope

and all the money spent.

Strategic Resources, Inc. (SRI)

B-6
NASA Fast Track Study

3. Earth OrbitingSatellite(EOS): The EOS Ante Meridian (AM) Project has

proceeded through all project life cycle phases, (i.e., A, B, C/D) but has done so within a number

ofditferent Projects and Programs. The Life Cycle Cost of EOS AM, which was originally set at

$2 billion, was reduced twice during Phase C; first to $1.3 billion and later to $1 billion.

Originally, the EOS AM Project was expected to be an Integration and Test (I&T) Project,

receiving a spacecraft bus from the Space Station Work Package C Polar Orbiting Platform

Project, instnmaents from the EOS Instruments Project, and a ground system from the Earth

Science Data and Information System (ESDIS) Project.

The launch vehicle was not determined until early 1995, around Critical Design Review (CDR),

and resulted in significant additional cost due to carrying the flexibility to fly on a number of

different launch vehicles all the way to CDR, Additionally, mergers/restructuring of the

spacecraft contractor, a major instrument contractor, and the solar array contractor caused

substantial problems for the project. These included loss of key technical personnel, changes in

accounting systems, rate increases and government payment of Restructuring Fee Agreements.

A) Practice Oriented Ideas: A project plan for the EOS AM Project was written and

approved, but not until long after the project had started. In this case, it might have been

better to use the Phase B work, culminating in the Preliminary Design Review (PDR), to

replace the project plan. Documentation requirements for the Prime Contractor, LMMS,

were reduced to approximately 100 CDRLs, with only ten requiring government approval;

a decrease of more than 800 CDKLs.

Two key goals of the risk management activities were:

(1) To significantly reduce the amount of new design hardware in the

program, replacing much of it with heritage hardware from other

programs.

(2) To integrate civil servants into the IFFs giving the government

better insight into EOS AM- 1 development.

A Performance Measurement System is probably most useful for initial planning,

identifying forgotten items, and replanning at the end of CDR. In a development program

the linkage between cost, schedule and performance is difficult to predict a priori;

however, as copies are produced, a valid PMS with strong coupling between cost,

schedule and performance is possible and would be a useful tool. Probably the most

significant lesson learned is to not duplicate work. Every task should be done once, by

whoever is most appropriate to do it (whether government or contractor) with

participation by all appropriate personnel Nothing should be undertaken from scratch

unless it is specifically necessary to achieve mission objectives. Use heritage designs

wherever they will achieve mission objectives, rather than going to new designs. New

Strategic Resources, Inc. (SRI)

B-7
NASA Fast Track Study

design efforts which typically increase cost and risk should be pursued to meet mission of

programmatic goals.

B) Policy Related Comments: Eliminate the Program Operating Plan (POP) review and

use the Independent Annual Review (IAR) to replace it, or combine the two reviews.

A massive three day PDR or CDR does not add value to the project. For reviewers to be

helpful, they should be involved in the subsystem PDKs and CDRs, working with the

technical team to improve the designs.

The system-level PDR and CDR should be more of a summary review, with most of the

issues worked out with the review team before hand. The Center Flight Assurance

Directorate, refused to structure the EOS AM-1 reviews in this manner, although they

have subsequently changed to this structure for LANDSAT 7 and other missions.

NASA needs an exemption or flexa'bility t_om the FAR to respond to the Research and

Development (R&D) nature of our activities. Often it is desirable to accomplish tasks in a

more streamlined fashion, or by forging partnerships with industry and academia or

between companies.

NASA should pursue Phase B/C/D efforts that allow the science team, industry and

government to develop an early understanding of the goals, risks and methodologies to

accomplish the mission. This will reduce the risk and time required for detailed design and

build. The NASA procurement system should support this type of an activity.

C) Cultural Changes: A management structure based on the use of Integrated Product

Teams (IPT) comprised of both civil servants and industry partners in a specific discipline,

t_equently a specific spacecraft subsystem, is needed. The government gains and

maintains insight into contractor efforts through normal IPT interactions by incorporating

civil servants and contractors into a single project team

D) Technology/Tools Recommended: Use early breadboards and technology

demonstrations, and proper incorporation of computer models and analyses. Engineering

models can reduce risk, but only if original designers and manufacturing team remain.

E) Other Observations: Having worked both the DoD and NASA side, it is clear that

NASA works in a more efficient manner and typically employs more aggressive

technology.

Strategic Resources, Inc. (SRI)

B-8
NASA Fast Track Study

4. Explorers: Small Explorers (SMEX) was managed out of Code 700, Engine_ing

Directorate and Medium Explorers (MIDEX) was managed by Code 400, Flight Projects

Directorate. Each have a very small project office with all discipline support, including system

engineering, matrixed from the appropriate directorates. MIDEX has a project office with a

mission manager and, if the instrument is procured outside the Center, an instrument manager for

each mission that falls under MIDEX. The project office provides overhead functions,

Configuration Management, office support, etc., for each of its missions. Matrixed personnel are

appointed by their home organization and selected by the skills needed for the project.

Each mission has internal meetings as needed with a monthly report provided to the Project

Manager. SMEX had informal weekly reports to the Program Manager at Headquarters until the

latest organization of Headquarters was established. Reporting to Code 400 for MIDEX is

standard at the Center. Within the project, most communication is kept informal.

A) Practice Oriented Ideas: A project plan has no value for these missions. The proposal

responses to the Announcement of Opportunity (AO) have become, in effect, the project

plan in combination with the information in the PDR and the NAR. It was found on the

first two small explorer missions that the project plan was not useful and, in fact, was

signed after the launch, or not at all. A project plan was never written for the third

mission. The consensus was that writing a formal project plan for these small missions

was a waste of time and the same purpose could be served by using the information in the

proposal, the Preliminary Design Review (PDR) and the Non-Advocate Review (NAR).

Therefore, a project plan was considered bad for Fast Track Projects.

• Solar, Anomalous and Magnetospheric Particle Explorer (SAMPEX) - Project

plan signed off one month after launch

• Fast Auroral Snapshot Explorer (FAST) - Project plan never signed off

• Sub Millimeter Wave Astronomy Satellite (SWAS) - No project plan

No pre-Phase A or Phase A studies. The process of developing the proposal could be

considered the equivalent of a Phase A study, with the project immediately entering Phase

B upon proposal selection. Mission performance requirements were developed from

Level One requirements during the definition phase reflected in the PDK. Metrics are in

the form of technical accomplishments, schedule, and cost. A fidl Peformance

Measurement System is not used. In lieu of a fidl formal performance measurement

system, key parameters were chosen for performance measurement assessment. As this

saves time and money, is good for Fast Track Projects.

Areas of risk are identified prior to Phase C/D, and watched closely throughout Phase

C/D. Cost tradeoffs are done continuously and contingencies are created throughout

Phase B. By the end of Phase B, the mission has been descoped as far as it can and should

Strategic Resources, Inc. (SRI)

B-9
NASA Fast Track Study

be. Formal descope plans are not necessary or valuable, as there isn't time or money to

do deseoping in Phase C/D in fast, low cost missions.

Firm funding commitments are obtained prior to starting the project. Procurement is

completed using the AO process. Contracts are cost reimbursable with cost capped

missions. Mission requirements are defined by performance, but are not totally divorced

from technologies.

The project was not guided by NMI 7120. The documentation and deliverables in the

NMI were considered too burdensome for a small project. Specific items, such as the

timing of the end of Phase B, the funding profile, and some of the reviews, were selected

and followed if appropriate for the project. Project reviews added value, particularly in

getting the team together at key points in the development process.

Share the wisdom. A Fast Track Project should build a team as early as possible to effect

tradeoffs. They should have early agreements on how to work as a team. At the start,

define the project and mission design, have a plan to match the budget, then "go for it."

Early agreements are needed for deciding how to measure performance.

Instruments require special attention. Principle Investigators (PI) generally don't have the

full infrastructure that should be provided to them. Control gates should be the

PDR/NAR followed by a running gate with a cost cap. If the project is expected to

exceed the cost cap, the project manager and PI's should immediately see the Associate

Administrator (AA).

Barriers were overcome by working one-on-one with other Organizations/Directorates.

This has proven to be a long and sometimes painful process. For example, problems with

Flight Assurance took three years to resolve.

B) Policy Related Comments: As currently conducted, independent reviews don't have

value added of inside and outside people in key areas of expertise of the project. Having

input into the selection of a mix would improve the independent reviews.

Regarding management controls by the Center, action items from the institution take away

from the streamlined process and use up resources, particularly stafftime, that is needed

elsewhere.

C) Cultural Changes: Most barriers are due to the Center. For example, the project

manager feels that it is part of his/her job to respond to changes made by Headquarters,

and therefore, does not consider them to be barriers. Barriers have typically been in the

areas of procurement, review process, flight assurance, the cost, and sometimes lack of

engineering support. The environment at the Center has been geared towards large

projects and the tendency is to throw large numbers of people at a project and/or

problems.

Strategic Resources, Inc. (SRI)

B-IO
NASA Fast Track Study

Overcoming barriers is very specific to the people the project deals with, and the

resolutions agreed to by those people are not always supported by their home

organization. Small teams with creative ideas are often shot down by their own

organizations. A Fast Track Project is different from other projects by its lack of

bureaucracy, its approach, and its willingness to be flexible.

D) Technology/Tools Recommended: Use paperless Configuration Management (CM)

with all documents on the Web. The standard CM process is quite involved with large

databases of changes to documents, etc., a great deal of paper required to be filled out,

and a long time is required to get changes approved through formal boards. This generally

requires at least one full time person to manage the effort. The recommendation is to keep

all documents in a location accessible through networks of the Interact, and to handle

change control through electronic means.

E) Other Observations: Full Cost Accounting would not really change the way the

project is managed. It would, however, provide more visibility as to where the money is

spent, particularly for in-house hardware. The project would also pay closer attention to

overhead functions.

The least successful experience was dealing with the technology on SMEX. New

technology and its infusion needs to be visible, with a well understood agreement on its

status. It should be managed separately and kept highly visible.

The most successful part of the project was that missions were ready on time and within

cost. People had a tendency to overwork. Since Explorers are a continuing line,

individuals could endure perhaps two missions, but not a third. Recommendations to

resolve this include:

• Manage time carefully from the beginning of the project and streamlined

processes from the start.

• Do not keep a rolling wave of undone work moving into Integration and Test

(I&T).

• Be willing to pay overtime early, as the hours are expended, then save time

later on.

• Attain depth in understanding the mission as early as possible.

The most innovative aspects were the architecture of the technologies that simplified the

design and were more cost effective.

Strate_lic Resources, Inc. (SRI)

B-11
NASA Fast Track Study

The combination of training and experience have been very valuable. The Project

Manager completed a large amount of formal training in addition to experience gained at

GSFC. The additional training was considered valuable for conducting a Fast Track

Project.

Strategic Resources, Inc. (SRI)

B-12
NASA Fast Track Study

5. Marshall Space Center (MSC) Interviews

a) Optical Transient Detector (OTD): The OTD payload was originally scheduled as a

six month development project with a firm $4.9 million budget and classified as a Class C payload

but was Class D referenced. The OTD slipped to nine months, however, estimates show it should

have been a 12 month project. The project plan stated that the payload had an up to 65 percent

probability of failure.

A) Practice Oriented Ideas: OTD employed some Quality support, but no receiving

inspections. OTD used Project software to develop logic schedule diagrams and it used

critical path methodology. Weekly scheduled reviews were conducted which, eventually,

became daily tag-ups.

Two major reviews were conducted:

• Pre-environmental- considered worthwhile

• Pre-ship - not considered to be value-added

System level specifications were used, but, in retrospect, specifications should have flowed

down to the box level. There was no formal tracking of specification verification, and

only Principle Investigator (PI) acceptance of payload performance. OTD used unreleased

drawings as it was felt that formal Configuration Management/Verification was not

needed for building one of a kind. There was formal approval of the project plan by the

Center and NASA Headquarters.

There is a need for coherent parts procurement and recommend _rongly that a dedicated

materials acquisition specialist track all parts procurements.

B) Policy Related Comments: Recommend eliminating low level procurement requests.

Government credit cards should allow for increased spending limits to expedite small

project-related procurements. Surge capacity is critical. It is felt that Full Cost

Accounting will constrain surge capacity ability.

C) Cultural Changes: The chief engineer role is absolutely critical to motivate the S&E

troops to get the job done.

D) Technology/Tools Recommended: None

E) Other Observations: None

Strategic Resources, Inc. (SRI)

B-13
NASA Fast Track Study

_[SC Interviews (cont.)

b) AXAF: The AXAF project is cost-driven with recognized and accepted increased risks.

The program has few equipment spares, an avionics engineering breadboard, and the protoflight

model which is being flown. Project cost is $750 million with an operational life expectancy

between five and ten years. The project employs about 200 FTEs in the S&E labs and about 500

FTEs overall, including TRW, Kodak, Ball Aerospace, MIT and SAO.

The project has faced numerous reviews with review teams checking review teams. While there is

a need for external oversight, teams need to keep the same people from start to finish to minimiTe

the learning curve and enhance the ability to streamline reviews.

A) Practice Oriented Ideas: Internal reviews at the Center that used Earned Value and

were led by the Comptroller's Office were considered helpful.

The project plans were not really used other than to get initial commitment, to serve as an

audit trail, and to allow project people to initially get their arms around the project.

Tracking and scheduling were performed by TRW using monthly scheduling and tracking

teleconferences.

While one materials acquisition specialist was used at TRW to track all procurements,

another dedicated specialist is recommended to procure/track the EEE parts.

Requirements must be defined up front. AXAF baselined requirements at the

SRR allowing requirements to change only in response to problems.

The project office maintained configuration control with support from SAIL and tried to

resolve issues below the CCB level.

Used Center Quality Personnel and placed them at the prime contractor site.

No interface with Headquarters except for formal reviews. The Progrmn Management

Council (PMC) is perceived to add to the overhead burden or, at minimum, add another

layer with which to deal.

Risk assessment should be performed in the project office and driven by fiscal reserves

constraints, resulting in risk ratings ofhigh-medittm-low and assigned dollar values.

Audits are performed every six months for every subsystem and Center personnel are

invited to observe. These audits provide a wealth of information for Center people.

B) Policy Related Comments: None

Strate__icResources, Inc. (SRO

B-14
NASA Fast Track Study

C) Cultural Changes: Chief Engineer functions could be in the project office itselt_ not

worrying about whether someone is assigned to S&E Directorate or Project Office.

Cost of program was driven by political issues and decisions that forced the use of the

Shuttle as the launch vehicle.

Need dedicated S&E personnel on the project who are co-located and have accountability

regarding the end product.

D) Technology/Tools Recommended: None

E) Other Observations: Lesson learned is not to use universities for science instrument

development.

It is critical to have a strong systems engineer and chief engineer with key people

integrated on the team.

Most successful aspects of the project dealt with the competence of the prime, the

reviews, international agreements/contracts, and tracking.

Least successful aspects dealt with schedule variances caused by university involvement.

Strategic Resources, Inc. (SRI)

B-15
NASA Fast Track Study

MSC Interviews (cont.)

c) Systems Analysis and Integration Laboratory (SAIL): SAIL handles requirement's

definition, Configuration Management, Systems Engineering, etc., for all Center projects. SAIL

is working with Preliminary Design (PD) Office (Program Development Directorate) to work the

requirements up front. When PD starts work on a new concept/project, SAIL is invited to attend

meetings early on. This really helps to work things on a system level, including requirements.

However, a lot of the PD concepts never work out for whatever reasons (political, funding, etc.)

and cause a manpower drain for SAIL.

A) Practice Oriented Ideas: Many projects fail to identify the requirements up front

(especially in-house projects), so people just start designing with requirements as an

afterthought. Need better Systems Engineering and up front stafl]oading. Need to

strengthen the up front planning and development of a valid schedule.

Need to allow more flexibility in Configuration Management. Advocate the Responsible

Equipment Engineers (REE) role of cradle to grave responsibility.

B) Policy Related Comments: Full Cost Accounting will make matters worse in regard to

Systems Engineering and up front staffloading; people will be assigned only as they are

viewed as adding value to the project.

Each project must have clearly defined systems integration responsibilities. There is now a

push for Design Certifications Reviews (DCRs) which seems to be mmther layer of

overhead.

C) Cultural Changes: There is a general problem of declaring when reviews will be

conducted and then executing the review regardless of whether the team is ready or not.

Responsibility and accountability for project success (or some portion of the project) is

often delegated without the authority to carry it out. There is also a tendency to start

projects without knowing really what is wanted.

There must be communication between laboratories; especially designers with the avionics

and propulsion personnel.

D) Technology/Tools Recommended: None

E) Other Observations: None

Strategic Resources, Inc. (SRI)

B-16
NASA Fast Track Study

MSC Interviews (cont.)

d) Transient Pressure Test Article (TPTA) Faeili_: The TPTA Facility Fast Track

Project began in January 1987 and ran nine months at an approximate cost of $39 million with up

to 600 personnel and 29 companies plus vendors. USBI performed all procurements. The project

used dedicated co-located Quality personnel. A standard six day workweek was used with work

on Sundays as well. Cost was not a driver since the objective was to get the Shuttle returned to

flight. Requirements definition and facility design were worked concurrently. Critical design

drivers included access and serviceability.

Industrial safety should have been brought in earlier. All design reviews (TRRs, PD1L CDR)

were informal and only action items. Other than weekly briefings to the Center Director, no

outside briefings were conducted. The project enjoyed top priority status throughout the nine

month period.

A) Practice Oriented: The Center Configuration Management system was employed for

the project. All Center personnel were dedicated full-time to the project and were colocated. There was no formal project management plan. The project's own fidl-time

administrative officer was key to the team

The project often worked to red-lined drawings. They held weekly and sometimes twiceweekly tag-ups that were usually oral discussions with team leaders.

There was significant customer involvement from the beginning which was sometimes

difficult, but critical for project success.

B) Policy Related Comments: People need to be assigned to the project full-time, rather

than part-time, if the project lasts more than a few months. The personnel should be colocated.

C) Cultural Changes: Center supervisors need to be willing to give up people for short

range projects and still ensure that their people get credit for their work.

NASA Barriers include:

• NASA Matrix structure (people detailed to project vs. being assigned full-time)

• NASA Site Security had to get a guard out to the compound each time someone

had to go in or out on Saturdays which impacted morale and productivity.

Strategic Resources, Inc. (SRI)

B-17
NASA Fast Track Study

• Communications problems with various parts of the contractor organization.

• Management oversight (need mlnim.m management involvement).

D) Technology/Tools Recommended: None

E) Other Observations: None

Strategic Resources, Inc. (SRI)

B-18
NASA Fast Track Study

MSC Interviews (cont.)

e) STABLE: No background information available.

A) Practice Oriented Ideas:

• Total communication.

• Timely decisions.

• Bring all disciplines on board at beginning and describe tasks and schedule.

B) Policy Related Comments:

• No creep in basic requirements.

• Create only the documents required to verify integrity and safety.

• Limit reviews to those absolutely necessary; STABLE had only PDR with no

RIDs and only action items; every meeting was normally a technical review less

than one hour weekly but, sometimes up to three times a week.

• CCB drawing system streamlined by review/signing meetings (one day

turnaround).

• Credit card purchases essential.

C) Cultural Changes:

• Team dedication to end product.

• Technical respect required.

• Must have management priority.

Strategic Resources, Inc. (SRI)

B-19
NASA Fast Track Study

D)

E)

• Must have a realistic schedule (originally four months allotted was too short;

actually was a nine month project).

• The Fast Track approach needs to be embraced by all Center personnel

Technology/Tools Recommended: None

Other Observations: None

Strategic Resources, Inc. (SRI)

B-20
NASA Fast Track Study

6. Advanced Solid Rocket Motor (ASRM): ASRM was a Shuttle project canceled by

Congress in late 1993. The Center's responsa'bilities for ASRM were limited to the

design/construction of the test stand, Data Acquisition and Control System (DACS), test stand

activation, and motor testing. Overall project responsibilities resided at another Center. The

ASRM project is a typical example of a large NASA project that followed the dictates of NMI

7120.4 and NHB 7120.5 with all the resulting phases, reviews, and reporting requirements. It

should be noted that the project was canceled during Phase D, Development, prior to entering

Phase E, Operations.

A) Practice Oriented Ideas: Maintaining an accurate cost tracking system is imperative to a

successful project.

Don't micro-manage. Projects Management should be delegated, and budgeted at the lowest

practical level with managers held accountable.

A key component to a successful cost tracking system is a strong resource control manager.

The project manager is primarily involved in the technical aspects, schedules and high level

resource control of a project. The day-to-day cost functions should be delegated to resource

control personnel to allow the project manager to concentrate on the technical and schedule

aspects of the project.

A strong project team is one that is small, co-located and dedicated to the project. Colocation results in improved team coordination, communication, and decreased distraction

_om the institution.

Environmental permitting processes are di_cult and drawn out. Adequate planning is

imperative to ensure schedule impacts are minimized.

The Earned Value of a sub-contractor must be verified (failure to do so on ASRM resulted in

a cost overrun on the Data Acquisition and Control System (DACS)).

Match contractor capability to the job. Center had a high-technology aerospace company in

charge of brick and mortar construction which led to high cost forecast. NASA made the

decision to cancel the construction part of the contract, brought the construction contract inhouse, and were able to bring the construction back in line for both cost and schedule.

B) Policy Related Comments: As a Space Shuttle Project, ASRM followed all the

requirements outlined in the 7120 series of instructions and handbooks. The National Space

Transportation System (NSTS) 07700 series of controls forces a tremendous amount of paper

on the systen_

Strategic Resources, Inc. (SRI)

B-21
NASA Fast Track Study

C) C3xlmral Changes: A specific NASA barrier that interferes is the "mind set" or "¢ultuxe"

of middle level NASA managers that are loyal to the infrastructure requirements with resulting

reporting and management requirements. This culture drives a large infi'astructure that is selfsustaining and serf-justifying.

D)

E)

Technology/Tools Recommended: None

Other Observations:

Non-NASA barriers included:

• Environmental Permitting

• Restricted Air-Space Identification

• Congressional Budgetary Processes

Strategic Resources, Inc. (SRI)

B-22
NASA Fast Track Study

7. Evolved Expendable Launch Vehicle (EELV): The EELV is a medium to heavy

payload launch vehicle to replace the Delta, Atlas and Titan launchers. It is a DoD created

program to fulfill the nation's need for a lower-cost, more reliable space launch systenl One of

the potential contractors for EELV is proposing to use the Space Shuttle Main Engine (SSME)

as the engine for the EELV. The SSME would then jettison during flight, parachute for water

recovery, be refurbished and re-used for another launch. The Center's participation, through a

Space Act Reimbursable Agreement with Boeing, is to conduct a SSME drop test and recovery.

Following recovery, the SSME will be test fired.

A) Practice Oriented Ideas: Small co-located teams ensure effective communication and

minimization of outside distractions. Team members must be self-motivated, dedicated,

and committed to the project. Matrix management of the team allows the project manager

to adjust manpower levels to meet the activities' peaks and valleys. A strong project

manager leads by example. It is hard to expect team members to put in extra hours if the

project manager doesn't. Additionally, people need to be allowed to do their jobs, they do

not need micro-management. Let people know what is expected and what they are

responsible for and hold them accountable for results. The project manager should have

final authority in the selection of the team.

If the project team does not have resource control assistance, the project manager should

maintain close contact with the customer, support contractors and Comptroller's Office to

ensure costs are contained.

Small projects should use a simplified project plan that includes the purpose, scheduling,

and issues/concerns. Requirements should be maintained in the Project Requirements

Document (PRD).

B) Policy Related Comments: Under Full Cost Accounting, the proposed overhead rates

applied to civil servants both for Headquarters' overhead and local site's overhead rates,

may exceed the cost of using a contractor. In this case, project managers may choose to

spend scarce project dollars on contractors which could have the opposite effect on

NASA's stated goal of maintaining the core capability, knowledge and experience within

NASA.

C) Cultural Changes: A project manager must be careful concerning the issue of using a

contractor versus government personnel It is felt that, as a general rule contractors will

comply with the project manager's directions, while government personnel are more likely

to challenge or question the project manager's decisions. This makes it imperative that the

contractor team members should be strong individuals that are equal members of the team,

willing to voice their opinions.

Strategic Resources, Inc. (SRI)

B-23
NASA Fast Track Study

The NASA procurement system is a slow to react, bureaucratic system with many gates,

statusing, and reporting requirements. It is much faster to have the contractor do

procurements and, many times, a contractor is an expediter whose sole fimction is to make

sure the procurement is completed and the product delivered when needed.

The cost reporting system does not adequately status for short projects. Data is at least a

week old and may be as much as a month old.

The biggest barrier to project management, is statusing to senior management. NASA

management needs to be more proactive in finding out what goes on and come to where

the action is.

D) Technology/Tools Recommended: None

E) Other Observations: A Fast Track Project is defined by time flame and not cost. It is

more an aspect of time with respect to complexity and cost is not a factor.

Strategic Resources, Inc. (SRI)

B-24
NASA Fast Track Study

8. Halogen Occultation Experiment {HALOE): The HALOE project was one of ten

instruments on the Upper Atmosphere Research Satellite (UARS) Platform hunched in

September 1991. HALOE was designed to take polar (northern) region measurements during

two consecutive winter seasons. R was required to operate on orbit for a minimum of two years.

UARS was originally designed as a recoverable platform to operate two years on an orbit

operational lifetime and contained international experiments. Total cost for the UARS mission

was approximately $700 million. The HALOE life cycle schedule was approximately 20 years

from concept to launch.

Technical performance was treated as the number one risk factor required to meet technical

performance at the 100 percent level. Once UARS was identified as the platform then the

HALOE project schedule was fixed and was non-negotiable. Resources were manager

(controlled) to ensure schedule did not slip. Schedule was the driving metric, and could not slip

at any cost.

The HALOE project was originally under contract to TRW, but was brought in-house about eight

years prior to launch and reworked. The Systems Engineering Division of the Systems

Engineering and Operations Directorate provided the project management function.

A) Practice Oriented Ideas: NASA needs to pay more time and attention to the details of

system integration and system testing as part of the planning done during early Phase A.

A NASA integration team was sent to the General Electric Astro

Division in Heightstown, NJ during spacecrat_ integration.

A standup briefing was held between all personnel of the changing shifts;

line workers did most of the talking during the standup briefings.

Clear lines of authority existed, no ambiguity of responsibility.

Leadership was provided, there was no management by consensus. (This

was helpful to this project and would probably be critical to a Fast Track

Project).

To obtain key team personnel, Branch approval was required for a person

within the same Division and Group Director approval was required if'the

desired person was in a different Division.

Strategic Resources, Inc. (SRI)

B-25
NASA Fast Track Study

• Project Status reporting requirements:

• Weekly status reports on the formal Action kem list

• Monthly reports to the Group Director

• Quarterly Center Director's management meeting

HALOE had a written and approved project plan. (This was helpful

to this project and would probably be critical to a Fast Track

Project).

A formal process operated out of the Project Office was used to

maintain configuration control. Several of the fabrication technicians a

lso served as QA technicians. (Helpful to this project).

Funding and workforce profiles were re-baselined almost annually. The

workforce profile tended to follow the funding profile.

• No form of official risk management was performed.

Launch was controlled by KSC and/or JSC. The instrument due date was

controlled by the Integration Manager at General Electric.

Daily morning stand up information exchanges were very

useful/successful.

Having a written contingency plan for the failure event during

Thermal Vac testing was very successful.

More early discussion and detailed planning of the integration and test

activities would enhance Fast Track management, and reduce the amount

of resources expanded during this time frame.

B) Policy Related Comments: Under Full Cost Accounting, the total life cycle cost of the

project (instrument only) would be about the same.

Recommend no Preshipment Readiness Review (PSRR) for a Fast Track Project. The PSSR

occurs too late in the development to really help identify problems.

C) Cultural Changes: A good project manager has a hard time returning to the line

organization. They become used to the intensity of project work, exercising authority and life

at a faster speed.

Strate(tic Resources, Inc. (SRI)

B-26
NASA Fast Track Study

The Integrated Management Team (IMT) was viewed as ineffective. Firm lines of authority

are required to expedite the decision process. (This is critical to a Fast Track Project).

This instrument was built in the era where science requirement creep was expected. If

anything, scope was added not subtracted.

The current relation between NASA and space platforms is in the process of changing just like

the past relationship between NASA and sounding rockets. A fundamental shift is occurring

in the nature of how NASA does space platforms.

D) Technology/Tools Recommended: None were prevalent at the time HALOE was built.

E) Other Observations: HALOE was driven by science requirements, however due to the

extremely long life-time of the project from original concept to flight a lot of technology

driven requirements crept into the mission.

In-house managers (two Deputy Project Managers and two Technical Project Engineers) were

replaced due to advancement. No one was removed due to incompetence.

Strategic Resources, Inc. (SRI)

B-27
NASA Fast Track Study

9. In Space Technology Experiment Proffram (INSTEPS: Code X Small Flight

Experiments started in 1988. One hundred ninety were proposed to the Announcement of

Opportunity (AO). Thirty experiments were funded for Phase A ($50-100K), six for Phase B,

and four advanced to Phase C/D and flew in space. The total life cycle cost of the project was

approximately $5 million and the total life cycle schedule was 12 months.

Risk was limited to the individual projects. There was not a written and approved project plan

due to the small size of the project. The project lost funding in the last year.

A) Practice Oriented Ideas:

• Quarterly written reports to Headquarters.

• Agency control gates: Request for Proposal (RFP), AO, Non-Advocate

Review (NAR), Flight Experiment Board (FEB), Formal Annual Review.

• Pre-phase A was done on Center funding; Headquarters funded Phase A study.

• Mission Performance Requirements were developed from '_ears of

experience."

• Configuration control managed by individual projects.

• Project status and progress tracked by simple schedule, Assessment by Project

Manager's feeling on cost and expense.

• A barchart of resources was used as a periodic report of costs and fimding.

Funding profile was a classic ramp-up over two to three years for Phase B.

Workforce profile was uncontrolled; management did not know what labor

was needed.

• Procurement process began when funding was received from Headquarters at

the end of Phase/k

• Recommend working with one procurement person for a general standard

contract.

• Technical Division at Center shared approximately $3 million of the $5 million

total cost.

• If using Graduate Students, quality and quantity ofworkforce is variable.

Strate_lic Resources, Inc. (SRI)

B-28
NASA Fast Track Study

Fast Track management should get the right people (discipline and experience)

around a table to brainstorm before initiating design, including technical input,

as they know what has worked in the past.

B) Policy Related:

• May have driven some projects over cost if Full Cost Accounting was used.

A realistic schedule to actually build the project should drive the location of

reviews for a Fast Track Project. Magnitude of work should drive the

schedule.

Recommended project control gates for a Fast Track Project; NAIL chaired

by Headquarters, System Requirement Review, and some Design Reviews.

Reviews force a project to stand up and explain all details and to explain why

internal decisions were made. The Systems Requirement. Review is the real

key review. There needs to be a methodical process to decompose goals and

objectives into specifications.

C) Cultural Changes: Overhead of a shuttle or space station (MIR) experiment not well

thought out in the planning phase. Upfront planning is essential for Fast Track Projects.

International projects may be impacted by language barriers and require careful planning.

Finding a Project Manager and Principle Investigator (PM/PI) who wanted the job is a

barrier. Keep pushing on upper-level Center management to reject proposal if no people

are available. Need to ensure that PM's and PI's knowledge, skills, abilities and expertise

match the project's goals and objectives.

D) Technology/Tools Recommended: None

E) Other Observations:

• Most projects were US; concerned about spending US dollars to fund technical

development overseas.

• The most challenging part of the project was finding a competent project

manager at Headquarters to sponsor.

Strategic Resources, Inc. (SRI)

B-29
NASA Fast Track Study

• Most projects that flew were suocessfuL

• Most significant lessons learned: make processes orderly with reasonable

schedules; not too short and not too long (between phases).

• Most innovative practices: not too rigorous; minimal paperwork; possible

because of low cost ($1 million to $5 million).

Strategic Resources, Inc. (SRI)

B-30
NASA Fast Track Study

I0. Lidar In-Space Technology Experiment (]Lite Instrument): A totally in-house project

with mostly civil servant workforce with local contract support. Project was shuttle space

experiments with a $25 million Research and Development (R&D) budget, (including civil

servant personnel funding). The total life cycle was ten years and started in January 1985. Post

flight test was in January 1995.

The project management function was performed by the Lite Project Office for five years, moved

to project development for four years; then the project development was reorganized into five

project offices.

Project success was defined by 45 hours Light Direction and Ranging (Lidar) data over many of

different atmospheric conditions (no metric on data quality).

A) Practice Oriented Ideas:

• A written and approved project plan was developed.

• Approval for key personnel was up to Flight Electronic Division (FED) if

within own line or up to Director Level if within Directorate.

• Project Status and Reporting requirements:

• Weekly staffmeetings

• Monthly Mission Information Control (MIC) [no longer used].

• Reporting driven by Center Director.

• Did not use IFFs used classic hierarchy. Project Manager (PM), Assistant

Project Manager (APM), and IM were co-located.

• Center level control gates were not formal. Went through Center Director.

• Agency level control gates: a few presentations in late 1985 and CDR at

Center in January 1986 with visit by Headquarters and approval for year two.

Center sent representative to Headquarters on task to generate job for Center

on laser development. Feasibility study conducted by NASA personnel from

January 1984 to January 1985.

• No formal process to develop Mission Performance Requirements.

Strategic Resources, Inc. (SRI)

B-31
NASA Fast Track Study

A Product Assurance Plan called for configuration control to start at

instrument integration. Latitude given on eleotronics design up to insmmaent

integration.

• No formal metrics were established. Weekly status staff meeting fostered

environment of open communication where it was okay to state problems.

• Project was re-baselined annually due to fluctuations in funding source. Code

at Headquarters would fund what was left in budget.

• Risk management was not formally addressed.

• Funding profile changed annually. Workforce profile was independent of

funding profile.

* Wrote Laser SOW and Center issued RFP to attract potential mission partners.

Contract was a cost plus fee and Center managed.

• Design requirements: bottom up to build best Lidar possible for available

funding. Should have a technology demonstration.

B) Policy Related Comments:

Near the end of 1993, Mr. Dan Goldin, NASA Administration, imposed

Independent External Independent Readiness Review (EIRR). This was a

waste of time as it came too late in the project (three months prior to launch).

Review panel had no authority.

• Keep the Critical Design Review (CDR), Preliminary Design Review (PDR),

Flight Readiness Review (FRR); but not the E1R1L

Recommended project control gates for a Fast Track Project: Conceptual

Design Review (CODR), PD1L CD1L Preshipment Readiness Review (PSRR),

FRR

C) Cultural Changes:

Under Full Cost Accounting, approximately $50 million for Full Time

Equivalent (FTE) over ten years and $25 million Research and Development

(R&D) funds for instrument development.

• Funding was to be as Level of Effort (LOE), but actual funding varied as

project expanded from five to ten years.

Strate_lic Resources, Inc. (SRI)

B-32
NASA Fast Track Study

• Consumed a lot of time and people to prepare and do reviews when they

should have been training for operations.

• Procurement is a problem Barrier is education of procurement people that

space flight parts are not going to be procured the same way as office supplies

are; lots of sole source buy.

• Have to live with barriers and work the problem (Space Tec and other

Centers); contractor taking direction from civil servant at Center and incurring

cost that project office did not budget.

D) Technology/Tools Recommended: None

E) Other Observations:

• Project was not process oriented.

• Most important lessons learned from experience with this project:

• Entire management was school of hard knocks, and no failure resulted

in loss of career potential

• A Fast Track Project must be strictly focused and use readily available

technology; Instrument and Spacecraft, no technical development.

• Recommend a ,not to break" 36 month development Authority to Proceed

(ATP) to hunch for full mission; part of mission for less time.

• If really looking to do Fast Track Projects, PM must have full authority and

resources; PM needs autonomy to do job; if PM fails, fire PM, don't try to

direct the work; and be receptive to request for problem/resolution

suggestions.

Strategic Resources, Inc. (SRI)

B-33
NASA Fast Track Study

IL Non-Interview Data Gathering

1. Goddard Space Flight Center (GSFC) Senior Group Fast Track Brainstorming

Ideas: This is not an interview, but a summary of a Senior Project Manager Group

Brainstorming session conducted July 17, 1996. The group was asked to brainstorm new ideas

on Fast track Project management.

A) Practice Oriented Suggestions:

The Project Organization:

• Should be co-located, a skilled team

• Centralized support office

• Project tailors activities to the mission

Risk Management Approaches:

• Project delivers a risk plan

• Same team does similar missions (product line)

• Low technology requirements

Engineering:

• Choose from a Component Supermarket (Off-the-SheLf)

• Use System Engineers (SE) as architects

• Must provide an early bridge with customer

• Do Concurrent Engineering

Resources:

• Partnering arrangements

• Reinvestment

• Encourage unsolicited proposals

Status Reporting: Limit to chosen metrics

Strategic Resources, Inc. (SRI)

8-34
NASA Fast Track Study

B) Policy Related Suggestions:

Project Organization:

• Multi-faceted, cross-trained

• Cycle or rotate "doers and watchers"

• Report to a single management level

• Project "contact" with agency

• Put trades in project plan

Risk Management:

• Portfolio of risk (low & high)

• Up front definition of success

• Project does Quality Assurance (QA) oversight

• Minimize Center requirements

• High level acceptance of failure

• Allow for trades

• No added requirements

Engineering: None suggested

Resources:

• Total up front, timeless dollars

s No Federal Acquisition Regulations (FAR)

• Liberal interpretation of NASA FAR Supplement

• Unrestricted subcontracting

• Go to fixed funding cap arrangements

• No unfunded mandates

Status Reporting:

• Team self-assessment

• Respond to single customer

• Status problems only

• Limit performance reviews

Strategic Resources, Inc. (SRI)

B-35
NASA Fast Trick Study

C) Cultural Changes:

• Project Organization is an enabled, small team

• Risk Management should be a no fault experiment in the early years of Fast Track

Projects.

• Engineering should make the customer part of SE team

• Management should add incentives for saving money on contracts

• Status Reporting: no suggestions

D) Technology/Tools: Recommend use of electronic status reporting

E) Other Observations: None

Strategic Resources, Inc. (SRI)

B-36
NASA Fast Track Study

2. GSFC Junior Group Fast Track Brainstormine Ideas: This is not an interview, but a

summary of a NASA/GSFC Junior Project Manager Group Brainstorming session conducted July

25, 1996. The purpose of the session was to brainstorm radical ideas on fast track project

management. Participants were asked "how I would manage my own company" in a Fast Track

Project environment. The results emphasized minimizing paperwork, presentational status

reporting, and duplication of efforts.

A) Practice Oriented Suggestions:

• The Project Organization:

• Teams consist of less than five people each

• Concentrated project team

• Co-located project team, including contractor

• Replace handbooks with cookbooks; (ideas rather than rigid

guidelines)

• Annual internal re-assessment of project

• Risk Management:

• Build Quality Assurance (QA) into design; not an oversight

function

• Benchmarking at start of project

• Risk assessment of new technology

• Bottomline threshold of pain on risk (acceptable impact

assessment)

• Minimize new technology

• Document Lessons Learned for the future

• Resources:

• More cooperative agreements with contractors

• Use contractor's procedures in WBS, minimize scheduling,

Reporting, QA, etc., to avoid duplication

• L_eral interpretation of Federal Acquisition Regulations (FAR),

NASA FAR Supplement

• Use past performance as primary means for choosing

suppliers

• Maximize objectives in award fee determinations

• Use blanket travel orders to minimize paperwork

Strategic Resources, Inc. (SRI)

B-37
NASA Fast Track Study

Status Reporting:

• Daily videoconferencing

• Replace formal reviews with peer reviews

• Eliminate paper documents

• Review hardware and software, not presentations

• Hands on reviews

Miscellaneous:

• Limit specification and requirements documents to less than

six pages

• Build items that are more integrated

• Minimize technical interfaces

• Take advantage of every loophole

• Eliminate redundancy

• Use existing software and ground systems

B) Policy Related Suggestions:

• Project Organization:

• Phases should be fluid and flexible

• Use merit pay to reward accomplishments; negative pay for failures

• Project manager has more authority/responsibility for project

• Government project office located at contractor's plant

• Resources:

• More Commercial OffThe Shelf(COTS) contracts

• Develop penalties to your suppliers for poor performance

• Modify cost sharing in cooperative arrangements

• Up front, timeless funding including reserves

• Shorten, simplify procurement

• No mandatory goals in contracting

• Tie award fee into on-orbit performance

• Utilize cost incentives in contracting

• Miscellaneous: None

Strategic Resources, Inc. (SRI)

B-38
NASA Fast Track Study

c) Cultural Changes:

• Project Organization:

• Minimize power centers

• Rotate people

• Term limits on management

• Risk Management: Up front risk management philosophy

D)

E)

• Status Reporting:

• Fewer major reviews, less people

• Interactive around the table rather than presentational

• Miscellaneous: think small, fast in all project areas

Technology/Tools:

• Resources: computerized Configuration Management (CM)

• Status Reporting: do electronically

Other Comments: None

Strategic Resources, Inc. (SRI)

B-39
NASA Fast Track Study

3. Lewis Research Center (LeRC) Pyramid Team (Pare Down sue2estions for NHB

7120.5_: This is not an interview, but a collection of suggestions from LeRC. The team

consisted of three persons, two experienced Space Experiments Dixrision Project Managers, and a

new Aeronautics Program Project Manager who has Reliability/Quality Assurance experience.

They were chartered with using the existing (Nov 1993) version of 7120.5 and paring down the

stated requirements to the core elements necessary to adequately plan and execute a Fast Track

Project. Their output was related back to specific elements of the 7120.5, with appropriate

interpretive comments and ideas. Background discussion output from APM-23 SSG deliberations

on the nature of Fast Track Projects was shared with the group. The group was briefed on the

overall charter, and tasked with a separate focused approach to developing Fast Track guidelines,

and were encouraged to define the nature of a project that could be deemed Fast Track.

A) Practice Oriented Suggestions:

• Each Center should develop a Fast Track implementation plan, consistent with

the types of projects for which they are responsible.

• For a project to go the Fast Track route it must:

State the intent for Fast Track at RDR/NAR (in Phase B) and in the

Project Plan, agreed to by primary stakeholders (project manager,

program manager, principal investigator, industry partners, etc.)

• Receive approval for Phase C/D/E Fast Track at Authority to Proceed

(ATP).

B) Policy Related Suggestions:

* The Fast Track definition applies only to phases C/D/E of the project life cycle:

Not required to pay for the assurance of lowered risk (per risk

definition of NHB 7120.5, 11/95 version, table 6-1), which is

manifested by removing control gates such as formal reviews,

inspections, etc. (safety requirements will not be compromised).

Strategic Resources, Inc. (SRI)

B-40
NASA Fast Track Study

A project is appropriate for Fast Track when:

• There are no undeveloped technologies necessary for mission

accomplishment.

• Requirements are clearly and completely defined (system design

specifications are complete with no development issues) at RDR/NAR.

The minimum requirements for Fast Track Projects are:

• Phase C Requirements

(NHB 7120.5A, Table 2-3, p. 2-25)

- Validated PCA - annually

- PMC Review Baseline - as necessary

- CDR Specs Baseline - as necessary

• Phase D Requirements

(NHB 7120.5A, Table 2-4, p. 2-29)

- Validated PCA - annually

- PMC Review Baseline - as necessary

- IRK Reports - per PCA

- Operational System- end of Phase D

• Phase C/D Requirements and Reviews

(NHB 7120.5A, Figure 4-I, p. 4-2)

- Omit all IARs

- Omit first IRR review and maintain only second one (at end

of Phase D)

- Have PMC review in Phase C or D only as necessary

- Omit all QSRs

C) Cultural Changes: None

D) Technology/Tools: None

E) Other Comments: None

Strategic Resources, Inc. (SRI)

B-41
NASA Fast Track Study

4. Lewis "Clean-Sheet" Team: This is not an interview, but a collection of suggestions

from Lewis Research Center. Background discussion output from APM-23 SSG deh'berations

on the nature of Fast Track Projects was shared with the group. The group was briefed on the

overall charter, and tasked with a separate focused approach to developing Fast track guidelines.

They were encouraged to define the nature of a project that could be deemed Fast Track.

Our team spent some time discussing what a Fast Track project should be in terms

of size, schedule, complexity, safety risk, and so forth. We concluded that the

definition of what project should be Fast Tracked was really not critical to the

determination of how the project should be managed flit is to be successful in the

world of "Better, Faster, Cheaper". In fact, we felt that these particular changes to

project management were relevant to all projects around NASA in the interest of

getting them done at whatever rate

Key assumption: The project manager on a Fast Track program must be experienced and

motivated.

A) Practice Oriented Suggestions:

The Fast Track Project manager must have been through the traditional project

development so that he/she is aware of what things are necessary and what are

not. Experience is a must to be able to make the decisions that are required

throughout the project.

The project manager must also believe in the value of the project itsel£

otherwise he/she will not be able to motivate the team.

• There must be a clear agreement on deliverables between project manager and

his/her single point of contact.

Project requirements are agreed upon at the beginning of the project.

• Whatever the nature of the project, the mission goals need to be

explicitly spelled out in the beginning of the project. The project

manager needs to be able to make trade-off decisions during the

development of the project that will effectively get the project

completed but not at the expense of the mission goals.

• Goals are agreed to directly between the project manager and the

customer, whoever that may be.

• Procurement must be streamlined for Fast Track projects.

• Positive incentives should be used to meet schedule and cost.

Strate_lic Resources, Inc. (SRI)

B-42
NASA Fast Track Study

B) Policy Related Suggestions:

• The project manager needs the freedom to choose his/her own team members

and to change them during the project if necessary.

• The project manager reports to a single person for project progress reviews

• The format of the project progress review would be at the discretion of the

project manager as a way for him/her to best represent the project status.

• The design reviews are for the project manager and are held within the project

team The review process should include reasonable but not burdensome

documentation.

The project manager must have financial and schedule flexibility.

direct access to decision makers, both

c)

• The project manager must have

programmatic and financial.

Cultural Changes:

If a project manager saves money, he/she is usually rewarded with less money

to spend in the future. This system does not encourage practical spending of

money throughout the life-cycle of a project and certainly does not provide for

a management reserve which is imperative in a program

Individual recognition for doing a job well needs to be improved. People work

better when they are appreciated and when their work accomplishes something

that is valuable to someone.

D) Technology/Tools: None

E) Other Comments: None

Strategic Resources, Inc. (SRI)

B-43

NASA Fast Track Study

L

HI

me

Vl

Appendix C

SRI Space Programs Interview Compilations

Jet Propulsion Laboratory (JPL)

1. JPL Reengineering Effort (E. Kane Casani, Michael J. Sander,

Bob Metzgar)

2. Mars Explorer Program (Donna Shirley)

3. New Millennium Program

a. Deep Space 1 Project (David H. Lehman)

b. Mars Microprobe Project (Sarah Gavit)

4. Pluto Express Project (Robert L. Staehle)

5. Clementine I (Dr. Henry B. Garrett)

Applied Physics Laboratory (APL) at Johns Hopkins University

1. Discovery Program, Near Earth Asteroid Rendezvous (NEAR)

Project (Thomas B. Coughlin, A. Santo, Larry Crawford)

Hughes Space and Communications Company

1. Communications Satellite Development

Lockheed-Martin Corporation

1. Iridium Project

Spectrum Astro Corporation

1. New Millennium Program, Deep Space 1 Project (Start Dubyn)

PPMI_

C-5

C-7

C-9

C-11

C-16

C-19

C-21

C-23

Strategic Resources, Inc. (SRI)

C-1

NASA Fast Traok Study

L ,let Propulsion Laboratory (JPL)

1. JPL Reengineeriw, Effort: A 1991-1992 benchmarking study looked at development

process reengineering in industry and divided companies into two categories: Product

Organizations for those with Space oriented products similar to JPL; and Process Organizations if

they had different products but similar development issues and processes. An example of the later

would be an automobile company with the process of developing a new car line.

Documentation had been lost for this study, but contacts from the interviews were made available.

Results showed that success for these organizations was due to about 10-20 percent as a mix of

technology applications, and 80-90 percent as cultural change. JPL has proven these results,

cultural change is what ensures success. This study also interviewed business managers in the

organizations contacted to do a capabilities evaluation and arrived at similar conclusions.

Following this study, the JPL director, adapted the principles of Michael Hammer, Hammer &

Champy (Reengineering the Organization). One of Hammer and Champy's basic tenets, is that

you cannot re-engineer everything, you must be selective. JPL sculpted this selectivity with key

JPL process personnel, focusing on JPL customers and stakeholders. It initially encompassed

four basic JPL processes with a fifth added several months ago. These processes are: 1)

Development of New Products; 2) Growth and Assignment of People; 3) Development and

Management of the Institutional Environment; 4) Business Systems; and 5) Enterprise

Information Systems. These five areas were seen to have the best potential influence along the

lines detailed by the JPL Strategic Plan. While Development of New Products perhaps has the

most influence in this study's areas of interest, it was clear that each of the areas contributed to

the ability to identify beneficial change leading to operation in a "Better, Faster, Cheaper"

environment.

Strategic Resources, Inc. (SRI)

C-2
NASA Fast Traok Study

_trs Man Exuloration Prom'am: Ms. Donna Shirley is the JPL Program Manager for the Exploration Program series of missions. The Mars Exploration Program is one of the

pioneering approaches to "Better, Faster, Cheaper" space exploration which will be active at least

over the next ten years and perhaps longer.

The Program's goal is to visit Mars as often as possible within budget constraints with a

"continuous infusion of new technology', t Much of Ms. Shirley's ideas are in her paper, Mars on

$300K A Day, reprinted in Appendix E of this study. In addition to a summary of the overall

Program, there are valuable thoughts on a business analysis approach to the Program, a summary

of the Program Strategic Plan, and Program organization, including industry and science

partnering, and cost containment practices.

A) Practice Oriented Ideas: Donna identified a problem with organization buy-in to

projects by those who hope to buy-in just to be sure to be in on a project. Buy-in has been

a common practice for projects. There is nothing wrong with a company being aggressive

to capture a program and make up for an initial low profit through efficiencies on later

missions. However, buy-in could be a real barrier to long term success in a "Better,

Faster, Cheaper" era. The first mission in a series could achieve goals, but later missions

could become more expensive. It is important to get a good solid bid the first time that

will carry through an entire project. Once on the project, it is very important for project

managers to be able to hold really good cost reviews and descope if necessary to hold

costs. There are several other practice related suggestions in her paper.

B) Policy Related Comments: A very large barrier to cost and schedule management in

any project relates to project requirements. Ms. Shirley's First Law of Project

Management is: "A requirement ain't a requirement until somebody's willing to pay for

it." Ms. Shirley further stated that "This is the key to everything." She even suggested

expunging the word requirement from the NASA Project Management lexicon until we

can get to a better, common understanding of just what we mean when we say

requirement. Requirement should mean that desires and capabilities have been defined,

compared and matched. Once this is competed they can become a contract. When

requirements for a project are agreed to, management must hold firm to that common

definition with respect to cost and schedule objectives until, and unless, a sponsor for

changed requirements steps forward with the ability and willingness to pay for their

addition and accept the schedule impact.

1 =Mars on $300K per Day': The Mars Exploration Program, Donna Shidey

Strategic Resources, Inc. (SRI)

C-3
NASA Fast Track Study

In addition, the project office, when soliciting participants, nmst learn that in order to keep

costs low we nmst change _om detail specifications of the past to performance

(m_imlli_) specifications. This is analogous to the military's move away _om 1Wditary

Specifications (lVfilSpec) and Mifitary Standards (l__dStd). Other benefits also accrue with

this approach, in simplicity of development and integration.

C) _ Cultural Changes: Ms. Shirley's suggestions here dealt mostly with the need to have a

more business-like approach, including knowing how to relate to your gain (profit) _om

the program and to know your competition. Her paper also addresses these ideas in more

detail

D) Technology/Tools Recommended: None

E) Other Observations: None

Strategic Resources, Inc. (SRI)

C-4
NASA Fur Track Study

3. New ]Vlillennium 1)ro21-am

a) Deep Space I (DS1) Project. First in a series of New Millemfium Program missions,

DS1 will validate technology for use on fitture space missions into the 21st Century. There is a

list of primary requirements and secondary goal technologies to validate, plus Mission Goals that

are not part of Primary Mission Success Requirements. These Mission Goals include a plan to fly

by one comet and one asteroid, returning imagery and spectral data. The basic Project is capped

at $138.5 million. The Project is managed for JPL by David I-L Lehman as the JPL Flight Lead.

The spacecrait development contractor is Spectrum Astro Corporation, and were interviewed

separately for this study.

A) Practice Oriented Ideas: This project drew much from the JPL geengineering

Process Action Team (PAT) output and these included using a revised parts

development/procurement process and an improved schedule-receivable/deliverable

process system.

Another practice which aided in successful management of a "Better, Faster, Cheaper"

project was in selection of suppliers for the development effort. The project management

team saw that there would be six Integrated Product Development Teams from which

would flow 13 technologies. Requests for Proposal (RFP) were released for each of the

teams, requesting a three to five page proposal. After a down-select process, more detail

was asked from those remaining. This submission was scored and a final selection made.

B) Policy Related Colnments: The management approach which is most responsible

for any success achieved to date is the variable scope and capabilities given by NASA for

this project. This came from the above mentioned definition of primary requirements and

secondary goals for mission success. In the past everything was a requirement, leaving the

project manager with little latitude if part of the development for the mission ran into

di_culty. Reserve may be better applied on just the primary requirements, with secondary

goals not driving cost and schedule

A second area, the result of ongoing efforts within JPL for better management of projects,

is an attempt to limit documentation to only that necessary for the project.

Documentation requirements had been cut by one third between Casini and Mars

Pathfinder. DS 1 has cut that by another hal£ This makes the project more cost effective,

with the team making decisions based on planned management needs during the course of

the project.

C) Cultural Changes: Encouragement of innovation and improvement in processes to

meet firm requirements was a key tool to ensure cutting development time. Previous

missions would have been given, and would have taken, six years from start to launch. It

is now given two years, and the team expects to make it. In other words, everyone works

together as a team on cost, schedule and technical performance to ensure project success.

Strategic Resources, Inc. (SRI)

C-5
NASA Fast Track Study

D) Technology/Tools Recommended: A good communication system is a big factor in

holding down cost for the project. The key tool in this system is a top of the line video

conferenoing capability, paid for by JPL, both at their end and with their primary partners,

in this case Spectrum Astro. The system was paid for with approximately one month's

savings in travel costs. Time away, travel time and non-productive spin-up/spin-down

time for individuals has also been cut drastically, making for more productive project

activities.

E) Other Observations: A copy of the DS-1 Project Plan is provided in Appendix E of

this study.

Strategic Resources, Inc. (SRI)

C-6
NASA Fast Traok Study

3. New ]Vlillfnninm l_'OE]'am

b) Mars Microprobe Project. Mars Microprobe is one of several experiments planned

for missions to the Planet Mars into the 21st Century.

A) Practice Oriented Ideas: Teamwork is the key to success. Management basics are

reduced for the project manager: 1) getting the right people and keeping them motivated;

2) being the one person who steps and stands back to see the big picture and big

problems; and 3) having a plan and making sure everyone understands it. These are

expanded in order.

Getting the fight people is somewhat subjective and involves knowing the necessary

discipline/fimctional areas to be used, the people in them, and asking for them by name

when the project starts. Motivation involves giving team members real responsibilities on

the project and with the team; "ensuring they all understand that..." which involves

directing team members to write down their plan and schedule, and expecting it to be

followed. This further involves enabling team members to be able to communicate and be

involved with the rest of the team.

Directing things to be done, means getting the team to sit down, deal with the project

from a big picture standpoint and ensure that they understand that they will live with their

plan.

Being the Big Picture person for the team means making sure things make sense overall.

This again involves ensuring "they all understand that..." which involves getting the team

members to understand each other's activities as well as their own.

Having a good plan comes from two management concepts. The first is having and

documenting good requirements. This will include using common terminology that is

thoroughly documented. The second is using this documentation to lead the team in

creating a bottom-line schedule from each of the detailed schedules.

B) Policy Related Comments: Functional and design reviews are team functions, held

when needed, not at a fixed date, and kept as informal as possible. "Better, Faster,

Cheaper" is not an excuse for sloppiness, and documentation of issues and necessary

actions is very important. The project manager must strive for distinction between

excessive formality and making the review process be a good solid comnmnications tooL

Each element of the project (sub-systems, QA, ME/SE, Packaging, etc.,) is required to

have three reviews leading up to corresponding Project Reviews. Reviews at the element

level are in-process peer reviews and are called Conceptual, Interim and Final Reviews, as

are the corresponding Project Reviews. They take the place of the more formal

Preliminary Design Review and Critical Design Review.

Strategic Resources, Inc. (SRI)

C-7
NASA Fast Trek Study

Since these reviews are focused on issues, nmzh of the work is resolved at the dement

level Outside team members are brought in to ensue that the reviews are real team

functions. Action items for these reviews are called advisories and are consolidated and

documented before the meeting ends.

The end objective for the reviews, as for the project, seems to be to have a plan that

everyone understands. This is accomplished by documenting the plan and ensuring that it

is built on good requirements.

C) Cultural Changes: No specific changes were listed, but allowing the project manager

the flexibility to set and manage the project design review process may be something that

would enable Fast Track Projects to move ahead smoothly with minimal disruption.

D) Technology/Tools Recommended: Contract Management is a tool the project

manager must have in order to ensure subcontractors can fully participate as team

members. The project manager must understand FAS.

E) Other Observations: None

Strategic Resources, Inc. (SRI)

C-8
NASA Fast Trek Study

4. lquto Express Proiect. 1lie JPL Ice and Fire Pre-Project Manager, which inc_des Pluto

Express, Europe Orbiter and Solar Probe, is Mr. Robert Staelde. Several times Pluto Express

was mentioned by others in the past few months as an icon for "Better, Faster, Cheaper" projects.

Pluto Express is an effort which _ launch two sciencecrafls to explore Pluto and its moon

Charon at the end of a 10-13 year set of trajectories. The idea of a Science, craft approach

identifies "scientists and engineers working together with a high degree of communication, from

the beginning." 2

A) Practice Oriented Ideas: A large part of Mr. StaeMe's focus for success in "Better,

Faster, Cheaper" is on Design-to-Cost (DTC) as a way of managing. The design process

also considers Test and Operation requirements. It is easier to ask questions related to

these areas in the design phase, and to make necessary changes based on objective

conclusions, than it is to determine problems and issues later in order to resolve negative

impacts. The approach that allows this to occur naturally is to form teams based on Cross

Functional Integration, including the Flight System Design Team, mission assurance,

software, mission operations, launch systems, radioisotope approval, procurement, etc.

Team member co-location was not possible on this project, so Mr. Staehle said that he

considered it important to get the team together frequently, in lieu of co-location.

B) Policy Related Comments: Mr. Staehle had no specific recommendations for policy

related changes, but provided a list of process improvement initiatives developed for Pluto

Express by the team. This list is in Appendix E of this study. One item, commonly

discussed by other organizations meeting with some success in reengineering development

processes, and present in this program, is minimal formal documentation to help hold

down costs. Examples given for areas being looked at currently include on-line

documentation; requirement documents no more than one page; and, documenting new

designs by as-built next-earlier generation documentation, plus deltas.

C) Cultural Changes: Mr. Staelde said that as Pre-project Manager he wanted to move

from a normal mode of periodic operation to one of continuous operation. This refers to

managing team members from all areas continuously, rather than as a series of short and

periodic project involvement. This has implications for dedication of matrixed personnel

from the line organizations, and translates to a project goal of maintaining equal to or

greater than 0.90 dedicated personnel time, supplemented by equal to or under 0.20

consultant time, referring to ratios of Full Time Equivalent personnel. It is also very

important to carry as many Pre-project personnel into the project as dedicated personnel

as possible. In this part of the discussion he referred to Robert Townsend's U__ the

Organization as a source of ideas for this approach.

2 JPL Pluto Express Brochure, version 1.1,616/96

Strategic Resources, Inc. ({JR/)

C-9
NASA Fast Track Study

D) Tedmology/To0ls Recommended: A Cost Analysis Tool, PCAT is recommended by

Mr. Staehle that allows everybody to do Design Analysis and Cost Impact Analysis

together. It is a simple spreadsheet tool developed at JPL that allows each area to

separately develop Work Breakdown Structures based on cost estimates that roll up into a

total project cost analysis. A related project trade-off model allows quick assessment of

end-to-end cost impacts of proposed mission and design changes.

E) Other Observations: None

Strate_/ic Resources, Inc. (SRI)

C-10
NASA Fast Trek Study

5. Clemcntine I Prelect: Clementine I was a Ballistic hfts_e Defense Organization (BMDO)

sponsored program l_om the Naval Research Laboratory (NRL), performed in conjunction with

Lawrence Livermore Labs and NASA. Science objectives led to methodology which included a

lunar fly-by and asteroid rendezvous. The project was characterized by a very short, tight

schedule and a very limited, capped budget.

A) Practice Oriented Ideas: Clementine I used a less conservative approach to this

project than other space projects. There was more of a willingness to try new, untried

approaches to meet cost, schedule and science objectives. The severe budget and schedule

constraints forced the team to constantly consider different, innovative ways to solve

management problems.

Also, there was a very streamlined Program and Project Management structure with a

definite lack of management layering. This enabled decisions to be made quickly and their

implementation to happen quickly.

In addition, ACT Corporation, hired to do imagery data management and daily archiving

of science and engineering data, were responsive and innovative. ACT Corporation used

a PC based system versus a main_ame, handling up to 25,000 images per day, and

managing the daily archiving of data with only three people. They implemented their

approach in a very short time, and did it at one quarter to one tenth the usual cost for such

a mission.

A pro-active, hands-on approach used by the Clementine BMDO Program Manager was

effective in creating the team approach which characterized Clementine, and in managing

cost and schedule parameters. The teamwork importance was also recognized in

comments made by both men about the importance of 'cradle-to-grave' assignment of

engineering team members and the effectiveness of training and co-location of team

members.

It is important to closely monitor cost and schedule data.

B) Policy Related Comments: The project management limited the number of standard

reviews. This was found extremely effective from an efficiency standpoint. Keviews were

primarily to satisfy issue resolution held, and was an effective methodology for staying on

schedule.

Strategic Resources, Inc. (SRI)

C-11
NASA Fast Track Study

C) Cultural Changes: Clememine embodied many approaches useful for "Better, Faster,

Cheaper" projects. NASA should consider adopting these successful approaohes, but they

will involve significant cultural changes.

D) Technology/Tools Recommended: None mentioned outside of comments referenced

above regarding the role ACT Corporation played. This indicated that standard

approaches using computer technology may need to be revisited as capabilities within such

tools as PCs become more sophisticated.

E) Other Observations: See attached view-graphs from Dr. Garrett's Personal

Perspective and Recommendations for future "Faster, Better, Cheaper" projects.

Strategic Resources, Inc. (SRI)

C-12
Strategic Resources, Inc. (SRI)

C-13
Strate(?ic Resources, Inc. (SRI)

C-14
UJ

Z

m

I-

,_1

_1

LU

0 0 0

z u. (n

W it_

I%1 ILl

_ =J

LU o x .<

-- I%: ..I I%:

!-- < 0

0 _= z >- ..

I.U u) -_

o_ = o _-

(/) m z <

LU Om

__ _ _,o= o_ _o

>" =_._ _-z ._o <o o z- <o ___ == _-=

-IW

Om wm

>-

ILl

b_ Z , , , , , '

Strategic Resources, Inc. (SRI)

C-15
NASA Fast Track Study

IL Applied Physics Laboratory (APL) at John Hopkins University

1. Discovery Program, Near Earth Asteroid Rendezvous (NEAR): NEAR is the

first of a series of missions under the Discovery Program which are designed to explore space

from the moon out to Mars and the Asteroid belt. NEAR was awarded to APL in the late fall of

1993, with a total cost to launch of $150 million. NEAR achieved their success at under $112

million. It is designed for rendezvous to orbit with the large near-earth asteroid, Eros in January

1999.

A) Practice Oriented Ideas: It is important to receive and understand fixed project

objectives. 'Tile objectives at the start must be those you finish with." The project must

maintain focus. These are not only the Level I technical objectives but the cost and

schedule objectives as well. One key to success is having support, not oversight, from the

NASA Program Manager. The program office maintained a single point of contact

approach on all matters, keeping interruptions down to a minimum_

It is important to understand the cost and schedule driven nature of "Better, Faster,

Cheaper". A Design-to-Cost (DTC) and Design-to-Schedule (DTS) focus was evident

throughout all discussions which translated to delivering project products. A true DTC

approach requires a technical design that recognizes performance as the dependent

variable with the design being changed to meet cost objectives. Design margin is critical

to success. Should any requirement changes be contemplated, cost and schedule impact

questions must be answered. Because of this, it must be understood that the first three

months are the most important part of the project. In the first three months the project

management team for NEAR:

1. Set policy (DTC/DTS, team environment, communication),

2. Picked partners properly (a most important step), and

3. Set philosophy of simplicity in the project plan.

Picking partners properly was stressed. The corporate flexibility for a partner to work in

the planned environment is critical. Flexibility must be evident at least one level above the

partner project manager to ensure the team leader will be able to do what is needed. This

is critical because of the flatness of the project management team structure and the need to

create a badgeless commamity. During the project, team members develop an allegiance to

each other and to the project manager, based on trust.

The APL approach to the structure of the project team was to use a lead engineer concept

over an entire subsystem, working under the Project System Engineer. Scope was defined

with the lead engineer who was held to it. The subsystem team leader was also a worker

and had ownership of the problems and the problem solutions.

The team environment extended to ensuring that all those working on and affecting the

project were made part of the team as early as possible. The contracting officer was

Strategic Resources, Inc. (SRI)

C-16
NASA Fast Track Study

assigned to the team. Manufa_uing and purchasing were brought in early, atteuding

davy meetings, discussing problems, and developing on the spot solutions. The extra

insight and belief in the schedule gained by manufacturing and fabrication was deemed

invaluable. Other team members also gained appreciation for involvement and overtime

impact of schedule decisions.

Throughout the process, extra hours worked came as a result of commitment to other

team members. Prior experience had seen commitment to the boss versus the team,

Having short term goals, such as fixed delivery dates, hardware-on-the-floor dates, etc.

became team rallying objectives. People believed in the objectives and in achieving them.

Team communication was also stressed. Person-to-person communication was used

wherever poss_le. Low-level design reviews were used to build in quality and to

communicate.

In the cost and schedule constrained environment of NEAR, this depended on being able

to leverage people with a track record of experience. (It is mentioned as a concern that a

more standard process on some projects should be available to teach newer people, in

order to be able to provide those with experienced track records in the future.)

B) Policy Kelated Comments: The low level design reviews worked into scheduled

Project Reviews. The Project Manager worked with NASA to schedule mandatory

NASA reviews (NARS) in conjunction with the Project Reviews to minimize disruption.

The NAR panel was asked to attend the Project Reviews with a day added to address

remaining NAR objectives. The NAR panel's willingness to fit into a different format

provided for a less intrusive, less costly approach.

C) Cultural Changes: Although the DTC focus has been mentioned, all stressed that this

is truly different from past practice in these types of projects. Performance margin must

be designed in. Buying the 90 percent version gains you wiggle room in the development

process.

It must be accepted up front that performance optimization will be limited and perhaps not

possible. The project manager uses margin gains in one subsystem to help relieve another

team member. This was aided by 50 percent use of the shell equipment.

From the beginning, the approach used on NEAR stressed the need to be able to use

simplicity, standards and cleverness to solve problems in lieu of a more expensive high

technology solution. "What made this program go was keeping it simple." An example

given was that the analysis of space craft geometry at the asteroid with respect to the earth

and sun allowed use of less costly, less complex fixed solar panel arrays and

communications antennae.

Strategic Resources, Inc. (SRI)

C-17
NASA Fast Track Study

Another cost and time saving innovation was the use of standard aerospace 1553 data bus

architecture for the first time on such a craft. Designing to an existing standard that

people are used to saved time over designing a unique interface. This saved an estimated

month of test and integration time.

D) Technology/Tools Keoommended: While use of tools was not stressed, the use of

CAD/CAM tools is standard at APE

E) Other Observations: Appendix E contains the paper, Cost Estimation and Modefing

for Space Missions at APL/JHU

Strate_lic Resources, Inc. (SRI)

C-18
NASA Fast Track Study

HL Hughes Electrgni_ Snace pnd Communications Comnanv

1. Communications Satellite Mgnufaeture and Launch: Hughes has reengineered their design-to-launch cycle for oonmamications satellites to stay competitive in the

world market. Past practice took as long as five years from contract to launch date and Hughes is

now working toward a 12 month cycle.

A) Practice Oriented Ideas: There is a great value in consolidation and co-location in the

space craR development and delivery process. Hughes has moved from a development,

assembly and test process using several buildings and locations to a single facility large

enough to accommodate co-location of people and functions. While the program manager

provides customer interface and budget management, the design engineering team, using a

work-cell, or production line approach, moves with the space craft as it develops. More

and better dialogue with the government customer regarding cost-containment practices is

recommended.

The continuous improvement approach in the evaluation of Hughes' completion of

spacecraft to launch process has allowed them to advertise a ship and shoot time of 31

days. This has encompassed the entire launch integration and processing focus. They

have tightened their hunch team requirements to a team of only 15 people. This came by

examining all costs and functions with a view to reducing cycle time. Shipping,

decontamination, fueling, etc. have all been examined and reduced in time and cost where

possible. Launch operations achieved a reduction from 38 days to 17 days in fueling the

spacecraft. A 33 percent reduction had been achieved just by looking at the model of the

process cycle time. Savings have also been achieved by going to off-the- shelf resources

as well For instance, a commercially available stainless steel container used in the dairy

industry was found adequate for use in' the process at a significant savings.

B) Policy Related Comments: The government should examine the procurement

turnaround time which is currently cumbersome and costly for the commercial provider.

Allowance for more creativity in the type of contract offered with an eye to the

possibilities allowed in the FAR is also important in a "Better, Faster, Cheaper"

environment.

C) Cultural Changes: No specific areas were mentioned, except to note that as

government tightens its belt, industry is also tightening its belt. Corporations such as

Hughes are looking at partioipation more and more in terms of return on net investment

and feel they can contribute on that basis with their cost reduction efforts.

D) Technology/Tools Recommended: None

IVo

E) Other Observations: None

Lockheed Martin Corporation

Strategic Resources,

C-19
NASA Fast Track Study

1. lltmIUM Global Cellular Satellite Prom'am: Lockheed Martin is a

subcontractor to Motorola on the IRIDIUM Global Cellular Satellite Network The founder

company for IRIDIUM has designed the network to consist of multiple satellites in low earth

orbit. Each satellite will be able to be produced t_om scratch in five days and put into orbit eight

days later. It is designed to allow phones of only six ounces to communicate with any other

phone on the network anywhere on earth.

A) Practice Oriented Ideas: Earned Value management is a practice that promotes cost

control. This grew out of a need to be more competitive, and seems to drive down all

costs. The concept began by performing a comparison between military and commercial

practices, benehmarking commercial practices and comparing them to MILSTAIL The

resulting approach focuses on identifying indicators that will be standardized on a set of

those things Senior Management will look at; the things everyone must do.

Two most important concepts which are part of this approach are:

1. Emphasis on insight versus oversight; answering the question, "Is

the management system doing what it is supposed to do?"

2. Replacing rule-based Cost/Schedule Control Systems Criteria (C/SCSC)

with analysis and judgment.

The benchmarking effort examined what was working, and identified those best practices

that, for example, combined management of cost, schedule and technical performance.

Lockheed Martin's approach is to use Integrated Product Teams (lIrI ") and keep project

manager reviews to one and a half hours using charts that communicate information, but

are not expensive or time consuming to produce. The charts being used to brief the

program should be the variance charts/reports that are used by the project team. The

chart should show trends, indexes, and other necessary data and provide an opportunity

for the team leader to say 'Tiffs is what I need to fix this problem" MILSTAR has

accepted the commercial practices identified. The president of Lockheed Martin refers to

this as '_.he basic block-and-tackling of management."

B) Policy Related Comments: The way NASA funds accruals with the 533 system and

mitigates the benefits of using Earned Value. This is at least partly because NASA is not

funded for termination liability. Actuals differ for the two systems.

C) Cultural Changes: First NASA should recognize that the practices described above

differ l_om past practices. Second, these new practices will require education for these

practices to be standardized.

D) Technology/Tool Recommended: None

Strategic Resources, Inc. (SRI)

C-20
NASA Fast Track*Study

E) Other Obm=)'ations: None

Strategic Resources, Inc. (SRI)

C-21
NASA Fast Track Study

V. Spectrum-Astro Corporation

1. New Millennium Program_ Deep Space One Project: Spectrum Astro is the

Prime Contractor to the Jet Propulsion Laboratory for Spacecraft development on the Deep

Space One (DSI) mission (see earlier description of DS1 under JPL).

A) Practice Oriented Ideas: As a small company, Spectrum Astro examined practices and

organization structure of larger competitors. They saw that in the past, 20 percent of the

people on such programs did 80 percent of the work and added 80 percent of the value.

Spectrum wanted to try to structure its involvement so that they could get that 20 percent

number to be 100 percent. They used inherently smart people, that could take on end-toend responsibilities. In addition, the organization adopted a work as a skunk works

organization model, encouraged innovation and rninimiTed outside interference. This was

supported by the JPL Project Management team

Cost and schedule need to be treated as main drivers. The project manager must control

creeping requirements. "Anything impacting cost and schedule goes through a Spanish

Inquisition; it is very painful to add new things."

There is also a need to treat design in the same way. Spectrum Astro uses a flexible bus

architecture to accommodate the various technologies which are to be part of the mission.

Once the Integrated Product Development Teams baseline specific technology

demonstrations, the specific accommodation to the spacecraft can be rapidly evaluated and

resolved

Although, the interfaces provided by Spectrum Astro as integrator needed to be flexible,

the project manager moved the participants forward and allowed interfaces to be frozen at

a reasonable time. The key is early flexibility, with the realization of a set time for freezing

design.

Spectrum Astro realizes that innovative ways must be found to control cost and schedule.

Their spacecraft design is a proven, low risk architecture that allows accommodation of a

variety of technologies on a variety of missions, allowing for accommodation of DS 1 's

rather high risk, and high pay-offapproach to science.

Management of the team recognized the need to have sprit division of responsibilities

between Spectrum Astro and JPL, but end-item responsibilities must be given to one.

This decision is based on company or organizational strengths and discrete responsibilities

are assigned.

B) Policy Related Comments: None

C) Cultural Changes: In keeping with the search for innovative ways to save costs,

Spectrum Astro heartily endorses JPL's investment in video teleconferencing (VTC)

Strategic Resources, Inc. (SRI)

C-22
NASA Fast Track Study

capability as a way to maintain momentum and to avoid unnecessary disruption. While

Spectrum Astro is located in Phoenix and only one hour away by air _om JPL, such travel

can still be disruptive.

Using VTC, team members can have daily meetings, including small team or working

group meetings of one to two hours, while only planned all-day meetings require team

members to travel to a single site.

D) Technology/Tools Recommended: For data and information communication/dissemination, a system using a collaborative file server checked on each end is used.

The system resides on a Web site and includes Level I, II and HI Requirements, a daily

project calendar, phone list, PDR/CDR documents, etc. The system is working and allows

Spectrum Astro and JPL to do project management as part of daily business.

E) Other Observations: None

Strategic Res_

C-23
NASA Fast Track Study

Appendix D

SRI Non-Space Industry Interview Compilations

L

IL

m6

vo

ve

v_,

U.S. Air Force/Office of the Secretary of Defense

1. Joint Direct Attack Munition (JDAM)

(Terry 1L Little, Diane M. Wright)

U. S. Army/Natick Laboratories

1. Soldier-System Science and Technology Program

Motorola

1. Soldier-System Science and Technology Program

Boeing Vertol (of Bell-Boeing V-22 Team)

1. V-22 Vertical Takeoffand Landing Aircraft (Stuart Dodge)

Boeing

1. Boeing Laboratory Reinvention (Rick Becker)

Texas Instruments

1. Factory Improvement (Dave McDearmont)

D-2

D-5

D-7

D-9

D-11

D-12

Strategic Resources, Inc. (SRI)

D-1

NASA Fast Track Study

L U.S. Air Force/Office of Secretary of Defense

1. Joint Direct Attack Munitions (JDAM)/ Joint Aid-to-Surface Standoff

Missile (JASSM): Both are developmental programs jointly managed by the U.S. Air Force

and the U.S. Navy. Mr. Terry Little was government program manager for JDAM during a

critical period and is now government program manager for JASSM. JDAM is considered a

success within DOD for its use of new management approaches. These approaches are being

refined by Mr. Little for JASSM.

A) Practice Oriented Ideas: Mr. Little's view of project management is that the

government should have a very limited role, which includes: stating requirements;

choosing the contractor able to do the job; facilitating; expediting; and working interfaces.

It is the contractor's role to figure out the 'how -to' and then to do it. It is the primary

role of the government program office to help the contractor be successful. The

government can almost totally leave the how-to to the contractor. The only exception to

this is in identifying and helping to manage safety issues.

A significant factor in proposal selection criteria on JDAM, expanded to 50 percent of the

selection criteria in choosing the fight contractor for the job on JASSM, is use of past

performance. The contractor proposal is used only to present the design and technical

issues. Past performance was used as the exclusive (or sole) means to evaluate contractor

processes. This is done by examining past on-time development and technical

performance. As will be seen, there is a true atmosphere of trust in the contractor, but the

basis for trust is primarily past performance.

Contractor management capabilities are examined to determine:

1. The ability to manage cost and schedule,

2. Responsiveness to problem resolution actions, and

3. The extent the contractor has required past government

intervention.

In short, the government must look for proven ability to meet cost, schedule and

performance objectives. Also critical to success of Mr. Little's approach is that the

government and the contractor team operate as a team, Because of this there was initial

training, provided by the contractors through an external source, followed by

reinforcements at six to nine months into the effort. Mr. Little's office also provided a

facilitator to help spend time with the team to "define one or two goals which could be

measured to determine progress and value added."

Mr. Little indicated that this approach works for him because he is comfortable with

delegation and with trusting those on the team to make decisions. He indicated that with

Strategic Resources, Inc. (SRI)

D-2
NASA Fast Track Study

both those above him and those working for him, the atmosphere is one of creating

expectations and then trusting people to live with those expectations and to rise to them

B) Policy Related Comments: Because the JDAM program was being managed in an

atmosphere of acquisition reform, still on-going in JASSM, Mr. Little was given broad

policy waiver authority. Some of the waivers are being examined to see if they make

sense for broader application to other programs. The key factor, as seen in other

interviews, is the need to minimize activities and functions that add cost or time to a

program by looking at the value they add.

One of the unique (for DoD) actions taken was to not require formal design reviews. The

review process was put in the contractor's hands. Design reviews were run as peer

reviews. There was no government requirement of contractor satisfaction of action items.

This is in keeping with allowing the contractor to be responsible for the how-to.

There is a recognition within OSD of the need to streamline documentation. For instance,

there is a tailored Defense Acquisition Executive Summary (DAES) and the monthly

Acquisition Report, a single bulleted page.

Also, while there is still a requirement for a quarterly program review, it is much more

informal and cognizant of the IPT structure. There is more trust and less looking for

problems. There is also a recognition of a need to manage "By the Budget" as provided

by Congress through the services.

C) Cultural Changes: Mr. Little mentioned that to achieve real teaming the government

roles mentioned above needed to be reaL This was done by significantly reducing the size

of the government program office. The current JASSM office is one fourth the size of a

previous attempt at a conventional cruise missile program. Program office members are

grouped on one of the three team alignments, consisting of Helpers, Interfaeers and

Resourcers.

Each group has a charter which specifically excludes responsibility for oversight of the

contractor. They truly are there to help. The Helpers are integrated into contractor IPTs

and their role is to assist the contractor in fulfilling obligations. They take directions from

the contractor program manager and input from him/her is a major input on that Helper's

annual performance review. Interfacers help the contractor deal with problems outside the

contractor's control. The gesourcers work issues of money and facilities, etc.

Because this is such a change from prior relationships with contractors and a reformed

style of management, those not comfortable with this approach are moved to other more

traditionally managed offÉces. Mr. Little believes however, that this is the management

approach of the future and raises the issue of the need to educate people on this new

approach and to reexamine the way program/project managers are selected.

Strategic Resources, Inc. (SRI)

D-3
NASA Fast Track Study

It was also noted that the program made those with traditional oversight or inspector roles

part of the team The OSD oversight team developed more of a stakeholder mentality and

attitude. Mr. Little fostered the attitude that it is every members' job to make this

program successful with the result of the OSD team moving from oversight and inspection

to a helping hand approach.

The difference that this management style makes was seen clearly in comparison to a

traditionally-structured progran_ The same OSD team that was fully integrated under

JDAM, was not integrated on the team of another progran_ They filled the traditional

inspection/oversight role with that program and became adversaries rather than advocates.

Mr. Little has clearly demonstrated that trusting your team and valuing their opinions

yields good results.

D) Technology_ and Tools recommended: None

E) Other Observations: Further information about JDAM is in Appendix E.

Strate_lic Resources, Inc. (SRI)

D-4
NASA Fred Track Study

IL Natick U.S. Army Laboratories

1. Soldier System Science and Technoloev Development Proeram: A concept

program which treats the U.S. Army soldier as the system, the objective is to design the soldier's

clothing and equipment that acts in concert to both protect and enhance warfighting capabilities,

including communication. This effort is analogous to the technical challenge of many NASA

missions where several different science and technology applications must be separately developed

and integrated within a constrained schedule and a limited budget.

A) Practice Oriented Ideas: The most critical aspect for success is team structure and

buy-in to that structure. It is also the most difficult aspect to get up front, especially

making the user part of the teanx Many users in DoD are accustomed to 'q_elling the

developer what we want, then coming back at the end of the effort and telling the

developer how bad they did." Making a way for the user to be involved throughout the

effort is a challenge.

The philosophy of trust in those chosen to be on the team to perfoma a necessary

management function is evident. We should refuse to pay for something twice. There

was no government duplication of contractor positions. This allowed for a smaller than

usual government staff and more reliance than usual on the contractor to assist in such

things as cost management.

Due to the nature of the need to develop science and to manage cost and schedule closely,

there were six Contracting Officer Technical Representatives (COTR) versus the usual

one, each assigned to an IPT. This allowed each team to make decisions quickly that

made sense, and still be responsible for cost, schedule and performance.

Reviews were managed as internal project tools for the benefit of the teams. Outsider

involvement and input at these reviews was carefully limited. Major reviews were

opportunities for IPTs to share information. In this project management approach, one is

either part of the team or is an outsider.

Contractor selection is seen as very important also. Selection is based on choosing

someone with a proven history of managing multiple subteams, system integration and an

approach to Integrated Product Teams/ Integrated Product and Process Development

(IPT/IPPD). How well the contractor has worked with sub-contractor in a team

environment was examined (Was the best person selected to lead a team, or were teams

always led by the prime?).

Once a contractor is selected, it takes time to get IPTs working well together, perhaps as

much as six months. All of the teams are taught the tools to be used (Metric

Measurement, Design for Manufacturing, Six Sigma, Quality Function Deployment, etc.,)

as well as team building/team formation. A new set of management/personnel _ is

Strate i__Resources, Inc. (SR 0

D-5
NASA Fast Track Study

needed to be successful in this environment, including consensus building, team leading

and others. While the special skills were not explicitly provided in training for the IPT

leaders, they probably would be in similar future projects. While it takes longer to build

team consensus and to work in IFFs than in a leader-directed environment, the decision

achieved is better and restarts are significantly reduced. In addition, late discovery of

integration problems are also greatly reduced. This moderated the fact that 70 percent of

downstream cost is cemented in the design phase.

As mentioned, the teams are responsible for managing their cost, schedule and

performance. This was facilitated by requiting each IPT to develop at least eight metrics

to be tracked, four each for performance (weight, power, etc.) and four more which

addressed such items as cost, management reserve, estimated spend rate versus actual,

software reuse rate, etc. The management functions for the project also developed four

performance and four cost metrics for the overall project, as well as a project risk

mitigation plan. Risk mitigation was addressed at each review.

The project also analyzed how well the IPTs seemed to be doing in general by examining

qualitative metrics, including team turn over rate, and how issues and action items are

closed/handled, etc. This type of metric may be used to identify problem areas early

without micro-managing.

B) Policy Related Comments: None

C) Cultural Changes: The entire area of close teamwork with the contractor, as well as

inclusion of the user as part of the process, can be cultural change for many. Recognizing

this and addressing it early in the project's life can help. Underlying the success of

teamwork is the issue of trust. It takes time to develop and it is easy to break. The bearer

of bad news must be given consideration and the information treated fairly.

D) Teclmology/Tools Recommended: None

E) Other Observations: None

Strategic Resources, Inc. (SRI)

D-6
NASA Fast Track Study

HL Motorola

1. Soldier-System Science and Technology Development Program: The program

is described in the previous interview compilation. Motorola is the prime contractor for this

effort.

A) Practice Oriented Ideas: Motorola echoed the thoughts of the government Project

Manager, in that teamwork is a key to the success of this project. The objectives in this

are to be cross-functional/cross disciplinary wherever possible so that people talk with

each other. Teamwork removes walls. To this end, the project manager must foster

communications and either convert skeptics or remove them

Motorola backs this both internally and on the projects they manage. Internally Motorola

provides up to four weeks of training yearly to employees on teamwork, quality and

related material. On projects, time is invested up front, on training in project management

tools to be used and team building. This is done for each product IPT, including

contractor, subcontractor and government team members.

In general, the Motorola philosophy is to push down the decision making process to the

lowest level. It is both a privilege and a responsibility designed to empower the managers.

In other words, it is done deliberately to develop managerial capability. The objective,

beyond human development, is to have distributed versus centralized control and

management. On this project in particular, the situation of the project forced distributed

control. There were several technologies for which Motorola did not have all the organic

technical capabilities required.

It is valuable to include the user on the project team This allowed significant involvement

of the user at the practical level, ensuring that specifications really represented operational

requirements. For one example, a subcontractor identified a specification which was

proving to be a significant recurring cost driver. By working closely with the

user/customer to gain a better understanding of minimum requirements, the cost was

better contained.

One way that teams worked to manage the limits was to develop effective metrics. These

metrics are not things you measure because you can quantify them, but things you measure

that will make a difference; those which will tell the manager there is a problem on the

horizon. Since the design of this system was soldier-driven, not all of the performance

metrics were easily quantifiable. As mentioned by the government project manager, each

team was tasked with metrics that were drivers for the project that needed monitoring. As

the project progressed, the manager/team was able to propose changes to metrics chosen

due to program dynamics, or if the chosen metrics proved to be ineffective. Choosing

which project drivers to watch was an output of good Systems Engineering and Risk

Management practices. Each team also developed a risk chart or table to depict

Strategic Resources, Inc. (SRI)

D-7
NASA Fast Track Study

probability of risk and impact as High, Medium, or Low, and planned risk mitigation

strategies.

Management of reviews was very important. The IPTs were allowed to manage quite

freely, holding their own reviews that took the place of pre_reviews for larger, inclusive

reviews. The larger design and management reviews were developed to address '_dae

white spaces" between the IPT boundaries, including interface and integration issues.

Even the larger design reviews at the project level including the government project

manager, were active discussions about the white space issues, an interactive design

interchange versus a military briefing. As seen in several other Fast Track Projects, the

reviews are held for the project management team rather than for oversight purposes.

A Motorola initiative to shorten the developmental cycle by using incremental

development was also successfully used on this project. It is a more piecemeal approach

and can be more difficult to manage, but also saves time. This includes incremental design

reviews, held in a cross-functionaL user influenced environment, rather than

comprehensive reviews. This approach also provided the user pieces/parts to test

throughout the process which, in turn, provided the developers early useful feedback when

change was easier to manage.

B) Policy Related Comments: None

C) Cultural Changes: While there was none mentioned explicitly, it is important to note

that Motorola has established a climate which includes teamwork, a focus on quality

(continuous improvement) and acceptance of change. This is brought into the project

environment through training and the corporate structure.

D) Technology/Tools Recommended: None

E) Other Observations: None

Strategic Resources, Inc. (SRI)

D-8
NASA Fast Track Study

IV. Boeing Vertol

1. V-22 Vertical Take-off and Landing Aircraft Program: The V-22 is in the

DoD Engineering and ManufaeaLfng Development (EMD) Phase of development, which

corresponds to NASA Phases C/D of the Life Cycle Development process. The management

approaches applied on this program have resulted in significant cost and schedule control with a

proven track record of meeting or exceeding most design objectives. The objectives here were

not to constrain the schedule, but to contain and reduce costs on the final product.

A) Practice Oriented Ideas: Boeing has applied many of the lessons learned from their

commercial aircraft experience with the 777 aircraft. The concept of the 777 DesignBuild Teams has translated well to Integrated Product Teams in use on V-22. One of the

lessons learned t_om 777 experience was a need to provide a way to bring separate team

activities together. On the V-22 this is done with Analysis and Integration Teams (AIT).

Older, experienced 'gray-beards' are brought in for short periods to help work integration

issues. This concept is carried forward to the air-vehicle Integrated Product Team

Commtmication between contractor components and between contractor and government

was obviously important in a team environment. Close contact is important and can be

provided by video teleconference capability. There were periods where the team used

video teleconferencing up to six hours per day with the government. Tiffs is especially

important in the team environment of the V-22 effort where both contractor and

government are committed to sharing information with each other as soon as possible.

Design-to-Cost (DTC) plays a significant role in the ability to provide a final design which

comes in at or under the program cost target. As mentioned earlier, the goal of the V-22

team was to reduce final production and other lifecycle costs of the V-22. As such, their

approach is applicable to any team based engineering-design effort. In this approach each

IPT set their own targets for cost reduction from initial estimates and were also given

discrete performance improvement targets, such as weight reduction, part unit reduction,

etc. This was based on analysis of data provided by the IPT. As the project progressed,

in some cases goals could be re-allocated where needed.

B) Policy Related Comments: To create the team environment, the type of contract in

place should be examined. A Firm Fixed Price type of contract tends to develop an

adversarial, non-stakeholder atmosphere. The Cost Plus Award Fee contract provides

more incentive to a team.

Strate_lic Resources, Inc. (SRI)

D-9
NASA Fa_d Track Study

C) Cultural Changes: In addition to the proper contracting vehicle and the enabling

communication technology, it takes time and effort to create and sustain a team attitude

between government and contractor, and between fimctional disciplines. A change in

culture is worth the effort because the payoffis significant.

D) Technology/Tools Kecommended: A Boeing tool with value derived from their

experience with the 777 is an enhanced CAD/CAM capability. Drawing on an engineering

database, it allows designers and producers on the team to see the product in a three

dimensional moving presentation. Interfaces are displayed for multiple teams to examine

simultaneously. The tool is programmable to highlight tolerance problems. As with team

related cultural changes the cost of implementing this tool was more than paid back in

savings of both cost and schedule. There were significant reductions in error, change and

rework, as well as elimination of the need for several costly, standard machine modeling

approaches and mockups. This tool became the central design hub of all IFF activities.

E) Other Observations: None

Strategic Resources, Inc. (SRI)

D-IO
NASA Fast Track Study

V. Boeing Laboratories

1. General Downsizin_: While not directly tied to Fast Track projects, several

comments were considered worth including with regard to reengineering and cultural change.

In 1989-1990, Boeing examined its laboratory structure and recognized a larger, broader

structure was present than necessary to support its needs (456 separate laboratory organizations).

In 1991, all labs were centralized under one head, which is now Boeing Defense & Space Group

Laboratory Operations. The problem was how to move faster in process change in support of

Boeing's corporate mission. In the process of consolidation the lab structure was reduced to 128

laboratories over a four year period. At the same time, Boeing tried to maintain the level of

scientists and scientific capability (i.e., The goal was better management, not reduction in

technical personnel).

In evaluating the structure, everything was put up against the corporate strategic plan. Anything

not connected to the mission and goals identified in the plan was considered for consolidation.

Part of this structural examination was to examine the customer's definition of requirements and

criteria. This is part of an overall process of walking through consolidation corporation wide.

Boeing has developed this process to a high level of detail. The exact details were not shared

because they are considered competition sensitive.

There have been many lessons learned from the success of Boeing 777 effort. Process change

was a big focus for that venture. In this area of embracing process change, care must be taken to

evaluate impact and to keep the number of process changes manageable. It is easy for the impacts

of several on-going changes to be compounded and very difficult, if not impossible, to manage.

There were three or four major process changes associated with the 777 effort. There are now

three to four dozen candidate process changes identified that could cause loss of the ability to

control the overall change.

Strate_]ic Resources, Inc. (SRI)

D-11
NASA Fast Track Study

VL Texas lnsU'uments CI'1)

1. Single Process Initiative/Common Process Factory: With the move in the

military to adopt commercial practices versus MilStds/MilSpecs, TI has seen an opportunity to

provide significant savings to its customers and to itself.. This falls under the title of Single

Process Initiative. This describes an attempt to arrive at single acceptable commercial processes

for several customers replacing multiple approaches of the past. In doing this they have replaced

65 versions of 38 Mil Specs with just eight Commercial Specifications. TI has done a block

modification to implement this on 770 separate DoD contracts. They are now extending this

concept to those companies for which TI is a subcontractor, to either do block modifications or

some other implementation of this idea.

They are also working with the DoD as a designated Keinvention Lab and they include the

Defense Contract Management Agency (DCMA) as a team member. They are working with the

U.S. Navy on an effort termed Alpha Acquisition which is working toward a highly streamlined

cost estimation.

Strategic Resources, Inc. (SRI)

D-12

NASA Fast Track Study

Appendix E

Supplementa ! Information

L Mars on $300K A Day: The Mars Exploration Program

Donna L. Shirley E-2

IL New Millennium Program Deep Space One

David I4_ Lehman E-15

IlL Pluto Process Improvement Initiatives

Robert Staehle E-61

IVo Discovery Program, Near Earth Asteroid Rendezvous (NEAR)

Project - L.J. Crawford, T.B. Coughlin, W.L. Ebert E,.66

Ve JDAM - The Value of Acquisition Streamlining

Internet Home Pages E-76

NOTE: The articles and information appear in this section by the express permission of

the authors and may not be reproduced without their permission.

E-1

"MARS ON $300K A DAY'*: THE MARS EXPLORATION PROGRAM

Second IAA International Conference on Low-Cost Planetary IVfissions

Donna L. Shirley

Manager, Mars Exploration Program

Jet Propulsion Laboratory/California Institute of Tedmology

4800 Oak Grove Drive

Pasadena, CA 91109

Fax 818-393-6800

E-Mail: dshirloy_c_aiLjpLnasa.gov

ABSTRACT

The Mars Exploration Program is pioneering a series of'_etter, faster, cheaper" missions to Mars

over at least the next 10 years, and possibly well into the 21st century. The overall cost of the

program is somewhat over $100M per year, including hunch vehicles and operations, for two

launches every 26 months. (This works out to about $300K per day.) A programmatic strategy

has been developed involving selection of a science theme for the program, a long-term industrial

partnerd_ for spacecraft development, continuous infusion of new tedmology, and integration of

the individual projects into an overall program so that synergy between projects can be

maximized. A number of process improvements have been made in procurement, concurrent

engineering, rapid prototyping, and integrated payload selection, among others.

INTRODUCTION AND PROGRAM DESCRIPTION

The Mars Exploration Program Office was formed at the Jet Propulsion Laboratory in July 1994

to integrate the robotic exploration of Mars. The two major elements of the program are Mars

Pathfinder (one of the first two Discovery missions) and the Mars Surveyor Program (a program

to send missions to Mars every opportunity within severe cost constraints). The Mars Surveyor

Program was established in the wake of the loss of Mars Observer in 1993. The first of its

missions, Mars Global Surveyor, w_l carry six of the eight Mars Observer insmunents, and the

other two instruments are planned to be flown in 1998 and 2001. The contemt and

implementation strategy of the Mars Exploration Program are described in Reference 1.

The Mars Exploration Program will continue the investigation of the red planet by robotic

spacecr_ that began in 1965. The new U.S. program of exploration will begin with launches in

late 1996 of the Mars Global Surveyor and Mars Pathfinder missions and extemd through at least

2005. And studies are under way for activities leading up to a human mission to Mars as early as

2018. The focus of the first 10 years of the Mars Exploration Program is on building up

knowledge steadily and incrementally to a thorough characterization of the planet in terms of life,

climate, and resources.

Mars Pathfinder will be the second mission in the Discovery Program of planetary exploration. It

will launch in December 1996 on a McDonnell Douglas Delta 7925 rocket, and land on July 4,

E-2
1997. It will imagethe terrain in 14 differmt spectra,monitor the weather, and deploy a small

rover to explore the region around the lander and measure the composition of the surface.

Mars Global Surveyor, which will launch in December 1996 (also on a Delta 7925), will go into

orbit around Mars in September 1997 and, after aerobraking into a circular polar orbit, wiU scan

the surface of Mars for a full Martian year (about two Earth years) using six of the eight

instruments that were o "nginally flown on Mars Observer (which was lost in 1993---4he first

planetary spacecraI_ failure in 27 years).

After Mars Global Surveyor, the Mars Surveyor program will fly two missions to Mars every

opportunity (about every 26 months) and, with Pathfinder, is pioneering the "oetter, faster,

cheape_' approach to planetary missions. In late 1998 Mars Surveyor 1998 will launch m orbiter

and a lander on a Delta 7325 hunch vehicle (at a savings over a Delta H). The orbiter will carry

an infrared spectrometer to survey the atmosphere over a yearly cycle. The lander will place a

payload near the south pole to search for volatile elements such as water.

The final element of the lost Mars Observer payload (a gamma ray spectrometer) is being

considered to search for water in 2001 on the Mars Surveyor 01 orbiter, and another lander

payload is planned to have a rover investigate an ancient highland lake bed to study the climate

history of the planet. One of the 2001 missions may be conducted in partnership with the

Russians. For the 2003 opportunity the Mars Surveyor program is exploring a partnership

(InterMarsNet) with the European Space Agency to place three landers on the surface. And the

Mars Surveyor 2005 mission will be part of an attempt to return a sample from the Martian

surface.

Over the next 10 years the Mars Exploration Program will result in a detailed understanding of

Mars, which is of interest not only to Mars scientists but _ help in understanding more about

the Earth's environment, and can eventually provide the basis for human exploration of Mars. The

entire program will be conducted for about one-third the cost of the V'ddng missions, which

orbited and landed on Mars 20 years ago. Each mission costs about the same as a major motion

picture, and the total cost of 10 missions to Mars is about the same as that of a single major

Figure I graphically illustrates the program--including a U.S.-only mission set (on the bottom)--

which can be augmented by international partnerships (on the top). Each launch year is shown

across the top from 1996 through 2005.

E-3
MARS PATHFI IDER _ ..... " "

tANO ER/ROVER M AR8 MARS M AR8 :':_

(DISCOVERY) SURVEYOR SURVEYOR SURVEYOR I

i $1,1MDEV/$14MOP8 I "n ORBITER "01 ORBrrER "0s ORBITER (PMIRR) (GR8) &/OR OR

+$81M DELTAT925 & POLAR & LANDER LANDER(S)

LANDER" W/MOBLITY (NETWORK?)

v Y v

MARS GLOBAL

8URVEYORORBITER. i $1871M DEV

$154M OEV /l I +$72M (2 DELTA T2:S-)

m

I ,1.o__,.,o.,.v.,..o____ I

i +$61M DELTA7925

_l-- MARS SURVEYOR OPERATIONS PROJECT': $20M/YEAR

"POSSIBLY W/NEW MILLENN RIM MICRO LANDER _ oust,.1_s

Figure 1. Mars Exploration Program Strategy

The funding available for each mission is shown in the boxes in real-year dollars: the first number

is development cost, the second is hunch vehicle cost. (The assumption here is that Delta 7325s

cost only $36M each, but this may not be the case when the 'WIed-Lite" contract is finally

negotiated.) For Pathfinder, a $14M operations cost is shown for the 7-month cruise and a year

of surface operations. For the Mars Surveyor Program, a project has been formed to operate all

the missions, and there is $20M per year available for this (not inoluding traoking and data

acquisition costs of the Deep Space Network).

STRATEGIC PI.ANNINCr--THINKING LIKE A BUSINESS

Because the new Mars exploration missionswere to be, conducted at only a fraction of the cost of

previous missions, cost became the major driver--with performance and risk being the dependent

variables--a major paradigm shift for NASA missions. Consequently, it was decided to approach

planning for the program as though it were a business. This approach was based on a

methodology for business planning desor_ed in Reference 2. A diagram of the approach is shown

in Figure 2, mapped as a series of questions.

Since Mars Global Surveyor and Pathfinder (including the rover technology experiment being

funded by lqASA's technology program) were in existence when the Mars Exploration Program

was formed, they were our "current business." Businesses of the future were the remainder of the

U.S. Mars Surveyor Program, plus international missions that could augment Surveyor. We

E-4
What Business AreWe In'?

• _(xluotS?

•Servloes?

• Customers?

What Business Should

We Be In?

Mars

Surveyor

Program

Mars

InterMarsNet

MGS96

Pathfinder

Rover

Align With Strategy:

• Systems

• Processes

• People

• Organization

Figure 2. Mars Exploration Program Strategic "Business" Planning Process

investigated several '_ousinesses" which we must be in to conduct a program of Mars exploration

in today's environment, and arrived at five: knowledge gemeration, exploration, education,

inspiration, and technology development/transfer. For each business we identified the customers

and the specific products and services we would be providing these customers. We identified

how we were currently meeting the needs of these customers; that is, what our current business

strategy is. Then we identified what our business strategy shoed be to meet the customer's

needs. We selected which businesses should be our primary focus, and which should be primarily

carried out by '_)artner" organizations at JPL. We identified the systems, technology, and

processes needed to carry out our business strategy. Then we staffed the program office, and,

finally, we evolved an organization to conduct the selected businesses.

Table 1 shows the selected '_ousinesses" with their descriptive information (summarized).

This business analysis showed us that the "knowledge" and "exploration" businesses are very

similar in terms of the missions needed to produce the knowledge products. These knowledge

products (an understanding of Mars, plus knowledge about the technology and processes of

conducting missions to Mars) could be produced synergistically by currmt and fimue Mars

Exploration Program missions, provided the missions are conducted properly. If the information

received _om the missions can be synthesized and translated into products suitable for the

E-5
classroom, the needs of the "education" business can be met. And if the information is presented

in a clear and exciting way, the Mars Exploration Program can provide "inspiration" to the public.

Since the only way to conduct a series of Mars missions on the very low budgets allocated is to

use new techniques, processes, and technologies, the "technology" business becomes a byproduct of the knowledge and exploration businesses. The step of transferring technology

developed or transformed by the Mars Exploration Program is needed to infuse that technology

into other applications (preferably commercial). We decided to focus on the knowledge and

exploration businesses as primary, with education, inspiration, and technology as secondary

businesses. We then began developing a strategic plan to conduct our '°ousinesses." The

following is from a draft of this plan.

Table 1. Mars Exploration Program Business Analysis (Summary)

Bnsiness

Knowledge

Exploration

Education

Inspkation

Technology

Development

& Transfer

Product Technology in

Product

Information

about Mars

Information

about Mars

Educational

products

Inspiring

information

Advanced

technology

Information

management

Information

management

Information

presentation

Information

presentation

Technology

itself

Technology

to Produce

Product

]V[ars

Missions

Mar8

_tssions

Translation

of knowledge

Computer

graphics, etc

Techniques for

technology

production

Features of

Product

Understandable

information

Synthesized

information

In currioula

Customers

Scientists,

everyone

NASA

Educators,

students

Exciting

Costeffective

The Public

Missions,

industry

E-6
MARS EXPLORATION PKOGRAM STRATEGIC PLAN

Mars Extdoration ]ProEram (MEP) Vision

To Know Mars

MEP Mission

Explore Mars to understand it in terms of life, climate, and resources.

Provide a first step in the search for life beyond Earth.

Better understand the processes that shape the solar system, including the Earth.

Pave the way for human exploration of Mars.

MEP Goals

1. Pave the way for human exploration of Mars.

2. Find evidence of past or present life (or find evidence that life never existed on Mars) to

understand the potential for life elsewhere in the universe.

3. Better understand the climato--4he weather, processes, and history--in order to understand

the relationship to Earth's climate change processes.

4. Identify and learn to utilize the resources available on Mars--understand the solid planet, how

it evolved, and what resources it provides for future exploration.

5. Identify and characterize safe, interesting, and productive sites for human landings.

6. Find and learn to use water and other resources.

7. Characterize Mars' atmosphere, surface environment, and other factors that bear on the design

of human missions.

8. Pioneer technologies for low-cost, high-return planetary exploration_

MEP Strategies

Strategies are specific actions and approaches we will take to guide the Mars Program toward our

customers' goals.

• MEP will operate in the manner of a business, determining the needs of our customers and

developing "contracts" with them for meeting those needs.

• Our primary products are knowledge and exploration_ Our secondm7 products--tedmology,

education, and inspiration--will be produced in partnership with other elements of JPL.

• We will employ best business practices and will create technical and managerial innovations to

manage our projects within the terms of the "contracts."

• We will implement our projects with common processes and support operations to minimiz_

costs.

• We will partner with NASA, industry, the science comnnmity, the educational community,

and other elements of JPL to accomplish our goals.

E-7
• International partioipafion will be sought for each MEP mission within the program

oonmmt_

Top-level Program strategies are supported by specific thematic emphases in knowledge,

exploration, technology, education, and inspiration.

Knowled2e Strate_

• Build a progressively deeper knowledge of Mars by implementing a series of missions that

build on the knowledge gained by previous missions.

• Partner with other countries to acquire and analyze information.

• Maintain a global compendium ofthe state ofknowledge of Mars.

• Provide a steady stream of new Mars informational and educational products that summarize

the state of knowledge and the remaining unanswered questions about Mars.

Extdoration Strateev

• Implement a series of missions focused on discovery.

• Use new instnunents and instrument deployment techniques.

• l.amd at different sites.

• Investigate more of the Martian surface and more aspects of the atmosphere over longer

periods of time with more capable sensors as the MEP progresses.

• Ketum samples to the Earth.

• Tailor investigations to support future human exploration of Mars.

Technolosy Strate2v

• Utilize new technology at every opportunity.

• Promote cost-effectiveness by integrating MEP technology needs with the needs of other JPL

flight projects.

• Partner with the JPL Technology and Applications Program (TAP) OtEce to develop

technology products tailored to MEP needs.

• Partner with industry to develop and demonstrate technology and infuse it into our projects.

• Support TAP in the transfer of our technology outside JPL.

Edugation S_at¢_

• Transform our knowledge of Mars into educational products.

• Partner with universities to disseminate these products and to train educators to understand

and use them

• Partner with the JPL Public Education Office to insert these products into curricula.

Inspiration Str_teev

• Share the exoitement and knowledge generated by discoveries about Mars with the world.

• Partner with JPL Public A/Fairs OtEce and with nmsemns, NASA Centers, and other

organizations to disseminate our information products widely and in innovative ways.

• Partner with industry to develop oommeroial products based on our progran_

E-8
OKGANIZING TO CONDUCT OUR BUSINESSES

Figure 3 shows the Mars Exploration Program Otiice organization that has evolved over the last

20 months. Before the JPL reorganization that created the office, preprojects were in one

organization, ongoing projects in another. Now the preprojects and flight projects have been

brought together so that there can be maximum benefit to the new projects by close association

with the old projects. There is considerable personnel sharing. At the program level, the Business

Operations and Outreach offices were created to allow resource sharing between the projects.

The program office staff is very small, and this has enabled our overhead to be reduced from

about 15% of the project business base to under 6%, of which less than 1% is program office

overheacL (The rest supports the JPL teohnical infrastructure.)

The Mars Pathfin¢ler spacecraft is being developed by JPL in a subsystem mode. All Mars

Surveyor spacecraft will be system contracted. Mars Global Surveyor selected Martin Marietta

Astronautics in Denver as its spacecraft system contractor, and Martin Marietta also won the

contract to build both Mars Surveyor 98 spacecraft. JPL and (the now) Lookheed Martin

Astronautics (LMA). are thus in a long-term partnership. We are working to "mirror" our

organizations to facilitate teaming and communication. Spacecraft design and operations are to

be conducted at LMA, with mission design and operations being at JPL.

Payloads for the spacecraft are procured through the announcement of opportunity process from

NASA Headquarters, with support from the projects in defining the allowable envelopes for

accommodation of the payloads.

I

I

I

TECHNOLOGY & APPLICATIONS [

DIRECTORATE I

I

I

I

I

I

I

I

MARS EXPLORATION

TEC HN OLOG Y

DEVELOPMENT

PATHFINDER ]

(.WL m-HOUSZ s/c)

I '

I

M ARS PATH F IND ER

MICROROV ER

MAR S GLOBAL

SURV EY OR

(LOC KtlEEDMARTIN)

JPL DIRECTOR

I

MARS EXPLORATION

PROGRAM OFFICE ]

BUSINESS

OPERATIONS

PROGRAM SCIENTIST

OUTREACH

IN TF, R NAT IO NAL

I

MARS SURVEYOR

98

• ORBITER

• LAN DER

(LOC KHKEDMARTIN)

!

MARS SURVEYOR

PREP ROJECT S

01

03

05

(LO¢ gHgZDMA RTIN)

FLIGHT MARS SURVEYOR

EXPERIMENT OPERATIONS PROJECT

Figure 3. Mars Exploration Program Organization Chart

E-9
As personnelroll off PathfinderandMars Global Surveyor,somearegoing to Mars Surveyor98

and Mars Preprojects, while others are transferring their knowledge of'°oetter, faster, cheaper" to

other programs. For example, the project manager of the first New lVfillemnium mission was

formerly the task manager for the Pathfinder Attitude and Information Management subsysten_

Because of the relatively short development times for Mars projects, this training and knowledge

diffusion can be effidently done.

PARTNERSHIP WITH INDUSTRY

The Mars Surveyor Program is built around an industry partnership for the spacecraft (both

orbiters and landers). Payloads are procured through an announcement of opportunity process,

except for the reflight of the Mars Observer instruments on Mars Global Surveyor and the 1998

and 2001 orbiters. So not only has the overall cost of Mars exploration declined drastically, but

there has been a major shift from in-house JPL spacecraft workforce to contractor workforce. In

real-year dollars, Mars Observer (not counting launch vehicle and operations) cost about $460M,

of which 54% was spent in industry (mostly through a system contract). Mars Global Surveyor,

carrying six of the eight Mars Observer instruments, costs $154M, with 69% going to industry.

Mars Surveyor 1998 has $187M for two missions, and 83% of the money is spent by industry.

Figure 4 shows the continuing decline in overall funds for the follow-on missions, and the

continued trend to spend the lion's share of the money with industry.

The Pathfinder spacecraR is being built in subsystem mode and assembled at JPL. LMA is the

contractor for both the MGS and Mars Surveyor 1998 spacecraft. These were two separate

procurements, with the second being for both the orbiter and lander. Under the terms of the Mars

Surveyor 98 contract, LMA can be selected as the contractor for follow-on missions, provided

their performance is adequate for Mars Surveyor 98. JPL and LMA are developing a teaming

relationship. The JPI.A.bIA partnership entails a division of labor rather than oversight of the

contractor by JPL. For instance, on Mars Surveyor 98, JPL personnel are located in Denver as

part of the Lockheed Martin spaoecraR team, where JPL skills can augment LMA skills. This

enables a very small JPL project office. The LMA spaceorafL team will operate the Mars

Surveyor 98 spaceoraR from Denver, while JPL will operate the mission.

E-IO
U_ of commcneiAI products. A commercial computer, the IBM (now Loral) RS6000 has bern

flight qualified for Pathfinder. This computer is now the basis for not only future Mars programs,

but for Discovery missions, New Millennium missions, and other projects being planned well into

the next century. A commercial operating system is being used by Pathfinder, and the Pathfinder

flight soRware developed at JPL is being transferred to Lockheed Martin for use by Mars

Surveyor 98. Commercial products used in the Pathfinder rover (Sojourner) include motors,

radio modems, and power converters. Use of commercial electronics has required new processes

for determining how to flight qualify then_

Standard b_siness practices and technio_ues. The formation of the Business Operations Office has

allowed us to standardize financial and reporting formats and scheduling tools and techniques, and

to integrate an electronic library for the progran_

Partnering. As described above, the program is providing JPL '_vorkers" to LMA. The

possibility for long-term partnering with LMA also allows an evolutionary approach to---and is

key to---keeping the costs down for future missions. This reduces procurement cogs, enables the

project to be done on 26-month centers, and increases efficiency of long-term planning.

New technology infusiom Mars Global Surveyor will be the first planetary spacecraft to utilize a

composite structure. Pathfinder is using the RS6000 computer and has developed a new

approach to entry, descent, and landing. Sojourner, the rover, is a technology demonstration to

prove that small rovers can operate on the surface of Mars. Mars Surveyor 1998 is improving the

RS6000 -based computer system and is introducing a robotic arm as part of the lander payload.

For 2001 and beyond, technology infusion is absolutely required to reduce the mass of the

spacecraft (because the launch energy requirements are higher in 2001 than in 1998) and to allow

more payload percentage. LMA and JPL have developed a joint plan for _ technology

developed with LMA IRAD funds, for infusing New Millennium technologies such as advanced

electronic packaging, and for developing technologies such as the Small Deep Space Transponder

(SDST) currently being built by Motorola. The SDST is being funded by a consortium of future

flight projects, including Mars, New lVfillenninm, Discovery, and Pluto Express.

Partnering with universities for outreach. The Outreach Program has a series of small contracts

with universities to develop and apply educational products in their states. They utilize graduate

and undergraduate students for outreach to schools and comnmnities, and they train teachers in

understanding Mars information, how to use the Internet, and other skills. All this is done at a

fi'action of the cost of other methods of providing these services.

SUMMAKYANDCONCLUSIONS

The Mars Exploration Program is pioneering the use of'_oetter, faster, cheaper" techniques within

a program structure to achieve a steady advance in knowledge within an extremely limited budget.

We have made great strides in streamlining, partnering, improving our processes, empowering our

colleagues. However, much remains to be done. In particular, things still needed to protect the

vlab'dity of the program include:

E-11
I nl I!

i_IO:4,,POg_UUION I_I_'OGRAM

_ I"H_H

150-

tO0

5O

O,

FY96 FY97 FY96 FY_ II

• MARS SURVEYOR OPERATIONS - LMA

D MARS 2001 -LMA

E MAR8 8UIWEYOR 96 - t.t_

U MGSg6- LMA

• MARS 8UR_-YOR OPERATK_IS JPLJCTF

n MARS 2001 - JPI./CTR

II M_ _ - JPtJC_'R

U PATI-_NDER 96 - JPI./CTR

Figure 4. Mars Exploration Program Workforce

PARTNERING WITH SCIENTISTS

Science instruments are procured through the announcement of opportunity process, but are then

managed by the projects. In some cases JPL is also partnered with the scientists to develop

payload elements. For example, the Mars Surveyor 98 lander payload includes an integrated

instrument/robot arm suite where the arm is being developed at JPL and the payload is being

integrated by JPL, while the instruments are developed outside JPL. The principal investigator

for the integrated payload is at a university. In these cases the announcement of opportunity

proposals that involve JPL are kept carefu_ segregated within JPL _om the Mars Exploration

Program Office or any of its projects.

SOME COST CONTAINING PRACTICES

The Mars Exploration Program is pioneering several practices required to contain costs while still

conducting rewarding missions. These include the following (with a few examples):

Rapid, goal-based procurement. We have reduced the lead-time for spa_ proctweme_t by a

factor of four, from ten to twelve months for previous flight system proeuremeats to ten weeks

for Mars Surveyor 98 and twelve weeks for MGS. New procmmneat strategies have been

introduced for one-day turnaround for small commercial items, and for drastically reducing the

procurement time for electronic parts, etc.

Concurrent engineering. Pathfinder pioneered concurrent engineering of the ground and flight

systems. The new Mars Surveyor Operations Project is supporting all aspects of the project

development cycle so that this concurrent engineering can continue throughout the program

E-12
• Adhermce to a steady, inoremeata], evolutionary set of miss/ons w/thin a stable program

structure, inoluding a long-term indum3, partnership.

• Access to low-cost launch services.

• A stable base of support for evolutionary tedmology developmmt and infusion_

• Establishment of a minimal set of requiremeets for the program, i.e., "achieve the m_x-inmm

knowledge within the cost constraints."

• Clear and stable programmatic priorities betweee knowledge aoquisition, foreign policy,

commercialization, sdeace, education, small Imsiness objectives, etc.

• Continuous improvement in JPL and contractor support services for the projects ; e.g.,

financial services, tedmical infi_stmcture, streamlined managemmt processes.

• A process for information synthesis and translation into layperson's language and educational -_

products_

W_nh the two initial launches of the Mars Exploration Program this fall and winter, a new and

exciting era or" planetary exploration will emme. We plan to land on Mars on

July4, 1997 and orbit in September 1997, and thee we will visit Mars every 26-months or so for

at least the next decade. There are many risks because of the minimal fimds available, and

because of the need to do things that have never been done before, on 26-month centers. These

risks are being mitigated by creative new processes in management, engineering, and soieece. The

jury is out on the eventual success of the program, but we befieve that we have started welL

ACKNOWLEDGMENT

The research descried in this publication was carried out by the Jet Propulsion Laboratory,

California Institute of Technology, under a contract with the National Aeronautics and Space

Administmtion_

Refereece herein to any specific commerdal product, process, or service by trade name,

trademark, manufacturer, or otherwise, does not constitute or imply its endorsement by the

United States Government or the Jet Propulsion Laboratory, California Institute of Technology.

REFERENCES

1. Donna L. Shirley and Dr. Daniel J. MoCleese, AIAA-96-0333, "Mars Exploration Program

Strategy: 1995-2020," 34th Aerospace Sdeeces Meeting and Exhibit, Keeo, Nevada, January

15-18, 1996.

2. Dan Thomas, Busine_ Sense: Ererclstng Management's Five _reedoms, The Free Press,

Division of Macmillan, Inc., New York, 1993.

3. Glenn Cunningham, "Management of Mars Global Surveyor IVfission," International Academy

of Astronautics IAA-L-0801, Low-Cost Planetary Missions Conference, Laurel, Maryland,

April 19, 1996.

E-13
4. John B. McNamee, "1998 Mars Surveyor Orbiter and Lander Project," International Academy

of Astronautics IAA-L-0403, Low-Cost Planetary kftssions Conference, Laurel, Maryland,

April 17, 1996.

5. Sylvia L. Miller, "Mars Surveyor Missions: 2001 & Beyond," International Academy of

Astronautics IAA-L-0404, Low-Cost Planetary l_ssions Conference, Laurel, Maryland, April

17, 1996.

6. Anthony J. Spear, ''Mars Pathfinder Project Progress Report" International Academy of

Astronautics IAA-L-0401, Low-Cost Planetary l_ssions Conference, Laurel MmTland, April

17, 1996.

E-14
New MIIlenn/um Program (NMP)

Deep Space One (DSI)

Project Plan

Document No. JPL D-13510, Rev. 1

David EL Lehman

August 19, 1996

Jet Propulsion Laboratory

Califonfia Institute of Technology

Pasadena, California 91109-8099

E-15
!_1 Project P._

Contents

Section P.ag_

1. Introduction ..................................................................................................... ......... :....... 1-1

1.1

1.2

1.3

1.4

Purpose; ............................................................... .................................................. I- I

Scope .................................................................................................................... 1-1

Governing Documents ......................................................................................... 1-1

Organization of the Plan ................................................................... :.................. 1-2

2. Governing Documents ........... .......................................................................................... 2-1

3. Level-1 Requirements, Goals and Mission Success Criteria ........................................... 3-1

4. Technical Plan .................................................................................................................. 4-1

4.1

4.2

Introduction .......................................................................................................... 4-1

4.1.1 Flight Team .............................................................................................. 4-1

4.1.2 Industry Partner ........................................................................................ 4-1

4.1.3 Integrated Product I_velopment Teams .................................................. 4-1

System Description ............................................... ,.............................................. 4-1

4.2.1 Advanced Technology Payload ............................................................... 4-1

4.2.2 Spacecraft Overview ............................................................................ ....4-2

4.2.3 Launch Servic,,es ....................................................................................... 4-3

4.2.4 Mission Operations and Ground Data System (Ground Segment) .......... 4-4

5. Management Plan ......... .. .................................................................................................. 5-1

5.3

5.4

Management Philosophy ....................... ............................................................... 5-1

5.1.1 Low-Cost, Quick, Reaction Approach ...................................................... 5-1

Program Relationships and Responsibilities ........................................................ 5-3

5.2.1 Program Office ......................................................................................... 5-3

5.2.2 Integrated Product Development Teams .................................................. 5-3

5.2.3 Flight Team .............................................................................................. 5-3

Organization and Work Breakdown Structure ..................................................... 5-4

5.3.1 DS1 Team Composition ........................................................................... 5-4

5.3.2 Work Breakdown Structure .................................................................... 5-6

Resomve Estimation, Allotmtion, and Reserve Strategies ................ ........ ........... 5-9

5.4.1 Resotm3e Estimation and Allocation ........................................................ 5-9

E-16
Secaon

.

D61 Project Plan

Contents (Cont'd)

gtm

5.5

5.4.2 Reserve Strategy/Descope Plan ..... .......................................................... 5-9

5.4.3 Resource Transfers and Fiscal Year Carryover ........................................ 5-9

Project Control ................................................................. ,, .................................. 5-9

5.5.1 Work Package Agreements ...................................................................... 5-9

5.5.2 Project Reporting .................... .............................................................. 5-10

5.5.3 Schedule ................................................................................................. 5-11

5.5.4 Review Plan ........................................................................................... 5-14

Resource Plan ................................................................................................................... 6-1

6.1

6.2

Funding Plan ........................................................................................................ 6-1

Facilities ............................................................................................................... 6-1

Table

2-1

5-1

5-2

5-3

5-4

5-5

5-6

5-7

6-1

Tables

DS 1 Document Definitions and Responsibilities ............................................................ 2-3

Quick Reaction, Low Cost Project Implementation Mode .............................................. 5-2

DS 1 Working Group Charters .......................................................................................... 5-8

DS1

DS1

DSI

DSI

DSI

DSI

Work Breakdown Structure ..................................................................................... 5-6

System or Subsystem Review Objectives ......................... _.................................... 5-15

System or Subsystem Review Process ................................................................... 5-16

System or Subsystem Interim Design Concurrence (IDC) Review ....................... 5-17

System or Subsystem Detailed Design Concurrence (DDC) Review .................... 5-18

Cost Summary .......................................................................................................... 6-2

Fdgmt

1-1

2-1

5-1

5-2

5-3

Figures

DS 1 Memorandum of Agreement ........................................ . ........................................... 1-3

DSI Documentation Tree .................................... ............................................................. 2-2

DSI Organization ......... . ................................................................................................... 5-5

DS1wort oroups....................................................................................................... 5-7

DS 1 Schedule. ................................................................................................................ 5-12

E-17
NMP DSl Project Plan

Section I

Introduction •

1.0 Introduction

The Deep Space One (DS 1)mission is the first in a series of New Millennium Program (NMP)

missions. The NMP will enable 2 lst-century space and Earth-science missions through the

identification, development, and flight validation of key technologies. Breakthrough

technologies are selected from the existing technology "pipeline" - consisting of ongoing NASA

technology programs, other government agencies, industry, nonprofit organizations, and

academia - and are developed in partnership with these organizations. These technologies will be

flight validated so that future science missions can take advantage of them without assuming the

risks inherent in their first use. NMP technology-validation flights are to be launched during

fiscal years 1998 and beyond. While these flights are essentially for technology validation they

will also provide meaningful science.

In addition to developing and validating key enabling technologies, NMP will pioneer new ways

of partnering with industry, nonprofit organizations, and academic institutions. Further, the

development and implementation of NMP's innovative management practices will advance the

competitive edge for NASA and other high-technology organizations. An overview of the DS I

team approach to implementing the project has been documented in a memorandum of

agreement which is provided as Figure 1-1.

1.1 Purpose

This document defines requirements accepted by the DS 1 mission (See Section 3.0) from NMP.

This plan also defmes the management and technical approach to be used by the NMP DS 1

project team. It includes an overview of the plans to manage the development of the flight,

ground, and mission operations systems.

1.2 Scope

The requirements in this document pertain to the DS 1 project objectives, technical approach,

budgetary and schedule constraints, and certain institutional and management relationships

between DS 1 and NMP. It represents the understanding between NMP and the DS 1 Project and

spans the entire project life cycle.

1.3 Governing Doeumeats

This document is governed by NASA guidelines and JPL institutional policies and practices, and

shall be implemented "maccordance with them, recognizing that NMP flights are technology

validation flights as opposed to science driven missions.

E-18
NMP D81 Project Plml

1.4. Organization of the Plan

This plan is composed of 5 sections in addition to the Introduction. Section 2 discusses

documents governing, and applicable to the conduct of the project. Section 3 details the Level 1

guidelines for DS 1 from the program office. The technical and mm_ement approaches are

described in Sections 4 and 5, respectively. Section 6 describes the resources necessary to

implement DS I.

E-19
NMPDSZ _t_m

Figure 1-1 DS 1 Memorandum of Agreement

JETPP.OPUI_ONLABORATORY INI1BROI_OZMlZMO_UM

DSl-DHL-96-012

May 24. 1996

TO: CharlesElachi

FROM: E. Kane Casani and David ELLehman

SUBJECT:. Deep Space One Memorandum of Agreement

We have agreed that DSI will have the following _cs:

_.aunch date:

]VJissiontvvc:

l..aunch vehicle:

onor before 31 July 1998

Cometand A_uoid Piyby

Delta 7326 (or TaurusXL/AUS-$1 with noCoazt flyby)

Technoloqy

IPS

SCARLEt

SDST

MICAS

RA

AUTONAV

3DS

PEPE

PASM

L.vEX

BMOX

KAX

NSTAR Ion Propulsion System

Advanced Solar Array

Small Deep-Space Transponder

Miniature Integrated Camera And Spectrometer

Autonomy - Remote Agent Architecture

Autonomy - Onboard Optical Navigation

3D-Stack Processor

Plasma Experiment forPlanetary Environments

Power Actuation and Switching Module

Low Power Eleetronim

Autonomy- Beacon Monitor Operations _-

Ka Band Solid State Amplifier

The funding required to seppoa DSI h as shown in Table 1.

T_ZcZDSZZ_,xU_e,cquC, uncm

DS1 Life Cycle Costs

OSl

Reselve8

D81 Lemeh sewloee

Total I)Sl Fm'dn 9 Needs

FY96

S3,SS0

0

ss,sso

(3

:m,s_

FY97

SK

SS,SS0

6,000

S9,650

0

SS.SS0

FY'98

SK.

17,020

4.000

el,O_0

• s2,oo_

6S.0_

I:Y99

$K

s_oo

0

s_oo

©

:3.90O

FYO0

$K

2,1(X

0

2,100

O

2,100

Total

SK

90,100

10.000

100,10U

• _.,oou

lS2,1oo

• Sl)ST cmuxeem fead_ md m_tve im_ded m dmiled in the SD,_ Cmmceum Pinn-

• IPS provided at m cha_e to DSI _. dettUed hi _e NSTARIi_! MOU.

• SCARLErpmvidedbyBMDOatttochaqptoDSl _delalledinzheBMDO/DSI MOU.

• Multi-miu/on elccu/cal OSB andion ddve 0ow thrust)missiondedlPn_mu,vigadon, andtrajectorycontrol

mftware provided by/PL institutional funding.

E-20
6_=VdW. ,m._m. _ m,_jr_41. K-ltmJJ8

Figure 1-1 DSI Memorandum of Agreement (Cont'd)

- *._t

• MS11 imrdware provided by BMDO m m chtrge to JRL(includes Ixtuery & m_ ny _).

• PandingforlhegaBmtdSSPAw/UbepmvldedbyC. ElachlmttlinjgS300KspUtasfolIows: $330Kin

FY'96, $120K In PY'97 ma/S20K in FY'98.

• Mlssioaopmutom funding, in 1:Y'99 tnd !_00 oniy includes funds tocollax dmfa'Udmolo_

validation; it does not _ funding for technology valldetio 9 and mud351s. The rmpomibllity for this

work is with the WDT co..k_ls and sepam_ funding for the anal3nds will be provided to the co-leads from

the program offl_. Fimdly, mission operations funds in FY'99 and FY'00 are very WeHndnm7 estimates

(allocations that do not even include project management) 8nd are subject to change as the design evolves.

• i_ launoh services, it is assumed that any co_ increase to the abovo allocation will be co_ by _

off'tee and/or NASA.

• IPDT co-leads will deliver their specified technologies in accordance with the agreed upon funding from the

project. Needs for additional funds and/or attendant descopes will be the responstbtl/ty of _e IPDT coleads. Descopes that impact the system design must have the concurrence of the prbject. This approach to

covering the funding needs is not common and results in high risk to the project - it is being done because

of low funding to rite project. _fore, if funding is not available from this approach, and desco_ options

prove to be fruitless and the _es of DS! are insufficient, JPL management will work to provide

alternate sources of funding.

• Funds to cover NEPA compliance will be paid for by the program office.

Management:

• Program office is responsible for NASA headquarters interface funedons. Program office will work

with the DSI team and NASA headquarters to close on the definition of the DSl launch services

approach. This issue needs to be resolved now.

• Program office is responsible for all TAP interfa_ functions.

• DSI project manager will have full authority to manage the budget noted above (including Code X

funds). TheDS ! project manager will be provided and have full control of the reserves noted above,

he will have full authority to use them at his discretion to ensure a suecessful projecL A formal

reserves usage plan (request and usage) will be developed that ensures that reserve usage is

minimized to the extent feasible. The NMP program office will not be in series with the approval of

DSI reserve usage.

• DSI project manager manages the DSI mission including the IPDT technologies (budgets, schedule

and risk) - program office will not "end ran" hisdecisions. Any decisions or direction on DS I from

the program office must be discussed with and agreed to by thc DSl project manager bcfote thcy can

be considered to be in effect.

• Mouthly pmject reviews will be conducted by the DSl pmjeetmanagerwhendnthepmgramoffice

WIll be invited to attend. At these meetings technical pm_ w,hedule, cost _d use oflxojeet

reserv_ will be_ No etber mviews are required except those authodzed bytheDSl project

manager (e.g. DDC, pre-ddp 8ridinternal peer reviews).

•Thc DSl lgojeet manager will work with his team members to develop a.descopc plan htohllohttng

_ suvings that 8re available vs. time. Once developed he will have full mthodty to lmplemem

the plan. He will make a dillgeat effort to find suvtngs in the cwtutt plan to pay for et_t pewth m

they oecur. However, oece the reua'ves usage has beaxne _ 0menne epptmch is alw_ to

have a minimum of 11_ resorv_ with respect to the current cost to otuplete), the desoepe plan will

be tmptmnmte o=e fet=d torote thetbove0ec tom. there, Upmmmserwm fuUymppm the

DSt pmJecgmmm . of.dnn=d teetmotogtmwm be Lqer t Up mmm 

Iresbeeainfesmed of the mmmm fer the clmage tmthe teehnolou _

• C omemofDStpmmnetof4$peoptewmbeanomu tuteUeiq301. Am=0mm t0of

these pe6ple will occulW the "Design Hub." Themove mustooeurialbe Juaetime frame. Moalm

for the move will oeme ftmn ntm_$1 funds.

• ADS I msuagemeat coeasel will b6 formed to provide sdviee and eoumel to the DS I wojeet

manager. This group will meet on a mgulsr basis (monthly or _y) and report its flnd_ to

the program manager.

E-21
NMP DSI ProJect Plan

Figure I-I DSI Memorandum of Agreement (Cont'd)

• XDSllT,_aCcpmmswill_hlredtoasslstinmanagiugthepmjem. Insdditloa.theresU/cfionsoa'

the splxoval e4e_ forDS! by theprogramoffice will he ndmd from _'K to $1000K.

• Bffecl/ve and I/meJy executloa of DSI in • "I)¢_, faster, _ mode will be employed.

• DSI will work with indusuy to build effective pmnetsldps and Ixoducdve technology mmsfer.

Kane Casan/

ConculTcncc:

Distribution:

B. Wilson

R.Me_g_

IPDT co-leads

DS 1 team members

DS I management counsel (M. Sander, J.Savino, and T. Spear)

E-22
NMP D$1 Project Plu

Section 2

Governing Documents

2.1 Project Documents

The document tree detailing the project documents for DS 1 are shown in Figure 2-1. The

definition of the documents are shown in Table 2-1.

E-23
OZ

n"o

ZIU

z_

II

II

II

III

II

II

II

II

II

II

II

II

II

II

II

II

II

II

II

II

II

II

II

II

II

!III

iiJ!

J.

E-24
NMP DS1 Project Plan

Table 2-1

[

t,

E

, t.

e

E

i

FI ,

!- l

bz •

if' E

• n J

;I I

! i

i

I

i]!l

E

r

|

!

i

, 1

i I

I I

L =

! t

II t •

pu

m la

E-25
NMP !_I Project Plan

Section 3

Level One Requirements, Goals and Mission Success Criteria

The following two pages provide the DS 1 level one requirements, goals, and mission

success criteria. The requirements and goals listed have been agreed to and signed off by NASA

headquarters, the NMP office, and DS I personnel. The primary difference between the

requirements and the goals listed on the first page is that changes (descopes or deletions) to the

goals do not require headquarters approval. The mission success criteria has not been fmalized,

though its contents have been tentatively approved by NASA as a working document. It is not

required to be finalized until one week prior to the launch of DS 1.

E-26
DEEP SPACE ONE LEVEL-1 RE_QUIREMENTS &

GOALS - .=--_,

Requlremmts

I. Validate the following prime technologies through space flight

Solar electric propulsion as primary propulsion

Advanced sohlr array

Autonomous navigation as primary navigation

3-D stack computer as the flight computer

Miniature imaging camera spectrometer

It is expected that other advanced technologies vital for 21st century science missions will also

be validated on DS1 to the greatest extent possible within the approved funding (see goals

below).

2. Launch by the end of July 1998.

3. Obtain validation data for the technologiesdemonstrated on DS I within 2 years of launch.

. Complete the Project within a Project cost cap of $138.5M. This cost cap includes funding

from the Office of Space Science, the Office of Space Access and Technology, and all costs

for launch services funded by the Expendable Launch Vehicle Program. Contributions from

separately funded technology programs, such as NASA's NSTAR and BMDO's SCARLET

Programs, are not capped.

Goals

o Fly by one asteroid and one comet and return images and spectra. Monitor solar wind

throughout the mission and measure the interaction of the solar wind with the targets during

the flybys.

2. Validate the following additional technologies through space flight:

Small deep:space transponder

Autonomy - remote agent architecture

Miniature ion and electron spectrometer

Autonomy - beacon monitor operations

Ka-band solid state power amplifier

Low power electromcs experiment

Multi-functional structure

Power actuation and switching module

P p.dby_ 'J

DavidH.Lehman,FlightTeamManager

DeepSp_One

E.KeneOmmi,PrognenManager

New MllleenlmnPmgram "

David A.Gtlman, _ _ve

omee ofspaceseteeee

Mission & Payload Development Division

E-27
NMP DSl Project Pllm

E-28
NMP DSI Project Plan

Section 4

Technical Plan

4.1 Introduction

This section provides a top-level description of the spacecraft, the technologies to

be incorporated, and the technical implementation strategy to be used by the DS 1 project. The

technical plan will be implemented by a team that is described in more detail in the Management

Section (5) of this plan. The DS 1 flight team is composed of the following elements:

4.1.1 JPL Flight Team

The JPL flight team has primary responsibility for the detailed mission and

system design, system integration and test, launch integration, and mission operations.

4.1.2 Industry Partner Flight Team

An industry partner has been selected competitively and is a member of the flight

team. A general approach is that the industry partner provides all spacecraft hardware (and

software) except those that are part of the advanced technology payload.

4.1.3 Integrated Product Development Teams

The NMP advanced technology Integrated Product Development Teams fIPDTs)

score the advanced technologies to be flight validated by NMP missions. Once an advanced

technology is selected, a hardware and/or software cognizant engineer, technicalmanager or

project element manager is selected to lead the development." This leader is appointed by the

IPDT co-lead and concreted upon by the DS 1 flight lead. The technology development leader

and his/her temn are also members of the DS 1 flight team.

4.2 System Description

4.2.1 Advanced Technology Payload

Through validation on NMP missions, the risk associated with key enabling technologies will be

reduced sufficiently to facilitate rapid (and affordable) incorporation into future missions. The

new technologies employed by DS1 have gone through the NMP selection process, and the

selection has been conctm'ed upon by NASA. The list of technologies to be flight validated on

DS I are as follows:

IPS

SCARIEr

SDST

MICAS

RA

AUTONAV

NSTAR Ion PropuMon System

Advanced Solar Array

SmanVeep-Spa Transponder

Miniature Integrated Camera And Speclrome_r

Autonomy - Remote Agent Architecture

Autonomy - Onboard Optical Navigation

I

I

I

rl

II

II

E-29
NMP DSl ProJect Plan

3DS 3D-Stack Processor H

PEPE Plasma Experiment for Planetary Environments In

PASM Power Actuation and Switching Module In

LPEX Low Power Electronics Experiment HI

BMOX Autonomy- Beacon Monitor Operations III

KAX Ka Band Solid State Amplifier HI

MFS Multi-Functional Structure HI

Note that the MFS shown above has been added to the payload since the MOA shown in Figure

I-I was signed off. All advanced technology payload items noted above must demonstrate

readiness through a series of technology readiness gates as defmed in the New Millennium

Program Technology Selection Process Plan (JPL D-13361).

4.2.2 Spacecraft Overview

Instrument Payload: Miniature Integrated Camera Spectrometer (MICAS)

Plasma Experiment for Planetary Environments (PEPE)

Attitude Control: 3-axis spacecraft with:

Spacecraft-mounted HGA

Solar array gimbaled in one axis

Spacecraft-mounted science instruments

NSTAR SEP thruster gimbaled in 2 axes.

- Slew rate requirements consistent with comet/asteroid flyby for

technology validation of the science instruments.

Power:. SCARLET solarconcentratorfrom BMDO which produces 2600 W (at1

AU) for both SEP engine and s/cloads.

Propulsion: Solar electric propulsion (using NSTAR supplied components) and a

hydrazine chemical propulsion system for attitude control.

Telecommunications: X Band using the SDST and the Mars Pathfinder lander antenna. Ka band

down link communication using the Ka band SSPA.

Lifetime: 24 months (excludes extended mission)

E-30
NMP DSl Project

4.2.3 Launch Vehicle

The launch vehicle for DS 1 is a Delta 7326 which will be provided through

NASA Goddard's Orbital Launch Services office. The DS1 performance requiremont at a

Declination of Launch Asymptote (DLA) of 28 degrees is 437 kg to a C 3 of 0 km2/s '. Note that

the costs listed in the MOA of Figure l- l have since been updated (see Table 6- l) to reflect the

cost of the Delta - the prior estimate was based on a Taurus launch vehicle.

4.2.4 Mission Operations System and Ground Data System (Ground Segment)

The approach for the ground segment for DS 1 is to utilize the Mars Patht'mder and

MGS ground data systems and baseline and upgrade them to support the mission. The main

difference between these projects and DS I is the incorporation of autonomy for the mission.

E-31
NMP _1 Project Plan

Section 5

Management Plan

5.1 Management Philosophy

This project must be conducted efficiently and economicafiy in order to meet cost

and schedul e requirements. There are several guiding principles that will be employed to

facilitate this outcome:

Select the right people for the job and empower them to do it. Encourage

team members to stretch beyond their capabilities.

Make agreements simple and interface designs robust

Project leaders will strive to facilitate the execution of tasks given team

members by giving them what they need to get their jobs done faster, better

and cheaper.

Fmd new ways of doing business; the DS 1 team will support new ideas,

processes and practices that have the potential to improve productivity and

have acceptable risk..

Attack multi-discipline problems by using concurrent engineering

- Encourage open communication among team members.

We will also develop and foster for our team members a "DS 1 mind set." This

mind set will include the following thoughts:

Design to cost - and the recognition that we must maintain our schedule tomaintain our cost.

Capability driven mission - recognition that the project will be dynamic

because of our extensive use of concurrent enginee "ring and because we are

flight validating breakthrough technologies.

Everyone must be a system engineer - we must all look at the big picture.

People must feel that they are empowered to solved there own problems -

and they aren't waiting for others to get their job done

- Work to create a "can-do attitude" through out the team

$.1.1 Low-Cost, (_hflck-Reactfon Approach

Table 5-1 contains an overall approach intended to be used by the project to meet

the objectives of developing a high quafity product at low cost and short schedule.

E-32
NMP DSl Project Plan

Table 5-1. Quick-Reaction, Low-Cost Project Implementation Mode

NMP DS 1 Quick-Reaction, Low-Cost Project Implementation Mode

JPL and its industry partner will establish and support a DS I team motivated to do more for

less. This team will:

Work closely with, be responsive to the customer, achieve up front agreements which

cannot change

Establish a clear set of mission objectives that are understood by ali parties

Strive to do more for less, walk line between over enthusiasm and ultra conservatism

Be projectized, make use of and be co-located near the JPL flight system testbed

Include in team dedicated finance, procurement, product assurance personnel

Use a mix of scarred old timers and bright energetic youth, give them responsibility. Trust

tbun

Focus application of resources to mission objectives, avoid unnecessary institutional

charges, work around unneeded bureaucracy, minimize external interfaces

Sensitize team members to cost, give engineers cost, _hedule and technical responsibility

Move technical planners to designers to testers to operators: womb to tomb

Establish a thorough WBS, cost cap major WBS items, accomplish thorough cost

estimates, track actual costs vs. plan monthly, maintain adequate reserves

Accomplish IDC early in project. Hit the street with p_urements as early as possible

Establish and maintain an integrated project schedule and control milestones, track

performance weekly

Use streamlined product assurance, less documentation, fewer independent checks, more

reliance on irnplementors, more reliance on extensive system test prior to launch

Go to best sources, use their methods, use expert consultants and peer reviews

- Be aware of the status of the JPL re-engineering teams. Serve as a proto-type for their

concepts where appropriate.

Achieve adequate margins, robust designs

Do up front planning, analysis and design, but assemble quickly and test thoroughly

Start rapid prototyping testbed activities at project start for the flight system & GDS.

Demo new concepts ASAP, such as autonomy, MICAS and SEP in simulated environment

Accomplish concurrent engineering, push to get necessary funding profile for this

• Emphasize up front mission and system engineering

Design mission, system, flight, and ground segments, including instruments.

Accomplish integrated fright and ground system S/W de.v.e.l.oTnnent teams ks

• Establish incremental deliveries with demonstrable capabilities for progress chec

• Clearly communicate status,plans, problems, solutions on a weekly basis

GDS = Ground Data System IDC = Interim Design Concurrence Review

H/W = Hardware S/Wffi Software

MOS ffi Mission Operations System WBS = Work Breakdown Structure

E-33
NMP DS1 Project Plan

$.2 Program RelatlonslMps and Respons/bflitles

$.2.1 Program Office

The agreement between the NMP manager and the DS 1 flight lead on how they

will work together is documented in a memorandum of agreement (see Figure 1-1).

5.2.2 Integrated Product Development Teams (IPDTs)

The IPDTs provide advanced technology road maps and candidate technologies

for the mission. The NMP program office, IPDT co-leads, and the DS 1 flight team work

together to select recommended technologies which are presented to NASA for a decision (see

JPL D-13361, "The NMP Technology Selection Plan").

A closely integrated, working, non-adversarial relationship between the flight

team and the IPDTs will be fostered. This approach is needed for the following reasons:

The flight team must ensure that candidate technologies from each of the

IPDTs are compatible and can be successfully integrated into the first flight.

Technical and integration issues must be addressed early in the design

process.

- It is consistent with concurrent engineering and a teaming environment.

NMP's tight cost and schedule constraints will not allow a "throw it over the

wall" approach.

- It fosters a "team concept" for future missions.

Selected technology providers will become part of the Flight Team while

maintaining membership on their IPDTs. This approach is needed to ensure successful

integration of the technology in the flight system. The relationship between the IPDT Cog. E

and the Flight Team will be one of supplier and customer.

5.2.3 Flight Team

An industrial partner, Spectrum Astro, Inc. (SAD, has been selected as a DS 1

flight team member wofldng as a true partner with JPL. The flight team (industry partner and

JPL) will identify, define, develop, and document a detailed mission and spaeeerafl" system

design. The flight team, including SAL will perform the following tasks:

(1) ComPlete and document, with other flight team members, the DS 1 detailed

mission and system design.

(2) Define, with flight team members, the following spacecraft capabilities and

design:

(a) Details of _e spacecraft system design, functions, interfaces,

subsystems, assemblies, flight software, and advanceA technology

E-34
NMI'DSl Pr ea man

hardware and software. Advanced technology hardware and software

will be identified and delivered to the flight team members from the

IPDTs

(b) Long-lead items with required procurement dates

(c) Spares philosophy and approach

(d) Spacecraft advanced technology and payload interfaces, including but

not limited to, the following:

(i) Mechanical interface descriptions

(ii) Electrical interface descriptions

(iii) Command and data interfaces

(iv) Environmental descriptions, including thermal envelope, static and

dynamic loads, and electromagnetic fields

(v) Cruise performance for onboard cruise autonomy

(e) Spacecraft metrics including, but not limited to, the following:

(i) Mass at launch

(ii) Power and thermal margins at launch

(3) Participate, as a flight team member, in the following meetings:

(a) System Requirements Review

(b) Mission Interim Design Concurrence (IDC) Review

(c) Mission Detailed Design ConcmTence (DDC) Review

(e) Sage Review

(0 Pre-Ship Review

(g) Launch Readiness Review

A contractual relationship with SAI will be established to facilitate and encourage a good

teaming arrangement. The goal is to have a "badgeless" environment, with clear

communications of and buy-in to project goals, and minimal formal documentation. In addition,

high speed data links and video communications will be established between SAI and JPL.

It is understood that the industry partner's role will contihue to grow in scope and responsibility,

with the intent that they become long term suppliers of deep space spacecraft for science

missions of the 21st century.

5.3 Organization and Work Breakdown Structure

5.3.1 DSl Team Composition

Figure 5-1 shows the project flight team composition. The roles for the lead

individuals on the team are as follows:

night Lead (JPL)

Flight Lead (SAT)

Responsible fo/achieving DS1 mission requirements and the planning,

implementation and control of the DS 1.

Respons_le for overall planning, control and direction of the SAI team.

Project manager for all SAI work. Assistant to the JPL flight lead.

E-35
- NMP i_1 Project Plan

¢q

i!i

Z n_

m

-J i

E-36
__lProJectl_m

System Architect

Flight Segment

Manager

Deputy Flight Segment

Manager

Chief Mission

Engineer

Chief Mission

Operations Engr

Procurement Lead

Finance & Schedule

Lead

Science _ad

Responsible for the overall mission and systems technical design "and the

achievement of the DS 1 technical requirements.

Responsible for the flight segment (including spacecraft and launch

vehicle integration) and the overall planning and control of the effort.

Deputy to the Flight Segment Manager. Has full authority as the Flight

Segment Manager

Responsible for the mission design development for DS 1 mission

engineering and navigation planning, and the overall planning and

control of the effort

Responsible for the mission operations development and

mission operations for DS 1 and the overall planning and control

of the effort

Responsible for the overall management of all DS 1 procurements

Responsible for development of budgets and schedules for DS 1

Responsible for DS 1 science

The above team leaders will work together and form sub-groups (or teams) to

implement the mission. The DS 1 working groups are shown in Figure 5-2. The charters of the

groups are defined in Table 5-2. As the project evolves it is expected that the charters of the

groups will change to fit the needs of the development cycle.

5.3.2 Work Breakdown Structure

The major Work Breakdown Structure (W'BS) for DS 1 is shown in Table 5-3

below.

WBS Item

Pro_ect Office

S_stem Architecture

sp,ce=.ft

Tom

Oround Se[gment Development

13round Selgment Operations

LaunehV_de

_Reserve

_OD

Table 5-3 Project Work Breakdown Structure (WBS)

E-37
•_ !_I Project Plan

-,]1

m

m

[.

i

...... %..._ ° ."

i

u i

|

z

Ill ! E

,q_1 i ii IIIIIIH IIIII

lilt Illllll

m

,ItI

!

E-3B
[...,

NMP DSI PrOjeCt Pllm

E-39
NMP DSI Project Plan -.

5.4 Resom'ce Ks'Umation, Allocation, and Reserve/Descope Strategies

5.4.1 Resource Estimation and Allocation

At least once each fiscal year, or as directed by the Flight Lead or designee, each

person responsible for a DS 1 work breakdown structure (WBS) element shall:

1. Estimate the element's resource needs to complete the work.

2. Negotiate a resource budget with the element's funding source.

$.4.2 Reserve Strategy/Descope Plan

Because of the tight budget reserve for the project a detailed reserve

strategy/descope (or risk management) plan has been developed. This plan highlights the uses of

the reserves for-theproject and where they can be applied. In addition, the plan shows how

descopes in the implementation plan will be made vs. time as reserves are used up throughout the

life of the project. (see JPL IOM DS1-DHL-96-016A, dated 7/25/96).

5.4.3 Resource Transfers and Fiscal Year Carryover

Resource wansfers may be traded among different WBS elements.

Individual WBS elements with unused budgetary allocation at the end of a fiscal

year, based on the concurrence of the Flight Lead or his designee, may carry the allocation

forward into the next year.

It is the responsibility of the NMP office to ensure that sufficient total funds are

carried over at the end of each _ year to provide funding for DS 1 until the new fiscal year

funding from NASA is on contract at JPL and available to the Project.

Funds aLlocated for a future year cannot be advanced to an earlier year without the

approval of the NMP manager.

$.q Project Control

$.S.1 Work Package Agreements

At the start of each fiscal year, each team element (cost acxount) leader will

prepare a Work Package Agreement (WPA) using standard JPL format. The WPA will be

negotiated with the Flight Lead, or designee,and will establish the cost account budgets, key

delivembles, and schedule for the task. -WPAs can be revised and a new WPA generated with

written concurrence from both parties.

IE-40
$.5.2 ProjectReporting

Project reporting will be kept to the minimum level required to track

progress, assess problems, and inform project, program, Laboratory, and NASA management.

The reporting plan includes the following elements.

S.5.2.1 NASA

Weekly reports of significant events will be provided to NAS A management

personnel via the program office.

Weekly teleconferences with NASA Headquarters covering the Flight Lead's

assessment of project status, accomplishments, issues and concerns will be conducted.

Inputs will be provided to the NASA's Program Operating Han as requested.

5.5.2.2 JPL

Weekly reports of significant events will be provided to $PL management

personnel.

The DS 1 Flight Leads will have weekly quiet hours with the NMP manager

to discuss the status of the project.

Monthly Management Reviews (MMRs) will be held at the project level by

the flight lead wherein the NMP manager is invited to attend. Topics tq be addressed include

accomplishments, goals for next month, cost status, cost to complete analysis, reserve status,

staffing status, schedule status, and issues and concerns (plus resolution plans).

ADS 1 management counsel will be formed to provide advice and counsel to

the DS 1 project manager. This group will meet on a regular basis (monthly or quarterly) and

report its findings to the program manager.

 ̧5.5.2.3 Technical Reporting

' The system architect, with the support of the DS 1 system element leads will

prepare a "state-of-DS 1" memorandum on a monthly basis. This memo will detail the stares of

the mission design and current margins for key parameters and metrics defining the mission.

5.5.2.4 Administrative

Weekly tracking of labor will be made by the flight lead, finance & schedule

lead, and chief engineers. In addition, informal Monthly Management Reviews (MMRs) will be

held for each mission element and/or spacecraft subsystem. At these meetings, the following

general items will be covered:

E-41
NMP DSI Project Plan

-Accomplishments of last period

-Goals for next period

-Schedule status

-Cost and work force status

-Cost to complete analysis

-Issues and recommended closure plan

-Help needed from flight team leads

5.5.3 Schedule

Schedule control will be achieved as a result of the weekly schedule reporting

meetings. Figure 5-3 shows the overall schedule for DS 1. The level 1 milestones are as follows:

a) Project start [October 1995]

b) Complete DSI Mission and Systems Interim Design

Concurrence (IDC) Review

[December 1995]

c) Complete Sage Review [May 1996]

d) Complete DS1 Mission and Systems Detailed Design

Concurrence (DDC) Review & Sage Review

[September 1996]

e) Complete Sage Review [May 1997]

f) Deriver P/trtial Bus from SAI to JPL. Commence

Assembly, Test and Launch Operations (ATLO)

[August 1997]

g) Complete system tests & ship toKSC [April 1998]

h) Launch Commitment to NASA

i)Complete advanced technology flight validation

j) End of Primary Mission

[July 1998]

[January 1999]

[July 2000]

At a detailed level, each system lead will develop and coordinate REC/DELs for all intersubsystem and key ifitra-subsystem milestones. The Finance/Schedule I.ead will coordinate this

system and provide a weeHy status report to DS1 team leaders

E-42
NM]P DSI Project Plan

E-43
NMP DSI Pro|ect Plan

E,-44
NMP DSI Project Plan

Section 6

Resource Plan

6.1

DSI.

Funding Plan

See Table 6-1 which shows the cost plan for each major work element of

Major costing assumptions for this table are shown below:

DS1 Fundin_ Assumptions

i

Re-engineering team support

SDST funding per Consortium Agreement

FY'96

IPDT funds for DS1 data/analysis & science

77O

40O

_('97

4926

1100

FY'98

1325

FY'98

330

FY'99 Total

7351

1500

Ka Band SSPA funds from Dr. Elachi 140 160 10 310

Multi-functional structure from E. K. Casani 6 90 18 18 6 138

Low power expedrnent from L Alkalai TBD iBD TBD i uD TBD

1000 1000 2000

The difference between the costs shown in Table 6-1 and that provided in the

MOA of Figure 1-1 is as follows:

Cut'rent estimate includes updated costs for launch services.

Current estimate shows revised assumptions for support from the JPL re-engineering team.

6.2 Facilities

NMP has developed a co-location plan (JPL D-13425). As part of that plan,

DSI will co-locate in building 301 by July 1996. The initial parts of that plan have been

executed with the core team in place. In addition, the following support is needed and is

described in the following documents:

PDC/DS 1 Support Plan, IOM by C. Briggs dated 16 Feb. 1996

FST/DS 1 Support Plan, IOM by P. Gluck dated 16 Feb. 1996

TMOD/DS1 Support Plan (D-13548, dated 3 July 1996)

E-45
NMP DS1 Project Plan

Detailed Design Concurrence (DDC) Review

_ Systems or Subsystems shall conduct DDCs at the time detailed

designs/drawings are ready for release and before fabrication begins.

Minimize coverage of areas known to be in good shape.

- Minimal overview tutorial, including configuration and/or block

diagram

- Dominant Level-2, 3, and 4 functional requirements/capabilities

driving System or SIS

- Major external and internal interfaces affecting design

- Details of design (and analysis, if appropriate)

- Solutions to major problems from IDC (functional and interface)

- Applicable test data

- Design/resource margins (timing, memory, torque margins, etc.)

- Drawing status

- QA document/aids/test plan/safety status

- Key schedule items/schedule problems

- Issues/planned actions

- Cost and W/F summary for board's "gut level" assessment

NOTE 1: Any requirement not being met (formal. or informal) shall be

a review item

_ NOTE 2: The DDC is not met to be a decision point for a go-ahead for

procurements.

Table 5-7 DS 1 System or Subsystem Detailed Design Conctmence (DDC) Review

E-46
NMP DSI Project Plan

Interim Design Concurrence (IDC) Review

IDCs shall be conducted after design concepts are established and

detailed design is beginning. As little time as possible shall be spent

on routine description of areas that are in good shape. The agenda

shall emphasize items selected from new, controversial, or problem

items including the following areas, as appropriate:

- Minimal overview tutorial

_ Major task description

_ Configuration and/or block diagram

_Major design options considered

- Dominant Level-2, 3, and 4 functional requirements/capabilities

driving System or S/S

- Major external and internal interfaces affecting design

_ Inheritance review of h/w & s/w as appropriate

- Design details/description

- Safety design approach

- Proof-of-concept test results (as appropriate)

- Schedule

- Issues/planned actions

- Cost and W/F summary for board's "'gut level" assessment

NOTE: Any violation of Project Requirements (PDs, FRs, etc.) or

other requirements not being achieved will be a review item

Table 5-6 DS 1 Interim Design Concurrence (IDC) Review

E-47
NMP DSI Project Plan

Review Process

_ Utilize a series of system or subsystem-level peer reviews for IDCs and

DDCs

- Reviews are chartered under the Flight Lead (system reviews) or

Chief Engineer (subsystem review) and conducted by the system

or subsystem lead (as appropriate)

- All reviews are attended by a core board selected by the Chief

Engineer and chaired by a senior JPL engineer outside the project

- Attendance will be limited to a small set of:

_ Technical specialists and peers from the System or SIS being

reviewed

_ Representatives of the flight team and key interfaces

- Attendance shall be limited to JPLgmdustry partner personnel,

except where special outside expertise is required. Non-team

attendees to be approved by YPL lead.

- All attendees are members of the "board" and expected to

contribute

_ Review board responsibilities:

- Probe technical approach and schedule for adequacy and

practicality

- Bring experience from past programs to identify potential problem

areas

- Recommend areas for special attention

- Recommend on the continuation of the effort to the next phase

_ Review secretary (e.g., Review chairman or System or SIS Lead) will

capture all action items, advisories, and comments in a running memo

during the meeting (no RFAs)

- To be reviewed at and issued after the review

- Focus should be on identifying potential problems and potential

solutions

Table 5-5 DS 1 System or Subsystem Review Process

E-48
NMP DSI Project Plan

5.5.4 Review Plan

Consistent with the NMP policy of minimizing cost and expediting schedule

performance, project reviews will be held to a minimum. All reviews are in accordance with

NMP requirements. The following reviews are planned.

m

D

Initial Technology and

System Requirements Review

Mission and System Interim Design

Concurrence(IDC) Peer Review

Sage Review

Mission and System Detailed Design

Concurrence(DDC) Peer & Sage Review

Sage Review

Spacecraft Pre-Ship Review

Launch Readiness Review

[October 1995]

[December 1995]

[May 1996]

[September 1996]

[April 1997]

[April 1998]

[June 1998]

DS 1 reviews at the system level and below will be peer reviews, with the

major objective to help engineers get their job done. They are not intended to be status reviews

for management. The summary purpose of the reviews are as follows:

Review

Requirements and

Design Reviews

Sage Reviews:

Technology Readiness

Review Gates:

Spacecraft er=-Ship

Review

Launch Readiness

Review

purpose

Peer review of the design

Sage review of mission implementation plan

Reviews to assess development status of a technology product and to

determine the likelihood that the product will meet its objectives

Assess readiness status to ship the DS 1 spacecraft to KSC

Assess readiness status to launch the DS 1 spacecraft

The IDC and DDC reviews will be conducted in accordance with the objectives and approach

shown in Tables 5-4, 5-5, 5-6, and 5-7. Technology readiness gates for all new developments

will be conducted in accordance with NMP JPL D-13361.

E-4g
NMP DSI Project Plan ..

Design Review Objectives

_ Detailed, "shirtsleeves," hands-on peer review of a system, subsystem

(S/S) or part of a SIS. Not a push up for management. These reviews

are to help engineers

_ Demonstrate technical definition consistent with the phase of

development

- IDC

_ Complete system SIS architecture (block diagram,

configuration, flowchart, etc.)

_ Preliminary designs (layouts, circuits, algorithms, procurement

specifications, ICD's, eU_.)

Inheritance review of h/w & s/w as. appropriate

- DDC

_ Complete system or S/S designs, ready for fabrication/coding

(drawings, data flow & state transition diagrams, proto code,

etc.)

_ Design analysis, failure modes & effects analysis, error

detection and recovery

_ Demonstrate adequatedefinition and understanding of driving

requirements, interfaces & capabilities, and verification method

_ Demonstrate adequacy of margins

_ Demonstrate understanding of driving technical risks, means for

handling risks & safety

_ First-order "gut feel" assessment by core board of whether designs and

pr0blem/risk areas are consistent with planned work force, budget, and

reserves

- Following technical, schezlule, and issues, a one-page budget

summary will be presented by the Cog E

- Core board will be asked to react with nothing more than "looks

OK" or "I think you've got a problem"

Table 5-4. DS 1 System or Subsystem Review Objectives.

E-50
!

s: p

| i

o <> |

I_ EO

o li

O O

O ¢ O

! !F

° ! i

_! |

_ Y

° !

I I I I t f I liliR_'_lllltlilit!l,.,, iIi1!iiI_

I I I _t I l J,lil_Iil i iI'_,1;i;I! It ii

i

s s s s • s s s • • s!! s s s sli_z,_ii t z

I F

li

l>

I:>

!

s

I:>

.

" : .ll _ !Sl It!!! _)!!!_'!_ _'ii_ iii,,:, !i':':l lira: ,

I II,iI Iri

Ii i ii'ii' i ,'lsi'Jl ' lJI ,

....... i

I#1#11 ltl I!! I| I!!

E-51
II

I !

I:!

|

0

|

! |,,

i

I ...................... i

!

It

l!!lltlllil!!!! !I

i

t1111IIt1_111'_it

! !_+I_!II _II I_t_l'tit I It l+iI*Iii_I!I'l Iii !litil_! I I I!I i I

tS ill _ +"_"-_t' Ii ;!t l ,.il,,, ++ .-:

i !ii"i':+" ,i,,, i. ill+ :,,

I=I,I.I.I,I=I=

E-52

i

1

Ii
NMP IlSl Prelect Pilm ....

Design Review Objectives

_ Detailed, "shirtsleeves," hands-on peer review of a system, subsystem

(S/S) or part of a S/S. Not a push up for management. These reviews

are to help engineers

_ Demonstrate technical definition consistent with the phase of

development

- IDC

_ Complete system S/S architecture (block diagram,

configuration, flowchart, etc.)

_Preliminary designs (layouts, circuits, algorithms, procurement

specifications, ICD's, etc.)

Inheritance review of h/w & s/w as appropriate

- DDC

_ Complete system or S/S designs, ready for fabrication/coding

(drawings, data flow & state transition diagrams, proto code,

etc.)

_ Design analysis, failure modes & effects analysis, error

detection and recovery

_ Demonstrate adequate definition and understanding of driving

requirements, interfaees & capabilities, and verification method

_ Demonstrate adequacy of margins

_ Demonstrate understanding of driving technical risks, means for

handling risks & safety

_ First-order "gut feel" assessment by core board of whether designs and

problem/risk areas are consistent with planned work force, budget, and

reserves

- Following technical, schedule, and issues, a one-page budget

Summary will be presented by the Cog E

- Core board will be asked to react with nothing more than "looks

OK" or "I think you've got a problem"

I I

Table 5-4. DS I System or Subsystem Review Objectives.

E-53
NMP DSI Project Plan

5.5.4 Review Plan

Consistent with the NMP policy of minimizing cost and expediting schedule

performance, project reviews will be held to a minimum. All reviews are in accordance with

NMP requirements. The foUowing reviews are planned.

_ Initial Technology and

System Requirements Review

_ Mission and System Interim Design

Concurrence(IDC) Peer Review

_ Sage Review

_ Mission and System Detailed Design

Concurrence(DDC) Peer & Sage Review

_ Sage Review

_ Spacecraft Pre-Ship Review

Launch Readiness Review

[October 1995]

[December 1995]

[May 1996]

[September 1996]

[April 1997]

[April 1998]

[June 1998]

DS 1 reviews at the system level and below will be peer reviews, with the

major objective to help engineers get their job done. They are not intended to be status reviews

for management. The sunmmry purpose of the reviews are as follows:

Review

Requirements and

Design Reviews

Sage Reviews:

Technology Readiness

Review Gates:

Spacecraft Pre-Ship

Review

Launch Readiness

Review

PUrD_Ose

Peer review of the design

Sage review of mission implementation plan

Reviews to assess development status of a technology product and to

determine the likelihood that the product will meet its objectives

Assess readiness status to ship the DS 1 spacecraft to KSC

Assess readiness status to launch the DS 1 spacecraft

The IDC and DDC reviews will be conducted in accordance with the objectives and approach

shown in Tables 5-4, 5-5, 5-6, and 5-7. Technology readiness gates for all new developments

will be conducted in accordance with NMP JPL D-13361.

E-54
NM]P DSl ProJect

Review Process

_ Utilize a series of system or subsystem-level peer reviews for IDCs and

DDCs

- Reviews are chartered under the Fright _ (system reviews) or

Chief Engineer (subsystem review) and conducted by the system

or subsystem lead (as appropriate)

- All reviews are attended by a core board selected by the Chief

Engineer and chaired by a senior JPL engineer outside the project

- Attendance will be limited to a small set of:

_ Technical specialists and peers from the System or S/S being

reviewed

_Representatives of the flight team and key interfaces

- Attendance shall be limited to JPL.rmdustry partner personnel,

except where special outside expertise is required. Non-team

attendees to be approved by JPL lead.

- All attendees are members of the "board" and expected to

contribute

_ Review board responsibilities:

- Probe technical approach and schedule for adequacy and

practicality

- Bring experience from past programs to identify potential problem

areas

- Recommend areas for special attention

- Recommend on the continuation of the effort to the next phase

_ Review secretary (e.g., Review chairman or System or S/S Lead) will

capture all action items, advisories, and comments in a running memo

during the meeting (no RFAs)

- To be reviewed at and issued after the review

- Focus should be on identifying potential problems andpotential

solutions

Table 5-5 DS 1 System or Subsystem Review Process

E-55
DSI Project Plan

Interim Design Concurrence _) Review

_ IDCs shall be conducted after design concepts are established and

detailed design is beginning. As little time as possible shall be spent

on routine description of areas that are in good shape. The agenda

shah emphasize items selected from new, controversial, or problem

items including the following areas, as appropriate:

- Minimal overview tutorial

_ Major task description

_ Configuration and/or block diagram

_ Major design options considered

- Dominant Level-2, 3, and 4 functional requirements/capabilities

driving System or S/S

- Major external and internal interfaces affecting design

_ Inheritance review of h/w & s/w as appropriate

- Design details/description

- Safety design approach

- Proof-of-concept test results (as appropriate)

- Schedule

- Issues/planned actions

- Cost and W/F summary for board's "gut level" assessment

NOTE: Any violation of Project Requirements (PDs, FRs, etc.) or

other requirements not being achieved will be a review item

Table 5-6 DS 1 Interim Design Concurrence 0DC) Review

E-56
NMPi_I_-oJmFI_

Detailed Design Concurrence (DDC) Review

_ Systems or Subsystems shall conduct DDCs at the time detailed

designs/drawings are ready for release and before fabrication begins.

Minimize coverage of areas known to be in good shape.

.- Minimal overview tutorial, including configuration and/or block

diagram

- Dominant Level-2, 3, and 4 functional requirements/capabilities

driving System or S/S

- Major external and internal interfaces affecting design

- Details of design (and analysis, if appropriate)

- Solutions to major problems from IDC (functional and interface)

- Applicable test data

- Design/resource margins (timing, memory, torque margins, etc.)

- Drawing status

- QA document/aids/test plan/safety status

- Key schedule items/schedule problems

- Issues/planned actions

- Cost and W/F summary for board's "gut level" assessment

NOTE 1: Any requirement not being met (formal or informal) shall be

a review item

_ NOTE 2: The DDC is not met to be a decision point for a go-ahead for

procurements.

Table 5-7 DS 1 System or Subsystem Detailed Design Concurrence (DDC) Review

E-57
NMP DSI Project Plan

Section 6

Resource Plan

6.1

DSI.

Pending Plan

See Table 6-1 which shows the cost plan for each major work element of

Major costing assumptions for this table are shown below:

DS1 Funding Assumptions

R_-engineedng team support

SDST funding per Consortium Agreernent

Ka Band SSPA funds from Dr. Elachi

FY'96

IPDT funds for DS1 data/analysis & science

770

400

140

FY'97

4926

1100

160

FY_J8

1325

10

FY'98

330

FY'99

7351

Total

6

TBD

1000

1500

310

Multi-functional structure from E. K. Casanl 6 90 18 18 TBD 138 Low power experiment from L Alkalai TBD ITBD TBD

1000 2000

The difference between the costs shown in Table 6-1 and that provided in the

MOA of Figure 1-1 is as follows:

Current estimate includes updated costs for launch services.

Current estimate shows revised assumptions for support from the JPL re-engineering team.

6.2 Facilities

NMP has developed a co-location plan (JPL D- 13425). As part of that plan,

DS1 will co-locate in building 301 by July 1996. The initial parts of that plan have been

executed with the core team in place. In addition, the following support is needed and is

described in the following documents:

PDC/DS 1 Suppon Plan, IOMby C. Briggs dated 16 Feb. 1996

FST/DS 1 Support Plan, IOM by P. Gluck dated 16 Feb. 1996

TMOD/DS1 Support Plan (D-13548, dated 3 July 1996)

E-58
N_, Dsz ]Proleet_

|___

m m

E-59
_1 ]_M_ Pbm

E-60
PlutoWA&I_lan_CostRev9_slmprov.lnltiatives

Pluto Proems lmurovement Initiatives (select list 93/10/4)

key a) already implemented or included with baseline

b)/n progress or planned

c) possible under consideration

How We Do Business

Worldorce Cap

Tight Team development

Sciencecraft implementation of intemmted science and eng'g, teams

Skunk Works TM collocation and management style

allpeople. >0.9 (near full time) or <0.15 (consulting only) FTE

I.&ILQ,..S.bJ_, _ Ops & Contractors as Full Partners

Groupware

All project meetings avail by telecon or in person

multimission applications

Make/buy based on cost

"Rat" Organizational slxucture

Concurrent development team from start

Clear aeeountability_ for WBS elements

Less hierarchy-->te0_ m members comfortable suggesting improvements to peers

and for Project as a whole

Within-co_t, history of meeting all deliverables from start

Earned valfue, up-to-date accounting; product-based WBS

Responsive to redirection as environment changes: ATI, NMP Coordination, Life

Cyele cost

Should stick to plan from here forward

Difficult ob_iectives-->Conservative estimates->often exceed performance

Minimum documentation

All documentation on-line, maintained by WBS.elethent Cog

No >1 page Requirements Does; proceed direct to draft interface specs

and equipment descriptions/designs

Document new design by As-built next-earlier generation plus deltas

Team Life Cycle Cognizance

Now until data returned (attitude: new people "'carry the mantle" of

predecessors on infrequent attrition)

ComaiT_ant, individuals resoonsible for their element's reliability_

Co-__iT_nee by individual, not line organization

Build early to work out interfaces, find flaws

Design to Test ltlll[_QW.Ial_

Rapid prototyping of hardware and software

Sim-->prototype_ -->flight units in testbed

Processlmprovlnitiatives RLS 960710

E-61
facilitates alliance between industrial technology develovers and

JPL equipment and o_Derations designers

Science Teams

. Minimal Pluto science team durin_ cruise

Compete added encounter team slots E-2 yrs for final encounter plan

No-Nonsense contractin_

Set difficult spas where they count: _OSt, m_s, power

Not over-specified where unim_nortant

Hardware & software: not desit, ns & studies

Select best bid from qualified source, not last mi.q_ion incumbent

Limit cost with conviction: No ATI overnm_ _0 far

Firm fixed price development contracts and cost-capped contr'0ct_

Minimum RFP addenda

No contract mods without Prepro_iect ram'. permission: none so far

Streamlined university_ contracting: latest @ 2 pg + coy letter

Use contractor product assurance to meet lifetime spec.

Use contractor reporting format

Frequent informal cost & schedule status discussions between JPL &

Communicate im_tmrtance of cost performance tO CEOs

Solicit ideas on commercial awlications, comnetitiveness,

educational benefits. _Lr_dways JPL & b/ASA can help amplify.

Compete next prototype w/ option for flight; keep ATI/NMP incumbent if

best.

Active promotion of small business. SDB, and HBCU/MI involvement.

Simple contract costschedule monitoring & management tools

frequent informal reporting

JPL contract technical monitors & negotiators work closely

Wide contracting and geom'aphic involvem.ent [see PlutoParticipants Map]

We have gone for best. rather than chosen location; we will continue

to select the best.

Lower burden (went from 2.1 to 1.8 x salary_ FY95 > FY96), or recycle burden to augment

development where available

Among first users of JPL flight system testbed, project design center.

PCAT. PTM

Young. mixed tea_,

Mix of JPL workforee

Creative and innovative

Balanced with senior exnerienfe (more to be recruited)

Clearly defined, highest priority_ science objectives: nothing else drives df._it)n)

TOM-driven emtmwerment and awareness

Focus off customer need._

Primary_ accountable team members, eo_tmiz.a!3__tengineers, accountable _

Processlmprovlnitiatives RLS 960710

E-62
(__ _r. _ and b_m_r as everyday desire1 metrics.

Senior JPL rnanRm__meat ]ist_nin_and helt)ing

Processlmprovlnitiatives RLS 960710

E--63
Draft Generic Instructions to Project or Task Managers

INSTRUCTIONS FOR LEADING THIS PROJECT

lo You are responm'ble for successful execution ofthe mission on time and within

budget, i

2. You may not go back to the welL

. You nmst be ready to launch on yy/mm/dd. Successful integration with the launch

vehicle is your responsibility.

. You may recruit and dismiss personnel to and from your project at wilL (This

amounts to authorizing use of one of your charge numbers.)

5. You must provide technical, financial, and schedule visibility into your project at

any time, but only to (N) designated individuals. You may use tools and service

organizations at your discretion to assist with this.

. You nmst maintain explicit schedule and budget margins. You may use up to

(X%) of any margin without prior management approval

o You may alter the performance of your system in order to meet schedule and

budget constraints. Your performance will be evaluated in part based on how

much performance your system delivers. If the customer is dissatisfied with the

performance ofyour system, you may be reassigned or dimnisse_

, You are responsible for operating within the laws and regulations applicable to

JPL and its employees. The Lab will make available to you, for use at your

discretion, skilled personnel to assist with procurement, safety, product assurance,

media relations, launch approvaL...

. System performance requirements will be given to you on one page, and will not

be altered after yy/mm/dd without an opportunity to revisit schedule and budget

commitments.

10. The Laboratory _ stand behind the implementation plan agreed to between you

and your ALD, and w_ not re-assign the personnel or fatalities needed by your

project until the release dates in your approved implementation plan.

Among the three variables, schedule, performance, and cost, two may be fixed. These instructions are

based on fixing cost and schedule, based largely on you reports of what Headquarters managers are calling

for. Other activities, such as technology development, may call for a performance objective on a go-asyou-pay basis, leaving schedule as the dependent variable.

E.-64
11. Upon the su_ oompletion of various phases of your project, you will be

provided with promotion and associated raises as agreed in a memorandum of

agreemem betwee_ you and your ALD.

E--65
2nd IAA International Conference on Low Cost Planetary Missions

April 16-19, 1996

COST ESTIMATION AND MODELING FOR SPACE MISSIONS AT

APL/JHU

•L. J. Crawford, T. B. Coughlin, W. L. Ebert

Applied Physics Laboratory

The Johns Hopkins University

Johns Hopkins Road

Laurel, MD 20"123-6099 U. S. A.

ABSTRACr

Cost estimation of space science missions

at APL over the past two decades has

been singularly successful in arriving at

program costs that are within a few

percen t of the actual costs at program

completion. The most recent example is

the Near Earth Asteroid Rendezvous

(blEAR) mission which was estimated at

approximately 112 million FY-92 dollars

and came m at approximately three

percent under the estimated cost. This

demonstrated performance has been

achieved without the benefit of a formal

cost model, such as those used in

government and industry (GSFC,

MSFC, SAIC, etc.). In light of this

performance,/t is important to understand

the parameters that are used in the cost

estimating process in an effort to quantify

those elements in a program that arc most

impoRant to the final cost. Wehave

identified a number of areas which

contribute to eventual cost performance;

these inclu&: (a) _ and mission

complexity; Co) use of aLready-developed

(facility-class) iusmm_ents versus ',to be

develo._" new insUmnents; (c)

syaergzsm among programs being

implored conctLrrently; (d) program

implementation length; (e) design-to-cost

practice for all major subsystems and

instruments without contingency; (0 lead

engineer responsibility throughout

design, layout,, fabrication, test,

integration,and initialflightopczations;

(g) dcsigned-4n qualityand testabilityto

minimDe rework; 01) inooIporsfiOn of

reliability and quality assurance

engineering within the program stmoaLr¢;

(i) minimization of documentation and

encouragement of oral and electronic

communication as required. We have

found that gross pammeUization of costs

such as the traditional weight, power, and

length of the program commonly included

in typical models do not reliably predict

acuml costs. A methodology will be

presented, whereby, the elements

identified above plus others are used to

describe the process implemented by APL

in previous missions to generate cost

estimates and tocontrolcosts.Actual data

over severalmissions during the past two

decades will also be presented that

illustrate APL's cost performance while

utilizing this methodology.

INTRODUCTION

In today's funding-constrained

environment, governments are searching

for solutions that permit the current

robust national and international -space

programs to continue while their science

and technology budgets decline. To

reduce the cost of space science missions,

and thus to allow more. missions to be

conducted for the same dollars, they are

demanding that space hardwaro and

software be produced faster, cheaper

and with equal or better oapabilities. All

of these goals are to be accomplisbed

through technology insom'on with

acceptable risk (read this as low riskl)

and through improved design,

fabrication, testing and other procedures

that are more efficient. NASA's

Discovery Program was established in

1992 to implement this faster,, cheaper

and better process for future planetary

space missions. The Near Earth Asteroid

Rendezvous (NEAR) Project being

E-66
conducted by the MU/APL is the first of

the NASA Discovery Program missions.

With the spacecraft successfully launct_

on a Delta II rocket on 17 February 1996,

it began its journey to a rendezvous with

the asteroid Eros. To date, the NEAR

Project has exceeded all Discovery

Program cost, schedule and performance

goals.

The extreme pressure placed on meeting

cost goals for space science missions has

resulted in a search by private and public

organizations to fmd ways of reducing

the costs of space systems, of predicting

these costs and of actually achieving the

originally estimated costs at the end of the

project A few papers are beginning to

appear in the literauue that attempt to

address cost reduction via small satellite

development and technology insertion

and that discuss the associated

relationships needed for cost estimating.

Very little attention has been given in the

literature, however, to cost control. Cost

control is perhaps the most important

element in this regard since, in the end, it

is the magnitude of the cost and its

comparison to the initial estimate that

ultimately matters in today' s

environment. In an attempt to arrive at

simple cost_g relationships for

small satellites, previous efforts have

focused on relating a few technical

parameters such as schedule, dry bus

mass, pointing accuracy, data storage,

average powerand others to development

costs. While these simple relationships

are instructive and _taJnly have some

value, we believe that they are not very

good estimators of cost for the general

case. Each project is unique and,

therefore, has its own cost drivers that

differ from cost drivers for other projects.

With a very large number of possible cost

elements and with each project having its

own unique •constraints, simple

relationships between costs and a few

technicfl patamete_ have not proven to

be very useful in predicting costs of space

science missions conducted at APL.

Consequently, no formal cost estimating

formulas are currently used at APL.

This paper will mt cost estimation Of

space science missions at APL. We will

discuss the system el_ts involved in

establishing and controlling the costs of

our space systems, we will identify many

of the important cost drivers and we will

discuss cost control techniques. A

methodology will be presented, "whereby,

the elements identified above plus others

are used to describe the process

implemented by APL to generate cost

estimates and to control costs for

previous missions. Actual data over

several missions during the past two

decades will also be presented that

illustrate APL's cost performance while

utilizing this methodology.

SYSTEM ELEMENTS AND COST

CENTERS

In discussing cost estimates and cost

control, for space science missions, it is

important to capture all of the major cost

centers and all of the major external

factors that either influence cost or affect

cost control. Our view of this system of

cost centers and external factors is shown

in Figure 1. Besides capturing the

traditional spacecraft bus and payload

systems, our system view includes

Government Furnished Equipment

(GFE), the APL management system

element and the customer driven e_

enviro_t system elements. These

three entities, the Project, consisting of

the spacecraft bus, payload systems, and

the GFE interface, the APL organizational

unit and the customer interact from a cost

perspective. They are principally

responsible for the magnitude of the

initial cost estimate and the subsequent

actual cost at Project completion. Each of

these will be discussed separately below.

The work breakdown structure (WBS)

and cost centers shown below are the

ones typically used at APL, and are

similar to those used by other apace

systems development organizations (also

see Reference 5, Figure 9). They are also

E-.67
many years ago, APL has developed a

way of doing business that minimizes the

organization's impact in those areas. In

this regard, some of the important

attributes of our culture that minimize cost

impacts

1. Foster quick program

execution with short overall development

schedules (< 36 months).

2. Enforce design-to-c_st

practice for all major subsystems and

instruments without cost contingency.

3. Empower lead engineers

with responsibility throughout design,

layout, fabrication, test, integration, and

initial fright operations (proposal to postlaunch.)

4. Practice designing - in

quality and testability to minimize rework

and the associated cost and schedule

impacts.

5. Incorporate a reliability

and quality assurance engineer within the

Project structure as a team player rather

than an outside adversary.

6. Minimize written

documentation required by the APL

organization and encourage oral and

electronic forms of communications.

7. Empower Project Manager

with complete responsibility and authority

to execute all aspects of the project.

8. Recognize the synergism

among programs being implemented

concurrently and the consequent costsavings potential for customers.

9. rmimize Projectreportmg

requirements to upper management and

oversight requirements by upper

management.

I0. Attempt to establish a

single customer interface with minimnm

customer imposed docaunentation,

oversight and other extra cost-generating

requirements.

I 1. Reduce Project

Management overhead - a typical project

office sm_cture would include a project

manager, a system engineer, and a parttime secretary.

12. Provide a stable work

environment to reduce annual staff

turnover and permit giving a large portion

of a job to a single person.

Customer

This system element is not a cost center

per se either. However, the customer can

significantly affect cost estimation and

control at his own organization and at the

contractor's organization through the

customer's policies, procedures and

practices. Listed below are some of the

important attributes of some of our

customers that have helped to minimize

cost impacts in the past.

1. Establishment of clear and

achievable Project cost, schedule and

performance requirements and goals.

2. Ability to stay focused on

achieving the project goals and remain

committed to them in the face of a

changing environment.

3. Management of APL by

the sponsor with a single person interface

and practice a hands-on-lightly

management approach to minimiTe

imposed documentation, oversight and

other extra cost-generating requirements.

4. Creation of a problemsolving, Wust-me atmosphere with APL

to promote open communications and to

create an environment where the customer

helps in arriving at solutions.

METHODOLOGY

Our performance record during the past

two decades will be presented later after

discussing here the formal methodology

utilized to manage it. At APL, our

principal focus is to manage the end

product of a project which involves

delivering a specific technical product, for

a specific cost and within a specific

schedule, rather than solely managing

cost-for cost sake. This focus begins

early in the project life cycle during which

an initial cost estimate is ge_ that

meets project cost, schedule and

goals. The same focus continues

throughout the life of the project until the

technical products are delivered, within

E-68
DATABASE

Figure 2 Cost Estimation Methodology

the cost and schedule goals. The

difference in the initial cost estimate and

the actual incurred cost is then one figure

of merit used for evaluating project

performance. We will present next, our

formal methodology for generating our

initial cost estimate followed by a

discussion of the methodology utilized to

control those costs during project

execution. This methodology, shown in

Figure 2, will be presented in the context

of the system elements discussed above

and involves assessing the impact that the

project, the APL organizational unit and

the customer have on both cost estimation

and cost control.

Cm l imati a

Cost estimation of space science missions

at APL is a proofs that _ m|n|mum

use of cost-estimating relationships

genet_od by statistical regressions of

industry-wide or even company-wide

actual space systems cost data. The

process involves interactions among the

project, the APL organiz_onal unit and

the customer, draws on our historical data

base and utilizes the experience of our

technical staff acquired over decades of

functioning in a stable work environment.

The process consists of executing the

following five steps; (a) Identify

Customer Requirements; (b) Cmnorate

Project Rough-Order-of-Magnitude

(ROM) Cost Estimate; (c) Gemram

Bottom-Up Cost Estimate; (d) C_nemm

Top-Down Cost Estimate and (e)

Generate Initial APL Cost Estimate. The

first four of these focus on a cost risk

(and schedule and performance risk) as a

design driver. Thus the initial cost

estimate is supported by a risk

management approach.

1. Identify Customer Requirements

The first step is to establish open and

frequent comrrmn|c-qfiorm with

customer at both the APL Project Office

level and at the APL organizational

management level. The outputs from this

step provide the answers to three key

questions that drive the initial magnitude

E-69
JHU/APLORGANIZATIONAL CONS'IIIAIN_

_MER

EXTERNAL

PROJECT

CONSTRAINTS

Figure 1 APL Spacecraft Development System View Point

similar to those found in the literature for

typical satellite programs 4

° Spacecraft Bus Cost Center

(a) Program Management and System

Engineering; Co) Mechanical; (c)

Telecommunications; (d) Power and

Thermal; (e) Attitude; (f) Ground Support

Bquipmont; (g) Integration, Test and

Launch; (h) RF System (Antennas &

Transmitters); (i) Reliability and Quality

Assurance (R&QA); (j) Propulsion.

2. Payload Cost Center

3. GFE Government Furnished

Equipment Interface Cost Centers

The Project staffs role in the estimating

proems for the initial costs is to ensure

the lowest possible project cost is

developed consistent with the Project's

cost, schedule and pc_forman_ goals.

During the execution phase, the Project

•staff is the front line of defense for APL

in controlling the actual cost incurred.

The Project Managex implements APL's

an.a

described m the section oelow during me

execution of the Project. Additionally,

the Project practices the following

gnidelin_ to _ impacts to cost:

I. Focus the team's effort on

delivering the end product of the Project

which involves delivering a specific

technical product, for a specific cost and

within a specific sehedule.

2. Design for simplicity. We

have found that perhaps the biggest single

factor that drives space systems cost and

our ability to control cost is spacecraft

and mission complexity.

3. Where possible, we use

already-developed instrmnents (facilityclass) and components versus "to-bedeveloped" new systems.

4. Immediate placement and

fight technical monitoring of subcontracts

and use of incentive-based controls.

5. Tight techmcal monitoring

of GFE. A separate interface engineer is

assigned to each GIVE component for the

purpose of ensuring that the GFE is

delivered on time and that. it's fit, form

and function are consistent with the

spacecraft bus development.

APL _iT_fi_onR! Unit

This system element is not a cost center

per se, but through an orgai_afion's

business policies, procedures and cost

accounting pra_o¢_, it could

significantly affect cost estimation and

control. Having recognized this point

E-70
of our cost estimate and ultimately our

ability to control the actual project costs.

These questions (cost drivers) are: (a)

What is the customer really trying to

accomplish as reflected in the stated

project technical, cost and schedule

parameters7 It is important to establish,

particularly at this early stage in the

project life cycle, which customer desires

are hard and fixed requirements and

which customer desires can be treated as

goals. (b) How does the customer intend

to manage APL during the execution of

the project? We always attempt to

establish a single customer interface with

minimum customer-imposed

documentation, oversight and other extra

cost-generating requirements. We believe

that our organization functions best under

these conditions as reflected in our past

performance and that this approach

results in lower overall cost to the

customer. (c) Is the customer's attitude,

commitment and/or will strong enough to

stay the course in holding the line on

project requirements and goals?

Although the answer to this question is

subjective, the commitment or, more

importantly, the lack of commitment of

the customer's will, significantly affects

our ability to control actual project cost.

2. Generate Project Rough-Order-ofMagnitude (ROM) Cost F__timate

A small technicalteam led by a Project

Manager empowered with complete

responsibilityand anthofityto execute all

aspe_ of the project, develops a

conceptual solution that satisfies the

customer, s technical,cost and schedule

desires. This is the point where the

system engineering practice of designing

a system that is good enough rather than

Attempti. "ng to find the perfect or optimal

solution is employed. The goal at this

stage of the system definition is, to

develop at least one acceptable conceptual

solution and to identify any possible

show-stoppers. This solution includes a

functional block diagram, a ROM cost

estimate, a ROM schedule and a list of

assumptions, key cost and risk drivers

and other project level constraints and

guidelines. In arriving at this solution,

lead engineers use a design-to-cost

approach without identifying any

contingency.

3. Generate Bottom-Up Cost

Estimate

Next, the Project-derived preliminary

concept definition as de_ribed in step 2

above is presented to skill center line

supervisors in APL's

organization. The line supervisors,

working with their technical staffs, then

generate a separate detailed cost estimate

for each technical area. In generatin__g

their cost estimates, the line supervisors

and the technical staff draw heavily upon

their past experience and upon our 40

year historical space systems performance

data base as they analyze the preliminary

concept. The most recent history carries

the most weight. Comparisons are made

between the current mission requirements

and those of previous space missions

conducted at APL. Analysis of these

differences is a critical component in

establishing initial cost estimates and in

establishing mission feasibility. Each of

these cost estimates is then reviewed

separately with the project staff to ensure

consistencywith the project direction and

to ensure the lowest possible cost

estimate has been obtained. The Project

Manager then sums all cost center

estimates to obtain a bottom-up project

cost estimate.

4. Generate Top-Down Cost

Estimate

In a manner similar to the bottom-up cost

estimation process described above, an

independent estimate is generated by a

few senior technical staff members and a

fewconceptseniordefinition managers.along Using with _e p_

level consu'aints and guidelines and using

the relevant aspects of our historical data

base, they generate a separate cost

estimate. This estimate is called a topdown cost estimate. It is generated by the

top level managers and technical staff

having analyzed the proposed space

E-71
TABLE 1

PROGRAM

MAGSAT

_AMPTE

GEOSAT.A

POLAR BEAR

DELTA 180

DELTA 181

DELTA 183

NEAR

RECENT JHU/APL SPACECRAFT BUS COST HISTORY

SPONSOR

NASA

NASA

NAVY

USAF

SDIO

SO_

SDK)

NASAJHQ

WEIGHT

BUS (LBS)

349

343

1198

208

713

2384

558

946

LAUNCH

DATE

1979

1984

1985

1986

1986

1988

1989

1996

PR_

DURATION

IMONTHS)

24

36

37

24

12

17

13

26

RELATIVE

COST*

1.3

1.3

1.6

0.9

1.0

5.6

1.1

3.8

* Cost in constant year dollars relative to cost of Delta 180 (as 1.0) at project

completion (Launch +30 days)

"* Percent cost growth from initial cost estimate at start of Phase C/D

COST

GROWTH**

%

8

0

7

-5

-3

-1

8

-3

Table 2

Spacecraft Bus Cost by WBS Element. (% of Total Bus. Cost)

WBS COST CENTER

Program Management & System Eng.

Mechanical

AMPTE GEOSAT POLAR BEAR

i|

12 12 13

NEAR

4

11 9 11 6

Telecommunications/RF 13 14 19 25

Power & Thermal 13 15 13 8

Attitude & Propulsion 13 13 12 30

Doppler Beacon 4

Ground Support Equipment 7 9 3 11

Integration & Test 17 14 20 4

Reliability & Quality Assurance 14 10 9 12

Figure 3 Spacecraft Relative Cost vs. Weight

6,0

5.0

_o4-0

03.0

2.0 •

_.1,0 • •

0.0 I ' I I I

0 500 1000 1500 2000 2500

Weight (LBS)

E-72
months, and relative costs vai'ying from

one to about six.

Table 2 shows the cost breakdown for

four previous APL missions (AMPTE,

GEOSAT, POLAR BEAR and NEAR)

according to the WBS mentioned earlier.

The data are presented as a percentage of

the total spacecraft bus cost. While

instructive, it is clear from examination of

this data that additional information is

required to interpret the obvious

variances. For example, NEAR is the

only deep space mission listed in Table 2.

Consequently, the NEAR propulsion,

attitude, telecommunication and RF

mission requirements were unique among

this set of APL missions and represent

significant cost drivers relative to our

previous experience. This situation is an

example of how mission unique

requirements or complexities can

dominate total mission costs and can

render useless cost estimates derived

from overly simple cost estimating

formulas.

Shown in Figure 3, is a graph of

spacecraft bus relative cost versus

weight. Again, while instructive, there

can be significant variances from the

indicated trendline for a SlX_ific mission.

It is clear, however, that there is a definite

dependence of spacecraft bus cost on the

bus weight.

COST PERFORMANCE RESULTS

Included in Table 1 is the cost growth of

various Projects experienced during

spacecraft development through launch

plus 30 days. Cost growth is defined as

the difference between the initial cost

estimate as deteamined by the

methodology described above and the

actual inctm_ cost following launch plus

30 days. This cost growth is used as a

parameter to characterize our cost

performance on each Project. As can be

seen, within the experience base listed in

Table 1 and by applying the methodology

described in this paper, APL has a very

good demonstrated performance record.

We are as likely to return money to the

customer as we are to ask for ad/litional

money. The range of cost growth is

between -5% and + 8%.

GENERAL COMMENTS

CONCLUSIONS

AND

In developing space science" mission

hardware and software in today's

environment of limited funding, the most

important element to focus on is the final

cost at project completion, defined here as

the time period ending at launch plus 30

days. Initial cost estimates matter only if

costs can be controlled during project

execution allowing favorable actual costs

to be realized at project completion. We

have tried to highlight the key features

associated with APL's spacecraft

development history extending over 40

years. These feature are: (1) recognizing

that the customer and the performing

contractor's organizational unit, as well

as the project goals effect initial cost

estimates and the ability to control cost,

(2) having a process in place that

supports the generation of accurate initial

project cost estimates and finally, (3)

being able to control cost during project

execution leading to minimal overall

project cost growth. Key parameters to

note are simple hardware and software

systems designs, favorable business

policies, procedures and practicesatboth

the executing contractor and the customer

and a focus and dedication by the project

staff, the executing contractor and the

customer to delivering specified technical

products or schedule and within cost in

the face of a changing environment.

Specific details on these topics have been

presented reflecting APL's extremely

good performance in the past 20 years of

conducting space science missions for

several govemme, nt agencies.

REFERENC'ES

[I] The Marshall Space Flight

Center's NASA cost model (NASCOM)

database, Vol. 1, Executive Summary,

Ver. 2.1, Applied Research, Inc. Dec.

1993.

E-73
mission at a gross level concentrating on

analysis of the similarities and differences

with previous missions conducted at

APL. It can occasionally result in

management decisions to make major

changes in the concept or approach and

require a revised step three. This

approach could be characterized as

looking at the forest rather than looking at

the trees.

5. Generate Initial APL Cost

Estimate

The final step in APL's cost estimating

methodology is accomplished in a short

meeting between the Project staff, the

skill center supervisors and a few of the

senior technical staff members and

managers. The purpose of this mee_g

is to determine APL's official cost

estimate by reconeiling differences among

the Project generated ROM cost estimate,

the bottom-up cost estimate and the topdown cost estimate. This estimate may

be adjusted by management, depending

on the requirements of the customer. The

estimate obtained from this process is

called our initial Project cost estimate.

Cost Control

As the customer, the APL organizational

unit and the Project staff execute the

program, they each have a role to play in

controlling costs. Having set the project

technical, cost and schedule goals, they

must never lose focus on achieving them

in light of the coustanfly changing internal

and external environments. Open and

frequent communications with the

customer are essential to controlling

costs. The customer must keep the funds

flowing into the project and resist the

_on to involve other organizations

that would provide tmn_ and

unproductive oversight.

For APL's management part, we

empower a Project Manager to deliver the

desired products on time, within budget

and to satisfy all technical requirements.

The Project Manager is completely

responsible for all deliverables and is

given the corresponding authority to

exercise this responsibility. This

situation allows for quick and low cost

solutions to problems by solving them

on-the-floor. Additionally, a reliability

and quality assurance engineer is

integrated directly into the Project as a

team player. The engineer is empowered

to find ways to "get it done" as reliably

and as economically as possible rather

than operate in an adversarial and usually

more costly role.

Projects usually function as small teams,

holding weekly team meetings, often in

the presence of the customer.

Managen_nt of technical subcontracts

and Government Furnished Equipment

(GFE) require special attention in order to

manage costs. They are tightly monitored

by a single APL technical person and

interfaces are designed to be as simple as

possible to minimize problems during

integration. This approach reduces the

risk of cost overruns and helps assure

that projected costs can be achieved.

APL HISTORICAL DATA BASE

In APL's forty year history of developing

space mission system hardware and

software, we have delivered 55 spacecraft

and over I00 instruments with a

reliability record exceeding 95%. Our

most recent history is summarized in

Table 1 which lists spacecraft data for the

last twenty years. This time period was

chosen as being the most relevant to

today's missions. The data are

parameterized according to a few key cost

drivers. These are program sponsor,

project duration through launch plus 30

days, spacecraft weight, and spacecraft

relative cost at completion in constant

year dollars. Spacecraft currently under

development at APL will be added to this

table after they have been launched. As

the data indicate, our recent exporience

extends over several government

customers (NASA, USN, SDIO/BMIX),

USAF), with spacecraft weights varying

from about 200 pounds to about 2,500

pounds, development schedules through

launch plus 30 days of 12 months to 36

E-74
[2] Burgess, E. L., N. Y. Lao, D. A.

Bearden, S_/-sate///t_ cost-estimating

relationships, 9th Annual AIAMUtah

State University Conference on Small

Satellites, Sept. 18-21, 1995.

[3] Pisacane, V. L., R. C. Moore,

Fundamentals of space systems, Oxford,

1994.

[4] Wertz, J. R., W. J. Larson,

Space mission analysis and design,

Kluwer Academic Publishers, 1991.

[5] Peter, J. J., Use of engineering

development life cycle model for

determining weighted progress, In

Control, Vol. 3, No. 2, Performance

Management Association, March, 1991.

[6] Augustine, N. R. Augustine's

laws, Penguin, Revised Ed., 1986.

E-75
JDAM- The Value of Acquisition Streamlining

PURPOSE:

The purpose of this paper is to briefly describe some of the successful JDAM acquisition reform

approaches applied over the past 18 months. Specifically, this paper presents some of the contributors to

getting the price from $40,000 dollars per round m the 40,000 unit, to McDonnell's price of $18,000 per

round at the very first unit.

BACKGROUND:

The JDAM program has evolved substantially since the Milestone I DAB. The primary reasons have been

the designation of the JDAM as a Defense Acquisition Pilot Program in the 1994 Federal Acquisition

Streamlining Act (FASA) and the extraordinary support from the Undersecretary of Defense" for

Acquisition. This support has more than any other factor, allowed us to break down traditional barriers by

asking "why does it have to be this way---why can't we accomplish the task using commercial

approaches?" Our focus is to reduce acquisition management costs--costs related specifically to the way

the Government does business rather than to the nature of the product the Government is buying,

PROGRAM DESCRIPTION:

* Joint Air Force/Navy program to develop a very affordable, adverse weather, and accurate guidance

kit for 1000 and 2000 pound bombs currently in inventory

• INS/GPS guidance kit attaches to the bomb and uses controlled tail fin movements to direct bomb to

the target

• JDAMisplannedforintegrafiononB-1,B-2, B-52, F-15,F-16, F/A-18, F-22andAV-8

• Designated as DoD Pilot Program by Federal Acquisition Streamlining Act (FASA) of 1994

PROGRAM STATUS:

• Two 18 month EMD- 1 contracts awarded to Lockheed Martin and McDonnell Douglas in Apr 94

• _ 1 emphasis on lowering average unit cost and manufacturing risk

• Downselected to one contractor (McDonnell Douglas) occurred on 11 Oft 95

• EMD-2 will complete development and emphasis development and operational testing

• EMD-2 is scheduled for 40 months; Lot I Production begins delivery in FY98

• Contract types

• Cust plus fixed fee for FAVID-1

• Cost plus award fee for EMD-2

• Firm fixed price for initial production

E-76
HOW HAS JDAM STREAMLINED ITS ACQUISITION PROCESS?

The number one reason JDAM has been able to successfully implement acquisRion streamlining is

A'f'YITUDEI

• JDAM asks "why can't we do it another way'" a "cheaper/fz_r way?" instead of complying with "the

way its always done" EVERY RESULT IS A SYMPTOM OF THIS ATYITUDE

• JDAM compares itself with an analogy of an individual building a house, how would that individual

react?

• Most of the initiatives implemented by JDAM were in the realm of common sense, i.e., things that

could always be done before but the environment and culture were not right to allow execution of the

ideas

• The number one objective of JDAM's first development phase was acquisition reform

• The contractor is responsible for his design

• The number one objective of JDAM's second development phase is to help MDA beat schedule

• The number one objective of JDAM's production is to help the contractor maximize his profit,

GIVEN

• MDA delivers JDAM at the price he has projected to the Government

Contracts

Selecting the "Right" Contractor

• Selection of smart contractors with known commercial buying experience

• Developed a "Rolling Downselect" process

• Provided feedback along the way - helped the contractor work his weak areas

• Used common sense - A parent wants his student to receive continuous feedback to ensure student

does his/her best

• Helped ensure the Government would have a "tough" final decision between two viable contractors

• Reduced timeline to final downselect

• Reduced chance of protest - contractors knew where they stood

Statutory- Regulatory Relief

• Statutory - the most significant relief provided by FASA was allowance to use commercial practices

• From a regulatory standpoint, JDAM requested waivers for 25 FARs and 13 DFARS; OSD granted

wa/ver approval for 15 FARs and II DFARs

• No n_ntl_l use of military specs or standards, although the conUactors elected to use a few rail

where no commercial equivalents were available interface with the Warfighter

E-77
Oversight

• Streamlined oversight of both the program and the contractors including little audit and DCMC

involvement with the contractors

• Minimal--Contractor responsible for his desigu--Government works the interfaces

Warranty

• Use of a commercial type warranty which guarantees the contractor will repair or replace any

defective item; any warranty costs are included in the price

Dispute Resolution

• Avoid the costs and timeline normally associated with resolving Government/Contractor conflicts

• Allowance for Alternate Dispute Resolution instead of the normal legal means of solving disputes

(EMD 2 and later)

• Impanel independent 3 person team (1 nominated by KR, I by Government, 1 selected by other two)

Management Process

Personnel

• Critical use of Integrated Program Teams (IPTs) including the actual inclusion of government people

as part of the contractor's team at their facilities

• JOB I for each Government IPT during I" phase of JDAM EMD was to help his contractor WIN

• Decreased overhead accounts---use multi-faceted direct charge talent both in Government and MDA

Manufacturing

• Heavy use of Manufacturing Development Initiative, meaning concentration of the actual

manufacture of hardware very early in the development cycle and the allowance of tradeoffs within

the design cycle for manufacturing and cost

• Emphasis on a long term relationship with the primes and their subcontractors

• Management at a higher level---the systems specification only, no sub-tier specs

• Contractor formats for critical design reviews and PRRs---held during 1't 18 months

Data/Conflg

• Significantly reduced data requirements---support system safety and flight clearance organizations

mainly

• Use of earned value reporting from the contractors in their own format

• At the beginning of EMD-2, configuration control is being given to the contractor for all Class II

changes

E-78
TECHNICAL Interface with the Warflghter

• TRADITIONAL: An unfortunate aspect of military acquisition is "requirements creep." When the

dollars aren't yours, human nature is to want more. The Air Force recognized this trend several years

ago and instituted a "4-Star Summit" process 4-Star Summit helped control this costly aspect, but

extremely painful and demands huge resources.

• JDAM: JDAM wanted to capture the benefits of the 4-Star Summit process and eliminate the burden

• JDAM instituted an "0-6 Requirements Review Council" consisting of 0-6s from ACC, N88, PMA201, and the JDAM program orifice

• JDAM contractors told to b_g any requirements/$ trades or questions for clarification to council

• RESULTS: Both JDAM contractors took advantage of this forum---reducing unit costs of their design

• EXAMPLES:

• McDonnell asked for reduction of low altitude captive carriage time in "worst case ops scenario"

• Warfighters approved and plastic encapsulated parts became baseline---Saved $535 per

• Lockheed Martin presented a trade offering $250 per round savings for trades using DSU-33 fuze

Aircraft Interface

• TRADITIONAL: Smart Weapons create their own interface with their aircraft using MI].,-STD- 1760

• Hire a weapon contractor to create their own implementation with each aircraft

• Time consuming, and expensive - new learning curve each time

• JDAM: Hired some aircraft interface support during JDAM original source selection to create

strawman

• Capitalized on TSSAM and JSOW existing interfaces with thc_e aircra_ in common with JDAM

• Supported commonality working group with JSOW and WCMD programs

• Reduced costs to prepare and execute aircraft interface

• Test Warranty motivates contractor to move reliability growth to the left providing a production

representative weapon earlier in program

• Characterization of weapon and any problems earlier in the program

• True combined DT/OT

• Worked with AForI'EC and COMOPTEVFOR to achieve extremely close working relationship

• OT will help plan DT mimions---DT data will feed and help reduce OF mrties---less cost

• "N_ with related programs (e.& Joint Programmable Fuze) to _k testing

• Results in reduced costs for JDAM and the other programs

• Learn from past _ more operationally representative missions early

• Get away from rote traditional DT test cards early---wring the weapon out in warfighter environment

• Use warfighter to provide "rapid Wototyping" feedback while developer has chance to fix

• Helps warflghter develop tactics and helps developer streamline training

• Base testing requirements on whole program, not piece parts

• AF using Navy FA-18 results to reduce (not e"luninate) required sorties on AF aircraft

• Tests feeds composite, integrated test matrix

• Warfighter requirements in JORD---DT testing normally oriented to specification

E-79
Motivation

• Milestone billings will also be allowed to improve the cash flow of the contractor in later phases of

the program

• Innovative use of "price based negotiation": allows the use of price rather than cost as a basis for

negotiation; also allows the use of rewards and punishment for a contractor's commitment for first

five lots of production price

• "Carrots and Sticks"

• Carrots (Contractor price meets or is better than their commitments to the Government)

• Contractor submit a price-no need for cost data as long as contractor meets price commitments

• Contractor has configuration control--as Long as changed does not affect system specification

• No in-plant, in-process oversight of prime or his subs

• Government will heLp prime reduce his costs--not his price

• Contractor will receive an incentive fee if lfis units' reliability is met or bettered

• Long term commitment_rnment will not recompete if contractor meets his commitments

• Sticks (Contractor price is greater than the commitments made to the Government)

• Contractor must deliver certified cost and pricing data

• Configuration control reverts to the Government

• Contractor must develop a qualified 2"_ source at no cost to the Government

• Government may place in-plant, in-process oversight

• Incentive fee goes away

"Measures of Effectiveness"

MIL STDS/SPECS

SOW PAGES

CDRLS

PERF SPEC

WARRANTY

AUPR ($93)

SPO S 173_

DEVELOPMENTS

DEVELOPMENT

TIME

PRODUCTION

TIME

87

137

243

YES YES

5YRS 5YRS

68K 48K

70 70

$380M

46 MOS

15 YRS

8O 0

100 10

250 85

YES

PRODUCrION 0 0 0

SAVINGS

DEVELOPMENT

PRODUCTION NO LIMIT

PROPOSAL

DOWNSELECT 3 MOS

DURATION

20 YRS

40K

70

SOO-2

29

YES

20YRS

<20K

63(96/1_-44) (96/4)

$310M

30 MOS

IIYRS

$1.5B

15 PAGES

6 WEEKS

E-80
• JDAM combined DT/OT considers JORD early and up front

• Eliminate oversight and checking the checkers

• Contractor is responsible for his design---Govermnent fly's the aircraft, but

• Contractor is customer for results

• Emphasize "margin "testing.up front

• Find out where design breaks, not just that design performs to spec

• Significant testing to break design and characterize performance early in program

• Treat DT/OT and OSD as partners

• Developed partnership with all TEMP organizations

• TEMP is continually updated for program changes--not signed and ignored until next milestone

• SUMMARY: The first phase of the JDAM development program met with many successes and some

failures--some ideas did not pan out. However, the concepts that did work fostered an environment

for the contractors to reduce costs. We've seen for the first time contractors who never before would

bid a Government contract submit bids. We've seen significant prime/subcontractor teaming

relationships in a "selfless manner" where a member may propose a change which decreases his

profit, but reduces system cost and price.

We are not able to quantify a cost savings with each of the initiatives and new ways of doing business that

JDAM pursued. In the aggregate though, the change in attitude, the change in paradigm, has allowed the

JDAM team to reduce unit cost by approximately 60% below the JORD requirement. There also have been

attendant savings in development which allowed the JDAM program to free FY97 and 98 funding to

accelerate the JDAM program and provide funding for earlier capability on the B-1 and B-52 bomber

aircraft.

In summary, JDAM is experiencing lower development costs/faster schedules, lower production

costs/faster schedules, and lower unit costs.

There remain some barriers to acquisition reform---some of which are outlined below.

Barriers to Acquisition Reform

• Chief among these barriers are:

• Continued functional (engineers, procurement specialists, etc) resistance to change

• Risk adversity, mistrust by both the government and contractors to wide scale changes

• Substitution of templates for innovation; people want a checklist on how to do a task--quick/easy

• Some people don't recognize the need to change attitudes--to tailor new ideas to their own program

"personalities'"

• Too much consensus decision making and conservatism, especially on the part of the government

• Increased inspection/audit for Pilot Programs since it is "new'"

• Strong senior DoD leadership, but continued mistrust by staffs

E-81