CORRECTIVE ACTION PLAN

In Response to Recent Programmatic Performance and 

NASA’s Designation on GAO’s High Risk List

14 December 2018
1 | Page

Contents

Purpose .........................................................................................................................................................2

Approach.......................................................................................................................................................2

NASA Legacy of Programmatic Improvements: the 2007 Corrective Action Plan........................................3

Corrective Action Plan Overview ..................................................................................................................4

Ownership and Responsibilities................................................................................................................4

Initiatives...................................................................................................................................................4

Areas of Emphasis.....................................................................................................................................5

Severability................................................................................................................................................5

Progress Tracking and Reporting ..............................................................................................................5

Corrective Action Plan Update Schedule..................................................................................................5

Initiatives to Implement................................................................................................................................6

Enhance Earned Value Management Implementation.............................................................................7

Improve HEOMD Portfolio Insight and Status........................................................................................11

Implement Programmatic (PP&C) Training Curriculum..........................................................................15

Include Original Agency Baseline Commitments for Performance-Driven Re-baselined Projects.........19

Enhance Annual Strategic Review Process.............................................................................................22

Create Technology Readiness Assessment (TRA) Best Practices Document..........................................27

Update Probabilistic Programmatic Policy .............................................................................................29

Initiative to Pilot..........................................................................................................................................32

Create a Schedule Repository .................................................................................................................33

Initiative to Research ..................................................................................................................................36

Enhance Implementation Indicators for Trends and Projections...........................................................37

Areas of Emphasis.......................................................................................................................................40

Area of Emphasis: Improve NASA’s Governance of Strategic Acquisitions............................................41

Area of Emphasis: Risk Assessment, Requirements, and Concept Definition Early In the Formulation 

Phase.......................................................................................................................................................42

Area of Emphasis: Contractually Incentivizing High Performance..........................................................44

Appendices..................................................................................................................................................45

Appendix A: NASA Associate Administrator Memorandum Initiating CAP Effort......................................46

Appendix B: Working Group and Steering Committee Membership .........................................................48

Appendix C: Development Timeline ...........................................................................................................49

Appendix D: Acronyms List .........................................................................................................................50
2 | Page

Purpose

This Corrective Action Plan (hereinafter referred to as the “Plan,” or the “CAP”) encompasses a 

collection of specific initiatives and areas of emphasis that the National Aeronautics and Space 

Administration (the “Agency,” or “NASA”) commits to pursuing as it matures Agency program and 

project management policies and processes, as well as related surveillance of contractors through 

appropriate insight and oversight. NASA developed this Plan in response to recent challenges in cost and 

schedule growth experienced by several of the Agency’s highest profile missions, as well as the 

continued inclusion of NASA’s acquisition practices in the Government Accountability Office’s (GAO) 

biennial High Risk Report. The Plan is also developed in support of the Agency’s steadfast commitment 

to good governance and effective stewardship of the resources entrusted to it. The overall goal of the 

Plan is to strengthen the Agency’s cutting-edge program and project management efforts across the 

board and to improve transparency for NASA’s stakeholders. 

Approach

GAO originally designated NASA’s acquisition management as a “high risk” area in its inaugural High-Risk 

List released in 1990, citing what was at the time considered a history of persistent cost growth and 

schedule delays in the majority of the Agency’s major products. In 2007, NASA established a Corrective 

Action Plan consisting of five broad focus areas and seven tactical initiatives that provided an Agencywide coordinated approach to improve NASA’s program and project management activities. The 

initiatives included in the 2007 Plan were all successfully closed by the end of 2012, and GAO has 

acknowledged that considerable progress toward strengthening and integrating NASA’s acquisition 

management functions resulted from those efforts. Both the 2015 and 2017 High Risk Reports credited 

NASA with fully meeting three of the five criteria for removal from the High Risk List (leadership 

commitment, action plan, and monitoring), as well as partially meeting the remaining two criteria 

(capacity and demonstrated progress).

In September 2018, Agency senior leadership determined that a new CAP was necessary to continue 

driving improvements in NASA’s program and project management policies and processes. NASA’s

Associate Administrator (AA) and the Chief Financial Officer (CFO) jointly issued a memorandum to this 

effect on September 6, 2018. The memo required that a new Plan be in place by the end of the 2018 

calendar year, and established a working group comprised of relevant experts from across the Agency to 

develop the initiatives. It also called for the creation of a steering committee to provide guidance to the 

working group at key milestones in the development process. Finally, top-level direction for the new 

Plan would reside with the NASA AA (in coordination with the CFO), with official approvals routed 

through the Agency Program Management Council (APMC).

The working group held a kick-off on September 19, 2018, and met or communicated daily throughout 

the development of the Plan. The working group considered a variety of inputs during the formulation of 

each of the individual initiatives that comprise the Plan. These inputs include, but are not limited to, 

previous GAO High Risk Reports, GAO’s 2018 Priority Recommendations Letter, reports issued by GAO 

during its annual programmatic reviews of NASA’s major projects, as well as internal analyses conducted 

by the Agency. Direction from NASA senior leadership, the advice of subject matter experts drawn from 

across NASA, and feedback from GAO were also considered. Agency-wide stakeholder review was 
3 | Page

conducted via the APMC community during November 2018, and the final document was approved at 

the December 6, 2018 APMC meeting. 

Please see Appendix B for working group and steering committee membership. Please see Appendix C 

for a full development timeline that includes all APMC and steering committee meetings, as well as 

iterative touchpoints with GAO. Appendix D includes a list of spelled-out acronyms used in this 

document.

NASA Legacy of Programmatic Improvements: the 2007 Corrective 

Action Plan

NASA sees excellence in program and project management as a core capability and critical if the Agency 

is to successfully develop and operate technologies and systems for the human exploration of deep 

space; execute robust programs of robotic missions to monitor the Sun and Earth, explore the planets of 

our solar system, and observe the universe beyond; and continue to make aviation safer, more efficient, 

and more environmentally friendly. To that end, the Agency continually assesses how to manage 

projects and prepare people to lead. As a result, NASA’s project management and oversight practices 

have seen significant improvement since the Agency was first added to GAO’s High Risk List in 1990. 

A key milestone in the maturity of NASA’s programmatic discipline was the Corrective Action Plan 

developed in 2007. The 2007 plan contained seven initiatives to address potential shortcomings in 

NASA’s acquisition management practices. Those initiatives were: (1) Program/Project Requirements 

and Implementation Practices; (2) Agency Strategic Acquisition Approach; (3) Contractor Cost 

Performance Monitoring; (4) Project Management Training and Development; (5) Improve Life-Cycle 

Cost/Schedule Management Processes; (6) Integrated Enterprise Management Program (IEMP) Process 

Improvement; and, (7) Procurement Processes and Policies. Six of these seven initiatives were 

operationalized by 2012 through the introduction of new requirements, policies, procedures, training, 

and other tools to improve how we manage our major acquisitions and ensure our workforce has the 

necessary associated tools. In 2014, NASA declared that the one outstanding initiative, Contractor Cost 

Performance Monitoring, was closed. This initiative was originally designed to improve the availability of 

contractor data to support performance monitoring of programs and projects. The initiative would have 

been accomplished through the use of enhanced business systems and changes to the contractor cost 

reporting process. NASA performed analyses at that time to identify gaps in the existing key business 

systems and concepts and courses of action that could be implemented to address those gaps. As a 

result of this analysis, NASA and GAO agreed to replace the original objective, and instead instituted 

several process improvements designed to achieve greater insight into project performance, including 

contractor cost performance. 

These operationalized initiatives have yielded the desired results for NASA’s small and medium-class 

missions, though the Agency recognizes that there is still work to be done. In particular, NASA needs to 

do better managing our larger, more complex projects, which typically involve the development of a 

significant number of new technologies, greater risk, and early estimation challenges. The 2018 

Corrective Action Plan provided below intends to build upon the successful legacy of its 2007 

predecessor, ensuring that NASA continues to enhance its programmatic rigor while pushing forward 

with the activities that will be necessary to initiate a bold new era of discovery.
4 | Page

Corrective Action Plan Overview

Ownership and Responsibilities

The NASA AA assumes ownership of the CAP. The Office of the Chief Financial Officer, Strategic 

Investments Division (OCFO SID), is responsible for maintaining CAP documentation, tracking and 

reporting progress against the CAP on an annual basis, and conducting any process updates for 

subsequent iterations on the CAP. Lead Executives or Lead Organizations as described in the various 

initiatives in the CAP are responsible for executing upon the initiatives as written, and reporting progress 

to either OCFO SID or other specific forums as described. The Supporting Organizations, where listed, 

will support the identified Lead Executives or Organizations in execution of the initiatives, as necessary. 

Initiatives

The Corrective Action Plan is comprised of a set of initiatives evaluated by the working group and crossAgency stakeholders to provide value for Agency acquisition management improvements. The initiatives 

are categorized by the following actionable characteristics:

• Implement: Initiatives that NASA has determined should proceed and become part of regular 

Agency business cadence. Any actions taken to support execution of the described initiatives will 

follow all established Agency control and oversight boards, as applicable, to ensure no 

unintended consequences are experienced.

• Pilot: Initiatives that NASA has determined show promise to provide value related to Agency 

acquisition management, but will initially be executed to a limited degree in scope and time 

until the Agency assesses and reaffirms continued execution.

• Research: Initiatives that are less conceptually mature but warrant dedicated effort to explore 

and develop with respect to generating value for Agency acquisition management.

Each initiative in the CAP includes the following sections:

• Lead Executive/Organization: The individual or office responsible for leading the initiative as 

described, and periodically reporting progress to OCFO SID for internal and external 

communications and for coordination purposes.

• Supporting Organization(s): The organization(s) required to support the Lead 

Executive/Organization execute the initiative as described. While the organizations listed are 

necessary and required to support the initiative, it is not necessarily an exhaustive list, as the 

Lead Executive/Organization may call on other Organizations to support.

• Initiative Description: A brief high-level summary of the initiative.

• Background/Current State: A description of the status quo and, as necessary, an overview of 

relevant background information and any policies, procedures, constraints, or other areas 

influencing the status quo.

• Expected Benefits: A review of the expected payoff from conducting and completing the 

described initiative.

• Recent Accomplishments: Where applicable, a review of recent progress and accomplishments 

with regard to the described initiative. Some initiatives already have momentum, which will be 

reflected here. Others, such as research initiatives for new ideas, will not have content for this 

section.
5 | Page

• Planned Next Steps: A high-level description of the planned progress for the initiative to occur 

over a period ranging up to two years.

• Output and Outcome Metrics: Where applicable, a list of appropriate methods of measurement 

to track progress and effectiveness of the described initiative.

• Interdependencies: Where applicable, a description of key dependencies that necessitate 

coordination or cognizance in support of initiative success.

• Impediments and Challenges: A brief description of possible pitfalls, risks, impediments, and 

challenges that can be reasonably expected to occur during execution of the described initiative.

• Required Resources: A preliminary assessment of possible resource requirements for successful 

execution of the described initiative. This does not represent committed resources and does not 

reflect a refined estimate of resources. Any funds required to execute any initiatives will 

proceed through the regular budget formulation processes for Agency approval.

Areas of Emphasis

NASA will emphasize adherence to current policies and practices in certain areas listed in this section, 

and encourage improvements that better position the Agency to manage cost and schedule 

performance. In contrast to the initiatives in the CAP, Areas of Emphasis do not currently have direct, 

measureable plans of action for the purposes of the Corrective Action Plan, but nevertheless are 

deemed critical to improving Agency performance. 

Severability

The initiatives contained herein are interrelated with respect to their connection to improving Agency 

acquisition management, but are not mutually dependent on one another for execution purposes, 

unless otherwise identified. As such, if the Agency should determine that any individual initiative be

removed from the Corrective Action Plan for any reason, the Plan and remaining initiatives will remain in 

effect. 

Progress Tracking and Reporting

Each initiative in the CAP includes planned next steps and metrics, where applicable. The Lead 

Organization(s) cited in the CAP will pursue actions as described. A subset of initiatives include specific 

forums for reporting progress or deliverables (e.g., APMC or Business Performance Review (BPR)). For all 

efforts in the CAP, OCFO SID will conduct an annual checkpoint to measure progress against the CAP. For 

odd-numbered years, the progress checkpoint will occur in the summer months. For even-numbered 

years, the progress checkpoint will be folded into the CAP update (see below). OCFO SID will provide the 

overall progress and status update to the AA. OCFO SID will also share and discuss progress with GAO 

annually at a minimum, and more often when applicable.

Corrective Action Plan Update Schedule

NASA will keep this Corrective Action Plan current and up to date until GAO removes the High Risk 

designation for the Agency. The update process in which initiatives and/or Areas of Emphasis are added, 

revised, or resolved will occur in the approximate period of May to September of even-numbered years. 

This timeframe allows an informed GAO consideration of any changes made to the CAP, and supports

the GAO’s timeline for preparation of the biennial publication of the High Risk Report 

(~January/February of odd-numbered years). The AA will retain the authority to make changes and 

revisions to the CAP at any time.
6 | Page

Initiatives to Implement
7 | Page

Enhance Earned Value Management Implementation

Lead Executive/Organization

Office of the Chief Financial Officer

Supporting Organization(s)

Mission Directorates; Centers; Programs and Projects

Initiative Description

NASA will improve and strengthen the Earned Value Management (EVM) discipline, and work to foster a 

culture at NASA where EVM is accepted by Programs and Projects and embraced by managers and 

employees.

Background/Current State

Earned Value Management (EVM) is an integrated management control system for assessing, 

understanding, and quantifying what a contractor or field activity is achieving with program dollars. EVM 

is a discipline with established industry-adopted standards outlined in EIA-748 that integrates technical, 

cost, schedule, and risk management. It requires an established performance measurement baseline 

(PMB) which allows for objective assessment and quantification of current project performance as well 

as helps to predict future performance based on trends. EVM provides project management with 

objective, accurate, and timely data for effective decision-making. 

EVM system compliance and use is required on all acquisitions for development designated as major in 

accordance with the Office of Management and Budget (OMB) Circular A-11 and the Capital 

Programming Guide. At NASA, EVM is required for development or production contracts and 

subcontracts (including those for flight systems, ground systems, and institutional requirements (facility, 

information technology, investment, etc.)) valued at or greater than $20 million. EVM is required on 

NASA spaceflight projects with a lifecycle cost (LCC) of $250 million or greater. Science Mission 

Directorate (SMD) Mission Risk Class-D space flight projects and contracts with a LCC of up to $150

million (not including launch costs) have an approved deviation from the EVM requirements of NFS 

1834.201. Class-D missions with a LCC of up to $150 million (not including launch costs) should use the 

processes per the SMD Class-D Tailoring/Streamlining Policy. The Aeronautics Research Mission 

Directorate (ARMD) has also recently tailored governance to require EVM on large crewed flight 

projects.

EVM reporting begins no later than 60 days post Key Decision Point-C (KDP-C). The primary 

consideration for EVM applicability is the nature of the work, associated risks, and the value of the 

effort. EVM is not recommended on Firm Fixed Price contracts or contracts that are exclusively Level of 

Effort (LOE).

Expected Benefits

Improved EVM will:

• Encourage rigorous upfront planning to establish a performance measurement baseline to assess 

NASA projects performance for cost, schedule and Estimate at Completions (EACs).

• Provide project managers with objective, accurate, and timely data for effective decision-making.

• Help the Agency progress toward removal from the GAO’s High Risk List by enabling NASA programs 

and projects to identify and address issues and take corrective actions.
8 | Page

• Enable a central repository that provides relevant historical data for NASA to develop better cost 

and schedule estimates for NASA’s future programs and projects.

Recent Accomplishments

While areas for improvement still exist, the Agency has taken concrete steps to advance the EVM 

capability, including:

• Rolled out NASA’s EVM Capability to Marshall Space Flight Center (MSFC), Kennedy Space 

Center (KSC), Goddard Space Flight Center (GSFC), and Johnson Space Center (JSC), comprising 

98 percent of space flight project EVM requirements.

• Completed the EVM skills gap survey, developed an EVM training plan, and updated existing 

courses. Worked with the Program, Planning, and Control (PP&C) Working Group to include 

EVM training as part of the Agency’s overall PP&C training curriculum.

• Developed an EVM change management plan that was approved by the EVM Steering 

Committee.

• Strengthened EVM surveillance to improve reliability of data and management use by updating

the NASA/SP-2018-599 EVM Implementation Handbook, developing EVM surveillance job 

aids/tools, and conducting EVM System Validation Reviews where Defense Contract 

Management Agency (DCMA) delegated surveillance is not present.

Planned Next Steps

Mission Directorates (MD) and Centers will provide points of contact to work with the Agency EVM 

Steering Committee to jointly review and assign the following next steps and considerations:

• Roll out EVM Capability to remaining Centers as identified and approved by the EVM Steering 

Committee.

• Emphasize senior management support for implementation of the established EVM change 

management plan. Develop and issue policy per senior management, outlining the new 

requirements and expectations in regards to EVM reporting, surveillance, and data submittals.

• Ensure EVM considerations are included in the ongoing SMD implementation plan for improving 

project management.

• Improve EVM flow-down for contracts by updating training materials and procurement guide.

• Require EVM reporting periodically at the Baseline Performance Review (BPR) for applicable 

projects. At a minimum, require cost and schedule indices and EVM independent EAC 

calculations in these reports.

• Enhance in-house EVM surveillance:

o Conduct EVM Assessments on a cadence negotiated for each project by OCFO, MDs and 

programs/projects.

o Update annual EVM surveillance plan for project surveillance

• Enhance contracted EVM surveillance with DCMA and/or internal NASA resources:

o Quantify existing contracts for DCMA delegations and delegate contracts with a 

remaining period of performance of 2 or more years

o Projects will review DCMA surveillance reports as they are issued, and monitor EVM 

corrective action plans
9 | Page

o Projects that do not delegate EVM System (EVMS) surveillance to DCMA should develop 

an EVMS surveillance plan to include EVM guidelines to be reviewed by contract on a 

cadence negotiated for each project by OCFO, MDs and programs/projects.

 OCFO will identify contracts that require EVMS surveillance annually

 Projects will document EVMS surveillance findings in a report

 Projects will require corrective action plans for discrepancies and track them to 

closure

• Conduct EVM surveillance using NASA resources on other major suppliers, such as JPL, APL, and

(Southwest Research Institute (SwRI), where DCMA does not have an existing presence. 

Evaluate whether all or a subset of projects should perform a monthly EVM data anomaly 

assessment. Require projects to report all EVM data anomalies to Mission Directorates and 

OCFO/SID, and require corrective action plans for resolution of any material issues.

• Require Integrated Program Management Report (IPMR) submittals to a central repository. The 

PP&C Working Group, in consultation with the EVM Steering Committee, will work 

implementation details seeking integration with related efforts where possible to minimize 

burden.

Output and Outcome Metrics

• All applicable contracts use DCMA for contract EVMS surveillance or an equally effective method 

as outlined in the project plan that is consistent with the agency overall surveillance approach.

• All issues identified in the EVM Assessment have been satisfactorily dispositioned with 

corrective action plans within a reasonable time of identification.

• A reduction of EVMS data integrity issues identified by GAO.

• EVM capability is rolled out to all NASA Centers with EVM requirements.

• Status meetings (e.g., BPR) include EVM data where applicable.

Interdependencies

The success of EVM implementation is predicated upon successful adoption and execution by all 

affected projects, programs, and mission directorates. It is also critical that NASA senior leadership 

emphasize the need for EVM performance data and act upon the data when anomalies are identified. 

Impediments & Challenges

• Enforcement of current requirements and guidance is a challenge and calls for senior 

management emphasis and action. OCFO owns the functional responsibility, but projects 

implement the EVM requirement. Currently, the EVM community is splintered because the 

resources are owned by projects, which have varying approaches for implementation that may 

not be consistent with the overall Agency policies and goals.

• There is a perception that the cost of EVM is too high, which leads projects and programs to 

resist EVM and to request waivers and deviations from flow-down of EVM requirements to 

contractors. 

• EVM is a disciplined project management process that promotes rigorous planning and control, 

and provides objective metrics. Without good PP&C skills, it is difficult to implement EVM. 

Increased training needs to continue across the agency.

• Frequent replanning and resetting of the EVM metrics erode the usefulness of the EVM metrics. 
10 | Page

Required Resources

Demands on the EVM staff within OCFO and at the Centers indicate that the Agency has been underresourced and under-staffed in recent years for EVM implementation and surveillance. To implement 

the steps in this plan, additional resources will be required depending on the Agency’s final strategy. If 

the full EVM surveillance is assumed by the Agency for both total project (in-house) and contracts, it is 

estimated that an additional 10 full-time equivalents (FTE)/work-year equivalents (WYE) would be 

required. If the Agency continues to utilize DCMA for contracts, and focuses more on project (in-house) 

surveillance and increasing EVM analysis at the project level, then approximately three additional 

FTE/WYEs would be required. Finally, to conduct EVM surveillance at contractor specific institutions 

(such as the Jet Propulsion Laboratory (JPL), the Applied Physics Laboratory (APL), and the Southwest 

Research Institute (SwRI)), to continue to conduct EVM assessment in their current state, and to use 

DCMA for contracts, then approximately five additional FTE/WYEs would be required. The CAP Working 

Group (CAPWG) recommends that the level of adequacy for EVM resources be assessed during the next 

logical Planning, Programming, Budgeting, and Execution (PPBE) budget formulation process and that 

the results of that assessment be reflected in the budget. 

NOTE: If the agency were to choose not to use DCMA for EVM surveillance on contracts, the risk would 

be that the current memorandum of understanding (MOU) for surveillance could be rescinded by 

DCMA. This could reduce the likelihood of DCMA conducting these types of reviews in the future 

because they could adjust their workforce based on the reduced scope of work with NASA. Finally, 

DCMA has a stronger presence in the industrial base because the Department of Defense’s budget is 

much larger than NASA’s and DCMA EVMS surveillance findings carry more influence with the industrial 

base than findings by civil agencies carry. 

Notional resource levels (to be reviewed in established NASA budget formulation process):

NASA does all of EVMS 

surveillance including all 

contracts

NASA does EVMS surveillance 

on total project (in-house) and 

JPL, APL, SwRI, etc.

Current State, plus additional 

support to implement CAP plan

+~10 FTE/WYEs +~5 FTE/WYEs +~3 FTE/WYEs
11 | Page

Improve HEOMD Portfolio Insight and Status

Lead Executive/Organization

HEOMD RMO Special Assessments and Analysis Branch

Supporting Organization(s)

HEOMD Programs; OCFO SID

Initiative Description

In an effort to improve the Human Exploration and Operations Mission Directorate’s (HEOMD) internal 

portfolio analysis and planning, the HEOMD Resource Management Office (RMO) is enhancing the 

process for cross-portfolio risk assessments (including technical, schedule, and cost risk), cross-portfolio 

interdependency tracking, acquisition and development status tracking, and performance evaluation 

against HEOMD and Agency goals and objectives. These processes are being unified within the RMO 

assessment and analysis team to enable appropriate level-setting of risks and impacts across the Mission 

Directorate and to support enhanced management actions on emerging issues and contractor 

performance. 

Background/Current State

Each program within the HEOMD portfolio performs and manages cost, schedule, and risk progress and 

analysis. The results of each program’s progress and analysis is communicated to HEOMD leadership 

through formal meetings, such as the HEO Directorate Program Management Council, RMO Quarterly 

Reporting, and other less formal meetings and exchanges (including weekly senior leadership tag-ups

and executive teleconferences). In addition to program-level analysis, broader analysis and review is also

conducted. For example, Exploration Systems Development (ESD) is an enterprise-level organizational 

structure for programmatic and technical integration for the Space Launch System (SLS), Orion, and 

Exploration Ground Systems (EGS) programs. ESD provides in-line and independent review of the SLS, 

Orion and EGS programs regarding integrated technical, cost, schedule, and risk assessment for the 

HEOMD Deputy Associate Administrator for ESD. While program progress and analysis is managed by 

each program with management oversight by HEOMD leadership, there is currently a limited amount of 

systematic HEOMD-wide portfolio analysis and assessment. 

Budget estimation in HEOMD is performed in phases. Initial budget estimation during pre-formulation 

and initial formulation is handled through concept study teams, parametric cost modeling, and other 

analogy based cost modeling. These methods are state-of-the art, but still inherently inexact; they are 

based on system modeling based on sub-characteristics of previous systems, that may or may not have 

eventual similarity to the one-of-a-kind type of development needed in human spaceflight planning. 

Reserve posture, acquisition type, and risk level are additionally factored to modify the base cost 

estimate to fit the projected mission. These initial cost estimates are used in initial budget projections 

for new system developments such the current Gateway and lander program. As a program moves 

through formulation into development, more detailed cost estimating is performed prior to major 

contract initiation. Depending on the type of acquisition model, a detailed vehicle model may be 

generated leading to a detailed bottom up cost model based on specific sub-system and manufacturing 

choices, or requirements definitions may lead to RFPs being used to solicit direct competing proposals to 

inform system cost. In both cases the government estimates are used until specific contractual 

agreements are in place that fully scope the work to be performed and financial agreements put in 

place. Once contractual agreements are established, initial detail planning is completed, and schedule 
12 | Page

baselines are put in place, detailed cost projections and commitments can be formed. For vehicle 

development projects this typically encompasses a single operational flight with potential for 

development flight tests included. For production and operational systems this typically encompasses 

multiple flight sets, missions, or vehicles so manufacturing bases can be established and maintained and 

savings for production quantity can be realized. 

Within ESD, currently both SLS and Orion are in between the development mode and the production 

and operation mode, which is further complicated by having different contractual models for various 

vehicle elements. 

• The SLS Program was congressionally directed to use the existing Space Shuttle Main 

Engines (SSMEs, or generically RS-25s) stockpile as much as possible. This led to a multiphased approach. First, modifying the existing engines and testing necessary new 

components for use in a new vehicle. Second, restarting the extinct production line of RS-25 

Engines so that more than 4 flights of SLS could be performed (16 existing engines, 4 engines 

per flight). And finally, producing engines to sustain flight rate capability of SLS. The SLS 

Program is in the midst of the contract for restarting the production of RS-25s and is 

initiating the follow-on production purchases for RS-25 engines beyond the fifth flight of SLS 

as is needed due to long-lead parts procurement for the complicated engine production. 

This mixed status means cost per flight is not a clean story even in regards to just the corestage engines for SLS, as the contractual environment and content inclusion varies between 

flights 1-4, 5, and 6+. 

• The SLS booster contract is handled separately. For the first 8 flights of SLS, the boosters are 

using partially used systems from shuttle heritage. Booster casings are available through 

flight 8, but remaining subsystem components are mixed between refurbished, new builds, 

and sustaining production. As such, contracts for booster production are released in batches 

of flights sets through flight 8, with additional funding to handle the obsolescence of the 

booster cases and other systems after flight 8. Through flight 8 the booster systems are 

independent of SLS block configuration (Block 1 vs 1B); as such the acquisition and cost 

phasing is based on batch buys to sustain booster production capability at cost-efficient 

prices. 

• SLS core and upper stages are handled separately from Engines and Boosters. The SLS 

Program is currently developing and building both the first and second core stage, with 

primary focus on completing development and first flight unit production. The contract for 

the overall development has different line items for various portions of the development, 

but has not been extended past the second flight for the core stage or past the first flight 

development and production unit of the exploration upper stage. As of December 2018, the

manifest shows that the first flight of SLS Block 1B with EUS is not planned until after the 

second flight of SLS. Consequently, a continuation of the purchasing of ICPS upper stages is 

underway, and there is a mix of contractual and detailed planning for the various portions of 

the core and upper stage for flights 1 and 2 and no definitized contract for content beyond 

the second flight. 

• The Orion Program has a contract for the initial development and production of first an 

uncrewed test flight and then the first crewed flight of the Orion crew vehicle with a 

separate international partnership for the service module. The Orion Program is mid build 

for both of these first two flights, but does not have a ratified contract for flights beyond 
13 | Page

EM-2 as it is currently in-between RFP and establishing a contract for a set of flights beyond 

the first crewed flight. 

Due to the mixed nature of the major contracts for the major vehicle elements of SLS and Orion, ESD is 

currently managing cost estimation and control based on the individual elements and contracts and not 

based on individual flights. This is currently deemed the most efficient and effective way to provide 

specific accountability for the entire system by managing accountability at the element level over the 

course of multiple flight sets per element. As part of the affordability planning, ESD is working towards 

aligning the various element contracts to a common set of flights, but that alignment will be earlier than 

the sixth flight of SLS due to the status of currently established contracts and new hardware 

production. HEOMD agrees that Agency Commitments and Joint Cost and Confidence Levels (JCLs) for a 

single vehicle development are effective tools for insight, cost control, accountability, and 

communication, but single flight commitments within a multi-flight production environment are not 

effective or reasonable. Communication of individual element performance by contract can be achieved 

taking into account the variable nature of ESDs contracts over multiple prime contracts until a time 

when contract alignment allows for consistent communication of flight aligned hardware production.

Expected Benefits

The initiative is expected to improve HEOMD program and project management effectiveness. It is 

designed to lead to:

• Proactive investigation of program progress such that problems are detected early and overall 

project performance can be improved;

• Comprehensive review of existing metrics, processes, and tools to monitor program execution;

• An increase in cross-HEOMD communication of management best-practices and unsuccessful 

initiatives. 

Secondary benefits of the initiative include the improvement of cost and schedule projections for 

programs and projects in formulation by consolidating the historical programmatic database. This will 

help to identify effective acquisition and formulation strategies and will improve collaborative analysis 

with other mission directorates and OCFO. This internal process supports already-existing reporting 

structures within HEOMD and the Agency and is designed to enable an increase in information quality 

for internal and external communication. 

Recent Accomplishments

• Establishment of ESD/Programmatic and Strategic Integration (PSI) Monthly Integrated 

Performance Review (October 2017)

• Establishment of HEOMD RMO Special Assessments and Analysis Branch (July 2018)

Planned Next Steps

• Synthesis of data reports from existing monthly and quarterly reviews and reporting (Initial: Q1 

FY19)

• Synthesis of performance tracking metrics (Initial: Q2 FY19)

• Portfolio-level schedule overview with depicted interdependencies including major program and 

procurement milestones (Initial: Q3 FY19)

• Portfolio-level analysis assessing portfolio health and risk-exposure (Initial: Q3 FY19)
14 | Page

• Communication of enhanced portfolio assessment capability through internal and external 

reporting forums (Initial: Q4 FY19)

• Initiation of biannual discussion between GAO and HEOMD leadership on portfolio status 

(Initial: FY20)

• Sustainment and enhancement of overall process (Ongoing)

Output and Outcome Metrics

• Internal output of enhanced HEOMD integrated products

• Integration of best practices and data impact into program execution and management cycle

• External output of communication of enhanced processes through existing communication 

paths to OCFO and external stakeholders

• Outcome of increased speed of decision making and response time to emerging issues

• Outcome of increased agency and external stakeholder confidence in HEOMD program and 

project management due to increase in management effectiveness

Interdependencies

• HEOMD internal interdependencies between program PP&C managers, HEOMD RMO, and 

HEOMD leadership team

• Coordination of HEOMD analysis and assessment work with ongoing OCFO SID oversight and 

analysis

Impediments & Challenges

There are no major impediments to the process of enhancing HEOMD portfolio analysis. Challenges exist 

in unifying data into a coherent reporting format due to the diverse nature of HEOMD programs 

between formulation, development, operations, and closeout and due to the various formats in which 

data is collected and analyzed. 

Required Resources

Resource impact on programs due to time related to information gathering and synthesis for requests 

from integration team. Resource impact on integration team to appropriately support scale of data 

gathering and analysis. 
15 | Page

Implement Programmatic (PP&C) Training Curriculum

Lead Executive/Organization

OCFO SID

Supporting Organization(s)

The training lead coordinates with subject matter experts (SMEs) from Agency Centers and from each 

programmatic function to ensure that all NASA best practices and processes are reflected in the skillsets 

captured in the competency model matrices. 

Initiative Description

The purpose of the Programmatic Training Curriculum is to help bridge the gap between the currentstate workforce and future-state workforce of highly trained analysts. Establishing a training curriculum 

where courses are reflective of the Agency’s best practices and methods is necessary to grow and 

strengthen the Agency’s programmatic capabilities.

Background/Current State

The Budget and Management Business Services Assessment (BSA) core team conducted a study to 

evaluate the health of the Agency’s PP&C functions, specifically Integration, Resource Management, 

Schedule Management, and Cost Estimation and Assessment (CE&A). The outcome of this study 

revealed that, within the PP&C (herein referred to as Programmatic) workforce, there is an inadequate

number of advanced, proficient analysts. In addition, there are limited resources describing how a 

programmatic analyst develops into a programmatic SME. 

Expected Benefits

The approach to address the concern for the low number of advanced-level programmatic analysts is 

two-fold: to refine the Agency hiring process to recruit ideal candidates, and to establish a 

Programmatic Career Development and Progression Framework to provide training and guidance to 

those candidates and other programmatic employees. The hiring and interview processes should be 

strategic to attract candidates who possess an inherent analytical and technical aptitude to support 

programmatic work. The Programmatic Career Development and Progression Framework will offer a 

roadmap of how to enhance or develop programmatic skills. It will include tiered competency matrices 

of entry-level to advanced-level programmatic analysts that is reinforced with a common, centralized 

Programmatic Training Curriculum. The establishment of the Programmatic Training Curriculum will 

increase the Agency’s programmatic proficiency level and encourage adherence to Agency best 

practices and processes. This will also promote the consistent application of analytical methods and 

techniques, which will produce more coherent, reliable work products. Although a robust training 

program lays a foundation for acquiring certain programmatic skills, it is important to note that on-thejob experiences and mentoring is also needed to develop programmatic SMEs. Taking concepts learned 

in formalized classroom settings and applying them in day-to-day job functions will enable an analyst to 

grow in their discipline, strengthen any weak programmatic areas, and continue their progression to an 

advanced-level programmatic analyst. 

Recent Accomplishments

• Identified subject matter experts for each programmatic function and from different Centers to 

participate in training working groups
16 | Page

• Established training working groups for each programmatic function. Working groups met 

biweekly to monthly to:

o Define the skill sets of proficient and advanced analysts for each function for the

creation of competency matrices

o Assess whether existing training courses support the development of the key skill sets 

previously identified and notate when these skill sets are not supported by current 

training offerings. This is considered a training gap. 

o Outline the learning objectives, prerequisites, and target audiences of any newly 

identified training course that addresses a potential training gap. 

• Defined and integrated the training courses (existing and newly identified) from each working 

group to create the initial draft of the synergistic Programmatic Training Curriculum

Planned Next Steps

The planned next steps are to:

• Coordinate with Training Working Group members to peer review course content to ensure that 

information is consistent and addresses all known gaps

o Update the training curriculum document to reflect the working group inputs (1st

quarter of FY2019)

• Present the Programmatic Training Curriculum draft to the PP&C Steering Committee for 

concurrence to begin course development and deployment (1st quarter of FY19)

o Address any changes to the training curriculum based on PP&C Steering Group 

recommendations (2nd quarter FY19)

o Identify the required number of dedicated resources to support course development 

(2nd quarter of FY19)

o Establish a course development plan/timeline based on resource availability (2nd

quarter of FY19) 

• Socialize the curriculum to Mission Directorates and AA’s so that senior leadership is aware of 

potentially new and/or updated courses for their employee’s development. Incorporate any 

recommendations/suggestions (2nd quarter of FY19)

• Coordinate with Office of Human Capital to discuss career path development as it relates to 

competency models. Reflect guidance within the curriculum where applicable (2nd quarter of 

FY19)

• Begin update of existing and development of new courses (3rd quarter of FY2019 – 3rd quarter 

FY20)

• Deploy new training courses (4th quarter of FY20)

Output and Outcome Metrics

• A percent increase on the number of proficient analysts within the Agency. Similar to the BSA 

study, assess the proficiency levels of the Programmatic community within a year of course 

deployment. 

• Common Competency Models exist for each PP&C discipline and are used toward Career 

Planning and 100% of programmatic job announcements reflect competencies listed within 

models 
17 | Page

o Supervisors and hiring officials refer to competency models to assess the proficiency 

levels of programmatic analysts and to gauge their ability to support advanced 

programmatic analysis

• Development and deployment of training courses

o Courses are being developed according the planned timeline. 100% of plan deviations 

are documented and communicated to the Programmatic Portfolio lead. 

o Each PP&C professional has taken at least one programmatic course from the 

curriculum and/or have identified courses within the Individual Development Plan (IDP). 

Supervisors reinforce the training needs at mid-year and annual performance 

discussions

o New hires and early career programmatic professionals have taken at least two courses 

within one year of activity within the PP&C community 

Interdependencies

An interdependency exists with the GAO Corrective Action Plan initiative for a PP&C NASA Procedural 

Requirement (NPR) document and other Agency efforts that involve updating programmatic guidance 

documents and handbooks (e.g., Schedule Management Handbook, EVM handbook, Cost Estimating 

Handbook, Risk Management Handbook). The guidance and requirements outlined in related 

programmatic material will be reinforced within the training courses. 

There are similar training initiatives that exist within the Agency that are interdependent with the 

Programmatic Training Curriculum efforts. NASA’s Chief Financial Officer University (CFOU) has an 

established curriculum that caters to the job functions related to the finance and resource management 

domains as well as the management of risk used for programmatic and institutional decision-making. 

Some of these courses can be expanded or amended to include the proper programmatic context to 

ensure that concepts are properly reflected in related training material. Additionally, OCFO’s 

Professional Development Framework (PDF) has created a career path website that enables employees 

to view the competency matrices of OCFO job functions. The competency levels of the Programmatic 

functions can also be added to this website so that information is provided uniformly and consistent to 

other related technical and programmatic fields. 

Impediments & Challenges

The challenges associated with this effort are as follows:

• Dedicated resources

o Although many individuals were involved in the development of the Programmatic 

Training Curriculum, this initiative does not have any dedicated resources. Members of 

the Programmatic Training Working Groups participate on a voluntarily basis. The only 

dedicated staff member of this effort is the Training Lead. Not having a dedicated set of
18 | Page

individuals to carry out the development and deployment of training materials will 

impede the establishment of a beneficial training program.

• Establishing a common terminology standard for certain analysis processes

o Centers use slightly different processes and terminologies to execute their analyses. 

With a centralized training curriculum, Centers will be encouraged to adhere to the 

processes and terminology adopted within the Agency-developed curriculum, thus 

potentially abandoning some prior processes. 

Required Resources

The resources required to execute the establishment of the Programmatic Training Curriculum include a 

group of dedicated SMEs that are given the time to develop courses and teach the material. The intent is 

not to solicit assistance from outside the Agency, but to use programmatic SMEs who are familiar with 

NASA’s best practices and processes. However, full-time dedication is not the requirement; depending 

how often a course is taught, an instructor will need a budget to develop course material, prepare for a 

class, travel to the class site (if applicable), and teach the course. Portions of the budget will also help 

ensure that there is a central repository where all related materials of the Programmatic Training 

Curriculum can reside. 
19 | Page

Include Original Agency Baseline Commitments for Performance-Driven Re-baselined 

Projects

Lead Executive/Organization

Office of the Chief Financial Officer

Supporting Organization(s)

N/A

Initiative Description

For projects that have been re-baselined due to performance (vice scope change), NASA will begin to 

include original cost and schedule Agency Baseline Commitments (ABC) in our quarterly, semi-annual, 

and annual external cost and schedule reports. OCFO will report these data alongside the current 

reporting structure for transparency purposes.

Background/Current State

NASA collects cost and schedule data to satisfy a number of purposes and customers, both internal and 

external to NASA. In 2004, the GAO published a study1 stating that a lack of disciplined project cost 

estimating processes was resulting in project management problems, schedule slippage, and cost 

growth. Soon after, Congress passed the NASA Authorization Act of 2005, which created the first 

external reporting requirements to Congress, the Major Program Annual Report (MPAR)2

. The MPAR, 

which requires NASA to report on projects in development with an estimated life cycle cost exceeding 

$250 million, is the nexus for all external reporting. All other requirements for cost and schedule 

reporting stem from the MPAR.

Baseline control is the basis for external reporting, which is tied to project lifecycle cost (LCC). For 

projects with a LCC of at least $250 million, NASA externally commits to a cost and schedule baseline for 

LCC, development cost (Phases C and D), and launch readiness date (or alternate key schedule 

milestone) established at KDP-C. Per NPR 7120.53

, programs or projects shall be re-baselined when: 

1. The estimated development cost4 exceeds the Agency Baseline Commitment development cost 

by 30 percent or more (for projects over $250 million, also that Congress has reauthorized the 

project)5

; 

2. The NASA AA judges that events external to the Agency make a re-baseline appropriate; or 

3. The NASA AA judges that the program or project scope defined in the ABC has been changed or 

the tightly coupled program or project has been interrupted. 

Project ABCs are not re-baselined to reflect cost or schedule growth that does not meet one or more of 

these criteria.

NASA reports cost and schedule variance against ABCs on a quarterly, semi-annual, and annual basis. 

Quarterly, NASA’s OCFO submits cost and schedule reports to OMB. Semi-annually, OCFO submits cost 

 1 NASA: Lack of Disciplined Cost Estimating Processes Hinders Effective Program Management [GAO-04-642] 2 Public Law 109-155 Section 103 Baselines and Cost Controls (codified at 51 U.S.C §30104) 3 NPR 7120.5E, Paragraph 2.4.1.7 4 “Development cost” includes all project costs from authorization to proceed to Implementation (Phase C) 

through operational readiness at the end of Phase D.

5 51 U.S.C §30104(f)
20 | Page

and schedule reports to GAO. Annually, NASA submits cost and schedule reports to Congress in the 

President’s Budget Request (PBR) Congressional Justification (CJ). In all of these reports, for projects that 

have been re-baselined, cost and schedule variations are reported against the re-baselined figures and 

not the original ABC. GAO has recommended that, for re-baselined projects, NASA report cost and 

schedule growth from original baselines in order to provide stakeholders and Congress with a more 

accurate view of project performance and to enhance accountability.

6

In certain cases, re-baselines occur for reasons beyond the control of NASA (e.g., weather-related 

events, international partnerships, etc.). To fairly represent NASA-driven cost and schedule 

performance, original baselines will only be reported for re-baselined projects that have been rebaselined primarily due to NASA performance issues. As with other elements of the Corrective Action 

Plan, the NASA AA may, in future iterations of the Plan, make changes to the implementation of this 

initiative as challenges or opportunities are encountered.

Expected Benefits

The majority of NASA’s major projects proceed through the development phases within original baseline 

cost and schedule commitments. In the event that a project is re-baselined, NASA reports future cost 

and schedule variation from the newly established Reporting Baseline. This is a valuable metric because 

it tracks cost and schedule variation from the newly authorized commitment, which is thoroughly 

reviewed and approved by all levels of the Agency, OMB, and Congress. NASA will continue to 

emphasize that once a project is re-baselined, it is appropriate to measure success against the most 

recent baseline commitments and not the original commitments.

However, in the spirit of transparency, NASA will begin to include the original baseline figures (for 

projects that have been re-baselined) in external cost and schedule reports alongside the current 

reporting structure. NASA has long recognized the value in tracking cost and schedule variation against 

original baseline commitments, including such variation in many internal analyses. The Agency’s external 

partners and the public will benefit from this added transparency in our cost and schedule reports, as 

they have benefitted from knowledge of other cost and schedule variation across NASA major projects 

absent a re-baseline.

Recent Accomplishments

N/A

Planned Next Steps

• In FY19, NASA will review and determine the best way to present the original ABC information in 

its routine external cost and schedule reports in such a way that it is clear to all parties why the 

project was re-baselined, and why future measurements of growth should be measured against 

the re-baselined figures.

• Beginning in FY20 (or earlier if possible), for projects that have been re-baselined due to 

performance, NASA will include original cost and schedule baselines in external reports 

(quarterly to OMB, semi-annually to GAO, annually in PBR/CJ). Projects that have been rebaselined due to a scope change will be excluded.

 6 GAO High Risk Series [GAO-17-317] Page 473
21 | Page

Output and Outcome Metrics

N/A

Interdependencies

N/A

Impediments & Challenges

N/A

Required Resources

N/A
22 | Page

Enhance Annual Strategic Review Process

Lead Executive/Organization

Performance Improvement Officer (PIO) and Program Management Improvement Officer (PMIO)

Supporting Organization(s)

OCFO/Mission Directorates/Mission Support Offices

Initiative Description

Enhance the Agency’s existing annual Strategic Review process to leverage the program portfolio 

reviews required by the Program Management Improvement and Accountability Act (PMIAA) aligned 

with OMB guidance. The portfolio reviews will provide additional information for Agency senior 

leadership to assess the effectiveness of an Agency program’s management and performance, identify 

opportunities for improvement, and serve as a basis to enable additional external stakeholder

communication regarding the outputs of the review.

Background/Current State

NASA currently conducts an annual assessment of progress made against its strategic objectives 

identified in the 2018 Strategic Plan (NPD 1001.0C). This Strategic Review is mandated within the 

Government Performance Results Act Modernization Act of 2010 (GPRAMA), with specific requirements 

for each exercise codified in annual updates to OMB Circular A-11, Part 6, as well as additional 

requirements communicated via standalone Strategic Review Guidance memoranda (e.g., M-18-15).

Per GPRAMA, the Strategic Review is conducted under the authority of the Chief Operating Officer 

(COO) (NASA Associate Administrator) and managed by the Performance Improvement Officer (SID 

Division Director). The Strategic Planning, Performance Management, and Reporting Branch within 

OCFO SID is responsible for the coordination of the review. The BPR is used as the forum for the Agency 

meeting that concludes the internal portion of the review. The annual Strategic Review cycle officially 

concludes with a Principals meeting held at OMB, where the results of the internal review are discussed 

with representatives of the Administration. 

Tactically, the review begins with each MD or Mission Support Office (MSO) developing an assessment 

of their ongoing and planned activities in support of each strategic objective assigned to their 

organization. In addition, OCFO SID develops an independent assessment of the progress the Agency has 

made, or committed to, in support of each strategic objective. The results of both assessments are 

presented to the NASA AA, who in turn determines a final “rating” for each strategic objective. The 

ratings, along with a high-level summary of the considerations for each rating, are provided to OMB in a 

“Summary of Findings” report, which in turn is utilized to develop the agenda for the Principals Meeting 

that concludes the review cycle. Guidance for ratings and specific requirements for the Summary of 

Findings deliverable are variable and dependent on annual guidance from OMB.

The PMIAA will be implemented in a phased approach over several years per OMB guidance 

memorandum M-18-19, with the provision of incremental OMB guidance as the agencies implement it. 

Aligned with OMB guidance, in 2019 NASA will integrate a pilot portfolio-level review component into its 

existing Strategic Review framework, and incrementally expand on the portfolio reviews in the future. 

The initial portfolio reviews will include non-IT major programs, and IT major projects will be added in 

the future. OMB will also conduct portfolio reviews of the most at-risk agency programs designated as 

high risk by GAO, however, plans for these reviews are pending future OMB guidance. This Corrective 
23 | Page

Action Plan initiative captures the Agency’s strategy to implement the portfolio-level review, to include 

expanding beyond the requirements of the PMIAA legislation to incorporate additional external 

stakeholder access of the review’s outputs.

Expected Benefits

The current Strategic Review, per OMB requirements, narrowly focuses on the Agency’s annual progress 

made against its 13 long-term strategic objectives established in its 2018 Strategic Plan. The measures 

described in this initiative will enhance the value of this exercise as follows:

• An additional layer of review at the portfolio level will enable additional insight into the 

challenges affecting the Agency’s Mission Directorates.

• Additional acquisition-specific data sets will be collected during the review (starting in 2019).

• Major non-IT investments will be included in the review for the first time starting in Phase II7 of 

PMIAA implementation (expected NLT 2021). 

• Assessment of the Agency’s larger grant programs to be included in the review for the first time 

starting in Phase III (expected NLT 2023).

In addition to the enhanced data collection and consideration that will occur during the review as a 

result of this initiative, NASA will implement a process to discuss the findings of its review with the GAO, 

starting with the 2020 assessment cycle. This will ensure additional stakeholder communication 

regarding the outputs of the review. 

Recent Accomplishments

An annual review of the Agency’s strategic objectives was first conducted in 2013, per GPRAMA and 

OMB guidelines. That initial exercise included an assessment of the Agency’s strategic objectives 

previously established in its 2011 Strategic Plan. The COO decisional meeting that concluded the review 

occurred within a stand-alone internal stakeholder meeting, outside of a recognized management 

council, and utilized a uniform set of success criteria. NASA has successfully implemented improvements 

to this process over the subsequent 5 assessment cycles, to include:

• Increased integration with the Agency’s nascent Enterprise Risk Management (ERM) framework.

• Full integration with the Agency’s existing monthly Baseline Performance Review (BPR), which 

has greatly increased the audience of the assessment, as well as the availability of the data to 

senior decision makers across the Agency.

• Development of success criteria unique to each strategic objective, established in tandem with 

the development of each objective during the formation of the Agency’s 2018 Strategic Plan.

• Continued refinement of the templates utilized to collect inputs into both the independent and 

stakeholder assessments, as well as additional templates used to summarize assessment 

findings for NASA AA/COO consideration.

In addition, the Agency has made impressive strides to integrate the outputs of the review within its 

ongoing strategic planning efforts. 

 7 M-18-19 mandates that CFO Act agencies follow an incremental 3 phase approach to implementing PMIAA 

legislative requires. All 3 phases are to be completed within a 5 year window, beginning in December 2018. 
24 | Page

Planned Next Steps

The Agency will utilize the upcoming 2019 review cycle as a pilot year prior to consider a permanent 

format for the portfolio review component of the exercise. The pilot will also seek to inform what 

subsequent phases of PMIAA implementation may include. 

During the pilot exercise, the 2019 Strategic Review will be modified as follows:

• One portfolio8 will be selected to participate in this initial exercise

• Selection will be made by the COO/NASA AA

• The Mission Directorate Associate Administrator (or their designee) for the selected 

organization will provide a high-level assessment of their portfolio’s strengths and weakness at 

the beginning of the annual Strategic Review meeting held at the April BPR

• A template developed by the PMIO will be provided to the selected portfolio owner that will 

include the following considerations:

o The performance of the non-IT major programs and projects in their portfolio (life cycle 

cost greater than $250 million)

o Opportunities for improvement, as well as identification of any barriers to achieving 

program outcomes

o Identification of 2-3 major acquisitions that have demonstrated particular success in the 

last two years

o Identification of 2-3 major acquisitions that have encountered challenges in the last two 

years 

o Any additional lines of inquiry deemed necessary by the Program Management 

Improvement Officer

• The NASA AA will provide feedback/guidance to the selected portfolio manager (NASA MDAA), 

as necessary. In addition, corrective actions or follow-on investigations may be assigned during 

the Strategic Review meeting

• A high-level synopsis of these discussions, as well as any follow-up actions, will be included in 

the Summary of Findings report provided to OMB (expected May 21, 2019)

• An agenda item will be added to the Principals Meeting at the conclusion of the 2019 review 

cycle (expected NLT June 28, 2019), to ensure a dialogue between OMB and the Agency in 

regards to the selected portfolio’s successes and/or challenges

The portfolio level review component of the broader 2019 Strategic Review will be overseen by the 

PMIO and will incorporate a set of broadly applicable program management principles, practices, and 

standards associated with successful program outcomes, in addition to more specific standards based 

on the type of program being reviewed. Agency managers will be held accountable for addressing areas 

identified for improvement during the portfolio level component of the review. The PIO will remain 

responsible for all other components and activities associated with the 2019 Strategic Review.

 8 For the purposes of PMIAA implementation, NASA’s major acquisition portfolios will be broadly defined utilizing 

the organizational structure authorized within NPD 1000.3E. This includes 4 Mission Directorates (Aeronautics 

Research, Human Exploration and Operations, Science and Space Technology)
25 | Page

NASA and OMB will evaluate the results of the portfolio review pilot prior to initiating the 2020 Strategic 

Review. The results of that analysis, along with any additional guidance communicated in future 

iterations of OMB Circular A-11, Part 6, will be used to enhance the portfolio review process starting in 

2020. The GAO’s 2019 High Risk Report (expected February 2019) may introduce additional 

considerations. Any process adjustments to the portfolio review process will be codified in future 

updates to the Agency’s 5-year implementation plan for PMIAA. 

Integrated Strategic Review/Portfolio Review Pilot Schedule:

Output and Outcome Metrics

• Specific metric TBD – Likely to consider additional performance information that will be provided 

in the FY 2021 Volume of Integrated Performance, which will be published in February 2020.

• Specific metric TBD – Likely to consider additional agenda topics included in the Principals 

Meeting held with OMB and/or potentially GAO every summer.

• Specific metric TBD – Likely to consider growth in number of data elements included in the 

assessment templates year over year, starting in 2019. 

Interdependencies

• OCFO SID Director

o Acts as NASA’s designated Performance Improvement Officer (PIO), responsible for 

ensuring Agency compliance with all GPRAMA commitments

• Program Management Improvement Officer (PMIO)

o Acts as NASA’s responsible party for ensuring compliance with all PMIAA commitments

• OCFO SID Strategic Planning, Performance Management, and Reporting Branch (SPPM&R)
26 | Page

o Leads coordination for all Strategic Review requirements embedded within GPRAMA 

and OMB Circular A-11, Part 6

• OCFO SID Portfolio Intelligence and Assessment Branch

o Independent assessment team

• Mission Directorates (HEOMD, ARMD, SMD, and STMD) and Mission Support Directorate (MSD)

o Each Mission Directorate has responsibility for implementing activities in support of at 

least 1 strategic objective established in the 2018 Strategic Plan

o HEOMD, ARMD, SMD, and STMD have responsibility for portfolios included in 2019 pilot

• Chief Operating Officer (COO)/Associate Administrator (AA)

Impediments & Challenges

• The OMB PMIAA guidance is based on a phased approach over several years with additional 

OMB guidance that will be provided as the agencies implement in parallel. This may cause 

changes to the agency’s implementation approach.

• Assessing strategic objective progress, as well as portfolio-level challenges, simultaneously may 

introduce confusion that could impair the value of the assessment’s findings to senior 

leadership. 

• PMIAA delegated responsibility to OMB to issue annual tactical guidance regarding 

implementation of the requirements embedded within the legislation. As such, NASA is 

dependent on to-be-released external guidelines that could complicate its ability to successfully 

coordinate its Strategic Review and Portfolio Review requirements while maintaining its current 

value to senior leadership. 

• PMIAA delegated aspects of the legislation’s oversight to the GAO. Future GAO assessments of 

OMB’s government-wide implementation of PMIAA could impact the mechanics or reporting of 

NASA’s internal assessment activities. 

Required Resources

• Input and support from offices identified above as interdependencies

o OCFO and MDs/MSOs will continue to provide analytical support

o OCFO SID SPPM&R Branch to lead implementation 

• NASA COO, PIO, and PMIO approvals

• MD partner for pilot exercise

o Identify a portfolio owner to participate in the initial pilot (scheduled to be selected in 

January 2019)
27 | Page

Create Technology Readiness Assessment (TRA) Best Practices Document

Lead Executive/Organization

Office of the Chief Technologist

Supporting Organization(s)

Office of the Chief Engineer; Various Centers and Mission Directorates

Initiative Description

This initiative is expected to develop a document that will capture the technology readiness assessment 

(TRA) information that is scattered throughout the Agency, provide links to governing documents, and 

document best practices across the Agency. The intent is to provide best practice information and 

governing statements and to document a generic process for how to conduct assessments.

As the Agency continues discussing risk, one element that could help inform decisions is an independent 

technology readiness assessment, which would provide an independent view of the state of a new 

technology or system of technologies. The assessment could inform projects and missions about the 

current technology readiness level (TRL), as well as provide an informed assessment of what will be 

needed to integrate into a larger system. 

Background/Current State

In 2014, a study was conducted by a partnership of the Office of the Chief Technologist and the Office of 

the Chief Engineer. Concurrently, GAO was developing its Technology Readiness Assessment Guide. 

The internal study made several recommendations, one of which was to “develop a TRA Handbook that 

will consolidate all TRA and TRL processes, guidance, best practices, examples, and other related 

content into a single reference source.” This initiative is in response to that recommendation, in order to 

gather information regarding best practices into a single source rather than the multiple sources 

documented by the study team. 

Expected Benefits 

It is expected that this document, when completed, will gather the information regarding technology 

readiness and its assessment into a single source. Currently, as the study team found, TRA information is

found in multiple NPRs and in the Systems Engineering Handbook, as well as in many Center 

documents/practices. These governing documents will not be changed. The purpose of this effort is to 

gather the high-level information into a single document with citations to governing documents. The 

other intent of the document is to provide information on how to conduct an assessment, based on the 

expertise of current TRA practitioners across the Agency.

Recent Accomplishments

N/A

Planned Next Steps

NASA will stand up a team of interested and knowledgeable participants to gather best practices across 

the Agency. Once engaged, the team will compile the best practices into a document for Agency review.

Output and Outcome Metrics

Outputs will include documented best practices for use by Mission Directorates, Offices, and Centers for 

independently determining the technology readiness level of a given technology or system of 
28 | Page

technologies. This information can be used for decision-making by programs and projects to provide 

additional information regarding the state of a technology prior to incorporating it within the program 

or project. 

Interdependencies

Technology readiness assessments could help inform risk discussions, as well as assist practitioners and 

projects to respond to the GAO TRA Guide, which is currently in the review process.

Impediments & Challenges

• It will take time to survey, compile, and assess the many practices across the Agency to 

adequately identify, document, and link best practices.

• Participants will be limited to working on an as-available basis. 

Required Resources

The document will be researched and written by TRA practitioners across the Agency on an as-available 

basis.
29 | Page

Update Probabilistic Programmatic Policy

Lead Executive/Organization

OCFO/SID/APARC

Supporting Organization(s)

Supporting organizations, beyond OCFO and the NASA PP&C communities, will include the Office of the 

Chief Engineer (OCE) and NASA MDs. Flight projects will be the implementing organizations and NASA 

Standing Review Boards (SRBs) will conduct assessments based on the requirements. The PP&C Steering 

Group will provide the needed technical expertise and guidance.

Initiative Description

This initiative readdresses NASA’s current probabilistic programmatic policy. NASA will add a 

requirement for flight projects to conduct a Joint Cost and Schedule Confidence Level (JCL) 

corresponding with the System Requirements Review (SRR) leading up to KDP-B. NASA will also add a 

requirement for flight projects to conduct/update a JCL at the Critical Design Review (CDR). Further, 

flight projects that are not performing to plan will have a requirement to conduct/update a JCL at the 

System Integration Review (SIR) leading up to KDP-D. These new requirements will apply to all major 

flight projects with a life cycle cost of at least $1 billion, or otherwise designated by the NASA AA. The 

existing JCL requirement for the Preliminary Design Review (PDR) leading up to KDP-C will not change.

Background/Current State

NASA currently uses a variety of cost and schedule methodologies to formulate, plan, and implement 

projects. One of the general methodologies deployed by NASA is the use of probabilistic programmatic 

analysis. NASA found that budgeting projects to a risk-adjusted estimate that reflects a project’s risk 

exposure, in combination with communicating that risk posture internally and externally, facilitates 

more realistic budget positions. It also aids in communication of the budgets. NASA currently requires 

projects to perform probabilistic analysis once at KDP-B and once at KDP-C. At KDP-B, NASA requires all 

flight projects with lifecycle costs (LCC) over $250 million to provide probabilistic analysis on their cost 

estimates and their schedule estimates. The intent of KDP-B policy is to position projects to have a 

healthy programmatic risk posture going into KDP-C. At KDP-C, NASA requires all flight projects with LCC

over $250 million to provide a joint cost and schedule confidence level (JCL) analysis. After KDP-C, NASA 

policy does not require a project to update any probabilistic analysis. However, the Agency uses a 

variety of performance metrics to assess how well the project is performing against its plan. If these 

metrics show that a project’s performance varies significantly from its plan, the project may need to 

replan, but Agency policy only encourages updates to the probabilistic programmatic analysis (i.e., JCL

analysis) in the event the project requires a re-baseline9

. 

NASA probabilistic programmatic policy has been empirically successful as measured by an observable 

reduction in the number and magnitude of project baseline overruns since the implementation of the 

policy. Additionally, the JCL policy has assisted in communicating Agency risk posture and quantifying 

the need for unallocated future expenses10 (UFE). 

 9 Re-baselined projects are not required, but are encouraged, to do a JCL analysis by NPR 7120.5E. 10 Per NPR 7120.5E, UFE is “the portion of estimated cost required to meet specified confidence level that cannot 

yet be allocated to the specific project WBS sub-elements because the estimate includes probabilistic risks and 
30 | Page

External stakeholders have documented the value of NASA’s probabilistic policy and have recommended 

that the policy be augmented.

11 Furthermore, the probabilistic policy, and specifically the JCL policy, has 

been a forcing function for projects to consider how their programmatic, technical, and risk products 

interact, which allows for more traceable programmatic integration.

Expected Benefits

Improvements to probabilistic estimating processes for our largest projects at KDP-B, CDR, and KDP-D

might further reduce cost and schedule overruns. Producing a JCL at SRR leading up to KDP-B will better 

inform leadership of the range of potential cost and schedule impacts on the portfolio through Phase B 

and through the life cycle. Updating the JCL at CDR will capture evolving and emergent risks and provide 

leadership with an enhanced awareness regarding projections of the project’s cost and schedule 

through Implementation. By the time of SIR and KDP-D, most work-to-go involves “marching army” costs 

and a JCL will provide diminishing returns. However, for projects that are off plan, updating a JCL will 

better inform decision-makers at KDP-D of the confidence of hitting cost and schedule targets given 

evolving risk postures. By focusing these new JCL requirements on the largest projects (LCC of at least $1 

billion), NASA is intelligently applying its programmatic resources to mitigate the largest potential 

impacts to its portfolios. 

Recent Accomplishments

N/A

Planned Next Steps

The following milestones have been identified to address augmentation of NASA probabilistic 

programmatic policy.

• APMC approval of the CAP will serve as approval for the new JCL requirements.

• Upon concurrence of the APMC, NASA will publish an executive memo detailing any new 

probabilistic programmatic requirements and the expectations on how those requirements will 

be transitioned in the current Agency manifest. This memo will focus on the implementation 

details for the KDP-B JCL.

• Language specified in the APMC executive memo will be formal Agency guidance until NASA 

NPR 7120.5 is updated.

Output and Outcome Metrics

• Objective measurement and communication of changes to cost and schedule risk exposure and 

performance from the KDP-B, CDR and SIR milestones12 for the affected projects. 

 

specific needs that are not known until these risks are realized.” It is commonly referred to as “reserves” in 

industry.

11 “Ensure that program offices regularly and consistently update their Joint Cost and Schedule Confidence Levels 

(JCL) across the portfolio. As a project reaches the later stages of development, especially integration and testing, 

its risk posture may change. An updated project JCL would provide both project and agency management with 

data on relevant risks that can guide project decisions. ” Government Accountability Office. GAO-17-317

“High-Risk Series,” p. 473. February 2017. 12 Performance is a function of the risk posture the Agency takes at each milestone. For example, higher risk 

programmatic postures at KDP-B may result in lower performance – but both metrics should be internally 

consistent.
31 | Page

Interdependencies

N/A

Impediments & Challenges

The challenges associated with this effort are as follows:

• Dedicated Resources: Additional resources may be needed to implement any new requirement.

• Communication: Communicating expectations and implementation down to projects and up to 

stakeholders will be a challenge.

Required Resources

• Implementation Resources

o Probabilistic programmatic requirements will require more programmatic support to 

projects. Implementing JCLs at SRR, CDR, and SIR will require projects to produce and 

communicate a work product that has not been baselined in their current workload and 

expectations.

o Depending on the ultimate policy selected, jumpstart capability may be needed to assist 

projects and implementing Centers for the first year of the initiative’s implementation 

to ensure the consistency of expectations and products. For example, when the nowdisbanded Office of Evaluation’s (OoE) Cost Analysis Division did a similar jumpstart 

activity for JCL implementation, the total cost to OoE for two years of implementation of 

that jumpstart was approximately $400,000 annually. However, JCLs are no longer new 

to the Agency, and the Centers (as well as many projects) have developed the capability 

to do probabilistic analysis; therefore, the cost of implementing any increases to the 

current JCL requirement should be significantly lower than the initial jumpstart costs 

paid by OoE.

o Jumpstart support would handle the mechanics, but implementation would additionally 

require projects to dedicate hours of personnel support to produce the proper inputs to 

any increased probabilistic programmatic requirement.

• Communication

o Additional internal and external communication would be needed to support 

expectations and implementation. Resources for communication can be covered by the 

current PP&C Community. 
32 | Page

Initiative to Pilot
33 | Page

Create a Schedule Repository

Lead Executive/Organization

OCFO/SID/APARC will lead the Schedule Repository effort with direct support of the Agency’s PP&C 

Portfolio Lead and the PP&C Steering Group.

Supporting Organization(s)

OCFO/SID/APARC will coordinate with Center points of contact (primarily the Schedule Community of 

Practice (SCoPe) POCs), and individual programs and projects on the collection of schedule data to 

support the Schedule Repository effort. 

Initiative Description

A Schedule Repository is a collection of program and project (P/p) schedules for completed and current 

missions and technology developments. Schedule information is organized and archived in a searchable 

library. The purpose of the Schedule Repository is to:

• Provide the schedule management community with access to historical and analogous schedules 

to aid in the planning and development of schedules for future missions.

• Allow for the continuous improvement of schedule management guidance and best practices. 

The schedule data collection cadence for the Schedule Repository will be determined by the PP&C 

Steering Group, which is represented by the Centers and Mission Directorates (MD).

Background/Current State

In 2016, as part of the Agency’s Schedule Initiative (SI), information was provided to the schedule 

management community on SI efforts in order to collect feedback from the community regarding areas 

of schedule management in need of improvement. The findings, which were presented at the 2017 

NASA Cost and Schedule Symposium, highlighted the need for access to past P/p schedules and for

schedule management guidance that is more detailed, consistent, and consolidated. 

With respect to access to past P/p schedules, currently, detailed schedule information is not collected at 

the Agency, MD, or Center levels on a routine basis because no requirements or formal processes exist 

(e.g., Cost Analysis Data Requirement (CADRe) collects only mission-level integrated master schedules 

(IMS) at lifecycle reviews). Thus, there is no centralized repository of detailed P/p schedule information 

from which other programs or projects can draw analogous data for schedule planning, development, 

and analysis purposes. In addition, schedule data is sometimes lost (e.g., during computer refreshes that 

go wrong or during employee attrition without proper data transition), preventing future programs or 

projects from locating relevant data. Schedulers then must create schedules from scratch instead being 

able to use valuable, NASA-specific data that include realized risks and lessons learned to inform 

planning and potentially to produce more realistic schedule estimates. This limits the efficiency with 

which schedules are produced and assessed. In many cases, it also constrains the ability to document 

and support the schedule basis of estimate (BoE) effectively. A Schedule Repository will alleviate these 

problems by providing an accessible, centralized archive of NASA schedule data.

To address schedule management guidance needs, the Agency decided to update the Schedule 

Management Handbook. While the draft is almost complete, it is not yet released. Updates will include 

the identification of NASA-specific best practices to address how NASA conducts schedule management 

(based on requirements specific to NASA, such as JCL and “risk-informed” schedules). It will incorporate 
34 | Page

practices and processes used across various Centers. The handbook will also include significantly more 

guidance on how to assess and analyze a schedule using different methods and techniques. A Schedule 

Repository that supports analysis across a variety of schedules will facilitate ongoing handbook updates 

to ensure the Agency’s schedule management guidance stays relevant and addresses current concerns. 

Expected Benefits

A Schedule Repository will provide a centralized source for quick and convenient access to schedule data 

(integrated master schedules, as well as supporting schedules and data) from which schedulers and 

analysts will be able to research analogous missions and use them as a basis for developing program or 

project schedules. A Schedule Repository will also allow the Agency to fine-tune best practice guidance 

for schedule planning, development, assessment, analysis, maintenance, and control. Looking across P/p 

schedules can inform the Agency regarding how well schedule management best practices are 

implemented, which in turn can lead to improvements in schedule guidance and training. In addition, 

having a routine collection of schedules will aid the Agency in understanding how and at what point in 

project lifecycle schedules typically experience issues. The trending of performance-based indices across 

a repository of programs and projects can help uncover the critical points in P/p lifecycles when 

schedule performance issues are more likely to arise. This can also facilitate continuous improvement in 

guidance and training.

Recent Accomplishments

• Two meetings have been held with Goddard Space Flight Center (GSFC) to understand their 

Center OCFO-led initiative for the routine collection of schedule data into a consolidated library. 

Future meetings are planned for further discussion.

• One meeting has been held to explore options for housing the Schedule Repository. Future 

meetings are planned for further discussion. 

Planned Next Steps

• To proceed with the Schedule Repository effort, the OCFO/SID/APARC Lead will work with 

supporting organizations to identify Center SCoPe and P/p POCs and assign targeted dates for 

the following next steps:

o Determine what schedule information (e.g., past or current program and project IMS 

and supporting schedules) can be collected for past and current programs or projects at 

each Center.

o Determine the cadence of schedule information collection.

o Determine the most effective way to transfer schedule data from the data source to the 

Schedule Repository, whether as a cloud function, server capability, direct-from-user 

delivery, or a hybrid approach. 

o Develop a framework for the Schedule Repository to determine where the data will be 

stored and what organizational structure will be used. 

o Collect and organize P/p schedules for both completed and in-work missions according 

to the identified framework.

Output and Outcome Metrics

The Schedule Repository will consist of past and current schedules (as of the most recent key decision 

milestone) from each Center collected in a centralized repository. Schedule collection metrics (e.g., 

percentages of available data collected, percentages of projects with complete or current data sets) will 

be developed and schedule data that is collected for the repository will be tagged (e.g., by work 
35 | Page

breakdown structure, subsystem type, and/or responsible organization) to enable various 

categorizations as appropriate for Agency analysis and use. 

Impediments & Challenges

There are several challenges to implementing a data collection activity to produce a consolidated 

Schedule Repository:

• Establishing an Agency Framework: to be supportive of all stakeholders from practitioners to 

decision makers

o Different schedule file formats (e.g., Primavera versus Microsoft Project files)

o Schedules may be at different levels of detail for different programs and projects

o Schedules may be at different levels of integration for different programs and projects

(e.g., giver/receiver relationships versus all work captured in one IMS)

• Defining Scope: Mission thresholds (e.g., category) and types (e.g., flight, technology)

o Implementation Approach for Data Collection

o Implementation Timeline

o Data Fidelity

o Ownership/Access

• Governance: Forcing mechanism (requirements/guidance)

o Not “adding” product requirements

o Not “dictating” schedule processes beyond what is already in policy and guidance

o Guidance on the way the IMS should be set up to facilitate these uploads (e.g., 

necessary fields, preventing the deletion of tasks once they are complete, putting in the 

actual achieved duration at completion, use of Microsoft Project or compatible format, 

etc.)

Required Resources

No additional resources are required, however, FTE/WYE resources will be prioritized for this effort.
36 | Page

Initiative to Research
37 | Page

Enhance Implementation Indicators for Trends and Projections

Lead Executive/Organization

Office of the Chief Engineer (OCE)

Supporting Organization(s)

Office of the Chief Financial Officer; Office of Safety and Mission Assurance; Mission Directorates; 

Centers

Initiative Description

The intent of this initiative is to establish a cross-Agency working group, including outside experts, led by 

OCE to identify indicators that will advance NASA’s ability to detect emerging issues that may affect a 

project’s implementation. Actions for the working group may include, but are not limited to:

• Review the adequacy of the metrics and leading indicators (e.g., mass margin, power margin, 

etc.) that are currently used to measure the progress and health of a flight project in 

development and their effectiveness in forecasting upcoming cost and schedule challenges. This 

assessment may consider the extent to which these metrics and leading indicators anticipated 

recent schedule problems in SMD and HEOMD. 

• Discuss options and make any recommendations, if warranted, to ensure programs and projects 

in the Implementation phase achieve an adequate frequency of independent review. For 

example, are additional mid-course reviews or independent assessments needed? Do 

entrance/success criteria for reviews need to be updated?

• Review current practices of other agency, industry, or academic procedures for new state-ofthe-art practices for monitoring and controlling projects to see if they are useful for NASA 

projects.

• As part of this effort, examine how program and project managers have responded to the 

current set of metrics and leading indicators, especially those that are particularly effective in 

predicting cost and schedule growth. Are program and project managers identifying, in a timely 

manner, corrective actions or risk mitigation plans in response to predictions of technical 

maturity and cost and schedule growth? Are they addressing these predictions, tracking the 

implementation and results of any identified corrective actions and risk mitigation plans, and 

responding effectively? Are there any corrective actions and risk mitigation plans that have been 

effective in correcting cost and schedule growth trends?

The working group will present findings and any recommendations to the Associate Administrator for 

consideration of follow-on action.

Background/Current State

Because each NASA program and project is unique, it is critical that the metrics and leading indicators 

chosen for any particular project represent true indicators of that project’s performance. By monitoring 

trends in the metrics and leading indicators, program managers, project managers, systems engineers, 

program analysts, other team members and management can more accurately assess the health, 

stability, and maturity of the program or project. Consequently, problems can often be mitigated before 

they become too costly.
38 | Page

Expected Benefits

Program and project personnel expend a significant amount of time and effort to generate and maintain 

voluminous data and report progress and performance at various levels and meeting venues across the 

Agency. Agency review of programs and projects at the BPR focuses on key problem areas, but there are 

still challenges in proactively identifying and addressing emergent issues. It will be valuable to review 

the current set of metrics, leading indicators, and standards; and make the necessary modifications, 

updates, and adjustments to ensure that these efforts are effectively supporting identification of 

emergent issues. 

The NPR 7120.5E lifecycle review structure is based on the engineering reviews defined in NPR 7123.1B, 

which establish the SRR/SDR (System Definition Review), PDR, CDR, SIR, and ORR (Operational Readiness 

Review) milestones. From design, manufacturing, test, and assembly perspectives, these standard life 

cycle reviews are reasonable milestones. For projects with long gaps between formal life-cycle reviews, 

sufficient independent oversight/insight between the formal milestone reviews is critical to provide 

timely independent reports to Agency and project management. Experienced independent board 

members can often help a project find or avoid problems and mistakes that the project team is missing 

by leveraging their past lessons learned and their distance from the daily project management activities. 

The working group tasked under this initiative will assess and make recommendations as deemed 

appropriate, and whether there may be a potential benefit of additional independent assessments that 

do not place undue burden on programs and projects. 

Benchmarking and review of other government, industry and academic procedures or studies may 

reveal new ways of viewing or predicting issues with project cost and schedule performance. This may 

include development of new indicators, advanced modeling, new display capabilities, or philosophies.

Recent Accomplishments

NASA has implemented a multitude of new and evolved project performance indicators and metrics 

since the establishment of the previous 2007 Corrective Action Plan. In response to the 2011 GAO report 

(GAO-11-364R) entitled “Additional Cost Transparency and Design Criteria Needed for NASA Projects,” 

the NASA OCE undertook an effort to determine and implement a set of common metrics or leading 

indicators to assess project design stability and maturity at key points in a project’s life cycle. These 

leading indicators provide additional insight and enhance NASA’s ability to monitor and assess the 

stability of programs and projects as well as the maturity of their products through the development

process. 

Based on an OCE-led analysis via the Program and Project Management Board (PPMB), leading 

indicators and augmented entrance criteria for key lifecycle reviews have been incorporated into NASA 

policy NPR 7123.1B, NASA Systems Engineering Processes and Requirements. In addition, a common set 

of recommended programmatic and technical indicators to support trending analysis throughout the life 

cycle have been incorporated into NASA policy NPR 7120.5E, NASA Space Flight Program and Project 

Management Requirements. 

Tightly coupled programs, single-project programs, and projects develop and track programmatic and 

technical leading indicators to ensure proper progress and management. A wide range of performance 

trends are used, including trends in requirements, interfaces, verifications, reviews, software, 

problem/discrepancy reports, cost, schedule, staffing, and technical measures.
39 | Page

In summary, NASA has been generally successful in identifying and incorporating a wide range of metrics 

and leading indicators that track performance of programs and projects in the Implementation phase. 

This has led to pre-emptive corrective action in countless cases across the NASA flight project portfolio.

Planned Next Steps

OCE will convene a cross-Agency working group and plan steps listed in the Initiative Description section 

leveraging the earlier OCE analysis on leading indicators. The working group will report to the APMC on 

progress and recommendations by October 2019.

Output and Outcome Metrics

The output metrics are to produce a set of recommended or required indicators/metrics and any 

proposed changes in existing NPRs, NPDs, or handbooks. The best outcome metric would be reduced 

cost and schedule growth; however, it will be several years after implementation of any new practices 

before progress can be measured.

Interdependencies

In response to a 2011 GAO audit, NASA identified three leading indicators required to be reported by all 

programs and projects. These include mass margin, power margin, and Request for Action (RFA), or other 

means used by the program/project to track review comments (e.g., Review Item Discrepancies (RIDs) 

and/or Action Items). These three leading indicators were incorporated into NPR 7123.1B, NASA Systems 

Engineering Processes and Requirements. If NASA is considering changes or alternatives to the three 

existing required indicators as a result of this study, NASA will need to coordinate with GAO to discuss 

and come to agreement on any changes.

Impediments & Challenges

Programs and projects will be reluctant to increase the number of metrics they need to gather and 

analyze, so the working group will have to ensure that selected metrics will truly be useful in 

determining progress and predicting future issues. It is critical to avoid undue reporting burdens on 

programs and projects.

One of the biggest impediments and challenges is the episodic communication breakdown between 

different management levels that can lead to miscommunication regarding emergent issues and 

proposed solutions. Problems identified by metrics and trends need to be accepted as credible, and 

appropriate corrective actions and mitigations must be implemented in a timely manner and tracked to 

completion.

NASA needs to prioritize program analysis at all levels of the Agency. More importantly, program 

analysts must be given responsibility and support from leadership. This will enable them to focus on 

analysis and develop the necessary expertise, tools and information-gathering networks to produce 

good analysis and clear recommendations for improvement. Agency, Center and program/project 

managers need to demand excellent analysis at all levels, and must actively use the analytical products 

to make decisions.

Required Resources

A team of Subject Matter Experts from across the Agency will have to be established. The time and 

travel resources for the team will need to be supported by each SME’s respective organization. OCE 

resources will support OCE efforts and leadership for this initiative.
40 | Page

Areas of Emphasis
41 | Page

Area of Emphasis: Improve NASA’s Governance of Strategic Acquisitions

Approximately 85% of NASA’s budget is procurement. The remaining 15% of in-house work represents a 

key component of acquisition decisions and NASA Center work allocations. Each year, NASA must 

consider the acquisition approach for missions entering pre-formulation where make/buy/partner 

decisions must be made. The aerospace industrial base has become more diverse and complex in recent 

years, leading to an increase in “blended” acquisition options across international partners, an evolved 

industrial base via mergers and new entrants, NASA in-house capabilities, and public/private 

partnerships. Long-term institutional planning largely depends on the acquisition approaches that NASA 

adopts. Consequently, NASA is strengthening its acquisition strategy governance, helping to create a 

unified story for leadership on why we make the choices we do, being clear on internal outcomes and 

messaging, and ensuring consistency in the system. The formation on October 16, 2018 of the NASA 

Acquisition Strategy Council (ASC) seeks to address acquisition holistically under a single Decision 

Authority.

Prior to the ASC, acquisition planning and implementation was distributed across seven decision 

authorities, eight meeting modes, seven support staff structures, and twelve directives owners. Under 

the new governance structure, the ASC will be responsible for acquisition policy (NPD 1000.5, Policy for 

NASA Acquisition), integration, and performance; specific acquisition strategy approvals; and associated 

Agency future work planning. The ASC will be aligned with and complementary to the existing Agency 

governance architecture that includes the Executive Council, Agency Program Management Council, and 

Mission Support Council.

There are numerous benefits to the implementation of the ASC. There will be better documentation and 

communication of acquisition decisions, stronger compliance with acquisition policy, a forum to address 

strategic contractor and partner management, a forum to develop criteria for make/buy/partner, and a 

strengthened link between strategy and execution. NASA’s renewed emphasis on strategic acquisitions 

will improve the Agency’s efficacy in intelligently moving forward on large (greater than $1 Billion or 

other high profile) acquisitions and making data-driven decisions, ensuring a holistic view of the 

aerospace industrial base, international partners, and NASA in-house performance and capacity. Making 

smarter strategic acquisition decisions in the earliest stages of public dollar investment will better 

position the Agency to efficiently and effectively manage its portfolio.
42 | Page

Area of Emphasis: Risk Assessment, Requirements, and Concept Definition Early In the 

Formulation Phase

To achieve our ambitious strategic objectives, NASA needs to be a technical risk-taking agency, but also 

needs to seek out ways to reduce programmatic (cost and schedule) risk. NASA can reduce its 

programmatic risk by emphasizing its early formulation phase risk assessment and management 

capabilities, as well as its processes and practices for defining and controlling requirements and 

concepts. To improve cost and scheduling estimating, NASA will emphasize the importance of reducing 

risk and uncertainty earlier in projects and minimizing late changes in requirements and concepts. 

NASA will also emphasize the adherence to its current formulation-related documentation (e.g., 

directives, policies, requirements, procedures and handbooks) and practices, and consider whether 

changes are needed to ensure that project requirements and concepts are developed properly and as 

early as possible in the formulation phase. NASA can improve its processes so that prior to allocating 

significant funds, the agency and its stakeholders know what end objectives the project is to achieve, 

the project has developed the requirements to achieve those ends, and the best possible concept has 

been selected to meet those requirements. Areas of emphasis will include the following:

• Requirements Definition (e.g., established and flowed down, configuration controlled, clear and 

concise)

• Concept Development (e.g., assessment and alignment of alternative concepts to requirements, 

feasibility of concepts to meet requirements within margin)

• Formulation Reviews (e.g., appropriate emphasis, appropriate entrance and exit criteria, adherence 

to policy)

The issue of cost increases and schedule slips relative to formulation phase estimates is caused primarily 

by the immaturity of the concept, technical approaches, and design. The immaturity in these areas at 

this point in the life cycle may result in the use of inaccurate parameters for cost and schedule 

estimating early in projects. Cost and schedule estimates are more reliable when NASA and its suppliers 

clearly understand what needs to be procured based on an established design driven by clearly defined 

requirements.

Given that the purpose of the formulation phase is to mature the concept, technical approaches, and 

design, it is impossible to eliminate all of the inaccuracy in the parameters early in projects. However, by 

focusing more of the early formulation efforts on identifying and controlling requirements, and 

improving the iterative process of requirements and concept definition, NASA can improve the 

formulation phase and minimize the inaccuracy of cost and schedule estimating. 

This area of emphasis is intended to improve NASA’s risk management, and its requirements and 

concept development processes and practices. Any improvements made as a result should enable 

better cost and schedule estimation, as well as reduce overall costs for the agency.

The areas of emphasis outlined here may yield significant improvements, including:

• Identifying and mitigating technical risks earlier in projects.

• Identifying and mitigating fabrication/production/integration/test risks earlier in projects.

• Mitigating the risk, cost and schedule increases resulting from requirement instability, 

incompatibility, and scope creep.
43 | Page

• Ensuring the final selected concept meets requirements with margin.

• Ensuring both concepts and requirements can be matured and baselined as early as possible.

• Reducing the time required to move the project past notional cost and schedule estimates.
44 | Page

Area of Emphasis: Contractually Incentivizing High Performance

NASA will emphasize the importance of appropriately structuring contracts, evaluation periods, and 

incentive fees to adequately encourage high performance on NASA’s largest contracts moving forward. 

The Office of Procurement will emphasize best practices guidance regarding the duration of contract 

award fee evaluation periods in the formulation, implementation, and mission operations project 

lifecycle phases. For example, during formulation (Phases A and B) and operations (Phase E), an award 

fee evaluation period of 12 months may be sufficient to evaluate contractor performance. However, 

once the project enters implementation (Phases C and D), it may be more appropriate to limit the 

defined award fee evaluation period to six months for a more focused evaluation of contractor 

performance. 

The use of award and incentive fees is a critical contract management tool by which the government can 

incentivize enhanced contractor performance during the execution of a design and development-type 

contract. As an example, the Performance Evaluation Board (PEB) evaluates contractor performance 

during the pre-determined award fee evaluation period against a pre-defined set of evaluation criteria, 

resulting in a score ranging from zero to 100. The PEB then provides the resultant recommended award 

fee score to the Fee Determination Official (FDO) for consideration. The FDO takes the PEB score under 

advisement, but retains the authority to freely determine the final award fee score. The award fee score 

directly corresponds to the level of award fee granted to the contractor.

The FDO is responsible for appointing PEB membership for a contract that uses an award fee incentive. 

The Office of Procurement considers it a best practice to staff the PEB with individuals who have the 

technical expertise to accurately evaluate contractor performance and who are organizationally situated 

such that they are familiar with the ongoing activities of the contractor. The PEB brings a broader 

management perspective to the evaluation process than exists at the monitor level, and its members 

accordingly should be at a relatively high management level.

Emphasizing intelligent use of award and incentive fees will lead to improved trust that the award fee 

contract management tool is being used fairly and accurately to incentivize contractor performance on 

the Agency’s highest visibility contracts. 
45 | Page

Appendices
46 | Page

Appendix A: NASA Associate Administrator Memorandum Initiating CAP 

Effort
47 | Page
48 | Page

Appendix B: Working Group and Steering Committee Membership

A-Suite: Steve Jurczyk

A-Suite: Melanie Saunders

OCFO: Jeff DeWit

HEOMD: Bill Gerstenmaier

SMD: Thomas Zurbuchen, Sandra Connelly

ARMD: Jaiwon Shin

STMD: Jim Reuter

OCE: Joe Pellicciotti

GSFC: Steve Shinn

JSC: Debbie Korth

CAPWG Membership

Team Lead: Kevin Gilligan

HEOMD: Scott Martinelli

SMD: Gary Rawitscher

ARMD: Tony Springer

STMD: Mike Green

OCIO: Dana Mellerio

OP: Jeff Cullen

OCE: Ellen Stigberg

OCFO SID: David Walters, Charley Hunt
49 | Page

Appendix C: Development Timeline
50 | Page

Appendix D: Acronyms List

AA – Associate Administrator

ABC – Agency Baseline Commitment 

APARC – Agency Programmatic 

Analysis & Research Capability

APL – Applied Physics Laboratory 

APMC – Agency Program 

Management Council

ARMD – Aeronautics Research Mission 

Directorate 

ASC – Acquisition Strategy Council

BoE – Basis of Estimate 

BPR – Baseline Performance Review 

BSA – Business Services Assessment 

CADRe – Cost Analysis Data 

Requirement 

CAP – Corrective Action Plan 

CAPWG –Corrective Action Plan 

Working Group

CDR – Critical Design Review 

CE&A – Cost Estimation and 

Assessment 

CFO – Chief Financial Officer

CFOU – Chief Financial Officer 

University 

CJ – Congressional Justification

COO – Chief Operating Officer 

DCMA – Defense Contract 

Management Agency 

EAC – Estimate at Completion 

EGS – Exploration Ground Systems 

ERM – Enterprise Risk Management 

ESD – Exploration Systems 

Development 

EVM – Earned Value Management 

EVMS – Earned Value Management 

System

FDO – Fee Determination Official 

FTE – Full Time Equivalent 

GAO – Government Accountability 

Office 

GPRAMA – Government Performance 

and Results Act Modernization Act

GSFC – Goddard Space Flight Center 

HEOMD – Human Exploration and 

Operations Mission Directorate

IEMP – Integrated Enterprise 

Management Program

IMS – Integrated Master Schedule

IPMR – Integrated Program 

Management Report

JCL – Joint Cost and Schedule 

Confidence Level

JPL – Jet Propulsion Laboratory

JSC – Johnson Space Center

KDP – Key Decision Point 

KSC – Kennedy Space Center

LCC – Life-Cycle cost

LOE – Level of Effort

MD – Mission Directorate

MPAR – Major Program Annual Report

MSO – Mission Support Office

MSFC – Marshall Space Flight Center

NASA – National Aeronautics and Space 

Administration

NPD – NASA Policy Directive

NPR – NASA Procedural Requirements

OCE – Office of the Chief Engineer

OoE – Office of Evaluation

ORR – Operational Readiness Review

P/p – Program/project

PBR – President’s Budget Request

PDF – Professional Developmental 

Framework

PDR – Preliminary Design Review

PEB – Performance Evaluation Board

PIO – Performance Improvement Officer

PMIAA – Program Management 

Improvement and Accountability Act

PMIO – Program Management Improvement 

Officer

PP&C – Program Planning and Control

RFA – Request for Action

RFP – Request for Proposal

RMO – Resource Management Office 
SCoPe – Schedule Community of Practice

SDR – System Definition Review

SI – Schedule Initiative

SID – Strategic Investments Division

SIR – Systems Integration Review

SLS – Space Launch System

SMD – Science Mission Directorate

SME – Subject Matter Expert

SPPM&R – Strategic Planning, Performance 

Management, and Reporting Branch

SRB – Standing Review Board

SRR – System Requirements Review 

SSME – Space Shuttle Main Engine

STMD – Space Technology Mission 

Directorate

SwRI – Southwest Research Institute

TRA – Technology Readiness Assessment

TRL – Technology Readiness Level

UFE – Unallocated Future Expenses

WYE – Work Year Equivalent