DOD BUSINESS 

SYSTEMS 

MODERNIZATION

Air Force Business 

System Schedule and 

Cost Estimates

Report to Congressional Requesters

February 2014

GAO-14-152

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-152, a report to 

congressional requesters

February 2014

DOD BUSINESS SYSTEMS MODERNIZATION

Air Force Business System Schedule and Cost 

Estimates 

Why GAO Did This Study

The Department of Defense (DOD) has 

stated that the development and 

implementation of DEAMS is critical to 

the department’s goal of producing 

auditable financial statements by 

September 2017. In October 2010, 

GAO reported that although the Air 

Force had developed a cost estimate 

that met best practices, it had not 

developed a schedule that met best 

practices for implementing DEAMS. 

GAO has published guides that identify 

the characteristics and associated best 

practices for developing reliable 

schedule and cost estimates. GAO 

was asked to review the schedule and 

cost estimates for selected DOD

systems. This report addresses the 

extent to which the current schedule 

and cost estimates for DEAMS were 

prepared in accordance with GAO’s 

Schedule and Cost Guides. 

Specifically, GAO’s review focused on 

the schedule and cost estimates that 

supported DOD’s February 2012 

Milestone B decision, which 

determined that investment in DEAMS 

was justified. GAO assessed the 

schedule and cost estimates and 

supporting documentation. GAO also

assessed an updated schedule dated 

October 2012. GAO interviewed 

DEAMS program officials, lead 

schedulers, and cost estimators. 

What GAO Recommends

GAO recommends that the Secretary 

of the Air Force update the cost

estimate as necessary after 

implementing GAO’s prior 

recommendation to adopt scheduling 

best practices. DOD concurred with the

recommendation.

What GAO Found

The Air Force’s schedule that supported the February 2012 Milestone B decision

for the Defense Enterprise Accounting and Management System (DEAMS) did 

not meet best practices. The cost estimate did meet best practices, but the 

issues associated with the schedule could negatively affect the cost estimate. 

GAO found that the schedule supporting the Air Force’s decision to invest in 

DEAMS partially or minimally met the four characteristics for developing a highquality and reliable schedule. For example, the schedule did not reflect all 

government and contractor activities, and resources were not assigned to 

specific activities. The schedule also lacked a valid critical path, preventing 

management from focusing on the activities most likely to cause critical program 

delays if they are not completed as planned. In addition, a schedule risk analysis 

was not conducted to predict a level of confidence in meeting the program’s 

completion date. 

Extent DEAMS Schedule Met Best Practices

Characteristic Assessment

Comprehensive Partially met

Well-constructed Partially met

Credible Minimally met

Controlled Partially met

Source: GAO analysis based on information provided by the Air Force.

GAO found that the October 2012 updated schedule estimate was not 

comprehensive, well-constructed, and credible, and contained weaknesses 

similar to those found in the previous schedule. In May 2013, program officials 

provided a third schedule that they said contained some improvements but 

acknowledged that issues remained that prevented the schedule from meeting 

best practices. 

GAO found that the DEAMS cost estimate fully or substantially met the four 

characteristics of a high-quality and reliable cost estimate. For example, the cost 

estimate included both government and contractor costs for the program over its 

life cycle and provided for an independent assessment and reconciliation. 

Because the cost estimate relies on dates derived from the schedule and GAO is 

questioning the reliability of the schedule, the credibility of the cost estimate 

could be affected. 

Extent DEAMS Cost Estimate Met Best Practices

Characteristic Assessment

Comprehensive Fully met

Well-documented Substantially met

Accurate Substantially met

Credible Substantially met

Source: GAO analysis based on information provided by the Air Force. 

View GAO-14-152. For more information, 

contact Asif A. Khan at (202) 512-9869 or 

khana@gao.gov and Nabajyoti Barkakati at 

(202) 512-4499 or barkakatin@gao.gov. 
Page i GAO-14-152 DOD Business Systems Modernization

Letter 1 

Scope and Methodology 2

Background 3

DEAMS Program Schedule Did Not Meet Best Practices, but the 

Cost Estimate Did Meet Best Practices 6

Conclusions 16

Recommendation for Executive Action 16

Agency Comments 16

Appendix I Assessment of the Defense Enterprise Accounting and Management 

System Program’s Schedule Estimate 19

Appendix II Assessment of the Defense Enterprise Accounting and Management 

System Program’s Cost Estimate 25

Appendix III Comments from the Department of Defense 28

Appendix IV GAO Contacts and Staff Acknowledgments 30

Tables

Table 1: Extent DEAMS Schedule Met Best Practices 7

Table 2: Extent DEAMS Cost Estimate Met Best Practices 14

Table 3: Assessment of DEAMS Schedule Estimate Compared to 

Best Practices 20

Table 4: Assessment of DEAMS Cost Estimate Compared to Best 

Practices 25

Contents
Page ii GAO-14-152 DOD Business Systems Modernization

Abbreviations

DEAMS Defense Enterprise Accounting and Management 

System

DOD Department of Defense

ERP enterprise resource planning

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-152 DOD Business Systems Modernization

441 G St. N.W.

Washington, DC 20548

February 7, 2014

Congressional Requesters

The Department of Defense (DOD) invests billions of dollars to develop 

and implement enterprise resource planning (ERP) systems, which it 

considers critical to transforming the department’s business operations 

and addressing some of its long-standing weaknesses, including those 

related to financial management and business systems modernization.1

DOD has stated that the development and implementation of the Air 

Force’s Defense Enterprise Accounting and Management System 

(DEAMS) is critical to the department’s goal of producing auditable 

financial statements by September 2017,2 as called for by the National 

Defense Authorization Act for Fiscal Year 2010.3

In October 2010, we reported that although the Air Force met best 

practices in developing a cost estimate, it did not meet best practices in 

developing the schedule estimate for implementing DEAMS.4 Having 

such a schedule is crucial to the Air Force’s ability to reliably estimate the 

program completion date.

To support Congress’s continuing oversight of DOD’s progress in 

implementing its ERP systems, you asked us to review the schedule and 

cost estimates for selected DOD ERP systems. The objective of this 

review was to determine the extent to which the current schedule and 

cost estimates for DEAMS were prepared in accordance with GAO’s 

 

1

An ERP system is an automated system using commercial off-the-shelf software 

consisting of multiple, integrated functional modules that perform a variety of businessrelated tasks, such as general ledger accounting, payroll, and supply chain management. 

2

For a complete list of the ERP systems that DOD has identified as essential to 

transforming its business operations, see GAO, DOD Financial Management: Reported 

Status of Department of Defense’s Enterprise Resource Planning Systems, GAO-12-565R

(Washington, D.C.: Mar. 30, 2012).

3

Pub. L. No. 111-84, § 1003(a), (b), 123 Stat. 2190, 2439-40 (Oct. 28, 2009). 

4

GAO, DOD Business Transformation: Improved Management Oversight of Business 

System Modernization Efforts Needed, GAO-11-53 (Washington, D.C.: Oct. 7, 2010).
 

Page 2 GAO-14-152 DOD Business Systems Modernization

Schedule and Cost Guides.5 We reviewed the most current schedule and 

cost estimates that supported DOD’s February 2012 Milestone B 

decision, which determined that investment in DEAMS was justified.6

We assessed the DEAMS schedule7 using the GAO Schedule Guide to 

determine whether it was comprehensive, well-constructed, credible, and 

controlled.8 To assess the schedule, we obtained and reviewed 

documentation, including the integrated master plan and work breakdown 

structure.9 In assessing the program’s cost estimate, we used the GAO 

Cost Guide to evaluate the DEAMS Program Management Office’s 

estimating methodologies, assumptions, and results to determine whether 

the cost estimate was comprehensive, well-documented, accurate, and 

credible. We obtained and reviewed documentation, including the 

program office estimate, software cost model, independent cost estimate, 

 

5

GAO, GAO Schedule Assessment Guide: Best Practices for Project Schedules—

Exposure Draft, GAO-12-120G (Washington, D.C.: May 2012), and GAO Cost Estimating 

and Assessment Guide: Best Practices for Developing and Managing Capital Program 

Costs, GAO-09-3SP (Washington, D.C.: March 2009). The guides document the specific 

characteristics and associated best practices for preparing schedule and cost estimates.

6

Milestone B is considered the official start of the program according to the Defense 

Acquisition Management System Framework. The framework is intended to translate 

mission needs and requirements into stable, affordable, and well-managed acquisition 

programs. 

7

Although the Air Force provided us with its integrated master schedule for the DEAMS 

program, we determined that it did not meet the characteristics of an integrated master 

schedule in accordance with GAO’s Schedule Guide. Specifically, an integrated master 

schedule is a document that integrates the planned work by the government, contractor, 

and other key parties; the resources necessary to accomplish that work; and the 

associated budget. Therefore, for purposes of this report, we use “schedule” to refer to the 

Air Force’s integrated master schedule.

8

A schedule is controlled if it is updated periodically by trained schedulers to realistically 

forecast dates for program activities and compared against a designated baseline 

schedule to measure, monitor, and report the project’s progress.

9

An integrated master plan provides an event-based hierarchy of program events, with 

each event supported by accomplishments and each accomplishment associated with 

specific criteria to be satisfied for its completion. The plan is normally part of the contract 

and is therefore contractually binding. A program’s work breakdown structure defines in 

detail the work necessary to accomplish a program’s objectives, including activities to be 

performed by both the government and contractors.

Scope and 

Methodology
 

Page 3 GAO-14-152 DOD Business Systems Modernization

and risk and uncertainty analysis.10 We also interviewed key program 

officials, such as the Program Manager, lead schedulers, and cost 

estimators, to obtain information, such as explanations to resolve 

identified discrepancies.

After we briefed DEAMS program officials on the results of our 

assessment, they provided an updated schedule dated October 2012. For 

this updated schedule, we determined the extent to which it met certain 

best practices for the comprehensive, well-constructed, and credible 

characteristics, because not implementing these best practices would 

affect the reliability of the entire schedule. In May 2013, program 

management officials provided another updated DEAMS schedule, which 

they acknowledged contained issues that prevented the schedule from 

meeting best practices. Although we did not independently assess the 

May 2013 schedule, we did confirm that it included certain information 

needed for long-term planning.

We conducted this performance audit from May 2012 to February 2014 in 

accordance with generally accepted government auditing standards.

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

DEAMS was initiated in August 2003 and is intended to provide the Air 

Force with the entire spectrum of financial management capabilities, 

including collections, commitments and obligations, cost accounting, 

general ledger, funds control, receipt and acceptance, accounts payable 

and disbursement, billing, and financial reporting for the general fund.11 In 

 

10An independent cost estimate is another estimate based on the same technical 

information that is used to validate and cross-check the cost estimate, but is prepared by a 

person or organization that has no stake in the approval of the program. A risk and 

uncertainty analysis recognizes the potential for error and attempts to quantify it by

identifying the effects of changing key cost drivers (e.g., costs associated with personnel 

or software development).

11An agency’s general fund accounts are those accounts in the U.S. Treasury holding all 

federal money administered by an agency that is not allocated by law to any other fund 

account. 

Background
 

Page 4 GAO-14-152 DOD Business Systems Modernization

February 2012, the DOD Deputy Chief Management Officer granted 

Milestone B approval for DEAMS to enter the Engineering Development 

Phase of the acquisition life cycle, which is considered the official start of 

the program. DEAMS program functionality is intended to be implemented 

across the Air Force in a series of releases in two increments—Increment 

1 will include six releases and Increment 2 will include two releases.12

DOD has approved the funding for the Air Force to proceed with the 

acquisition of the functionality for the first increment of DEAMS. This 

funding is approximately $1.6 billion, with deployment scheduled to occur 

during the fourth quarter of fiscal year 2016. The Air Force reported that it 

had spent about $427.5 million as of September 30, 2013, on the 

program.

As stated earlier, in October 2010, we reported that although the Air 

Force met best practices in developing a cost estimate, it did not meet 

best practices in developing the schedule estimate for implementing 

DEAMS.13 In particular, the Air Force had not developed a fully integrated 

master schedule that reflected all government and contractor activities. 

We recommended that the Air Force develop an integrated master 

schedule that fully incorporated best practices, such as capturing all 

activities, sequencing all activities, integrating activities horizontally and 

vertically,14 establishing the critical path for all activities, identifying float 

between activities,15 conducting a schedule risk analysis, and updating 

the schedule using logic and durations to determine dates. DOD 

concurred with our recommendation, and we discuss later in this report 

the status of DOD’s efforts to address this recommendation.

 

12An increment is a useful and supportable operational capability that can be developed, 

tested, produced, deployed, and sustained. It may consist of multiple capability releases to 

facilitate delivery of the system. 

13GAO-11-53. 

14A horizontally integrated schedule links products and outcomes with other associated 

sequenced activities, and a vertically integrated schedule ensures that the start and 

completion dates for activities are aligned with such dates on subsidiary schedules 

supporting tasks and subtasks. 

15Float is the amount of time by which a predecessor activity can slip before the delay 

affects the program’s estimated finish date.
 

Page 5 GAO-14-152 DOD Business Systems Modernization

In March 2009, we published the Cost Guide to address a gap in federal 

guidance about processes, procedures, and practices needed for 

ensuring reliable cost estimates.16 The Cost Guide provides a consistent 

methodology based on best practices that can be used across the federal 

government to develop, manage, and evaluate capital program cost 

estimates. The methodology is a compilation of characteristics and 

associated best practices that federal cost estimating organizations and 

industry use to develop and maintain reliable cost estimates throughout 

the life of an acquisition program.

In May 2012, we issued the Schedule Guide as a companion to the Cost 

Guide.17 A consistent methodology for developing, managing, and 

evaluating capital program cost estimates includes the concept of 

scheduling the necessary work to a timeline, as discussed in the Cost 

Guide. Simply put, schedule variances are usually followed by cost 

variances. Because some program costs, such as labor, supervision, 

rented equipment, and facilities, cost more if the program takes longer, a 

reliable schedule can contribute to an understanding of the cost impact if 

the program does not finish on time. In addition, management tends to 

respond to schedule delays by adding more resources or authorizing 

overtime. Further, a schedule risk analysis allows for program 

management to account for the cost effects of schedule slippage when 

developing the life-cycle cost estimate. A cost estimate cannot be 

considered fully credible if it does not account for the cost effects of 

schedule slippage.

A well-planned schedule is a fundamental management tool that can help 

government programs use public funds effectively by specifying when 

work will be performed in the future and measuring program performance 

against an approved plan. Moreover, as a model of time, an integrated 

and reliable schedule can show when major events are expected to occur 

as well as the completion dates for all activities leading up to them, which 

can help determine if the program’s parameters are realistic and 

achievable. A program’s success depends in part on the quality of its 

schedule.

 

16GAO-09-3SP. 

17GAO-12-120G. 

GAO’s Cost and Schedule 

Guides
 

Page 6 GAO-14-152 DOD Business Systems Modernization

We found that the schedule for the DEAMS program did not meet best 

practices. The cost estimate did meet best practices, but the issues 

associated with the schedule could negatively affect the cost estimate. 

Specifically, the DEAMS schedule supporting the February 2012 

Milestone B decision partially or minimally met the four characteristics for 

developing a high-quality and reliable schedule—it was not 

comprehensive, well-constructed, credible, or controlled. In addition, our 

assessment of the October 2012 updated schedule found that it was not 

comprehensive, well-constructed, and credible and thus was also not 

reliable.18 In contrast, the DEAMS cost estimate fully or substantially met 

the four characteristics of a high-quality and reliable cost estimate—it was 

comprehensive, well-documented, accurate, and credible. However, 

because the cost estimate is based on the schedule, the unreliability of 

the schedule could affect the cost estimate. For example, if there are 

schedule slippages, the costs for the program could be greater than 

currently estimated.

Our analysis found that the DEAMS program partially met three and 

minimally met one of the characteristics of a reliable schedule estimate 

and therefore did not provide the information needed to support the 

February 2012 Milestone B decision (see table 1). Appendix I contains 

our detailed analysis of the DEAMS schedule estimate. The success of 

any program depends on having a reliable schedule of the program’s 

work activities that will occur, how long they will take, and how the 

activities are related to one another. As such, the schedule not only 

provides a roadmap for systematic execution of a program, but also 

provides the means by which to gauge progress, identify and address 

potential problems, and promote accountability.

 

18We did not assess the extent to which the updated October 2012 schedule was 

controlled.

DEAMS Program 

Schedule Did Not 

Meet Best Practices, 

but the Cost Estimate 

Did Meet Best 

Practices

Program Schedule Was 

Not Developed in 

Accordance with Best 

Practices
 

Page 7 GAO-14-152 DOD Business Systems Modernization

Table 1: Extent DEAMS Schedule Met Best Practices

Characteristic Assessment

Comprehensive Partially met

Well-constructed Partially met

Credible Minimally met

Controlled Partially met

Source: GAO analysis based on information provided by the Air Force.

Note: GAO’s methodology includes five levels of compliance with its best practices. “Not met” means 

the program provided no evidence that satisfies any of the criterion. “Minimally met” means the 

program provided evidence that satisfies a small portion of the criterion. “Partially met” means the 

program provided evidence that satisfies about half of the criterion. “Substantially met” means the 

program provided evidence that satisfies a large portion of the criterion. “Fully met” means the 

program provided evidence that completely satisfies the criterion.

Comprehensive. A schedule should reflect all activities as defined in the 

program’s work breakdown structure, including activities to be performed 

by the government and the contractor; the resources (e.g., labor, 

materials, and overhead) needed to do the work; and how long each 

activity will take.

We found that the schedule used to support the Milestone B decision 

included the activities to be performed by both the government and 

contractor for Releases 1 through 3 of Increment 1. However, the 

schedule did not reflect activities to be performed for Releases 4 through 

6 of Increment 1 and for Releases 1 and 2 of Increment 2. The DEAMS 

Program Manager stated that a comprehensive schedule for Increment 1 

that included the activities for all six releases would not be completed until 

mid-2014. The Program Manager also stated that Increment 2 had not 

been included because program officials did not know the detailed 

activities to be performed that far in advance. To address this issue, the 

DEAMS program office developed a roadmap depicting Releases 1 

through 6 of Increment 1 and Releases 1 and 2 of Increment 2 with a full 

deployment date of fiscal year 2017.19 However, the program office did 

not provide a schedule that supported the estimated dates in the 

roadmap.

 

19A roadmap is a planning document that briefly outlines the program’s key phases 

(increments and releases) and the expected milestones for completion. A roadmap does 

not include specific activities or a detailed work breakdown structure, which would be 

included in the schedule that is prepared to support the roadmap. 
 

Page 8 GAO-14-152 DOD Business Systems Modernization

A comprehensive schedule should reflect all of a program’s activities and 

recognize that uncertainties and unknown factors in schedule estimates 

can stem from, among other things, data limitations. As such, a schedule 

incorporates different levels of detail depending on the information 

available at any point in time. That is, near-term effort will be planned in 

greater detail than long-term effort. Effort beyond the near term that is 

less well defined is represented within the schedule as long-term planning 

packages. Planning packages are a summarization of the work to be 

performed in the distant future with less specificity. Planning packages 

are planned at higher levels such that a single activity may represent 

several months of effort, generic work to be accomplished, or even a 

future contract or phase. Planning packages can be used as long as they 

are defined and estimated as well as possible. By not including all work 

for all deliverables for both increments and all releases, the DEAMS 

program could incur difficulties resulting from an incomplete 

understanding of the plan and what constitutes a successful conclusion 

for the program. DEAMS program officials provided a draft of the 

Schedule Management Plan that documented their intent to use a 

planning package approach when updating the DEAMS schedule in the 

future.

Resources were identified in the schedule; however, the resources were 

not assigned to specific activities in the schedule. Although our analysis 

determined that activity durations were manageable and reasonably 

estimated, resource availability affects estimates of work and its duration, 

as well as resources that will be available for subsequent activities. 

DEAMS program management officials told us that government resource 

allocations are determined by management as needed. These officials 

told us that management does not necessarily take into consideration the 

resource information captured in the schedule when determining resource 

allocations. However, DEAMS officials did not provide any documentation 

that specific resources were being mapped to the schedule. As 

mentioned above, the estimates of work required and duration for an 

activity are tied to the availability of resources; therefore, the lack of such 

information could hinder management’s ability to compute total labor and 

equipment hours, calculate total project and per-period cost, resolve 

resource conflicts, and establish the reasonableness of the plan.

Well-constructed. A schedule should be planned so that critical project 

dates can be met. To meet this objective, all activities should be logically 

sequenced—that is, listed in the order in which they are to be carried out. 

In particular, activities that must finish prior to the start of other activities 

(i.e., predecessor activities), as well as activities that cannot begin until 
 

Page 9 GAO-14-152 DOD Business Systems Modernization

other activities are completed (i.e., successor activities), should be 

identified and their relationships established. The establishment of a 

critical path is necessary for examining the effects of any activity slipping 

along this path. The calculation of a critical path determines which 

activities drive the project’s earliest completion date. The schedule should 

also identify total float so that the schedule’s flexibility can be accurately 

determined.

We found that the majority of logic used to sequence the activities within 

the schedule was generally error-free with a minimal use of lags,20 clearly 

indicating to program management the order of activities that must be 

accomplished. Although we found few missing logic relationships for 

Release 3 of Increment 1,21 approximately 25 percent of the remaining 

activities for Releases 1 and 2 of Increment 1 were missing logic 

relationships. Because interdependencies among activities were not 

identified, the DEAMS program management officials’ ability to properly 

calculate dates and predict changes in the future is impaired. We found a 

significant number of constraints for activities throughout the schedule.22

A schedule is intended to be a dynamic, proactive planning and risk 

mitigation tool that models the program and can be used to track progress 

toward important milestones. Schedules with constrained dates can 

portray an artificial or unrealistic view of the project. Constraints should be 

minimized because they can create false dates in a schedule.

Further, the schedule did not have a valid critical path and identified 

critical activities more by their constraints than by logic. Rather than 

relying on constraints, the schedule should use logic and durations in 

order to reflect realistic start and completion dates for activities. 

Successfully identifying the critical path relies on several factors, such as 

capturing all activities, properly sequencing activities, and assigning 

resources, which, as noted earlier, had not been done. Without a valid 

 

20A lag in a schedule denotes the passage of time between two activities. Lags have a 

specific use in scheduling but may be misused to force activities to begin on specific 

dates.

21The purpose of a logic relationship, or dependency, is to depict the sequence in which 

activities occur. Such relationships state when activities are planned to start and finish in 

relation to the start and finish of other activities. A logic relationship therefore models the 

effect of an on-time, delayed, or accelerated activity on subsequent activities.

22A constraint predefines the start, finish, or both dates of an activity. 
 

Page 10 GAO-14-152 DOD Business Systems Modernization

critical path, management cannot focus on activities that will have 

detrimental effects on the key project milestones and deliveries if they 

slip.

We found that total float was not reasonable, and that in some instances 

unreasonable float was a direct result of improper sequencing or missing 

logic. Releases 1 and 2 of Increment 1 showed that 25 percent of 

program activities had total float equal to or greater than 392 working 

days, meaning that those activities could slip almost 2 working years and 

not affect the end date of the program. Without knowledge of the reason 

float exists for a program activity, management cannot determine the 

flexibility of tasks and therefore cannot properly reallocate resources from 

tasks that can safely slip to tasks that cannot slip without adversely 

affecting the estimated program completion date.

Credible. A schedule should be horizontally and vertically integrated. A 

horizontally integrated schedule links products and outcomes with other 

associated sequenced activities, which helps verify that activities are 

arranged in the right order to achieve aggregated products or outcomes. 

A vertically integrated schedule ensures that the start and completion 

dates for activities are aligned with such dates on subsidiary schedules 

supporting tasks and subtasks. Such mapping or alignment among 

subsidiary schedules enables different groups—such as government 

teams and contractors—to work to the same master schedule, and 

provides assurance that the representation of the schedule to different 

audiences is consistent and accurate. A schedule risk analysis should 

also be performed using statistical techniques to predict the level of 

confidence in meeting a program’s completion date.

We found that Release 3 of Increment 1 exhibited horizontal integration, 

but Releases 1 and 2 of Increment 1 did not because date constraints 

prevented forecasted dates from being calculated realistically for future 

activities. If the schedule lacks horizontal integration, activities whose 

durations are greatly extended will have no effect on key milestones 

reflected in the schedule. We further found that Releases 1 and 2 of 

Increment 1 did not demonstrate vertical integration. For example, we 

found instances where the start dates for the same activities differed by 1 

day, 1 week, and 1 month between the government and contractor 

schedules. Unless the schedule is vertically integrated, lower-level 

schedules will not be consistent with upper-level schedule milestones, 

affecting the integrity of the entire schedule and the ability of different 

teams to work to the same schedule expectations.
 

Page 11 GAO-14-152 DOD Business Systems Modernization

DEAMS program management officials stated that a schedule risk 

analysis had not been conducted because the schedule had not been 

approved to be used as a baseline schedule—the target schedule against 

which program performance can be measured, monitored, and reported. 

These officials stated that although this analysis had not been conducted, 

they were collecting best-case and worst-case durations from the 

contractor with their periodic schedule delivery. These data can be used 

by program management to calculate more reliable estimates of durations 

for future activities. However, we found that the schedule did not contain 

best- or worst-case duration data for 600 of 605 detailed activities. For the 

five instances where duration data were contained in the schedule, we 

determined that four were questionable because two activities were 

already completed and two had already exceeded the worst-case 

estimate. If a schedule risk analysis is not conducted, program 

management cannot determine the likelihood of the project’s completion 

date, how much schedule risk contingency is needed to provide an 

acceptable level of certainty for completion by a specific date, risks most 

likely to delay the project, how much contingency reserve each risk 

requires, and the paths or activities that are most likely to delay the 

project. As discussed later, the lack of a schedule risk analysis can affect 

the credibility of the cost estimate.

Controlled. A schedule should be continuously updated using logic, 

durations, and actual progress to realistically forecast dates for program 

activities. A schedule narrative should accompany the updated schedule 

to provide decision makers and auditors a log of changes and their effect, 

if any, on the schedule time frame. The schedule should be analyzed 

continuously for variances to determine when forecasted completion 

dates differ from planned dates. This analysis is especially important for 

those variations that affect activities identified as being in a program’s 

critical path and that can affect a scheduled completion date. A baseline 

schedule should be used to manage the program scope, the time period 

for accomplishing it, and the required resources.

We found that DEAMS program management met weekly to discuss 

proposed schedule changes and updated the schedule’s progress. 

However, a schedule narrative was not prepared by DEAMS program 

management. In addition, we found a number of date anomalies 

throughout the schedule, including activities with planned start dates 

scheduled to occur in the past and activities with actual finish dates 

scheduled to occur in the future. We also found a number of out-ofsequence activities in the schedule—activities that started before their 

predecessors finished, in contradiction to the planned sequence. If the 
 

Page 12 GAO-14-152 DOD Business Systems Modernization

schedule is not continually monitored to determine when forecasted 

completion dates differ from planned dates, then it cannot be used to 

determine whether schedule variances will affect work needed to be 

accomplished at a future date.

We also found that there was no baseline schedule that could be used to 

measure program performance. DEAMS program management officials 

did maintain a schedule narrative document that contained a list of 

custom fields and assumptions; however, the document did not explain 

ground rules and assumptions, justifications for logic, and other unique 

features of the schedule. These officials stated that other process 

documents were being developed. Without a formally established 

baseline schedule to measure performance against, management cannot 

identify or mitigate the effect of unfavorable performance.

Our assessment of the updated schedule dated October 2012 found that 

it was not comprehensive, well-constructed and credible.23 Although the 

DEAMS Program Manager stated that the government and contractor 

activities for Releases 1 through 3 of Increment 1 had been integrated in 

the October 2012 schedule, this schedule was not comprehensive. 

Specifically, it excluded activities for both the government and contractor 

related to Releases 4 through 6 of Increment 1 and Releases 1 and 2 of 

Increment 2. If activities are missing from the schedule, then other best 

practices will not be met. The schedule was also missing relationships for 

a significant number of the remaining milestones and activities. In 

addition, the October 2012 schedule included a significant number of date 

constraints with little or no justification for their use in the schedule. 

Similar to the previous schedule, the updated schedule presented 

unreasonable float throughout and did not include a schedule risk 

analysis. As a result of these shortcomings, the updated schedule was 

not reliable. Further, program officials could not rely on this schedule as a 

baseline to effectively manage and monitor program performance.

In May 2013, program management officials provided another updated 

DEAMS schedule that they stated included some improvements, but they 

acknowledged that it contained issues that prevented the schedule from 

 

23Program officials stated that the updated schedule represented the DEAMS program 

baseline as of October 2012. The updated schedule included the period May 2, 2011, to 

December 17, 2015, and encompassed Releases 1 through 3 of Increment 1. 

Updated Schedule Also 

Did Not Follow Best 

Practices
 

Page 13 GAO-14-152 DOD Business Systems Modernization

meeting best practices. For example, these officials stated that the May 

2013 schedule included long-term planning packages for activities related 

to Releases 4 through 6 of Increment 1 and Releases 1 and 2 of 

Increment 2, integrated government and contractor activities, and 

reduced the number of constraints and out-of-sequence activities in the 

schedule. However, the officials acknowledged that several outstanding 

issues remained related to, for example, vertical and horizontal 

integration, missing logic relationships, and the lack of a schedule risk 

analysis. Although we did not independently assess the May 2013 

schedule to determine whether it met the four schedule characteristics, 

we did confirm that it included long-term planning packages, which are 

needed to create a complete picture of the program from start to finish 

and to allow the monitoring of a program’s critical path. The results of our 

analyses of the schedule that supported the February 2012 Milestone B 

decision and October 2012 DEAMS schedule reflect similar weaknesses 

to those we reported in October 2010.24 Therefore, given the findings of 

this review, our prior recommendation for improving the DEAMS schedule 

remains valid.

We found that the DEAMS program fully or substantially met the four 

characteristics of a reliable cost estimate to support the Milestone B 

decision, as shown in table 2. However, because the cost estimate relies 

on dates derived from the schedule and we are questioning the reliability 

of the forecasted program dates, the credibility of the cost estimate can 

be affected. Appendix II contains our detailed analysis of the DEAMS cost 

estimate. A reliable cost estimate is critical to the success of any 

program. Such an estimate provides the basis for informed investment 

decision making, realistic budget formulation and program resourcing, 

meaningful progress measurement, proactive course correction when 

warranted, and accountability for results.

 

24GAO-11-53. 

Program Cost Estimate 

Met Best Practices
 

Page 14 GAO-14-152 DOD Business Systems Modernization

Table 2: Extent DEAMS Cost Estimate Met Best Practices 

Characteristic Assessment

Comprehensive Fully met

Well-documented Substantially met

Accurate Substantially met

Credible Substantially met

Source: GAO analysis based on information provided by the Air Force.

Note: GAO’s methodology includes five levels of compliance with its best practices. “Not met” means 

the program provided no evidence that satisfies any of the criterion. “Minimally met” means the 

program provided evidence that satisfies a small portion of the criterion. “Partially met” means the 

program provided evidence that satisfies about half of the criterion. “Substantially met” means the 

program provided evidence that satisfies a large portion of the criterion. “Fully met” means the

program provided evidence that completely satisfies the criterion.

Comprehensive. A cost estimate should include costs of the program over 

its full life cycle, provide a level of detail appropriate to ensure that cost 

elements are neither omitted nor double-counted, and document all costinfluencing ground rules and assumptions. The cost estimate should also 

completely define the program and be technically reasonable.

We found that the cost estimate for DEAMS was comprehensive. The 

cost estimate included both government and contractor costs of the 

program over its life cycle—from the inception of the program through 

design, development, deployment, and operation and maintenance—as 

outlined in the roadmap prepared by program officials. As stated earlier, 

the roadmap provided an overall summary of the program’s key phases 

(increments and releases) and the expected milestones for completion. 

The cost estimate also included an appropriate level of detail, which 

provided assurance that cost elements were neither omitted nor 

duplicated, and included documentation of all cost-influencing ground 

rules and assumptions. The cost estimate documentation included the 

purpose of the cost estimate, a technical description of the program, and 

technical risks (e.g., the resolution for any identified deficiencies).

Well-documented. A cost estimate should be supported by detailed 

documentation that describes how it was derived and how the expected 

funding will be spent in order to achieve a given objective. Therefore, the 

documentation should capture such things as the source data used, the 

calculations performed, the results of the calculations, the estimating 

methodology used to derive each work breakdown structure element’s 

cost, and evidence that the estimate was approved by management.
 

Page 15 GAO-14-152 DOD Business Systems Modernization

We found that the cost estimate for DEAMS was well-documented. The 

cost estimate captured such things as the source data used, the 

calculations performed and the results of the calculations, and the 

rationale for choosing a particular estimating methodology. This 

information was captured in such a way that the data used to derive the 

estimate can be traced back to, and verified against, the sources so that 

the estimate can be easily replicated. However, there was no discussion 

of efforts taken, if any, to ensure the reliability of the data used. The 

DEAMS Program Management Office presented evidence of receiving 

approval of the estimate through briefings to management.

Accurate. A cost estimate should be based on an assessment of most 

likely costs (adjusted properly for inflation), updated to reflect significant 

changes and grounded in a historical record of cost estimating and actual 

experiences on other comparable programs.

We found that the cost estimate for DEAMS was accurate. The cost 

estimate provided results that were substantially unbiased, and the cost 

model detailed the calculations and inflation indexes underlying the 

estimate. Calculations within the model could be traced back to 

supporting documentation. The cost estimate was updated regularly to 

reflect significant changes in the program and updated annually to 

incorporate actual costs expended in prior fiscal years. Further, the cost 

estimate was based on historical data. However, the cost estimate did not 

discuss variances between planned and actual costs, which would enable 

estimators to assess how well they are estimating program costs and to 

identify lessons learned.

Credible. A cost estimate should discuss any limitations of the analysis 

because of uncertainty or biases surrounding data or assumptions. In 

addition, the estimate’s results should be cross-checked and reconciled to 

an independent cost estimate to determine whether other estimating 

methods produce similar results.

We found that the cost estimate was credible. The DEAMS Program 

Management Office conducted a risk and uncertainty analysis by 

identifying the cost elements with the greatest degree of uncertainty, 

determining the cost drivers for the program, and identifying the impact of 

changing major ground rules and cost driver assumptions. An 

independent cost estimate developed by the Air Force Cost Analysis 

Agency was reconciled to the program’s estimate. However, a sensitivity 

analysis was not completed for each of the major cost drivers. As a result, 

the cost estimator will not have a clear understanding of how each major 
 

Page 16 GAO-14-152 DOD Business Systems Modernization

cost driver is affected by a change in a single assumption and thus which 

scenario most affects the cost estimate. Further, as discussed previously, 

because a schedule risk analysis was not performed as required by best 

practices, the cost estimate does not include a contingency amount to 

account for any schedule slippage that could occur. To the extent that a 

schedule slippage does occur, there could ultimately be an impact on the 

cost estimate.

The Air Force did not meet best practices in developing a schedule for the 

DEAMS program. As a result, this raises questions about the credibility of 

the deadline for acquiring and implementing DEAMS to provide needed 

functionality for financial improvement and audit readiness. Because of 

these questions, the cost estimate, while following best practices, may not 

fully capture all costs associated with the program, particularly if there is 

significant schedule slippage. Moreover, Air Force management did not 

have a reliable schedule estimate when making its decision to invest in 

the DEAMS program. It is critical to correct the deficiencies identified with 

the schedule estimate to help ensure that the projected spending for this 

program is being used in the most efficient and effective manner.

To help provide for the successful implementation of DEAMS, we 

recommend that the Secretary of the Air Force direct the Under Secretary 

of the Air Force, in his capacity as the Chief Management Officer, to 

consider and make any necessary adjustments to the DEAMS cost 

estimate after addressing our prior recommendation to adopt scheduling 

best practices.

We provided a draft of this report to DOD for review and comment. In its 

written comments, reprinted in appendix III, DOD concurred with our 

recommendation. DOD also provided a technical comment, which we

incorporated.

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to the appropriate

congressional committees, the Secretary of Defense; the Secretary of the 

Air Force; the Assistant Secretary of Defense (Acquisition); the Deputy 

Chief Management Officer; the Under Secretary of Defense (Comptroller); 

the Under Secretary of the Air Force, in his capacity as the Chief 

Conclusions

Recommendation for 

Executive Action

Agency Comments 
 

Page 17 GAO-14-152 DOD Business Systems Modernization

Management Officer of the Air Force; and the Program Manager for 

DEAMS. In addition, the report will be available at no charge on the GAO 

website at http://www.gao.gov. 

If you or your staffs have any questions about this report, please contact 

Asif A. Khan at (202) 512-9869 or khana@gao.gov or Nabajyoti Barkakati 

at (202) 512-4499 or barkakatin@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff members who made key contributions to 

this report are listed in appendix IV.

Asif A. Khan

Director

Financial Management and Assurance

Nabajyoti Barkakati

Chief Technologist

Applied Research and Methods

Center for Science, Technology, and Engineering
 

Page 18 GAO-14-152 DOD Business Systems Modernization

List of Requesters

The Honorable Thomas R. Carper

Chairman

The Honorable Tom Coburn

Ranking Member

Committee on Homeland Security and Governmental Affairs

United States Senate

The Honorable Claire McCaskill

Chairman

Subcommittee on Financial and Contracting Oversight

Committee on Homeland Security and Governmental Affairs

United States Senate

The Honorable John McCain 

Ranking Member 

Permanent Subcommittee on Investigations 

Committee on Homeland Security and Governmental Affairs 

United States Senate
Appendix I: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Schedule Estimate

Page 19 GAO-14-152 DOD Business Systems Modernization

This appendix provides the results of our analysis of the extent to which 

the Defense Enterprise Accounting and Management System (DEAMS) 

schedule estimate supporting the February 2012 Milestone B decision1

met the characteristics of a high-quality, reliable schedule. Table 3 

provides the detailed results of our analysis.

GAO’s methodology includes five levels of compliance with its best 

practices.2 “Not met” means the program provided no evidence that 

satisfies any of the criterion. “Minimally met” means the program provided 

evidence that satisfies a small portion of the criterion. “Partially met” 

means the program provided evidence that satisfies about half of the 

criterion. “Substantially met” means the program provided evidence that 

satisfies a large portion of the criterion. “Fully met” means the program 

provided evidence that completely satisfies the criterion. 

 

1

Milestone B is considered the official start of the program according to the Defense 

Acquisition Management System Framework. The framework is intended to translate 

mission needs and requirements into stable, affordable, and well-managed acquisition 

programs. 

2

GAO, GAO Schedule Assessment Guide: Best Practices for Project Schedules—

Exposure Draft, GAO-12-120G (Washington, D.C.: May 2012).

Appendix I: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Schedule Estimate
Appendix I: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Schedule Estimate

Page 20 GAO-14-152 DOD Business Systems Modernization

Table 3: Assessment of DEAMS Schedule Estimate Compared to Best Practices

Characteristic Best practice Assessment Key examples of rationale for assessment

Comprehensive The schedule should reflect all 

activities as defined in the 

project’s work breakdown 

structure, which defines in detail 

the work necessary to 

accomplish a project’s objectives, 

including activities to be 

performed by both the 

government and contractors. The 

schedule should reflect the 

resources (e.g., labor, materials, 

and overhead) needed to do the 

work, whether those resources 

will be available when needed, 

and whether any funding or time 

constraints exist. The schedule 

should realistically reflect how 

long each activity will take, 

allowing for discrete progress 

measurement with specific start 

and finish dates.

Partially met The schedule reflected the activities to be performed by both 

the government and the contractor for Increment 1, Releases 

1 through 3, of the program. However, it did not reflect 

activities for Releases 4 through 6 of Increment 1 and for 

Releases 1 and 2 of Increment 2 of the program. To be 

comprehensive, the program schedule should reflect all effort 

necessary to successfully complete the program. In addition, 

the program schedule was not fully integrated. Specifically, 

although the schedule narrative noted that Release 2 of 

Increment 1 included contractor activities and that integration 

with government activities was in process, there were no links 

between the government and contractor activities for this 

release. For the most part, resources had been identified; 

however, the resources had not been assigned to specific 

activities in the schedule. The amount of available resources 

affects estimates of work and its duration and resources that 

will be available for subsequent activities. Information on 

resource needs and availability assists the program office in 

forecasting the likelihood that activities will be completed as 

scheduled. Because the current schedule did not allow insight 

into current or projected assignment of resources, the risk of 

program slippage was significantly increased.

The project duration was determined by a target date and the 

majority of activity durations in the schedule met the standard 

best practice for activity duration (44 days). Specifically, 98 

percent of the activities were less than 44 days. 
Appendix I: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Schedule Estimate

Page 21 GAO-14-152 DOD Business Systems Modernization

Characteristic Best practice Assessment Key examples of rationale for assessment

Well-constructed The schedule should be planned 

so that critical project dates can 

be met. All activities should be 

logically sequenced—that is, 

listed in the order in which they 

are to be carried out—with 

predecessor and successor logic. 

Date constraints and lags should 

be minimized and justified to help 

ensure that the interdependence 

of activities that collectively lead 

to the completion of events or 

milestones can be established 

and used to guide work and 

measure progress. The schedule 

should identify the program 

critical path—the path of longest 

duration through the sequence of 

activities—to determine the 

program’s earliest completion 

date. The schedule should 

identify total float—the amount of 

time by which a predecessor 

activity can slip before the delay 

affects the program’s estimated 

finish date—so that the 

schedule’s flexibility can be 

accurately determined. As a 

general rule, activities along the 

critical path have the least 

amount of float.

Partially met The schedule was not planned to ensure that critical project 

dates could be met. The majority of logic used to sequence 

the activities within the schedule was generally error-free with 

a minimal use of lags—the passage of time between two 

activities—and leads—negative lags used to accelerate a 

successor activity, clearly indicating which activities must 

finish. Our analysis found that the schedule for Increment 1, 

Release 3, accounted for predecessor and successor logic on 

97 percent of remaining activities and milestones. However, 

our analysis of the schedule for Releases 1 and 2 of 

Increment 1 showed that for the remaining activities and 

milestones 10 percent were missing predecessor links, 17 

percent were missing successor links, and 2 percent were 

missing both. That is, approximately one-fourth of all 

remaining detail activities and milestones were missing some 

logic link. Missing predecessors or successors reduce the 

credibility of the calculated dates. If an activity that has no 

logical successor slips, the schedule will not reflect the effect 

on the critical path, float, or scheduled start dates of 

downstream activities.

In addition, there were a small percentage of activities with 

dangling relationships in both schedules. A dangling 

relationship is a scheduling relationship that is not properly 

tied to an activity’s start or end date. Each activity’s start 

date—other than the start and finish milestones—must be 

driven by a predecessor activity, and each activity’s finish date 

must drive a successor activity’s start or finish. The schedule 

for Releases 1 and 2 of Increment 1 had 5 remaining activities 

whose start dates were not being driven by a predecessor 

activity and 11 activities that had finish dates that did not drive 

successor activities. The schedule for Release 3 of Increment 

1 had 1 remaining activity whose start date was not being 

driven by a predecessor activity and 24 activities that had 

finish dates that did not drive successor activities. Dangling 

relationships, a form of incomplete relationships, can interfere 

with the valid forecasting of scheduled activities. Both 

schedules had date constraints on about one-third of the 

remaining activities. A customer-mandated date, including 

contractual obligations, does not constitute a legitimate 

reason to constrain an activity. A schedule is intended to be a 

dynamic, proactive planning and risk mitigation tool that 

models the program and can be used to track progress toward 

important milestones. Schedules with constrained dates can 

portray an artificial or unrealistic view of the project.
Appendix I: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Schedule Estimate

Page 22 GAO-14-152 DOD Business Systems Modernization

Characteristic Best practice Assessment Key examples of rationale for assessment

Our analysis found that both schedules did not have a valid 

critical path. For example, there were numerous critical 

activities scheduled to start before the project status date had 

started. Successfully identifying the critical path relies on 

capturing all activities, properly sequencing activities, 

horizontal traceability, the reasonableness of float, accurate 

status updates, and—if there are resource limitations—

assigning resources. Until the schedule can produce a true 

critical path, the program office will not be able to provide 

reliable timeline estimates or identify when problems or 

changes may occur and their effect on downstream work. 

Also, without a valid critical path, management cannot focus 

on activities that will have detrimental effects on the key 

project milestones and deliveries if they slip.

Our analysis found that float calculations within the schedules 

were not reliable and that in some instances unreasonable 

float was a direct result of improper sequencing or missing 

logic. The schedule for Releases 1 and 2 of Increment 1 had 

average total float for the remaining detail and milestone 

activities of 239 business days. Twenty-five percent of 

activities had total float equal to or greater than 392 working

days, meaning they could slip almost 2 working years and not 

affect the end date of the project. In addition, 77 activities had 

negative float, ranging from -1 to -18 days. Negative float 

stems from constraining one or more activities or milestones 

in the network. Negative float indicates critical path effort that 

may require management action such as overtime or 

resequencing of work. The constraint should be examined and 

justified, and the resulting negative float should be evaluated 

for reasonableness. The schedule for Release 3 of Increment 

1 also had an average total float for the remaining detail and 

milestone activities of 289 business days; 25 percent of 

remaining activities had total float equal to or greater than 489 

days. There were no activities with negative float. Without 

reasonable float estimates, management may be unable to 

identify activities that could be permitted to slip and thus 

release and reallocate resources to activities that require 

more resources to be completed on time.
Appendix I: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Schedule Estimate

Page 23 GAO-14-152 DOD Business Systems Modernization

Characteristic Best practice Assessment Key examples of rationale for assessment

Credible The schedule should be 

horizontally integrated, meaning 

that it should link products and 

outcomes associated with other 

sequenced activities. These links 

are commonly referred to as 

“handoffs” and serve to verify that 

activities are arranged in the right 

order to achieve aggregated 

products or outcomes. The 

schedule should also be vertically 

integrated, meaning that the 

dates for starting and completing 

activities in the integrated master 

schedule should be aligned with 

the dates for supporting tasks 

and subtasks. Such mapping or 

alignment among levels enables 

different groups to work to the 

same master schedule.

A schedule risk analysis should 

be performed using statistical 

techniques to predict the level of 

confidence in meeting a 

program’s completion date, 

determine the time contingency 

needed for a level of confidence, 

and identify high-priority risks and 

opportunities. The analysis 

focuses not only on critical path 

activities but also on activities 

near the critical path, since they 

can affect the program’s status. 

Minimally met The schedule for Release 3 of Increment 1 exhibited 

horizontal traceability; however, the schedule for Releases 1 

and 2 of Increment 1 did not because date constraints 

prevented forecasted dates from being calculated realistically 

for future activities. Unless the schedule is horizontally 

traceable, activities whose durations are greatly extended will 

have no effect on key milestones.

Our analysis did not find vertical traceability within the 

schedule for Releases 1 and 2 of Increment 1—the ability to 

consistently trace work breakdown structure activities 

between detailed, intermediate, and master schedules. For 

example, we traced three activities between the government 

schedule and the underlying prime contractor schedule and in 

each case found mismatching start dates that differed by a 

day, a week, and a month. Vertical traceability provides 

assurance that the representation of the schedule to different 

audiences is consistent and accurate. Unless the schedule is 

vertically traceable, lower-level schedules will not be 

consistent with upper-level schedule milestones, affecting the 

integrity of the entire schedule and the ability of different 

teams to work to the same schedule expectations.

The program office had not conducted a schedule risk 

analysis. If a schedule risk analysis is not conducted, program 

management cannot determine the likelihood of meeting the 

program’s completion date, how much schedule risk 

contingency is needed to provide an acceptable level of 

certainty for completion by a specific date, risks most likely to 

delay the program, how much contingency reserve each risk 

requires, and the paths or activities that are most likely to 

delay the program. Unless a statistical simulation is run, 

calculating the completion date from schedule logic and the 

most likely duration distributions will tend to underestimate the 

program’s overall critical path duration. 
Appendix I: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Schedule Estimate

Page 24 GAO-14-152 DOD Business Systems Modernization

Characteristic Best practice Assessment Key examples of rationale for assessment

Controlled The schedule should be 

continuously updated using logic 

and durations to realistically 

forecast dates for program 

activities. A schedule narrative 

should accompany the updated 

schedule to provide decision 

makers and auditors a log of 

changes and their effect, if any, 

on the schedule time frame.

The schedule should be analyzed 

continuously for variances to 

determine when forecasted 

completion dates differ from 

planned dates. This analysis is 

especially important for those 

variations that impact activities 

identified as being on a 

program’s critical path and can 

impact a scheduled completion 

date. A baseline schedule is the 

basis for managing the program 

scope, the time period for 

accomplishing it, and the 

required resources. The baseline 

schedule is designated the target 

schedule, and is subject to a 

configuration management 

control process, against which 

program performance can be 

measured, monitored, and 

reported. A corresponding 

baseline document explains the 

overall approach to the program; 

defines custom fields in the 

schedule file; details ground rules 

and assumptions used in 

developing the schedule; and 

justifies constraints, lags, long 

activity durations, and any other 

unique features of the schedule.

Partially met Management met weekly to discuss proposed schedule 

changes and update schedule progress. The program 

management office followed general guidelines for updating 

the schedule; however, a schedule narrative did not 

accompany the updates, and we found a significant number of 

date anomalies in the schedules provided for Releases 1 

through 3 of Increment 1. For example, we found instances of 

activities with start dates in the past, activities with planned 

finish dates in the past, activities with actual start dates in the 

future, and activities with actual finish dates in the future. We 

also found a number of out-of-sequence activities, that is, 

activities that started before their predecessors finished. A 

schedule is a fundamental program management tool that 

specifies when work will be performed in the future and how 

well the program is performing against an approved plan. If 

the schedule is not updated, then it is impossible to tell what 

activities have been completed, are in progress, are late, and 

are planned to start on time. In addition, if a schedule is not 

updated to reflect what is actually occurring on the program, it 

may have inaccurate completion dates and critical paths. 

When this is the case, management cannot use the schedule 

to monitor progress and make decisions regarding risk 

mitigation and resource allocations.

There was no baseline schedule that could be used to 

measure program performance. As a result, it was not 

possible to track variances to past baselines using the 

schedules provided for Increment 1, Releases 1 through 3. 

The program management office did maintain a baseline 

schedule narrative document that contained a list of custom 

fields and assumptions. However, this document did not 

explain the overall program approach, ground rules and 

assumptions, justifications for logic, and other unique features 

of the schedule. Without a formally established baseline 

schedule to measure performance against, management 

cannot identify or mitigate the effect of unfavorable 

performance. A well-documented schedule is essential for 

validating and defending a baseline schedule, analyzing 

changes in the program schedule, and identifying the reasons 

for variances between estimates. 

Source: GAO analysis of Air Force data.
Appendix II: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Cost Estimate

Page 25 GAO-14-152 DOD Business Systems Modernization

This appendix provides the results of our analysis of the extent to which 

the Defense Enterprise Accounting and Management System (DEAMS) 

cost estimate supporting the February 2012 Milestone B decision1 met the 

characteristics of a high-quality cost estimate. Table 4 provides the 

detailed results of our analysis.

GAO’s methodology includes five levels of compliance with its best 

practices.2 “Not met” means the program provided no evidence that 

satisfies any of the criterion. “Minimally met” means the program provided 

evidence that satisfies a small portion of the criterion. “Partially met” 

means the program provided evidence that satisfies about half of the 

criterion. “Substantially met” means the program provided evidence that 

satisfies a large portion of the criterion. “Fully met” means the program 

provided evidence that completely satisfies the criterion.

Table 4: Assessment of DEAMS Cost Estimate Compared to Best Practices

Characteristic Best practice Assessment Key examples of rationale for assessment

Comprehensive The cost estimate includes all 

life-cycle costs. The cost 

estimate completely defines the 

program, reflects the current 

schedule, and is technically 

reasonable. The cost estimate 

work breakdown structure is 

product oriented, traceable to 

the statement of work, and at an 

appropriate level of detail to 

ensure that cost elements are 

neither omitted nor doublecounted. The estimate 

documents all cost-influencing 

ground rules and assumptions.

Fully met The DEAMS cost estimate included both government and 

contractor costs of the program over its life cycle. The 

documentation provided a description of the purpose of the 

estimate; a technical description of the program including 

functional relationships; and technical risks (e.g., hours per 

test activity and the resolution for any identified 

deficiencies).

The work breakdown structure clearly outlined the end 

product and major work to be performed. The program 

provided supporting documentation that showed the ground 

rules and assumptions, such as the total number of users, 

use of a commercial-off-the-shelf product, and licensing 

requirements. The estimate was based on a cost element 

structure as stated in the Department of Defense 

Automated Information System Economic Analysis Guide. 

 

1

Milestone B is considered the official start of the program according to the Defense 

Acquisition Management System Framework. The framework is intended to translate 

mission needs and requirements into stable, affordable, and well-managed acquisition 

programs.

2

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009).

Appendix II: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Cost Estimate
Appendix II: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Cost Estimate

Page 26 GAO-14-152 DOD Business Systems Modernization

Characteristic Best practice Assessment Key examples of rationale for assessment

Well-documented The documentation should 

capture the source data used, 

the reliability of the data, and 

how the data were normalized. 

The documentation describes in 

sufficient detail the calculations 

performed and the estimating 

methodology used to derive 

each work breakdown structure 

element’s cost. The 

documentation describes step 

by step how the estimate was 

developed so that a cost analyst 

unfamiliar with the program 

could understand what was 

done and replicate it. The 

documentation discusses the 

technical baseline description 

and the data in the baseline are 

consistent with the estimate. 

The documentation provides 

evidence that the cost estimate 

was reviewed and accepted by 

management.

Substantially met The documentation identified the source of the data used 

and that the data were normalized. Major cost drivers were 

also identified, such as personnel costs, support costs, and 

system development. However, there was no discussion of 

efforts taken, if any, to ensure the reliability of the data 

used.

The documentation identified the methodology used for 

each cost element in sufficient detail to replicate it. The 

document also contained the rationale for selecting the 

methodology. Technical descriptions in the cost model were 

consistent with the Cost Analysis Requirements Description, 

which contained a system overview outlining the 

incremental approach being used to develop DEAMS.

The DEAMS Program Management Office presented 

evidence of receiving approval of the estimate through 

briefings to management although there was no reference 

in the briefing to the risk and uncertainty analysis that was 

performed.

Accurate The cost estimate results are 

unbiased, not overly 

conservative or optimistic and 

based on an assessment of 

most likely costs. The estimate 

has been adjusted properly for 

inflation. The estimate contains 

few, if any, minor mistakes. The 

cost estimate is regularly 

updated to reflect significant 

changes in the program so that 

it is always reflecting current 

status. The estimate is based on 

a historical record of cost 

estimating and actual 

experiences from other 

comparable programs. 

Variances between planned and 

actual costs are documented, 

explained, and reviewed. 

Substantially met The program provided a risk and uncertainty analysis that 

documented the range of all possible costs. This analysis 

showed that the results were unbiased and reflected the 

most likely costs. The DEAMS cost model detailed the 

calculations and inflation indexes underlying the estimated 

costs. Calculations within the model can be traced back to 

supporting documentation. The estimate did not contain any 

mistakes. In addition, the cost model was updated annually 

to incorporate actual costs expended in prior fiscal years. 

The estimate was based on historical data as appropriate.

However, the cost estimate did not discuss variances 

between planned and actual costs, which would enable 

estimators to assess how well they are estimating program 

costs and identify lessons learned. 
Appendix II: Assessment of the Defense 

Enterprise Accounting and Management 

System Program’s Cost Estimate

Page 27 GAO-14-152 DOD Business Systems Modernization

Characteristic Best practice Assessment Key examples of rationale for assessment

Credible A risk and uncertainty analysis 

was conducted that quantified 

the imperfectly understood risks 

and identified the effects of 

changing key cost driver 

assumptions and factors. Major 

cost elements were crosschecked to see whether results 

were similar. An independent 

cost estimate was conducted by 

a group outside the acquiring 

organization to determine 

whether other estimating 

methods produce similar results. 

The cost estimate includes a 

sensitivity analysis that identifies 

a range of possible costs based 

on varying major assumptions, 

parameters, and data inputs. 

Substantially met The program management office provided a risk and 

uncertainty analysis that quantified the risks for the major 

cost drivers (e.g., personnel, deployment software and 

integrated logistics support) based on changes in the 

assumptions (e.g., pending negotiations and level of 

knowledge about support concepts). DEAMS program 

officials stated that some cross-checking was performed 

using cost models; however, the results of this analysis 

were not documented.

The Air Force Cost Analysis Agency developed an 

independent cost estimate that was reconciled to the 

program’s cost estimate.

However, a sensitivity analysis was not completed for each 

of the major cost drivers. As a result, the cost estimator will 

not have a clear understanding of how each major cost 

driver is affected by a change in a single assumption and 

thus which scenario most affects the cost estimate.

Source: GAO analysis of Air Force data.
Appendix III: Comments from the Department 

of Defense

Page 28 GAO-14-152 DOD Business Systems Modernization

Appendix III: Comments from the 

Department of Defense
Appendix III: Comments from the Department 

of Defense

Page 29 GAO-14-152 DOD Business Systems Modernization
Appendix IV: GAO Contacts and Staff 

Acknowledgments

Page 30 GAO-14-152 DOD Business Systems Modernization

Asif A. Khan, (202) 512-9869 or khana@gao.gov

Nabajyoti Barkakati, (202) 512-4499 or barkakatin@gao.gov

In addition to the contacts named above, Cynthia Jackson (Director), 

Karen Richey (Assistant Director), Beatrice Alff, Jennifer Echard, Patrick 

Frey, and Jason Lee made key contributions to this report.

Appendix IV: GAO Contacts and Staff 

Acknowledgments

GAO Contacts

Staff 

Acknowledgments

(197120)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.