United States Government Accountability Office

GAO Report to Congressional Committees

DEFENSE 

TRANSPORTATION 

Study Limitations 

Raise Questions about 

the Adequacy and 

Completeness of the 

Mobility Capabilities 

Study and Report 

September 2006 

 

GAO-06-938 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

September 2006

DEFENSE TRANSPORTATION 

Study Limitations Raise Questions about 

the Adequacy and Completeness of the 

Mobility Capabilities Study and Report 

Highlights of GAO-06-938, a report to 

congressional committees 

The Department of Defense (DOD)

issued the Mobility Capabilities 

Study (MCS), which was intended

to identify and quantify the mobility 

capabilities needed to support U.S. 

strategic objectives into the next

decade. The MCS found that 

projected capabilities are adequate

to achieve U.S. objectives with an

acceptable level of risk—that is, 

current U.S. inventory of aircraft, 

ships, prepositioned assets, and 

other capabilities are sufficient, in 

conjunction with host nation 

support, and assuming planned 

investments take place. 

The Senate report accompanying

the bill for the fiscal year 2005

Defense Authorization Act required

GAO to report on the adequacy and 

completeness of the MCS. GAO

assessed the extent to which the 

MCS met generally accepted 

research standards that this type of

study would be expected to meet to 

be considered sound and complete. 

DOD used an innovative approach in conducting the study and 

acknowledged methodological limitations in its report; however, it did not 

fully disclose how these limitations could affect the MCS conclusions and 

recommendations. Therefore, it is not transparent how the analyses done for 

the study support DOD’s conclusions. Measured against relevant generally 

accepted research standards, GAO has identified limitations in the MCS and 

its report that raise questions about their adequacy and completeness. GAO 

suggests that Congress and other decision makers exercise caution in using 

the MCS to make investment decisions. Among GAO’s findings: 

• Aspects of modeling and data were inadequate in some areas because

data were lacking and the models used could not simulate all relevant 

aspects of the missions. The report did not explain how these limitations 

could affect the study results or what the impact on projected mobility 

capabilities might be. Generally accepted research standards require that 

models used are adequate for the intended purpose, represent a 

complete range of conditions, and that data used are properly generated 

and complete. For example, the MCS modeled hypothetical homeland 

defense missions rather than homeland defense demands derived from a 

well defined and approved concept of operations for homeland defense, 

because the specific details of the missions were still being determined 

and the data used may be incomplete. The MCS also was unable to 

model the flexible deterrent options/deployment order process to move 

units and equipment into theater because of lack of data, but the study 

assumed a robust use of this process. In addition, the MCS report 

contains over 80 references to the need for improved modeling or data. 

• While the MCS concluded that combined U.S. and host nation 

transportation assets were adequate, in describing the use of warfighting 

metrics in its analyses, the report does not provide a clear understanding 

of the direct relationship of warfighting objectives to transportation 

capabilities. Additionally, the report stated that further analysis is

required to understand the operational impact of increased or decreased 

strategic lift on achieving warfighting objectives. Relevant generally 

accepted research standards require that conclusions be supported by 

analyses. The use of both warfighting and mobility metrics would allow 

decision makers to know whether combat tasks were achieved and how 

much strategic transportation is needed to accomplish those tasks. 

• In some cases, the MCS results were incomplete, unclear, or contingent 

on further study, making it difficult to identify findings and evaluate 

evidence. Relevant research standards require results to be presented in 

a complete, accurate, and relevant manner. For example, the report 

contains recommendations for further studies and assessments, five of 

which are under way. However, DOD has no plans to report the impact 

of these studies on the MCS results after the studies are complete. In 

addition, the report contains qualified information that is not presented 

clearly, such as varying assessments of intra-theater assets in three 

different places. 

www.gao.gov/cgi-bin/getrpt?GAO-06-938.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact William M. Solis

at (202) 512-8365 or solisw@gao.gov. 

What GAO Recommends

GAO recommends that the 

Secretary of Defense, in future

mobility s capabilities studies

beginning with any study currently

under way, develop models and

data for critical missions and 

processes; include in study reports 

an explanation of how stated 

limitations might impact results; 

and, incorporate both mobility and

warfighting metrics to determine 

capabilities. In comments, DOD

concurred with two of the 

recommendations and claimed they 

did not understand the third, which

GAO clarified.
Contents 

Letter 1

Results in Brief 2 

Background 5

Limitations in the MCS Study and Report Raise Questions about 

Adequacy and Completeness 7 

Conclusions 13

Recommendation for Executive Action 13 

Agency Comments and Our Evaluation 14 

Appendix I Scope and Methodology 18 

Appendix II Comments from the Department of Defense 22 

Appendix III GAO Contact and Staff Acknowledgments 29 

Table 

Table 1: Generally Accepted Research Standards Relevant to MCS 

Requirements 19

Figure 

Figure 1: MCS Hypothetical Ongoing Contingencies during 7-year 

Baseline Security Posture Time Frame 9 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page i GAO-06-938 Defense Transportation
United States Government Accountability Office

Washington, DC 20548 

September 20, 2006 

Congressional Committees 

The National Security Strategy of the United States requires global 

mobility through rapid, effective, and efficient projections of power at 

home and abroad to deploy and sustain America’s armed forces. To 

improve its mobility capabilities, the Department of Defense (DOD) plans 

to spend more than $50 billion from fiscal years 2006 through 2011 for 

aircraft, ships, ground transportation, prepositioned assets, and other 

mobility assets. DOD has conducted several studies to determine mobility 

requirements and recently completed a study of its mobility capabilities 

and issued a report in December 2005. The intent of the Mobility 

Capabilities Study (MCS) was to identify and quantify the mobility 

capabilities needed to support U.S. strategic objectives into the next 

decade. The MCS determined that the projected mobility capabilities are 

adequate to achieve U.S. objectives with an acceptable level of risk during 

the period from fiscal years 2007 through 2013; that is, the current U.S. 

inventory of aircraft, ships, prepositioned assets, and other capabilities are 

sufficient, in conjunction with host nation support. The MCS emphasized 

that continued investment in the mobility system, in line with current 

departmental priorities and planned spending, is required to maintain 

these capabilities in the future. This includes, for example, fully funding 

Army prepositioned assets as planned and completing a planned 

reengineering of the C-5 aircraft. The MCS report also made 

recommendations to conduct further studies, develop plans and strategies, 

and improve data collection and mobility models. In fact, DOD officials 

told us that a Mobility Capabilities Study--2006 is underway. 

In the Senate report accompanying the bill for the fiscal year 2005 Defense 

Authorization Act, you asked us to monitor the process used to conduct 

the MCS and report on the adequacy and completeness of the study.1

Specifically, our objective was to determine whether the MCS was 

adequate and complete. On March 1, 2006, we briefed your staff on our 

preliminary observations. This report expands on that briefing and makes 

recommendations to the Secretary of Defense. 

 1

 S. Rep. 108-260, at 126 (2004). 

Page 1 GAO-06-938 Defense Transportation
To conduct our review of the MCS, we analyzed the final MCS report, the 

MCS Terms of Reference and MCS Study Plan, as well as other DOD 

policies and guidance concerning how DOD would conduct the MCS and 

the databases and models used in the study. We identified generally 

accepted research standards that define a sound and complete quality 

study that were relevant to the MCS, and assessed the extent to which the 

MCS report met these standards. We interviewed study officials, study 

participants, and subject matter experts from several DOD entities, 

including the combatant commands and the military services. As we 

monitored the development of the MCS, we requested that DOD provide 

documentation supporting and verifying key analytical and decisionmaking processes.2

 DOD officials could not produce this documentation 

during the development of the MCS or following issuance of the report. 

Consequently, we were unable to fully determine whether the analytical 

and decision-making processes that we believe are significant to the 

credibility of the study supported the MCS effort and its conclusions. Our 

scope and methodology are discussed in more detail in appendix I. We 

conducted our work from July 2004 through July 2006 in accordance with 

generally accepted government auditing standards. 

DOD used an innovative approach in conducting the study and 

acknowledged some methodological limitations in its report, as any sound 

study should. However, it did not fully disclose how these limitations 

could affect the MCS conclusions and recommendations. Therefore, it is 

not transparent how the analyses done for the study support DOD’s 

conclusions. As measured against relevant generally accepted research 

standards, we identified limitations in the MCS study and report that raise 

questions about their adequacy and completeness. Among our findings: 

Results in Brief 

 2

 Examples of the documentation we requested to support and verify key analytical and 

decision-making processes used by DOD to conduct the MCS included (1) the accreditation 

report and supporting documentation or evidence of the verification, validation, and

accreditation process for the models and data used in the MCS; (2) copies of MCS working 

group meeting minutes that verify and validate the analytical processes the various MCS 

study teams and study participants used to vet and agree upon data, scenarios, 

assumptions, models, and associated risk; and (3) copies of MCS General Officer Steering

Committee and Executive Committee meeting minutes that verify and validate the

analytical and decision-making processes the DOD senior leadership used to vet and agree

upon the key data, scenarios, assumptions, models, and associated risk used to conduct the 

MCS, as well as agreement with the study results. 

Page 2 GAO-06-938 Defense Transportation
• Aspects of modeling and data were inadequate in some areas because data 

were lacking and some of the models used could not simulate all relevant 

aspects of the missions. The report did not explain how these limitations 

could affect the study results or what the effect on the projected mobility 

capabilities might be. Relevant research standards require that models 

used are adequate for the intended purpose, represent a complete range of 

conditions, and that data used are properly generated and complete. For 

example, the MCS modeled hypothetical homeland defense missions 

rather than homeland defense demands derived from a well defined and 

approved concept of operations for homeland defense, because the 

specific details of the missions were still being determined, and DOD 

acknowledged that the data used may be incomplete. The MCS also was 

unable to model the flexible deterrent options/deployment order process 

to move units and equipment into theater due to lack of data, but the study 

assumed a robust use of this process, which in one scenario accounted for 

approximately 60 percent of the airlift prior to beginning combat 

operations.3

 In addition, the MCS report contains more than 80 references 

to the need for improved modeling, and 12 of these references call for 

additional data or other refinements. Additionally, the MCS modeled the 

year 2012 to determine the transportation capabilities needed for the years 

2007 through 2013. The year 2012 did not place as much demand for 

mobility assets in support of smaller military operations, such as 

peacekeeping, as other years. However, DOD officials considered 2012—

the year modeled—as “most likely” to occur and stated that statistically it

was not different from other years in the 2007 to 2013 period even though

the number of smaller military operations is the least of any of the years 

reviewed. 

• While the MCS concluded that combined U.S. and host nation 

transportation assets were adequate to meet U.S. objectives with 

acceptable risk, the report, in describing the use of warfighting metrics in 

its analyses, does not provide a clear understanding of the direct 

relationship of warfighting objectives to transportation capabilities. 

Acknowledging this point, the report stated that further analysis is 

required to understand the operational impact of increased or decreased 

strategic lift on achieving warfighting objectives. Relevant generally 

accepted research standards require that conclusions be supported by 

analyses. The use of warfighting metrics is a measure to determine 

whether combat tasks, such as achieving air superiority, are achieved. 

 3

 Deployment orders are issued to deploy specific capabilities as commitment decisions are

made, rather than a deploying unit’s full set of equipment or capabilities. Flexible Deterrent

Options (FDOs) provide escalation options during the initial stages of a conflict. FDOs are 

employed under certain conditions to deter adversarial actions contrary to U.S. interests. 

Page 3 GAO-06-938 Defense Transportation
However, they do not measure whether appropriate personnel, supplies, 

and equipment arrived in accordance with timelines. As a result, we could 

not determine how the study concluded that planned transportation assets 

were adequate because the study did not contain a transparent analysis to 

support its conclusion or a clear roadmap in the report to help decision 

makers understand what that conclusion meant in terms of type and 

number of mobility assets needed. Previous DOD mobility studies 

primarily used mobility metrics, which measured success in terms of tons

of equipment and personnel moved per day to accomplish military 

objectives. The use of both warfighting and mobility metrics to measure 

success would allow decision makers to know whether combat tasks were 

achieved and how much strategic transportation is needed to accomplish 

those tasks. 

• In some cases, the MCS results were incomplete, unclear, or contingent on

further study, making it difficult to identify findings and evaluate evidence. 

Relevant research standards require results to be presented in a complete, 

accurate, and relevant manner. For example, the report contains several 

recommendations for further studies and assessments, five of which are 

under way. However, DOD has no plans to report the effect of these 

studies on the MCS results after the studies are complete. In addition, the 

report contains qualified information that is not presented clearly, such as

varying assessments of intratheater assets in three different places in the

report. The lack of clarity and conciseness of the reported results can limit 

the study’s usefulness to decision makers and stakeholders. 

• Verification, validation, and accreditation (VV&A) of models and data used 

to conduct the study was not complete because it was not done in 

accordance with DOD policy or relevant research standards. Moreover, 

relevant research standards state that a study report should include a 

VV&A accreditation report that is signed by the study director and 

addresses the models and data certification. DOD officials acknowledged 

that they did not comply with DOD VV&A policy when using legacy 

models in the MCS because they contended that long-term use of models 

and data constitutes an equivalent VV&A process. Other than a description 

of the process contained in the MCS report, DOD officials could provide 

no additional documentation to verify and validate this equivalent process 

to provide the assurance that models and data used in the MCS reduced 

the risk inherent in modeling and simulation and added to the credibility 

of the results. Moreover, officials could not provide documentation to 

support key analytical and decision-making processes used by senior DOD 

leadership, thus undermining the credibility of the reported study results. 

These limitations to the study’s methodology raise questions concerning 

the accuracy of the study’s finding that projected capabilities are adequate 

to achieve U.S. objectives with an acceptable level of risk. Until DOD 

Page 4 GAO-06-938 Defense Transportation
conducts an adequate and complete future MCS and clearly discloses all 

limitations and their effects on the study results, decision makers may be 

unable to clearly understand the operational implications of the study 

results and make fully informed programmatic investment decisions 

concerning mobility capabilities. We are recommending that the Secretary 

of Defense, when conducting future mobility capabilities studies, 

beginning with any study currently under way, develop and use models 

and data for critical missions and processes that are verified, validated,

and accredited as required; include in study reports an explanation of how 

stated limitations might impact the study results and, at a minimum, 

describe how recommended future studies might be conducted to enhance 

the results of the original study; and incorporate both mobility and 

warfighting metrics in determining capabilities. 

In commenting on a draft of this report, DOD concurred with the first and 

third recommendations and claimed that they did not understand the 

second. We have clarified that recommendation to the Secretary of 

Defense to include in study reports an explanation of how stated 

limitations might impact the study results and, at a minimum, describe 

how recommended future studies might be conducted to enhance the 

results of the original study. In its comments, DOD also stated that the 

report contained misleading information and factual errors. We disagree 

with DOD’s assertion. We did modify our report to respond to a DOD 

technical comment related to homeland defense missions. DOD’s 

comments and our evaluation of them are discussed in the agency 

comments section of this report. 

The MCS was the first assessment of DOD’s mobility system since 2000.

The study was designed to identify changes in DOD’s transportation force 

structure due to changes in threats and national security and military 

strategies. The MCS is the fourth in a series of major mobility studies that 

DOD has conducted since the end of the Cold War. The first study, the 

Mobility Requirements Study, conducted in 1992, was undertaken because 

of concern about the DOD’s strategic mobility capabilities in the wake of 

Operation Desert Shield and Operation Desert Storm. That study 

established mobility requirements for the post-Cold War era; defined 

baseline requirements for intertheater, or strategic, mobility; and proposed 

a long-range investment plan to meet these requirements. The Mobility 

Requirements Study Bottom-Up Review Update, conducted in 1994, 

reaffirmed the need for increases in key mobility components and 

validated the prior study’s recommendation for the procurement of 

additional ships for afloat prepositioning and for surge deployments of 

Background 

Page 5 GAO-06-938 Defense Transportation
forces based in the continental United States. The Mobility Requirements 

Study—2005, issued in 2001, projected future mobility requirements based 

on two nearly simultaneous major regional contingencies. It included a 

broader range of factors, including host nation support and enemy use of 

weapons of mass destruction, than the previous studies. 

The current MCS, which began in May 2004, reassessed DOD’s mobility 

capabilities against the backdrop of a revised National Military Strategy 

that included the ongoing war against violent extremism, an evolving 

global defense posture, a new force-sizing construct, revised campaign 

scenarios, and ongoing departmentwide transformation efforts. The study 

results were intended to support decisions on future strategic airlift, aerial 

refueling aircraft, and sealift procurements needed to meet varying 

military requirements. The study used an innovative “capabilities-based” 

approach, measuring existing and currently projected mobility capabilities 

against warfighting demands that could be expected in fiscal year 2012 

while also considering mobility demands during the 7-year period from 

fiscal year 2007 through fiscal year 2013. According to DOD officials, the 

Secretary of Defense believed this approach would give him greater 

flexibility in deciding which capabilities to fund in a constrained budget 

environment. In considering each aspect of the National Military Strategy,4

the MCS modeled warfighting scenarios in the year 2012 using different 

premises with varying assumptions to develop and evaluate mobility 

capability mix alternatives. The models were used to evaluate

transportation alternatives, including variations in alternative 

transportation modes (air, land, sea) and sources (military, civilian, 

foreign), as well as factors that affect transportation mode and source 

decisions. The scope of the MCS described the study as an assessment of 

the full range of transportation needs required to support (1) combat 

operations; (2) smaller military efforts, such as peacekeeping or overseas 

evacuation of American citizens; (3) homeland defense/civil support, such 

as disaster relief and antiterrorism response; and (4) other strategic 

missions, such as strategic nuclear and global strike missions. The study 

was coauthored by the Office of the Secretary of Defense, Office of the

Director, Program Analysis and Evaluation (PA&E), and the Chairman, 

 4

 The 2004 National Military Strategy of the United States calls for a force sized to defend 

the homeland, proactively dissuade adversaries in and from four global regions, and

conduct two overlapping “swift defeat” campaigns. Even when committed to a limited 

number of lesser contingencies, the force must be able to “win decisively” in one of the two 

campaigns. This “1-4-2-1” force-sizing construct places a premium on increasingly 

innovative and efficient methods to achieve objectives. 

Page 6 GAO-06-938 Defense Transportation
Joint Chiefs of Staff, Office of the Director of Logistics. Other DOD 

components involved in the study included the U.S. Transportation 

Command and its subordinate commands, the Office of the Under 

Secretary of Defense (Acquisition, Technology and Logistics), the 

combatant commanders, the military services, and others. The final report 

was signed on December 19, 2005, by the Deputy Secretary of Defense. 

As measured against relevant generally accepted research standards, 

limitations in the MCS study and report raise questions about their 

adequacy and completeness. For example, aspects of modeling and data 

were inadequate in some areas because data were lacking and some of the 

models used could not simulate all relevant aspects of the missions. 

Furthermore, the exclusive use of warfighting metrics in the MCS analyses 

limited the usefulness of the report. Moreover, in some cases the MCS 

results were incomplete, unclear, or contingent on further study, making it

difficult to identify findings and evaluate evidence. Finally, verification, 

validation, and accreditation of models and data used to conduct the study 

were incomplete because they were not done in accordance with DOD 

policy or relevant research standards, and supporting documentation for 

key processes could not be provided. 

Aspects of modeling and data were inadequate in some areas because data 

were lacking and some of the models used could not simulate all relevant 

aspects of the missions. Relevant research standards require that models 

used are adequate for the intended purpose, represent a complete range of 

conditions, and that data used are properly generated and complete. As 

DOD acknowledged in the MCS report as a study limitation, some 

modeling tools were not available to analyze key missions. The MCS cited 

deficiencies in several existing mobility models and the need for follow-on 

MCS analyses. The MCS report also acknowledged that the identified 

deficiencies in data hindered analysis of future requirements and 

transportation system performance. However, the report did not explain 

how these limitations could affect the study results or what the effect on 

the projected mobility capabilities might be. 

For example, the MCS modeled hypothetical homeland defense missions 

rather than homeland defense demands derived from a well defined and 

approved concept of operations for homeland defense, because the 

specific details of the missions were still being determined, and DOD 

acknowledged that the data used are incomplete. The MCS report 

recommended further analysis of mobility capabilities after homeland 

Limitations in the 

MCS Study and 

Report Raise 

Questions about 

Adequacy and 

Completeness 

Aspects of Modeling and 

Data Were Inadequate 

Page 7 GAO-06-938 Defense Transportation
defense needs are refined. However, the report did not identify the 

potential effect that using these hypothetical scenarios might have on the 

MCS results. The MCS also was unable to model the flexible deterrent 

options/deployment order process to move units and equipment into 

theater because of lack of data on how deployment orders have been 

issued in the past for major combat operations. However, the MCS 

assumed a robust use of the flexible deterrent option/deployment order 

process, which in one scenario accounted for approximately 60 percent of

the early airlift movement prior to the beginning of combat operations. 

Instead, the MCS modeled the flow of forces and equipment contained in 

the time-phased force deployment data process.5 Based on the scenarios 

provided for the MCS analyses, we could not determine how the 

deployment order process would affect the mobility assets required for 

major combat operations. The MCS report noted that additional analysis is 

required to determine the implications of the deployment order process 

and to provide sufficient information for decision makers concerning the 

amount of future mobility assets actually needed. 

In addition to these modeling and data issues, the MCS report contains 

more than 80 references to the need for improved modeling, and 12 of 

these references call for additional data or other refinements, such as 

follow-on analyses, further assessments, future evaluations, additional 

study, and investigation of alternatives in a wide range of areas, such as 

antiterrorism response, infrastructure availability, intratheater airlift, 

strategic sealift, air refueling, and availability of civil reserve aircraft. 

Some of these studies are currently underway, as discussed later in this 

report. 

Moreover, our analysis of the MCS report showed that the year modeled 

(2012) to determine the DOD transportation capabilities needed for the 

years 2007 through 2013 did not place as much demand for mobility assets

in support of smaller military operations, such as peacekeeping, as other 

years. To establish transportation demands for mission requirements, DOD 

developed and used a baseline security posture6

 that covered a 7-year 

 5

 In joint military planning, time-phased force deployment data are defined as a computer

database that contains detailed personnel and cargo planning data; it usually includes 

priority and sequencing of deploying forces.

6

 The baseline security posture projects the position from which combatant commanders

will perform future missions, including how they will address the global war on terrorism, 

ongoing operations, and other day-to-day activities to which U.S. forces remain committed 

and from which they are not likely to disengage entirely. 

Page 8 GAO-06-938 Defense Transportation
period. This baseline was developed, in part, using a historical analysis of 

DOD’s movement of personnel, equipment, supplies, and other items. 

According to DOD officials, Office of the Under Secretary of Defense for 

Policy, which developed the baseline security posture, selected the year 

modeled in the MCS because it was deemed the “most likely” to occur in

terms of transportation demands and because it was not statistically 

different from other years in the 7-year period. However, our analysis 

showed that 2012 involved the least demand for transportation assets in 

support of smaller military efforts than any year in the 7-year period and 

did not fully stress the defense transportation system. Figure 1 depicts the 

number of hypothetical ongoing contingencies for each year in the 

baseline as shown in the MCS. 

Figure 1: MCS Hypothetical Ongoing Contingencies during 7-year Baseline Security 

Posture Time Frame 

Note: A particular contingency may be ongoing in more than 1 year. Each contingency has unique 

cargo and passenger requirements. For example, a contingency that may be ongoing over a 2- or 3-

year time frame may require more or less mobility capability than a 6-month contingency.

Although not transparent in the study, DOD officials said scenarios in the 

year modeled were not intended to fully stress the defense transportation 

system. DOD officials provided no further explanation for the year 

selected to develop the DOD transportation capabilities other than it was 

Page 9 GAO-06-938 Defense Transportation
directed by Office of the Under Secretary of Defense for Policy and agreed

to by the study leadership. We believe that selection of a different year that

placed an increased demand on transportation assets for smaller military 

efforts may have revealed gaps in mobility requirements. Therefore, we 

found that the selection of 2012 as the modeling year was a limitation in 

the MCS with respect to smaller military efforts. 

Because of these modeling and data limitations, the MCS may have 

incorrectly estimated the future mobility requirements needed to support 

homeland defense missions, major combat operations, and smaller 

contingencies. Until DOD improves aspects of the modeling and data used

to conduct the MCS—to include defining its homeland defense mission 

needs, developing models for the deployment order process, and 

explaining how identified modeling and data limitations could affect the 

study results—decision makers may not have adequate and complete 

information about DOD’s mobility capabilities. 

While the MCS concluded that combined U.S. and host nation 

transportation assets were adequate to meet U.S. objectives with 

acceptable risk, the report, in describing the use of warfighting metrics in 

its analyses, does not provide a clear understanding of the direct 

relationship of warfighting objectives to transportation capabilities. The 

report acknowledged that further analysis is required to understand the 

operational impact of increased or decreased strategic lift on achieving 

warfighting objectives. Relevant generally accepted research standards 

require that conclusions be supported by analyses. The use of warfighting 

metrics is a measure to determine whether combat tasks, such as 

establishing air superiority, are achieved. However, they do not measure 

success in terms of whether appropriate personnel, supplies, and 

equipment arrived in accordance with timelines. As a result, we could not 

determine how the study concluded that planned transportation assets 

were adequate because the study did not contain a transparent analysis to 

support its conclusion. In our opinion, it is important for decision makers 

to have an understanding of both combat tasks that must be achieved and 

the amount of transportation assets needed to achieve those tasks with 

some level of success. This understanding would allow creation of a clear

roadmap for investment decisions. However, we could not determine how 

the study calculated the specific numbers of transportation assets needed 

or whether there are specific gaps, overlaps, or excesses in transportation 

assets, a key objective of the study. Previous DOD mobility studies, 

including the Mobility Requirements Study—2005, primarily used mobility 

metrics, which measured success in terms of tons of equipment and 

Exclusive Use of 

Warfighting Metrics in 

MCS Analyses Limited 

Usefulness of Report 

Page 10 GAO-06-938 Defense Transportation
personnel moved per day to accomplish military objectives. Million-tonmiles per day is a commonly accepted measure of airlift performance and 

reflects how much cargo can be delivered over a given distance in a given

period of time based on the capability of each type of aircraft. A similar 

mobility metric—short tons—is used to measure ship capability. However, 

these studies did not fully integrate combat tasks as a metric. The use of 

both warfighting and mobility metrics to measure success would allow 

decision makers to know whether there is sufficient capability to achieve 

warfighting objectives, as well as to understand the number, type, and mix 

of mobility assets that are actually needed. 

In some cases, the MCS results were incomplete, unclear, or contingent on

further study, making it difficult to identify findings and evaluate evidence. 

Relevant research standards require results to be presented in a complete, 

accurate, and relevant manner; conclusions to be sound and complete; and 

recommendations to be supported by analyses. Our analysis of the MCS 

report found that it contains several recommendations for further studies 

and assessments, five of which are under way. The five studies address 

intratheater lift capabilities; sealift petroleum, oil, and lubricants; logistics 

contingency operations capabilities; aerial refueling; and integrated global 

presence and basing. However, the report does not explain the potential

effect of these ongoing studies on the MCS results after the studies are 

complete, nor does DOD have plans to report the effect of these studies on 

the MCS results. 

In addition, the report contains qualified information that is not presented 

clearly in the report, such as varying assessments of intratheater assets in 

three different places. For example, the report states in the assessment 

section of the executive summary that projected transportation assets are 

sufficient to address intratheater demands in the fiscal years 2007 through 

2013 time frame. However, in the recommendations section of the 

executive summary, the report states that DOD should take action to 

determine the proper mix of intratheater assets needed to meet 

requirements. Then, in the part of the report that discusses intratheater 

airlift, the report states that a detailed analysis of intratheater airlift needs

would require improved modeling tools to accurately capture interactions

among land, sealift, and airlift capabilities and that data sets must be 

developed that accurately describe the requirement in light of emerging 

concepts. 

Results Are Not Always 

Complete or Presented 

Clearly and Are Qualified 

or Contingent on Further 

Study or Analysis 

Page 11 GAO-06-938 Defense Transportation
VV&A of models and data used to conduct the study was not complete 

because it was not done in accordance with DOD policy or relevant

research standards. DOD policy issued by the Under Secretary of Defense 

for Acquisition, Technology and Logistics requires that DOD models and 

data go through a VV&A process. Moreover, relevant research standards 

state that a study report should include a VV&A accreditation report that is

signed by the study director and addresses the models and data 

certification. DOD officials acknowledged that they did not comply with 

the VV&A policy when using legacy models in the MCS because they 

believed such an approach was not warranted for legacy models that have 

been used for many years and have proved reliable. Moreover, these 

officials believe that such long-term use constitutes a VV&A process 

equivalent to that required in the policy. However, the DOD policy does 

not specify that the actual use of a model constitutes an equivalent VV&A

process. VV&A of models and data reduces the risk inherent in the use of

models and simulations by improving the credibility of modeling and 

simulation results. We previously reported our concerns that DOD did not 

follow its policy in executing the MCS and had little documentation to 

support the VV&A process used.7

 We found that the final MCS report 

contained a description of the equivalent VV&A process, but DOD officials 

could provide no further documentation to verify and validate this 

equivalent process other than the description included in the report. We 

also found no documentation in the study report to support DOD claims 

that the models have proven reliable.

Furthermore, DOD officials were unable to provide documentation to 

support and verify key analytical and decision-making processes used by 

senior DOD leadership throughout the study. Relevant research standards 

support documenting the study’s analytical and decision-making processes 

to ensure the accuracy, completeness, and credibility of study results. 

DOD officials told us that the study’s key analytical and decision-making 

processes were validated and approved by study participants during 

working group meetings and by senior leadership during General Officer 

Steering Committee meetings and Executive Committee meetings. PA&E 

officials could not produce documentation of these meetings because they

said documentation did not exist. Nor could they produce other 

documents we requested during the development of the MCS or following 

VV&A of Models and Data 

Was Not Complete 

 7

 See GAO, Defense Transportation: Opportunities Exist to Enhance the Credibility of the 

Current and Future Mobility Capabilities Studies, GAO-05-659R (Washington, D.C.: Sept.

14, 2005), for a more detailed discussion. 

Page 12 GAO-06-938 Defense Transportation
issuance of the report. Consequently, we were unable to determine the 

adequacy and completeness of the analytical and decision-making 

processes that supported the MCS effort and that we believe are 

significant to the credibility of the study and its conclusions. 

The methodological limitations in the MCS that we identified—some of 

which were acknowledged by DOD in the MCS report—raise questions 

about the adequacy and completeness of the study and its report. Until 

DOD improves aspects of the modeling and data used to conduct the 

MCS—such as defining its homeland defense mission needs and 

developing models for the deployment order process—decision makers 

may not have adequate and complete information about DOD’s mobility 

capabilities to enable them to make fully informed investment decisions. 

Furthermore, in the absence of an explanation of how identified modeling 

and data limitations could affect the study results or how such limitations 

could affect projected mobility capability requirements, the accuracy of 

the study’s finding that projected capabilities are adequate to achieve U.S. 

objectives with an acceptable level of risk during the period from fiscal 

years 2007 through 2013 is unclear. Moreover, without a transparent 

comparison between existing mobility assets and projected needed assets, 

decision makers will be unable to use study results to identify and quantify

the specific types and mix of mobility assets needed to address mobility 

gaps, overlaps, and excesses. Until DOD conducts an adequate and 

complete future MCS and clearly discloses all limitations and their effects 

on the study results, decision makers likely will not have full information 

concerning DOD’s mobility capabilities. As a result, we suggest that 

Congress and other decision makers exercise caution in using the MCS to 

make programmatic investment decisions. 

To provide decision makers with adequate and complete information 

concerning mobility capabilities so they are able to clearly understand the 

operational implications of the study and make fully informed 

programmatic investment decisions, and to improve the usefulness of 

future mobility capabilities studies, we recommend that the Secretary of 

Defense take the following three actions, when conducting future mobility 

capabilities studies beginning with any study currently underway: 

Conclusions 

Recommendation for 

Executive Action 

• develop models and data for all critical missions, such as homeland 

defense, and processes, such as the flexible deterrent options/deployment

order process; 

• include in study reports an explanation of how stated limitations might 

impact the study results and, at a minimum, describe how recommended 

Page 13 GAO-06-938 Defense Transportation
future studies might be conducted to enhance the results of the original 

study; and 

• incorporate both mobility and warfighting metrics in determining 

capabilities. 

In commenting on a draft of this report, DOD concurred with our first and 

third recommendations. DOD stated it did not understand our second 

recommendation that the Secretary of Defense, when conducting future 

mobility studies, beginning with any study currently underway, include in 

study reports an explanation of how ongoing and follow-on studies and 

modeling and data limitations that are referenced in the report could affect 

the reported results. DOD also noted that it plans to continue its ongoing 

efforts to enhance the models and data collection processes used to assess 

mobility capabilities across the full range of strategic missions and 

supports the notion that continual improvements are needed. As we noted 

throughout our report, the MCS report contains numerous and repeated

references to the need for improved modeling and additional data or other 

refinements, such as follow-on analyses, further assessments, future 

evaluations, additional study, and investigation of alternatives in a wide 

range of areas. DOD further commented that while a completed study can 

recommend that follow-on studies be conducted, it cannot explain how 

future studies might affect the results of the current study. We 

acknowledge that DOD cannot quantitatively predict the outcome of an 

ongoing or future study. However, we believe DOD should be able to 

explain what ongoing follow-on studies or evaluations seek to determine, 

what changes are being made to the data inputs and modeling tools that 

are being used to conduct the studies, and how DOD expects the results 

may differ from current study results. While the explanation may be 

hypothetical, as are many operations research study hypotheses, it can 

provide decision makers with a better understanding of the current study’s 

limitations and results and how an ongoing or future study’s results may 

differ. Therefore, we refined our recommendation to recommend that the 

Secretary of Defense, when conducting future mobility studies, beginning 

with any study currently under way, include in study reports an 

explanation of how stated limitations might impact the study results and, 

at a minimum, describe how recommended future studies might be 

conducted to enhance the results of the original study. For example, if 

modeling and data are limitations to a study, the report should discuss the 

ways in which the results might change with better models and data. 

DOD also commented that our report contained misleading information 

and factual errors and that it stands by the adequacy and completeness of 

Agency Comments 

and Our Evaluation 

Page 14 GAO-06-938 Defense Transportation
the MCS. The department provided examples in its technical comments 

where it believed our report contained misleading information and factual 

errors. We disagree with the department’s comments regarding the facts 

in our report and have addressed each of the department’s comments in 

appendix II. 

Lastly, DOD stated that the MCS and its conclusions are well accepted by 

the civilian and military leadership of the department, and pointed out that 

in March 2006 testimony before the House Armed Services Committee, the 

Commander, U.S. Transportation Command, stated that the planned 

strategic airlift fleet determined by the MCS is “about the right capacity”. 

However we note that in the same hearing, the Commander also stated 

that he thought DOD needed “somewhere in the neighborhood of” 20 C-17 

cargo aircraft beyond what is planned. We also note that in the 

Commander’s April 2006 testimony before the Senate Armed Services 

Committee, Subcommittee on Sealift, he stated that, in an internal 

Focused Mobility Analysis to study strategic mobility from a 

Transportation Command perspective, the MCS will be the baseline, “but

we will explore how changes in key assumptions may impact the 

analytical outcome.” 

We are sending copies of this report to the Secretary of Defense; the 

Director of PA&E; and the Office of the Chairman, Joint Chiefs of Staff. We 

will also make copies available to others upon request. In addition, the 

report is available at no charge on the GAO Web site at 

http://www.gao.gov. If you or your staff have any questions regarding the

briefing or this report, please contact me at (202) 512-8365 or 

solisw@gao.gov. Contact points for our Offices of Congressional Relations 

Page 15 GAO-06-938 Defense Transportation
and Public Affairs may be found on the last page of this report. GAO staff 

who made key contributions to this report are listed in appendix III. 

William M. Solis 

Director, Defense Capabilities and Management 

Page 16 GAO-06-938 Defense Transportation
List of Congressional Committees

The Honorable John Warner 

Chairman 

The Honorable Carl Levin 

Ranking Minority Member 

Committee on Armed Services 

United States Senate 

The Honorable Ted Stevens 

Chairman 

The Honorable Daniel K. Inouye 

Ranking Minority Member 

Subcommittee on Defense 

Committee on Appropriations 

United States Senate 

The Honorable Duncan L. Hunter 

Chairman 

The Honorable Ike Skelton 

Ranking Minority Member 

Committee on Armed Services 

House of Representatives 

The Honorable C.W. Bill Young 

Chairman 

The Honorable John P. Murtha 

Ranking Minority Member 

Subcommittee on Defense 

Committee on Appropriations 

House of Representatives 

Page 17 GAO-06-938 Defense Transportation
Appendix I: Scope and Methodology Appendix I: Scope and Methodology 

To conduct our review of the Mobility Capabilities Study (MCS), we 

reviewed and analyzed the final MCS report; the MCS Terms of Reference; 

the MCS Study Plan; applicable Department of Defense (DOD) strategic 

planning guidance; as well as other DOD guidance, directives, instructions, 

and memos that describe how DOD would conduct its MCS. We also 

reviewed the National Security Strategy of the United States of America 

and the National Military Strategy of the United States of America; DOD 

guidance concerning data collection, development, and management in 

support of strategic analysis; DOD modeling and simulation instruction; 

Defense Modeling and Simulation Office guidance; descriptions of models 

used to conduct the study; and the databases used in the models. We 

interviewed study officials from the Office of the Secretary of Defense, 

Program Analysis and Evaluation (PA&E), and the office of the Chairman, 

Joint Chiefs of Staff, Logistics, as well as study participants and subject 

matter experts from the U.S. Transportation Command, Air Mobility 

Command, Surface Deployment and Distribution Command, the 

combatant commands, and the military services concerning the extent of

their input to the study. We also interviewed officials from the Office of 

the Secretary of Defense, Acquisition, Technology and Logistics, and the 

Modeling and Simulation Technical Director at the Defense Modeling and 

Simulation Office. 

Additionally, we reviewed research literature and DOD guidance and 

identified frequently occurring, generally accepted research standards that 

are relevant for defense studies such as the MCS that define a quality or 

sound and complete study. The following were our sources for these 

standards: 

• GAO, Government Auditing Standards: 2003 Revision, GAO-03-673G 

(Washington, D.C.: June 2003); 

• GAO, Designing Evaluations, GAO/PEMD-10.1.4 (Washington, D.C.: 

March 1991); 

• GAO, Dimensions of Quality, GAO/QTM-94-1 (Washington, D.C.: February 

2004); 

• RAND Corporation, RAND Standards for High-Quality Research and 

Analysis (Santa Monica, Calif.: June 2004); 

• Air Force Office of Aerospace Studies, Analysts Handbook: On 

Understanding the Nature of Analysis (January 2000); 

• Air Force, Office of Aerospace Studies, Air Force Analysis Handbook, A 

Guide for Performing Analysis Studies: For Analysis of Alternatives or

Functional Solution Analysis (July 2004); 

Page 18 GAO-06-938 Defense Transportation
Appendix I: Scope and Methodology 

• Department of Defense, DOD Modeling and Simulation (M&S) 

Verification, Validation, Accreditation (VV&A), Instruction 5000.61 

(Washington, D.C.: May 2003); 

• Department of Defense, Data Collection, Development, and Management

in Support of Strategic Analysis, Directive 8260.1 (Washington, D.C.: 

December 2, 2003); and 

• Department of Defense, Implementation of Data Collection, Development, 

and Management for Strategic Analyses, Instruction 8260.2 (Washington,

D.C.: January 21, 2003). 

During the process of identifying generally accepted research standards 

we noted that not all studies are conducted the same way. For example, 

while all studies use data, not all use baseline data. Likewise, all studies 

require analyses, but not all use models or simulation to conduct analyses. 

We tailored the research standards we identified as relevant to the MCS, as

shown in table 1. 

Table 1: Generally Accepted Research Standards Relevant to MCS Requirements 

Design: The Study is well designed 

I. Study plan, scope, and objectives follow Defense Planning Guidance

I.a (Do the study scope and objectives fully address the charter presented in the 2004 Defense Planning Guidance?) 

I.a.1 Does the study plan address specified guidance? 

I.b Was the study plan followed?

I.c Were deviations from the study plan explained and documented? 

I.d Was the study plan updated over the course of the study and the updates explicitly identified in the study and updated

study plan? 

II Assumptions and constraints are reasonable and consistent

II.a Are assumptions and constraints explicitly identified? 

II.a.1 (Are the study assumptions necessary and reasonable?) 

II.b Do the study assumptions support a sound analysis? 

II.c Are the assumptions used in analyses common throughout the study and models? 

II.d Do the assumptions contribute to an objective and balanced research effort? 

III Scenarios and threats are reasonable 

III.a. Are scenarios traceable back to formal guidance? 

III.b Were the threat scenarios validated and Defense Intelligence Agency approved and documented? 

III.c Do scenarios represent a reasonably complete range of conditions? 

III.d (Were the threats varied to allow for the conduct of sensitivity analysis?) 

Execution: The study is well executed

IV Methodology is successfully executed 

IV.a Was the study methodology executed consistent with the (MCS) study plan and schedule? 

Page 19 GAO-06-938 Defense Transportation
Appendix I: Scope and Methodology 

IV.b (Does the methodology support accomplishing the objectives presented in the study plan?) 

IV.c Were the models used to support the analyses adequate for their intended purpose? 

IV.d Were the model input data properly generated to support the methodology? 

V (Analytical ) Baseline data and other data used to support study and analyses validated, verified, and approved 

V.a Is the (analytical) baseline fully and completely identified and used consistently throughout the study for the various 

analyses? 

V.b Were data limitations identified (and the impact of the limitations fully explained?) 

V.c Were the (baseline security posture) data verified and validated? 

V.d Was the data verification and validation process documented? 

VI Models, simulations, and verification, validation, and accreditation are reasonable 

VI.a Was a VV&A accreditation report that addresses the models and data certification signed by the study director and 

included in the report? 

VI.b Were modeling and simulation limitations identified and explained? 

VI.c Has each model in the study been described? 

VI.d Are the model processes clearly explained, documented and understood?

VII Measures of effectiveness (MOEs) and essential elements of analysis (EEAs) are addressed

VII.a (Do MOEs adhere to the guidance in the study terms of reference?) 

VII.b (Are the MOEs fully addressed in the study?) 

VII.c (Are the EEAs addressed in the study?) 

Presentation of results: Timely, complete, accurate, concise, and relevant to the client and stakeholders 

VIII Presentation of results support findings 

VIII.a Does the report address the objectives?

VIII.b Does the report present an assessment that is well documented and conclusions that are supported by the analyses?

VIII.c Are conclusions sound and complete? 

VIII.d Are recommendations supported by analyses? 

VIII.e Is a realistic range of options provided? 

VIII.f Are the study results presented in the report in a clear manner? 

VIII.g Are study participants/stakeholders (i.e., services and Combatant Commands) informed of the study results and 

recommendations? 

Source: GAO analysis of industry and DOD study and research standards. 

We used these relevant standards as our criteria to assess the reported 

MCS results. All eight key areas of the study process were considered to 

have equal importance relative to the soundness and completeness of the 

study; that is, a sufficiently serious concern in any category could raise 

questions concerning the adequacy and completeness of the report. The 

analysts independently reviewed evidence relevant to each subquestion, 

including the study itself, the study Terms of Reference, and its strategic 

planning guidance. For each of the subquestions in the key study process 

areas, the analysts determined whether (1) the evidence had no limitations 

Page 20 GAO-06-938 Defense Transportation
Appendix I: Scope and Methodology 

or raised no concerns, (2) the evidence had some limitations or raised 

some concerns, (3) the evidence had significant limitations or raised 

significant concerns, or (4) we could not determine the extent of 

limitations or concerns because there was not sufficient information. The 

analysts then met, compared, and discussed their individual assessments, 

and reached an overall assessment for each subquestion. Areas of the 

study where we identified either “some” or “significant” limitations or 

concerns were considered to affect the adequacy or completeness of the 

study. Additionally, areas of the study that could not be assessed because 

of the lack of supporting documentation were considered to affect the 

credibility of the study. 

Throughout our review PA&E officials told us that the documentation 

needed to support and verify the key analytical and decision-making 

processes used to conduct the MCS, documentation that was vetted and 

approved by DOD leadership and all of the study participants, would not 

be completed and available for our review until the study report was 

issued. However, after the report was issued, we were told that the report

provides all of the supporting documentation needed and that the other 

documentation we requested could not be provided. As a result, we were 

unable to determine the adequacy and completeness of the analytical and 

decision-making processes that supported the MCS effort to evaluate the 

credibility of the study. We believe these processes are significant to the 

credibility of the study and its results. 

We conducted our review from July 2004 through July 2006 in accordance 

with generally accepted government auditing standards. 

Page 21 GAO-06-938 Defense Transportation
Appendix II: Comments from the Department Appendix II: Comments from the Department of Defense 

of Defense 

Page 22 GAO-06-938 Defense Transportation
Appendix II: Comments from the Department 

of Defense 

Page 23 GAO-06-938 Defense Transportation
Appendix II: Comments from the Department 

of Defense 

See comment 1. 

See comment 2. 

See comment 3. 

Page 24 GAO-06-938 Defense Transportation
Appendix II: Comments from the Department 

of Defense 

See comment 4. 

See comment 5. 

Page 25 GAO-06-938 Defense Transportation
Appendix II: Comments from the Department 

of Defense 

GAO’s Responses to 

DOD’s Technical 

Comments 

1. DOD disagreed with our assessment that the modeled year—2012—did 

not place as much demand for mobility assets in support of smaller 

military operations, such as peacekeeping, as other years. DOD also 

stated that we incorrectly focused on the number of operations, not 

the level of effort. We disagree. The MCS report (Annex A to Appendix 

F) made no distinction between the number of lesser contingencies 

and the level of effort. Specifically, the Vignettes for Baseline Security 

Posture Analysis did not report the level of effort by year and instead 

aggregated the data, in many instances across several modeled years. 

Consequently, we compared the number of operations conducted in 

the model year. Throughout our review, PA&E officials consistently 

told us that the completed MCS report would contain all the 

documentation needed to support its analyses. Furthermore, although 

demand in the modeled year may exceed previous efforts, the MCS 

was chartered to assess the ability of the mobility system to support 

the National Military Strategy into the next decade. The size of the 

selected model year in relation to efforts conducted between 1941 and 

2006 is not at issue. As our report makes clear, our concern is that 

modeling what appears to be the least demanding year does not 

address whether the United States has sufficient capability to support 

national objectives during a peak demand period and may 

underestimate and underreport demands to senior decision makers. 

2. DOD disagreed with our observation that the MCS report does not 

provide a clear understanding of the direct relationship of warfighting 

objectives to transportation capabilities. We disagree. We understand 

that achieving a combat task requires delivering the right commodity 

to the right place at the right time. However, the specific combat tasks 

(e.g., attaining air superiority) necessary to satisfy the commander’s 

campaign objectives are not a direct measure of mobility capability. 

For example, the problems in using a single metric are reflected in the 

MCS Appendix H, where the MCS report states that “the study itself 

still had difficulty in evaluating the operational impact of the delivery 

of theater support elements,” adding that “we [DOD] were unable to 

develop a satisfactory mechanism to capture the linkage of the closely 

related, but delayed, follow-on support needed...” Finally, the MCS 

concludes that “there was no way to model a decrease in [Air Force] 

squadron effectiveness if this support was late. Additional effort is 

required to develop a methodology for evaluation the operational 

impact of support equipment availability.” We continue to believe, and 

DOD agreed with our recommendation, that warfighting metrics, in 

conjunction with mobility metrics, can give decision makers a full 

picture of the capabilities needed to meet a specific warfighting goal. 

Page 26 GAO-06-938 Defense Transportation
Appendix II: Comments from the Department 

of Defense 

3. DOD disagreed with our statement that the MCS was unable to model 

the flexible deterrent options/deployment order process and that the 

study in fact analyzed flexible deterrent option (FDO) movements to 

theater. We do not dispute that DOD analyzed FDO movements as part 

of the MCS analysis. However, the degree to which the MCS analyses 

successfully modeled FDOs is in question. The MCS report, Appendix 

H, stated that an individual FDO is essentially the same as a 

deployment order. It also states in that section that “Deployment 

orders [DEPORDS] are issued to deploy specific capabilities as 

commitment decisions are made. This was not modeled due to lack of 

data on how DEPORDS would have been issued for an MCO [major 

combat operation] deployment.” In the same paragraph, the MCS 

concludes that “the impact on the mobility system of the DEPORD 

process should be assessed in follow-on MCS analyses,” adding that 

“there is a data deficit on how to model and execute a DEPORD 

process.” Furthermore, the MCS report states that “additional analysis 

is required to investigate the implications of the DEPORD process 

decisions and provide data for future decision-makers to develop a 

DEPORD execution process.” 

4. DOD believes our statement concerning homeland defense missions is 

misleading and is not sure what the report means by “actual homeland 

defense demands.” We removed the word “actual” and clarified our 

report to discuss “demands derived from a well defined and approved 

concept of operations for homeland defense”, which were not 

available for the study according to the MCS report. Furthermore, in 

chapter 4, the MCS report states that “maintaining a dedicated 

capability to support multiple, nearly simultaneous homeland 

defense/civil support events concurrent with the peak demand period 

of two overlapping warfights, greatly exceeds programmed lift 

capabilities”. This raises questions about the conclusions of the MCS 

that there are adequate mobility capabilities to meet national security 

objectives. Also, in Chapter 3 of the MCS report, it states that the DOD 

homeland defense concept of operations required refinement and was 

one of nine issue areas cited within the homeland defense portion of 

the study that “need to be addressed and investigated in more detail.” 

All of these nine areas potentially impact mobility support for 

homeland defense operations. Moreover, the MCS Executive Summary 

notes that reassessment of these missions is required as DOD’s role in 

homeland defense evolves. The MCS report, chapter 4, concludes by 

calling for further refinement of mission requirements, continuing risk 

assessments, and an effort to determine corresponding mobility 

solutions. We continue to believe that the MCS conclusion that 

adequate mobility capability exists is conditional given the results of 

Page 27 GAO-06-938 Defense Transportation
Appendix II: Comments from the Department 

of Defense 

the homeland defense portion of the study and that the accuracy and 

completeness of the data, modeling, and results for this portion of the 

MCS remain in question. 

5. We disagree with DOD’s characterization that our information was 

“misleading” regarding the adequacy of some aspects of the MCS’ 

modeling and data. Furthermore, we continue to disagree with DOD’s 

statement that the models and data used by the study were sound and 

adequate to assess relevant aspects of missions required to support the 

National Military Strategy, and that the results of the study are valid. In 

this report, as in our September 14, 2005 report,1

 we reaffirm our 

concern that the data and models used by the study may not be sound 

and the results may not be valid since the verification, validation, and 

accreditation (VV&A) of the models and data used to conduct the 

study was not done in accordance with DOD policy or relevant 

research standards. VV&A of models and data reduces the risk 

inherent in the use of models and simulations by improving the 

credibility of modeling and simulation results. We do not dispute 

DOD’s assertion that it has relied upon the same models to produce 

mobility studies done “since the end of the Cold War”. However, as we 

discuss in our report, the MCS report fails to explain or qualify the 

impact that identified data or modeling limitations might have on its 

results. For example, in the MCS chapter 4, entitled Operational Data, 

the MCS states that “data deficiencies negatively affected MCS’s ability 

to use current execution data to project future requirements and 

assess system performance.” Unclear is the extent to which these 

deficiencies impacted the MCS ability to meet the objective of 

identifying mobility capability gaps, overlaps, or excesses and provide 

associated risk assessments, an MCS objective. Similarly, in the 

section of chapter 4 entitled Analysis Tools, the report states that 

“MCS analysis revealed several deficiencies in existing mobility 

models.” The section concludes with five recommended tool 

enhancements but it does not explain the impact that the absence of 

these enhanced tools may have. We continue to believe that because of 

these modeling and data limitations, the MCS may have incorrectly 

estimated the future mobility requirements needed to support 

homeland defense missions, smaller contingencies, and major combat 

operations. 

 1

 See GAO, Defense Transportation: Opportunities Exist to Enhance the Credibility of the 

Current and Future Mobility Capabilities Studies, GAO-05-659R (Washington, D.C., 

September 14, 2005), for a more detailed discussion. 

Page 28 GAO-06-938 Defense Transportation
Appendix III: GAO

A

 Contact and Staff 

cknowledgments 

Page 29 GAO-06-938

Appendix III: GAO Contact and Staff 

Acknowledgments 

William M. Solis, (202) 512-8365 or solisw@gao.gov

Ann Borseth, Assistant Director; Brian Lepore, Assistant Director; 

Nabajyoti Barkakati; Renee Brown; Claudia Dickey; Ron La Due Lake; 

Oscar Mardis; Deborah Owolabi; Kenneth Patton; and Stephen Woods 

made significant contributions to this report. 

Defense Transportation

GAO Contact 

Acknowledgments 

(350558) 
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER