Defense Travel System: Reported Savings Questionable and Implementation Challenges Remain
Published: 2006-09-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 1995, the Department of Defense (DOD) began an effort to implement a standard departmentwide travel system. The Defense Travel System (DTS) is envisioned as DOD's standard end-to-end travel system. This report is a follow-up to GAO's January 2006, report which highlighted DTS implementation problems. Because of continued congressional interest in DTS, GAO initiated this follow-up audit under the Comptroller General's statutory authority. GAO determined whether (1) two key assumptions made in the September 2003 economic analysis were reasonable, (2) DOD is taking action to ensure full utilization of DTS and gathering the data needed to monitor DTS utilization, and (3) DOD has resolved the previously identified problems with DTS flight information. To address the above objectives, GAO (1) reviewed the September 2003 DTS economic analysis, (2) analyzed DTS utilization data, and (3) analyzed DTS flight information.
section_title: What GAO Found
paragraphs: GAO's analysis of the September 2003 DTS economic analysis found that the two key assumptions used to estimate annual net savings were not based on reliable information. Two cost components represent the majority of the over $56 million in estimated net savings--personnel savings and reduced commercial travel office (CTO) fees. In regard to the personnel savings, GAO's analysis found that the $24.2 million of personnel savings related to the Air Force and the Navy was not supported. Air Force and Navy DTS program officials stated that they did not anticipate a reduction in the number of personnel, but rather the shifting of staff from the travel function to other functions. The Naval Cost Analysis Division stated that the Navy will not realize any tangible personnel cost savings from the implementation of DTS. In regard to the CTO fees, the economic analysis assumed that 70 percent of all DTS airline tickets would either require no intervention or minimal intervention from the CTOs, resulting in an estimated annual net savings of $31 million. However, the sole support provided by the DTS program office was an article in a trade industry publication. The article was not based on information related to DTS, but rather on the experience of one private sector company. Furthermore, the economic analysis was not prepared in accordance with guidance prescribed by OMB and DOD. DOD guidance stated that the life-cycle cost estimates should be verified by an independent party, but this did not occur. The economic analysis did not undertake an assessment of the effects of the uncertainty inherent in the estimates of benefits and costs. Because an economic analysis uses estimates and assumptions, it is critical that the imprecision in both the underlying data and assumptions be understood. Such an assessment is referred to as a sensitivity analysis. DOD acknowledged that DTS is not being used to the fullest extent possible, but lacks comprehensive data to effectively monitor its utilization. DOD's utilization data are based on a model that was developed in calendar year 2003. However, the model has not been completely updated to reflect actual DTS usage. The lack of accurate utilization data hinders management's ability to monitor progress toward the DOD vision of DTS as the standard travel system. GAO also found that the military services have initiated actions that are aimed at increasing the utilization of DTS. Finally, GAO found that DTS still has not addressed the underlying problems associated with weak requirement management and system testing. While DOD has acted to address concerns GAO previously raised, GAO found that DTS's requirements are still ambiguous and conflicting. For example, DTS displaying up to 25 flights for each inquiry is questionable because it is unclear whether this is a valid requirement. Until DOD improves DTS's requirement management practices, the department will not have reasonable assurance that DTS can provide the intended functionality.

section_title: Letter
paragraphs: In 1995, the Department of Defense (DOD) embarked upon the daunting  challenge of implementing a standard, departmentwide travel system in  response to a report by the DOD Task Force to Reengineer Travel. The  report pinpointed three principal causes for DOD’s inefficient travel  system: (1) travel policies and programs were focused on compliance with  rigid rules rather than mission performance, (2) travel practices did not  keep pace with travel management improvements implemented by  industry, and (3) the various existing travel systems were not integrated.  To address these concerns, DOD established the Program Management  Office—Defense Travel System (PMO-DTS) to acquire travel services that  would be used DOD-wide. The department launched this program with the  goal of replacing existing travel systems with a single departmentwide  system to more effectively support nonintegrated travel processes and  procedures across its component organizations. The Defense Travel  System (DTS) is envisioned as being the department’s standard end-to-end  travel system.
The department estimates that DTS will be fully deployed at all 11,000  intended locations during fiscal year 2007. The September 2003 economic  analysis noted that DTS, when fully implemented, would result in annual  net savings of over $56 million. The economic analysis noted that savings  would be realized by the department during fiscal years 2009-2016. In  December 2003, the department’s Chief Information Officer approved a  DTS funding level of approximately $564 million. Of this amount, the  contract for the design, development, and deployment of DTS was for  about $264 million. The remaining costs are associated with areas such as  the operation and maintenance of DTS, operation of the PMO-DTS, the  voucher payment process, and management and oversight of the  numerous contracted commercial travel offices (CTO).
This report is a follow-up to our September 2005 testimony and January  2006 report in which we highlighted problems encountered by the  department in its efforts to successfully implement DTS. One of our major  findings was that DOD did not have reasonable assurance that flight  information was properly displayed for DOD travelers because the  department failed to properly test the system interfaces through which the  data are accessed for display. We further noted that the continued use of  the existing legacy travel systems at locations where DTS has been  deployed results in underutilization of DTS and reduces the envisioned  savings.
The objectives of this audit were to determine whether (1) two key  assumptions related to the estimated cost savings in the September 2003  economic analysis were reasonable, (2) DOD is taking action to ensure full  utilization of DTS and gathering the data needed to monitor DTS  utilization, and (3) DOD has resolved the previously identified problems  with properly displaying DTS flight information. To address the first  objective, we obtained and reviewed the September 2003 economic  analysis to (1) ascertain if the economic analysis was prepared in  accordance with criteria prescribed by the Office of Management and  Budget (OMB) and DOD; (2) analyze two key assumptions on which the  majority of the estimated savings were predicated—personnel savings and  reduced CTO fees; and (3) analyze the underlying supporting  documentation related to these two assumptions. In addition, we  interviewed PMO-DTS and military service officials to obtain an  understanding of the basis they used to compute the reported annual net  savings of over $56 million.
To address the second objective, we obtained and analyzed DTS utilization  data from the PMO-DTS. We also met with military service officials to  obtain an understanding of the efforts they have underway to help ensure  the full utilization of DTS. Further, we obtained from the military services  an understanding of the data they used to monitor DTS utilization. Finally,  to address the third objective, we analyzed 246 U.S. General Services  Administration (GSA) city pair flights to determine if the information  being displayed to the traveler was consistent with DTS’s stated  requirement. We also met with PMO-DTS and contractor officials.
Because of the continued widespread congressional interest in DTS, this  assignment was performed at our initiative under the statutory authority  provided to the Comptroller General of the United States. Our work  focused on the validity of the assumptions that were the principal drivers  of the net annual estimated savings of over $56 million. We determined  that the data were sufficiently reliable for this purpose. We did not review  the accuracy and reliability of the specific dollar amounts shown in the  September 2003 economic analysis. Our work was performed from  October 2005 through July 2006 in accordance with U.S. generally  accepted government auditing standards. Details on our scope and  methodology are included in appendix I. We requested comments on a  draft of this report from the Secretary of Defense or his designee. We  received written comments from the Under Secretary of Defense  (Personnel and Readiness), which are reprinted in appendix II.
subsections: section_title: Background
paragraphs: In September 1993, the National Performance Review recommended an  overhaul of DOD’s temporary duty (TDY) travel system. In response, DOD  created the DOD Task Force to Reengineer Travel to examine the travel  process. The task force found that the current process was expensive to  administer and was neither customer nor mission oriented with the net  result being a travel process that was costly, inefficient, fragmented, and  did not support DOD’s needs. On December 13, 1995, the Under Secretary  of Defense for Acquisition and Technology and the Under Secretary of  Defense (Comptroller)/Chief Financial Officer issued a memorandum,  “Reengineering Travel Initiative,” establishing the PMO-DTS to acquire  travel services that would be used DOD-wide. Additionally, in a 1997  report to Congress, the DOD Comptroller pointed out that the existing  DOD TDY travel system was never designed to be an integrated system.  The report stated that because there was no centralized focus on the  department’s travel practices, the travel policies were issued by different  offices and the process had become fragmented and “stovepiped.” The  report further noted that there was no vehicle in the current structure to  overcome these deficiencies, as no one individual within the department  had specific responsibility for management control of DOD TDY travel.
DOD management and oversight of the DTS program has varied over the  years. DTS was designated a “Special Interest” program in 1995. It retained  this status until May 2002 when it was designated a major automated  information system, with the Defense Finance and Accounting Service  (DFAS) being designated as the lead component for the program. This  meant that DFAS was responsible for the management oversight of DTS  program acquisition, including DTS compliance with the required DOD  acquisition guidance.
In September 2003, DOD finalized its economic analysis for DTS in  preparation for a milestone decision review. The highlights of the  economic analysis are shown in table 1. In December 2003, the DOD Chief  Information Officer granted approval for DTS to proceed with full  implementation throughout the department.
In October 2005, DOD established the Business Transformation Agency  (BTA) to advance DOD-wide business transformation efforts, particularly  with regard to business systems modernization. DOD believes it can better  address managing defensewide business transformation, which includes  planning, management, organizational structures, and processes related to  all key business areas, by first transforming business operations to support  the warfighter, while also enabling financial accountability across DOD.  BTA operates under the authority, direction, and control of the Under  Secretary of Defense for Acquisition, Technology, and Logistics, who is the  vice chair of the Defense Business Systems Management Committee— which serves as the highest ranking governing body for business systems  modernization activities. Among other things, BTA includes a Defense  Business Systems Acquisition Executive who is responsible for centrally  managing 28 DOD-wide business projects, programs, systems, and  initiatives—one of which is DTS. In October 2004, responsibility for the  policies and procedures related to the management of commercial travel  throughout DOD transferred to the Office of the Under Secretary of  Defense (Personnel and Readiness).
section_title: Validity of DTS Economic Analysis Questionable
paragraphs: Our analysis of the September 2003 DTS economic analysis found that two  key assumptions used to estimate cost savings were not based on reliable  information. Consequently, the economic analysis did not serve to help  ensure that the funds invested in DTS were used in an efficient and  effective manner. Two primary areas represented the majority of the over  $56 million of estimated annual net savings DTS was expected to realize— personnel savings and reduced CTO fees. However, the estimates used to  generate these savings were unreliable. Further, DOD did not effectively  implement the policies relating to developing economic analyses for  programs such as DTS. Effective implementation of these policies should  have highlighted the problems that we found and allowed for appropriate  adjustments so that the economic analysis could have served as a useful  management tool in making funding decisions related to DTS—which is  the primary purpose of this analysis. While the department’s system  acquisition criteria do not require that a new economic analysis be  prepared, the department’s business system investment management  structure provides an opportunity for DOD management to assess whether  DTS is meeting its planned cost, schedule, and functionality goals.
subsections: section_title: Personnel Savings Are Unrealistic
paragraphs: The economic analysis estimated that the annual personnel savings was  over $54 million, as shown in table 2.
As shown in table 2, approximately 45 percent of the estimated savings, or  $24.2 million was attributable to the Air Force and Navy. The assumption  behind the personnel savings computation was that there would be less  manual intervention in the processing of travel vouchers for payment, and  therefore fewer staff would be needed. However, based on our discussions  with Air Force and Navy DTS program officials, it is questionable as to  how the estimated savings will be achieved. Air Force and Navy DTS  program officials stated that they did not anticipate a reduction in the  number of personnel with the full implementation of DTS, but rather the  shifting of staff to other functions. According to DOD officials responsible  for reviewing economic analyses, while shifting personnel to other  functions is considered a benefit, it should be considered an intangible  benefit rather than tangible dollar savings since the shifting of personnel  does not result in a reduction of DOD expenditures. Also, as part of the  Navy’s overall evaluation of the economic analysis, program officials  stated that “the Navy has not identified, and conceivably will not  recommend, any personnel billets for reduction.” Finally, the Naval Cost  Analysis Division (NCAD) October 2003 report on the economic analysis  noted that it could not validate approximately 40 percent of the Navy’s  total costs, including personnel costs, in the DTS life-cycle cost estimates  because credible supporting documentation was lacking. The report also  noted that the PMO-DTS used unsound methodologies in preparing the  DTS economic analysis.
The extent of personnel savings for the Army and defense agencies, which  are reported as $16 million and $6.3 million respectively, is also unclear.  The Army and many defense agencies use DFAS to process their travel  vouchers, so the personnel savings for the Army and the defense agencies  were primarily related to reductions in DFAS’s costs. In discussions with  DFAS officials, they were unable to estimate the actual personnel savings  that would result since they did not know (1) the number of personnel,  like those at the Air Force and Navy, that would simply be transferred to  other DFAS functions or (2) the number of personnel that could be used to  avoid additional hiring. For example, DFAS expects that some of the  individuals assigned to support the travel function could be moved to  support its ePayroll program. Since these positions would need to be filled  regardless of whether the travel function is reduced, transferring  personnel from travel to ePayroll would reduce DOD’s overall costs since  DFAS would not have to hire additional individuals.
section_title: Savings Associated with Reduction of CTO Fees Are Unknown
paragraphs: According to the September 2003 economic analysis, DOD expected to  realize annual net savings of $31 million through reduced fees paid to the  CTOs because the successful implementation of DTS would enable the  majority of airline tickets to be acquired with either no or minimal  intervention by the CTOs. These are commonly referred to as “no touch”  transactions. However, DOD did not have a sufficient basis to estimate the  number of transactions that would be considered “no touch” since (1) the  estimated percentage of transactions that can be processed using the “no  touch” was not supported and (2) the analysis did not properly consider  the effects of components that use management fees, rather than  transaction fees, to compensate the CTOs for services provided. The  weaknesses we identified with the estimating process raise serious  questions as to whether DOD will realize substantial portions of the  estimated annual net savings of $31 million.
subsections: section_title: “No Touch” Transaction Volume Estimates Are Not Supported
paragraphs: DOD arrived at the $31 million of annual savings in CTO fees by estimating  that 70 percent of all DTS airline tickets would be considered “no touch”  and then multiplying these tickets by the savings per ticket in CTO fees.  However, a fundamental flaw in this analysis was that the 70 percent  assumption had no solid basis. We requested, but the PMO-DTS could not  provide, any analysis of travel data to support the assertion. Rather, the  sole support provided by the PMO-DTS was an article in a travel industry  trade publication. The article was not based on information related to  DTS, but rather on the experience of one private sector company.
The economic analysis assumed that DOD could save about $13.50 per “no  touch” ticket. Since that analysis, DOD has awarded one contract that  specifically prices transactions using the same model as that envisioned by  the economic analysis. This contract applies to the Defense Travel Region  6 travel area. During calendar year 2005, the difference in fees for “no  touch” transactions and the transactions supported by the current process  averaged between $10 and $12, depending on when the fees were incurred  because the contract rates changed during 2005. In analyzing travel  voucher data for Region 6 for calendar year 2005, we found that the  reported “no touch” rate was, at best 47 percent—far less than the 70  percent envisioned in the economic analysis.
PMO-DTS program officials stated they are uncertain as to why the  anticipated 70 percent “no touch” was not being achieved. According to  PMO-DTS program officials, this could be attributed, in part, to the DOD  travelers being uncomfortable with the system and making reservations  without using a CTO. Although this may be one reason, other factors may  also affect the expected “no touch” fee. For example, we were informed  that determining the airline availability and making the associated  reservation can be accomplished, in most cases, rather easily. However,  obtaining information related to hotels and rental cars and making the  associated reservation can be more problematic because of the limitations  in the data that DTS is able to obtain from its commercial sources.  Accordingly, while a traveler may be able to make a “no touch” reservation  for the airline portion of the trip, the individual may need to contact the  CTO in order to make hotel or rental car reservations. When this occurs,  rather than paying a “no touch” fee to the CTO, DOD ends up paying a  higher fee, which eliminates the savings estimated in the economic  analysis.
The economic analysis assumed that (1) DOD would be able to modify the  existing CTO contracts to achieve a substantial reduction in fees paid to a  CTO when DTS was fully implemented across the department and (2) all  services would use the fee structure called for in the new CTO contracts.  The first part of the assumption is supported by results of the CTO  contract for DOD Region 6 travel. The fees for the DTS “no touch”  transactions were at least $10 less than if a CTO was involved in the  transactions. However, to date, the department has experienced difficulty  in awarding new contracts with the lower fee structure. On May 10, 2006,  the department announced the cancellation of the solicitation for a new  contract. According to the department, it decided that the solicitation  needed to be rewritten based on feedback from travel industry  representatives at a March 28, 2006, conference. The department  acknowledged that the “DTS office realized its solicitation didn’t reflect  what travel agency services it actually needed.” The department would  not say how the solicitation would be refined, citing the sensitivity of the  procurement process. The department also noted that the new solicitation  would be released soon, but provided no specific date.
section_title: Navy Impact of CTO Management Fees Not Adequately Considered
paragraphs: The economic analysis assumed that the Navy would save about   $7.5 million, almost 25 percent, of the total savings related to CTO fees  once DTS is fully deployed. The economic analysis averaged the CTO fees  paid by the Army, the Air Force, and the Marine Corps—which amounted  to about $18.71 per transaction—to compute the savings in Navy CTO fees.  Using these data, the assumption was made in the economic analysis that  a fee of $5.25 would be assessed for each ticket, resulting in an average  savings of $13.46 per ticket for the Navy ($18.71 minus $5.25). While this  approach may be valid for the organizations that pay individual CTO fees,  it may not be representative for organizations such as the Navy that pay a  management fee. The management fee charged the Navy is the same  regardless of the involvement of the CTO—therefore, the reduced “no  touch” fee would not apply.
We were informed by Navy DTS program officials that they were  considering continuing the use of management fees after DTS is fully  implemented. According to Navy DTS program officials, they paid about  $14.5 million during fiscal year 2005 for CTO management fees, almost $19  per ticket for approximately 762,700 tickets issued. Accordingly, even if  the department arrives at a new CTO contract containing the new fee  structure or fees similar to those of Region 6, the estimated savings related  to CTO fees for the Navy will not be realized if the Navy continues to use  the management fee concept.
section_title: Effective Implementation of Existing Policies Should Have Identified Problems with the Economic Analysis
paragraphs: Effective implementation of DOD guidance would have detected the types  of problems discussed above and resulted in an economic analysis that  would have accomplished the stated objective of the process—to help  ensure that the funds invested in DTS were used efficiently and effectively.  DOD policy and OMB guidance require that an economic analysis be  based on facts and data and be explicit about the underlying assumptions  used to arrive at estimates of future benefits and costs. Since an economic  analysis deals with costs and benefits occurring in the future, assumptions  must be made to account for uncertainties. DOD policy recognizes this and  provides a systematic approach to the problem of choosing the best  method of allocating scarce resources to achieve a given objective.
A sound economic analysis recognizes that there are alternative ways to  meet a given objective and that each alternative requires certain resources  and produces certain results. The purpose of the economic analysis is to  give the decision maker insight into economic factors bearing on  accomplishing the objectives. Therefore, it is important to identify factors,  such as cost and performance risks and drivers, which can be used to  establish and defend priorities and resource allocations. The DTS  economic analysis did not comply with the DOD policy, and the  weaknesses we found should have been detected had the DOD policy been  effectively implemented. The PMO-DTS had adequate warning signs of the  potential problems associated with not following the OMB and DOD  guidance for developing an effective economic analysis. For example, as  noted earlier, the Air Force and Navy provided comments when the  economic analysis was being developed that the expected benefits being  claimed were unrealistic. Just removing the benefits associated with  personnel savings from the Air Force and Navy would have reduced the  overall estimated program cost savings by almost 45 percent. This would  have put increased pressure on the credibility of using a 70 percent “no  touch” utilization rate. The following are examples of failures to effectively  implement the DOD policy on conducting economic analyses and the  adverse effects on the DTS economic analysis.
The DTS life-cycle cost estimates portion of the economic analysis was not  independently validated as specified in DOD’s guidance. PMO-DTS  officials acknowledged that there was not an independent assessment of  the DTS life-cycle cost estimates. However, they noted that the  department’s Office of Program Analysis and Evaluation had provided  comments on the economic analysis. Program Analysis and Evaluation  officials informed us that they did not perform an independent assessment  of the DTS economic analysis because the data were not available to  validate the reliability of that analysis. Program Analysis and Evaluation  officials also noted that they had raised similar concerns about the July  2003 economic analysis, but those issues had not been resolved when the  September 2003 economic analysis was provided for their review. Because  the September 2003 DTS life-cycle cost estimates were not independently  assessed, the department did not have reasonable assurance that the  reported estimates were realistic, that the assumptions on which the  analysis was based were valid, or that the estimated rate of return on the  investment could reasonably be expected to be realized.
The September 2003 DTS economic analysis did not undertake an  assessment of the effects of the uncertainty inherent in the estimates of  benefits and costs, as required by DOD and OMB guidance. Because an  economic analysis uses estimates and assumptions, it is critical that a  sensitivity analysis be performed to understand the effects of the  imprecision in both underlying data and modeling assumptions. This  analysis is required since the estimates of future benefits and costs are  subject to varying degrees of uncertainty. For example, according to  DOD officials, the number of travel transactions has remained relatively  stable over the years. On the other hand, as discussed previously, the  number of transactions that can be processed as “no touch” is unknown.  Sensitivity analysis refers to changing the value of a given variable in a  model to gauge the effect of change on model results. More importantly, it  identifies key elements—data and assumptions—as discussed above—and  varies a single element while holding the others constant to determine  what amount of change in that element is required to raise or lower the  resulting dominant benefit and cost elements by a set amount. In this way,  data and assumptions can be risk-ranked for decisionmaking and auditing.  In the case of DTS, we requested that the PMO-DTS determine the effects  of a change in “no touch” transaction percentage. With all other factors  remaining the same, DTS would have to achieve a 35 percent “no touch”  transaction rate just to break even—where tangible costs and benefits are  equal. Had DOD performed such an analysis, it would have understood  that depending solely on an industry trade publication as its support for  the “no touch” transaction percentage had major implications on the  potential savings.
section_title: New Economic Analysis Not Required by DOD Criteria
paragraphs: Although the September 2003 economic analysis was not based on  supportable data, the department’s criteria do not require that a new  economic analysis be prepared. DTS has already completed all of the  major milestones related to a major automated system, which require that  an economic analysis be prepared or at least updated to reflect the current  assumptions and the related costs and benefits. However, the fiscal year  2005 defense authorization act requires the periodic review, but not less  than annually, of every defense business system investment. Further, the  department’s April 2006 guidance notes that the annual review process  “provides follow-up assurance that information technology investments,  which have been previously approved and certified, are managed properly,  and that promised capabilities are delivered on time and within budget.” If  effectively implemented, this annual review process provides an excellent  opportunity for DOD management to assess whether DTS is meeting its  planned cost, schedule, and functionality goals. Going forward, such a  review could serve as a useful management tool in making funding and  other management decisions related to DTS.
section_title: DTS Remains Underutilized by the Military Services
paragraphs: Our September 2005 testimony and January 2006 report noted the  challenge facing the department in attaining the anticipated DTS’s  utilization. While DOD has acknowledged the underutilization, we found  that across DOD, the department does not have reasonable quantitative  metrics to measure the extent to which DTS is actually being used.  Presently, the reported DTS utilization is based on a DTS Voucher Analysis  Model that was developed in calendar year 2003 using estimated data, but  over the years has not been completely updated with actual data. While  the military services have initiated actions to help increase the utilization  of DTS, they pointed out that ineffective DTS training is a contributing  factor to the lower than expected usage rate by the military services.
subsections: section_title: Metrics to Measure DTS Utilization Are Inadequate
paragraphs: The DTS Voucher Analysis Model was prepared in calendar year 2003 and  based on airline ticket and voucher count data that were reported by the  military services and defense agencies, but the data were not verified or  validated. Furthermore, PMO-DTS officials acknowledged that the model  has not been completely updated with actual data as DTS continues to be  implemented at the 11,000 sites. We found that the Air Force is the only  military service that submits monthly metrics to the PMO-DTS officials for  their use in updating the DTS Voucher Analysis Model. Rather than  reporting utilization based on individual site system utilization data, the  PMO-DTS continues to rely on outdated information in the reporting of  DTS utilization to DOD management and Congress. We have previously  reported that best business practices indicate that a key factor of project  management and oversight is the ability to effectively monitor and  evaluate a project’s actual performance against what was planned.
In order to perform this critical task, best business practices require the  adoption of quantitative metrics to help measure the effectiveness of a  business system implementation and to continually measure and monitor  results, such as system utilization. This lack of accurate and pertinent  utilization data hinders management’s ability to monitor its progress  toward the DOD vision of DTS as the standard travel system, as well as to  provide consistent and accurate data to Congress. With the shift of the  DTS program to BTA, which now makes DTS an enterprisewide endeavor,  improved metrics and training are essential if DTS is to be DOD’s  standard, integrated, end-to-end travel system for business travel.
Table 3 presents DTS’s reported percentage of utilization during the period  October 2005 through April 2006. PMO-DTS officials calculated these  utilization percentages by comparing the actual number of travel vouchers  processed through DTS to the outdated universe of travel transaction data  per the model, as described previously. Because the PMO-DTS was not  able to identify the total number of travel vouchers that should have been  processed through DTS (total universe of travel vouchers), the utilization  percentages shown in table 3 may be over- or understated.
PMO-DTS program officials confirmed that the reported utilization data  were not based on complete data because the department did not have  comprehensive information to identify the universe or the total number of  travel vouchers that should be processed through DTS. PMO-DTS program  and DTS military service officials agreed that the actual DTS utilization  rate should be calculated by comparing actual vouchers being processed  in DTS to the total universe of vouchers that should be processed in DTS.  The universe would exclude those travel vouchers that cannot be  processed through DTS, such as those related to permanent change of  station travel.
The Air Force was the only military service that attempted to obtain data  on (1) the actual travel vouchers processed through DTS and (2) those  travel vouchers eligible to be processed through DTS, but were not. These  data were site specific. For example, during the month of December 2005,  the PMO-DTS reported that at Wright-Patterson Air Force Base, 2,880  travel vouchers were processed by DTS, and the Air Force reported that  another 2,307 vouchers were processed through the legacy system—the  Reserve Travel System (RTS). Of those processed through RTS, Air Force  DTS program officials stated that 338 travel vouchers should have been  processed through DTS. DTS Air Force program officials further stated  that they submitted to the PMO-DTS the number of travel vouchers  processed through RTS each month. These data are used by the PMO-DTS  to update the DTS Voucher Analysis Model. However, neither the Air  Force nor the PMO-DTS have verified the accuracy and reliability of the  data. Therefore, the accuracy of the utilization rates reported for the Air  Force by the PMO-DTS is not known. As shown in table 3, PMO-DTS  officials reported utilization data for the Air Force from a low of 29  percent (January 2006) to a high of 48 percent (November 2005) during the  7-month period ending April 2006.
Because Army and Navy DTS program officials did not have the  information to identify the travel transactions that should have been  processed through DTS, the Army and Navy did not have a basis for  evaluating DTS utilization at their respective military locations and  activities. Furthermore, Navy DTS program officials indicated that the  utilization data that the PMO-DTS program officials reported for the Navy  were not accurate. According to Navy DTS program officials, the Navy’s  primary source of utilization data was the monthly metrics reports  provided by the PMO-DTS, but Navy DTS program officials questioned the  accuracy of the Navy utilization reports provided by the PMO-DTS.
For example, the Navy PMO-DTS utilization site report has a site name of  Ballston, Va.; however, Ballston, Va. is not listed on the map site names on  the DTS contractor’s database. As a result, the PMO-DTS Navy utilization  report for this location indicates no usage every month. Our analysis  indicated that this was 1 of at least 33 similar instances where no usage  was reported for a nonexistent location. Navy DTS program officials  stated that an effort is underway to “re-map” all Navy organizations to the  correct site name, but as of June 2006 this effort had not been completed.
Another example indicates the inconsistencies that exist in the different  information used by the Navy and the PMO-DTS program officials to  report utilization rates for the Navy. The PMO-DTS program officials  reported that the Navy had a total of 9,400 signed, original vouchers  processed through DTS during December 2005; however, this is less than  the 10,523 reported by the DTS contractor for the same month. According  to Navy DTS program officials, they have not been able to confirm  whether either figure is correct. Since the number of DTS vouchers is  required to calculate utilization, the Navy is unable to determine the  accuracy of the utilization metrics reported by the PMO-DTS officials, as  shown in table 3.
section_title: DOD Has Taken Steps to Improve DTS Utilization, but Further Action Is Needed
paragraphs: While the military services have issued various memorandums that direct  or mandate the use of DTS to the fullest extent possible at those sites  where DTS has been deployed, resistance still exists. As highlighted  below, deployed sites are still using non-DTS systems, or legacy systems,  to process TDY travel.
The Army issued a memorandum in September 2004 directing each Army  installation to fully disseminate DTS to all travelers within 90 to 180 days  after Initial Operating Capability (IOC) at each installation.  Subsequently in September 2005, DFAS officials reported that 390,388  travel vouchers were processed through the Army’s legacy system—the  Windows Integrated Automated Travel System, but DFAS officials could  not provide a breakout of how many of the 390,388 travel vouchers should  have been processed through DTS.
The Air Force issued a memorandum in November 2004 that stressed the  importance of using DTS once it was implemented at an installation. The  Air Force memorandum specifically stated that business, local, and group  travel vouchers should be electronically processed through DTS and that  travel claims should not be submitted to the local finance office for  processing. However, we found that Air Force travelers continued to  process travel claims through legacy systems, such as RTS. For example,  during the month of November 2005, the Air Force reported that 3,277  business vouchers, 1,875 local vouchers, and 1,815 group vouchers were  processed through RTS that should have been processed through DTS.  Additionally, a DFAS internal review analyzed Air Force vouchers during  the period January 2005 through June 2005, at locations where DTS was  deployed, and found that Air Force travelers used legacy systems to  process 79 percent of all routine TDY transactions.
The Navy issued a memorandum in May 2005 that directed the use of DTS  to generate travel orders throughout all Navy locations. Navy DTS  program officials reported in an April 2006 briefing that 18,300 travel  vouchers were processed in DTS during the month of March 2006, but that  over 90,000 travel vouchers were still being processed monthly through  the Integrated Automated Travel System—a legacy system.
Thus, despite memoranda issued by the military services, it appears that  DTS continues to be underutilized by the military services. As discussed in  our September 2005 testimony and January 2006 report, the unnecessary  continued use of the legacy travel systems results in the inefficient use of  funds because the department is paying to operate and maintain  duplicative systems that perform the same function—travel.
Besides the memorandums, DOD is taking other actions to increase DTS  utilization as the following examples illustrate.
The Assistant Secretary of the Army for Financial Management (Financial  & Accounting Oversight Directorate) holds monthly Senior Focus Group  meetings with the installation leadership of major commands to discuss  DTS utilization issues and possible corrective actions.
The Navy conducts quarterly video and telephone conferences with major  commands and contacts commands with low usage to determine the  causes for low DTS usage.
The PMO-DTS conducts monthly working group meetings with the military  service and defense agency DTS program officials to discuss DTS  functionality issues and concerns, DTS usage, and other related DTS  issues.
Although the military services have issued various memorandums aimed at  increasing the utilization of DTS, the military service DTS program  officials all pointed to ineffective training as a primary cause of DTS not  being utilized to a far greater extent. The following examples highlight the  concerns raised by the military service officials.
Army DTS program officials emphasized that the DTS system is complex  and the design presents usability challenges for users—especially for first- time or infrequent users. They added that a major concern is that there is  no PMO-DTS training for existing DTS users as new functionality is added  to DTS. These officials stated that the PMO-DTS does not do a good job of  informing users about functionality changes made to the system. We  inquired if the Help Desk was able to resolve the users’ problems, and the  Army DTS officials simply stated “no.” The Army officials further pointed  out that it would be beneficial if the PMO-DTS improved the electronic  training on the DTS Web site and made the training documentation easier  to understand. Also, improved training would help infrequent users adapt  to system changes. The Army officials noted that without some of these  improvements to resolve usability concerns, DTS will continue to be  extremely frustrating and cumbersome for travelers.
Navy DTS program officials stated that DTS lacks adequate user/traveler  training. The train-the-trainer concept of training system administrators  who could then effectively train all their travelers has been largely  unsuccessful. According to Navy officials, this has resulted in many  travelers and users attempting to use DTS with no or insufficient training.  The effect has frustrated users at each step of the travel process and has  discouraged use of DTS.
Air Force officials stated that new DTS system releases are implemented  with known problems, but the sites are not informed of the problems.  Workarounds are not provided until after the sites begin encountering  problems. Air Force DTS program officials stated that DTS releases did  not appear to be well tested prior to implementation. Air Force officials  also stated that there was insufficient training on new functionality. PMO- DTS and DTS contractor program officials believed that conference calls  to discuss new functionality with the sites were acceptable training, but  Air Force officials did not agree. The Air Force finance office was  expected to fully comprehend the information received from those  conference calls and provide training on the new functionality to  users/approvers, but these officials stated that this was an unrealistic  expectation.
section_title: Previously Reported DTS Requirements Management and Testing Deficiencies Have Not Been Resolved
paragraphs: Our September 2005 testimony and January 2006 report noted problems  with DTS’s ability to properly display flight information and traced those  problems to inadequate requirements management and testing. DOD  stated that it had addressed those deficiencies and in February 2006, we  again tested the system to determine whether the stated weaknesses had  been addressed. We found that similar problems continue to exist. We also  identified additional deficiencies in DTS’s ability to display flights that  comply with the Fly America Act. DTS’s inability to display flights that  comply with the Fly America Act places the traveler who purchases a  ticket or the individual authorizing, certifying, or disbursing a payment  made when a ticket is paid for directly by DOD through a centrally billed  account at unnecessary risk of personal liability. Once again, these  problems can be traced to ineffective requirements management and  testing processes. Properly defined requirements are a key element in  systems that meet their cost, schedule, and performance goals since they  define (1) the functionality that is expected to be provided by the system  and (2) the quantitative measures by which to determine through testing  whether that functionality is operating as expected.
We briefed PMO-DTS officials on the results of our tests and in May 2006  the officials agreed that our continued concerns about the proper display  of flight information and compliance with the Fly America Act were valid.  PMO-DTS officials stated that the DTS technology refresh, which is to be  completed in September 2006, should address some of our concerns.  While these actions are a positive step forward, they do not address the  fundamental problem that DTS’s requirements are still ambiguous and  conflicting—a primary cause of the previous problems. Until a viable  requirements management process is developed and effectively  implemented, the department (1) cannot develop an effective testing  process and (2) will not have reasonable assurance the project risks have  been reduced to acceptable levels.
subsections: section_title: Providing Complete Flight Information Has Been a Continuing Problem
paragraphs: In our earlier testimony and report, we noted that DOD did not have  reasonable assurance that the flights displayed met the stated DOD  requirements. Although DOD stated in each case that our concerns had  been addressed, subsequent tests found that the problems had not been  corrected. Requirements represent the blueprint that system developers  and program managers use to design, develop, and acquire a system.  Requirements should be consistent with one another, verifiable, and  directly traceable to higher-level business or functional requirements. It is  critical that requirements be carefully defined and that they flow directly  from the organization’s concept of operations (how the organization’s day- to-day operations are or will be carried out to meet mission needs).  Improperly defined or incomplete requirements have been commonly  identified as a cause of system failure and systems that do not meet their  cost, schedule, or performance goals.
Requirements represent the foundation on which the system should be  developed and implemented. As we have noted in previous reports,  because requirements provide the foundation for system testing,  significant defects in the requirements management process preclude an  entity from implementing a disciplined testing process. That is,  requirements must be complete, clear, and well documented to design and  implement an effective testing program. Absent this, an organization is  taking a significant risk that its testing efforts will not detect significant  defects until after the system is placed into production. Our February 2006  analysis of selected flight information disclosed that DOD still did not have  reasonable assurance that DTS displayed flights in accordance with its  stated requirements. We analyzed 15 U.S. General Services Administration  (GSA) city pairs, which should have translated into 246 GSA city pair  flights for the departure times selected. However, we identified 87 flights  that did not appear on one or more of the required listings based on the  DTS requirements. For instance, our analysis identified 44 flights  appearing on other DTS listings or airline sites that did not appear on the  9:00 am DTS listing even though those flights (1) met the 12-hour flight  window and (2) were considered GSA city pair flights—two of the key  DTS requirements the system was expected to meet.
After briefing PMO officials on the results of our analysis in February 2006,  the PMO-DTS employed the services of a contractor to review DTS to  determine the specific cause of the problems and recommend solutions. In  a March 2006 briefing, the PMO-DTS acknowledged the existence of the  problems, and identified two primary causes. First, part of the problem  was attributed to the methodology used by DTS to obtain flights from the  Global Distribution System (GDS). The PMO-DTS stated that DTS was  programmed to obtain a “limited” amount of data from GDS in order to  reduce the costs associated with accessing GDS. This helps to explain why  flight queries we reviewed did not produce the expected results. To  resolve this particular problem, the PMO-DTS proposed increasing the  amount of data obtained from GDS. Second, the PMO-DTS acknowledged  that the system testing performed by the contractor responsible for  developing and operating DTS was inadequate and, therefore, there was  no assurance that DTS would provide the data in conformance with the  stated requirements. This weakness was not new, but rather reconfirms  the concerns discussed in our September 2005 testimony and January 2006  report related to the testing of DTS.
Our analysis also found that DOD did not have reasonable assurance that  the system displayed flights in compliance with the requirements of the  Fly America Act. In 1996, Congress assigned the Administrator, GSA, the  responsibility to determine the situations for which appropriated funds  could be used consistent with the Fly America Act, and GSA has published  its rules in the Federal Travel Regulation (FTR). Within the basic  guidelines that GSA publishes, agencies must establish “internal  procedures” to ensure that agency reimbursements with federal funds for  travelers’ air carrier expenses are made only in compliance with the Fly  America Act and the FTR rules. As a result, DTS places the traveler who  purchases a ticket or the individual authorizing, certifying, or disbursing a  payment made when a ticket is paid for directly by DOD—such as those  tickets purchased using a centrally billed account—at unnecessary risk of  personal liability. DOD guidance expressly states that for code-sharing  airline tickets related to foreign travel (1) the entire airline ticket must be  issued by and on the U.S.-flag carrier (not necessarily the carrier operating  the aircraft) and (2) the flight must be between a centennial United States  and a foreign destination. If these conditions are not met, DOD requires a  determination that a U.S.-flag carrier is not available or use of a non-U.S.- flag carrier is necessary. These requirements are commonly referred to as  the Fly America Act requirements. According to PMO-DTS officials, DTS’s  requirements are intended to comply with the Fly America Act. However,  our analysis of March 2006 flight display data identified several instances  in which flights were displayed to the DOD traveler that did not meet the  requirements of the Fly America Act. For example, six of the first seven  flights displayed between Santiago, Chile, and San Antonio, Texas, did not  appear to comply with the Fly America Act requirements since they did  not involve a U.S.-flag carrier. More importantly, several flights that  appeared later in the listing and involved U.S.-flag carriers were more  advantageous to the traveler because they required less actual travel time.  Figure 1 shows the DTS display of flights.
According to DTS program officials, after our discussions relating to the  flight displays and compliance with the Fly America Act, they did a  “requirements scrub” to define the requirements that should be used to  display flights, including those requirements relating to displaying flights  that comply with the Fly America Act. The previous requirement stated  that “DTS shall examine international trip records for compliance with  DOD policy on the use of non-U.S.-flag carriers.” The revised requirement  relating to international flights stated that the system should display flights  that are (1) part of the GSA city pair program or (2) offered by U.S.  carriers. If the system cannot find flights that meet these criteria, then the  system is expected to instruct the user to contact their CTO to arrange the  flight. According to PMO-DTS officials, this change has been incorporated  into the production system. We conducted a limited nonstatistical test to  determine if the examples of flights not complying with the Fly America  Act identified in our earlier tests had been eliminated and found that these  flights no longer appeared on the DTS displayed flights. However, as we  noted, the DOD policy is compliant with the Fly America Act requirements  and this was a DTS requirement in effect when we identified the examples  of flight displays not complying with the Fly America Act. In effect, this is  another example of (1) inadequate testing by the DTS contractor and   (2) DOD’s inability to ensure the system is meeting its requirements. Until  DOD effectively analyzes and properly documents the functionality it  desires, it has little assurance that the proper requirements have been  defined.
section_title: DOD’s Planned Corrective Actions Will Not Address Fundamental Requirements Management Problems
paragraphs: While DOD’s planned actions, if effectively implemented, should address  several of the specific weaknesses we identified related to flight displays  and the Fly America Act, they fall short of addressing the fundamental  problems that caused those weaknesses—inadequate requirements  management. DTS’s requirements continue to be ambiguous. For example,  a system requirement was changed to “display,” that is, show the fares  relating to the full GSA city pair fare only if the GSA city pair fare with  capacity limits was not available. Based upon information provided by  PMO-DTS officials, after the requirement was supposed to have been  implemented, both fare types were shown on the DTS display screen.  PMO-DTS officials stated that although both fares were shown, DTS was  still expected to book the lower fare and that the requirement was really  designed to ensure that the lower fare was booked. This requirement is  ambiguous because it is not clear what the word “display” means in this  context. Based upon the stated requirement, the most common  interpretation would be that the word display implies information that is  provided (or shown) to the DOD traveler. However, based on the PMO- DTS official’s explanation, the word display, in fact, means the fare that is  booked. This type of ambiguity was one cause of problems we noted in the  past where testing did not identify system defects and DTS did not  properly display the proper flight information to the user.
Furthermore, DOD is currently undergoing a technology upgrade of DTS  that is scheduled for completion by September 30, 2006. This technology  upgrade is expected to provide additional functionality; however, DOD  still has not adequately defined the requirements that are needed to define  flight displays for DOD travelers. According to DTS program officials and  the contractor responsible for the technology upgrade, the upgrade is  intended to do the following:    Replace the current display of up to 25 flights on one page in a  predetermined order and separate the 25 flights into three categories— GSA city pair flights, Other Government Fares, and Other Unrestricted  Flights—and then sort the flights by additional criteria such as elapsed  travel time (rather than the current flight time), time difference from the  requested departure time, number of stops, and whether the flight is  considered a direct flight. This approach, if effectively implemented,  addresses one problem we noted with the current process where flight  time rather than elapsed travel time is used as one of the sorting criteria.  It will also present flights that have the shortest duration in relation to the  requested departure time at the top of the listing.
Display the prices on all flights returned to the traveler. The current  system displays the prices for the GSA city pair flights and allows the  traveler to request prices for up to 10 additional flights at a time. This  significantly improves the ability of the system to present information to  the traveler that can be used to select the best flight for the government  and allows the system to help ensure that the lowest cost flights are  selected by the user. This is especially true when a GSA city pair fare is  not available. According to DOD officials, it is cost prohibitive to obtain  the pricing information for non-GSA city pair flights using the current  technology.
Although these planned improvements should provide the DOD traveler  with better travel information, they still fall short of adequately defining  the requirements that should be used for displaying flights. For example,  DOD has retained a requirement to display 25 flights for each inquiry.  However, it has not determined (1) whether the rationale for that  requirement is valid and (2) under what conditions flights that are not part  of the GSA city pair program should be displayed. For example, we found  that several DTS flights displayed to the user “overlap” other flights.  Properly validating the requirements would allow DOD to obtain  reasonable assurance that its requirements properly define the  functionality needed and the business rules necessary to properly  implement that functionality. As previously noted, requirements that are  unambiguous and consistent are fundamental to providing reasonable  assurance that a system will provide the desired functionality. Until DOD  improves DTS requirement management practices, it will not have this  assurance.
section_title: Conclusions
paragraphs: Overhauling the department’s antiquated travel management practices and  systems has been a daunting challenge for DOD. While it was widely  recognized that this was a task that needed to be accomplished and  savings could result, the underlying assumptions in support of those  savings are not based on reliable data and therefore it is questionable  whether the anticipated savings will materialize. Even though the overall  savings are questionable, the successful implementation of DTS is critical  to reducing the number of stovepiped, duplicative travel systems  throughout the department. We have reported on numerous occasions that  reducing the number of business systems within DOD can translate into  savings that can be used for other mission needs. Furthermore, the shift of  DTS to BTA, which makes DTS an enterprisewide endeavor, should help in  making DTS the standard integrated, end-to-end travel system for business  travel. Management oversight is essential for this to become a reality.  Equally important, however, will be the department’s ability to resolve the  long-standing difficulties that DTS has encountered with its requirements  management and system testing. Until these issues are resolved, more  complete utilization of DTS will be problematic.
section_title: Recommendations for Executive Action
paragraphs: To improve the department’s management and oversight of DTS, which  has been declared a DOD enterprise business system, we recommend that  the Secretary of Defense direct the Under Secretary of Defense (Personnel  and Readiness) and the Director, Business Transformation Agency, to  jointly take the following four actions:    Evaluate the cost effectiveness of the Navy continuing with the CTO  management fee structure versus adopting the revised CTO fee structure,  once the new contracts have been awarded.
Develop a process by which the military services develop and use  quantitative data from DTS and their individual legacy systems to clearly  identify the total universe of DTS-eligible transactions on a monthly basis.  At a minimum, these data should be used to update the DTS Voucher  Analysis Model to report DTS actual utilization rates.
Require the PMO-DTS to provide a periodic report on the utilization of  DTS to the Under Secretary of Defense (Personnel and Readiness) and the  Director, Business Transformation Agency, once accurate data are  available. The report should continue until the department has reasonable  assurance that DTS is operating as intended at all 11,000 locations. The  report should identify at a minimum (1) the number of defense locations at  which DTS has been deployed, (2) the extent of DTS utilization at these  sites, (3) steps taken or to be taken by the department to improve DTS  utilization, and (4) any continuing problems in the implementation and  utilization of DTS.
Resolve inconsistencies in DTS requirements, such as the 25 flight display,  by properly defining the (1) functionality needed and (2) business rules  necessary to properly implement the needed functionality.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report from the Under  Secretary of Defense (Personnel and Readiness), which are reprinted in  appendix II. DOD concurred with three and partially concurred with one  of the recommendations. In regard to the recommendations with which  the department concurred, it briefly outlined the actions it planned to take  in addressing two of the three recommendations. For example, the  department noted the difficulties in obtaining accurate utilization data  from the existing legacy systems, but stated that the Office of the Under  Secretary of Defense (Personnel and Readiness) and BTA will evaluate  methods for reporting actual DTS utilization.
Additionally, DOD noted that the Defense Travel Management Office  developed and implemented a requirements change management process  on May 1, 2006. In commenting on the report, the department stated that  this process is intended to define requirements and track the entire life  cycle of the requirements development process. As reiterated in this  report, and discussed in our September 2005 testimony and January 2006  report, effective requirements management has been an ongoing concern,  and we fully support the department’s efforts to improve its management  oversight of DTS’s requirements. In this regard, the department needs to  have in place a process that provides DOD reasonable assurance that   (1) requirements are properly documented and (2) requirements are  adequately tested as recommended in our January 2006 report. This  process should apply to all existing requirements as well as any new  requirements. As discussed in this report, we reviewed some of the  requirements in May 2006, that were to have followed the new  requirements management process, and found problems similar to those  noted in our January 2006 report. While we did not specifically review the  new process, if it does not include an evaluation of existing requirements,  the department may continue to experience problems similar to those we  previously identified.
DOD partially concurred with our recommendation to evaluate the cost  effectiveness of the Navy continuing with the CTO management fee  structure. DOD stated that all military service secretaries should  participate in an evaluation to determine the most cost-effective payment  method to the CTOs. DOD’s response indicated that the Defense Travel  Management Office is currently procuring commercial travel services for  DOD worldwide in a manner that will ensure evaluation of cost  effectiveness for all services. If DOD proceeds with the actions outlined in  its comments, it will meet the intent of our recommendation.
Finally, DOD strongly objected to our finding that the personnel savings  are unrealistic. In its comments, the department stated that DOD is facing  an enormous challenge and the department continues to identify  efficiencies and eliminate redundancies to help leverage available funds.  We fully recognize that the department is attempting to improve the  efficiency and effectiveness of its business operations. In fact, the  Comptroller General of the United States testified in August 2006 that  increased commitment by the department to address DOD’s numerous  challenges represents an improvement over past efforts.
The fact remains, however, that the results of an economic analysis are  intended to help management decide if future investments in a given  endeavor are worthwhile. In order to provide management with this  information it is imperative that the underlying assumptions in an  economic analysis be supported by valid assumptions. The September  2003 economic analysis noted that personnel savings of $54.1 million, as  shown in table 2 of this report, would be realized by the department  annually for fiscal years 2009 through 2016. However, based upon our  review and analysis of documentation and discussion with department  personnel we found that the underlying assumptions in support of the  $54.1 million were not valid.
Furthermore, as noted in the report Air Force and Navy DTS program  officials stated that they did not anticipate a reduction in the number of  personnel with the full implementation of DTS. Further, as discussed in  the report, the Naval Cost Analysis Division review of the DTS economic  analysis noted that approximately 40 percent of the Navy’s total costs,  including personnel costs, in the DTS life-cycle cost estimates could not be  validated because credible supporting documentation was lacking.
The report does note that Air Force and Navy DTS program officials noted  that while they did not anticipate a reduction in the number of personnel,  there would be a shifting of personnel to other functions. The report  further points out that DOD officials responsible for reviewing economic  analyses stated that while shifting personnel to other functions is  considered a benefit, it should be considered an intangible benefit rather  than tangible dollar savings since the shifting of personnel does not result  in a reduction of DOD expenditures. Additionally, in its comments the  department provided no new data that was counter to our finding.
We are sending copies of this report to the Secretary of Defense; Under  Secretary of Defense (Comptroller); the Under Secretary of Defense for  Acquisition, Technology, and Logistics; the Under Secretary of Defense  (Personnel and Readiness); the Director, Business Transformation Agency;  and the Director, Office of Management and Budget. Copies of this report  will be made available to others upon request. In addition, the report is  available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions on matters discussed in this report,  please contact McCoy Williams at (202) 512-9095 or williamsm1@gao.gov  or Keith A. Rhodes at (202) 512-6412 or rhodesk@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be found   on the last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess the reasonableness of the key assumptions made by DOD to  arrive at the net annual estimated savings of over $56 million shown in the  September 2003 economic analysis addendum, we (1) ascertained if the  economic analysis was prepared in accordance with the prescribed  standards, (2) analyzed two key assumptions that represent the largest  dollar savings for the DTS program, and (3) analyzed the supporting  documentation related to these two assumptions to determine whether the  assumptions were valid. Furthermore, we met with the military services  and DFAS officials to ascertain their specific concerns with the estimated  savings. Further, we met with Program Analysis and Evaluation officials to  identify any issues they had with the DTS estimated savings. In performing  this body of work, we relied heavily upon the expertise of our Applied  Research and Method’s Center for Economics.
To determine the actions being taken to enhance the utilization of DTS, we  met with military services officials to obtain an understanding of the  specific actions that were being taken. In addition, we obtained and  reviewed various memorandums related to the utilization of DTS. We also  obtained an overview of the method and data used by the PMO-DTS to  report the rate of DTS utilization for the various DOD components. We  also met with the military services to ascertain how they use the PMO-DTS  data to monitor their respective utilization and whether they augment  these data with any other data and if so, the source of those data.
To ascertain whether DOD has reasonable assurance that the testing of  DTS was adequate, and thereby ensure accurate flight information was  displayed, we met with Northrop Grumman and the PMO-DTS officials to  obtain an explanation of the corrective actions that were to have been  implemented. To ascertain if the noted corrective actions have been  successfully implemented, we analyzed 246 GSA city pair flights to  determine if the information being displayed to the traveler was consistent  with DTS’s stated requirement.
We did not review the accuracy and reliability of the specific dollar  amounts shown in the September 2003 economic analysis. Given the  department’s previously reported problems related to financial  management, we have no assurance that the underlying data supporting  the economic analysis were complete. Furthermore, our emphasis was  directed more towards the validity of the assumptions that were used to  arrive at the net annual estimated savings of over $56 million. We  determined that the data were sufficiently reliable for the purpose of this  audit. We performed our audit work from October 2005 through July 2006  in accordance with U.S. generally accepted government auditing  standards.
We requested comments on a draft of this report from the Secretary of  Defense or his designee. We received written comments from the Under  Secretary of Defense (Personnel and Readiness), which are reprinted in  appendix II.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: In addition to the above contacts, the following individuals made key  contributions to this report: Darby Smith, Assistant Director;   J. Christopher Martin, Senior-Level Technologist; F. Abe Dymond,  Assistant General Counsel; Beatrice Alff; Harold Brumm, Jr.; Francine  DelVecchio; Jason Kelly; and Tarunkant Mithani.