Defense Transportation: Monitoring Costs and Benefits Needed While Implementing a New Program for Moving Household Goods
Published: 2003-04-18T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) spends more than $1.7 billion each year to move and store over 600,000 household goods shipments when relocating military personnel. It conducted and evaluated several pilot program studies aimed at fixing its problem-plagued program and, in 2002, issued a report to Congress with three recommendations. The 1997 Defense Appropriations Act Conference Report directed GAO to validate the results achieved by the pilot programs. In response, GAO examined the extent to which DOD's recommendations to Congress (1) offer solutions to long-standing problems in the current program and (2) are supported by the evaluation's findings and should be implemented. GAO also assessed the soundness of methodologies used by DOD to develop cost estimates to implement the recommendations.
section_title: What GAO Found
paragraphs: The recommendations in DOD's report to Congress have the potential to resolve several long-standing problems found in the current personal property program, which manages the transportation and storage of household goods. The recommendations, if implemented, would (1) reengineer the claims process to reduce the length of time it currently takes to resolve claims for lost, destroyed, or damaged household goods and increase the reimbursement rates that military personnel currently receive for their losses; (2) use performance-based service contracts to improve the generally low quality of service that DOD currently gets from the moving industry; and (3) put in place new information technology with interface capabilities to enable program managers and users to monitor in-transit shipments and track the number and cost of shipments processed each year. The recommendations in DOD's report to Congress are supported by the Transportation Command's evaluation of the pilot programs' findings and should be implemented within budget constraints. DOD's approach in conducting the evaluation was methodologically sound: It developed an evaluation plan to guide its work and adjusted the plan when necessary to address differences in the pilot programs' approaches. While the shipments included in the evaluation do not represent all shipment types managed annually by DOD, GAO believes that the evaluation results provide sufficient information to allow DOD to initiate actions to improve its current personal property program. GAO found that the soundness of methodologies used to develop DOD's cost estimates varied. Therefore, DOD's ability to implement changes to the existing program within the cost estimates DOD reported to Congress is uncertain. GAO found that the estimate to implement the information technology recommendation was $7 million rather than the $4 million to $6 million estimate DOD reported to Congress. In developing cost estimates for the remaining recommendations, DOD did not provide the same level of evidentiary support for one of the three adjustments it used to align the pilot programs' costs to current program costs. As a result, GAO questions the extent to which these recommendations can be implemented within DOD's estimated 13 percent increase over current program costs. While DOD believes it used a conservative approach in developing this 13 percent estimate, it has not quantified the risk associated with the projection, which could provide the military services and Congress information needed to develop and review future budget requests for this program. Without providing the range of possible cost increases and the risk regarding the likelihood of achieving this 13 percent projection within that range, DOD may find a repeat of what occurred during the pilots, where the military services terminated participation in one of the pilot programs due to costs exceeding projections. GAO also found that without carefully monitoring costs during the implementation phase and assessing costs and benefits from a period succeeding full implementation of the recommendations, DOD would not have the information needed to determine if anticipated improvements in the program are being achieved at a reasonable cost.

section_title: Letter
paragraphs: Military personnel and their families can expect to relocate many times  during a service member’s career. As the moving industry’s single largest  customer, the Department of Defense (DOD) spends more than $1.7 billion  annually for its personal property program, which provides household  goods transportation and storage services for military personnel and their  families when they relocate. The program manages more than 600,000  personal property shipments each year. DOD has experienced long- standing problems with its current personal property program, including  excessive loss or damage to property, high claims costs incurred by the  government, and poor quality of service from moving companies.  Moreover, the program’s data management system does not provide  reliable information on the status of individual shipments or on the types  of shipments and their costs.
In an effort to test alternative approaches and improve the quality of its  personal property program, DOD has carried out several initiatives over  the past 8 years. In 2000, the U.S. Transportation Command began to  collect data from one of the three pilot programs to evaluate alternative  approaches for improving the current program. The Transportation  Command compared the features of the current program with those of the  three pilot programs, and in June 2002 it submitted a report to the Deputy  Under Secretary of Defense (Logistics) outlining its evaluation strategy,  findings, and recommendations. In its November 12, 2002, report to  Congress, DOD included the three recommendations resulting from the  evaluation and the estimated additional costs required to implement  program improvements based on a subsequent assessment of pilot and  current programs’ operations.
Our involvement in this issue stems from the conference report on the  1997 Defense Appropriations Act, which directed us to review the data  collected from the pilot programs and validate the results before DOD  expands any of the pilot programs, and a subsequent request from the  Subcommittee on Readiness of the House Committee on Armed Services.  This report supplements our previous testimony on DOD’s efforts to  enhance its current program, as well as our status report and status  briefings to Congress on DOD’s efforts to implement a methodologically  sound evaluation of the pilot programs. As agreed with your offices, for  this report we assessed the extent to which the recommendations in  DOD’s November 2002 report (1) offer solutions to long-standing problems  in the personal property program and (2) are supported by the evaluation’s  findings and should be implemented. In addition, we assessed the  soundness of the methodology used by DOD to develop the cost estimates  for implementing the recommendations.
To conduct our assessment of DOD’s three recommendations to Congress,  we reviewed DOD and GAO prior reports on the personal property  program, interviewed DOD officials and private-sector contractors  involved in the current and pilot personal property programs and the  Transportation Command’s evaluation, examined the methodology and  findings of the Transportation Command’s evaluation of the three pilot  programs, reviewed the methodology and data generated by each of the  three pilot programs, and assessed the methodologies used to develop the  cost estimates for implementing DOD’s recommendations. We did not  make an assessment of whether the anticipated benefits to be derived  from implementing the three recommendations would warrant the  additional costs DOD projects will be required to fund these  improvements. The scope and methodology we used in our review are  described in further detail in appendix I.
subsections: section_title: Background
paragraphs: DOD’s personal property program is managed centrally by the Military  Traffic Management Command headquarters and administered locally by  about 200 military service and DOD transportation offices around the  world. The program relies on more than 1,200 domestic commercial  carriers and 150 freight forwarders for international shipments to provide  household goods transportation and storage services for military  personnel and their families when they relocate. The military services pay  shipment and storage-related costs from their military personnel accounts  and loss and damage claims and personal property shipment office  expenses through their operations and maintenance accounts. The  program has remained virtually unchanged for nearly 40 years. It involves  a complex process of qualifying carriers, soliciting rates, distributing  moves, evaluating transportation providers’ performance, paying invoices,  and settling claims. Among the program’s many challenges is ensuring that  the moving industry provides adequate year-round capacity, especially  during the summer peak-moving season when most service members, as  well as the general public, schedule their moves.
In prior reports, both DOD and GAO have identified problems related to  the loss and damage claims process and the low quality of service from  movers. In designing and implementing its evaluation plan, the  Transportation Command also noted that weaknesses in the current  program’s data management system precluded DOD from being able to  track shipments in transit and from being able to extract reliable data on  the number and types of shipments managed annually and their associated  costs.
In response to the long-standing problems, DOD has undertaken a number  of pilot program studies to find ways to improve the process of shipping  service members’ household goods. In August 1996, the Deputy Under  Secretary of Defense (Logistics) tasked the Transportation Command with  evaluating alternative approaches to the current program and  recommending changes in the program based on the results of its  evaluation. The Transportation Command identified three ongoing or  planned pilot programs to include in its evaluation and began to collect  data for its analysis from one of them in 2000. These three pilot programs  shared some common features, such as testing performance-based service  contracts and providing full replacement (rather than depreciated) value  for loss or damage. Each one also had some distinctive features, such as  allowing service members to participate in selecting their movers and  contracting out installation personal property shipment office functions to  private-sector move manager companies.
The three pilot programs are summarized below. Further information on  each program, as well as DOD’s current personal property program, is  provided in appendix II.
The Military Traffic Management Command’s Reengineered Personal  Property Program operated from the military services’ and the Coast  Guard’s installations located in North Carolina, South Carolina, and  Florida. It used military installation personal property shipment office  personnel, as the current program does, and developed a new data  management system that tracked both the movement of individual  shipments and information on the number and cost of shipments.
The Department of Defense’s Full Service Moving Project operated  from the military services’ and the Coast Guard’s installations located  in the National Capital Region (the Washington, D.C., metropolitan  area), Georgia, and North Dakota. It contracted the management of  shipments to private-sector companies and offered optional relocation  services, such as referrals for rental assistance and purchase and sale  of real estate services, to personnel participating in the pilot program.
The Navy’s Service Member Arranged Move Pilot Program operated  from Navy installations located in the states of California, Connecticut,  Virginia, and Washington. It designated current staff within the  installation personal property shipment offices as “move coordinators”  to provide assistance, allowed participants to pre-select transportation  providers, and paid for moves through government purchase cards.
In June 2002, the Transportation Command submitted a report containing  its evaluation results and proposed three recommendations to the Deputy  Under Secretary of Defense (Logistics). After reviewing the results and  receiving comments from the military services, DOD submitted its report,  dated November 12, 2002, to Congress. DOD’s report contained the same  three recommendations contained in the Transportation Command’s  report. DOD also provided cost estimates for implementing the  recommendations. The three recommendations were to  reengineer the liability/claims process by adopting commercial  practices of minimum valuation, simplifying the filing of claims, and  providing direct settlement with the carrier;    change the acquisition process to implement performance-based  implement information technology improvements, which could  interface functions across such areas as personnel, transportation,  financial, and claims.
In its report to Congress, DOD estimated that reengineering the  liability/claims process and changing the acquisition process to implement  performance-based service contracts would increase the current  program’s estimated $1.7 billion cost by 13 percent. Implementing the  information technology improvements to enhance its data management  capabilities and to provide training to users was estimated at an additional  $4 million to $6 million.
DOD also estimated that efforts to implement the changes to the current  program would be completed by the first quarter of fiscal year 2005. DOD  has developed a plan of action and milestones for designing the new  personal property program. This initial effort identifies several teams,  which are exploring the following issues: the acquisition/solicitation  process, quality assurance, the liability/claims process, information  systems technology, and electronic billing and payment. Four of these  issues address the recommendations included in DOD’s November 2002  report to Congress. The plan identifies a list of essential activities needed  to carry out the responsibilities required to build the future personal  property program. It also includes time lines and identifies a process to  monitor problems and delays. However, it does not include monitoring  costs and benefits during the implementation phase and the extent the  proposed changes are being achieved within an acceptable and a  predefined range. Further, it does not include evaluating the extent the  benefits from the pilot programs will be achieved after the new program is  implemented to determine whether the anticipated improvements were  achieved at a reasonable cost.
section_title: DOD’s Recommendations Offer Solutions to Long-Standing Problems
paragraphs: The three recommendations in DOD’s report to Congress offer solutions to  several of the current program’s long-standing problems, such as the  liability/claims process and the low quality of service. These problems  have been identified in DOD and GAO’s prior reports, as well as in surveys  conducted as part of the pilot program evaluation. The inability to monitor  shipments and shipping information has been long recognized and was  highlighted as an additional problem during DOD’s evaluation. If  implemented, the first recommendation (i.e., reengineering the  liability/claims process by adopting commercial practices of minimum  valuation, simplifying the filing of claims, and providing direct settlement  with the carrier) has the potential to help reduce the length of time it  currently takes to resolve claims for lost, destroyed, or damaged  household goods because the carrier recovery time would be eliminated  for most moves, increase the reimbursement rates military personnel  receive for their losses, and reduce DOD’s claims-related costs. The  second recommendation (i.e., changing the acquisition process to  implement performance-based service contracts) has the potential to help  improve the generally low quality of service that DOD currently receives  from the moving industry. The third recommendation (i.e., implementing  information technology improvements, which could interface functions  across areas such as personnel, transportation, financial, and claims) has  the potential to improve the program’s ability to reliably monitor and  collect data on the status and costs of shipments so that accurate  reporting can be provided to DOD and Congress.
subsections: section_title: Reengineered Liability/Claims Process Has the Potential to Shorten Time, Increase Reimbursement Rates, and Reduce Claims-Related Costs to DOD
paragraphs: As part of its evaluation, the Transportation Command cited that one of  the long-standing problems with military household goods shipments is  the liability/claims process, including the (1) length of time it takes to  resolve claims, (2) low reimbursement rates, and (3) high cost of claims  that DOD must pay.
subsections: section_title: Length of Time to Resolve Claims
paragraphs: In a study conducted in 1999, the Military Traffic Management Command  reported that 146 days are expended between the time a claim is filed by a  service member to recovery of costs from the carrier by the government.  During this period, military personnel file their claims for lost, destroyed,  or damaged household goods with their respective military service’s  claims offices and receive settlements (this occurs, on average, within          23 days), and then these offices file the claims against the carriers to  recover the costs (this step is completed within the 146 day period). In the  Transportation Command’s pre-evaluation survey completed in 2000,  responses from military personnel who had recently moved indicated that  one of the lowest performance ratings involved the time required to settle  a claim.
Based on the Transportation Command’s evaluation of the claims process  under the pilot programs, one of the results from implementing the pilot  programs was the 146 day average required under the current program to  settle claims and recover costs was reduced to an average of 30 days since  the service member filed directly with the carrier and the military services  did not have to recover costs. Under each pilot program, military  personnel settled claims directly with the carriers. Service members who  were not satisfied with offers made by the carriers could file their  unresolved claims directly with DOD. Military services worked these  claims with the carriers and if a claim was justified, the service member  received just settlement under the pilot programs (i.e., if the items were  lost or destroyed, the member received full replacement value, while  damaged items were repaired).
section_title: Low Reimbursement Rates for Loss and Damage
paragraphs: In its pre-evaluation survey, the Transportation Command found that  military personnel cited low reimbursement amounts that typically do not  cover the loss or damage of household goods as a major concern during  their moves. Under the current program, a carrier’s liability is limited to  $1.25 per pound multiplied by the shipment weight. Personnel receive only  the depreciated value of lost, destroyed, or damaged items, up to a  maximum of $40,000 per move. When arranging their moves under the  current program, military personnel can buy increased insurance coverage  from their carrier, up to a full replacement value limit of $3.50 per pound  times the shipment weight, at a cost of 85 cents per $100 of the stated  value of the shipment. However, only military personnel making moves  within the continental United States can buy this additional coverage; it is  not available to those moving to or from overseas posts. Another option  that military personnel have to increase their protection for loss and  damage is to buy additional coverage from private-sector sources.
Unlike the current program, the pilot programs provided full replacement  value for lost and destroyed goods, with maximum amounts ranging from  $63,000 to $75,000. Damaged items were repaired. Two of the three pilot  programs reported that their cost per pound times the shipment weight  rates were $3.50 for $63,000 maximum coverage and $6.00 for $75,000  maximum coverage, respectively. The remaining pilot program did not  give a cost per pound, stating only that its maximum coverage rate was  $72,000.
section_title: High Costs to Government for Claims
paragraphs: DOD has reported that, historically, approximately 35 percent of all moves  result in loss or damage claims. A 1997 Military Traffic Management  Command survey of 3,000 moves revealed that while 65 percent of  shipments had loss or damage, only 35 percent resulted in claims being  filed. DOD pays approximately $100 million a year in claims but recovers  only 60 to 65 percent of the amounts paid to military personnel from the  moving industry. These figures understate the actual loss and damage,  since all military personnel do not file claims, apparently because the  process takes a long time and reimbursement rates do not always cover  the losses. DOD incurs these losses due to the structure of its current  program. The military claims offices assist service members by arranging  to pay their claims and then submitting the claims to the respective  movers for reimbursement. As indicated above, DOD receives only partial  reimbursement from the moving industry.
If the recommendation is implemented, DOD expects to reduce a  substantial portion of the estimated $100 million it currently pays in claims  each year to service members and eliminate much of the 35 to 40 percent  in losses it incurs from settling claims with the moving industry because  service members will be settling claims directly with their carriers. DOD  also expects additional savings because fewer demands would be placed  on military claims officials to manage the claims process. DOD believes  that these savings will help offset the higher costs of providing full  replacement value to service members for any loss and damage incurred  during the shipment and storage of their personal property.
section_title: Change to Performance- Based Service Contracts Could Improve Quality of Moving Services
paragraphs: Our work has shown that another long-standing problem with the current  personal property program is the poor quality of moving services provided  to military personnel. The high number of loss and damage claims that  military personnel file underscores this problem. According to the two  pre-evaluation surveys cited in the Transportation Command’s evaluation,  around 55 to 65 percent of respondents reported suffering some loss or  damage of household goods during a recent move. Moreover, in the   pre-evaluation survey conducted by the Transportation Command, the top  four factors identified by service members as being of greatest importance  to them in the moving process were the quality of packing, the care in  handling personal property, the condition of their property upon receipt at  the end of the move, and the receipt of fair payment for any losses or  damages they suffered. In individual comments obtained during the   pre-evaluation survey, the Transportation Command reported that some  service members also cited the lack of professionalism and quality of  customer service on behalf of moving crews as a concern.
The problem stems primarily from the current program placing greater  emphasis on costs (i.e., the lowest bids) than on the quality of service that  carriers provide when moving shipments of military household goods.  While the current program established its Total Quality Assurance  Program to measure quality, data collected to develop scores for each  carrier includes three measures (timeliness of pickup, timeliness of  delivery, and reported loss and damage), which are not collected for all  household goods shipments. The best indication of quality, customer  satisfaction, is not measured in the current program. The problem of  quality is further exacerbated by the program’s use of a 20 year-old tariff  schedule that carriers use in developing their bids. This tariff contains  lower rates than the current commercial tariff used during the pilot  programs.
Unlike the current program, the pilot programs screened carriers that  wanted to participate in their programs by emphasizing the quality of  carriers’ prior performance rather than the amount of their bids. For  example, the Full Service Moving Project contracted a financial services  company to conduct a financial and performance assessment of potential  movers. The pilot program emphasized best value and placed more  emphasis on performance (70 percent) than cost (30 percent) in  determining which providers were awarded shipments. The pilot programs  showed that these types of contracts could allow the government to   pre-screen carriers for financial viability and, more importantly, to  institute and maintain a quality assurance process to reduce losses and  improve service.
In addition to prescreening carriers for quality control purposes, the pilot  programs also surveyed military personnel who participated in the  programs and used the results to distribute future shipments to carriers  that received the best performance scores. To address concerns about the  obsolete tariff schedule, the pilot programs adopted current commercial  tariffs for carriers to use in establishing their bids.
The pilot programs also showed that the solicitation process could be  streamlined by eliminating detailed statements of work and that the pilot  programs could place responsibility for successful performance on  carriers, allowing the government to focus on outcomes, rather than  processes. Finally, the pilot programs demonstrated that using these types  of performance-based service contracts did not have an adverse effect on  small business participation, a major concern of the moving industry. On  the basis of the total dollar value of shipments, each pilot program  exceeded the Small Business Administration’s goal of 23 percent  participation for the industry. Specifically, 48 percent of the Military  Traffic Management Command’s Reengineered Personal Property  Program’s revenues, 74 percent of the Full Service Moving Project’s  revenues, and 100 percent of the Navy’s Service Member Arranged Move’s  revenues went to small businesses.
section_title: Implementing Information Technology Improvements to Address Data Reliability Problems
paragraphs: Another ongoing problem with the current personal property program is  its inability to provide reliable data on the status of in-transit shipments or  on the number and associated costs of shipments managed by DOD each  year. Because of the lack of reliable data on shipments and costs, program  managers have no way of knowing the actual costs of moving military  personnel’s household goods. In addition, they have no access to real-time  tracking data that they could use to manage transportation and storage  costs and to help cut down on the need for temporary storage by reducing  the number of failed deliveries.
Two of the pilot programs included features to address the problems  associated with the current program’s stand-alone data management  system. The pilot programs each developed a Web-based data management  system to enhance the visibility of individual shipments and provide more  reliable data on shipments and costs. For example, the Military Traffic  Management Command’s Reengineered Personal Property Program’s data  management system provided in-transit visibility. This made it possible to  track the status of individual shipments and gave real-time access to those  sections of the shipment records that various parties involved in the  relocation process needed for data entry or status review. The pilot  program’s data management system provided a complete picture of the  service member’s move from start (the move application process) to finish  (the claims submission and resolution process). In addition, the data  management system demonstrated the potential to provide information to  personnel in various functional areas involved in the service members’  relocation process (such as personnel, transportation, financial, and  claims). Finally, the data management system demonstrated the potential  to provide data for planning and budgeting purposes on the types of  shipments made annually across DOD and their costs. The Full Service  Moving Project’s data management system was developed but not fully  implemented because the military services terminated their participation  in the pilot program due to its high costs. While the Navy’s pilot program  developed a database near the end of the Transportation Command’s  evaluation, the database was not fully implemented nor assessed as part of  the evaluation.
section_title: Transportation Command’s Evaluation of Pilot Programs Supports DOD’s Three Recommendations
paragraphs: Our analysis indicated that DOD’s three recommendations are supported  by the results of the Transportation Command’s evaluation of the three  pilot programs. The Transportation Command adopted a sound  methodology to conduct its evaluation, and it adjusted this methodology  when circumstances warranted. The results of the Transportation  Command’s evaluation are based on data collected from a limited number  of geographical areas. While the shipments included in the evaluation do  not represent all shipment types managed annually by DOD, we believe  that the evaluation results provide sufficient information to allow DOD to  initiate actions to improve its current personal property program within  budget constraints.
subsections: section_title: Transportation Command Implemented a Methodologically Sound Evaluation
paragraphs: We found that the Transportation Command used a methodologically  sound approach to evaluate the results of the three pilot programs and  make its recommendations. Before it started the evaluation process, the  Transportation Command considered some lessons learned that had  emerged from our review of the Hunter Pilot Program in 1999, and it  followed through with several of them. For example, it obtained assistance  from a contractor to design an evaluation plan that met professional  standards. The Transportation Command identified four aspects, or  factors, of the property program that served as the focus of its evaluation  (i.e., quality of life, total costs, small business participation, and process  improvements). The evaluation plan also prescribed that only one quality  of life survey be administered to each participating service member in  order to avoid survey “fatigue” that can result from subjecting a person to  multiple surveys, and thus avoid the resulting potential for questionable  results. In designing the evaluation plan, the Transportation Command  incorporated a number of important evaluation features. These features  included assessing the four factors consistently across all three pilot  programs, ensuring that the evaluation received data from the pilot  programs during the same time period to avoid the need to make  adjustments due to potential changes in carrier operations and costs,  conducting a survey of service members using the current program to  establish a baseline from which to measure the pilot program results, and  developing a method to provide estimates of what DOD would have paid  for comparable shipments under the current program for those shipments  completed under the pilot programs.
The Transportation Command made appropriate adjustments to the  evaluation plan when it learned that the three pilot programs would not be  underway at the same time and that they would not provide all of the  information originally outlined in the plan. For example, the Full Service  Moving Project began later and terminated earlier than was expected, the  Navy’s Service Member Arranged Move Pilot Program failed to conduct a  quality of life survey and collect cost data as outlined in the evaluation  plan, and none of the three pilot programs provided costs associated with  individual process improvements. The Transportation Command included  qualitative analytical techniques so that it could include as much  information on each pilot program as possible in its evaluation while also  dealing appropriately with data limitations. The Transportation Command  also shifted the evaluation focus from the individual pilot programs to  specific features from the three programs, such as full replacement value  for loss and damage and the screening process for carrier participation.
section_title: Survey and Analytical Data Support Command’s Recommendations
paragraphs: Our work indicated that the Transportation Command’s analysis of data  collected from the three pilot programs supports the three  recommendations that DOD included in its report to Congress. The  Transportation Command’s analysis of household goods shipment data  from the pilot programs showed that the average amount of time that  service members and DOD spend to settle claims and recover costs from  carriers fell dramatically in all three pilot programs. In comparison with  the current program’s 146-day average, it took only 30 days, on average, to  settle a claim under the Reengineered Personal Property Program and the  Full Service Moving Project and fewer than 14 days under the Navy’s  program. Survey results indicated that full replacement (rather than  depreciated) value, direct claims settlements, and anticipated  improvements in the claims process accounted for the highest increases in  satisfaction. Based on experiences during the pilot programs, DOD  believes that direct claims settlement between service members and  carriers should reduce claims costs DOD currently incurs. Under the  current program, DOD must collect from the carriers after it has paid the  service members’ claims. DOD expects that this step will be eliminated in  most instances because it is anticipated that service members will be  resolving most of their claims directly with their carriers.
The Transportation Command’s analysis of process improvement data,  interviews and observations during site visits, and survey results from the  pilot programs supported DOD’s recommendation to use performance- based service contracts to improve the quality of services that the moving  industry provides to the military. The process of prescreening carriers  desiring to participate in the pilot programs on the basis of their financial  viability and past performance helped to eliminate poor performers.  Furthermore, the pilot programs’ use of post-move surveys allowed them  to get immediate and continuous feedback on the carriers’ performance  and to use this information to distribute future work to those carriers with  the highest performance ratings and best value. In addition, two of the  pilot programs reduced the amount of paperwork associated with  soliciting proposals and approving carriers.
Finally, the Transportation Command’s review and observations of two of  the pilot programs’ Web-based data management systems supported  DOD’s recommendation to overhaul the current personal property  program’s computer system (the Transportation Operational Personal  Property Standard System). The Transportation Command found that the  Reengineered Personal Property Program’s data management system  significantly improved communications between the various DOD offices  and the moving industry. The system gave real-time access to shipment  records to DOD’s personal property shipment offices, certifying officers,  prepayment auditors, military service headquarters, and military service  claims offices and finance centers, as well as moving industry participants.  Similar results occurred with the Full Service Moving Project’s Best Value  Distribution Database system, but the military services terminated their  participation in this pilot program before the system’s full potential could  be demonstrated.
While the shipments included in the evaluation do not represent all the  shipment types managed annually by DOD, we believe that the evaluation  results provide sufficient information to allow DOD to initiate actions to  improve its current personal property program.
section_title: Ability to Implement New Program within Cost Estimates Reported to Congress Is Uncertain
paragraphs: Our review found that the estimates DOD reported to Congress might  understate the total initial cost for implementing the information  technology improvements recommendation and contain a questionable  adjustment for costs associated with the claims and contracting process  recommendations. Also, DOD did not quantify the risk associated with  implementing these latter recommendations within its projected                13 percent increase over the current program’s cost. Therefore, the ability  to implement changes to the existing program within the cost estimates  reported to Congress is uncertain.
Based on our discussions with Military Traffic Management Command  officials and review of available documents, we concluded that the total  initial cost to implement the information technology improvements  recommendation will more likely be $7 million rather than the $4 million  to $6 million estimate that DOD previously reported to Congress. In its  response to a draft of this report, DOD maintained that the costs to  implement a new Web-based data management system would fall within  its initial cost estimate of $4 million to $6 million. DOD’s projected cost  estimate includes $5 million for development and implementation of the  new system and $500,000 each for user training and system verification  and validation testing. At a minimum, based on these projected cost  estimates, the initial cost to implement the information technology  improvements recommendation would more likely be $6 million.
While we concur with the premise of two of the three adjustments used to  develop the 13 percent cost increase to implement the remaining  recommendations, we are less assured in the extent to which the projected  savings related to the third adjustment may occur. We found that the first  two adjustments were based on historical data. However, we question the  rationale DOD used to develop the third adjustment, as the savings  associated with this adjustment are based on assumed cost reductions  resulting from changes in program operations. Also, these reductions lack  the same quality of evidentiary support as DOD provided for the other two  adjustments.
DOD believes it took a conservative approach in developing the savings in  each of the three adjustments; therefore, it assumes that the proposed  changes to claims and the contracting process can be achieved within the  13 percent increase over the current program’s costs. Due to the long- standing problems with this program and the high pilot program costs that  contributed to the military services’ early termination of participating in  one of the pilot programs, we believe that by quantifying the risk  associated with this projection, DOD could provide the military services  and Congress information needed to develop and review future budget  requests for this program. Further, without carefully monitoring costs  during the implementation process and assessing costs and benefits from a  period succeeding full implementation of the recommendations, DOD will  not have the information needed to determine if anticipated improvements  in the program are being achieved at a reasonable cost. Currently, DOD is  beginning planning efforts to implement the recommendations. These  efforts do not include monitoring and evaluating costs and benefits during  the implementation phase and post implementation of the  recommendations in a new program.
subsections: section_title: Costs to Implement Information Technology Improvements Vary
paragraphs: The information DOD has provided on costs to implement the information  technology improvements recommendation varies. Information provided  during our review indicated that the total initial cost to improve the  current data management system would be higher than the $4 million to    $6 million DOD included in its report to Congress. DOD worked with the  contractor who developed the Reengineered Personal Property Program’s  Web-based data management system to develop an estimate of the cost to  expand the capabilities tested during the pilot program. Also included in  this estimate were funds to provide training for users of the new system.  Based on our discussions with officials from the Military Traffic  Management Command and our review of available documents, we  concluded that these costs would more likely be $6 million, as the data  management system development cost was projected to be $5 million with  an additional $1 million for user training. The need for this training as part  of a new personal property program was identified during DOD’s  evaluation of the pilot programs. We increased our overall projections for  the cost of the new system to $7 million when we learned that DOD  planned to continue spending at least another $1 million annually for  independent verification and validation testing and contractor support.  This latter expense was identified to us during discussions following  DOD’s submission of its report to Congress.
In its response to a draft of this report, DOD maintained that the costs to  implement a new Web-based data management system would fall within  its initial cost estimate of $4 million to $6 million.  It projected a cost of    $5 million for system development and implementation and $500,000 each  for user training and initial system validation. At a minimum, based on  these projected cost estimates, the initial cost to implement the  information technology improvements recommendation would more likely  be $6 million. Because we did not assess the sufficiency of DOD’s original  estimates of $1 million each for training and validation testing, we are  unable to assess the impact of the reduction on the improvements in  information technology across DOD.
Based on our discussion with DOD officials, we learned that the plan is to  implement this recommendation regardless of the status of the other two  recommendations because managers and users of the program need more  reliable information to manage the program’s shipments and their costs.  Funds to implement this recommendation would come from the military  services’ operations and maintenance accounts.
section_title: Soundness of Estimates to Develop Cost for Changing Claims and Contracting Processes Varies
paragraphs: The soundness of the three adjustments the Military Traffic Management  Command used to develop its estimated 13 percent increase over the  current program costs to implement the remaining recommendations—the  claims process and performance-based service contracts—varies. We  found that two of these adjustments are based on reasonable assumptions  and are supported by historical experience and by data. The savings  associated with the third adjustment are based on assumed cost  reductions resulting from changes in program operations and lack the  same quality of evidentiary support as DOD provided for the other two  adjustments. Therefore, we are less assured in the extent to which the  savings associated with this adjustment may occur. Finally, we found that  in its report to Congress, DOD did not quantify the risk of achieving these  recommendations within the projected 13 percent increase. This  information is important to the military services as they develop their  military personnel and operations and maintenance budget requests and to  Congress as it assesses the reasonableness of these requests.
subsections: section_title: Estimate Includes a Questionable Cost Adjustment
paragraphs: In developing the 13 percent estimate, the Military Traffic Management  Command determined that three adjustments to the average costs for the  pilot programs were required to develop the cost for the full rollout of a  new personal property program. The first two adjustments (i.e., reducing  the average weight of shipments and reducing costs to adjust for a mix of  small and large businesses) were made to offset differences between the  pilot programs’ shipments and those more typically managed across DOD.  The third adjustment was made to reduce the pilot programs’ costs to  reflect anticipated savings based on economies of scale. In developing  these adjustments, the Military Traffic Management Command worked  with a contractor and consulted with officials from the military services  and moving industry associations.
While we believe that the shipment weight and small business mix  adjustments are reasonable, we question the extent to which the  economies of scale or program efficiencies adjustment may be achieved.  For the weight adjustment, the Military Traffic Management Command  determined that the average weights of moves in the two pilot program  areas were higher than those experienced in typical departmentwide  moves. As a result, the Military Traffic Management Command reduced the  pilot programs’ average weights to reflect the lower, more typical weights  to be used in calculating a total cost for a departmentwide program. This  adjustment resulted in a 12 percent drop in average costs. We found the  approach of using historical data to more accurately reflect the typical  shipment weights to be reasonable.
Next, the Military Traffic Management Command further lowered the pilot  programs’ average costs because the pilot programs had higher small  business participation rates than the departmentwide average, and small  businesses are typically more expensive than large businesses. Small  businesses accounted for 48 percent of the cost of all moves under the  Reengineered Personal Property Program and 73 percent under the Full  Service Moving Project. In addition, small businesses were 14 percent  more expensive per shipment in the Reengineered Personal Property  Program and 74 percent more expensive in the Full Service Moving Project  than what each pilot program paid to large businesses. In developing its  departmentwide estimate, the Military Traffic Management Command  used a small business participation target rate of 30 percent. This               30 percent target rate is higher than the Small Business Administration’s  23 percent goal for government agencies conducting business with this  industry. On the basis of this lower participation rate, the Military Traffic  Management Command reduced the pilot programs’ average costs further  by 8 percent. We agree that this adjustment in costs based on differences  in the pilot programs’ small business participation rate and the new           30 percent goal is a reasonable way to reflect the differences between the  pilot programs’ costs and the departmentwide-projected costs.
We found that the third adjustment that the Military Traffic Management  Command made—to reduce the cost of departmentwide shipments  because of economies of scale or program efficiencies—was not  adequately supported based on either historical experience or data that  DOD later provided. The Military Traffic Management Command reduced  the pilot programs’ average costs by 5 percent on the assumption that  the pilot programs’ shipments involved only a limited number of  providers;  the pilot programs only included a limited number of shipments while  the current program manages over 600,000 shipments annually;   more accurate and timely management data that includes service   member counseling, reduced losses, and storage and indirect costs will  result in a more efficient program; and    overhead and operating costs will be spread due to a larger volume of  shipments.
While recognizing that some changes may result from these anticipated  program efficiencies, the effect of these changes on potential cost savings  is uncertain at this time. The Military Traffic Management Command did  not provide the same level of evidentiary support that it provided on the  other two adjustments. Further, we believe that only time will determine if  DOD’s assumption for this adjustment, in particular, proves to be correct.
section_title: DOD Did Not Quantify the Risk for Its Cost Estimate
paragraphs: We found that DOD has not provided a level of assurance to the military  services and Congress that its projected 13 percent increase over the  current program’s cost can be achieved. Quantifying the risk associated  with this projection could provide the military services assurance of the  viability of the projected 13 percent increase as they prepare budgets to  support the increased cost for this program. Congress could also use this  information as it reviews DOD’s requests for additional funds to  implement changes in this program. The need for this type of information  is further supported based on the long-standing problems associated with  the current program and the fact that shipment and storage costs under  the pilot programs were significantly higher than those that DOD  estimated it would have paid under its current program in the same  geographical areas. These costs ranged from 31 to 32 percent higher under  the Reengineered Personal Property Program and from 51 to 54 percent  higher under the Full Service Moving Project. These higher-than- anticipated costs contributed to the military services’ decision to  terminate their participation in the Full Service Moving Project before its  test period ended.
While DOD did not quantify the risk, per se, it believes a conservative  approach was taken in developing the savings in each of the three  adjustments. As a result, DOD assumes that the proposed changes to the  claims and contracting processes can be achieved with its projected  increase of 13 percent over the current program’s budget. We still believe  that the Military Traffic Management Command could have quantified the  risk and provided this additional information to the military services and  Congress as additional assurance of the likelihood of implementing the  two recommendations within its projected 13 percent increase. The need  for this information is further supported based on the long-standing  problems DOD has experienced in this program, the fact that the military  services terminated participation in one of the pilot programs due to the  high cost increases, and the need to determine whether the proposed  additional funds from military personnel and operations and maintenance  accounts will be sufficient to implement the recommendations.
In addition to the information that could be gained from quantifying the  risk of its cost projection, we believe that only by careful monitoring  during the implementation phase will DOD be able to ensure that the  proposed changes are being achieved within an acceptable and a  predefined range. Further, while we believe that the evaluation results  support implementing plans to enhance the current program, it should be  noted that the pilot programs’ shipments included in the evaluation were  not typical of all types of shipments managed annually. Therefore, DOD  was precluded from projecting the extent to which the recommended  improvements can be achieved DOD-wide. Unless a subsequent evaluation  is undertaken after the recommendations have been implemented, DOD  will not be able to assess the extent to which the projected benefits are  being achieved for military personnel, their families, and DOD, and  whether the benefits are being achieved at a reasonable cost. Selecting an  evaluation period to include the peak-moving season would also provide  DOD with the information its needs to determine if the proposed changes  can be achieved during the summer, when the demand for moving services  by DOD and the private sector is at its highest.
section_title: Conclusions
paragraphs: The three recommendations DOD developed from its evaluation of the  current and pilot programs, if implemented successfully, could enhance  the quality of life for relocating service members and their families; reduce  claims-related costs to DOD; and resolve problems related to the reliability  of management information on the status of shipments and on the  quantity, types, and costs of shipments that DOD and the military services  manage annually. Delaying implementation of the recommendations only  prolongs problems military personnel, their families, and DOD experience  under the current program.
DOD has not quantified the risk associated with achieving its projected    13 percent increase over the current program’s costs to implement the  claims process and performance-based service contract recommendations.  Without quantifying the risk, the military services and Congress cannot be  assured that these recommendations can be achieved within this estimate  or whether additional funding or trade-offs may be needed. Further,  without careful monitoring during the implementation phase, DOD will not  be able to ensure that the proposed changes are being achieved within an  acceptable and a predefined range.
Because the pilot programs’ shipments included in the evaluation were not  typical of all types of shipments managed annually, it was not possible for  DOD to project the extent to which the recommended improvements can  be achieved departmentwide. Without evaluating the program following  implementation of the recommendations, DOD will be unable to assess the  extent to which the projected benefits for military personnel, their  families, and DOD are being achieved and, if so, whether they are being  achieved within a reasonable cost. Also, if DOD does not select an  evaluation period that includes the peak-moving season, it will not have  the information needed to determine if the proposed changes can be  achieved in the summer, when the demand for moving services is at its  highest.
section_title: Recommendations for Executive Action
paragraphs: To improve the personal property program for military personnel, their  families, and program administrators, we recommend that the Secretary of  Defense direct the Commander, U.S. Transportation Command, to  initiate actions to implement the three recommendations contained in  DOD’s report to Congress within budget constraints,    provide the military services and Congress additional information to  quantify the risk associated with achieving the projected 13 percent   cost estimate before the claims process and performance-based service  contracts recommendations are implemented to provide the military  services with information needed for budgeting purposes,   monitor costs for all recommendations during the implementation  phase to ensure that the proposed changes are being achieved within  an acceptable and a predefined range, and   assess the effects of the three recommendations on the personal  property program (to be carried out after the summertime peak-moving  season once the recommendations have been implemented) to  determine whether the anticipated improvements in the program are  being achieved at a reasonable cost.
section_title: Agency Comments and Our Evaluation
paragraphs: In commenting on a draft of this report, DOD concurred with three of our  four recommendations. For the first of these recommendations, DOD  stated that it is developing a plan to implement those recommendations it  reported to Congress and anticipates its recommendations will be  implemented by the end of the first quarter of fiscal year 2006, assuming  the military services receive the additional funds needed to fund program  enhancements. In response to our recommendation to monitor costs  during the implementation phase, DOD stated that rolling out the new  program will require monitoring of costs to determine if the moving  industry partners are submitting bids that will allow DOD to enhance this  program within the projected 13 percent cost increase. Further, DOD plans  to include a process to conduct a rate reasonableness analysis upon  receipt of the rates. For rates found to be outside the range of  reasonableness, carriers will be given one opportunity to resubmit their  rates. DOD plans to only use those rates determined to be reasonable in  the new program. DOD also plans to include metrics, target/benchmark  performance indicators, and a methodology for data collection in an  updated program of action and milestone plan. For our recommendation,  i.e., assess the effects of the three DOD recommendations on the personal  property program to determine whether the anticipated improvements in  the program are being achieved at a reasonable cost, DOD plans to collect  data needed to determine if anticipated improvements have been achieved  on a continuing basis. DOD plans to use customer satisfaction surveys in  developing carrier performance ratings, which will be established  quarterly, with the exception of the peak season, when performance  ratings will be established monthly. If properly implemented, we believe  the proposed DOD actions will sufficiently address these  recommendations.
DOD partially concurred with the remaining recommendation, i.e., provide  the military services and Congress with additional information to quantify  the risk associated with achieving the projected 13 percent cost estimate  to provide the military services with information needed for budgeting  purposes. DOD continues to believe that the 5 percent reduction it made  to pilot programs’ average costs to adjust for economies of scale/program  efficiencies was reasonable and very conservative and that the program  can be implemented within the projected 13 percent increase over current  program costs. DOD also reported that one of the military services  validated the 13 percent cost increase following our audit. Further, DOD  stated that it did not see value added in providing the military services or  Congress a formal risk assessment but will continue to work with the  military services as execution progresses to make sure they have all  information required for budget purposes. Additionally, while not part of  this recommendation, DOD also said it did not concur with our finding  that the cost estimate for implementing its information technology  improvements recommendation would be $7 million.
In reviewing the response, we found that DOD still did not provide any  data to support its assumption of a 5 percent cost savings from economies  of scale/program efficiencies. DOD stated that the new program will be  about 200 times larger than the pilot programs and that the resulting  increase in volume will lower the cost per unit, a standard and accepted  law of economics. While we agree that the cost may decrease, it may also  increase or remain unchanged. Moreover, the cost may decrease by less  than 5 percent. Without specific data showing the per move costs will  decrease as the scale of operations increase, we continue to question the  basis for DOD’s assumption of a 5 percent reduction.
We believe that the validation effort completed by one of the military  services, along with the calculations and assumptions DOD used in  developing the 13 percent cost estimate, does not provide the military  services and Congress with information needed to reliably develop and  review budget requests to fund enhancements to the current program. We  continue to believe that DOD needs to qualify this estimate with a measure  of the risk associated with implementing its recommendations. Without  providing the range of possible cost increases and the risk regarding the  likelihood of achieving this 13 percent projection within that range, DOD  may encounter a repetition of its experience with one of the pilot  programs, which had to be terminated because actual costs exceeded  projected costs. Absent this risk information, the military services will  have to wait until after the transportation providers submit their bids in  order to learn whether the recommendations can be implemented within  the 13 percent projection. Should the bids result in costs that exceed this  estimate, DOD and the military services will need to make adjustments to  ensure that the recommendations are implemented within funding limits.  Therefore, we continue to believe that our recommendation has merit.
Our finding that the implementation of the information technology  improvements recommendation would likely cost $7 million rather than  the $4 million to $6 million that DOD projected was based on information  we received from DOD during the audit. Specifically, we calculated that  the costs to develop and implement the new system would be about          $5 million and that training for users of the enhanced system would cost  an additional $1 million. DOD had identified the need for this training  during its evaluation of the pilot programs. After DOD submitted its report  to Congress, it identified another potential cost—an additional $1 million  for independent verification and validation testing of the system. Our        $7 million estimated included all three of these cost elements. In its  response to a draft of this report, DOD maintained that its costs estimate  would fall within its initial cost estimate of $4 million to $6 million,  including $5 million for system development and implementation and an  additional $500,000 each for user training and system verification and  validation testing. At a minimum, based on these projected cost estimates,  the initial cost to implement the information technology improvements  recommendation would more likely be $6 million.  However, since we did  not originally assess the sufficiency of the $1 million estimates for training  and testing, we are unable to assess what impact DOD’s reduction for  these costs to $500,000 would have on the implementation of the system  across DOD. We have reflected DOD’s changes in the body of our report.
DOD’s comments are reprinted in appendix III. DOD also provided  technical comments, and we revised our report to reflect them where  appropriate.
We performed our review from April 2002 through February 2003 in  accordance with generally accepted government auditing standards.  Appendix I contains the scope and methodology for this report. DOD’s  comments are reprinted in their entirety in appendix III.
We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Commander, U.S.  Transportation Command; and the Director, Office of Management and  Budget. We will also make copies available to others upon request.  In  addition, the report will be made available at no charge on the GAO Web  site at http://www.gao.gov.
Please contact me at (202) 512-8365 or Lawson Gist, Jr., at (202) 512-4478  if you or your staff have any questions concerning this report. Key  contributors to this assignment were Robert L. Self, Jacqueline S. McColl,  Arthur L. James, Jr., Charles W. Perdue, and Nancy L. Benco.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess the extent to which the recommendations in the Department of  Defense’s (DOD) November 2002 report to Congress addressed major  problems in the personal property program, we took the following steps:    To identify the major problems facing the current personal property  program, we reviewed DOD and GAO reports addressing this program.  These reports identified problems associated with quality of service  and claims. We also conducted interviews with personal property  program officials and their contractors to gain an understanding of the  current data management system’s limitations and the long-standing  problems involving the lack of reliable information on shipments and  their costs.
To determine whether the proposed recommendations in DOD’s report  to Congress addressed the major problems of the current program, we  tracked the recommendations back to the U.S. Transportation  Command’s report on its evaluation results and assessed the extent to  which the recommendations are linked to and have the potential to  address problems.
To assess whether the recommendations in DOD’s report to Congress  were supported by DOD’s evaluation findings and should be implemented,  we took the following steps:    To determine if the Transportation Command developed a  methodologically sound evaluation plan, we assessed the command’s  efforts against the findings and recommendations contained in our  report on the Army’s Hunter Pilot Program results and against  professional standards we would use if conducting a comparable  evaluation. These sources addressed issues such as (1) seeking advice  in designing a methodologically sound evaluation plan, (2) developing  the evaluation plan prior to testing, (3) identifying factors to be  assessed and the data required for analyses to develop findings and  recommendations, (4) limiting quality of life surveys to only one for  each participant to preclude survey “fatigue,” and (5) conducting  simultaneous testing of the pilot and current programs.
To determine if the Transportation Command implemented an effective  evaluation strategy during the data collection phase of its evaluation,  we reviewed the pilot programs’ efforts to collect data for the four  factors as prescribed in the Transportation Command’s evaluation  plan. We also assessed the adjustments the Transportation Command  made in its evaluation strategy to address issues that could affect the  soundness of the results. An example of the issues addressed included  developing a constructed cost methodology to provide better estimates  of what DOD would have paid under the current program for  shipments made by the pilot programs.
To assess the Transportation Command’s development of findings and  recommendations to improve the current personal property program,  we reviewed the evaluation techniques (quantitative and qualitative  analyses) used to analyze data collected for the four factors. Further,  we assessed the extent to which the Transportation Command adjusted  the evaluation techniques to make up for differences in the way that  the pilot programs provided data for the evaluation.
To assess the methodology that DOD used to develop cost estimates for  implementing the recommendations, we took the following steps:    To determine the reliability of the cost estimates for the pilot programs  and for the proposed recommendations, we reviewed the cost  projection methodologies used by the Transportation Command and by  the Military Traffic Management Command.
To determine the reliability of pilot program shipment-related costs  used in the report, we reviewed the data collection efforts used by each  pilot program for the transportation and storage of household goods  included in the Transportation Command’s evaluation. Further, we  reviewed the constructed cost methodology used to develop the  estimates of what DOD would have paid to make comparable  shipments under the current program in the pilot programs’ test areas.
To determine the reasonableness of the assumptions and sources of  data used to develop cost estimates for implementing  recommendations for the personal property program, we met with  officials from the Military Traffic Management Command and their  contractor to discuss the methodology. We also reviewed the contents  of their briefing on the cost estimate work for implementing changes to  the claims process and performance-based service contracts and   additional information the Military Traffic Management Command  provided on the costs to implement information technology  improvements.
We did not make an assessment of whether the anticipated benefits to be  derived from implementing the three recommendations would warrant the  additional costs DOD projects will be required to fund these  improvements. Furthermore, we did not independently test the reliability  of data DOD extracted from its data system to develop costs. We found  that the department placed proper caveats on their use of such data, and  in the case of comparing pilot programs’ shipment costs to current  program costs, developed a constructed cost methodology to address  current program data management system weaknesses.
During this and prior reviews of DOD’s evaluation efforts, we met with  officials and obtained documents from the Office of the Assistant Deputy  Under Secretary of Defense (Transportation Policy), Washington, D.C.; the  U.S. Transportation Command, Scott Air Force Base, Illinois; the Military  Traffic Management Command, Alexandria, Virginia; the Department of  Defense Inspector General, Full Service Moving Project, and Hay Group  (Transportation Command Contractor), Arlington, Virginia; American  Management Systems (Transportation Command contractor),  PricewaterhouseCoopers (Military Traffic Management Command  contractor), and Systems Research and Applications (Military Traffic  Management Command contractor), Fairfax, Virginia; Logistics  Management Institute (Military Traffic Management Command  contractor), McLean, Virginia; the Navy’s Service Member Arranged Move  Pilot Program, Mechanicsburg, Pennsylvania; The Gallup Organization  (Full Service Moving Project contractor), Omaha and Lincoln, Nebraska;  and Parsifal Corporation (Military Management Traffic Command  contractor), Palm Bay, Florida. In addition to these agency meetings and  documents, we drew upon information contained in a testimony  statement, in reports, and in status briefings resulting from our prior  reviews of this program.
Our work for this review was performed from April 2002 through February  2003 in accordance with generally accepted government auditing  standards.
section_title: Appendix II: Overview of Current Personal Property Program and Pilot Programs
paragraphs: The Transportation Command evaluated three pilot programs to assess  alternative approaches that might address long-standing problems with its  current personal property program. The following tables provide features  of the current program and the three pilot programs. As the tables show,  the pilot programs had several features that provided enhancements to  military personnel and their families and to DOD that are not offered by  the current program.
Table 1 compares claims-related features. Specifically, the pilot programs  provided full replacement value rather than depreciated value for loss and  damage and guaranteed claims settlement with 45 to 60 days of filing the  claims.
Table 2 compares the quality of service-related features. Some of the  comparable features included emphasizing performance over cost in  selecting transportation providers and prescreening of transportation  providers.
Table 3 compares data reliability-related features. As noted, only one of  the pilot programs had a data management system that provided reliable  information to track individual shipments in transit and provide overall  data on shipments and their associated costs.
Additional information on the current program and on each pilot program  and its unique features follows.
subsections: section_title: Current Program
paragraphs: The current DOD personal property program, valued at over $1.7 billion  annually, moves more than 600,000 shipments each year for military  personnel and their families from the military services, Defense agencies,  and the Coast Guard. DOD is the moving industry’s single largest  customer. Managed centrally by the headquarters office of the Military  Traffic Management Command and administered locally by about 200  military and DOD transportation offices around the world, this program  relies on over 1,200 domestic commercial carriers and more than 150  forwarders for international traffic to provide moving and storage services.
subsections: section_title: Loss and Damage Claims
paragraphs: When loss and damage occur, military personnel can submit claims to  their respective military service claims office. Based on depreciated  values, the reimbursement rate is $1.25 per pound multiplied by the  shipment weight, with a maximum amount of $40,000 per move. Military  personnel have up to 2 years after receiving their shipments to file claims  but must submit notice of loss and damage within 70 days of delivery. The  current program does not have a specified time period in which the claims  are to be settled.
section_title: Counseling and Arranging Shipment Services
paragraphs: The current program provides counseling services and arranges the  shipment and storage of household goods and unaccompanied baggage  through government representatives, who are available to assist military  personnel and their families at the origin and destination points of their  moves. The current program does not have a real-time tracking system for  shipments nor does it provide a single point of contact to manage the  entire moving process; therefore, military personnel may interact with  several people at the origin and destination offices during their relocation.
section_title: Screening and Shipment Distribution Process for Transportation Providers
paragraphs: The current system is not designed to select transportation providers on  the basis of quality service; rather, transportation providers offering a  minimally acceptable level of quality are generally selected based on the  lowest rates. The program uses the Total Quality Assurance Program to  develop quality scores for each transportation provider. Each local  military installation distributes its traffic using a traffic distribution roster.  Transportation providers are placed on the rosters for each channel  (origin and destination areas) by order of rate level and quality score.  Transportation providers who participate in the domestic part of the  current program submit their rates as a percentage of the government  tariff, which is nearly 20 years old. The providers who participate in the  international part of the current program submit single factor or fixed  rates per hundredweight of the shipments.
The current program does not use customer satisfaction surveys as a  means to evaluate transportation provider performance. To remain in the  program, a provider must maintain a minimally acceptable level of quality– a 90 percent score. Three factors are measured: on-time pickup, on-time  delivery, and reported loss and damage to determine if points should be  deducted from transportation providers and allocation of shipments  should be reduced or terminated.
section_title: Visibility of Shipments during the Relocation Process
paragraphs: The current program does not provide service members with real-time  visibility of shipments during the relocation process.
section_title: Availability and Reliability of Data on Household Goods Shipments
paragraphs: Personnel at origin and destination personal property shipping offices  enter information on shipments to their respective Transportation  Operational Personal Property Standard Systems. However, data in these  individual systems does not include all shipments that occur during the  year, and the systems are not accessible to all parties involved in the  relocation process. Destination personal property shipment offices are  forwarded information on shipments via the current system; however,  payment data on these shipments is maintained in a separate system.
In addition to not providing information on all aspects of individual  shipments, the current program’s data management system does not  provide DOD and the military services information about the types of  shipments and related costs for planning and budgeting purposes. The  following are examples of the current system’s limitations:  the format of the system makes compiling data from multiple sites  difficult;    not all data is captured promptly; and    not all data and costs are captured/updated in the system.
section_title: Funding Sources
paragraphs: Under the current program, the military services reimburse carriers and  forwarders for shipment-related costs from military personnel accounts.  Personal property shipment office expenses and claims filed with the  government are funded from the military services’ operations and  maintenance accounts.
section_title: Military Traffic Management Command’s Reengineered Personal Property Program
paragraphs: Sponsored by the Military Traffic Management Command, the  Reengineered Personal Property Program included outbound shipments  for military and Coast Guard personnel departing from installations  located in North Carolina, South Carolina, and Florida (excluding Tyndall  Air Force Base). The pilot program ran concurrently with the existing  program at these installations. The pilot program’s goal was to include      50 percent of eligible moves from the above installations to continental  United States and European locations. The remaining shipments were to  be moved under the existing program. The Reengineered Personal  Property Program was initiated in January 1999 and operated for 12  months before data was submitted to the Transportation Command for  evaluation.
subsections: section_title: Loss and Damage Claims
paragraphs: Reimbursement for loss and damage claims was increased from  depreciated value to full replacement value, and the dollar amounts per  move increased from $40,000 under the current program to $63,000 under  the Reengineered Personal Property Program. Additionally, the pilot  program provided direct claims settlement between military personnel and  their transportation providers and a requirement that transportation  providers settle claims within 60 days of receiving claims forms from  military personnel.
section_title: Counseling and Arranging Shipment Services
paragraphs: Like the current program, the Reengineered Personal Property Program  relied on personnel in the personal property shipping offices to provide  counseling services and arrange for shipment and storage of household  goods and unaccompanied baggage. A central contact point in these  offices was not designated to manage the entire moving process; therefore,  military personnel may have interacted with several people at the origin  and destination offices during the relocation process. However, to improve  customer service, the program’s Pilot Transportation Operational Personal  Property Standard System provided real-time worldwide tracing  capability.
section_title: Screening and Shipment Distribution Process for Transportation Providers
paragraphs: Greater emphasis was placed on performance in awarding shipments to  transportation providers. Evaluation of financial status, elimination of  high-risk companies, and consideration of providers’ past performance,  rather than lowest bid, played the dominant role in selecting initial  transportation providers to participate in this pilot program.  Transportation providers who participated in the pilot program submitted  their bids for various origin and destination routes as a discount from the  commercial tariff. Prices were fixed for a year, with no provision for rate  increases during the contract period. Awards were made only to  transportation providers whose offers conformed to the solicitation and  represented the best overall value to the government.
The Military Traffic Management Command evaluated company  performance quarterly and compliance with terms and conditions of the  contracts annually. Subsequent performance reviews were conducted  based on customer satisfaction surveys and claims data. After  transportation providers received their minimum guarantee of business for  the year ($25,000), future awards were offered to the best performers.  Feedback was provided monthly to transportation providers, and those  that became poor performers were no longer offered household goods and  unaccompanied baggage shipments.
section_title: Visibility of Shipments during the Relocation Process
paragraphs: The Reengineered Personal Property Program provided the transportation  provider’s toll-free number to military personnel to enhance visibility over  their shipments throughout the relocation process.
section_title: Availability and Reliability of Data on Household Goods Shipments
paragraphs: The Reengineered Personal Property Program implemented its central,  Web-based Pilot Transportation Operational Personal Property Standard  System in part to address problems associated with visibility of and  availability of information on shipments during the relocation process. The  pilot program’s data management system provided real-time access to  both shipment and payment records. Access to the various modules of the  system was granted to personal property shipment office personal at  origin and destination locations, transportation providers, invoice  certifying officers, prepayment auditors, military service headquarters, and  military service claims offices and finance centers, based on each party’s  need for the information.
The system’s design allowed for entry of current address and telephone  numbers of military personnel to improve the process of delivering  household goods to a new residence. Data reliability was enhanced under  the Reengineered Personal Property Program, but one problem noted  during the evaluation was the need for military personnel to ensure that  their contact information (phone number and address) was current during  the relocation process. This had an effect on deliveries of household goods  and the quality of life survey contractor’s ability to reach military  personnel to ascertain their opinions about their relocation experience.
In addition to providing information on all aspects of individual shipments,  the Reengineered Personal Property Program’s data management system  demonstrated the potential to provide DOD and the military services with  information about the types of shipments and related costs managed  annually for planning and budgeting purposes.
section_title: Other Reengineered Personal Property Program Features
paragraphs: The Reengineered Personal Property Program achieved stronger  transportation provider commitment with long-term contracts, and it used  contractor support to conduct quality of life surveys with military  personnel moving under the pilot program and to perform audits of each  invoice submitted by transportation providers.
section_title: Funding Sources
paragraphs: Like the current program, the military services reimbursed carriers and  forwarders for shipment-related costs from their military personnel  accounts. Transportation office expenses and any claims filed with the  government were funded from the services’ operations and maintenance  accounts.
section_title: The Department of Defense’s Full Service Moving Project
paragraphs: Sponsored by the Office of the Assistant Deputy Under Secretary of  Defense (Transportation Policy), the Full Service Moving Project included  outbound shipments for military and Coast Guard personnel and DOD  civilian departing from locations in the National Capital Region, Georgia  (excluding Robins Air Force Base), and Minot Air Force Base, North  Dakota. The pilot program’s goal was to include 90 percent of the moves  from these locations to continental United States and to European and  Asian-Pacific locations. The remaining shipments were to be moved under  the current program. The Full Service Moving Project began in January  2001 and continued until its early termination in September 2001. Due to  continuing delays in implementing this pilot program and DOD’s decision  to terminate the pilot program in September 2001, the Full Service Moving  Project had limited operational experience before submitting data to the  Transportation Command.
subsections: section_title: Loss and Damage Claims
paragraphs: Reimbursement for loss and damage claims was increased from  depreciated value to full replacement value, with the dollar amounts  increasing from $40,000 per move under the current program to $75,000  per move under the Full Service Moving Project. Additionally, the pilot  program provided for direct claims settlement between military personnel  and their transportation providers and a requirement that the responsible  party (transportation providers or move managers) settle claims within 45  days of receiving claim forms from military personnel.
section_title: Counseling and Arranging Shipment Services
paragraphs: Unlike the current program and other pilot programs, the Full Service  Moving Project tested the use of private-sector relocation companies  (move managers) for outsourcing traditional transportation services  (counseling and arranging for the shipment and storage of household  goods and unaccompanied baggage) performed by origin and destination  personal property shipping offices. The pilot program’s goal was to  provide a single point of contact (move manager) for military personnel  and transportation providers to contact throughout the relocation process.
section_title: Screening and Shipment Distribution Process for Transportation Providers
paragraphs: The Full Service Moving Project made major changes to the existing  transportation provider approval, rate solicitation, and traffic distribution  processes. The pilot program emphasized best value and placed more  weight on performance (70 percent) than cost (30 percent) in determining  which transportation providers would be awarded shipments. The pilot  program contracted with a financial services company to conduct financial  and performance assessments of transportation providers and move  manager companies that wanted to participate in the pilot program. For  approved transportation providers, rates were established for a 1-year  cycle. The providers submitted their rates as a discount from the  commercial tariff for domestic shipments and negotiated single rate  factors for the overseas locations. Approved move managers were  awarded 2-year contracts with 1-year options. The move management  companies competitively bid their fees as flat rates, depending on whether  they were responsible for claims settlement or the transportation provider  carried this liability. Also, different fees were established for domestic and  international shipments.
The Full Service Moving Project used survey data from all personnel  participating in the pilot program to determine future percentages of  shipments that would be allocated to the transportation providers. The  pilot program also planned to use survey data on move manager  performance to determine future participation in the pilot program and  incentive payments. The Full Service Moving Project’s Web-based Best  Value Distribution Database maintained the transportation providers’  quality and cost scores based on survey information and costs associated  with prior shipments. Move managers used this data to assign future  shipments. However, in some instances (i.e., for group moves, when  meeting small business requirements, when there was a lack of  transportation provider capacity to handle shipments offered, for multiple  shipments to a single transportation provider, and for international  shipments to areas without an established rate), move managers were told  to deviate from the information provided by the data management system.
section_title: Visibility of Shipments during the Relocation Process
paragraphs: One of the goals of incorporating move managers into the relocation  process was to provide real-time information to military personnel and to  transportation providers regarding the status of household goods  shipments. The move managers, unlike the current program’s personal  property shipping office personnel, were responsible for the entire  relocation process from the point of origin in establishing entitlements,  arranging for transportation providers, and handling other personnel- related issues, to the destination in overseeing deliveries, approving  storage, and either settling claims or assisting military personnel with  issues involving settling claims with the transportation providers if the  liability fell with the providers. Working with both military personnel and  transportation providers, the move managers used contact information to  keep military personnel informed of their shipments’ status and to  coordinate the delivery of the shipments at the destination. Additionally,  as part of the pilot program, all participants were provided a toll-free  number to maintain visibility over their shipments throughout the process.
section_title: Availability and Reliability of Data on Household Goods Shipments
paragraphs: In addition to move managers, the Full Service Moving Project’s Web- based Best Value Distribution Database was implemented to address  problems associated with visibility of shipments during the relocation  process. The pilot program’s data management system had access to both  shipment and payment records via interface with US Bank’s PowerTrack  and the move managers’ systems. Access to the pilot program’s data  management system was granted to move managers, invoice certifying  officers, military service headquarters, and military service claims offices  and finance centers, based on each party’s requirements.
Move managers were responsible for keeping the status of the shipments  current in the pilot program’s data management system. However, the  move managers did not always update this information in the system.  Further, the ability of the move manager to contact the service member  was directly affected by the information provided by the member.
The Full Service Moving Project’s Web-based Best Value Distribution  Database was anticipated to provide DOD and the military services  information on the types of shipments and related costs managed annually  for planning and budgeting purposes. Unlike the Reengineered Personal  Property Program where various parties in the relocation process entered  data into that pilot program’s data management system, the majority of the  data in the Full Service Moving Project’s data management system was  predicated on the move managers gathering and entering the information.
section_title: Other Full Service Moving Project Features
paragraphs: The Full Service Moving Project achieved stronger transportation provider  commitment with long-term contracts and faster payment of invoices; it  offered binding cost estimates for shipments; it used contractor support to  conduct quality of life surveys with military personnel moving under the  pilot program and to perform audits of each invoice submitted by the  transportation providers; and it offered optional relocation referral  assistance for activities such as the purchase and sale of service members’  residences.
Move managers were required to perform prepayment audits and business  rules were established for an automatic payment method. Payment  methodology was predicated on the move manager entering the expected  invoice into PowerTrack and the transportation provider submitting a  notice of delivery and invoice. Payment timeliness was also driven by the  timeliness of documentation submitted by the transportation providers.  On some invoices, the contracting representative had to review and certify  payment in PowerTrack. This occurred when the match showed a  difference of more than $1.00.
section_title: Funding Sources
paragraphs: For this pilot program, the military services reimbursed carriers and  forwarders for shipment-related costs from military personnel accounts.  These accounts were also used to fund move manager expenses. Any  claims that might have been filed with the government would have been  funded from the military services’ operations and maintenance accounts.
section_title: Navy’s Service Member Arranged Move Pilot Program
paragraphs: Sponsored by the Navy, the Service Member Arranged Move Pilot Program  included only domestic outbound intrastate and interstate shipments for  Navy personnel moving from its installations located at Puget Sound,  Washington; San Diego, California; Norfolk, Virginia; New London,  Connecticut; and Whidbey Island, Washington. One of this program’s  objectives was to offer Navy military personnel a set of moving choices to  meet their specific needs. This pilot program was one of three choices  offered. Military personnel moving from the above locations could choose  to move under the current personal property program, move their own  household goods, or participate in the pilot program. The pilot program  was initiated in April 1997 and began operations in January 1998. Because  the Navy decided not to scope the Service Member Arranged Move Pilot  Program comparable to other pilot programs (i.e., operational at multiple  military services) and the pilot program did not provide data as outlined  by the Transportation Command’s evaluation plan, its inclusion in the  Transportation Command’s evaluation was limited to a qualitative  assessment.
subsections: section_title: Loss and Damage Claims
paragraphs: Reimbursement for loss and damage claims was increased from  depreciated value to full replacement value, with the dollar amounts per  move increasing from $40,000 under the current program to $72,000 under  the Service Member Arranged Move Pilot Program. Additionally, the pilot  program provided direct claims settlement between military personnel and  their transportation providers and a requirement that transportation  providers settle claims within 60 days of receiving claims forms from  military personnel.
section_title: Counseling and Arranging Shipment Services
paragraphs: Like the current program, the Service Member Arranged Move Pilot  Program also relied on personnel in the personal property shipping offices  to provide counseling services and arrange shipment and storage of  household goods and unaccompanied baggage. However, the shipping  office personnel at the origin installations participating in this pilot  program served as the single point of contact coordinating the service  members’ moves and remained available throughout the move to handle  all issues, including claims. Unlike those participating in the current  program and other pilot programs, service members participating in this  pilot program identified the transportation provider they desired to handle  their household goods shipments after they completed their reviews of  participating providers’ vendor quality books (containing provider  information and marketing materials) and of surveys completed by  previous pilot program participants. The personal property office  coordinator assigned to the service member had to concur with the  member’s request, and the coordinator made actual arrangements with the  carrier.
section_title: Screening and Shipment Distribution Process for Transportation Providers
paragraphs: Staff in the program management office and personal property shipping  offices participating in the Service Member Arranged Move Pilot Program  initially screened transportation providers that wished to participate in the  pilot program based on providers’ performance rather than low cost.  Letters of agreement were adopted to streamline the contracting process  and improve the quality of the move for Navy personnel. According to pilot  program officials, these letters of agreement provided commercial best  practices and enabled lessons learned from prior pilot program efforts and  industry to be incorporated into the Navy pilot program. Actual contract  awards were made on a case-by-case basis based on the best value  decision for each move. Transportation providers used commercial tariffs  in developing their bids for each move.
Transportation providers approved to participate in the pilot program  submitted their bids for various origin and destination channels using  commercial tariffs. Bids were rejected if they did not fall within acceptable  percentage discounts. Feedback was provided monthly to transportation  providers, and those that became poor performers were no longer offered  household goods and unaccompanied baggage shipments. Subsequently,  service members who were planning their upcoming moves relied on  information contained in a nine-question survey that other service  members had completed after their moves and claims process ended.  Service members who volunteered to participate in this pilot program had  to manually review carrier books, which included documents provided by  the carriers and prior surveys completed by service members who had  been moved by the carriers. According to pilot program officials, six  carriers were terminated or canceled from the pilot program–four for  providing poor service and two for price gouging.
section_title: Visibility of Shipments during Relocation Process
paragraphs: The Service Member Arranged Move Pilot Program relied upon the  shipping office personnel, who served as the single point of contact  coordinating the service members’ moves, to maintain visibility of  shipments during the relocation process. In addition, the pilot program  provided both the personal property shipping office’s and transportation  provider’s toll-free numbers, as well as a pager to service members to  enhance the members’ visibility of their shipments during the relocation  process.
section_title: Availability and Reliability of Data on Household Goods Shipments
paragraphs: The Service Member Arranged Move Pilot Program did not initially  develop an alternative data management system to capture data on  shipments and payment records. By the end of the pilot program, the Navy  had developed a database to capture shipment data; however, the system  was not fully implemented or evaluated.
Unlike the other pilot programs, the Navy pilot program used local  personal property program personnel rather than third parties to review  all invoices for payment. Navy personnel who participated in the pilot  program completed their own surveys, mailing the paper forms to their  respective personal property program offices.
section_title: Other Service Member Arranged Move Pilot Program Features
paragraphs: The Service Member Arranged Move Pilot Program was designed to offer  all shipments to small businesses, to provide direct claims settlements  between Navy personnel and the transportation providers, to make faster  payments to transportation providers through government purchase cards,  and to establish a stronger commitment from transportation providers by  offering long-term contracts.
section_title: Funding Sources
paragraphs: Like the current program, the Navy reimbursed carriers and forwarders for  shipment-related costs from its military personnel account. Personal  property shipment office expenses were funded from the Navy’s  operations and maintenance account. While information on claims filed  with the government was not provided, under this pilot program such  expenses would also be funded from the operations and maintenance  account.
section_title: Appendix III: Comments from the Department of Defense
section_title: Related GAO Products
paragraphs: Defense Transportation: Final Evaluation Plan Is Needed to Assess Alternatives to  the Current Personal Property Program. GAO/NSIAD-00-217R. Washington, D.C.:  September 27, 2000.
Defense Transportation: The Army’s Hunter Pilot Project Is Inconclusive but  Provides Lessons Learned. GAO/NSIAD-99-129. Washington, D.C.: June 23, 1999.
Defense Transportation: Plan Needed for Evaluating the Navy Personal Property  Pilot. GAO/NSIAD-99-138. Washington, D.C.: June 23, 1999.
Defense Transportation: Efforts to Improve DOD’s Personal Property Program.  GAO/T-NSIAD-99-106. Washington, D.C.: March 18, 1999.
Defense Transportation: The Army’s Hunter Pilot Project to Outsource Relocation  Services. GAO/NSIAD-98-149. Washington, D.C.: June 10, 1998.
Defense Transportation: Reengineering the DOD Personal Property Program.  GAO/NSIAD-97-49. Washington, D.C.: November 27, 1996.