Defense Transportation: DOD Needs a Comprehensive Approach to Planning for Implementing Its New Personal Property Program
Published: 2007-05-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) has been working to improve its personal property program since the mid-1990s to fix long-standing problems, such as excessive loss or damage to servicemembers' property and poor quality of service from moving companies. DOD plans to replace its current program with Families First, a program that promises to offer servicemembers an improved claims process and quality of service. GAO was mandated to (1) assess the steps DOD has taken to achieve the goals and benefits of the Families First program; (2) evaluate the growth in costs of the program, including the costs for a new information management system, since GAO's last assessment in 2003; and (3) assess the extent to which DOD faces management challenges--such as staffing--in implementing Families First. To address these objectives, GAO analyzed DOD's program, funding and staffing data, and interviewed personal property officials and stakeholders.
section_title: What GAO Found
paragraphs: DOD has taken some initial steps to achieve the goals and benefits of Families First, but delays in developing a new information management system have put the overall goals of improving the quality of service from moving companies and streamlining the claims process at risk. The information management system, the Defense Personal Property System (DPS), is now more than 2 years behind schedule. DOD has missed DPS milestones because of software development issues and is now working to address issues identified in recent software testing. Since DPS has been delayed, DOD is in the process of implementing a backup plan to meet a statutory mandate to provide servicemembers with the full replacement value of goods lost or damaged during a move by March 1, 2008. However, there are risks and costs associated with DOD's backup plan because it relies on an increasingly unreliable legacy computer system; also, DOD's plan may not cover all moves by March 1, 2008. The Families First program could increase costs to DOD by $1.4 billion over current program costs through fiscal year 2011 for two main reasons: (1) DOD estimates the program will increase costs to the services' household goods budgets by 13 percent and (2) DOD has significantly increased the cost estimate for a new information management system since GAO's last assessment. While DOD's estimate that the Families First program will increase costs by 13 percent has not changed since 2005, all of the services have not yet fully budgeted for this cost increase, which GAO analysis shows could be about $1.2 billion. Additionally, DOD has increased its estimate for an information management system for Families First because it decided to develop DPS rather than upgrade the legacy system. DOD estimated that the upgrade would cost $4 million to $6 million, and the program office estimated that DPS will cost about $180 million through fiscal year 2011. DOD's personal property program faces many management challenges--especially staffing, in addition to program requirements and funding problems--because it has not employed comprehensive planning. Sound management practices require a comprehensive approach that includes plans to assemble a qualified, trained, and well-led team; gain stakeholders' agreement about key program elements, such as business rules to define how the moving industry will serve military members; and estimate and plan for adequate resources. DOD has developed several draft plans to address individual portions of Families First and DPS, such as the draft transition plan for moving the DPS program office as part of a base realignment and closure move from Virginia to Illinois, but there is no overall plan that addresses how DOD will (1) fill significant staffing shortfalls in the newly formed DPS program office, (2) gain agreement from stakeholders, and (3) fund the significant and growing costs associated with the program. For example, DOD has not identified sources to fully fund DPS development and operations. Without a comprehensive plan, achieving the goals of the Families First program will likely remain difficult.

section_title: Letter
paragraphs: Military servicemembers and their families are typically relocated many  times during servicemembers’ careers. The Department of Defense’s  (DOD) personal property program provides household goods  transportation and storage services for military personnel and their  families when they relocate. DOD spent more than $1.8 billion in fiscal  year 2006 to move more than 680,000 shipments. The quality of these  moving services directly affects the quality of life for servicemembers and  their families, which has become increasingly important as  servicemembers face increased demands. However, as we have previously  reported, DOD has experienced long-standing problems with its personal  property program, especially poor quality of service from moving  companies that results in excessive loss or damage to servicemembers’  property and high claims costs incurred by the government. Moreover, a  DOD analysis indicated that the program’s legacy computer system for  data management does not meet DOD’s information technology standards  and is costly to operate and maintain. In an effort to test alternative  approaches and address some of these problems, DOD has spent more  than 11 years trying to improve the program through lessons learned from  its pilot programs.
DOD decided to replace its personal property program with a new  program known as Families First. The three goals of the Families First  program are improving the quality of service from moving companies,  streamlining the claims process for losses or damages incurred during a  move, and developing an integrated information management system. In  addition, the John Warner National Defense Authorization Act for 2007  mandated that DOD provide full replacement value coverage to  servicemembers for damages or losses incurred during their moves by  March 1, 2008. With full replacement value, a servicemember would  receive enough funds to replace or repair a lost or damaged item at its  present value. DOD’s current personal property program provides  servicemembers a minimum level of depreciated value coverage that does  not fully compensate them for lost or damaged goods. DOD plans to  incorporate full replacement value as an additional benefit of Families  First and provide this coverage at no additional cost to servicemembers.
Since 1999, we have completed several assessments of DOD’s personal  property program, including reviews of several aspects of DOD’s personal  property pilot programs. In 2003, we evaluated the methodology used to  estimate the costs associated with Families First that the services would  incur. The Families First program was initially expected to increase the  services’ costs for DOD’s personal property program by 13 percent, but we  questioned part of the methodology used to generate this estimate and  recommended that DOD provide the military services and Congress  additional information to quantify the risk associated with achieving the  projected 13 percent cost estimate. In 2005, DOD reevaluated its estimated  13 percent cost increase and quantified the risks of implementing Families  First within the expected cost increase. We reviewed the methodology  DOD used to estimate the cost of implementing the Families First  program. We found at that time that DOD used a reasonable methodology  to validate the estimated increase and quantify the risk.
In 2004, DOD began developing the Defense Personal Property System  (DPS), an integrated information management system that will be used to  support Families First. DOD’s initial assumption was that DPS could be  developed quickly using commercially available software. However, our  analysis indicates that DOD has missed milestones and costs have grown  in developing and implementing DPS. DOD entered into an 11-month  strategic pause in October 2005 to assess the next steps for DPS. DOD has  a history of challenges in implementing information management systems  similar to DPS, such as the Defense Integrated Military Human Resources  System and the Defense Travel System, which are part of DOD’s business  systems modernization efforts. Because of management challenges in  implementing these types of systems, we have placed DOD’s business  systems modernization on our high-risk list of programs that are at risk  because of their greater vulnerabilities to fraud, waste, abuse, and  mismanagement.
The John Warner National Defense Authorization Act for Fiscal Year 2007  mandated that we review the Families First program. Specifically, we   (1) assessed the steps DOD has taken to achieve the goals and benefits of  the Families First program; (2) evaluated the growth in the cost of the  program since the previous assessment, including the costs for DPS; and  (3) assessed the extent to which DOD faces management challenges— such as with staffing, developing capabilities, and resources within the  DPS program office—in implementing Families First.
To conduct our assessment of the Families First program, including its  costs, we reviewed prior DOD and GAO reports, documentation regarding  program development, program management staffing and resources, and  cost analyses for Families First and DPS. We also interviewed DOD  officials, private sector moving industry associations, and contractors  supporting DOD efforts. While we did not independently verify DOD’s  staffing and resource needs or its cost data, we did determine that the data  were sufficiently reliable for our purposes. We performed our audit work  from September 2006 through May 2007 in accordance with generally  accepted government auditing standards. The scope and methodology we  used is described in greater detail in appendix I.
subsections: section_title: Background
paragraphs: DOD’s personal property program is used by personal property shipping  office staff to manage household goods moves for all military  servicemembers and DOD civilians when they relocate. The military  services pay shipment and storage-related costs from their military  personnel accounts’ permanent change of station budgets and pay for loss  and damage claims and personal property shipment office expenses from  their operation and maintenance accounts. Servicemembers generally  work with DOD transportation officials at personal property processing  offices to coordinate their moves. These offices can either be service- specific offices or joint or consolidated property offices that assist  servicemembers from more than one service. These offices provide  servicemembers with local points of contact for counseling about their  moves and processing paperwork related to shipments of their personal  property. Prior to the reengineering efforts over the last 11 years, DOD’s  personal property program had remained virtually unchanged for nearly 40  years.
DOD’s personal property program involves a complex process of  qualifying carriers, soliciting rates, distributing moves, evaluating moving  companies’ performance, paying invoices, and settling claims. Among the  program’s many challenges is ensuring that the moving industry provides  adequate year-round capacity, especially during the summer peak moving  season when most servicemembers, as well as the general public, schedule  their moves. In an effort to test alternative approaches and address some  of its challenges, DOD previously evaluated three pilot programs. From  those three pilot programs, DOD submitted a report to Congress in 2002  with recommendations to improve the quality of household goods moves  for servicemembers that were originally contained in a U.S. Transportation  Command report. Those recommendations were as follows:    Reengineer the liability and claims process by adopting commercial  practices of minimum valuation, simplifying the filing of claims, and  providing the servicemember with direct settlement for claims with the  carrier.
Change the acquisition process to implement performance-based service  contracts (as opposed to the current practice of providing contracts to  the lowest bidder).  Implement information technology improvements, which could integrate  functions across such areas as personnel, transportation, financial, and  claims.
To respond to these recommendations, DOD developed a new program  called Families First to improve the quality of household goods moves for  servicemembers, DOD civilian employees, and their families. Families  First is a U.S. Transportation Command program that is executed by the  Military Surface Deployment and Distribution Command, an Army service  component command.
Since 1989, DOD’s personal property system has used the Transportation  Operational Personal Property Standard System, a legacy data  management system known as TOPS, which includes 25 additional legacy  systems that support it. The Surface Deployment and Distribution  Command determined that it was not feasible to upgrade TOPS to support  the goals of Families First for several reasons. TOPS is being phased out  because the software is no longer fully supportable and does not meet  DOD’s technology standards, including its security requirements. TOPS  also did not support the Business Management Modernization Program,  the program that preceded the Business Transformation Agency in  overseeing DOD’s business transformation efforts. In addition to these  technical considerations, TOPS also has poor reporting and data  capabilities. However, DOD expects that TOPS will need to be functional  for a large part of the Families First rollout, until DPS is fully operational.
Under the current system, servicemembers are provided with basic claims  coverage using depreciated value for losses or damages incurred during a  move that allows liability at a rate of $1.25 times the weight of the goods  being shipped. For example, if a shipment’s weight is 10,000 pounds, the  maximum liability for the moving company is $12,500. Additional coverage  options are available for the servicemember to purchase. Under the  current program, a servicemember has two options. Under option one, the  servicemember can purchase depreciated value coverage above what the  government currently pays, and under option two, the servicemember can  purchase full replacement value coverage. Under both options, the  servicemember shares the cost with the government. For moves within the  United States and overseas or stored shipments, servicemembers can  obtain additional coverage from a commercial insurance company. Some  private insurance companies and moving companies sell insurance to  cover certain items of personal property during moves. Additionally, some  homeowner policies may cover some items in shipment. See table 1 for  coverage and cost comparisons for the current DOD personal property  program versus what is planned under Families First.
We have completed several assessments that evaluated DOD’s pilot  programs and plans for implementing Families First. For example, in  2000, we reported that the U.S. Transportation Command needed to  complete an evaluation plan for its pilot programs and take necessary  actions to resolve outstanding cost issues. In 2003, we evaluated the  methodology used to estimate the costs associated with Families First that  the services would incur. The Families First program was initially  expected to increase the services’ costs for DOD’s personal property  program by 13 percent, but we questioned part of the methodology used to  generate this estimate. Specifically, we recommended that DOD quantify  the risks of implementing the Families First program within the 13 percent  estimate. As part of this evaluation, we also assessed a separate estimate  for the cost of upgrading the information technology system used for  managing the shipment of household goods. We questioned DOD’s ability  to implement the upgrades to the information technology system within its  cost estimate. We found that the estimate to implement the information  technology recommendation was slightly higher than the $4 million to   $6 million estimate DOD reported to Congress. In 2005, DOD reevaluated  its estimated 13 percent cost increase and quantified the risks of  implementing Families First within the expected cost increase. We found  at that time that DOD used a reasonable methodology to validate the  estimated increase and quantify the risk.
Congress has been concerned about problems in this program, especially  that servicemembers may receive less than what it would cost them to  replace or repair their household goods that are lost or damaged during  shipment. On November 24, 2003, the fiscal year 2004 National Defense  Authorization Act amended the U.S. Code to allow the Secretary of  Defense to include a clause for full replacement value in DOD’s contracts  with moving companies. The John Warner National Defense Authorization  Act for Fiscal Year 2007 mandated that DOD provide full replacement  value coverage by March 1, 2008, for servicemembers and DOD civilian  employees. With full replacement value, a servicemember would receive  enough funds to replace or repair a lost or damaged item at its present  value.
Additionally, the John Warner National Defense Authorization Act for  Fiscal Year 2007 mandated that the Secretary of Defense shall submit to  the congressional defense committees a report containing the  certifications of the Secretary on the following matters with respect to the  program of the Department of Defense known as Families First:   (1) whether there is an alternative to the system under the program that  would provide equal or greater capability at a lower cost; (2) whether the  estimates on costs, and the anticipated schedule and performance  parameters, for the program and system are reasonable; and (3) whether  the management structure for the program is adequate to manage and  control program costs. The mandate did not specify a date for DOD to  provide this information.
section_title: DOD Has Taken Some Initial Steps for Families First, but DPS Delays Put Achievement of Program Goals and Benefits at Risk
paragraphs: DOD has taken some initial steps to achieve the goals and benefits of the  Families First program, but delays in developing a new information  management system have put achieving the program’s goals and benefits  at risk. DOD continues to experience delays and missed milestones in  developing and implementing DPS, and the original estimated release date  for DPS has now been pushed back for more than 2 years. To meet the  statutory mandate, DOD has taken steps to provide servicemembers with  the full replacement value coverage benefit because of the delays in  implementing DPS. However, some servicemembers may not be covered  by March 1, 2008, and there are other risks associated with this backup  plan. Despite these challenges, Surface Deployment and Distribution  Command officials told us that they expect all types of moves will have  full replacement value by March 1, 2008. DOD continues to rely on the  implementation of DPS to achieve other program goals such as improving  the quality of service and claims processing.
subsections: section_title: DOD Has Taken Some Initial Steps to Achieve the Goals and Benefits of Families First
paragraphs: DOD has taken some initial steps to help achieve the goals and benefits of  the Families First program. To improve the personal property program,  DOD has established three goals for Families First: (1) improving the  quality of service from moving companies by using a best-value approach  that incorporates performance-based service contracts; (2) streamlining  the claims process used for claiming losses or damages incurred during a  move; and (3) developing an integrated information management system,  known as DPS. DOD designed Families First so that achieving the first two  goals of the program relies heavily on completion of the third goal of the  program, DPS. DOD identified numerous benefits of the Families First  program, including reduced storage costs and greater operational  flexibility for moving companies. Two of the program benefits identified  by DOD—full replacement value coverage and expanded counseling  support through a Web-based information system—are intended to  directly benefit servicemembers and promote quality service when moving  their personal belongings.
DOD developed a phased approach to implement Families First and has  taken some steps to accomplish the first and second phases. The first  phase, which began in March 2004, has two main parts: (1) electronic  billing and payment systems and (2) a customer satisfaction survey. The  electronic billing system, known as the Central Web Application, is a  government Web-based system for reviewing and approving services  online, as well as for pricing shipments. The electronic payment system,  U.S. Bank’s PowerTrack, is an online payment and transaction tracking  system. This system is expected to reduce the payment cycle for DOD’s  personal property moves. While DOD and all services other than the Air  Force have made some progress in implementing the electronic billing and  payment systems, the Air Force is not processing its own bills and  payments using these systems because it is reengineering its payment  process and cannot currently support these systems. DOD is working to  fully interface and integrate electronic billing and payment systems,  respectively, with DPS but continues to experience operational problems,  such as invoices being delayed or lost when being processed.
In addition, as part of the first phase, DOD began data collection for a  customer satisfaction survey, which is intended to support the Families  First goal of improving moving company performance through evaluation  of past performance. Under Families First, servicemembers are expected  to fill out a customer satisfaction survey about their moves, the results of  which will be combined with other data to generate an overall moving  company quality score. The moving companies with the best scores will be  awarded more shipments. This process contributes to the best-value  distribution of shipments. Under DOD’s current household goods  program, DOD awards shipments to the company that bids the lowest  price for a move. To generate data for ranking moving companies when  Families First is implemented, DOD has instituted an interim customer  satisfaction survey under the current program. However, interim customer  satisfaction survey response rates have been about 16 percent within the  past year, which has resulted in less than one-third of moving companies’  scores being usable. To compensate for the low response rate, DOD has  developed a methodology upon program implementation to make moving  companies’ scores statistically valid so the scores can be used when  allocating shipments. However, the moving companies are concerned  about how the low survey response rate will affect how DOD awards  business to them. In addition to developing the methodology to ensure  that moving companies’ scores are statistically valid, DOD has taken  several steps it says will increase the customer satisfaction survey  response rate. For example, it has released a commercial to increase  awareness about the survey and added information in its It’s Your Move  pamphlet. It also included the customer satisfaction survey requirement  in the Defense Transportation Regulation. DOD also expects that the  survey response rate will improve once DOD implements DPS and  servicemembers can file their surveys electronically within DPS. Both of  these components—the electronic billing and payment systems and the  customer satisfaction survey—are necessary to support Families First.
The second phase of Families First includes the development and  implementation of DPS, which DOD has been working on for more than 2  years. DOD plans to use DPS to implement many key improvements for  the Families First program. For example, Families First implementation  documents state that with DPS, DOD will be able to    use best-value distribution when awarding performance-based service    provide Web-based counseling to help servicemembers with their moves,    use a commercial-based tariff for domestic moves rather than the  antiquated government tariff currently being used,    provide direct claims settlement with the transportation service provider,    use a “rate reasonableness” strategy that will help DOD manage the costs  of the moving program.
DOD also plans to use DPS to provide the electronic customer satisfaction  survey to servicemembers and to help DOD monitor the rates moving  companies charge it for moving services.
The third, and final, phase of the Families First program includes adding  functionality to DPS so that it can handle more types of moves, including  nontemporary storage (about 16 percent of all moves) and direct  procurement moves (about 8 percent of all moves).
section_title: DPS Development and Implementation Has Experienced Delays
paragraphs: DOD continues to face delays and missed milestones in developing and  implementing DPS. DPS development and implementation has been  pushed back for more than 2 years from the original estimated release  date. DOD began DPS development in May 2004 and DPS was originally  scheduled to be available by October 2005. In October 2005, the Surface  Deployment and Distribution Command initiated a review of the program.  DOD then entered into an 11-month strategic pause for further program  review and software testing after it encountered significant software  validation and systems problems, which resulted in the system not  working. DOD subsequently developed two more implementation  timelines, the first in October 2006 and the second in March 2007. See  table 2 for a comparison of DPS implementation timelines.
During the October 2005 internal review of DPS, DOD’s review group  recommended improvements in areas such as management, the type of  contract used for DPS, and the DPS development process. The strategic  pause following this review ended in September 2006, but the next  schedule for DPS implementation was not developed until after a new DPS  program office was created in October 2006. This schedule incorporated a  phased rollout approach for DPS. Under this schedule, DPS software  acceptance testing was to occur in winter 2007, followed by increasing use  of DPS through summer and fall 2007. DOD expected DPS to be fully  operational in spring 2008.
DPS program managers developed what they described as an aggressive  implementation schedule for two reasons. First, they planned to use DPS  to meet the mandate to provide full replacement value by March 1, 2008.  Second, DPS implementation was needed because the legacy system used  with the current personal property system is not fully supportable and  does not meet DOD information technology security standards. Program  and service officials said that the legacy system has problems interfacing  with DOD’s networks. In addition, the legacy system’s hardware has been  breaking down. Surface Deployment and Distribution Command officials  said that the number of sites not functioning at any one time varies. To  keep the legacy system working, the Surface Deployment and Distribution  Command provided the services with legacy system “survival kits.” These  kits included motherboards and other hardware components that are  difficult to find and are no longer supported commercially. DOD estimates  that these survival kits will keep the legacy systems viable for 4 to 5 years,  but some service personal property officials have expressed concerns that  the legacy systems might not last that long.
DOD delayed the DPS implementation schedule again in February 2007,  after stakeholders from the services and the moving industry participated  in DPS software acceptance testing and found a significant number of  problems with the software. This testing generated more than 1,400  problem reports, almost 200 of which were collectively expected to result  in significant changes to the software. For example, for a shipment  awarded to one moving company, DPS sent the work order for the  shipment to a different moving company. Thus, the moving company that  was awarded the shipment did not know the shipment was awarded to it.  In addition, according to a U.S. Transportation Command official, some  test reports indicated that business rules still needed to be clarified, such  as whether moving companies will have one or two opportunities per year  to file the rates they will charge DOD to move servicemembers’ household  goods.
In March 2007, because of the number of test problem reports and overall  concern about DPS functionality, DPS program management officials  significantly altered the timeline for rolling out DPS to address concerns  expressed by military service and moving industry stakeholders regarding  DPS functionality and its implementation schedule. Stakeholders were  also concerned that the implementation schedule called for switching to  DPS during the peak summer season, when both the services and industry  would have to learn a new system while also moving shipments using the  current system. The revised DPS implementation schedule calls for fixing  the issues identified in the test problem reports, continued testing of DPS  through the summer, and adding high-priority changes requested by the  services. Program managers said that DPS should be available for some  shipping offices to use by fall 2007 on a test basis, with all offices using  DPS beginning in spring 2008 for all moves except those using  nontemporary storage and the direct procurement method. Once DPS is  functional for domestic and international household goods moves,  program managers will begin developing the functionality for the third  phase of Families First, which includes moves using nontemporary storage  and the direct procurement method.
section_title: DOD Has a Plan to Provide Full Replacement Value without DPS, but Does Not Have a Plan to Implement the Other Goals and Benefit of Families First without DPS
paragraphs: Because of the delays implementing DPS, DOD has developed a backup  plan to provide servicemembers with the full replacement value coverage  benefit, but its plan to implement the other goals and benefit of Families  First still relies on DPS. When the backup plan was published in December  2006, it called for the next version of the current program’s domestic tariff  and international rate solicitation to include language that made it  mandatory for moving companies to include full replacement value  coverage in the rates submitted to DOD. Using this backup plan in the  current program, the majority of shipments will receive full replacement  value protection by March 1, 2008. The schedule for implementing the  backup plan follows the current program’s winter 2007 rate filing  schedules for the domestic intra- and interstate programs and the  international programs. The Surface Deployment and Distribution  Command plans to begin accepting rates under the backup plan beginning  in May 2007, and the rates will be effective from October 1, 2007, through  April 1, 2008.
Risk factors associated with DOD’s backup plan challenge DOD’s ability to  implement the plan. First, the backup plan relies on a legacy system that is  no longer fully supportable. For example, the system still does not meet  security standards. DOD estimates that the survival kits it has sent to the  services can keep the legacy system running for 4 to 5 years. However,  some service officials had concerns that the system would not last this  long. Without the legacy system, staff at the personal property offices have  to work manually to accomplish administrative tasks. Furthermore, some  service officials expressed concern that providing full replacement value  without DPS would give the moving industry an opportunity to increase  prices with no control to limit the cost and may create some increase in  workloads for the claims offices because of the lack of automation for  claims filing. Moreover, most services expressed concern about the lack of  guidance for implementing full replacement value under the current  system instead of within DPS. Some service and Surface Deployment and  Distribution Command officials expressed concerns about possible  increases in their workload because of the magnitude of the procedural  changes as they work to implement full replacement value without DPS.
Another risk is that thousands of moves may not be covered before the  March 1, 2008, deadline. DOD’s contracts for moves within a theater of  operation, those using nontemporary storage, and those using the direct  procurement method do not include full replacement value and may  expire after the March 2008 mandate. DOD stated that it is initiating  various levels of action to ensure full replacement value is implemented by  March 2008. According to DOD, as these contracts expire, the new  contract will include full replacement value. The Surface Deployment and  Distribution Command directed that all eligible contracts be modified not  later than March 2008. However, it is still uncertain whether all contracts  in place on March 1, 2008, will cover full replacement value. According to  DOD estimates, in fiscal year 2006, moves that included servicemembers  transferring within a theater of operation accounted for about 7,800  personal property moves, or about 1 percent of the more than 680,000  shipments that occurred. DOD officials also stated that in fiscal year 2006  direct procurement method moves represented almost 8 percent of all  moves. About 16 percent of moves included nontemporary storage.
In a broader sense, DOD’s backup plan does not address the other goals or  counseling benefit of Families First; it is designed only to allow DOD to  meet its mandate to provide full replacement value coverage. DOD  officials said that they did not have a requirement to produce a backup  plan for the other goals or counseling benefit and they did not invest  resources to do so. Instead, DOD continues to rely on DPS to achieve the  goals and counseling benefit of Families First. The reason Surface  Deployment and Distribution Command officials said they created a  backup plan for full replacement value is because they were required by  statute to implement it by March 1, 2008, whether DPS was ready or not.  For example, the backup plan does not address how to provide  streamlined claims or improved quality of service from moving companies  without DPS, nor does it include a way to provide the other  servicemember benefit of expanded Web counseling services to help  servicemembers with their moves without DPS. Until DPS is operational,  some service officials have said that DOD has at least one other option for  providing expanded counseling services because the Navy has a program,  known as SmartWebMove, which can be used by members from other  services. However, this program is connected to the legacy system, and  deterioration of the legacy system may limit the feasibility of this option.
Despite these challenges, Surface Deployment and Distribution Command  officials told us that they expect all types of moves will have full  replacement value by March 1, 2008. According to these officials, this will  include nontemporary storage and direct procurement method moves.  DOD continues to rely on the implementation of DPS to achieve other  program goals such as improving the quality of service and claims  processing.
section_title: Families First Program Could Increase DOD Costs by About $1.4 Billion over Current Program Costs through Fiscal Year 2011
paragraphs: The Families First program could increase costs to DOD by about   $1.4 billion over current program costs through fiscal year 2011 for two  main reasons: (1) DOD estimates the program will increase costs to the  services by 13 percent and (2) DOD has significantly increased the cost  estimate for a new information management system since our last  assessment. DOD’s Families First program has not yet been implemented,  so we could not assess the actual growth in costs of the program, although  DOD continues to estimate that the Families First program will increase  the cost to the services for their household goods budgets by an estimated  13 percent, or as much as $1.2 billion through fiscal year 2011. In addition,  the DPS program office has significantly increased the estimated cost of  DPS and maintaining the DPS program office, which it now expects to cost  $180 million through fiscal year 2011.
subsections: section_title: Actual Growth in Costs of Families First Program Cannot Be Assessed, Although DOD Continues to Estimate a 13 Percent Increase in Services’ Costs to Implement Families First
paragraphs: We could not assess the actual growth in costs of Families First because  the program has not been implemented; however, DOD continues to  estimate that the costs to the services of the Families First program will be  13 percent higher than costs under the current program. In fiscal year  2006, the services’ total household goods budget was about $1.8 billion,  which would mean the services would have an increase of $240 million  annually above the existing budget in order to move servicemembers’  household goods under Families First in 2007, if the program were fully  implemented. DOD will incorporate a cost-control mechanism into DPS,  similar to the one employed in the current program, in an attempt to keep  the costs within the expected increase. However, until DPS is  implemented the impact of the use of this mechanism on the Families First  program will not be known. Based on DOD’s total household goods  budget, Families First could cost DOD about $1.2 billion more than the  current program over the next 5 years.
DOD continues to inform the services that the Families First program,  when fully implemented, will cost them an additional 13 percent over their  existing household goods budgets, which is a subset of the services’  permanent change of station budgets. According to U.S. Transportation  Command and some personal property officials, this cost increase is in  part because of an expectation by DOD that moving companies will  increase the rates they charge as a result of their additional  responsibilities under the Families First business rules, such as providing  full replacement value. The actual cost of Families First will not be known  until moving companies file the rates they will charge DOD to move  servicemembers, which is expected to take place in March 2008. DOD is  relying on features built into DPS to ensure that the costs remain at or  below the expected cost increase of 13 percent. DPS will incorporate a  cost-control mechanism known as rate reasonableness, which will  establish an acceptable range of rates for each combination of pickup and  destination locations.
The program delays in implementing Families First decrease the certainty  of the cost estimate because the methodology is based on certain  assumptions and data that may change with time. For example, the cost  methodology used to estimate the 13 percent increase was adjusted to  account for fewer small businesses participating DOD-wide than  participated in the pilot programs. However, according to DOD officials,  the percentage of small business participation in Families First will be  similar to the current DOD participation rate of 70 percent, which is  significantly larger than the 30 percent assumed in the 2002 cost estimate.  DOD’s evaluation of the pilot programs demonstrated that small  businesses were 14 to 74 percent more expensive per shipment compared  to the current program. As a result, DOD may have underestimated the  cost of having small businesses participate in Families First.
section_title: Estimated Costs for the Integrated Information Management System Have Significantly Increased since Our Last Assessment
paragraphs: DOD’s estimated costs for an integrated information management system,  known as DPS, have significantly increased since our last assessment in  2003. The estimates for developing an information management system to  support Families First have increased from $4 million to $6 million to   $86.0 million, and the total cost is expected to be about $180 million  through fiscal year 2011 once annual operating costs are added.
In a 2002 report, DOD estimated that implementing the information  technology improvements to enhance its data management capabilities for  Families First would cost $4 million to $6 million. This estimate was based  on expanding the use of an upgraded, Web-based version of the existing  legacy system that was implemented on a small scale during one of DOD’s  pilot programs. In our April 2003 review of that estimate, we questioned  whether DOD would be able to implement its new personal property  program, including the technology improvements, within the estimated  range. In addition, we noted that although DOD had developed a plan of  action for designing the new system, the plan did not include monitoring  the costs and benefits during its implementation or the extent to which  system changes were being achieved within an acceptable cost range, such  as the $4 million to $6 million estimate.
According to DOD officials and documents, in January 2004, DOD decided  to implement the technology improvements to support Families First by  developing an entirely new information management system, which came  to be known as DPS. In a January 2004 report, the Surface Deployment  and Distribution Command said that the legacy system evaluated under  the previous cost estimate was expensive to operate and maintain and  could not be modified to become compliant with DOD technology  standards or to support the objectives of the Families First program. At  that time, DOD estimated that DPS could be developed for about   $16.5 million, with an average annual cost of about $4.6 million after the  initial investment. This estimate assumed that DPS could be developed  using commercial-off-the-shelf or government-off-the-shelf software to  account for about 75 percent of the new system. The use of existing  commercial and government software was expected to keep the cost of  the system low, because using ready-made software reduces the need to  develop original software. For example, the Navy developed a counseling  program, known as SmartWebMove, which was originally planned to be  incorporated into DPS as its counseling module so that DOD would not  need to develop original software as part of DPS to provide this Families  First benefit. However, the Navy’s counseling module was not  incorporated into DPS. The Navy, however, is still using SmartWebMove  while DPS is being developed.
Since the 2004 estimate, the cost of DPS has continued to increase. As of  February 2007, DOD reports that it spent $51.5 million developing DPS,  which is significantly higher than any previous DOD estimate. This cost  includes about $8.2 million for capital hardware, $24.9 million for capital  software, and $18.5 million in operating costs. According to DOD Families  First officials, after the DPS contract was awarded, software developers  determined that DPS would require a much larger percentage of new  software development than expected because of the unique needs of the  DOD personal property stakeholders, which has caused the cost to rise  significantly. In addition, the costs for developing, testing, and making  DPS available for use now include the cost of the Joint Program  Management Office for Household Goods Systems, which was established  on October 1, 2006. The original estimates did not account for a separate  program office to manage the development and operation of DPS, sustain  the legacy system, or evaluate future options for DOD’s household goods  program. Based on our analysis of program office budget planning  documents from February 2007, the DPS program office estimated that the  costs for maintaining a program office, sustaining the legacy system  through retirement, developing and sustaining DPS, and implementing a  future household goods program through fiscal year 2011will be   $180 million if all of the requirements are funded. Additional delays in the  schedule are likely to further increase the costs associated with the  program. However, when the legacy system is no longer needed, DOD  expects that it will not have to budget for this additional cost, which is  about $21 million annually. Summary information on DPS cost estimates  appears in table 3.
section_title: DOD Faces Management Challenges for Families First Program and Has Not Developed a Comprehensive Implementation Plan
paragraphs: DOD faces management challenges for the Families First program, and it  has not employed comprehensive planning that incorporates many sound  management principles and practices. Families First offices, including the  DPS program office, continue to experience organizational challenges and  staffing shortfalls. Moreover, Families First does not have stakeholders’  agreement on some elements of the program, such as business rules and  essential DPS functions. Additionally, the Families First program faces  uncertain funding.
subsections: section_title: DOD Planning for Families First Did Not Incorporate Some Sound Management Principles and Practices
paragraphs: Sound management practices require employing comprehensive planning  to manage program implementation. Comprehensive planning should  include many things, such as integrated approaches to manage training  and workforce redeployment issues; a qualified, trained, and well-led team  to reengineer the program; stakeholder agreement about key elements of a  program, including the program’s business rules and its priorities; and full  cost information and funding resources. However, DOD’s planning for  Families First has not incorporated some of these sound management  practices. Instead, DOD has developed several nonintegrated plans to  cover individual portions of the Families First program. For example, DOD  has a draft transition plan for organizational changes and the DPS program  office has a plan for DPS development. However, DOD does not have a  comprehensive plan with timelines for implementing Families First that  manages all of its efforts simultaneously. Without an integrated,  comprehensive plan, the program’s implementation is at risk.
section_title: Families First Faces Significant Organizational Changes and Staffing Shortfalls
paragraphs: The offices supporting Families First, including the program office now  overseeing the development and implementation of DPS, are undergoing  organizational changes and experiencing staffing shortfalls that affect  DOD’s ability to support Families First at a critical time in its  implementation. When the first phase of Families First implementation  and DPS development began in 2004, the Surface Deployment and  Distribution Command, which is under the U.S. Transportation Command,  managed all elements of the program. In December 2005, almost 2 years  after DPS development started, the Surface Deployment and Distribution  Command established a DPS program management office based on a  recommendation made by a DOD review group. The review group  suggested that the Surface Deployment and Distribution Command  establish a clear management structure for DPS because there was no  single point of authority and there was no acquisition-certified program  manager. On October 1, 2006, the U.S. Transportation Command  transferred this office from the Surface Deployment and Distribution  Command to the U.S. Transportation Command. The new office, named  the Joint Program Management Office for Household Goods Systems, is  under the leadership of the U.S. Transportation Command’s Program  Executive Office for Distribution Services. The DPS program office and  the program executive office are now led by officials with acquisition  experience.
The DPS program office has several tasks:    mature the program office structure and processes;    sustain the legacy system currently being used through the development of  its replacement, DPS;    quickly implement DPS in phases; and    evaluate alternatives for the future of DOD household goods services,  including options for outsourcing.
Although DOD’s establishment of the DPS program office addresses some  of the concerns about DPS program management raised in DOD’s review,  the Joint Program Management Office for Household Goods Systems was  not established until a few months prior to a critical phase of DPS  development and continues to organize while also facing staffing  challenges. The DPS program office had a draft organization chart as of  March 2007, but filling staff positions has been complicated by a base  realignment and closure move to Scott Air Force Base in Illinois from the  office’s current location in Alexandria, Virginia. This move is scheduled to  take place in the fourth quarter of fiscal year 2007. The program office’s  draft transition plan transfers several positions from the Surface  Deployment and Distribution Command to the DPS program office.  However, this has created human capital challenges, as many of the staff  are choosing to retire or leave rather than move. These workforce  planning issues are significantly affecting the DPS program office.  According to DPS program management officials, as of April 2007, only 1  of 27 civilians in the program office planned to transfer to Scott Air Force  Base. Thus, while Families First and DPS are at a critical stage of  development, both the Surface Deployment and Distribution Command  and the DPS program office are losing many of their senior leaders who  possess technical and program knowledge.
The DPS program office is working to mitigate these human capital  planning challenges by seeking authority to hire over current staffing  limits, seeking temporary functional support from other Surface  Deployment and Distribution Command and U.S. Transportation  Command offices, and continuing to seek support from the services and  industry as software testers. As of April 2007, hiring actions had been  accelerated and some job announcements had been made. The program  office is also using contractor support but is facing challenges with this as  well. For example, in March 2007 several contractors were not able to  complete tasks for the program office because of paperwork processing  issues. In addition, in March 2007, the DPS program office asked each of  the services to provide two or three full-time servicemembers to continue  conducting DPS software testing at the Surface Deployment and  Distribution Command headquarters in Alexandria, Virginia. While the  services plan to provide some human capital support, current service  plans indicate that they cannot provide the servicemember support DPS  management officials originally sought because each service will need its  staff during the busy, peak moving season that coincides with DPS testing.  For example, the Navy is planning to provide five part-time testers at Navy  bases. The Army is planning to provide five part-time testers at Army  bases. The Marine Corps plans to provide one full-time person to test at  program headquarters as well as one support staff member at its testing  site at Camp Lejeune in North Carolina. The Air Force is also providing  full-time support from its joint personal property shipping office in  Colorado Springs, Colorado. Overall, it is not clear how successful these  temporary mitigation efforts will be in providing staff with the skills these  offices need to implement both DPS and Families First. However, DOD  stated that its joint stakeholder advisory team of testers will be sufficient  to fulfill the mission required by the DPS program office.
Surface Deployment and Distribution Command officials, who will manage  and provide oversight of the current DOD personal property program and  implementation of the Families First program, said that they are also  facing additional workload and workforce challenges as they administer  the electronic billing and payment systems as well as the customer  satisfaction survey. These officials are administering these processes  without the automation they expect DPS will provide while also  experiencing staff reductions and changes as a result of a base  realignment and closure move.
As of April 2007, the U.S. Transportation Command has made some  progress to staff the DPS program office, but it is not clear how successful  its measures will be. Until the U.S. Transportation Command is able to  ensure that the DPS program office has adequate and capable human  capital resources, it may be unable to successfully implement DPS.
The John Warner National Defense Authorization Act for Fiscal Year 2007  mandated that the Secretary of Defense submit to the congressional  defense committees a report containing the certifications of the Secretary  on the following matters with respect to the program of the Department of  Defense known as Families First: (1) whether there is an alternative to the  system under the program that would provide equal or greater capability  at a lower cost; (2) whether the estimates on costs, and the anticipated  schedule and performance parameters, for the program and system are  reasonable; and (3) whether the management structure for the program is  adequate to manage and control program costs. When the U.S.  Transportation Command established the DPS program office, it included  an evaluation of materiel alternatives for the future of household goods  services as part of the office’s mission. The mandate did not specify a date  for DOD to provide this information to the congressional defense  committees. The U.S. Transportation Command is responsible for leading,  with the assistance of the DPS program office, the evaluation of  alternatives. The DPS program office is responsible for evaluating how to  implement the chosen alternative. It is unclear when the DPS program  office will be able to evaluate materiel alternatives for the program  because (1) U.S. Transportation Command officials told us they were  focusing on developing and implementing DPS and (2) the DPS program  office has not yet been resourced to evaluate the materiel alternatives.
section_title: DOD Does Not Have Stakeholders’ Agreement on Key Elements of Families First
paragraphs: DOD does not have stakeholders’ agreement on some elements of Families  First, which puts the implementation of the program at risk. DOD does not  have stakeholders’ agreement in two interrelated areas: (1) business rules  issues, including whether existing and proposed rules will actually enable  accomplishment of a key program goal, and (2) the essential functions  needed for DPS.
subsections: section_title: Uncertainties Surround Families First Business Rules
paragraphs: Stakeholders, including the military services, have not all agreed to some  elements of Families First business rules and have not taken action to  implement all of the business rules because it is unclear to them if the  rules are final. At the end of our audit work, the U.S. Transportation  Command was still evaluating whether the business rules would have to  be published again for comment by stakeholders. However, in commenting  on a draft of this report in May 2007, DOD stated that the business rules  are now considered final. Business rules help define how policies are to  be implemented. DPS requirements and functions are derived from these  business rules. For example, in late March 2007, several months into DPS  testing, DOD was still evaluating a business rule as to whether moving  companies should have the opportunity to file the rates they charge DOD  to move servicemembers’ household goods once or twice per year. In the  current program, rates are filed twice per year. Under Families First  business rules, moving companies would file rates only once per year. In  April 2007, DOD decided to continue with its Families First business rule  where moving companies only file rates once per year.
Within DOD, debate continues about whether Families First business rules  will allow DOD to accomplish its goal of improving the quality of service  from moving companies by using a best-value approach that incorporates  performance-based service contracts. Some DOD officials (and industry  representatives) question DOD’s proposed practice of allocating business  to moving companies using a system where companies that receive less  than the best performance score are still allocated business. For example,  under Families First rules, moving companies will be ranked into four  groups based on their performance scores. Those companies with the best  scores will be placed into the first group and receive the most DOD  business. However, DOD officials said that even those companies that are  in the fourth group, with the lowest performance scores, are expected to  receive some business from DOD. According to DOD officials and industry  representatives, one reason DOD will do this is to keep providing business  to those companies that may not otherwise be able to stay open during the  nonpeak moving season. These stakeholders said that this helps ensure  that there will be enough capacity during peak moving season. However,  some servicemembers’ household goods may be moved by companies that  did not receive high performance scores, and therefore they may not  receive quality moves. If this is not resolved, DOD may be challenged to  meet the program’s goal of improving the quality of service from moving  companies.
Additionally, during the course of our work, stakeholders indicated that  they did not consider the business rules final. However, DOD, in  commenting on a draft of this report, stated that the business rules are  now considered final. Stakeholders indicated that they do not yet know  what will be expected of them under Families First or what DPS must  include to fully support the program. Stakeholders said that the business  rules are not considered final until they have been published in the  Defense Transportation Regulation. The U.S. Transportation Command  published business rules for phase one of the program in the Defense  Transportation Regulation on February 20, 2007. However, it has not  published business rules for the second and third phases of Families First  in the Defense Transportation Regulation. The business rules have only  been published on the Surface Deployment and Distribution Command’s  Web site and once in the Federal Register so that stakeholders could  comment on them. During our review, U.S. Transportation Command  officials indicated that DOD planned to publish the business rules again in  the Federal Register in June 2007 so that stakeholders could comment on  them again and said that DOD would finalize the business rules in July  2007. It is unclear whether DOD still plans to publish the rules again in the  Federal Register. Along with the uncertainty surrounding the business  rules, stakeholders do not have procedural guidance and do not yet know  what is expected of them under Families First. For example, an Air Force  personal property official told us the Air Force needs the finalized  Families First business rules so that it can train its staff on these new  rules, which the personal property official described as being vastly  different from the current program’s business rules. However, the Air  Force personal property official said the Air Force is hesitant to develop a  training curriculum on business rules that are not finalized. Additionally,  without final business rules, the services cannot set up internal regulations  to support the business rules. Moreover, representatives from the services  and the moving industry are concerned that without a formal set of  business rules on which to develop DPS, they cannot evaluate whether the  computer system fully supports the Families First business rules. Service  officials and industry representatives continue to have questions about  Families First business rules and DPS implementation.
Finally, the moving companies have concerns about the Families First  business rules that define how DOD generates the performance scores  used to rank them in the first, second, third, or fourth groups. The majority  of a moving company’s performance score comes from a customer  satisfaction survey. However, servicemember response rates for the  survey have been low (about 16 percent within the past year) and, because  of this, most moving companies’ scores are not statistically valid for  generating a performance score. Although DOD has, as part of its business  rules, devised a methodology to make moving company performance  scores valid until survey response rates improve, industry representatives  are still concerned that moving companies will be negatively affected by  low response rates. In commenting on a draft of this report, DOD stated  that while it values the opinion of the moving industry, its personal  property program does not require consensus by industry. DOD stated that  the main focus of the department is to provide a quality personal property  program for servicemembers while being good stewards of taxpayer  dollars.
section_title: Stakeholders Have Not Agreed to All Essential Functions for DPS
paragraphs: Another fundamental challenge facing DOD in implementing DPS is that  stakeholders, such as the military services and the moving industry, have  not agreed to all of the essential functions of DPS and how they should  operate when DPS is made available to servicemembers to use. Service  officials told us that prior to the development of the DPS program office,  the Surface Deployment and Distribution Command held many meetings  to understand what the services wanted DPS to provide servicemembers  and personal property officials. However, service officials said that  officials overseeing DPS development at that time did not include all of  those requirements when first developing DPS. A Surface Deployment and  Distribution Command official said that the contract for DPS was written  from a requirements list generated by military service and moving industry  stakeholder participation. For example, there was a General Officer  Steering Committee, Council of Colonels and Captains, and moving  industry stakeholder groups, which met to discuss DPS requirements. In  early 2007, after the U.S. Transportation Command took over DPS,  stakeholders had their first opportunity to test DPS. During these tests,  stakeholders identified functions that they expected within DPS but that  did not work the way they expected. This resulted in DPS not providing  the functionality service officials expected, and this, in turn, could affect  the services’ workloads. A U.S. Transportation Command official said that  it is possible that there was miscommunication during earlier meetings to  define requirements and that it was not until stakeholders were able to test  DPS functionality that these issues were identified. For example, DPS  users wanted to obtain the status of a moved shipment. When DPS was  programmed, it only displayed whether the shipment was in the system.  However, users wanted more detail in terms of where the shipment was at  a certain point in time. DPS program management is still in the process of  identifying and prioritizing the requirements for DPS, but currently lacks  stakeholders’ agreement about all of those requirements and their priority.  For example, some stakeholders disagree with the categories assigned to  some of the test problem reports, because none of the reports were placed  in categories 1 or 2, which are used for the most severe types of problems.  Further, the moving industry expected that DPS would interface with their  computer systems, but this is not yet part of DPS.
DPS program officials said that earlier phases of DPS development lacked  a mechanism for systematically reviewing DPS problems and requirements  and identifying how to fix them. The U.S. Transportation Command and  the Surface Deployment and Distribution Command formed a Functional  Requirements Board to review and prioritize the problems identified  during testing that must be fixed and to address other proposed changes  to DPS. The Functional Requirements Board is composed of  representatives from the services, the Surface Deployment and  Distribution Command, and the U.S. Transportation Command and meets  monthly to discuss which testing problems should be the highest priority  for correcting. The prioritized list of test problems is then reviewed by a  Configuration Control Board, which is composed of DPS program  managers, service representatives, DPS development contractors, and  software engineers who decide which of the DPS problems can be  corrected after considering the resources available. As of March 2007,  according to DOD, the Functional Requirements Board had developed  initial DPS functional requirements, reviewed many proposed system  enhancements, and prioritized the services’ top five needs in each DPS  module. In addition to stakeholders’ requirements, additional priorities for  DPS may also come from the business rules. This, too, could affect the  DPS implementation timeline, as well as implementation of Families First.
Another challenge is that without stakeholders’ agreement, DPS  requirements continue to change. DPS development is being administered  using a firm-fixed-price contract. With a firm-fixed-price contract all  major modifications to DPS require negotiation with the contractor, which  may lead to additional administrative costs.
section_title: Families First Is Not Fully Funded
paragraphs: Even though Families First is projected to cost the services about   $1.2 billion over the next 5 years, and DPS is expected to cost about   $180 million through fiscal year 2011, the department has not set aside  funding to fully cover these costs. The services have taken different  approaches in budgeting for the increased costs expected to implement  the Families First program, ranging from the Army requesting the entire  estimated 13 percent increase to the Navy not requesting any increase at  all, in part because they have not received clear guidance from DOD about  how to calculate the estimated increase to their budgets. Moreover, the  growing cost of DPS has led to funding shortfalls in the DPS program  office that are affecting both staffing needs and software development.
subsections: section_title: Services Vary in the Extent to Which They Have Budgeted for Families First
paragraphs: The services vary in the extent to which they have budgeted for the  increased costs expected to implement the Families First program. As  previously discussed, DOD estimates that the Families First program will  increase the services’ moving budgets by 13 percent above the current  budgets needed to move household goods, and DOD has informed the  services to budget based on this estimate. However, some personal  property officials expressed concerns about DOD’s ability to implement  the Families First program within the expected increase of 13 percent;  these officials expect that the cost increase will be significantly more. As a  result, the services vary in the degree to which they have budgeted for  Families First. According to service officials, the services have taken the  following actions to budget for Families First:    The Army has submitted a budget request for the entire 13 percent  increase to the household goods portion of its budget in fiscal years 2008  and 2009.
The Coast Guard requested the 13 percent increase based on its entire  permanent change of station budget, of which household goods is just a  portion.
The Air Force submitted a budget that included the 13 percent cost  increase for Families First in fiscal year 2008, but the Office of the  Secretary of Defense did not agree with the Air Force’s budget submission  and reduced its funding for permanent change of station moves.
The Marine Corps has requested funding in fiscal years 2007, 2008, and  2009 only for its estimated full replacement value cost.
The Navy has not requested any of the expected 13 percent cost increase.
In addition, some personal property officials stated that they are having  difficulty budgeting because they have not received clear guidance about  how to calculate the increase. As a result, the services also vary in how  they interpret the effect of the 13 percent cost estimate on their household  goods budgets. For example, the Air Force estimated its typical annual  expected increase in the current household goods program and then  added 13 percent. Army officials told us they were unclear whether the  household goods program would be increasing by 13 percent every year or  just the first year of Families First. A Coast Guard budget official  interpreted the 13 percent increase as an increase to just those portions of  the budget that apply to the rates charged by moving companies. Neither  the Surface Deployment and Distribution Command nor the Office of the  Secretary of Defense Comptroller have provided clear guidance on how  the services are supposed to apply the 13 percent estimate to their  household goods budgets. As a result, the services may continue to apply  the 13 percent in different ways, which could result in the program not  being fully funded.
According to some service officials, if the expected increase in Families  First cost is not included in their budgets and program costs begin to rise  as Families First is implemented, then the services may have to consider  measures to reduce their household goods budgets. This could affect the  number of moves the services can make and could ultimately impact the  services’ flexibility in meeting force management needs. Surface  Deployment and Distribution Command officials said they plan to monitor  the cost of Families First in two ways. First, these officials will use the rate  reasonableness methodology to keep the cost at the estimated 13 percent  increase. Further, they plan to use reporting functions in DPS to monitor  program costs. However, it is unclear how officials will monitor the costs  of the program without a fully functioning DPS. Additionally, although the  services plan to monitor the actual cost of the program as part of their  normal budget processes, there is no plan to provide updated estimates to  the services about the cost of the program prior to the services actually  incurring the cost. Further, without clear guidance to the services about  how to apply the 13 percent cost increase to their permanent change of  station budgets, it is unclear whether the services will budget  appropriately for the projected Families First cost increase. Without  updated information about whether the estimated increase remains  accurate as Families First is implemented, the services may not budget for  Families First or may budget inaccurately for the program.
section_title: Growing DPS Costs Are Creating Funding Shortfalls
paragraphs: As a result of the growing costs of DPS, the DPS program office is  experiencing funding shortfalls that are affecting both staffing needs and  software development. As of April 2007, the U.S. Transportation Command  had not identified how it would fully fund its projected costs of DPS,  which it estimated in February 2007 to be about $180 million through fiscal  year 2011. Without these funds, the U.S. Transportation Command will be  challenged to staff the DPS program office and complete DPS software  development. The U.S. Transportation Command has been trying to fund  DPS and the DPS program office from its transportation working capital  fund.
In fiscal year 2007, the U.S. Transportation Command reallocated about  $7.5 million from its transportation working capital fund to support DPS.  According to a U.S. Transportation Command budget planning document,  this resulted in other U.S. Transportation Command information  technology program delays or affected their ability to provide some  services. Even with the reallocation, the U.S. Transportation Command  had to defer about $9.7 million needed for high-priority software changes  and development essential for DPS to fiscal year 2008. The DPS program  office has an anticipated shortfall for fiscal year 2008 of $21 million, which  includes staff training, contractor support, and funds for staff travel.  Travel funds are important since the DPS contractor will be in Virginia and  the program staff will be located at Scott Air Force Base in Illinois.
However, the U.S. Transportation Command has not yet developed a  detailed budget plan to resource DPS or the DPS program office. The  information technology portion of the U.S. Transportation Command’s  transportation working capital fund has an annual budget of about   $400 million. The DPS program office estimates that it will cost from   $28 million to $43 million annually to support DPS and the program office  for fiscal years 2008 through 2012. This is from 7 to about 11 percent of the  U.S. Transportation Command’s information technology portion of the  transportation working capital budget. The U.S. Transportation Command  has asked the Office of the Secretary of Defense for $5 million during  fiscal year 2008, but the request has not yet been approved. Additionally,  U.S. Transportation Command officials said that they have informed the  services that they expect them to provide funds for some additional DPS  requirements.
DPS program office officials based their current cost estimates for DPS  and the program office on the aggressive timeline for implementing DPS  before it changed in February 2007. Additional delays in the schedule  because of problems developing the software will likely increase the costs  associated with the program. At the time of our review, it was too early in  the DPS implementation process to determine if the oversight provided by  the program office will be effective in keeping DPS costs under control  given the ongoing changes to the DPS implementation schedule and the  significant number of software changes identified during software testing.
section_title: Conclusions
paragraphs: Despite DOD’s recent focus on its personal property program, long- standing problems persist. The department has invested millions of dollars  trying to improve the program since the mid-1990s with little real progress.  DOD’s Families First program is intended to address many of these long- standing problems but has faced a myriad of management problems and is  now at a critical juncture in its implementation. The underlying problem is  that DOD has not developed a comprehensive plan to organize, staff, and  fund Families First. Until DOD develops a detailed plan to adequately  recruit and retain qualified personnel, gain stakeholder agreement about  essential requirements for DPS, and set aside resources such as funding  and staff, it will be unable to effectively address the challenges to the  program. Relying on DPS to achieve program goals—without analyzing  alternatives as required by Congress—puts Families First at risk.  Moreover, at a time when our nation faces increasing financial constraints  and it is increasingly important for DOD to maximize the return on its  investment in new systems, DPS costs are continuing to increase while  DOD has little to show for its 11 years of reengineering efforts and millions  of dollars of investment. Without a reexamination of the program as  required by the mandate and urgent attention commensurate with the  program’s importance to millions of servicemembers and their families,  these problems are likely to continue to negatively affect servicemembers’  quality of life when they are required to move.
section_title: Recommendations for Executive Action
paragraphs: We are making the following two recommendations to the Secretary of  Defense. To address long-standing problems in DOD’s personal property  program we recommend that the Secretary of Defense direct the  Commander, U.S. Transportation Command, to expedite the evaluation of  the Families First program the John Warner National Defense  Authorization Act for Fiscal Year 2007 mandated the department conduct.  This act mandates that the report contain the certifications of the  Secretary of Defense on the following matters with respect to the Families  First program: (1) whether there is an alternative to the system under the  program that would provide equal or greater capability at a lower cost;    (2) whether the estimates on costs, and the anticipated schedule and  performance parameters, for the program and system are reasonable; and  (3) whether the management structure for the program is adequate to  manage and control program costs.
We also recommend that DOD employ comprehensive planning to  implement the Families First program and its associated system. At a  minimum, this planning should address specific steps to    hire and train personnel so that the Surface Deployment and Distribution  Command personal property division and the DPS program office have the  human capital needed to develop and implement DPS and  reach agreement with stakeholders on the essential requirements for DPS  and their priority to facilitate the development of DPS.
In addition, this comprehensive plan should include an investment  strategy that reflects the full cost of accomplishing the goals of Families  First and milestones for implementation.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD concurred with both of  our recommendations and cited specific actions it has taken and will take  for each recommendation. We believe DOD’s planned actions, if  implemented, have the potential to improve the outcome of the Families  First program. However, we also believe it is critical that DOD sustain  focus on the program, especially given the delays the program has  experienced and the challenges it still faces.
DOD concurred with our recommendation to expedite the evaluation of  the Families First program that the John Warner National Defense  Authorization Act for Fiscal Year 2007 mandated the department conduct.  In its comments, DOD said that it plans to provide this evaluation of its  household goods program to Congress in August 2007. We modified our  recommendation to include the details of what the act requires.
DOD also concurred with our recommendation that it should employ  comprehensive planning to implement the Families First program. In  addition, DOD provided technical comments suggesting that we include  the Surface Deployment and Distribution Command personal property  division as part of this recommendation. We agree, and have modified this  recommendation accordingly. Our recommendation now indicates that at  a minimum, this planning should include specific steps to hire and train  personnel for the Surface Deployment and Distribution Command  personal property division and the DPS program office, address specific  steps to reach agreement with stakeholders on the essential requirements  for DPS and their priority, and include an investment strategy that reflects  the full cost of accomplishing the goals of Families First and milestones  for implementation.
DOD cited specific actions it has taken or will take to implement this  recommendation. For example, DOD said that hiring actions are in  progress to staff the DPS program office after its relocation to Scott Air  Force Base and that other actions are being implemented to staff the  Surface Deployment and Distribution Command. In addition, DOD stated  that it has implemented a process to reach agreement with stakeholders  on the essential requirements for DPS and their priority by establishing the  Functional Requirements Board and the Configuration Control Board.  DOD stated that this structure has helped to stabilize the functional  requirements for DPS. Finally, DOD stated that the U.S. Transportation  Command will provide almost $91 million for DPS development, testing,  fielding, and maintenance and that there is a DPS line item in the U.S.  Transportation Command’s transportation working capital fund budget.  The U.S. Transportation Command is also working with the Office of the  Secretary of Defense to provide almost $2.8 million of  operating/maintenance dollars from transformation funding. DOD’s  comments also state that the U.S. Transportation Command will provide  funds internally, if required, to fund DPS and the DPS program office.  However, DOD’s comments did not address how the department intends  to develop an investment strategy to cover the over $1 billion in increased  costs associated with implementing Families First. Developing such a plan  for Families First will be critical for the program’s future success.
DOD also provided technical and editorial comments, which we have  incorporated as appropriate. DOD’s comments are reprinted in appendix  II.
We are sending copies of this report to interested congressional  committees; the Secretary of Defense; the Secretaries of the Army, Navy,  and Air Force; the Commander, U.S. Transportation Command; the Office  of the Assistant Deputy Under Secretary of Defense (Transportation  Policy); and the Director, Office of Management and Budget. We will make  copies available to others upon request. In addition, the report will be  available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (404) 679-1816 or pendletonj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To evaluate the Department of Defense’s (DOD) Families First program,  we obtained documentation from and met with DOD officials and  stakeholders in the Washington, D.C., area from   the Military Surface Deployment and Distribution Command;  the Joint Program Management Office for Household Goods Systems;    Transportation and personal property offices of the Army, Navy, Air Force,  Marine Corps, and Coast Guard;  three moving industry associations, including the American Moving and  Storage Association, the Household Goods Forwarders Association of  America, and the Military Mobility Coalition; and  the Offices of the Secretary of Defense, including Transportation Policy  and the Comptroller.
We also met with DOD officials at Scott Air Force Base, Illinois, from the    U.S. Transportation Command’s Program Executive Office for Distribution    U.S. Transportation Command’s Strategy, Policy, Programs and Logistics    U.S. Transportation Command’s Program Analysis and Financial  Management Directorate.
To assess the steps DOD has taken to achieve the goals and benefits of the  Families First program with or without a new information management  system, we identified the goals and benefits of the Families First program  by analyzing Families First planning documents and related studies, such  as briefings to the U.S. Transportation Command, and verified these goals  with personal property officials from the Surface Deployment and  Distribution Command. We compared the status of the Families First  program to its stated goals by examining Families First implementation  timelines provided by program officials and by interviewing officials and  stakeholders from the offices listed. We limited our evaluation of the  benefits of the Families First program to those benefits that pertain to the  servicemembers, specifically full replacement value coverage and  expanded counseling services.
We determined DOD’s ability to achieve the goals and benefits of Families  First with or without a new information management system by  monitoring the status of the Defense Personal Property System (DPS)  throughout the course of our review. This included observing  demonstrations of DPS as it was presented to representatives from the  services and moving industry, reviewing test problem reports generated  during DPS software acceptance testing, and examining briefings in which  DPS program management and stakeholders reevaluated the DPS  implementation schedule. We also reviewed a Federal Register notice  provided by Surface Deployment and Distribution Command officials,  which described its plans for implementing the full replacement value  benefit of Families First without DPS. We analyzed the current program  business rules and requirements and compared them to the goals and  benefits of Families First to determine if alternatives existed in the current  program to implement them without DPS. We also interviewed the  officials and stakeholders listed. We asked these officials and stakeholders  to provide alternatives for achieving the Families First goals and benefits  without DPS. When an alternative was identified, we questioned officials  and stakeholders about the feasibility and possible challenges of  implementing Families First goals and benefits without DPS.
To evaluate the growth in the cost of the program since our previous  assessment, we determined that we could not evaluate the actual growth  in costs because data were unavailable as the program has not yet been  implemented. However, we determined that DOD was still advising the  services to budget for the previously estimated 13 percent cost increase  for Families First. To understand this estimated increase, we reviewed the  estimated cost of the Families First program from our two previous  reports related to the cost of the Families First program. We analyzed the  size of the services’ current permanent change of station budgets and  assessed how the cost of the Families First program would increase those  budgets using information provided by the Office of the Secretary of  Defense Comptroller. Families First program management officials stated  that DOD would use a cost-control mechanism known as rate  reasonableness to ensure that program costs do not exceed the estimated  increase. We analyzed Families First business rules and planning  documents, such as its concept of operations, to determine the feasibility  and limitations of the rate reasonableness approach. We interviewed  officials from the Military Surface Deployment and Distribution Command  and the U.S. Transportation Command Office for Transportation Policy to  determine whether their estimate of the cost of Families First has changed  since our previous assessment and what their plans were to keep the cost  of the Families First program within the estimate. We also asked officials  from the Military Surface Deployment and Distribution Command if they  had plans to monitor the costs during implementation of the program.  Although we did not independently test the reliability of data DOD  extracted from its data system to develop costs or independently verify the  analysis used to generate cost estimates, we determined the data were  sufficiently reliable for the purposes of our report because they are the  same data DOD decision makers use to manage the program.
To assess the growth in the estimated cost of DPS, we reviewed our  previous report, which contained DOD’s estimate of the cost of improving  its information technology system. We compared this estimate to  estimates contained in the U.S. Transportation Command budget planning  documents and the DPS economic analysis completed in 2003, which also  documented how DOD’s concept for information technology system  improvements changed since our last review. To obtain updated  information about the current costs of developing and fielding DPS, we  analyzed budget documents provided by the Joint Program Management  Office for Household Goods Systems as of February 2007. To understand  the U.S. Transportation Command’s transportation working capital fund  Chief Information Officer Program Review Process, we reviewed related  documents, such as the Chief Information Office Program Review Process  business flow and interviewed budget officials at the U.S. Transportation  Command’s Program Analysis and Financial Management directorate.  Although we did not independently test the reliability of data DOD  extracted from its data system to develop costs or the analysis used to  generate cost estimates, we determined that the data were sufficiently  reliable for the purposes of our report because they are the same data  DOD decision makers use to manage the program.
To assess the extent to which DOD faces management challenges in  implementing the Families First program, we analyzed documents, such as  Families First program meeting minutes and management briefings to the  General Officer Steering Committee, the Council of Colonels and Captains,  and the U.S. Transportation Command, which identified difficulties in  implementing the Families First program. We also identified best practices  for business reengineering and transformation, which we used to compare  the process by which DOD is implementing the Families First program.  These best practices were found in prior GAO reports. We also reviewed  documents pertaining to the Joint Program Management Office for  Household Goods Systems, including the draft organization chart, the draft  transition plan for the office’s move to Scott Air Force Base as part of a  base realignment and closure move, and draft budget documents for  developing and implementing DPS. We also interviewed officials and  stakeholders. We did not evaluate DOD’s decision to implement the  Families First program or develop DPS, nor did we evaluate the  solicitation process for awarding the DPS contract.
We conducted this performance audit from September 2006 through May  2007 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Lawson Gist, Jr., Assistant  Director; Krislin Bolling; Renee S. Brown; Michelle Cooper; Arthur L.  James, Jr.; Tina M. Kirschbaum; Lonnie McAllister; Maewanda Michael- Jackson; Charles W. Perdue; and Bethann Ritter made key contributions to  this report.
section_title: Related GAO Products
paragraphs: Defense Transportation: DOD Has Adequately Addressed Congressional  Concerns Regarding the Cost of Implementing the New Personal  Property Program Initiatives. GAO-05-715R. Washington, D.C.: June 9,  2005.
Defense Transportation: Monitoring Costs and Benefits Needed While  Implementing a New Program for Moving Household Goods. GAO-03-367.  Washington, D.C.: April 18, 2003.
Defense Transportation: Final Evaluation Plan Is Needed to Assess  Alternatives to the Current Personal Property Program. GAO/NSIAD-00- 217R. Washington, D.C.: September 27, 2000.
Defense Transportation: The Army’s Hunter Pilot Project Is Inconclusive  but Provides Lessons Learned. GAO/NSIAD-99-129. Washington, D.C.:  June 23, 1999.
Defense Transportation: Plan Needed for Evaluating the Navy Personal  Property Pilot. GAO/NSIAD-99-138. Washington, D.C.: June 23, 1999.
Defense Transportation: Efforts to Improve DOD’s Personal Property  Program. GAO/T-NSIAD-99-106. Washington, D.C.: March 18, 1999.
Defense Transportation: The Army’s Hunter Pilot Project to Outsource  Relocation Services. GAO/NSIAD-98-149. Washington, D.C.: June 10, 1998.
Defense Transportation: Reengineering the DOD Personal Property  Program. GAO/NSIAD-97-49. Washington, D.C.: November 27, 1996.