Defense Transportation: DOD Needs a Comprehensive Approach to Planning for Implementing Its New Personal Property Program
Released: 2007-05-31T00:00:00

Letter

Background

DOD’s personal property program is used by personal property shipping office staff to manage household goods moves for all military servicemembers and DOD civilians when they relocate. The military services pay shipment and storage-related costs from their military personnel accounts’ permanent change of station budgets and pay for loss and damage claims and personal property shipment office expenses from their operation and maintenance accounts. Servicemembers generally work with DOD transportation officials at personal property processing offices to coordinate their moves. These offices can either be service- specific offices or joint or consolidated property offices that assist servicemembers from more than one service. These offices provide servicemembers with local points of contact for counseling about their moves and processing paperwork related to shipments of their personal property. Prior to the reengineering efforts over the last 11 years, DOD’s personal property program had remained virtually unchanged for nearly 40 years.

DOD’s personal property program involves a complex process of qualifying carriers, soliciting rates, distributing moves, evaluating moving companies’ performance, paying invoices, and settling claims. Among the program’s many challenges is ensuring that the moving industry provides adequate year-round capacity, especially during the summer peak moving season when most servicemembers, as well as the general public, schedule their moves. In an effort to test alternative approaches and address some of its challenges, DOD previously evaluated three pilot programs. From those three pilot programs, DOD submitted a report to Congress in 2002 with recommendations to improve the quality of household goods moves for servicemembers that were originally contained in a U.S. Transportation Command report. Those recommendations were as follows: Reengineer the liability and claims process by adopting commercial practices of minimum valuation, simplifying the filing of claims, and providing the servicemember with direct settlement for claims with the carrier.

Change the acquisition process to implement performance-based service contracts (as opposed to the current practice of providing contracts to the lowest bidder). Implement information technology improvements, which could integrate functions across such areas as personnel, transportation, financial, and claims.

To respond to these recommendations, DOD developed a new program called Families First to improve the quality of household goods moves for servicemembers, DOD civilian employees, and their families. Families First is a U.S. Transportation Command program that is executed by the Military Surface Deployment and Distribution Command, an Army service component command.

Since 1989, DOD’s personal property system has used the Transportation Operational Personal Property Standard System, a legacy data management system known as TOPS, which includes 25 additional legacy systems that support it. The Surface Deployment and Distribution Command determined that it was not feasible to upgrade TOPS to support the goals of Families First for several reasons. TOPS is being phased out because the software is no longer fully supportable and does not meet DOD’s technology standards, including its security requirements. TOPS also did not support the Business Management Modernization Program, the program that preceded the Business Transformation Agency in overseeing DOD’s business transformation efforts. In addition to these technical considerations, TOPS also has poor reporting and data capabilities. However, DOD expects that TOPS will need to be functional for a large part of the Families First rollout, until DPS is fully operational.

Under the current system, servicemembers are provided with basic claims coverage using depreciated value for losses or damages incurred during a move that allows liability at a rate of $1.25 times the weight of the goods being shipped. For example, if a shipment’s weight is 10,000 pounds, the maximum liability for the moving company is $12,500. Additional coverage options are available for the servicemember to purchase. Under the current program, a servicemember has two options. Under option one, the servicemember can purchase depreciated value coverage above what the government currently pays, and under option two, the servicemember can purchase full replacement value coverage. Under both options, the servicemember shares the cost with the government. For moves within the United States and overseas or stored shipments, servicemembers can obtain additional coverage from a commercial insurance company. Some private insurance companies and moving companies sell insurance to cover certain items of personal property during moves. Additionally, some homeowner policies may cover some items in shipment. See table 1 for coverage and cost comparisons for the current DOD personal property program versus what is planned under Families First.

We have completed several assessments that evaluated DOD’s pilot programs and plans for implementing Families First. For example, in 2000, we reported that the U.S. Transportation Command needed to complete an evaluation plan for its pilot programs and take necessary actions to resolve outstanding cost issues. In 2003, we evaluated the methodology used to estimate the costs associated with Families First that the services would incur. The Families First program was initially expected to increase the services’ costs for DOD’s personal property program by 13 percent, but we questioned part of the methodology used to generate this estimate. Specifically, we recommended that DOD quantify the risks of implementing the Families First program within the 13 percent estimate. As part of this evaluation, we also assessed a separate estimate for the cost of upgrading the information technology system used for managing the shipment of household goods. We questioned DOD’s ability to implement the upgrades to the information technology system within its cost estimate. We found that the estimate to implement the information technology recommendation was slightly higher than the $4 million to $6 million estimate DOD reported to Congress. In 2005, DOD reevaluated its estimated 13 percent cost increase and quantified the risks of implementing Families First within the expected cost increase. We found at that time that DOD used a reasonable methodology to validate the estimated increase and quantify the risk.

Congress has been concerned about problems in this program, especially that servicemembers may receive less than what it would cost them to replace or repair their household goods that are lost or damaged during shipment. On November 24, 2003, the fiscal year 2004 National Defense Authorization Act amended the U.S. Code to allow the Secretary of Defense to include a clause for full replacement value in DOD’s contracts with moving companies. The John Warner National Defense Authorization Act for Fiscal Year 2007 mandated that DOD provide full replacement value coverage by March 1, 2008, for servicemembers and DOD civilian employees. With full replacement value, a servicemember would receive enough funds to replace or repair a lost or damaged item at its present value.

Additionally, the John Warner National Defense Authorization Act for Fiscal Year 2007 mandated that the Secretary of Defense shall submit to the congressional defense committees a report containing the certifications of the Secretary on the following matters with respect to the program of the Department of Defense known as Families First: (1) whether there is an alternative to the system under the program that would provide equal or greater capability at a lower cost; (2) whether the estimates on costs, and the anticipated schedule and performance parameters, for the program and system are reasonable; and (3) whether the management structure for the program is adequate to manage and control program costs. The mandate did not specify a date for DOD to provide this information.

DOD Has Taken Some Initial Steps for Families First, but DPS Delays Put Achievement of Program Goals and Benefits at Risk

DOD has taken some initial steps to achieve the goals and benefits of the Families First program, but delays in developing a new information management system have put achieving the program’s goals and benefits at risk. DOD continues to experience delays and missed milestones in developing and implementing DPS, and the original estimated release date for DPS has now been pushed back for more than 2 years. To meet the statutory mandate, DOD has taken steps to provide servicemembers with the full replacement value coverage benefit because of the delays in implementing DPS. However, some servicemembers may not be covered by March 1, 2008, and there are other risks associated with this backup plan. Despite these challenges, Surface Deployment and Distribution Command officials told us that they expect all types of moves will have full replacement value by March 1, 2008. DOD continues to rely on the implementation of DPS to achieve other program goals such as improving the quality of service and claims processing.

DOD Has Taken Some Initial Steps to Achieve the Goals and Benefits of Families First

DOD has taken some initial steps to help achieve the goals and benefits of the Families First program. To improve the personal property program, DOD has established three goals for Families First: (1) improving the quality of service from moving companies by using a best-value approach that incorporates performance-based service contracts; (2) streamlining the claims process used for claiming losses or damages incurred during a move; and (3) developing an integrated information management system, known as DPS. DOD designed Families First so that achieving the first two goals of the program relies heavily on completion of the third goal of the program, DPS. DOD identified numerous benefits of the Families First program, including reduced storage costs and greater operational flexibility for moving companies. Two of the program benefits identified by DOD—full replacement value coverage and expanded counseling support through a Web-based information system—are intended to directly benefit servicemembers and promote quality service when moving their personal belongings.

DOD developed a phased approach to implement Families First and has taken some steps to accomplish the first and second phases. The first phase, which began in March 2004, has two main parts: (1) electronic billing and payment systems and (2) a customer satisfaction survey. The electronic billing system, known as the Central Web Application, is a government Web-based system for reviewing and approving services online, as well as for pricing shipments. The electronic payment system, U.S. Bank’s PowerTrack, is an online payment and transaction tracking system. This system is expected to reduce the payment cycle for DOD’s personal property moves. While DOD and all services other than the Air Force have made some progress in implementing the electronic billing and payment systems, the Air Force is not processing its own bills and payments using these systems because it is reengineering its payment process and cannot currently support these systems. DOD is working to fully interface and integrate electronic billing and payment systems, respectively, with DPS but continues to experience operational problems, such as invoices being delayed or lost when being processed.

In addition, as part of the first phase, DOD began data collection for a customer satisfaction survey, which is intended to support the Families First goal of improving moving company performance through evaluation of past performance. Under Families First, servicemembers are expected to fill out a customer satisfaction survey about their moves, the results of which will be combined with other data to generate an overall moving company quality score. The moving companies with the best scores will be awarded more shipments. This process contributes to the best-value distribution of shipments. Under DOD’s current household goods program, DOD awards shipments to the company that bids the lowest price for a move. To generate data for ranking moving companies when Families First is implemented, DOD has instituted an interim customer satisfaction survey under the current program. However, interim customer satisfaction survey response rates have been about 16 percent within the past year, which has resulted in less than one-third of moving companies’ scores being usable. To compensate for the low response rate, DOD has developed a methodology upon program implementation to make moving companies’ scores statistically valid so the scores can be used when allocating shipments. However, the moving companies are concerned about how the low survey response rate will affect how DOD awards business to them. In addition to developing the methodology to ensure that moving companies’ scores are statistically valid, DOD has taken several steps it says will increase the customer satisfaction survey response rate. For example, it has released a commercial to increase awareness about the survey and added information in its It’s Your Move pamphlet. It also included the customer satisfaction survey requirement in the Defense Transportation Regulation. DOD also expects that the survey response rate will improve once DOD implements DPS and servicemembers can file their surveys electronically within DPS. Both of these components—the electronic billing and payment systems and the customer satisfaction survey—are necessary to support Families First.

The second phase of Families First includes the development and implementation of DPS, which DOD has been working on for more than 2 years. DOD plans to use DPS to implement many key improvements for the Families First program. For example, Families First implementation documents state that with DPS, DOD will be able to use best-value distribution when awarding performance-based service provide Web-based counseling to help servicemembers with their moves, use a commercial-based tariff for domestic moves rather than the antiquated government tariff currently being used, provide direct claims settlement with the transportation service provider, use a “rate reasonableness” strategy that will help DOD manage the costs of the moving program.

DOD also plans to use DPS to provide the electronic customer satisfaction survey to servicemembers and to help DOD monitor the rates moving companies charge it for moving services.

The third, and final, phase of the Families First program includes adding functionality to DPS so that it can handle more types of moves, including nontemporary storage (about 16 percent of all moves) and direct procurement moves (about 8 percent of all moves).

DPS Development and Implementation Has Experienced Delays

DOD continues to face delays and missed milestones in developing and implementing DPS. DPS development and implementation has been pushed back for more than 2 years from the original estimated release date. DOD began DPS development in May 2004 and DPS was originally scheduled to be available by October 2005. In October 2005, the Surface Deployment and Distribution Command initiated a review of the program. DOD then entered into an 11-month strategic pause for further program review and software testing after it encountered significant software validation and systems problems, which resulted in the system not working. DOD subsequently developed two more implementation timelines, the first in October 2006 and the second in March 2007. See table 2 for a comparison of DPS implementation timelines.

During the October 2005 internal review of DPS, DOD’s review group recommended improvements in areas such as management, the type of contract used for DPS, and the DPS development process. The strategic pause following this review ended in September 2006, but the next schedule for DPS implementation was not developed until after a new DPS program office was created in October 2006. This schedule incorporated a phased rollout approach for DPS. Under this schedule, DPS software acceptance testing was to occur in winter 2007, followed by increasing use of DPS through summer and fall 2007. DOD expected DPS to be fully operational in spring 2008.

DPS program managers developed what they described as an aggressive implementation schedule for two reasons. First, they planned to use DPS to meet the mandate to provide full replacement value by March 1, 2008. Second, DPS implementation was needed because the legacy system used with the current personal property system is not fully supportable and does not meet DOD information technology security standards. Program and service officials said that the legacy system has problems interfacing with DOD’s networks. In addition, the legacy system’s hardware has been breaking down. Surface Deployment and Distribution Command officials said that the number of sites not functioning at any one time varies. To keep the legacy system working, the Surface Deployment and Distribution Command provided the services with legacy system “survival kits.” These kits included motherboards and other hardware components that are difficult to find and are no longer supported commercially. DOD estimates that these survival kits will keep the legacy systems viable for 4 to 5 years, but some service personal property officials have expressed concerns that the legacy systems might not last that long.

DOD delayed the DPS implementation schedule again in February 2007, after stakeholders from the services and the moving industry participated in DPS software acceptance testing and found a significant number of problems with the software. This testing generated more than 1,400 problem reports, almost 200 of which were collectively expected to result in significant changes to the software. For example, for a shipment awarded to one moving company, DPS sent the work order for the shipment to a different moving company. Thus, the moving company that was awarded the shipment did not know the shipment was awarded to it. In addition, according to a U.S. Transportation Command official, some test reports indicated that business rules still needed to be clarified, such as whether moving companies will have one or two opportunities per year to file the rates they will charge DOD to move servicemembers’ household goods.

In March 2007, because of the number of test problem reports and overall concern about DPS functionality, DPS program management officials significantly altered the timeline for rolling out DPS to address concerns expressed by military service and moving industry stakeholders regarding DPS functionality and its implementation schedule. Stakeholders were also concerned that the implementation schedule called for switching to DPS during the peak summer season, when both the services and industry would have to learn a new system while also moving shipments using the current system. The revised DPS implementation schedule calls for fixing the issues identified in the test problem reports, continued testing of DPS through the summer, and adding high-priority changes requested by the services. Program managers said that DPS should be available for some shipping offices to use by fall 2007 on a test basis, with all offices using DPS beginning in spring 2008 for all moves except those using nontemporary storage and the direct procurement method. Once DPS is functional for domestic and international household goods moves, program managers will begin developing the functionality for the third phase of Families First, which includes moves using nontemporary storage and the direct procurement method.

DOD Has a Plan to Provide Full Replacement Value without DPS, but Does Not Have a Plan to Implement the Other Goals and Benefit of Families First without DPS

Because of the delays implementing DPS, DOD has developed a backup plan to provide servicemembers with the full replacement value coverage benefit, but its plan to implement the other goals and benefit of Families First still relies on DPS. When the backup plan was published in December 2006, it called for the next version of the current program’s domestic tariff and international rate solicitation to include language that made it mandatory for moving companies to include full replacement value coverage in the rates submitted to DOD. Using this backup plan in the current program, the majority of shipments will receive full replacement value protection by March 1, 2008. The schedule for implementing the backup plan follows the current program’s winter 2007 rate filing schedules for the domestic intra- and interstate programs and the international programs. The Surface Deployment and Distribution Command plans to begin accepting rates under the backup plan beginning in May 2007, and the rates will be effective from October 1, 2007, through April 1, 2008.

Risk factors associated with DOD’s backup plan challenge DOD’s ability to implement the plan. First, the backup plan relies on a legacy system that is no longer fully supportable. For example, the system still does not meet security standards. DOD estimates that the survival kits it has sent to the services can keep the legacy system running for 4 to 5 years. However, some service officials had concerns that the system would not last this long. Without the legacy system, staff at the personal property offices have to work manually to accomplish administrative tasks. Furthermore, some service officials expressed concern that providing full replacement value without DPS would give the moving industry an opportunity to increase prices with no control to limit the cost and may create some increase in workloads for the claims offices because of the lack of automation for claims filing. Moreover, most services expressed concern about the lack of guidance for implementing full replacement value under the current system instead of within DPS. Some service and Surface Deployment and Distribution Command officials expressed concerns about possible increases in their workload because of the magnitude of the procedural changes as they work to implement full replacement value without DPS.

Another risk is that thousands of moves may not be covered before the March 1, 2008, deadline. DOD’s contracts for moves within a theater of operation, those using nontemporary storage, and those using the direct procurement method do not include full replacement value and may expire after the March 2008 mandate. DOD stated that it is initiating various levels of action to ensure full replacement value is implemented by March 2008. According to DOD, as these contracts expire, the new contract will include full replacement value. The Surface Deployment and Distribution Command directed that all eligible contracts be modified not later than March 2008. However, it is still uncertain whether all contracts in place on March 1, 2008, will cover full replacement value. According to DOD estimates, in fiscal year 2006, moves that included servicemembers transferring within a theater of operation accounted for about 7,800 personal property moves, or about 1 percent of the more than 680,000 shipments that occurred. DOD officials also stated that in fiscal year 2006 direct procurement method moves represented almost 8 percent of all moves. About 16 percent of moves included nontemporary storage.

In a broader sense, DOD’s backup plan does not address the other goals or counseling benefit of Families First; it is designed only to allow DOD to meet its mandate to provide full replacement value coverage. DOD officials said that they did not have a requirement to produce a backup plan for the other goals or counseling benefit and they did not invest resources to do so. Instead, DOD continues to rely on DPS to achieve the goals and counseling benefit of Families First. The reason Surface Deployment and Distribution Command officials said they created a backup plan for full replacement value is because they were required by statute to implement it by March 1, 2008, whether DPS was ready or not. For example, the backup plan does not address how to provide streamlined claims or improved quality of service from moving companies without DPS, nor does it include a way to provide the other servicemember benefit of expanded Web counseling services to help servicemembers with their moves without DPS. Until DPS is operational, some service officials have said that DOD has at least one other option for providing expanded counseling services because the Navy has a program, known as SmartWebMove, which can be used by members from other services. However, this program is connected to the legacy system, and deterioration of the legacy system may limit the feasibility of this option.

Despite these challenges, Surface Deployment and Distribution Command officials told us that they expect all types of moves will have full replacement value by March 1, 2008. According to these officials, this will include nontemporary storage and direct procurement method moves. DOD continues to rely on the implementation of DPS to achieve other program goals such as improving the quality of service and claims processing.

Families First Program Could Increase DOD Costs by About $1.4 Billion over Current Program Costs through Fiscal Year 2011

The Families First program could increase costs to DOD by about $1.4 billion over current program costs through fiscal year 2011 for two main reasons: (1) DOD estimates the program will increase costs to the services by 13 percent and (2) DOD has significantly increased the cost estimate for a new information management system since our last assessment. DOD’s Families First program has not yet been implemented, so we could not assess the actual growth in costs of the program, although DOD continues to estimate that the Families First program will increase the cost to the services for their household goods budgets by an estimated 13 percent, or as much as $1.2 billion through fiscal year 2011. In addition, the DPS program office has significantly increased the estimated cost of DPS and maintaining the DPS program office, which it now expects to cost $180 million through fiscal year 2011.

Actual Growth in Costs of Families First Program Cannot Be Assessed, Although DOD Continues to Estimate a 13 Percent Increase in Services’ Costs to Implement Families First

We could not assess the actual growth in costs of Families First because the program has not been implemented; however, DOD continues to estimate that the costs to the services of the Families First program will be 13 percent higher than costs under the current program. In fiscal year 2006, the services’ total household goods budget was about $1.8 billion, which would mean the services would have an increase of $240 million annually above the existing budget in order to move servicemembers’ household goods under Families First in 2007, if the program were fully implemented. DOD will incorporate a cost-control mechanism into DPS, similar to the one employed in the current program, in an attempt to keep the costs within the expected increase. However, until DPS is implemented the impact of the use of this mechanism on the Families First program will not be known. Based on DOD’s total household goods budget, Families First could cost DOD about $1.2 billion more than the current program over the next 5 years.

DOD continues to inform the services that the Families First program, when fully implemented, will cost them an additional 13 percent over their existing household goods budgets, which is a subset of the services’ permanent change of station budgets. According to U.S. Transportation Command and some personal property officials, this cost increase is in part because of an expectation by DOD that moving companies will increase the rates they charge as a result of their additional responsibilities under the Families First business rules, such as providing full replacement value. The actual cost of Families First will not be known until moving companies file the rates they will charge DOD to move servicemembers, which is expected to take place in March 2008. DOD is relying on features built into DPS to ensure that the costs remain at or below the expected cost increase of 13 percent. DPS will incorporate a cost-control mechanism known as rate reasonableness, which will establish an acceptable range of rates for each combination of pickup and destination locations.

The program delays in implementing Families First decrease the certainty of the cost estimate because the methodology is based on certain assumptions and data that may change with time. For example, the cost methodology used to estimate the 13 percent increase was adjusted to account for fewer small businesses participating DOD-wide than participated in the pilot programs. However, according to DOD officials, the percentage of small business participation in Families First will be similar to the current DOD participation rate of 70 percent, which is significantly larger than the 30 percent assumed in the 2002 cost estimate. DOD’s evaluation of the pilot programs demonstrated that small businesses were 14 to 74 percent more expensive per shipment compared to the current program. As a result, DOD may have underestimated the cost of having small businesses participate in Families First.

Estimated Costs for the Integrated Information Management System Have Significantly Increased since Our Last Assessment

DOD’s estimated costs for an integrated information management system, known as DPS, have significantly increased since our last assessment in 2003. The estimates for developing an information management system to support Families First have increased from $4 million to $6 million to $86.0 million, and the total cost is expected to be about $180 million through fiscal year 2011 once annual operating costs are added.

In a 2002 report, DOD estimated that implementing the information technology improvements to enhance its data management capabilities for Families First would cost $4 million to $6 million. This estimate was based on expanding the use of an upgraded, Web-based version of the existing legacy system that was implemented on a small scale during one of DOD’s pilot programs. In our April 2003 review of that estimate, we questioned whether DOD would be able to implement its new personal property program, including the technology improvements, within the estimated range. In addition, we noted that although DOD had developed a plan of action for designing the new system, the plan did not include monitoring the costs and benefits during its implementation or the extent to which system changes were being achieved within an acceptable cost range, such as the $4 million to $6 million estimate.

According to DOD officials and documents, in January 2004, DOD decided to implement the technology improvements to support Families First by developing an entirely new information management system, which came to be known as DPS. In a January 2004 report, the Surface Deployment and Distribution Command said that the legacy system evaluated under the previous cost estimate was expensive to operate and maintain and could not be modified to become compliant with DOD technology standards or to support the objectives of the Families First program. At that time, DOD estimated that DPS could be developed for about $16.5 million, with an average annual cost of about $4.6 million after the initial investment. This estimate assumed that DPS could be developed using commercial-off-the-shelf or government-off-the-shelf software to account for about 75 percent of the new system. The use of existing commercial and government software was expected to keep the cost of the system low, because using ready-made software reduces the need to develop original software. For example, the Navy developed a counseling program, known as SmartWebMove, which was originally planned to be incorporated into DPS as its counseling module so that DOD would not need to develop original software as part of DPS to provide this Families First benefit. However, the Navy’s counseling module was not incorporated into DPS. The Navy, however, is still using SmartWebMove while DPS is being developed.

Since the 2004 estimate, the cost of DPS has continued to increase. As of February 2007, DOD reports that it spent $51.5 million developing DPS, which is significantly higher than any previous DOD estimate. This cost includes about $8.2 million for capital hardware, $24.9 million for capital software, and $18.5 million in operating costs. According to DOD Families First officials, after the DPS contract was awarded, software developers determined that DPS would require a much larger percentage of new software development than expected because of the unique needs of the DOD personal property stakeholders, which has caused the cost to rise significantly. In addition, the costs for developing, testing, and making DPS available for use now include the cost of the Joint Program Management Office for Household Goods Systems, which was established on October 1, 2006. The original estimates did not account for a separate program office to manage the development and operation of DPS, sustain the legacy system, or evaluate future options for DOD’s household goods program. Based on our analysis of program office budget planning documents from February 2007, the DPS program office estimated that the costs for maintaining a program office, sustaining the legacy system through retirement, developing and sustaining DPS, and implementing a future household goods program through fiscal year 2011will be $180 million if all of the requirements are funded. Additional delays in the schedule are likely to further increase the costs associated with the program. However, when the legacy system is no longer needed, DOD expects that it will not have to budget for this additional cost, which is about $21 million annually. Summary information on DPS cost estimates appears in table 3.

DOD Faces Management Challenges for Families First Program and Has Not Developed a Comprehensive Implementation Plan

DOD faces management challenges for the Families First program, and it has not employed comprehensive planning that incorporates many sound management principles and practices. Families First offices, including the DPS program office, continue to experience organizational challenges and staffing shortfalls. Moreover, Families First does not have stakeholders’ agreement on some elements of the program, such as business rules and essential DPS functions. Additionally, the Families First program faces uncertain funding.

DOD Planning for Families First Did Not Incorporate Some Sound Management Principles and Practices

Sound management practices require employing comprehensive planning to manage program implementation. Comprehensive planning should include many things, such as integrated approaches to manage training and workforce redeployment issues; a qualified, trained, and well-led team to reengineer the program; stakeholder agreement about key elements of a program, including the program’s business rules and its priorities; and full cost information and funding resources. However, DOD’s planning for Families First has not incorporated some of these sound management practices. Instead, DOD has developed several nonintegrated plans to cover individual portions of the Families First program. For example, DOD has a draft transition plan for organizational changes and the DPS program office has a plan for DPS development. However, DOD does not have a comprehensive plan with timelines for implementing Families First that manages all of its efforts simultaneously. Without an integrated, comprehensive plan, the program’s implementation is at risk.

Families First Faces Significant Organizational Changes and Staffing Shortfalls

The offices supporting Families First, including the program office now overseeing the development and implementation of DPS, are undergoing organizational changes and experiencing staffing shortfalls that affect DOD’s ability to support Families First at a critical time in its implementation. When the first phase of Families First implementation and DPS development began in 2004, the Surface Deployment and Distribution Command, which is under the U.S. Transportation Command, managed all elements of the program. In December 2005, almost 2 years after DPS development started, the Surface Deployment and Distribution Command established a DPS program management office based on a recommendation made by a DOD review group. The review group suggested that the Surface Deployment and Distribution Command establish a clear management structure for DPS because there was no single point of authority and there was no acquisition-certified program manager. On October 1, 2006, the U.S. Transportation Command transferred this office from the Surface Deployment and Distribution Command to the U.S. Transportation Command. The new office, named the Joint Program Management Office for Household Goods Systems, is under the leadership of the U.S. Transportation Command’s Program Executive Office for Distribution Services. The DPS program office and the program executive office are now led by officials with acquisition experience.

The DPS program office has several tasks: mature the program office structure and processes; sustain the legacy system currently being used through the development of its replacement, DPS; quickly implement DPS in phases; and evaluate alternatives for the future of DOD household goods services, including options for outsourcing.

Although DOD’s establishment of the DPS program office addresses some of the concerns about DPS program management raised in DOD’s review, the Joint Program Management Office for Household Goods Systems was not established until a few months prior to a critical phase of DPS development and continues to organize while also facing staffing challenges. The DPS program office had a draft organization chart as of March 2007, but filling staff positions has been complicated by a base realignment and closure move to Scott Air Force Base in Illinois from the office’s current location in Alexandria, Virginia. This move is scheduled to take place in the fourth quarter of fiscal year 2007. The program office’s draft transition plan transfers several positions from the Surface Deployment and Distribution Command to the DPS program office. However, this has created human capital challenges, as many of the staff are choosing to retire or leave rather than move. These workforce planning issues are significantly affecting the DPS program office. According to DPS program management officials, as of April 2007, only 1 of 27 civilians in the program office planned to transfer to Scott Air Force Base. Thus, while Families First and DPS are at a critical stage of development, both the Surface Deployment and Distribution Command and the DPS program office are losing many of their senior leaders who possess technical and program knowledge.

The DPS program office is working to mitigate these human capital planning challenges by seeking authority to hire over current staffing limits, seeking temporary functional support from other Surface Deployment and Distribution Command and U.S. Transportation Command offices, and continuing to seek support from the services and industry as software testers. As of April 2007, hiring actions had been accelerated and some job announcements had been made. The program office is also using contractor support but is facing challenges with this as well. For example, in March 2007 several contractors were not able to complete tasks for the program office because of paperwork processing issues. In addition, in March 2007, the DPS program office asked each of the services to provide two or three full-time servicemembers to continue conducting DPS software testing at the Surface Deployment and Distribution Command headquarters in Alexandria, Virginia. While the services plan to provide some human capital support, current service plans indicate that they cannot provide the servicemember support DPS management officials originally sought because each service will need its staff during the busy, peak moving season that coincides with DPS testing. For example, the Navy is planning to provide five part-time testers at Navy bases. The Army is planning to provide five part-time testers at Army bases. The Marine Corps plans to provide one full-time person to test at program headquarters as well as one support staff member at its testing site at Camp Lejeune in North Carolina. The Air Force is also providing full-time support from its joint personal property shipping office in Colorado Springs, Colorado. Overall, it is not clear how successful these temporary mitigation efforts will be in providing staff with the skills these offices need to implement both DPS and Families First. However, DOD stated that its joint stakeholder advisory team of testers will be sufficient to fulfill the mission required by the DPS program office.

Surface Deployment and Distribution Command officials, who will manage and provide oversight of the current DOD personal property program and implementation of the Families First program, said that they are also facing additional workload and workforce challenges as they administer the electronic billing and payment systems as well as the customer satisfaction survey. These officials are administering these processes without the automation they expect DPS will provide while also experiencing staff reductions and changes as a result of a base realignment and closure move.

As of April 2007, the U.S. Transportation Command has made some progress to staff the DPS program office, but it is not clear how successful its measures will be. Until the U.S. Transportation Command is able to ensure that the DPS program office has adequate and capable human capital resources, it may be unable to successfully implement DPS.

The John Warner National Defense Authorization Act for Fiscal Year 2007 mandated that the Secretary of Defense submit to the congressional defense committees a report containing the certifications of the Secretary on the following matters with respect to the program of the Department of Defense known as Families First: (1) whether there is an alternative to the system under the program that would provide equal or greater capability at a lower cost; (2) whether the estimates on costs, and the anticipated schedule and performance parameters, for the program and system are reasonable; and (3) whether the management structure for the program is adequate to manage and control program costs. When the U.S. Transportation Command established the DPS program office, it included an evaluation of materiel alternatives for the future of household goods services as part of the office’s mission. The mandate did not specify a date for DOD to provide this information to the congressional defense committees. The U.S. Transportation Command is responsible for leading, with the assistance of the DPS program office, the evaluation of alternatives. The DPS program office is responsible for evaluating how to implement the chosen alternative. It is unclear when the DPS program office will be able to evaluate materiel alternatives for the program because (1) U.S. Transportation Command officials told us they were focusing on developing and implementing DPS and (2) the DPS program office has not yet been resourced to evaluate the materiel alternatives.

DOD Does Not Have Stakeholders’ Agreement on Key Elements of Families First

DOD does not have stakeholders’ agreement on some elements of Families First, which puts the implementation of the program at risk. DOD does not have stakeholders’ agreement in two interrelated areas: (1) business rules issues, including whether existing and proposed rules will actually enable accomplishment of a key program goal, and (2) the essential functions needed for DPS.

Uncertainties Surround Families First Business Rules

Stakeholders, including the military services, have not all agreed to some elements of Families First business rules and have not taken action to implement all of the business rules because it is unclear to them if the rules are final. At the end of our audit work, the U.S. Transportation Command was still evaluating whether the business rules would have to be published again for comment by stakeholders. However, in commenting on a draft of this report in May 2007, DOD stated that the business rules are now considered final. Business rules help define how policies are to be implemented. DPS requirements and functions are derived from these business rules. For example, in late March 2007, several months into DPS testing, DOD was still evaluating a business rule as to whether moving companies should have the opportunity to file the rates they charge DOD to move servicemembers’ household goods once or twice per year. In the current program, rates are filed twice per year. Under Families First business rules, moving companies would file rates only once per year. In April 2007, DOD decided to continue with its Families First business rule where moving companies only file rates once per year.

Within DOD, debate continues about whether Families First business rules will allow DOD to accomplish its goal of improving the quality of service from moving companies by using a best-value approach that incorporates performance-based service contracts. Some DOD officials (and industry representatives) question DOD’s proposed practice of allocating business to moving companies using a system where companies that receive less than the best performance score are still allocated business. For example, under Families First rules, moving companies will be ranked into four groups based on their performance scores. Those companies with the best scores will be placed into the first group and receive the most DOD business. However, DOD officials said that even those companies that are in the fourth group, with the lowest performance scores, are expected to receive some business from DOD. According to DOD officials and industry representatives, one reason DOD will do this is to keep providing business to those companies that may not otherwise be able to stay open during the nonpeak moving season. These stakeholders said that this helps ensure that there will be enough capacity during peak moving season. However, some servicemembers’ household goods may be moved by companies that did not receive high performance scores, and therefore they may not receive quality moves. If this is not resolved, DOD may be challenged to meet the program’s goal of improving the quality of service from moving companies.

Additionally, during the course of our work, stakeholders indicated that they did not consider the business rules final. However, DOD, in commenting on a draft of this report, stated that the business rules are now considered final. Stakeholders indicated that they do not yet know what will be expected of them under Families First or what DPS must include to fully support the program. Stakeholders said that the business rules are not considered final until they have been published in the Defense Transportation Regulation. The U.S. Transportation Command published business rules for phase one of the program in the Defense Transportation Regulation on February 20, 2007. However, it has not published business rules for the second and third phases of Families First in the Defense Transportation Regulation. The business rules have only been published on the Surface Deployment and Distribution Command’s Web site and once in the Federal Register so that stakeholders could comment on them. During our review, U.S. Transportation Command officials indicated that DOD planned to publish the business rules again in the Federal Register in June 2007 so that stakeholders could comment on them again and said that DOD would finalize the business rules in July 2007. It is unclear whether DOD still plans to publish the rules again in the Federal Register. Along with the uncertainty surrounding the business rules, stakeholders do not have procedural guidance and do not yet know what is expected of them under Families First. For example, an Air Force personal property official told us the Air Force needs the finalized Families First business rules so that it can train its staff on these new rules, which the personal property official described as being vastly different from the current program’s business rules. However, the Air Force personal property official said the Air Force is hesitant to develop a training curriculum on business rules that are not finalized. Additionally, without final business rules, the services cannot set up internal regulations to support the business rules. Moreover, representatives from the services and the moving industry are concerned that without a formal set of business rules on which to develop DPS, they cannot evaluate whether the computer system fully supports the Families First business rules. Service officials and industry representatives continue to have questions about Families First business rules and DPS implementation.

Finally, the moving companies have concerns about the Families First business rules that define how DOD generates the performance scores used to rank them in the first, second, third, or fourth groups. The majority of a moving company’s performance score comes from a customer satisfaction survey. However, servicemember response rates for the survey have been low (about 16 percent within the past year) and, because of this, most moving companies’ scores are not statistically valid for generating a performance score. Although DOD has, as part of its business rules, devised a methodology to make moving company performance scores valid until survey response rates improve, industry representatives are still concerned that moving companies will be negatively affected by low response rates. In commenting on a draft of this report, DOD stated that while it values the opinion of the moving industry, its personal property program does not require consensus by industry. DOD stated that the main focus of the department is to provide a quality personal property program for servicemembers while being good stewards of taxpayer dollars.

Stakeholders Have Not Agreed to All Essential Functions for DPS

Another fundamental challenge facing DOD in implementing DPS is that stakeholders, such as the military services and the moving industry, have not agreed to all of the essential functions of DPS and how they should operate when DPS is made available to servicemembers to use. Service officials told us that prior to the development of the DPS program office, the Surface Deployment and Distribution Command held many meetings to understand what the services wanted DPS to provide servicemembers and personal property officials. However, service officials said that officials overseeing DPS development at that time did not include all of those requirements when first developing DPS. A Surface Deployment and Distribution Command official said that the contract for DPS was written from a requirements list generated by military service and moving industry stakeholder participation. For example, there was a General Officer Steering Committee, Council of Colonels and Captains, and moving industry stakeholder groups, which met to discuss DPS requirements. In early 2007, after the U.S. Transportation Command took over DPS, stakeholders had their first opportunity to test DPS. During these tests, stakeholders identified functions that they expected within DPS but that did not work the way they expected. This resulted in DPS not providing the functionality service officials expected, and this, in turn, could affect the services’ workloads. A U.S. Transportation Command official said that it is possible that there was miscommunication during earlier meetings to define requirements and that it was not until stakeholders were able to test DPS functionality that these issues were identified. For example, DPS users wanted to obtain the status of a moved shipment. When DPS was programmed, it only displayed whether the shipment was in the system. However, users wanted more detail in terms of where the shipment was at a certain point in time. DPS program management is still in the process of identifying and prioritizing the requirements for DPS, but currently lacks stakeholders’ agreement about all of those requirements and their priority. For example, some stakeholders disagree with the categories assigned to some of the test problem reports, because none of the reports were placed in categories 1 or 2, which are used for the most severe types of problems. Further, the moving industry expected that DPS would interface with their computer systems, but this is not yet part of DPS.

DPS program officials said that earlier phases of DPS development lacked a mechanism for systematically reviewing DPS problems and requirements and identifying how to fix them. The U.S. Transportation Command and the Surface Deployment and Distribution Command formed a Functional Requirements Board to review and prioritize the problems identified during testing that must be fixed and to address other proposed changes to DPS. The Functional Requirements Board is composed of representatives from the services, the Surface Deployment and Distribution Command, and the U.S. Transportation Command and meets monthly to discuss which testing problems should be the highest priority for correcting. The prioritized list of test problems is then reviewed by a Configuration Control Board, which is composed of DPS program managers, service representatives, DPS development contractors, and software engineers who decide which of the DPS problems can be corrected after considering the resources available. As of March 2007, according to DOD, the Functional Requirements Board had developed initial DPS functional requirements, reviewed many proposed system enhancements, and prioritized the services’ top five needs in each DPS module. In addition to stakeholders’ requirements, additional priorities for DPS may also come from the business rules. This, too, could affect the DPS implementation timeline, as well as implementation of Families First.

Another challenge is that without stakeholders’ agreement, DPS requirements continue to change. DPS development is being administered using a firm-fixed-price contract. With a firm-fixed-price contract all major modifications to DPS require negotiation with the contractor, which may lead to additional administrative costs.

Families First Is Not Fully Funded

Even though Families First is projected to cost the services about $1.2 billion over the next 5 years, and DPS is expected to cost about $180 million through fiscal year 2011, the department has not set aside funding to fully cover these costs. The services have taken different approaches in budgeting for the increased costs expected to implement the Families First program, ranging from the Army requesting the entire estimated 13 percent increase to the Navy not requesting any increase at all, in part because they have not received clear guidance from DOD about how to calculate the estimated increase to their budgets. Moreover, the growing cost of DPS has led to funding shortfalls in the DPS program office that are affecting both staffing needs and software development.

Services Vary in the Extent to Which They Have Budgeted for Families First

The services vary in the extent to which they have budgeted for the increased costs expected to implement the Families First program. As previously discussed, DOD estimates that the Families First program will increase the services’ moving budgets by 13 percent above the current budgets needed to move household goods, and DOD has informed the services to budget based on this estimate. However, some personal property officials expressed concerns about DOD’s ability to implement the Families First program within the expected increase of 13 percent; these officials expect that the cost increase will be significantly more. As a result, the services vary in the degree to which they have budgeted for Families First. According to service officials, the services have taken the following actions to budget for Families First: The Army has submitted a budget request for the entire 13 percent increase to the household goods portion of its budget in fiscal years 2008 and 2009.

The Coast Guard requested the 13 percent increase based on its entire permanent change of station budget, of which household goods is just a portion.

The Air Force submitted a budget that included the 13 percent cost increase for Families First in fiscal year 2008, but the Office of the Secretary of Defense did not agree with the Air Force’s budget submission and reduced its funding for permanent change of station moves.

The Marine Corps has requested funding in fiscal years 2007, 2008, and 2009 only for its estimated full replacement value cost.

The Navy has not requested any of the expected 13 percent cost increase.

In addition, some personal property officials stated that they are having difficulty budgeting because they have not received clear guidance about how to calculate the increase. As a result, the services also vary in how they interpret the effect of the 13 percent cost estimate on their household goods budgets. For example, the Air Force estimated its typical annual expected increase in the current household goods program and then added 13 percent. Army officials told us they were unclear whether the household goods program would be increasing by 13 percent every year or just the first year of Families First. A Coast Guard budget official interpreted the 13 percent increase as an increase to just those portions of the budget that apply to the rates charged by moving companies. Neither the Surface Deployment and Distribution Command nor the Office of the Secretary of Defense Comptroller have provided clear guidance on how the services are supposed to apply the 13 percent estimate to their household goods budgets. As a result, the services may continue to apply the 13 percent in different ways, which could result in the program not being fully funded.

According to some service officials, if the expected increase in Families First cost is not included in their budgets and program costs begin to rise as Families First is implemented, then the services may have to consider measures to reduce their household goods budgets. This could affect the number of moves the services can make and could ultimately impact the services’ flexibility in meeting force management needs. Surface Deployment and Distribution Command officials said they plan to monitor the cost of Families First in two ways. First, these officials will use the rate reasonableness methodology to keep the cost at the estimated 13 percent increase. Further, they plan to use reporting functions in DPS to monitor program costs. However, it is unclear how officials will monitor the costs of the program without a fully functioning DPS. Additionally, although the services plan to monitor the actual cost of the program as part of their normal budget processes, there is no plan to provide updated estimates to the services about the cost of the program prior to the services actually incurring the cost. Further, without clear guidance to the services about how to apply the 13 percent cost increase to their permanent change of station budgets, it is unclear whether the services will budget appropriately for the projected Families First cost increase. Without updated information about whether the estimated increase remains accurate as Families First is implemented, the services may not budget for Families First or may budget inaccurately for the program.

Growing DPS Costs Are Creating Funding Shortfalls

As a result of the growing costs of DPS, the DPS program office is experiencing funding shortfalls that are affecting both staffing needs and software development. As of April 2007, the U.S. Transportation Command had not identified how it would fully fund its projected costs of DPS, which it estimated in February 2007 to be about $180 million through fiscal year 2011. Without these funds, the U.S. Transportation Command will be challenged to staff the DPS program office and complete DPS software development. The U.S. Transportation Command has been trying to fund DPS and the DPS program office from its transportation working capital fund.

In fiscal year 2007, the U.S. Transportation Command reallocated about $7.5 million from its transportation working capital fund to support DPS. According to a U.S. Transportation Command budget planning document, this resulted in other U.S. Transportation Command information technology program delays or affected their ability to provide some services. Even with the reallocation, the U.S. Transportation Command had to defer about $9.7 million needed for high-priority software changes and development essential for DPS to fiscal year 2008. The DPS program office has an anticipated shortfall for fiscal year 2008 of $21 million, which includes staff training, contractor support, and funds for staff travel. Travel funds are important since the DPS contractor will be in Virginia and the program staff will be located at Scott Air Force Base in Illinois.

However, the U.S. Transportation Command has not yet developed a detailed budget plan to resource DPS or the DPS program office. The information technology portion of the U.S. Transportation Command’s transportation working capital fund has an annual budget of about $400 million. The DPS program office estimates that it will cost from $28 million to $43 million annually to support DPS and the program office for fiscal years 2008 through 2012. This is from 7 to about 11 percent of the U.S. Transportation Command’s information technology portion of the transportation working capital budget. The U.S. Transportation Command has asked the Office of the Secretary of Defense for $5 million during fiscal year 2008, but the request has not yet been approved. Additionally, U.S. Transportation Command officials said that they have informed the services that they expect them to provide funds for some additional DPS requirements.

DPS program office officials based their current cost estimates for DPS and the program office on the aggressive timeline for implementing DPS before it changed in February 2007. Additional delays in the schedule because of problems developing the software will likely increase the costs associated with the program. At the time of our review, it was too early in the DPS implementation process to determine if the oversight provided by the program office will be effective in keeping DPS costs under control given the ongoing changes to the DPS implementation schedule and the significant number of software changes identified during software testing.

Conclusions

Despite DOD’s recent focus on its personal property program, long- standing problems persist. The department has invested millions of dollars trying to improve the program since the mid-1990s with little real progress. DOD’s Families First program is intended to address many of these long- standing problems but has faced a myriad of management problems and is now at a critical juncture in its implementation. The underlying problem is that DOD has not developed a comprehensive plan to organize, staff, and fund Families First. Until DOD develops a detailed plan to adequately recruit and retain qualified personnel, gain stakeholder agreement about essential requirements for DPS, and set aside resources such as funding and staff, it will be unable to effectively address the challenges to the program. Relying on DPS to achieve program goals—without analyzing alternatives as required by Congress—puts Families First at risk. Moreover, at a time when our nation faces increasing financial constraints and it is increasingly important for DOD to maximize the return on its investment in new systems, DPS costs are continuing to increase while DOD has little to show for its 11 years of reengineering efforts and millions of dollars of investment. Without a reexamination of the program as required by the mandate and urgent attention commensurate with the program’s importance to millions of servicemembers and their families, these problems are likely to continue to negatively affect servicemembers’ quality of life when they are required to move.

Recommendations for Executive Action

We are making the following two recommendations to the Secretary of Defense. To address long-standing problems in DOD’s personal property program we recommend that the Secretary of Defense direct the Commander, U.S. Transportation Command, to expedite the evaluation of the Families First program the John Warner National Defense Authorization Act for Fiscal Year 2007 mandated the department conduct. This act mandates that the report contain the certifications of the Secretary of Defense on the following matters with respect to the Families First program: (1) whether there is an alternative to the system under the program that would provide equal or greater capability at a lower cost; (2) whether the estimates on costs, and the anticipated schedule and performance parameters, for the program and system are reasonable; and (3) whether the management structure for the program is adequate to manage and control program costs.

We also recommend that DOD employ comprehensive planning to implement the Families First program and its associated system. At a minimum, this planning should address specific steps to hire and train personnel so that the Surface Deployment and Distribution Command personal property division and the DPS program office have the human capital needed to develop and implement DPS and reach agreement with stakeholders on the essential requirements for DPS and their priority to facilitate the development of DPS.

In addition, this comprehensive plan should include an investment strategy that reflects the full cost of accomplishing the goals of Families First and milestones for implementation.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD concurred with both of our recommendations and cited specific actions it has taken and will take for each recommendation. We believe DOD’s planned actions, if implemented, have the potential to improve the outcome of the Families First program. However, we also believe it is critical that DOD sustain focus on the program, especially given the delays the program has experienced and the challenges it still faces.

DOD concurred with our recommendation to expedite the evaluation of the Families First program that the John Warner National Defense Authorization Act for Fiscal Year 2007 mandated the department conduct. In its comments, DOD said that it plans to provide this evaluation of its household goods program to Congress in August 2007. We modified our recommendation to include the details of what the act requires.

DOD also concurred with our recommendation that it should employ comprehensive planning to implement the Families First program. In addition, DOD provided technical comments suggesting that we include the Surface Deployment and Distribution Command personal property division as part of this recommendation. We agree, and have modified this recommendation accordingly. Our recommendation now indicates that at a minimum, this planning should include specific steps to hire and train personnel for the Surface Deployment and Distribution Command personal property division and the DPS program office, address specific steps to reach agreement with stakeholders on the essential requirements for DPS and their priority, and include an investment strategy that reflects the full cost of accomplishing the goals of Families First and milestones for implementation.

DOD cited specific actions it has taken or will take to implement this recommendation. For example, DOD said that hiring actions are in progress to staff the DPS program office after its relocation to Scott Air Force Base and that other actions are being implemented to staff the Surface Deployment and Distribution Command. In addition, DOD stated that it has implemented a process to reach agreement with stakeholders on the essential requirements for DPS and their priority by establishing the Functional Requirements Board and the Configuration Control Board. DOD stated that this structure has helped to stabilize the functional requirements for DPS. Finally, DOD stated that the U.S. Transportation Command will provide almost $91 million for DPS development, testing, fielding, and maintenance and that there is a DPS line item in the U.S. Transportation Command’s transportation working capital fund budget. The U.S. Transportation Command is also working with the Office of the Secretary of Defense to provide almost $2.8 million of operating/maintenance dollars from transformation funding. DOD’s comments also state that the U.S. Transportation Command will provide funds internally, if required, to fund DPS and the DPS program office. However, DOD’s comments did not address how the department intends to develop an investment strategy to cover the over $1 billion in increased costs associated with implementing Families First. Developing such a plan for Families First will be critical for the program’s future success.

DOD also provided technical and editorial comments, which we have incorporated as appropriate. DOD’s comments are reprinted in appendix II.

We are sending copies of this report to interested congressional committees; the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; the Commander, U.S. Transportation Command; the Office of the Assistant Deputy Under Secretary of Defense (Transportation Policy); and the Director, Office of Management and Budget. We will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (404) 679-1816 or pendletonj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To evaluate the Department of Defense’s (DOD) Families First program, we obtained documentation from and met with DOD officials and stakeholders in the Washington, D.C., area from the Military Surface Deployment and Distribution Command; the Joint Program Management Office for Household Goods Systems; Transportation and personal property offices of the Army, Navy, Air Force, Marine Corps, and Coast Guard; three moving industry associations, including the American Moving and Storage Association, the Household Goods Forwarders Association of America, and the Military Mobility Coalition; and the Offices of the Secretary of Defense, including Transportation Policy and the Comptroller.

We also met with DOD officials at Scott Air Force Base, Illinois, from the U.S. Transportation Command’s Program Executive Office for Distribution U.S. Transportation Command’s Strategy, Policy, Programs and Logistics U.S. Transportation Command’s Program Analysis and Financial Management Directorate.

To assess the steps DOD has taken to achieve the goals and benefits of the Families First program with or without a new information management system, we identified the goals and benefits of the Families First program by analyzing Families First planning documents and related studies, such as briefings to the U.S. Transportation Command, and verified these goals with personal property officials from the Surface Deployment and Distribution Command. We compared the status of the Families First program to its stated goals by examining Families First implementation timelines provided by program officials and by interviewing officials and stakeholders from the offices listed. We limited our evaluation of the benefits of the Families First program to those benefits that pertain to the servicemembers, specifically full replacement value coverage and expanded counseling services.

We determined DOD’s ability to achieve the goals and benefits of Families First with or without a new information management system by monitoring the status of the Defense Personal Property System (DPS) throughout the course of our review. This included observing demonstrations of DPS as it was presented to representatives from the services and moving industry, reviewing test problem reports generated during DPS software acceptance testing, and examining briefings in which DPS program management and stakeholders reevaluated the DPS implementation schedule. We also reviewed a Federal Register notice provided by Surface Deployment and Distribution Command officials, which described its plans for implementing the full replacement value benefit of Families First without DPS. We analyzed the current program business rules and requirements and compared them to the goals and benefits of Families First to determine if alternatives existed in the current program to implement them without DPS. We also interviewed the officials and stakeholders listed. We asked these officials and stakeholders to provide alternatives for achieving the Families First goals and benefits without DPS. When an alternative was identified, we questioned officials and stakeholders about the feasibility and possible challenges of implementing Families First goals and benefits without DPS.

To evaluate the growth in the cost of the program since our previous assessment, we determined that we could not evaluate the actual growth in costs because data were unavailable as the program has not yet been implemented. However, we determined that DOD was still advising the services to budget for the previously estimated 13 percent cost increase for Families First. To understand this estimated increase, we reviewed the estimated cost of the Families First program from our two previous reports related to the cost of the Families First program. We analyzed the size of the services’ current permanent change of station budgets and assessed how the cost of the Families First program would increase those budgets using information provided by the Office of the Secretary of Defense Comptroller. Families First program management officials stated that DOD would use a cost-control mechanism known as rate reasonableness to ensure that program costs do not exceed the estimated increase. We analyzed Families First business rules and planning documents, such as its concept of operations, to determine the feasibility and limitations of the rate reasonableness approach. We interviewed officials from the Military Surface Deployment and Distribution Command and the U.S. Transportation Command Office for Transportation Policy to determine whether their estimate of the cost of Families First has changed since our previous assessment and what their plans were to keep the cost of the Families First program within the estimate. We also asked officials from the Military Surface Deployment and Distribution Command if they had plans to monitor the costs during implementation of the program. Although we did not independently test the reliability of data DOD extracted from its data system to develop costs or independently verify the analysis used to generate cost estimates, we determined the data were sufficiently reliable for the purposes of our report because they are the same data DOD decision makers use to manage the program.

To assess the growth in the estimated cost of DPS, we reviewed our previous report, which contained DOD’s estimate of the cost of improving its information technology system. We compared this estimate to estimates contained in the U.S. Transportation Command budget planning documents and the DPS economic analysis completed in 2003, which also documented how DOD’s concept for information technology system improvements changed since our last review. To obtain updated information about the current costs of developing and fielding DPS, we analyzed budget documents provided by the Joint Program Management Office for Household Goods Systems as of February 2007. To understand the U.S. Transportation Command’s transportation working capital fund Chief Information Officer Program Review Process, we reviewed related documents, such as the Chief Information Office Program Review Process business flow and interviewed budget officials at the U.S. Transportation Command’s Program Analysis and Financial Management directorate. Although we did not independently test the reliability of data DOD extracted from its data system to develop costs or the analysis used to generate cost estimates, we determined that the data were sufficiently reliable for the purposes of our report because they are the same data DOD decision makers use to manage the program.

To assess the extent to which DOD faces management challenges in implementing the Families First program, we analyzed documents, such as Families First program meeting minutes and management briefings to the General Officer Steering Committee, the Council of Colonels and Captains, and the U.S. Transportation Command, which identified difficulties in implementing the Families First program. We also identified best practices for business reengineering and transformation, which we used to compare the process by which DOD is implementing the Families First program. These best practices were found in prior GAO reports. We also reviewed documents pertaining to the Joint Program Management Office for Household Goods Systems, including the draft organization chart, the draft transition plan for the office’s move to Scott Air Force Base as part of a base realignment and closure move, and draft budget documents for developing and implementing DPS. We also interviewed officials and stakeholders. We did not evaluate DOD’s decision to implement the Families First program or develop DPS, nor did we evaluate the solicitation process for awarding the DPS contract.

We conducted this performance audit from September 2006 through May 2007 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Lawson Gist, Jr., Assistant Director; Krislin Bolling; Renee S. Brown; Michelle Cooper; Arthur L. James, Jr.; Tina M. Kirschbaum; Lonnie McAllister; Maewanda Michael- Jackson; Charles W. Perdue; and Bethann Ritter made key contributions to this report.

Related GAO Products

Defense Transportation: DOD Has Adequately Addressed Congressional Concerns Regarding the Cost of Implementing the New Personal Property Program Initiatives. GAO-05-715R. Washington, D.C.: June 9, 2005.

Defense Transportation: Monitoring Costs and Benefits Needed While Implementing a New Program for Moving Household Goods. GAO-03-367. Washington, D.C.: April 18, 2003.

Defense Transportation: Final Evaluation Plan Is Needed to Assess Alternatives to the Current Personal Property Program. GAO/NSIAD-00- 217R. Washington, D.C.: September 27, 2000.

Defense Transportation: The Army’s Hunter Pilot Project Is Inconclusive but Provides Lessons Learned. GAO/NSIAD-99-129. Washington, D.C.: June 23, 1999.

Defense Transportation: Plan Needed for Evaluating the Navy Personal Property Pilot. GAO/NSIAD-99-138. Washington, D.C.: June 23, 1999.

Defense Transportation: Efforts to Improve DOD’s Personal Property Program. GAO/T-NSIAD-99-106. Washington, D.C.: March 18, 1999.

Defense Transportation: The Army’s Hunter Pilot Project to Outsource Relocation Services. GAO/NSIAD-98-149. Washington, D.C.: June 10, 1998.

Defense Transportation: Reengineering the DOD Personal Property Program. GAO/NSIAD-97-49. Washington, D.C.: November 27, 1996.