GAO/NSIAD-00-217R Defense Transportation

United States General Accounting Office

Washington, DC 20548

National Security and

International Affairs Division

B-285940

September 27, 2000

Congressional Committees

Subject: Defense Transportation: Final Evaluation Plan Is Needed to Assess

Alternatives to the Current Personal Property Program

The Department of Defense spends about $1.2 billion annually to move the household

goods and unaccompanied baggage of relocating servicemembers and their families.

The Department has long been concerned about the quality of services it provides

military personnel and their families when they relocate. Past problems include poor

service from movers, excessive loss or damage to property, and high claims costs to

the government. All of these problems have contributed to poor quality of service for

people using the personal property program.

Currently, the Department’s Transportation Command is developing an evaluation

plan to assess alternatives to the existing personal property program. The Command

originally expected to have its plan in place by spring 1999 and to begin collecting

data from the current program and two pilot projects no later than January 2000 to

develop recommendations for a new personal property program. These goals have

not been achieved because the Department has not yet concluded what methodology

will be included in the evaluation plan to analyze costs and because one of the pilot

projects will not be ready to be assessed until late February 2001. In the meantime,

the Command decided to begin collecting data in June 2000 for completed sections of

the evaluation plan from the current program and the other pilot project, which is in

its second year of operation. The Transportation Command and the Military Traffic

Management Command, the office administering the ongoing pilot project, have

awarded contracts to obtain assistance in developing the evaluation plan and in

gathering and analyzing data for this year’s assessment. The Military Traffic

Management Command has also awarded contracts to private sector transportation

companies that will be moving the servicemembers’ household goods and

unaccompanied baggage.

This report provides an update on the Transportation Command’s efforts to evaluate

alternatives to the current personal property program. Our objective was to assess

the Command’s decision to begin data collection efforts before the evaluation plan

was completed. Our involvement in this issue stems from the conference report1

 on

 

1

 House Report 104-863 p. 866 (Sept. 28, 1996).
B-285940

2 GAO/NSIAD-00-217R Defense Transportation

the fiscal year 1997 Defense Appropriations Act and a request from the House Armed

Services Subcommittee on Military Readiness after our testimony2

 during its March

18, 1999, hearing on the Department’s personal property program.

RESULTS IN BRIEF

The Transportation Command’s decision to begin collecting data from the ongoing

pilot project and current program in June 2000 was reasonable because (1) the

Command needs to determine this year whether its evaluation plan can be

successfully executed so that any necessary changes can be made before any further

assessments of pilot projects are initiated in 2001; (2) the current evaluation plan,

although incomplete, provides sufficient guidance to allow the Command to begin

collecting time-sensitive and other data required for analysis; and (3) the collection of

data during the high-demand summer moving season in this year’s assessment allows

the Department to begin determining whether its pilot projects can provide the level

of services desired for its servicemembers and their families when they relocate. The

current evaluation plan is incomplete because methodological issues related to how

total costs will be measured remain unresolved. If the Command does not finalize its

evaluation plan and complete its current assessment, it will be unable to meet a

February 15, 2001, internal reporting deadline. This will impact the Command’s

ability to make any needed changes to the plan before conducting its initial

assessment of the second pilot next year and a reassessment, if needed, of the

ongoing pilot before it expires at the end of 2001. If these problems are not

addressed in a timely manner and should assessments continue into 2002, the

Department will need to modify existing contracts or award new contracts. Should

this occur, the Department’s efforts to improve its personal property program will be

further delayed and cost more.

We are recommending that the Department resolve outstanding cost issues this fall

so that the Transportation Command can have a complete evaluation plan in place to

begin assessing alternatives to the current personal property program. In oral

comments provided on a draft of this report, the Department concurred with our

recommendation and agreed to take the necessary actions to complete its evaluation

plan.

BACKGROUND

The Transportation Command is responsible for developing an evaluation plan for

assessing alternatives to the Department of Defense’s personal property program.

This plan will be used to assess two pilot projects. The Reengineered Personal

Property Program, managed by the Military Traffic Management Command, has been

under way since January 1999 and is scheduled to end on December 31, 2001. The

Full Service Moving Project, managed by a project team reporting to the Assistant

Deputy Under Secretary of Defense (Transportation Policy), was scheduled to start in

January 2000 but is currently not expected to be fully operational until the end of

 

2 Defense Transportation: Efforts to Improve DOD’s Personal Property Program (GAO/T-NSIAD-99-106,

Mar. 18, 1999).
B-285940

3 GAO/NSIAD-00-217R Defense Transportation

February 2001. A third pilot project, Navy’s Servicemember Arranged Move, will not

be assessed using the Transportation Command’s evaluation plan. Rather, this pilot

project will be evaluated separately by the Transportation Command using data from

the Navy’s 1999 assessment.3

The evaluation plan includes four assessment factors: quality of life, small business

participation, process improvements, and total costs. The specific criteria for the

first three factors were established by May 2000. The fourth assessment factor,

related to total costs, will consider both direct and indirect costs for the pilot projects

and the current program.4

 Discussions among the Transportation Command and

other Department offices are ongoing to reach agreement on the methodologies for

the cost factor.

The Transportation Command is considering comments from three other offices in its

efforts to complete the cost section of the plan. The Assistant Deputy Under

Secretary of Defense (Transportation Policy) issued guidance to the Command

stating that the evaluation plan and the first-year assessment of the Reengineered

Program pilot project should use the same direct and indirect cost methodologies to

assess the current program. The Command is also reviewing the results of the

Department of Defense Inspector General’s evaluation of the current program’s cost

methodologies that the Military Traffic Management Command plans to use to

evaluate its Reengineered Program pilot project’s first-year operations. The Inspector

General sent its draft report, dated June 30, 2000, to the Transportation Command

and requested that comments on the report’s findings and recommendations be

submitted by August 29, 2000. The Inspector General has agreed to the

Transportation Command’s request for an extension to September 15, 2000, to allow

the Transportation Command additional time to facilitate coordination within the

Department before submitting its formal comments. The third office, the Military

Traffic Management Command, previously concurred with the cost data elements

included in the evaluation plan; however, its Reengineering Program pilot project

team recently raised concerns about the availability of cost data as specified in the

plan.

 

3

 The Reengineered Program pilot project includes about 18,500 moves from installations in North

Carolina, South Carolina, and Florida. The Full Service pilot project will involve about 45,000 moves

from installations in the National Capital Region, Georgia, and North Dakota. Both pilots will include

participants from all military services and the Coast Guard. The Navy’s assessment of its pilot project

included 223 moves from 5 installations between January 1998 and March 1999. While government

personnel administer the current program and two of the pilot projects, the Full Service pilot project

will employ contractors to perform this function.

4

 Direct costs include charges for transportation, storage, and accessorial services (e.g., number of

boxes used and movement of large items such as pianos). Indirect costs include program

management, counseling, quality assurance, and financial transaction processing costs. Direct costs

can be linked to a specific shipment; indirect costs cannot.
B-285940

4 GAO/NSIAD-00-217R Defense Transportation

DECISION TO PROCEED WITH DATA COLLECTION WAS

REASONABLE, BUT EVALUATION PLAN HAS NOT BEEN FINALIZED

Although the Transportation Command has not completed its evaluation plan and

was unable to assess both pilot projects and the current program as projected this

year, its decision to begin collecting data from the ongoing pilot project and the

current program in June 2000 was reasonable. However, unless outstanding cost

issues are resolved this fall, the Command will be unable to finalize the evaluation

plan and complete this year’s assessment to determine whether any changes are

needed to the evaluation plan before any further assessments are conducted in 2001.

Furthermore, should the delay cause assessments to continue beyond 2001, the

Department will need to modify existing contracts or award new contracts.

Decision to Proceed With Data Collection Provides Benefits

The Transportation Command’s decision to begin collecting data from the

Reengineered Program pilot project and the current program in June 2000 was

reasonable for the following reasons:

• The Command needs to collect and analyze data from the ongoing pilot project

and current program this year to determine whether its evaluation plan can be

successfully executed before the initial assessment of the Full Service pilot

project begins next year. Also, if the evaluation plan needs to be revised, the

Command projects it will have sufficient time to make the necessary changes and

reassess the Reengineered Program pilot project before it ends on December 31,

2001.

• The evaluation plan, though incomplete, contains sufficient guidance to allow the

Command to begin collecting data required for the completed sections of the plan,

including quality of life, small business participation, and process improvements.

Of particular significance is the need to conduct time-sensitive surveys of

participants in the Reengineering Program pilot project and of a control group

representing users of the current program for the quality-of-life factor.5

• Data collected this summer will allow the Department to begin assessing whether

its pilot projects can provide the level of services desired for its servicemembers

and their families when they relocate. It is during the summer that the demand

for moving services is at its highest and that the Department competes the most

with other government agencies and private-sector users for quality services from

the moving industry.

 

5

 To assess the quality-of-life factor for the current program, the Transportation Command plans to

survey 1,000 randomly selected servicemembers from geographic locations outside the Reengineered

Program and Full Service pilot project areas.
B-285940

5 GAO/NSIAD-00-217R Defense Transportation

Effects of Further Delays in Completing the Evaluation Plan

If the Department does not resolve the outstanding cost issues this fall, before data

collection efforts end, the Command will be unable to finalize its evaluation plan and

meet its February 15, 2001, deadline for issuing an internal status report on this year’s

assessment of the Reengineered Program pilot project and current program. This will

also impact the Command’s ability to determine whether changes are needed to the

cost and other sections of the plan in time for its initial assessment of the Full Service

pilot project in 2001 and, if required, a reassessment of the Reengineering Program

pilot project. Summer 2001 will be the last opportunity to obtain meaningful data

from the Reengineered Program pilot project before it ends on December 31, 2001.

The Command has identified several steps that it believes must be completed to meet

the February 15, 2001, reporting deadline. Currently, the Command is collecting data

on moves initiated and completed from June 1 through November 30, 2000.

Extending the assessment period from August 31 to November 30 allows for the

inclusion of more overseas moves and shipments involving temporary storage (less

than 60 days). Throughout the assessment period, the Command plans to collect and

analyze data for its report on all four of the assessment factors and will also identify

and correct any problems, to the extent possible, to maintain the integrity of this

year’s execution of the evaluation plan. With the exception of the final quality-of-life

survey data, the Command plans to have most of its data collection and analysis

completed by December 31, 2000. Since the Reengineered Program pilot project will

be given 60 days to compile quality-of-life survey results after each move is

completed, the Command anticipates receiving the final survey data by January 30,

2001.

By February 15, 2001, the Command plans to have all data collection and analysis

tasks completed, issue an internal status report on its assessment of the pilot project

and current program, and identify any remaining deficiencies in its execution of the

evaluation plan. The Command projects that it will then have less than 4 months to

address these deficiencies before any further assessments, including those carried

out in the summer of 2001, could begin. If the remaining deficiencies are not

corrected by that time and should assessments continue into 2002, the Department

will need to modify existing contracts or award new contracts. Any assessments

beyond next year will cause further delays and higher costs for the Department to

continue its efforts to improve the current personal property program.

CONCLUSIONS

The Department’s decision to begin collecting data on the Reengineering Program

pilot project and the current program was reasonable, even though the evaluation

plan was incomplete. However, if the Department does not resolve outstanding

issues on how total costs are to be calculated, the Transportation Command will be

precluded from finalizing its evaluation plan, completing this year’s assessment, and

correcting any remaining deficiencies in the plan before further assessments are

conducted in 2001. Until these problems are addressed, the Command is also

precluded from developing recommendations for an improved personal property
B-285940

6 GAO/NSIAD-00-217R Defense Transportation

program. In the meantime, servicemembers and their families will continue to be

subjected to problems with the current program when they relocate.

RECOMMENDATION FOR EXECUTIVE ACTION

We recommend that the Secretary of Defense direct the Assistant Deputy Under

Secretary of Defense (Transportation Policy), the Transportation Command, and the

Department of Defense Inspector General take the necessary actions to resolve

outstanding cost issues this fall so that the Transportation Command can have a

complete evaluation plan in place to begin assessing alternatives to the current

personal property program.

AGENCY COMMENTS AND OUR EVALUATION

The Assistant for Surface Transportation to the Assistant Deputy Under Secretary of

Defense (Transportation Policy) and representatives from the Transportation

Command and the Department of Defense Inspector General provided oral comments

on a draft of this report. They all concurred with our recommendation and stated

that they plan to take necessary actions so that the U.S. Transportation Command

can have a complete evaluation plan in place this fall to begin assessing alternatives

to the current personal property program. The officials also provided technical

comments, and we revised our report to reflect them where appropriate.

SCOPE AND METHODOLOGY

To assess the Transportation Command’s decision to proceed with data collection for

the Reengineered Program pilot project and the current program, we reviewed the

latest version of the Command’s evaluation plan to determine to what extent it

addresses the four assessment factors: quality of life, small business participation,

process improvements, and total costs. We met with Department officials and their

contractors to discuss prior problems the Department encountered in studying

alternatives to the current personal property program and the ongoing efforts to

address these problems. We discussed lessons learned from our review of the Army

Hunter Pilot Project and the criteria we would use to assess the evaluation plan. 6

 We

also provided the Transportation Command with excerpts of our General

Policies/Procedures and Communications Manual on sampling methodology and

copies of two reference guides,7

 as requested.

We met with officials from the Office of the Assistant Deputy Under Secretary of

Defense (Transportation Policy), Washington, D.C.; the U.S. Transportation

Command, Scott Air Force Base, Illinois; the Military Traffic Management Command,

Alexandria, Virginia; the Department of Defense Inspector General, Full Service

 

6

 Defense Transportation: The Army’s Hunter Pilot Project Is Inconclusive but Provides Lessons

Learned (GAO/NSIAD-99-129, June 23, 1999).

7 Developing and Using Questionnaires (GAO/PEMD-10.1.7, Oct. 1993) and Using Structured

Interviewing Techniques (GAO/PEMD-10.1.5, June 1991).
B-285940

7 GAO/NSIAD-00-217R Defense Transportation

Moving Project, and Hay Group (Transportation Command Contractor), Arlington,

Virginia; and American Management Systems (Transportation Command contractor),

PricewaterhouseCoopers (Military Traffic Management Command contractor), and

Systems Research and Applications (Military Traffic Management Command

contractor), Fairfax, Virginia. We conducted our review from February through

August 2000 in accordance with generally accepted government auditing standards.

- - - - - -

We are sending copies of this report to the Honorable William S. Cohen, Secretary of

Defense; the Honorable Louis Caldera, Secretary of the Army; General Charles T.

Robertson, Jr., Commander in Chief, U.S. Transportation Command; Major General

Kenneth L. Privratsky, Commander, Military Traffic Management Command; and the

Honorable Jacob J. Lew, Director, Office of Management and Budget. We are also

sending copies to interested congressional members. Additional copies will be

available to others upon request.

Please see enclosure I for contacts and key contributors to this assignment.

David R. Warren, Director

Defense Management Issues
B-285940

8 GAO/NSIAD-00-217R Defense Transportation

LIST OF CONGRESSIONAL COMMITTEES

The Honorable John W. Warner

Chairman

The Honorable Carl Levin

Ranking Minority Member

Committee on Armed Services

United States Senate

The Honorable Ted Stevens

Chairman

The Honorable Daniel K. Inouye

Ranking Minority Member

Subcommittee on Defense

Committee on Appropriations

United States Senate

The Honorable Floyd D. Spence

Chairman

The Honorable Ike Skelton

Ranking Minority Member

Committee on Armed Services

House of Representatives

The Honorable Solomon P. Ortiz

Ranking Minority Member

Subcommittee on Military Readiness

Committee on Armed Services

House of Representatives

The Honorable Jerry Lewis

Chairman

The Honorable John P. Murtha

Ranking Minority Member

Subcommittee on Defense

Committee on Appropriations

House of Representatives
ENCLOSURE I ENCLOSURE I

9 GAO/NSIAD-00-217R Defense Transportation

GAO CONTACTS AND STAFF ACKNOWLEDGMENTS

GAO CONTACTS

David R. Warren (202) 512-8412

Charles I. Patton, Jr. (202) 512-8412

ACKNOWLEDGMENTS

In addition to those named above, Lawson Gist, Jr., Robert L. Self,

Jacqueline S. McColl, Marjorie L. Pratt, Arthur L. James, Jr., Molly B. Boyle, and

John G. Brosnan made key contributions to this assignment.
10 GAO/NSIAD-00-217R Defense Transportation

RELATED GAO PRODUCTS

Defense Transportation: The Army’s Hunter Pilot Project to Outsource Relocation Services

(GAO/NSIAD-98-149, June 10, 1998).

Defense Outsourcing: Better Data Needed to Support Overhead Rates for A-76 Studies

(GAO/NSIAD-98-62, Feb. 27, 1998).

Defense Transportation: Reengineering the DOD Personal Property Program

(GAO/NSIAD-97-49, Nov. 27, 1996).

(709473)