United States General Accounting Office

GAO Report to Congressional Committees

June 1998 DEFENSE

TRANSPORTATION

The Army’s Hunter

Pilot Project to

Outsource Relocation

Services

GAO/NSIAD-98-149

GAO United States

General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-278548 

June 10, 1998

Congressional Committees

Because of long-standing concerns and problems associated with the

relocation of military personnel, the Department of the Army began a pilot

project in July 1997 to test an alternative approach for providing relocation

services for its personnel stationed at Hunter Army Airfield, Savannah,

Georgia. The statement of managers in the conference report1

 on the

Department of Defense Appropriations Act, 1997, directed us to validate

the results and savings achieved from this and any other personal property

pilot program. While the conference report originally envisioned us

reporting and evaluating the Army’s experience under this pilot program

during fiscal year 1997, the late start of the pilot in the last quarter of the

fiscal year provided limited data for assessment purposes. Accordingly, we

are providing an interim report. As discussed with your offices, this report

provides (1) information on the data collected to date and (2) our

observations on how the Army plans to evaluate the data. We are also

providing information on other pilot tests that are underway or planned in

the Department of Defense (DOD).

Background DOD has long been concerned about the quality of service it provides

military personnel and their families when they relocate. DOD reported that

in 1996 it paid roughly $2.8 billion to move 800,000 families. Despite the

fact that it moves more household effects than any U.S. corporation, DOD

has found that its system has provided military personnel some of the

worst service in the Nation. DOD has reported that 25 percent of its moves

have resulted in damage claims, compared to 10 percent of the moves

undertaken for the private sector. Further, best-in-class movers have had

customer satisfaction rates of 75 percent, compared with 23 percent for

DOD personnel moves.

The Army recognizes that in today’s environment of limited resources, it

cannot continue to move personal property in the usual way. Currently,

the personal property program is run centrally by the headquarters office

of the Military Traffic Management Command (MTMC) and relies almost

exclusively on more than 1,250 motor van carriers and forwarders to

handle its domestic and international moving needs. The current system is

not specifically designed to select carriers on the basis of quality service;

rather, carriers offering a minimally acceptable level of quality are

1

House Report 104-863, Sept. 28, 1996.

Page 1 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

generally selected based on the lowest rates. DOD offers carriers meeting

the low rate on each route the opportunity to share in any traffic moving

on that route. Further burdening the current system are “paper

companies,” which are affiliates of larger or parent companies established

to increase shares in DOD traffic. These companies increase DOD’s

administrative activities but do not add hauling and storage capacity.

To test whether commercial business practices can help alleviate these

problems and to acquire quality service for its military personnel and

families during the relocation process, the Army began developing a pilot

project in February 1996 to outsource transportation and relocation

services. The company selected for outsourcing was expected to offer

Army personnel and their families a wide range of relocation services. In

addition to packing, transporting, and storing household goods, the

company was to provide other services that included entitlement

counseling; point-to-point move management; help with selling, buying, or

renting a home; and property management and mortgage services.

On January 31, 1997, the Army selected PHH Relocation, a

relocation/move management company, now named Cendant Mobility, of

Bethesda, Maryland, from among 11 offerors as the contractor for its pilot

project.2

 The evaluated price for 3 years—a base year and two 1-year

option periods—was $22.5 million.

The pilot program involves Army personnel stationed at Hunter Army

Airfield moving household goods, unaccompanied baggage, mobile homes,

and boats to other authorized locations, including Georgia, other states in

the continental United States, Alaska, Hawaii, and foreign countries. It

excludes privately owned vehicle shipments and shipments moving into

nontemporary storage. Prior to the pilot project, Hunter was part of the

MTMC-managed program at Fort Stewart, Georgia, although Hunter staff

arranged the moves.

Results in Brief Through the first 6 months of operation, Cendant Mobility, the Hunter

pilot contractor, had received and processed nearly 800 requests for

transportation and relocation services, arranged for transportation of

nearly 600 personal property shipments, and assisted Army members

initiate another 200 Do-It-Yourself moves. Because many of the shipments

had not been completed at the time of our review, the Army had paid for

2

Although the Army awarded the contract on January 31, 1997, a protest to our office delayed the start

of operations. The protest was denied and operations began July 1, 1997. (See decision in Suddath Van

lines, Inc; The Pasha Group, B-274285.2, B-274285.3, May 19, 1997.)

Page 2 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

only about 200 of these shipments, at a cost of about $500,000, but it was

actively collecting data on all the factors—quality of life, cost, and impact

on small businesses—it was planning to use in its evaluation of the pilot.

However, at the time we completed our work, the Army had not

determined what it would consider successful within each factor or how

much weight each factor would have in determining overall success.

Specifically, the Army had not determined criteria to judge the success or

failure of the pilot to help it assess whether the pilot was performing

better or worse than the existing program. Making such determinations

before pilot program data are analyzed would be important to enhancing

the credibility of the Army’s assessment and for use in making

comparisons with other pilot programs that are underway or planned in

DOD. These include pilots underway by the Navy or planned by MTMC. Also,

DOD and the Army are considering expanding the pilot now being tested at

Hunter Army Airfield to other DOD sites.

Subsequently, in response to our draft report, the Army provided us with

new information on how it will determine the results of the pilot. The

information, with some minor exceptions, represents important steps

toward formulating a sound evaluation plan. However, the evaluation

method has not yet been finalized or made a formal part of the evaluation

plan, and questions remain about the definition of terms and small

business group measurements. We recognize that further issues could be

identified as the evaluation method is refined and finalized.

Information Provided

on the Contractor’s

Activity at Hunter

Army Airfield

Through 

December 31, 1997

Contract operations at Hunter Army Airfield began on July 1, 1997. The

contractor, Cendant Mobility, receives the authorizations for relocation

from Army members and arranges to move household goods,

unaccompanied baggage, and mobile homes from Hunter to worldwide

destinations. It also assists a member who may want to make a

Do-It-Yourself personal property move.

Data provided by the Army on the first 6 months of contractor operations

indicated that Cendant had received and processed 793 relocation

authorizations and arranged (booked) 581 moves. These moves included

244 domestic (within the continental United States) household goods

shipments, 227 international household goods shipments, 106 international

unaccompanied baggage shipments, and 4 mobile home shipments.

Cendant also assisted members in initiating 181 Do-It-Yourself moves. (See

table 1.)

Page 3 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

Table 1: Contractor Activity Through December 31, 1997

July

1997

August

1997

September

1997

October

1997

November

1997

December

1997 Total

Orders received authorizing

relocation 218 102 184 100 100 89 793

Shipments booked (total) 112 91 114 89 71 104 581

Domestic household goods

shipments 53 46 28 42 32 43 244

International household goods

shipments 11 26 11 20 15 23 106

International unaccompanied

baggage shipments 46 49 44 26 24 38 227

Mobile homes 2 1 0 1 0 0 4

Do-It-Yourself moves 53 19 34 21 21 33 181

Source: Department of the Army.

The Army’s Plans to

Evaluate the Pilot

Project Need to Be

Further Refined

The Army has developed an evaluation plan to facilitate data collection on

quality of life, total cost, and impact on small business; define agency

responsibilities; and explain which data will be used for comparative

purposes. It has indicated that data will be collected for a minimum of 

12 months. Quality of life is defined primarily in terms of customer

satisfaction, measured principally from customer surveys at both Hunter

and Fort Stewart. Additionally, quality of life will be assessed in terms of

claims for loss and damage based on incidence, amounts, and settlement

times. Total costs include transportation, storage, administration, and

overhead. Transportation and storage costs under the pilot will be

compared with what the Army would have paid under the MTMC program.

Administrative and overhead costs for the pilot will be compared with the

cost of doing business under the MTMC program. Claims costs under the

pilot will be compared with claims costs on shipments made in calendar

year 1995. Impact on small business will be based on the percentage of

moves, tonnage, and dollar amount awarded to small and disadvantaged

businesses under the pilot compared with calendar year 1995 experience.

The Army Audit Agency has been tasked to validate all the data used to

conduct the Army’s evaluation.

Prior to commenting on a draft of this report, the Army had not explained

what constituted success for the pilot, how much weight each factor

(quality of life, total cost, and impact on small business) should have in

Page 4 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

determining overall success, and whether there were thresholds that

specific factors should attain to be considered successful. Such advance

determinations are typically part of a sound study methodology to

enhance the credibility of the results and to avoid any perception of bias.

In response to our draft report, DOD provided us with additional

information it had received from the Army. The information provides

thresholds for determining the success of the overall Hunter pilot and the

relative importance of each evaluation factor. While the Army has not

finalized its evaluation method, the new information clarifies the

description of the three evaluation factors, establishes a point-scoring

system and threshold for each factor and subfactor, and prescribes the

relative weight of each factor for determining the pilot success. We will

continue to evaluate the Army’s evaluation method as it is finalized and

have included the new information in the following sections of this report.3

Evaluating the Factors We recognize that the relationship among the factors—quality of life,

costs, and impact on small business—complicates the process of setting

absolute targets for each factor or for assessing the trade-offs among the

factors. However, the Army may consider certain levels critical for certain

factors (e.g., the budget may limit the amount of increased costs that can

be paid for increased quality of life levels, measured by customer

satisfaction survey and claims experience). On the other hand, there may

also be certain customer satisfaction levels (e.g., those similar to the

commercial market) that the Army expects to obtain. Therefore, making

these kinds of decisions early in the evaluation process should provide the

Army a more definitive basis for determining whether the pilot project is

successful.

Accordingly, the Army has taken action in response to our draft report

recommendation that it determine what results it requires to judge its pilot

a success. The Army’s evaluation plan indicates that overall, quality of life

is the most important of the evaluation factors. The additional information

recently provided suggests that the Army will use a point-scoring system

to determine the relative weight of each factor and subfactor, as shown in

table 2.

3

The full text of the Army’s new information is contained in appendix I.

Page 5 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

Table 2: Evaluation Factors and

Subfactors and Relative Weights

Assigned to Each Evaluation factor/subfactor

Subfactor

weight

Factor

weight

Quality of life 4 points

Customer satisfaction rating 1 point

Claims settlement time 1 point

Percentage of direct deliveries 1 point

Congressional/White House inquiries 1 point

Total cost 1 point

Impact on small business 1 point

Total 6 points

Source: DOD.

If the scores of the pilot total 3.75 or more, the Army will consider the pilot

a success; if below 3.75, or if any of the individual factor scores falls below

its threshold, the Army will consider it unsuccessful.

Quality of Life Quality of life is the key element the Army is considering in assessing the

Hunter pilot. The Army’s most recent information indicates that quality of

life will be based on results of four subfactors—customer satisfaction,

claims settlement time, percentage of direct deliveries, and number of

congressional or White House inquiries concerning dissatisfaction of

Hunter test participants.

Customer satisfaction will be determined by examining responses from

two surveys: (1) Cendant will obtain comments made by all pilot test

members after their moves and (2) an independent contractor will survey

both the pilot test members and those making similar moves at Fort

Stewart under the MTMC program. Responses from both surveys will be

compared. Although the surveys will be quite detailed, the single question

for determining pilot test success will be “ How satisfied are you with your

relocation moving experience?”

The Cendant survey consists of 12 questions, the key question being: “How

satisfied are you with your relocation moving experience?” Responses are

used to determine the amount of “performance” pay that the contractor

can earn in accordance with the contract terms. Data provided to us for

the first 6 months of activity, covering 88 responses where the response

choices were “excellent,” “very good,” “good,” “fair,” and “poor,” showed

that 98 percent of the members had received “excellent,” or “very good,”

Page 6 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

service. None indicated dissatisfaction with their overall relocation

moving experiences. The Army Audit Agency, however, has not yet

completed its validation of those responses.

The independent contractor survey is more extensive and probing than the

Hunter contractor survey, consisting of about 100 questions. For example,

it includes a number of questions on the members’ previous moves and

services used. The key question, however, is: “How satisfied are you with

your relocation moving experience?” When the Hunter participants are at

least 10 percent more satisfied than the control group, which are the

personnel moving from Fort Stewart during the same time as the Hunter

personnel, the customer satisfaction rating subfactor will be deemed

successful. While there are a number of possible responses to this question

(“very satisfied,” “satisfied,” “neither satisfied nor dissatisfied,”

“dissatisfied,” or “very dissatisfied”), it is not clear how “more satisfied”

will be defined.

Results to date are limited because the independent contractor did not

start until October 5, 1997, and it needed time to test and revise the survey

before proceeding with the full survey process. The survey results are

expected to be reported to the Army in October 1998.

The second subfactor related to quality of life is claims settlement time.

This is based on the contract requirement that the contractor offer

settlement to a member within 30 days of claim receipt for damage or lost

property. Consequently, the threshold is 30 days.

Information on claims data is preliminary and may take some time to fully

accumulate since servicemembers may file claims to the Army up to 

2 years from delivery of property. As of December 31, 1997, however, over

500 shipments had been arranged by the contractor, and 33 of the 

40 claims filed by customers against the contractor had been closed. The

average time it took the contractor to settle claims was 13 days, the

average amount claimed was $919.18, and the average amount settled was

$455.15. To date, none of the Hunter servicemembers has exercised his or

her option to file a claim with the Army.

The third subfactor is direct delivery, which is defined as delivering a

shipment to a servicemember’s residence without storage intransit. Direct

delivery is a contract performance measure in which the contractor is paid

an incentive for maintaining a direct delivery rate above 60 percent.

Consequently, the threshold is 60 percent. Of the shipments delivered

Page 7 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

during the first 6 months of contractor operations, 35 percent were direct

deliveries.

The fourth subfactor under quality of life will be the number of

congressional or White House inquiries that the Army Office of Deputy

Chief of Staff for Logistics will receive concerning dissatisfaction of a

Hunter test participant. The established threshold is one inquiry.

Total Cost Total cost is another factor that will be used in the pilot evaluation, and

the carrier industry has expressed concerns that the pilot program could

cost the Army more than it would incur under the MTMC program.

According to the Army, total cost is the sum of (1) transportation, which

includes line-haul transportation, accessorials, and storage prior to

delivery; (2) claims, those costs paid by the Army; and (3) management

cost, which includes pay for performance, management price, and

overhead and administrative expenses. The Army has indicated that the

Hunter pilot will be determined successful if its total cost does not exceed

the baseline cost, which is the cost, including management cost, that the

Army would have incurred had the shipments moved at MTMC program

rates, by more than 30 percent.

Reported pilot costs are still preliminary. To date, the only validated pilot

cost data relate to transportation, storage, accessorial services, claims, pay

for performance, and management fees. Through late January 1998, the

Army paid the contractor $503,835 for 203 moves—150 full service

shipments and 53 Do-It-Yourself moves (see table 3). The costs for these

same shipments and moves, priced at the current MTMC program rates, are

being developed but are still being validated.

Page 8 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

Table 3: Test Program Reported Costs

(Based on Payments Made to the

Contractor Through January 1998) 

Element of cost Amount paid

Transportation and related services

Transportation $461,372

Accessorial services 5,594

Storage 10,913

Claims

Claims paid by the Army 0

Management cost

Pay for performance 3,231

Management fees 22,725

Overhead/Administration costs a

Total $503,835

aData are still being collected and validated by the Army.

Source: Department of the Army.

Developing overhead costs has historically been difficult in government

agencies, including DOD, because such data are often unreliable and

unavailable.4

 The Army, however, is collecting data on administrative or

overhead costs, which include costs for personnel, supplies, claims

processing, voucher processing, inbound shipment processing, electronic

automation, excess cost processing, building overhead, telephone and

copier, and MTMC overhead. Even though overhead and administrative

costs for both the pilot and the comparable MTMC program are being

developed, they have not yet been fully validated by the Army Audit

Agency. The Agency’s responsibilities include providing the Army program

managers with guidance in capturing administrative and overhead costs,

guidance in statistical sampling techniques, and the validation of the test

and baseline data.

Impact on Small Businesses The impact of the Army’s pilot on small businesses has been a source of

concern to the carrier industry. The Army is collecting data on the

percentage of moves, tonnage, and dollar amounts awarded to small and

disadvantaged businesses, including carriers, forwarders, and their agents,

under the pilot program. It will then compare that data with calendar 

year 1995 Hunter data.

4

Defense Outsourcing: Better Data Needed to Support Overhead Rates for A-76 Studies

(GAO/NSIAD-98-62, Feb. 27, 1998).

Page 9 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

The Hunter contractor is a relocation firm who subcontracts or who has

written agreements with various service providers. Some of these

providers are carriers and some are carriers’ agents. The Army has

indicated that it will measure impact on both types of providers and has

established a threshold of 23 percent to consider the pilot small business

participation a success. It is not clear, however, which small business

groups—carriers, agents, or both—on which measurements will be taken

to establish small business participation in the pilot.

As of December 31, 1997, 74 of 533, or 14 percent, of the shipments booked

by Cendant during the first 6 months were awarded to small businesses.5

In comparison, 32 percent of the calendar year 1995 shipments booked by

Hunter under the MTMC program were awarded to small businesses. The

data are still being verified by the Army Audit Agency.

According to Cendant, moving industry opposition to DOD’s reengineering

plans and participation of relocation companies, such as Cendant, in DOD

and civilian government household goods business has had a major impact

on the availability to Cendant of small business subcontractors in the

Hunter pilot. According to Cendant, a number of major van lines have

been reluctant to sign contracts with it for the pilot. Eight of the 15 small

business agents who had been doing business with Hunter are affiliated

with van lines that have not signed Cendant contracts.

In December 1997, Cendant initiated efforts to obtain signed contracts and

broaden the base of potential small business certified agents who could

participate in the pilot. Specifically, on December 8, 1997, Cendant held a

meeting in which 20 officials, representing 15 local moving companies,

expressed an interest in providing relocation services. According to

Cendant officials, several of the local carriers will ask to be considered for

future business and Cendant intends to continue encouraging small

business participation and increase its small business supplier base.

Even as efforts are being made to increase small business participation in

the Army’s pilot, a long-standing issue remains—the lack of verifiable data

establishing company size and ensuring that companies qualify as small

businesses. Historically, such data have been based on carrier/forwarder

self-certification, which MTMC has considered questionable. Under the

MTMC program, company size is established when MTMC initially approves

companies’ participation in the program. MTMC, however, does not verify

5

For the moving and storage industry, small businesses are defined as those with average annual gross

revenues not exceeding $18,500,000.

Page 10 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

the data or require the data to be updated. In addition, MTMC has informed

us that, in its view, some businesses have certified themselves as small,

but, in fact, they rely exclusively on the assets of their parent companies to

perform the actual transportation services. Although these so-called

“paper companies” have certified themselves as small businesses, when

the revenues of their parent companies are considered, MTMC believes

these companies may exceed the small business size threshold. Moreover,

few of the MTMC-approved carriers and forwarders, whether paper

companies or not, are identified in the Small Business Administration data

files. Consequently, carrier/forwarder size determination will be difficult,

and the Army’s ability to compare test data with baseline data could be

questionable.

Other Pilot Projects or

Tests Underway or

Planned

Other pilots or tests are also underway or are being planned in DOD. MTMC,

as DOD’s overall personal property program manager, is contracting for a

pilot covering selected shipments moving from all military bases and

installations throughout North Carolina, South Carolina, and Florida. This

pilot will use one or more companies to handle moves on specific traffic

channels—from one state to a group of states, excluding Hawaii and

Alaska, and to various locations in Europe—under long-term contracts.

Key goals of this pilot are to simplify DOD’s current transportation

acquisition process and to substantially improve and put on par with

corporate customer standards, the quality its military personnel and their

families receive from DOD’s contracted movers. This pilot differs from the

Army Hunter pilot in that it is larger, covering a wider geographical origin

area and more shipments, and involves the relocation of members of all

branches of service. The Army Hunter pilot, on the other hand, is limited

to one location, Hunter, and one service, the Army, although it provides

for move management services, including entitlement counseling and

destination relocation assistance, for all Hunter Army families, relocating

to anywhere in the world. Although the original MTMC solicitation was

issued on March 14, 1997, the start of the MTMC pilot has been

delayed—first, by a disagreement between DOD and the moving industry

over the approach to take for the pilot test and, second, by a series of

protests filed with us concerning the terms and conditions of the

solicitation.6

6

See our report to the congressional defense committees on the dispute over the approach to take

(Defense Transportation: Reengineering the DOD Personal Property Program, GAO/NSIAD-97-49, 

Nov. 27, 1996) and on the solicitation dispute in four decisions (Aalco Forwarding, Inc. et.al.,

B-277241.8, B-277241.9, Oct. 21, 1997; Aalco Forwarding, Inc. et.al., B-277241.12, B-277241.13, Dec. 29,

1997; Aalco Forwarding, Inc. et.al., B-277241.15, Mar. 11, 1998; and Aalco Forwarding, Inc. et.al.,

B-277241.16, Mar. 11, 1998).

Page 11 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

The Navy began a test in January 1998 to use commercial practices at its

personal property shipping office at the Fleet Industrial Supply

Center-Puget Sound, Bremerton, Washington. The initial test is limited to

moves being made to five U.S. destinations. It allows sailors to select from

a list of eligible moving companies the company they would like to move

their property. Like the ongoing Hunter and proposed MTMC pilots, sailors

receive counseling, full replacement value protection for any loss or

damage, and direct claims settlement with the contractor or moving

company.

The U.S. Transportation Command (USTRANSCOM) plans to evaluate the

Hunter pilot and the two other military personal property pilots or tests.

USTRANSCOM will also maintain oversight of the personal property program

pilots and tests and assure consistent evaluation criteria and assessment

of the pilot program results. Upon completion of the pilots and tests, and

in coordination with each service, USTRANSCOM will determine which pilot

or test, or portions thereof, if any, could provide better long-term results

for DOD. It will then recommend the follow-on course of action and

time-lines for implementation throughout DOD, if applicable. We have

previously provided informal comments to USTRANSCOM on their evaluation

plan and will continue to monitor the personal property pilot tests.

Notwithstanding this and other pilots and tests planned or underway, DOD

and Army officials briefed us at the end of this review about plans to

expand the Hunter pilot by testing it at selected DOD sites. Specifically, the

expansion would include a higher shipment volume (between 40,000 and

50,000 shipments per year) and a larger range of services (such as adding

nontemporary and vehicle storage) to determine if the Hunter pilot results

can be successfully duplicated throughout DOD. DOD would like to begin

implementing the test by October 1, 1998. However, to date, all of the

expansion details have not been worked through, such as determining how

the expansion will be managed and evaluated and identifying all of the

installations and services that would participate.

Conclusions The Army is well underway in implementing its pilot project for

outsourcing relocation services at Hunter Army Airfield. It has identified

quality of life, costs, and impact on small businesses as important factors

to be used to determine the success of the pilot. In response to our draft

report recommendation, the Army provided additional information that

states what results it will use to judge the pilot a success. The information,

with some minor exceptions, represents important steps toward

Page 12 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

formulating a sound evaluation plan. However, the evaluation method has

not been finalized or made a formal part of the evaluation plan, and

questions remain about the definition of terms and small business group

measurements. We recognize that further issues could be identified as the

evaluation method is finalized.

In addition, the Hunter pilot is only one initiative among several that will

be tested to determine how DOD can improve the delivery of relocation

services to military personnel. Should DOD decide to expand the Hunter

pilot to include significantly more military customers and services, this

could provide additional information for assessing the concept and for

evaluating results and savings achieved. However, USTRANSCOM, which has

primary responsibility for analyzing overall results and savings achieved,

will ultimately need to assess the relative strengths and/or limitations of

each pilot project to determine how DOD can best improve the personal

property program.

Recommendation We recommend that the Secretary of Defense direct the Secretary of the

Army to finalize its method for evaluating the pilot results to include the

new evaluation information provided in response to our draft report. In

doing so, the Secretary of Defense should direct the Secretary of the Army

to strengthen the evaluation plan by better defining terms and more clearly

defining the small business groups on which measurements will be taken

to establish small business participation in the pilot.

Agency Comments

and Our Evaluation

DOD stated that it concurred, with comment, with the report and

recommendation. Our draft report contained a recommendation that the

Secretary of Defense direct the Secretary of the Army to determine what

results the Army will require to judge its pilot a success, such as specifying

the relative weight of each factor it is measuring and determine if certain

factors must meet certain thresholds, and if so, specify those factors and

levels. In agreeing with our draft recommendation, DOD provided

additional information regarding the Army’s method for determining what

results it plans to use to judge the pilot a success. While we must reserve

conclusions until an evaluation method is finalized, our initial assessment

of the additional information provided indicates that it begins to clarify the

description of the three evaluation factors (quality of life, total cost, and

impact on small business), establishes a point-scoring system and

threshold for each factor and subfactor, and prescribes the relative weight

of each factor for determining the pilot’s success. To the extent this

Page 13 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

information is incorporated into the Army’s evaluation plan, it will provide

a more complete basis for the Army’s analysis. However, our initial

assessment of the information indicates that the Army could strengthen

the evaluation by better defining terms, such as “more satisfied” in

reference to comparing customer satisfaction ratings and by more clearly

defining the small business groups—carriers, agents, or both—on which

measurements will be taken to establish small business participation in the

pilot. Consequently, we have revised our report to discuss the additional

information and modified our recommendation to indicate that the

evaluation method should be finalized and be made a formal part of the

Army’s evaluation plan.

DOD also had comments on three specific areas. First, DOD disagreed with

our references that the Hunter pilot was being expanded “DOD-wide.” DOD

indicated that while expansion of the Hunter pilot is being considered as

part of its total reengineering effort, it has not approved or decided to

expand the program “DOD-wide.” We revised the report to reflect the

current status of DOD’s plans.

Second, DOD stated that characterizing the DOD overhead cost data as often

unreliable and unavailable undermines the entire effort. We acknowledge

DOD’s position; however, the Army Audit Agency’s validation of overhead

data will not be completed for several months. Further, while the

preliminary data represent a full 6 months of performance, the Army has

not developed the same full 6 months of transportation cost data. (The

data are needed to accurately portray the pilot’s cost.) Until the Army has

developed the cost data both for overhead and transportation for the full 

6 months, and the audit validation process is complete, any conclusions

about the total cost of the pilot are premature.

Third, DOD expressed concern that we had not mentioned that greater

small business participation in the Hunter pilot could be realized if the

parent companies of small businesses that previously serviced Hunter

agreed to allow their small business concerns to sign contracts with the

Hunter contractor. We have added that information in our discussion of

the pilot’s impact on small business.

DOD’s comments, along with those prepared by the Army, are included as

appendix I.

Page 14 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

Scope and

Methodology

We asked the Army for data and supporting documents concerning the

first 6 months of activity at Hunter, and the Army provided us that data

and documents on February 2, 1998. We reviewed the data and observed

how it was being collected. We did not independently verify the accuracy

of the data; however, the Army Audit Agency will be validating the data.

To date, sufficient data have not been collected for comparison purposes.

We also met and discussed matters related to the pilot with officials of the

following:

• Office of Deputy Under Secretary of Defense (Logistics-Transportation

Policy), Washington, D.C.;

• USTRANSCOM, Scott Air Force Base, Illinois;

• Office of Army Deputy Chief of Staff for Logistics, Washington, D.C.;

• Hunter Army Airfield, Georgia;

• Army Audit Agency, Hunter Army Airfield, Georgia;

• Fort Stewart, Georgia; and

• MTMC, Falls Church, Virginia.

We reviewed the Army’s plans for evaluating the pilot project and

reviewed the chronology of the pilot from announcement of plan,

presolicitation, solicitation, award, protest, and implementation.

Regarding the management of the pilot project, we visited both Hunter

Airfield and the personal property shipping office at Fort Stewart and

discussed the information being collected by Army officials. We also

reviewed the data prepared by the Army and the Army Audit Agency.

We met with Hunter contractor officials; observed their contract

operations, including the actions of their contracted carrier and agent

activities; and observed a relocation counseling session with a Hunter

soldier. We observed similar activities at Fort Stewart and were briefed or

received documentation on the other planned or ongoing DOD personal

property tests. We also discussed matters related to the Army Hunter pilot

with three agents in Savannah who are on the MTMC-approved list of agents

serving the military in the Fort Stewart/Savannah area.

Our review was conducted between July 1997 and March 1998 in

accordance with generally accepted government auditing standards.

Page 15 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

We are sending copies of this report to the Secretaries of Defense and the

Army; the Commander in Chief, USTRANSCOM; the Department of Army

Deputy Chief of Staff for Logistics; and the Commander, MTMC. We will

also make copies available to others upon request.

Please contact me on (202) 512-8412 if you or your staff have questions

concerning this report. Major contributors to this report are listed in

appendix II.

David R. Warren, Director

Defense Management Issues

Page 16 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
B-278548 

List of Congressional Committees

The Honorable Strom Thurmond

Chairman

The Honorable Carl Levin

Ranking Minority Member

Committee on Armed Services

United States Senate

The Honorable Ted Stevens

Chairman

The Honorable Daniel K. Inouye

Ranking Minority Member

Subcommittee on Defense

Committee on Appropriations

United States Senate

The Honorable Floyd D. Spence

Chairman

The Honorable Ike Skelton

Ranking Minority Member

Committee on National Security

House of Representatives

The Honorable C. W. Bill Young

Chairman

The Honorable John P. Murtha

Ranking Minority Member

Subcommittee on National Security

Committee on Appropriations

House of Representatives

Page 17 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Contents

Letter 1

Appendix I 

Comments From the

Department of

Defense

20

Appendix II 

Major Contributors to

This Report

27

Tables Table 1: Contractor Activity Through December 31, 1997 4

Table 2: Evaluation Factors and Subfactors and Relative Weights

Assigned to Each

6

Table 3: Test Program Reported Costs 9

Abbreviations

DOD Department of Defense

MTMC Military Traffic Management Command

USTRANSCOM U.S. Transportation Command

Page 18 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Page 19 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix I 

Comments From the Department of Defense

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Page 20 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix I 

Comments From the Department of Defense

See comment 1.

Page 21 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix I 

Comments From the Department of Defense

Page 22 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix I 

Comments From the Department of Defense

Page 23 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix I 

Comments From the Department of Defense

See comment 2.

Page 24 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix I 

Comments From the Department of Defense

Page 25 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix I 

Comments From the Department of Defense

The following are GAO’s comments on the Department of Defense’s letter

dated May 21, 1998.

GAO Comments 1. We have included Tab C containing new information that the Army will

use to evaluate the pilot. However, we have not included Tab A and Tab B,

which are administrative and technical comments and were addressed in

the report as appropriate.

2. An Army Hunter pilot program official advised us that the wording in

paragraph 1e (Total Cost) was incorrect and should have indicated that

the Hunter pilot will be determined successful if the total contract-case

cost of the pilot does not exceed the baseline-case cost by more than

30 percent.

Page 26 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Appendix II 

Major Contributors to This Report

National Security and

International Affairs

Division, Washington,

D.C.

Barry W. Holman, Associate Director

Nomi R. Taslitt, Assistant Director

J. Kenneth Brubaker, Evaluator-in-Charge

Barbara L. Wooten, Evaluator

Leo G. Clarke, III, Evaluator

(709237) Page 27 GAO/NSIAD-98-149 Army’s Hunter Pilot Project
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100