Defense Logistics Agency: Process for Selecting Defense Distribution Center Site Contained Weaknesses
Released: 1998-04-09T00:00:00

Letter

Background

Since 1993, DLA has operated with two defense distribution regional headquarters, an eastern headquarters in New Cumberland, Pennsylvania, and a western headquarters in the vicinity of Stockton, California. These regional headquarters provided operational oversight to over 20 geographically dispersed distribution depots. See figure 1 for DLA’s distribution structure prior to the decision to consolidate the regional headquarters.

Actions Resulting From the BRAC Process

DLA reduced its number of distribution regions from three to two in 1993 and soon thereafter began exploring the potential of having just one. Although a study was initiated, it was not finalized and no proposals or recommendations were approved. During the 1995 base realignment and closure (BRAC) process, DLA examined the military value of the eastern and western regional headquarters and found that they rated nearly equal. At that time, DLA officials concluded that changing the command and control structure would present significant risks in the efficient management of day-to-day operations and the ability to effectively support two major regional conflicts simultaneously. Further, they determined that span of control of future operations and the requirement to continue to accommodate contingency, mobilization, and future total force requirements made two regions essential.

Site Selection Process

Subsequent to BRAC 1995, DLA officials revisited the consolidation issue and began another study. According to DLA headquarters officials, a preliminary assessment was made and the study was terminated before any recommendations were made. Meanwhile, DLA continued to restructure its distribution organization. Then, in February 1997, DLA reinitiated an effort to consolidate the two regions. This was expected to result in the creation of a single distribution command, known as the Defense Distribution Center. Under this plan, the center was to assume the regional distribution functions and manage the two primary distribution sites and all remaining distribution depots.

Two steering groups were established to work concurrently on the consolidation effort. A “missions and functions steering group” was established to determine the distribution center staffing requirements and organizational design. A “site selection steering group” (hereafter referred to as the steering group) was established to develop a decision process to determine a recommended site. The Principal Deputy Director of DLA was responsible for selecting the site in consultation with the DLA Director and other senior officials who made up DLA’s executive leadership team.

The site selection process included the use of a contractor, KPMG Peat Marwick, LLP to assist the steering group in developing the decision model and identifying the data needed. The contractor and an experienced DLA facilities engineer gathered, validated, and evaluated the data used in the model.

The steering group had 12 members, with 6 voting members from various offices within DLA headquarters and the Office of the Secretary of Defense; 2 voting members each from the eastern and western regions; and 2 nonvoting members—the steering group chair (a military colonel) and a Department of Defense (DOD) Inspector General representative, who participated as an advisor. The group developed cost and other site selection criteria that were approved by the selecting official and the executive leadership team. (See table 1.)

While costs played a major role in the evaluation, the preferred site did not necessarily have to be the site with the lowest costs. Rather, it was the one with the highest point total.

Three sites were initially considered for the new center: DLA headquarters at Fort Belvoir, Virginia, and the existing eastern and western regional headquarters. The Fort Belvoir site dropped out before completion of the first data request due to a lack of available space at the headquarters building.

Requests for Data

The steering group sponsored two rounds of data requests from officials at the competing sites. According to KPMG, the responses to the first data request were not used because some of the questions were not clear and the respondents did not fully understand them. Thus, a second data request was required. For the second data request, the questions were redefined in an attempt to be clearer and obtain better information. The steering group expected that responses to the second data request would be analyzed using the decision model to identify the preferred site.

Analysis of the second data request was completed on August 11, 1997. Although the steering group was not convened to review the results, the steering group chair and KPMG jointly presented the results to the selecting official (DLA’s Principal Deputy Director), who told us that he was dissatisfied with how some aspects of the approved methodology had been implemented and saw the need for additional data. He requested revisions, including having a second facilities engineer, not previously involved in the process, oversee the collection and validation of some new data—essentially a third data request. The steering group was not made aware of the selecting official’s actions, including the third data request, until the group was given both the results of its work on the second data request and the revisions, on September 15, 1997. On the basis of the results of the revised study, the eastern location was selected by the Principal Deputy Director of DLA as the site for the new center also on September 15, 1997.

Status of the Consolidation

As of October 1, 1997, DLA had officially established its command and control of all distribution functions at New Cumberland, Pennsylvania. As of February 25, 1998, personnel performing distribution headquarters functions at the western location were reporting to management at the eastern location, and DLA was in the process of implementing other aspects of the consolidated operation at that location.

Weaknesses in the Site Selection Process

The process used by DLA to support the site selection decision for its consolidated distribution headquarters contained several weaknesses, including insufficient data on personnel and facilities requirements, a questionable methodology for evaluating and comparing costs, and subjective responses used by steering group members for two criteria. Subsequent changes to the process, made at the request of the selecting official, did not correct these weaknesses and created concerns about the perception of bias. Also, these actions significantly altered investment and operating cost results between the second and third data requests. (See apps. I and II.)

Insufficient Data on Personnel and Facilities Requirements

Because information on staff size and functions was being determined concurrently with the site selection process, the steering group was not given complete information on the staffing requirements, organizational design, and facility requirements for the new headquarters. Because these requirements have a substantial impact on space utilization and costs, it is important that they be properly defined in advance of facility space planning.

The steering group was initially told by senior DLA management that an estimated 400 persons would be needed to staff the consolidated distribution headquarters. Therefore, the first data request asked officials at the competing sites for facility requirements and costs based on the requirements of 400 staff. Subsequently, the missions and functions steering group provided an estimated personnel strength of 347 persons, which was used in the second data request. In both instances, the competing locations were not given a more detailed breakdown of the operational functions or the number of persons associated with them.

Respondents from the regional locations stated that they could only estimate floor plan requirements and then compute associated costs. According to a KPMG official, the floor plans developed by each site contained certain unrealistic aspects and did not present a clear picture of the investment costs that would be required.

Although the structure and functions of the new headquarters were determined prior to the third data request by the mission and functions steering group, DLA officials considered the information too sensitive to release to the competing locations because it could lead to speculation about layoffs. For the third data collection effort, requested by the selecting official, a DLA facilities engineer provided the regional respondents with the number of functions and the number of staff per function, but not the identity of the functions. As a result, the third data request also resulted in hypothetical floor plans and associated costs. Officials at the eastern location told us that, after becoming aware of the final plans for staffing, they made some good guesses in identifying some of the functions and could have implemented their floor plan if required to do so. However, they did not believe the floor plan presented the optimal solution. For example, although they knew they needed a law library, they did not know which function it would be associated with; therefore, their floor plan resulted in locating their legal staff offices at one end of one building and the law library at the opposite end of a second building. Although some changes were to be expected, according to regional officials, the lack of definitive information meant that neither site’s floor plan would have been fully implemented if selected.

DLA officials are still in the process of finalizing the floor plan for their headquarters location in New Cumberland, but the officials said they do not expect the costs to exceed the estimates provided in the site selection competition. Nevertheless, questions still exist regarding what differences might have existed between the plans and costs initially provided by the two competing locations if they had had a clearer picture of the performing functions and space requirements.

Both steering group members and regional respondents said the lack of information on personnel and facilities hampered their ability to perform their tasks. A KPMG official agreed, noting that although he had been involved in numerous site selections, this was the first time he had participated in a selection process in which the functions and related staffing had not been determined before the site selection process began.

Questionable Cost Comparison Methodology

Analysts often assign varying weights to evaluation criteria in this type of analysis to distinguish the relative importance of individual criteria. This approach is also used to assign different weights between cost and noncost variables to distinguish their relative importance. However, DLA’s site selection steering group assigned different weights to individual cost criteria, which produced a distorted picture of the comparative costs of the two competing locations. For example, a dollar spent on a service order became more significant or of more value than a dollar spent on real property maintenance. The distortion caused by this weighting was so significant for the second data request that, even though the eastern location was $3.8 million more expensive than the western location based on a straight comparison of costs, the assignment of points made it appear that the eastern location had come out ahead in the cost categories. (See app. I for the results of the second data request.)

The DOD Inspector General representative who participated in the steering group’s proceedings told us that he had questioned the weighting of individual cost elements and recommended to the group that costs be evaluated on a straight comparison basis between locations. Further, KPMG officials told us that they had also told the group repeatedly that this methodology was not comparing dollars equally. Individual steering group members we spoke with could not recall their rationale for a disproportionate ranking of dollars and did not understand the potential impact of their actions until they saw the results of the second data request.

Subjective Evaluation of Two Noncost Criteria

Specific criteria for evaluating work environment and commute time were not established because the steering group could not determine data sources for these two noncost criteria. Thus, steering group members subjectively determined point values for these criteria at each location. For work environment, members told us they considered everything from distance between the parking lot and the office to perceived lifestyles to ongoing working relationships. For commute time, because some members had been to each location only twice, their experience with commute time consisted of traveling between their hotel and the site. Some members argued that commute time was not really a valid criterion, because it was a matter of personal choice. The subjectivity of these responses and the inconsistency in what members considered made the determinations of questionable value. These two criteria represented 100 points, or half the points awarded in the noncost category and 10 percent of the total points.

Selecting Official Requested Changes Affecting the Analysis

DLA’s site selection decision support model was reviewed and approved by its Principal Deputy Director who served as the selecting official, in consultation with DLA’s executive leadership team, before data collection efforts were initiated to ensure the objectivity of the process. However, after being briefed on the results of the second data request analysis, the selecting official requested changes in the analysis and asked for additional data. He stated that he did this to better ensure the comparability of data between the sites. This produced the requirement for a third data request. The selecting official decided that equal points should be awarded to each site for personnel costs, that real property maintenance costs be reassessed, and that changes should be made to requirements for furniture and space. The selecting official’s actions to negate the impact of one criterion and to have data reassessed after receiving the results of the analysis and without consulting the steering group created concerns among various steering group members about the perception of bias.

Personnel Costs

The selecting official disagreed with the steering group’s use of personnel cost as a criterion. As a result, following a briefing on the second data request analysis, the selecting official, acting independently of the steering group, decided to eliminate personnel costs as a consideration. The selecting official reasoned that the grade structure of the new headquarters should be independent of the chosen site, making personnel costs irrelevant, so he gave equal points to both locations. (See app. II for the results of the third data request.) The steering group, however, had considered personnel cost to be an important criterion and had based it on average grade levels of the current structure at each location. The steering group reasoned that, although the employees of the new site would be downsized, the grade structure at the new headquarters would be similar to that of the region where it was located. DLA headquarters officials told us that in setting up the new distribution headquarters at New Cumberland, they expect to restructure and downgrade positions at that location to meet the requirements for the new organization. To what extent this would lessen the higher personnel costs for the eastern location is unclear given employee bumping rights and save pay provisions that would likely be associated with such restructuring. Also, the importance of personnel costs as a decision factor should not be minimized since savings in this area can mean the potential for significant recurring savings in the long term. The results of the second data request showed that the eastern location had higher average grade levels, resulting in a $3.1 million difference in personnel costs between the two locations over a 5-year period. (See app. I.)

Real Property Maintenance Costs

The selecting official had the facilities engineer responsible for the third data request reassess real property maintenance costs. The selecting official told us that he did this in the interest of obtaining more realistic data. For the second data request, the steering group used the Navy Public Works Center real property maintenance estimates used in the 1995 BRAC process. These estimates had been reviewed in 1995 by the DOD Inspector General, who found the procedures used to be reasonable and the cost estimates to be consistently generated, generally supported, and reasonably accurate. According to the facilities engineer responsible for the second data request, the data had also been used in DLA’s real property maintenance project development, budgeting, and execution processes. The DLA facilities engineer made minor changes to the Navy Public Works Center data as part of his data validation efforts before approving their use for the second data request analysis.

The selecting official told us he believed that the data used in the second data request were not realistic, based on his personal knowledge of the conditions of the two sites, past experience in the distribution area, and knowledge of flaws in repair and maintenance data used for BRAC. He reasoned that the BRAC data were not comparable because the Navy Public Works Center had different people with different perceptions and evaluation criteria assess the individual sites. For example, he said the eastern location’s database included about $47,000 for the cost of painting a building with a dryvit exterior, which, according to the facilities engineer, does not need cyclic painting. However, the facilities engineer had already removed this item from the eastern location’s database during his validation of the second data request.

A DLA facilities engineer, not previously involved in the process, reassessed real property maintenance costs for the third data request, producing a significant change in costs between the two locations. The results of the second data request had shown a difference of about $643,000 in the real property maintenance category in favor of the western location. The results of the third data request produced a difference of about $182,000 in favor of the eastern location.

For the third data request, the facilities engineer went one step further than the previous engineer and had the sites submit justification for removing additional projects from the Navy Public Works Center database. The projects submitted for removal included some cyclical projects that the respondents did not believe were needed during the 5-year time frame covered by the analysis. The eastern location submitted and received approval for removing about $791,000 worth of projects. According to western location officials, they had one such project, valued at $95,000, but they did not submit it because their efforts to have it removed during the second data request were unsuccessful. Both facilities engineers said that if the western location had resubmitted this project for the third data request it would have been considered and may have been removed. The removal of this item alone would not have significantly impacted the cost or point spread in the final analysis of the third data request.

However, the facilities engineers later identified an error of about $210,000 in the western location’s database for costs that KPMG agreed should have been excluded. The correction reducing the western location’s real property maintenance costs was not made in the final analysis because, according to KPMG, it was identified after the third data request analysis was completed. Although these reductions—the possible removal of the $95,000 project and the $210,000 correction—in the western location’s costs would have changed the dollar and point spread advantage for real property maintenance costs from the eastern site to the western site, they would not have been enough alone to effect the overall outcome of the study. (See app. II.)

New Facility Requirements

The selecting official requested revisions to the requirements for furniture and space. These changes significantly affected the relative position of the competing locations within the facilities and information technology cost category. The eastern location’s cost advantage in the investment cost category, which includes facilities and information technology costs, went from only about $19,000 in the second data request analysis to about $1.7 million in the third data request analysis. The facilities engineer who developed the third data request told us that the requirements were instituted to ensure a level playing field. However, some steering group members disagreed with the changes and told us that the requirements gave an advantage to the eastern location, which already had modular systems furniture required by the new data request.

In developing the second data request, the steering group had voted to disregard the selecting official’s direction for the new headquarters to include modular systems furniture. While DLA officials told us that such furniture was used at two other newly renovated sites, various steering committee members told us they did not believe that an official standard requiring such furniture currently existed within DLA. Nonetheless, this became a requirement under the third data request, consistent with the selecting official’s earlier guidance. The facilities engineer who developed the data request required that the competing regions resubmit floor plans to include the systems furniture and stipulated that its life expectancy not exceed 10 years within the 5-year time frame covered by the analysis. As a result of these new requirements, the western location submitted a cost of about $901,000 for purchasing new systems furniture in the third data request because it could not verify the age of the systems furniture stored in its warehouse.

Moreover, in addition to requiring systems furniture, the third data request included other new requirements such as a minimum of 22 meeting rooms and floor-to-ceiling walls for conference rooms and offices to meet what the facilities engineer described as an idealistic view of what DLA offices should look like. The engineer said that he used these requirements to ensure that both proposals would be based on comparable work space. Neither the steering group nor officials from the competing locations agreed with all of these requirements. For example, they protested the need for 22 meeting rooms, calling it excessive and wasteful. Officials at both locations told us that they currently had more people with fewer meeting rooms and had encountered no difficulties in doing their work.

Impact of Changes

The changes made for the third data request analysis had the effect of improving the position of the eastern location, including shifting the cost advantage (on an absolute dollar basis) to the eastern location. However, it should be noted that these costs were largely one-time costs that could easily be offset over time should there be significant recurring savings in another cost area, such as personnel. Steering group members told us they had no role in the third data collection effort. They received the results of the second and third data requests on the same day, September 15, 1997. According to the steering group’s minutes, the group accepted the results of the third data request because the request was at the discretion and authority of the selecting official. However, they cautioned the selecting official that the process of the third data request would appear biased to outside parties, considering they had not been consulted regarding this phase. The results of the third data request analysis showed that the eastern location scored much better in both cost and point totals than the western location—the eastern location was the least costly by about $2.1 million. However, the results of the second data request showed the western location was the least costly by about $3.8 million. Again, because of problems identified in the process, we could not validate either set of data.

No Evidence to Validate Claims of Predetermination

Allegations had been made that DLA officials had selected the eastern site for the Defense Distribution Center before the site selection study took place. We found no evidence to validate concerns that the site selection decision was predetermined. Previous studies examined the consolidation issue but left the two regions intact. We found no evidence that the prior studies influenced the current site selection process or outcome.

DLA officials told us they had considered consolidating their regional distribution headquarters for a number of years and had eliminated one of three regional headquarters in 1993. Subsequently, they had studied options for consolidating the two remaining regional headquarters; however, the study was not finalized and no proposals or recommendations were approved. The issue of consolidating the two regions had also been separately addressed as part of DLA’s BRAC deliberations in 1995. Even though DLA’s BRAC 1995 assessment emphasized the importance of retaining two regions, we learned that following BRAC 1995, DLA officials once again began revisiting the issue and began another study. A DLA official told us that the post-BRAC study was justified because the 1995 BRAC process had produced decisions to close six depots. (See fig. 1.) However, a DLA headquarters official told us that, although a preliminary assessment was made, this study was not completed and no report was issued.

We were provided documents that various officials from the western location said raised concerns about whether the decision had been predetermined. However, we found no evidence to support that the information provided in these documents reflected the official position of DLA or influenced the current site selection process. For example, a 1995 briefing document from a previous study indicated a planned future staffing level of 387 at the eastern location and the phaseout of staff at the western location. According to DLA headquarters officials, the briefing document was preliminary and this study was not finalized.

The selecting official and DLA officials associated with the most recent consolidation study told us that all the previous studies were outdated, given changes in DLA’s structure. Thus, the selecting official said that he did not consider them in the most recent study effort.

Additionally, claims were made by DLA officials at the western location that actions were taken to better position the eastern location in the competition for the consolidated distribution center. These actions included DLA’s moving a general flag officer, its Defense Distribution Systems Center, and the DLA Operations Support Office to the eastern location in 1996. While this may have given the appearance that the eastern site was being preselected, we found no support indicating that this was considered in the site selection process. Moreover, DLA officials told us the flag officer would have moved to the western location if it had been the selected site.

Conclusions

DLA’s efforts to establish a steering group and formulate decision-making criteria indicate that DLA recognized the need for a credible process to guide its decision-making. However, the process used by DLA to support the site selection for its consolidated distribution headquarters contained a number of weaknesses, and raised questions about the soundness of the decision-making process. The evaluation was completed without adequate information concerning facility requirements, which forced an assessment based on hypothetical costs; technical weaknesses further skewed the results. Subsequent changes to the process, requested by the selecting official, did not correct these weaknesses and created concerns about the perception of bias. Additionally, an incomplete assessment of personnel costs minimized opportunities to fully assess the potential for long-term recurring savings.

Although various officials from the western location raised concerns about whether the decision had been predetermined, we found no evidence to validate that the information they provided to us reflected official DLA positions. Also, we found no evidence that prior studies examining the consolidation issue influenced the current site selection process or outcome.

Recommendation

Because of the weaknesses in the process supporting DLA’s site selection decision and subsequent questions raised about the soundness of the decision-making process, we recommend that the Secretary of Defense independently and expeditiously reassess DLA’s site selection decision, taking into consideration issues and questions raised in this report.

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report, and they are included in their entirety in appendix III along with our evaluation of them. DOD nonconcurred with the report’s findings pertaining to (1) insufficient data on personnel and facilities requirements, (2) questionable cost comparison methodology, (3) subjective evaluation of two noncost criteria, and (4) selecting official’s requested changes affecting the analysis.

DOD noted that DLA could have made the site selection decision unilaterally but chose to put a process in place that solicited input from the Office of the Secretary of Defense, the DOD Inspector General, DLA headquarters, and DLA regional experts. It further stated that DLA structured the evaluation process based on other successful models (including BRAC) and adjusted it to accommodate the special considerations felt to be important by representatives of the sites most impacted. We agree with DOD that sound and supportable decision-making processes are needed in making consolidation decisions. Our concern is that the process DLA decided to use was not well implemented. In particular, it contained weaknesses in methodology. The cumulative effect of these weaknesses raised questions about the soundness of the site selection process and the ultimate decision. We believe that the majority of issues raised in DOD’s response were already adequately addressed in our report and, accordingly, we made only minor modifications to the report regarding the requirement for systems furniture.

DOD partially concurred with our recommendation. While disagreeing with the report’s findings, DOD nonetheless agreed with our recommendation that an expeditious review of the site selection decision should be done, taking into account issues and questions raised in this report. However, DOD did not set a time frame for doing so. Also, DOD did not specifically address that portion of our recommendation that stated that the Secretary of Defense should independently conduct the reassessment. We continue to believe it is important that an independent and expeditious assessment be made by the Secretary.

Scope and Methodology

To assess the soundness of the process DLA used to recommend and select a site for the Defense Distribution Center, we reviewed supporting documentation for the criteria, weights, and analysis used in the selection process. We interviewed all participants in the process. Participants included the

Site Selection Steering Group—the steering group chair, an Air Force colonel in the DLA Logistics area; a DOD Inspector General representative; four representatives from DLA headquarters offices, four regional headquarters representatives, including two from each region; and two officials from the Office of the Secretary of Defense, including one from the Comptroller and one from the Logistics offices;

KPMG Peat Marwick, LLP contractor personnel;

DLA facility and installation officials involved in this process; the DLA selecting official, the Principal Deputy Director; the DLA Executive Director, Logistics Management; the Commander and Deputy Commander at Defense Distribution Region West and Defense Distribution Region East, respectively; and

DLA officials from both Defense Distribution Region East and Defense Distribution Region West that responded to data requests.

We visited both sites evaluated in the analysis and reviewed proposed floor plans. We traced and verified selected data inputs used to support DLA’s analysis to verify the reliability of selected DLA and KPMG data validation. We also reviewed documents from BRAC and other DLA consolidation studies, as available, to compare methodologies used. Documentation associated with studies other than the BRAC process was limited.

To address the question of site selection predetermination, we interviewed DLA officials who had participated in or had knowledge of BRAC studies and DLA consolidation studies and reviewed documents relevant to these studies. We also interviewed participants in the Defense Distribution Center site selection process to determine whether they had prior knowledge of these studies. Additionally, to follow up on allegations of predetermination, we spoke to representatives of the DLA Council of American Federation of Government Employees union locals from both the eastern and western regions.

Given the sensitive nature of this assignment, we met with senior DLA officials on two separate occasions to brief them on the results of our work and to solicit their comments on preliminary drafts of this report. We incorporated their comments, as appropriate, to enhance the technical accuracy and completeness of our report.

We conducted our work from October 1997 to February 1998 in accordance with generally accepted government auditing standards.

We are providing copies of this report to the Chairmen and Ranking Minority Members of the Senate Committees on Armed Services and on Appropriations; the House Committees on National Security and on Appropriations; Members of Congress of the affected congressional districts; the Director, Office of Management and Budget; the Secretary of Defense; and the Director, Defense Logistics Agency. We will also make copies available to others upon request.

Please contact me at (202) 512-8412 if you or your staff have any questions concerning this report. Major contributors to this report were Barry W. Holman, James R. Reifsnyder, Kathleen M. Monahan, Jacqueline E. Snead, and Gary W. Ulrich.

Site Selection Steering Group’s Decision Model Results

Operations costs over a 5-year period (discounted)

Both sites were given equal points for the cost of living because comparable data were not available, according to the steering group.

Selecting Official’s Revised Analysis Results

Operations costs over a 5-year period (discounted)

For the third data request the facilities engineer changed the threshold for service orders to clarify definition problems. He requested that the data request respondents capture the costs only for the maintenance and repair projects greater than $2,000.

The selecting official decided to give both sites equal points for personnel costs, because he reasoned that the costs would be identical after formation of the Defense Distribution Center, regardless of the site chosen.

Both sites were given equal points for cost of living because comparable data were not available, according to the steering group.

Comments From the Department of Defense

The following are GAO’s comments on the Department of Defense’s (DOD) letter dated April 6, 1998.

GAO Comments

1. We did not assume that each data request result would be implemented without any changes. Our point is that, while we would not expect the selected site to implement the floor plan exactly as submitted, we do believe the requirements should have been more fully defined and shared with the data request respondents. This was of particular importance in this study because investment costs were a major factor in the site selection criteria. Likewise, better clarity of personnel requirements by function could have led to better estimates of space requirements and cost. 2. We agree that the site selection steering group was given the responsibility to develop the criteria and weights for the decision support model and followed its established process to do so. Although one can assign different weights to costs as compared to a straight cost comparison, it is not a methodology that we have typically seen in such analyses, and as noted in our report, steering group members we spoke with could not recall their rationale for using this approach. Furthermore, both the Defense Logistics Agency’s (DLA) contractor and the DOD Inspector General advised against such a methodology. Varying weights can be assigned to evaluation criteria in this type of analysis to distinguish the relative importance of individual criteria, particularly when distinguishing between cost and noncost variables. However, assigning different weights to individual cost criteria, reduced DLA’s ability to perform the most meaningful comparison of cost. We saw no reason that costs should not have been evaluated on a straight comparison basis to provide a more accurate picture of costs. 3. On the basis of our discussions with steering group members and our review of the site selection backup documentation, we do not believe that the steering group had and used a valid basis for evaluating work environment and commute time. Each steering group member ranked each of the sites between 1 and 10 for quality of work environment and then commute time, with 10 being the most favorable score. Average scores were calculated and the highest average scores received the maximum points in the site selection analysis. The weakness in this method was that the basis for the ranking was not clearly established. Work environment and commute time were not clearly defined; had they been, the steering group might have identified objective measures for assessing these criteria. No data were used, and group members’ knowledge of commute times and working environment was limited. As a result some group members used commutes from nearby hotels and based working environment on personal working relationships. Alternatively, we note that another DLA site selection study, pertaining to the issue of consolidating cataloging functions, used quantitative factors to assess quality of life at work, including factors such as individual office space per person; average commute time measured in average number of miles traveled; availability of public transportation; types and numbers of amenities such as day care, gym, and credit union; parking fees; and distance to the airport. Clearly, the method used in the current study raised questions about the soundness of the method used to evaluate these two criteria. According to steering group meeting minutes, senior DLA leadership expressed concern about the subjective evaluation of these criteria. Additionally, while it is true that these criteria constituted only 10 percent of the total points assigned overall, they constituted 50 percent of the noncost criteria. 4. To what extent time requirements did not allow for consultation with the site selection steering group about the changes requested by the selecting official is unclear. The time constraint—making the site selection decision by the end of the fiscal year—appeared to have been self-imposed. Additionally, we agree that comparable data and information should have been used in the site selection process. However, to avoid questions about the objectivity of the evaluation, standards need to be clearly stated and agreed to up front. The selecting official had approved the site selection decision support model before data collection efforts were initiated to ensure the objectivity of the process. While it is within his authority and discretion to make changes, he did not do so until after he saw the results of the second data request analysis and did so without consulting the steering group. His actions did not correct the weaknesses we identified but resulted in negating personnel costs, reassessing real property, and establishing new facilities standards—the impact of which dramatically altered the resulting costs and point values in the site selection analysis. These actions make it difficult for us to be certain that DLA had the best comparable data it needed for its analysis. 5. We agree that locality pay was not relevant after Fort Belvoir was removed from the site selection process. As DOD stated, the locality pay was the same for both of the remaining sites. However, the steering group was correct in initially identifying personnel costs as an important criterion. The importance of personnel costs should not be minimized since savings in this area can mean the potential for significant recurring savings in the long term. As our report notes, the results of the second data request showed that the eastern location had higher average grade levels, resulting in a $3.1 million difference in personnel costs between the two locations over a 5-year period. While it may be difficult to project bumping rights along with voluntary early retirement and separation incentive pay, it can be done. For example, DLA officials planned to conduct a mock reduction-in-force to determine the effects on personnel, but had not yet done so. It should also be noted that, absent definitive data, DLA and other DOD components previously used standard factors in prior base closure rounds to project some personnel impacts and costs. 6. While we agree that the real property maintenance data should have been comparable, DLA’s site selection backup documentation indicated that the data were reviewed and some modifications were made to it to ensure comparability between the two competing locations before it was used in the second data request analysis. Our concern relates to the decision-making process. The DLA Chief of the Real Property Maintenance Team was approved by the steering group as the facilities engineer responsible for validating the data. He told us that he validated the data as a routine matter of prudent facilities engineering management. During his data validation of the responses to the second data request, he removed the requirement for the eastern site to repaint a nonpaintable exterior surface. Subsequently, the selecting official told us that he based his decision to reassess real property maintenance on his personal knowledge and experience. Having the data reassessed after they had already been validated raised concerns among various steering group members about the perception of bias. 7. DLA officials suggested that not requiring systems furniture and other facilities requirements would result in a substandard work environment and indicated that these requirements were used at two other DLA locations not part of this site selection process. We have modified our report to reflect DLA’s point about these other locations. However, as noted in our report, members of the steering group told us they did not perceive this as an official DLA standard for furniture and workspace.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.