Public-Private Competitions: Reasonable Processes Used for Sacramento Depot Maintenance Award
Released: 1999-05-12T00:00:00

Letter

Background

As a result of a 1995 Base Realignment and Closure (BRAC) Act decision, the Sacramento and San Antonio ALCs, including their maintenance depots, are to close by 2001. To mitigate the impact of the closings on the local communities and employees, the administration announced its intention to maintain employment levels by privatizing the maintenance depots’ workloads in place at each location. The Air Force followed by announcing a strategy to privatize in place five prototype depot maintenance work packages at the two closing centers. In response to congressional concerns regarding this strategy, the Air Force decided to use public-private competitions to determine the most cost-effective source of repair for the closing maintenance depots’ work. Appendix II provides a more detailed description of the closure history for both the Sacramento and San Antonio centers.

On March 20, 1998, the Air Force issued a solicitation for the purpose of conducting a public-private competition for various aircraft and commodity depot maintenance workloads being performed at the Sacramento ALC. The Air Force received one private sector proposal from Lockheed Martin Corporation, which had AAI Aerospace Corporation and GEC-Marconi Avionics Incorporated as major subcontractors, and one public sector proposal from the Air Force’s Ogden ALC, which was teamed with Boeing Aerospace Corporation. After performing technical and cost evaluations, on September 21, 1998, the Air Force selected the Ogden ALC’s proposal as the best value to the government. Following resolution of a bid protest filed with our Office, the Air Force proceeded to award the work to Ogden, on October 9, 1998.

Sacramento Air Force Depot Competition Placed No Limitation on Performance Location

Under 10 U.S.C. 2469a(d), a competitor must be allowed to perform at the location of its choosing and is not to be given preferential treatment for, or be limited to, performing the work in place or at any other single location. On the basis of our review of the Air Force’s evaluation and selection documents related to the Sacramento competition, we found no basis to conclude that the procedures did not provide a substantially equal opportunity for the offerors to compete without regard to performance location. For example, while, in its evaluation, the Air Force expressed concerns about the risks inherent in Ogden’s plan to transition the workloads to facilities at San Antonio, Texas, and Ogden, Utah, these concerns were based upon legitimate performance considerations related to the transition plan and did not reflect a bias toward performing the work at the closing Sacramento facility. Appendix III provides the details of our analysis.

Competition Procedures Complied With Applicable Laws and Regulations

Overall, the Air Force’s evaluation and selection of Ogden appeared to be reasonable, fair, and consistent with the solicitation and depot competition procedures. We found no reason to conclude that the competition procedures used in selecting Ogden deviated in a material way from 10 U.S.C. 2469a and other applicable laws and regulations. (See app. III for our detailed analysis.) In assessing the Air Force’s compliance with applicable laws and regulations relating to the competition for Sacramento’s work, we reviewed the Air Force’s evaluation of the proposals and the selection in the context of applicable laws and regulations. This review included examining documents, reviewing processes and procedures, and discussing the competition with Air Force and DOD officials.

Competition Procedures

Pursuant to 10 U.S.C. 2469a and its depot competition procedures, the Air Force issued the solicitation in accordance with the Federal Acquisition Regulation (FAR), part 15, which sets forth the source selection procedures for competitively negotiated acquisitions. The solicitation called for proposals from public and private sector sources for some of the aircraft and commodity work currently being performed at the closing Sacramento ALC at McClellan Air Force Base. The solicitation also provided for award to the public or private competitor that was responsible and whose proposal conformed with the solicitation and represented the best value to the government. The proposals were to be evaluated using transition, operation, and cost criteria; a risk assessment; and other general considerations.

Applicable Laws and Regulations

Several statutes, in particular, 10 U.S.C. 2469a, govern the solicitation and award process for public-private competitions for the depot workloads of the closing Sacramento and San Antonio ALCs. Because the Air Force used the competitive acquisition system, the standards in chapter 137 of title 10 of the United States Code and the FAR apply to the extent they are consistent with 10 U.S.C. 2469a and the other applicable provisions relating to the outsourcing of depot workloads.

Consistent with these standards, the Air Force followed the criteria announced in the solicitation, which in this case included those required by 10 U.S.C. 2469a, and exercised its judgment in a reasonable manner in selecting the successful competitor.

Air Force Considered Factors Other Than Cost

While the competitor selected represented the lowest evaluated cost to the government, the Air Force considered the relative merits of the technical and management approaches of both proposals. For example, the Air Force considered the private competitor’s plan to recruit and maintain the existing work force in place at the Sacramento facility was beneficial. On the other hand, the Air Force concluded that Ogden’s plans for relocating the workloads to San Antonio and Ogden were risky. Thus, for these and other reasons, we found no basis to conclude that factors other than cost were not appropriately considered.

Evaluation Resulted in the Lowest Total Cost to the Government

The Air Force’s award of aircraft and commodity depot maintenance work previously performed at the Sacramento depot was made to the Ogden ALC. The award, which was valued at $1,794,488,861, was made in accordance with the provisions of the solicitation and resulted in the lowest total cost to the government. Overall, the cost evaluation results appear reasonable. However, while not affecting the selection, we do question some of the estimates supporting the evaluation. However, the selection decision would not have been affected by these questions. Questions relate to estimating costs for (1) overhead, (2) commodity rate risk, (3) warehousing, (4) base operating support, and (5) material surcharge.

Cost Evaluation Appears Reasonable

Ogden’s total evaluated cost of $1,794,488,861 for the competed Sacramento depot maintenance workloads was about 6 percent less than Lockheed’s evaluated cost of $1,902,848,080. The Ogden proposal, after cost comparability adjustments provided for in the solicitation and the depot maintenance cost comparability handbook, was determined by the source selection authority to offer the lowest total evaluated cost to the government. Both before and after the cost comparability adjustments, the Ogden proposal was evaluated lower than the private sector proposal.

We examined the accuracy and soundness of the data, assumptions, and methodology supporting a number of these adjustments, including an analysis of the various cost elements in each proposal and the final adjustments made by the cost evaluation team in its proposal analysis report. For our analysis, we selected cost elements having variances of 10 percent or more between the competitors or between amounts contained in the competitors’ final proposals versus the final evaluated cost estimated by the evaluation team. For these cost elements, we (1) discussed with members of the evaluation team, the methodology they used in determining the evaluated cost; (2) reviewed the calculations and supporting documentation for the various cost elements; (3) attempted to independently collect data to corroborate the evaluated cost estimates, where warranted; and (4) offered to discuss competition issues with both the public and private sector competitors. In some instances, our review was limited by a lack of supporting source data. Notwithstanding this limitation, our analysis did not disclose any material weaknesses in the overall cost evaluation. However, as discussed below, in several cases, we identified weaknesses in the evaluated cost estimates.

Cost Estimate Issues

While the overall cost evaluation was reasonable, we question several of the cost estimates. In each case, these questions relate to actions that would have decreased the evaluated cost of the public sector's offer. Therefore, these cases had no impact on the award decision. However, we present them as potential opportunities for improving cost estimates for future competitions. These issues relate primarily to refining cost estimating methodologies and using more accurate data.

Overhead Savings

The Air Force evaluation team reduced Ogden’s projection of overhead savings by 85 percent—from $294.5 million to $46.2 million. The team based the reductions primarily on (1) the Defense Contract Audit Agency’s (DCAA) assessment of Ogden’s overhead savings analysis, (2) its decision to limit the number of years overhead savings would be considered, and (3) its assessment of Boeing Aircraft’s proposed cost savings on the C-17 maintenance program. The first two adjustments were based on conservative assumptions and likely understated the savings between the proposals. Also, in some cases, supporting documentation was lacking or inconsistent approaches to estimating costs were used. Given these issues, we did not attempt to determine a cumulative effect of these adjustments.

Directions regarding the preparation of overhead savings were provided in the Sacramento solicitation. It stated that the evaluation of overhead savings would emphasize a competitor's analysis and documentation of proposed management initiatives to ensure that the projected savings would occur—particularly those predicted for more than 24 months after award. The solicitation evaluation criteria provided that the proposed first-year savings, if determined to be reasonable, would be allowed. The second-year savings, if supportable, would also be allowed but discounted for risk. The solicitation also stated that proposed savings for 3 years and beyond might be allowed if clearly appropriate but would be considered under the best-value analysis.

Ogden used a regression-based methodology to develop its estimate of projected overhead savings that should result from consolidating the commodity and A-10 aircraft work with existing work at Ogden. Ogden based its analysis on 8 years of historical data to capture the relationship between changing workloads and their effect on overhead rates. In its assessment, Ogden normalized the data to reflect cost accounting changes that occurred over the 8-year period.

After reviewing Ogden’s projections, the Air Force evaluation team concluded that the regression methodology was an adequate starting point for projecting future overhead and general and administrative savings. However, they expressed concern about Ogden’s application of this methodology and asked DCAA to evaluate Ogden’s overhead savings analysis. DCAA concurred with the use of the regression methodology but questioned the workload baseline Ogden used in developing the savings estimate. DCAA found that in establishing the baseline, Ogden did not include all the workload that is expected to be transferred into the depot separate from the competition process. The evaluation team reduced Ogden’s proposed overhead savings for the commodity and A-10 workload by a significant amount over the 8-year performance period. The team said they based this reduction on a more realistic projection of Ogden’s baseline due to the transition of workloads transferring separate from the competition. We were unable to reconstruct how this figure was derived because the Air Force did not provide supporting documentation.

After making this adjustment, the evaluation team determined the number of years that overhead savings would be allowed. Team members said they had a general lack of confidence in the regression analysis, the overhead rates, and the application of savings beyond the initial years of the performance period. They said that all overhead fixed costs associated with excess capacity would be eliminated in the long run. Consequently, the team said that production overhead savings would occur only on a prorated basis for the first 3 years. Likewise, the team estimated general and administrative savings for 8 years—with the annual amount prorated progressively beginning with the second year. Taken together these estimates represent the evaluation team's $46.2 million estimate for overhead savings.

The Air Force’s estimate of overhead savings is conservative and is likely understated. We question the Air Force’s assumption that overhead fixed costs associated with excess capacity would be eliminated beginning in the first and second year through reductions in force or attrition. For example, a significant percentage of Ogden’s proposed production overhead cost savings were related to nonpersonnel costs such as facilities and capital equipment, which by their nature are long-term assets and would not likely be eliminated in the evaluators’ estimated time frame. Additionally, the projected organizational structure in the directorates and divisions projected to gain competition work showed that some positions have only one person assigned and that the costs associated with these positions would likely remain fixed for the life of the requirement. Therefore, it appears reasonable to assume that some level of overhead savings relative to these positions would be achieved during the entire performance period.

We noted an inconsistency in how the evaluation team treated Ogden's regression analysis. On the one hand, the evaluation team accepted the proposed overhead costs for the competition workload that had been developed using Ogden’s regression analysis. DCAA officials expressed confidence in this procedure and stated that it provided a reasonable estimate of savings and is applied fairly regularly to commercial firms. On the other hand, when assessing the overhead savings associated with existing workload at the Ogden facility, the evaluation team expressed a lack of confidence in the same regression analysis and, based on these concerns, prorated projected overhead savings associated with this workload. This approach also likely resulted in underestimating overhead savings.

Commodity Rate Risk

Commodity rate risk refers to the uncertainty the evaluation team placed in Ogden’s proposed overhead rates on the commodity workloads. The evaluation team did not question Ogden’s identification of projected overhead costs for the combined competition and noncompetition work, but it was concerned that not enough of the total overhead had been allocated to the competition work. Consequently, it made an adjustment to increase Ogden’s proposed overhead costs for the competed commodity workloads. However, the team did not make a corresponding adjustment to reduce the overhead costs for the noncompetition work to fairly represent the total government cost. Consequently, this resulted in a corresponding overstatement of the noncompetition overhead costs and a corresponding understatement of overhead savings for the noncompetition workloads. While making this corresponding adjustment would not have had a material effect on the selection, it would have increased overhead savings. Ensuring that adjustments of this nature are made correctly is important in future competitions.

Warehouse

While an upward cost adjustment to the Ogden proposal for the warehousing function was appropriate, the method the evaluation team used for making this adjustment could have been more accurate. As a result of the adjustment, the team significantly overstated costs.

The Defense Logistics Agency (DLA) provides the Air Force material storage, warehousing, and issuing and receiving support. DLA accumulates costs for these services and allocates them to its customers based on standard prices that are computed annually for each type of service. The solicitation required that competitors for the Sacramento workloads provide these services as a part of their proposals.

The evaluation team concluded that Ogden’s offer did not represent all the costs associated with the warehousing, storage, and receipt and issuance services. The team estimated that DLA’s full costs to support the competed workload at the closing Sacramento depot would be significantly higher than Ogden’s proposed costs. Air Force evaluators said that while it is possible that Ogden could add the competed work to its existing workload at a marginal cost, they had no basis for estimating the incremental costs of the Ogden warehouse operations. Therefore, they used the full costs of the Sacramento operations.

After considering cost information and discussions with agency officials, we estimated that not more than about half of DLA’s costs for the warehousing function are variable. Consequently, the added warehousing costs were overstated by a significant amount.

Base Operating Support

The evaluation team added a cost comparability adjustment to capture base operating support costs not included in Ogden’s proposal. The methodology used to make this estimate was inappropriate, and as a result, the adjustment was overstated by almost half. “The Defense Depot Maintenance Council Cost Comparability Handbook” provides procedures and techniques to address cost comparability when competing depot maintenance workloads between the public and private sectors. Base operating support cost is one cost category recognized by the handbook. According to the Air Force’s evaluation report, DCAA performed a review of base operating support costs and recommended an adjustment. However, we found no references to base operating support costs in DCAA’s report. Additionally, DCAA officials said that they had not recommended any adjustments in this area.

Evaluation team members said that their adjustment was based on McClellan Air Force Base operating support costs for 1996. They said that they did not collect base operating support cost data for Hill Air Force Base because there should be no difference in the base operating support costs at the two locations. A more appropriate approach would be to use reported costs at the location being evaluated. Based on reported Hill Air Force Base operating cost data for 1996, we estimated that the comparability adjustment should have been about half of the evaluation team’s adjustment.

Material Surcharge

The evaluation team disallowed Ogden’s proposed cost comparability adjustment for a material surcharge. While this action was appropriate for this competition, one technical issue may be important for future competitions. The team said that it disallowed the adjustment because Ogden did not include material in its cost proposal. The more appropriate rationale for disallowing this adjustment was that the cost of government furnished material was added as an adjustment to both proposals. Therefore, there was no need for a comparability adjustment. While this was not a factor for this competition, it may be relevant for future competitions if a private competitor chooses to use contractor furnished material.

Conclusions

The Air Force met the requirements of applicable laws and regulations in the competition for depot maintenance work at the Sacramento ALC. However, the process used for estimating overhead, commodity rate risk, warehousing, base operating support, and material surcharge costs provides issues for the Air Force to consider in its future competitions. Specifically, the evaluation team could have better documented support for certain key cost estimates, followed more appropriate or consistent approaches for estimating costs, and used more accurate or appropriate data.

Agency Comments and Our Evaluation

We provided a draft copy of this report to the Air Force for comment and review for procurement sensitive information. Responsible officials stated that they did not have sufficient time to review and comment on the report. As agreed with the responsible committees, to respond to the congressionally mandated reporting date, we issued prepublication and printed copies of this report with appropriate markings to indicate that the report contained procurement sensitive information that must be safeguarded. Subsequently, Air Force officials identified specific data that they said should be removed from the published report. We have removed the sensitive data identified by the Air Force from this version of the report.

Scope and Methodology

In conducting our work, we obtained information from and interviewed officials at the Air Force Headquarters, Washington, D.C.; the Air Force Materiel Command Headquarters, Wright Patterson Air Force Base, Ohio; the Sacramento ALC, McClellan Air Force Base, California; and the Ogden ALC, Hill Air Force Base, Utah. We also discussed contracting issues with DCAA officials. We offered to discuss the Sacramento competition and award with both the public and private sector competitors; however, because of the pending litigation the Air Force has not provided either competitor with a debriefing. Since the competitors were not familiar with the specifics of the evaluation, they could not provide us with detailed concerns regarding the evaluation process. Particularly, when we contacted the private-sector competitor, its representative stated that discussions of the selection with our Office, without the benefit of a debriefing, would not be productive.

To analyze the Air Force’s decision to award the Sacramento depot maintenance workload to the Ogden ALC, we interviewed officials and collected relevant documents from Headquarters, Department of the Air Force; Headquarters, Air Force Materiel Command; Air Force source selection team members; representatives from the two competitor; and DCAA. To verify compliance with the Sacramento competition and selection with applicable laws and regulations, our Office of the General Counsel performed a legal compliance review. To determine whether cost elements considered in the source selection evaluation were complete and reasonable, we discussed the selection structure with cognizant Air Force and DOD officials, as well as the qualified competitors. We also reviewed the Air Force evaluation team's calculating methods for the various cost estimates for reasonableness and compared the cost elements between competitors to identify material drivers and to further test for reasonableness. We discussed with the evaluation team members their rationale for treating cost elements in the evaluation and in some cases recalculated cost estimates. A list of our related reports is provided at the end of this report.

We performed our review between September and November 1998 in accordance with generally accepted government auditing standards.

We are sending copies of this report to the Honorable William S. Cohen, Secretary of Defense; the Honorable F.W. Peters, Acting Secretary of the Air Force; the Honorable Jacob J. Lew, Director, Office of Management and Budget; and interested congressional committees and members. We will also make copies available to others upon request.

Please contact me at (202) 512-8412 if you or your staff have questions concerning this report. The major contributors to this report are listed in appendix IV.

Summary of Depot Reporting Requirements in the National Defense Authorization Act for Fiscal Year 1998

The National Defense Authorization Act for Fiscal Year 1998 contained the following depot-related reporting requirements for our Office.

I. Report on DOD's Compliance with 50-Percent Limitation (Section 358)

The act amended 10 U.S.C. 2466(a) by increasing from 40 to 50 percent the amount of depot-level maintenance and repair workload funds that the Department of Defense (DOD) can use for contractor performance and revised 10 U.S.C. 2466(e) by requiring the Secretary of Defense to submit to Congress by February 1, 1998, a report identifying the percentage of funds expended for contractor performance.

Within 90 days of DOD’s annual report to Congress, we were required to review DOD’s report and inform Congress whether DOD had complied with the 50-percent limitation.

II. Reports Concerning Public-Private Competitions for the Depot Maintenance Workloads at the Closing San Antonio and Sacramento Air Logistics Centers (Section 359)

The act added section 2469a to title 10 the United States Code to provide for special procedures for public-private competitions concerning the workloads of these two closing depots. It also required us to issue four reports.

First, within 60 days of its enactment, the 1998 Defense Authorization Act required us to review the C-5 aircraft workload competition and subsequent award and report to Congress on whether (1) the procedures used provided an equal opportunity for offerors to compete without regard to performance location, (2) the procedures complied with applicable law and the Federal Acquisition Regulation (FAR), and (3) the award resulted in the lowest total cost to DOD.

Second, the act required the Secretary of Defense to submit a determination to Congress if any of the workloads were bundled in a single solicitation. We were required to report our views on the DOD determination within 30 days.

Third, the act required us to review all DOD solicitations for the workloads at the San Antonio and Sacramento ALCs and report to Congress within 45 days of the solicitations’ issuance whether the solicitations provided “substantially equal” opportunity to compete without regard to performance location and otherwise complied with applicable laws and regulations.

Fourth, the act required us to review all DOD awards for the workloads at the two closing ALCs and report to Congress within 45 days of the contract award whether (1) the procedures used complied with applicable laws and regulations and provided a “substantially equal” opportunity to compete without regard to performance location, (2) “appropriate consideration was given to factors other than cost” in the selection, and (3) the selection resulted in the lowest total cost to DOD for performance of the workloads.

This report addresses the fourth requirement for the award of the Sacramento aircraft and commodity workloads.

San Antonio and Sacramento Air Logistic Centers’ Closure History

The 1995 Base Realignment and Closure (BRAC) Commission recommended closing the Sacramento and San Antonio Air Logistics Centers (ALC) and transferring their workloads to the remaining depots or to private sector commercial activities. In making these recommendations, the Commission considered the effects of the closures on the local communities, on workload transfer costs, and the potential effects on readiness and concluded that the savings and benefits outweighed the drawbacks. The Commission’s report noted that given the significant amount of excess depot capacity and limited DOD resources, closure was a necessity and would increase the use of the remaining centers and substantially reduce DOD operating costs. The specific Commission recommendations were as follows:

Realign Kelly Air Force Base, including the ALC; disestablish the defense distribution depot; consolidate the workloads to other DOD depots or to private sector commercial activities as determined by the Defense Depot Maintenance Council; and move the required equipment and personnel to the receiving locations.

Close McClellan Air Force Base, including the ALC; disestablish the defense distribution depot; move the common-use ground communication electronics to Tobyhanna Army Depot, Pennsylvania; retain the radiation center and make it available for dual use and/or research, or close as appropriate; consolidate the remaining workloads with other DOD depots or private sector commercial activities as determined by the Council; and move the required equipment and any required personnel to receiving locations. All other activities and facilities at the base were to close.

In considering the BRAC recommendations to close the two centers, the President and the Secretary of Defense expressed concerns about the near-term costs and potential effects on local communities and Air Force readiness. In response to these concerns, the President, in forwarding the Commission’s recommendations to Congress, indicated that the ALCs’ work should be privatized in place or in the local communities. He also directed the Secretary of Defense to retain 8,700 jobs at McClellan Air Force Base, which had been recommended for closure, and 16,000 jobs at Kelly Air Force Base, which had been recommended for realignment, until 2001 to further mitigate the closures’ impact on the local communities.

Additionally, the size of the workforce remaining in the Sacramento and San Antonio areas through 2004 was expected to remain above 4,350 and 11,000, respectively.

The Air Force initially focused on privatizing five prototype workloads— three at Sacramento (for hydraulics, electric accessories, and software) and two at San Antonio (for C-5 aircraft paint/depaint and fuel accessories). The Council approved the Air Force’s plans for the five prototype workloads on February 1, 1996. The prototype workloads involved about 11 percent of the San Antonio depot’s maintenance personnel and about 27 percent of the Sacramento depot’s personnel.

Shortly after the Council approved the prototype program, the concept’s appropriateness was questioned. Community and industry groups expressed an interest in having larger packages, and DOD officials were concerned about the cost of administering a large number of smaller contracts. Implementation of the prototype concept was put on hold in May 1996 as the Air Force considered various options. In April 1996, we testified that, if not effectively managed, privatizing depot maintenance activities, including the downsizing of remaining DOD depot infrastructure, could exacerbate existing excess capacity problems and the inefficiencies inherent in underused depot maintenance capacity. Privatizing workloads in place at two closing Air Force depots would not reduce the excess capacity in the remaining depots or the private sector and consequently would not be a cost-effective approach to reducing depot infrastructure. Later that year, we reported that privatizing in place, rather than closing and transferring the depot maintenance workloads at the Sacramento and San Antonio centers, would leave the Air Force with costly excess capacity at its remaining depots that a workload consolidation would mitigate. Our analysis showed that transferring the depot maintenance workloads to other depots could yield additional economy and efficiency savings of over $200 million annually.

We recommended that the Secretary of Defense require the Secretary of the Air Force to take the following actions:

Before privatizing any Sacramento or San Antonio center workload, complete a cost analysis that considers the savings potential of consolidating the two centers’ depot maintenance workloads at other DOD depots, including savings that can be achieved for existing workloads by reducing overhead rates through more efficient capacity utilization of fixed overhead at underused military depots that could receive this workload.

Use competitive procedures, where applicable, for determining the most cost-effective source of repair for workloads at the closing Air Force depots.

In August 1996, the Air Force announced a revised strategy for allocating the depot workloads at the Sacramento and San Antonio centers. The strategy involved several large consolidated work packages, essentially one at Sacramento and two at San Antonio (one for the C-5 aircraft and one for engines). In December 1996, the Air Force issued procedures to conduct public-private competitions for the workloads and to allow one of the remaining public depots to compete with the private sector for each of the three workload packages. The Air Force's procedures allowed evaluation credit for public and private sector proposals that offered overhead savings to other government workloads.

In February 1997, the Air Force issued a request for proposals for the C-5 aircraft depot maintenance workload. In September 1997, the Air Force awarded the C-5 workload to the Warner Robins Air Logistics Center based on the Air Force conclusion that it had the lowest total evaluated cost. As required by the 1998 Defense Authorization Act, we reviewed the C-5 award, issuing our report on January 20, 1998. We concluded that (1) the C-5 competition procedures provided an equal opportunity for public and private offerors to compete without regard to where the work could be performed; (2) the procedures did not appear to deviate in any material respect from the applicable laws or the FAR; and (3) based on Air Force assumptions and conditions at the time of award, the award resulted in the lowest total cost to the government.

On December 19, 1997, DOD submitted to Congress a determination and report to support bundling the engine workloads at the San Antonio depot and a determination and report to support bundling the commodity and aircraft workloads at the Sacramento depot. DOD was required to submit these documents before issuing single solicitations at each location for the combined work. In response to 1998 Authorization Act requirements and subsequent requests from the Senate Committee on Armed Services and the House Committee on National Security, we issued two reports and two testimonies providing our assessment of DOD’s determinations that it was more logical and economical to combine the workloads being competed at the closing depots. We reported that: the determination and reports contained significant weaknesses in logic, assumptions, and data;

DOD had not considered alternatives that appeared to be logical and

DOD’s assumption that efficiencies from shared personnel and facilities would be best achieved with a single solicitation for combined workloads at each location was questionable; and the Air Force’s conclusion from its cost analysis that the workload combination would save $22 million to $130 million at Sacramento and $92 million to $259 million at San Antonio was questionable because the Air Force did not consider all cost factors, such as the cost benefits of increased competition resulting from solicitations for individual workloads.

On March 20, 1998, the Air Force issued a solicitation for the combined aircraft and commodity workloads at the Sacramento depot and on March 30, 1998, issued a solicitation for the combined engine workloads at the San Antonio depot. We issued our required report on the Sacramento solicitation on May 4, 1998. We concluded that the Air Force had not provided a sufficient basis to show that soliciting the workloads on a combined basis was necessary to satisfy its needs. Otherwise, we found that the solicitation complied with applicable laws, including 10 U.S.C. 2469a. On May 14, 1998, we issued our report on the San Antonio solicitation, similarly concluding that the Air Force had not provided a sufficient basis to show that soliciting the workloads on a combined basis was necessary to satisfy its needs but that otherwise the solicitation complied with applicable laws, including 10 U.S.C. 2469a.

Legal Review of Competition for Sacramento Air Logistics Center Workloads

On March 20, 1998, the Department of the Air Force, Sacramento ALC at McClellan Air Force Base, California issued requests for proposal (RFP) No. F04606-98-R-0007 for the purpose of conducting a public-private competition for the depot-level workloads being performed at the closing Sacramento ALC. The Air Force received proposals from one private sector offeror—Lockheed Martin Corporation (Lockheed)—and from one public offeror—the Air Force’s Ogden ALC. Following technical and cost evaluations, the Air Force selected Ogden ALC to perform the Sacramento workloads on the basis that its proposal represented the best value to the government. The Ogden ALC proposal also represented the lowest “most probable total evaluated” cost at $1,794,488,861 over the 9-year requirement.

Section 359 of the National Defense Authorization Act for Fiscal Year 1998, Public Law 105-85 (1998 Authorization Act), added section 2469a to title 10 of the United States Code, which provided for special procedures for public-private competitions for the workloads at the closing Sacramento and San Antonio ALCs. Section 2469a also requires us to review the selection process for the awards made for the workloads at the two closing ALCs and report to Congress within 45 days of each award on whether (1) the procedures used to conduct the competition provided a substantially equal opportunity for offerors to compete without regard to performance location and complied with 10 U.S.C. 2469a and all applicable laws and regulations, (2) appropriate consideration was given to factors other than cost in the selection, and (3) the award resulted in the lowest total cost to the DOD for the performance of the workloads.

Our review is based on the record of the proposal evaluation and the selection. In addition, we spoke to Air Force officials and considered concerns raised informally by one of the competitors. We recognize that an offeror may file a protest with our Office pursuant to 31 U.S.C. 3551-3556, or with the courts, or may file an objection to the award with DOD under 10 U.S.C. 2469a(h). If a protest is filed, factual information, issues, and arguments raised by the interested parties will be reviewed in the context of an adversarial process. For that reason, the result of a protest may differ from that of our current review. Similarly, the result of an objection filed with DOD may differ from our review.

Based on our review of the procedures the Air Force used to conduct the Sacramento competition in context of the concerns that were raised by the competitor, we found no basis to conclude that (1) the procedures did not provide a substantially equal opportunity for the offerors to compete without regard to performance location, (2) appropriate consideration was not given to factors other than cost in the selection, and (3) the procedures used in selecting the successful offeror deviated in any material respect from the applicable laws and regulations. While not affecting the legal sufficiency of the selection, we nevertheless identified several issues related to the estimates supporting the cost evaluation. These issues are discussed in the body of the report.

In an earlier review of the Sacramento solicitation, we concluded that the Air Force had not provided a sufficient basis to show that soliciting the workloads on a combined basis was necessary to satisfy its needs. We also concluded that the solicitation was otherwise in compliance with applicable laws, including the provisions of 10 U.S.C. 2469a, and that it provided a substantially equal opportunity for offerors to compete without regard to performance location.

On June 17, 1998, Pemco Aeroplex, Inc. (Pemco) filed a protest of the solicitation’s provisions with our Office pursuant to 31 U.S.C. 3551-3556. Pemco objected to the solicitation of the workloads on a combined basis. In a decision dated September 25, 1998, our Office sustained the protest, concluding that the Air Force was unable to show that combining the requirements was reasonably required to satisfy its needs and recommending that the agency cancel the solicitation and resolicit its requirements without combining the workloads. On October 9, the Air Force decided to proceed with the award to Ogden ALC notwithstanding the protest recommendation. On October 13, Pemco filed civil action no. Cv. 98-B-2584-S in the United States District Court for the Northern District of Alabama, Southern Division, seeking a declaration that the award is void and an injunction preventing the Air Force from going forward with performance.

The following describes the legal standards applicable to the Sacramento competition, relevant aspects of the solicitation and evaluation procedures the Air Force used, and our analysis of those procedures under the applicable legal standards.

Applicable Legal Standards

The basic authority for the Sacramento workload competition is 10 U.S.C. 2469a, which provides procedures for public-private competitions for the workloads of the closing Sacramento and San Antonio ALCs that are proposed to be outsourced after the November 18, 1997, enactment of the 1998 Authorization Act. Section 2469a sets forth a number of requirements that the Air Force must satisfy in the solicitations it issues and the source selection process it uses, to make awards for the specified workloads. Particularly, the solicitation and the source selection process must (1) permit both public and private offerors to submit offers; (2) take into account the fair market value of any land, plant, or equipment at a closed or realigned military installation that is proposed to be used by a private offeror in the performance of the workload; (3) take into account the total estimated direct and indirect costs that will be incurred by DOD and the total estimated direct and indirect savings (including overhead) that will be derived by DOD; (4) use cost standards to determine the depreciation of facilities and equipment that provide, to the maximum extent practicable, identical treatment to public and private offerors; (5) permit any offeror, whether public or private, to team with any other public or private entity to perform the workload at any location or locations of their choosing; and (6) ensure that no offeror may be given any preferential consideration for, or in any way be limited to, performing the workload in place or at any other single location.

In addition to 10 U.S.C. 2469a, there are a number of other laws that are generally applicable to the outsourcing of government-performed depot workloads. One of the principal laws is 10 U.S.C. 2469, which provides for the use of “competitive procedures for competitions among private and public sector entities” when DOD contemplates changing the performance of a depot workload, valued at $3 million or more, to contractor performance. In addition, section 8039 of the Department of Defense Appropriations Act for Fiscal Year 1998, Public Law 105-56, authorizes public-private competitions for depot workloads as long as the “successful bids" are certified to "include comparable estimates of all direct and indirect costs for both public and private bids.” Both provisions state that Office of Management and Budget Circular A-76 is not to apply to the competitions. Other than the reference in section 8039 to the use of comparable estimates of all costs, neither provision prescribes the elements that constitute a competition. Further, 10 U.S.C. 2470 provides that depot-level activities are eligible to compete for depot workloads.

There are other provisions that apply, generally, to converting DOD functions to private-sector performance. Section 8014 of the 1998 DOD Appropriations Act requires that DOD certify its in-house estimate to congressional committees before converting any activity performed by more than 10 civilian employees to contractor performance; the provisions of 10 U.S.C. 2461 require that whenever a DOD-performed function, such as the workloads involved in this competition, is converted to performance by a contractor, DOD must provide to Congress a cost comparison that shows that a savings will result. Under 10 U.S.C. 2462, DOD is generally required to contract with the private sector if a source can provide the supply or service at a lower cost than DOD can and to ensure that all costs considered are realistic and fair.

The Air Force implements these outsourcing authorities through the Air Force Materiel Command’s Procedures for Depot Level Public-Private Competition, December 20, 1996 (Depot Competition Procedures). The procedures are supplemented by the “Defense Depot Maintenance Council Cost Comparability Handbook” (CCH), including the January 28, 1998, revision, the Air Force Materiel Command “Guide to the Cost Comparability Handbook” and the SAF/AQ Public-Private Competition Cost Procedures of February 21, 1998. The procedures provide for issuing a solicitation calling for offers from public and private sector sources and establish the criteria, including those listed in 10 U.S.C. 2469a, for deciding how the Air Force will select a source from either sector to perform depot workloads. According to these procedures, a competitive solicitation is to be issued in accordance with the applicable provisions of the FAR, which set forth uniform policies and procedures for the competitive acquisition system that all executive agencies use and implements the provisions of chapter 137 of title 10 of the United States Code, which govern DOD acquisitions.

This use of the competitive acquisition system subjects a depot workload competition to the applicable provisions of chapter 137 and the FAR to the extent that they do not conflict with the public-private competition statutes cited previously. (Newport News Shipbuilding and Dry Dock Company, B-221888, July 2, 1986, 86-2 CPD 23.) Further, aspects of a competition that fall outside the competitive acquisition system's parameters as defined by chapter 137 and the FAR, such as the comparison of public and private offers for the workloads from the two closing ALCs, are governed by 10 U.S.C. 2469a and the other statutes applicable to public-private depot competitions as implemented by the Air Force.

In general, the standards in chapter 137 and the FAR (1) require that a solicitation clearly and unambiguously state what is required so that all offerors can compete on an equal basis and (2) allow restrictive provisions to be included only to the extent necessary to satisfy an agency’s needs. Under these standards, an agency must follow the criteria announced in the solicitation, which in this case include those required by 10 U.S.C. 2469a, and exercise its judgment in a reasonable manner in determining which of the competing offers is to be selected. (Dimensions International/QSOFT, Inc. , B-270966.2, May 28, 1996, 96-1 CPD 257.)

Solicitation

The RFP for the Sacramento workloads provides for the award of several line items representing various performance phases for each of the different workloads to be competed. For example, line item no. 0001, among other things, calls for offers on a cost-plus-award fee basis for the transition period for the KC-135 aircraft, the A-10 aircraft, and commodities, including hydraulics, instruments/electronics, electrical accessories, and nonrouted backshop/manufacturing. Other line items provide for firm-fixed priced offers for the performance of these various workloads, including “over and above” work, once the transition is completed, and for other miscellaneous work requirements. The RFP also provides for a transition period, which is to begin at the award and to end by September 30, 1999, a 5-year basic performance period, and up to 3 additional years based upon the performance of the awardee. The line items representing the work for the KC-135 aircraft and the A-10 aircraft during the basic performance period are to be awarded on a multi-year basis, with guaranteed minimum quantities, while the other workloads are to be awarded on a requirements-type basis, with no minimum quantity guaranteed.

According to the solicitation, the competition is to be conducted in accordance with FAR 15.101, which sets forth the source selection processes and techniques to be used in competitive negotiated acquisitions, as well as the applicable Air Force and Air Force Materiel Command supplements. Further, the solicitation provides that the Depot Competition Procedures, the CCH, and their updates are to govern the selection.

The solicitation states that the award will be made to the offeror—either public or private—who is deemed responsible in accordance with the FAR, whose proposal conforms with the solicitation and is judged to represent the best value to the government. According to the RFP, the Source Selection Authority (SSA) will integrate the source selection team's assessments of the proposals under the evaluation criteria listed in the solicitation to arrive at a best value selection.

The evaluation criteria consist of criteria for transition, operations, cost, and assessment. Transition is made up of the integrated master plan, personnel plan, and integrated master schedule. The operations criteria consist of five factors representing the major workloads: (1) KC-135 aircraft, (2) hydraulics, (3) instruments/electronics, (4) electrical accessories, and (5) A-10 aircraft. The assessment criteria, which will be used for measuring the extent to which a proposal meets the transition, operations, and cost criteria, are made up of two parts: (1) understanding of/compliance with the solicitation requirements and (2) soundness of approach.

Under the cost criteria, proposals will first be assessed for completeness, realism, and reasonableness. Then each offeror’s total proposed cost is to be determined by calculating the various cost estimates, unit prices, and hourly rates proposed for the different line items. Next, the offerors’ total alternative cost is to be developed by factoring in the numerous adjustments to public and private offerors’ total proposed cost in accordance with the CCH and the RFP. Finally, the offerors’ total evaluated cost is to be determined by adjusting the total alternative cost to reflect the “dollarized impact of significant discriminators, to the extent that a dollar value can be assigned to such discriminators, based on identified proposal strengths, weaknesses and risks.”

Further, the RFP provides for the evaluation of general considerations such as the results of preaward surveys, site visits, and “fair market value.” In addition, the proposals are to be the subject of two risk assessments: proposal and performance. A proposal risk assessment is to measure the risk that is associated with an offeror’s proposed approach to accomplishing the solicitation requirements relating to each of the three transition area factors and each of the five operations area factors. A performance risk assessment is to assess, based on an offeror’s present and past performance, the probability of the offeror successfully accomplishing the proposed effort.

Finally, the solicitation provides that in the SSA’s best value assessment, the criteria for transition and operations areas, and cost criteria are to be equally important, while the general considerations are to be “considered substantially less important than Cost, Transition, or Operations.” According to the RFP, this assessment is also to include “as appropriate” items listed in the solicitation as “Other Considerations.” This category essentially reiterates five of the six requirements for the competition listed in the 1998 Authorization Act.

The proposals were first evaluated by specialized teams, which reported to a source selection evaluation board (SSEB), which in turn, reported its conclusions to a source selection advisory council (SSAC). The SSAC then advised the SSA, who made the final selection decision on the merits of the proposals.

Evaluation of Proposals

Two offerors submitted proposals in response to the solicitation. Ogden ALC, the public depot chosen by the Air Force to submit the public sector offer, proposed (1) to perform the commodities work and the A-10 aircraft work at its facilities in Utah and (2) to have the KC-135 aircraft work performed by the Boeing Aerospace Corporation (Boeing) at the Boeing Aerospace Support Center located at the closing San Antonio ALC at Kelly Air Force Base. Boeing, a partner to Ogden in the public sector offer, proposes to use the San Antonio facilities that have been transferred by the Air Force to the Greater Kelly Development Corp. (GKDC) and leased by GKDC to Boeing. The private sector offeror, Lockheed, proposed to perform all of the work at the closing Sacramento ALC at McClellan Air Force Base, where the workloads are currently being performed by government employees. The Sacramento ALC facilities are to be transferred by the Air Force to Sacramento County. Under the Lockheed proposal, the facilities would be leased by the county to Lockheed. Lockheed’s major subcontractors are AAI Aerospace Corporation (AAI) for the hydraulics workload and GEC-Marconi Avionics Incorporated (GEC-Marconi) for the instruments/electronics and the electrical accessories.

The proposals were initially evaluated to determine whether they were to be included in the competitive range in accordance with FAR 15.306(c) and considered for award. On June 23, 1998, the Air Force determined both proposals to be within the competitive range.

Accordingly, discussions were held with the offerors consisting of written evaluation notices raising concerns about each of the proposals and face-to-face and telephone exchanges about the concerns. As a result, each offeror submitted proposal revisions. The Air Force requested final proposal revisions on August 26, which were the subject of the Air Force's final cost adjustments and evaluation. Based on the results of the evaluations and cost adjustments, the advice of the SSAC, and the SSA's own analysis in the context of the RFP evaluation criteria, the SSA decided that the Ogden ALC proposal met all of the RFP requirements and represented the best value to the government over the life of the requirement. The SSA’s conclusion was based upon Ogden ALC’s “slight advantage” in the operations area and its lower “most probable total evaluated cost” of performance. Consequently, the SSA selected Ogden ALC to perform the Sacramento workloads.

Technical Evaluation

As noted previously, the solicitation evaluation criteria provided that the offerors’ management approaches were to be evaluated in the transition and operations areas. Under transition, three factors were to be evaluated: (1) integrated master plan, (2) personnel plan, and (3) integrated master schedule. Operations included five factors: (1) KC-135 aircraft, (2) hydraulics, (3) instruments/electronics, (4) electrical accessories, and (5) A-10 aircraft. Under each of the factors, the proposal risk was assessed.

Transition

The first factor under transition, integrated management plan, was to assess the management, transition activities, and logistics plans and activities of the offerors. Under this factor, the SSA noted that both offerors had been assigned a green, or acceptable, rating, with low risk by the SSAC. The SSA stated that Lockheed’s approach, which was to perform all of the work at the Sacramento ALC facilities, had less potential for disruption to the ongoing operations than the Ogden ALC plan, which involved the transition of the KC-135 aircraft work to the Boeing facility at the San Antonio ALC. The A-10 aircraft and the commodities work was to be done at the Ogden ALC facilities in Utah. While recognizing that the Ogden ALC approach was sound and would minimize disruption to workload production, the SSA nevertheless concluded that the Lockheed plan, which would maintain the existing, experienced government workforce in place at Sacramento “was a benefit.” Since the SSA concluded that neither offeror had significant strengths or weaknesses under this factor, she did not propose to make a “dollarization” adjustment.

Under the personnel plan factor, an offeror was to provide a plan that detailed the staffing necessary to perform the workloads and a plan to acquire, train, and maintain the staff. The SSA concurred with the SSAC's green rating for both proposals and the assignment of a low risk rating to the Lockheed proposal and a moderate risk rating to the Ogden ALC proposal. The SSA noted that Lockheed would use the trained and experienced workforce at the Sacramento ALC. The SSA believed that there was risk that some of the current workforce would not wish to leave government service but concluded that a significant portion would want to continue working at the Sacramento location. On the other hand, the SSA had concerns about the Ogden ALC’s ability to hire 328 people from the Sacramento workforce to work on the commodities at Ogden. In this regard, the SSA noted that Boeing intended to hire its workforce for the KC-135 aircraft from San Antonio, Texas, and Wichita, Kansas, where it currently maintains a facility. The SSA found “real differences” between the proposals under this factor, concluding that the weakness in the Ogden ALC proposal regarding the availability of the experienced workforce resulted in a moderate risk rating. Consequently, the SSA concluded that the risk should be offset by an upward “dollarization” cost adjustment in the determination of Ogden ALC’s most probable total evaluated cost.

Under the integrated master schedule factor, an offeror was to submit a comprehensive transition time line based on the integrated master plan that identified all tasks and major event milestones required to transition all of the workloads. Both offerors were given green, or acceptable, ratings with a low risk by the SSAC. The SSA had a concern with the Ogden ALC approach, which would transfer the workload to two locations: Ogden, Utah, for the commodities and the A-10 aircraft and San Antonio, Texas, for the KC-135 aircraft, which would involve completely a new workforce for the KC-135. This concern did not raise to the level of a weakness because the SSA had confidence in Ogden ALC's risk mitigation strategy of using intense integrated product team oversight. Overall, the SSA considered both offerors to be equal under this factor and saw no need for a “dollarization” adjustment.

Operations

Under the KC-135 factor, an offeror must submit a contractor work specification that will become the contract statement of work, define all of the work requirements, and identify work flow and processes to reflect the offeror’s specific approach to accomplishing the work. Other information to be provided includes (1) a work activity flow plan for one complete aircraft to show the sequence of movement through the required processes and (2) “waterfall” plans, which are to describe the work flow activity using the number of aircraft represented by the best estimated quantity and, in the alternative, the maximum order quantity. In addition, any process improvements to current practices of performing the work are to be explained under this factor.

For the KC-135 aircraft, the SSAC rated both proposals as blue (exceptional) and assigned Ogden ALC a low risk rating and Lockheed a low/moderate rating. In adopting the ratings, the SSA noted that both proposals contained strengths concerning the reduction of the flowdays needed for the programmed aircraft depot maintenance. She nevertheless concluded that the Ogden ALC proposal was the strongest under this factor. According to the SSA, this advantage was grounded in Ogden ALC’s proposal to exceed the RFP flowday requirement of 175 days by contractually committing to a reduction to 150 days by fiscal year 2002 and to 140 days by fiscal year 2003. In addition, the SSA was impressed by Ogden ALC’s proposal to use in-line jacking to accomplish “over and above” structural work concurrently with the planned programmed work and its plan to use a continuous fluid sampling analysis to ensure early recognition of defects. The SSA, however, did note a weakness in Ogden ALC’s approach concerning its plan to use fewer labor hours than considered readily achievable by the evaluation team. The SSA concluded that this risk could be mitigated by normal monitoring and was not significant enough for a moderate risk rating.

The SSA recognized that Lockheed also had proposed to reduce the required flowdays (to 165 in fiscal year 2002 and to 155 by fiscal year 2004), but Lockheed’s approach, which involved using two shifts, had some risk because Lockheed had not documented the availability of the workforce for the extra shift or the way the proposed productivity increases would be achieved. Another element contributing to Lockheed’s low/moderate risk rating was the SSA’s concern about the plan to use field team employees to meet the proposed productivity increases and flowday reductions. The SSA decided that Ogden ALC’s superior flowday reduction could not be “dollarized” because the KC-135 was not an aircraft that generated revenue for the Air Force. However, the SSA recognized that this was a significant benefit and concluded that Ogden ALC offered an approach to the KC-135 aircraft work that had greater strengths and was more advantageous than Lockheed’s approach.

Under the hydraulics factor, the SSA concurred with the SSAC’s rating of each proposal as green and low risk. The SSA concluded both proposals were essentially equal and proposed no “dollarization.”

Under the factor for instruments/electronics, the SSA agreed with the SSAC’s ratings of both proposals as green and low risk. Again, the SSA concluded that both offerors were essentially equal and proposed no “dollarization.”

Under the electrical accessories factor, the SSAC rated both proposals as green; Lockheed was given a low risk rating, while Odgen ALC was rated as a moderate risk. This rating was caused by a process improvement concerning the pretesting of oil-cooled generators and a change in the painting sequence of the repair process. According to the SSA, the new test process could create production bottlenecks due to test equipment constraints with the potential of increased flowdays and the new paint process could increase flowtime. Thus, the SSA concluded that to offset this moderate risk, an upward “dollarization” adjustment to the evaluated cost of the Ogden ALC proposal would be added. This adjustment would reduce, in the SSA's view, the risk rating to low.

Under the A-10 aircraft factor, the SSA concurred with the SSAC's ratings of both proposals as blue and low risk. The SSA concluded that both had equal strengths and proposed no “dollarization.”

Risk and General Considerations

Under the performance risk analysis for the transition, operations, and cost areas, the SSA determined that each represented a low risk and that there was not a discriminator among them. Further, under the general consideration category, the SSA concluded that both offerors meet all solicitation and responsibility requirements and had acceptable small business plans.

The SSA agreed with the SSAC’s analysis of fair market value of the assets of the closing McClellan Air Force Base that were to be transferred to Sacramento County and then leased to Lockheed to perform the workloads. According to the SSA, Lockheed would lease the facilities at a composite rate of $0.33 per square foot a month, the local market rate for similar industrial facilities. Further, the SSA concluded that the Air Force would transfer the facilities to the County at fair market value. Finally, the SSA agreed that the SSAC’s $25,038,804 adjustment to Lockheed’s cost proposal for depreciation of the government furnished equipment (GFE) reflected the fair market value of the equipment.

The SSA also evaluated Boeing’s lease of the closing San Antonio ALC facility in connection with its performance of the KC-135 aircraft work under the Ogden ALC proposal and concluded there was no basis to make any further adjustment (other than that already made for GFE) to either proposal for fair market value. In this regard, the SSA concluded that the lease between GKDC and Boeing was an “arms length” transaction, not contingent on the competition, made at a fair market price.

Cost Evaluation

As noted previously, the cost evaluation consisted of (1) an assessment of the realism and reasonableness of the cost proposals; (2) a determination of the “total alternative cost” of each proposal, calculated through adjustments required by the CCH and RFP; and (3) a determination of the total evaluated cost of each proposal, calculated by taking the total alternative cost and adjusting it to reflect the “dollarization” of significant discriminators among the proposals. In determining the total evaluated cost, the evaluators used ranges based on different estimates for overhead savings and risk “dollarization.” The results for each of these analyses conducted by the cost evaluators are summarized below.

Realism and Reasonableness Evaluation

The cost team evaluators initially reviewed each offeror’s cost proposal to determine its completeness, realism, and reasonableness. The evaluators ultimately were satisfied that each cost proposal met these standards. In accordance with the Depot Competition Procedures, the Defense Contract Audit Agency (DCAA) audited the Ogden ALC cost proposal and reviewed the public offeror’s disclosure statement and accounting and estimating systems. The disclosure statement was found to be adequate and the proposal was determined to be realistic.

DCAA also reviewed Ogden ALC’s accounting and estimating systems and found in a report dated December 22, 1997, that the accounting system was inadequate, in part, for assuring that all workload costs are properly recorded and that the estimating system was also inadequate, in part, because it relied on accounting system data. Nevertheless, we were informed by the DCAA auditors who performed the review that these inadequacies did not affect the validity of the Ogden ALC cost proposal.

Determination of Total Alternative Cost

The cost evaluators determined each offeror’s total alternative cost by first calculating the offeror’s “customer cost”--in essence, its proposed price for performing the requirement, and then making upward and downward adjustments to this figure in accordance with the RFP and the CCH. Ogden ALC’s customer cost was calculated to be $1,097,615,652. Lockheed’s customer cost was $1,256,721,586.

Using the customer cost for each offeror as a base, the evaluators made the comparability adjustments called for in the CCH and the RFP. Two sets of adjustments were made. The first set, required by form number 1 of the CCH, encompassed adjustments to the public sector offer and the second set, required by form number 2 of the CCH, governed adjustments applicable to the public and private sector proposals.

The CCH form number 1 adjustments made to the Ogden ALC proposal included upward and downward changes in a number of categories. The most significant were upward adjustments for base operating support, unfunded civilian retirement, and retiree health benefits. The upward and downward adjustments resulted in an adjusted cost of $1,153,415,260 to the proposal. The adjusted cost was lower than Lockheed's customer cost of $1,256,721,586.

Some CCH form number 2 adjustments were made to both proposals. Upward adjustments were made to both types for GFM for both the commodities and the KC-135 aircraft, contract administration, reduction-in- force (RIF) costs; costs associated with the transition of government personnel (i.e., costs of retaining current work force at McClellan that will be subject to a RIF and not be rehired by the new source after the workload is transitioned pending their separation); and costs of performing the work in process during the transition that each offeror elected not to perform. Most of these adjustments were similar in size for both proposals. The largest difference was in the RIF cost adjustment, resulting in increases to Ogden ALC’s and Lockheed’s costs. Examples of downward adjustments were those made for the payment of federal income tax on profits to the Lockheed and Ogden ALC proposals.

Form number 2 also provided for a downward adjustment for a public or private offeror that proposed and supported overhead savings to other government work resulting from the increased work from the competition sharing the costs of fixed assets. While no such savings were proposed by Lockheed, the evaluators determined that a downward adjustment of between $70,357,189 and $24,665,837 could be applied to the Ogden ALC proposal to represent the savings applicable to other workloads at the Ogden ALC facility during the performance period.

The net result of the form number 2 comparability analysis for the Lockheed proposal was an upward adjustment of $630,058,494. The form number 2 upward adjustments to the Ogden ALC proposal included the high and low ranges for overhead savings.

This final cost adjustment resulted in a total alternative cost for Lockheed of $1,886,780,080 and a range of between $1,694,862,974 and $1,740,554,326 for Ogden ALC. No single adjustment accounts for the cost difference at this point, and at the highest range for Ogden ALC, its alternative cost was more than $140,000,000 lower than Lockheed's.

Determination of Total Evaluated Cost

To arrive at the total evaluated cost of each proposal, the evaluators took the total alternative cost and applied “dollarization” adjustments.

The initial aspect of the Ogden ALC proposal that was considered to be suitable for quantification was the transition risk as it related to both the KC-135 aircraft and the commodities. The first concern was Ogden ALC's proposal to have the workforce at the closing Sacramento depot perform most of the work in process on the KC-135 aircraft before the workload was transitioned to the Boeing facility at San Antonio. While the adjustment for this work in form number 2 assumed normal workforce efficiency (80 percent), the evaluators, based on the experience of previous transitions from closing depots, concluded that it was likely that the closing depot would experience declining efficiency. Consequently, the evaluators calculated the impact of a lower efficiency rate (45 percent) on the work to be completed at Sacramento. This rate resulted in a quantification of the risk of performing the remaining KC-135 work at Sacramento. Next, the evaluators added another adjustment that represented the risk that Boeing would not achieve its proposed 90-percent efficiency rate as it began the KC-135 aircraft work at its new facility.

The evaluators were similarly concerned about the transition risks in the Ogden ALC proposal for commodities workload. The evaluators calculated the Sacramento efficiency rate for the completion of commodities work in process to be 65 percent. Further, the evaluators reduced the efficiency rate proposed for the new commodity work to be performed by the combined workforces at Sacramento and Ogden. The total recommendation for “dollarized” transition risk for all the affected workloads (KC-135 aircraft and commodities) under the Ogden ALC proposal was $20,732,000.

The evaluators were also concerned about transition risk in connection with the Lockheed proposal. Their concerns centered around the efficiency of the Sacramento workforce in accomplishing the work in process for both the KC-135 aircraft and commodity workloads. In both cases the evaluators assumed a Sacramento efficiency rate of 65 percent (higher than the rates of Ogden ALC and its partner Boeing, as Lockheed proposed to perform the work at the closing Sacramento facility using much of the current workforce). Further, for both workloads, the evaluators did not think that Lockheed could achieve its proposed efficiency rates at the start of full performance. The calculations resulted in a proposed total “dollarization” of $16,068,000 for the Lockheed proposal.

The evaluators concluded that the risks inherent in two aspects of Ogden ALC’s proposed performance of the commodities workload after transition should be “dollarized.” The first risk involved Ogden ALC’s process improvements and whether those improvements (particularly those in the electrical accessories area) would result in a reduction of workhours. The evaluators were skeptical that the reduction could be achieved, and they consequently calculated a proposed upward “dollarization” adjustment of $17,380,738 to represent the risk that the predicted reduction would not occur.

The second risk related to the nature of the public depots’ funding under the working capital fund and the possibility that the government will have to shoulder the additional cost if Ogden ALC cannot perform at its proposed labor rates. The evaluators requested DCAA to conduct a rate- risk analysis on the Ogden ALC cost proposal, comparing the proposed rates to Ogden ALC’s current and past labor rates. DCAA calculated what it believed to be more realistic rates and concluded that its rates would result in a $37,373,690 increase in Ogden ALC's cost. The evaluators concluded that a “dollarized” risk range representing the high and low historical rates analyzed by DCAA should be established for this factor.

As the result of these evaluations, the SSAC presented the SSA with a recommended total evaluated cost range for each offeror. The recommendation consisted of a high range, including the lowest overhead savings, if any, combined with the highest upward “dollarization” adjustments, and a low range consisting of the highest overhead savings, if any, and the lowest upward “dollarization” adjustments. The high range was $1,819,717,982, and the low range was $1,707,243,712 for Ogden ALC. The high range for Lockheed was $1,902,848,080, while the low range was $1,886,780,080. Ogden ALC's high range is below Lockheed's low range by over $65,000,000.

The SSA reviewed the recommended total evaluated ranges and concluded that the high range estimate for Lockheed of $1,902,848,080, including the recommended $16,068,000 “dollarization” for transition risk represented Lockheed's most probable cost. The SSA concluded that the most probable cost for Ogden ALC was $1,794,488,861, about $25,000,000 below the SSAC's high range. The SSA’s estimate for Ogden ALC included $46,217,730 for overhead savings (about midway between the SSAC high and low ranges), the total amounts recommended by the SSAC for transition risk ($20,732,000) and for performance risk related to commodity workload process improvements ($17,380,738) and the DCAA recommendation of $37,373,690, for overall cost risk for the workloads to be performed at the public depot. A summary of the cost adjustments adopted by the SSA is shown in table III.1.

Table III.1: Cost Adjustments Adopted by the SSA (46,217,730)

Award

Based on the evaluation results, the SSA concluded that the offerors were “essentially equal” in performance risk and general considerations. The SSA noted that Lockheed had a slight advantage in transition, while Ogden ALC had a slight advantage in operations. The SSA selected Ogden ALC for award, as the competitor representing the best value to the government based upon the public offeror’s operations advantage combined with its lower most probable total evaluated cost.

GAO Analysis

As discussed previously, several statutes govern the solicitation and award process for public-private competitions for the depot workloads of the closing Sacramento and San Antonio ALCs. In particular, 10 U.S.C. 2469a sets forth the elements that must be considered in selecting the public or private source to perform the workloads. Further, because the Air Force used the competitive acquisition system, the standards in chapter 137 of title 10 of the United States Code and the FAR apply to the extent they are consistent with 10 U.S.C. 2469a and the other applicable provisions relating to the outsourcing of depot workloads and to conversions of DOD functions to private-sector performance. See (Newport News Shipbuilding and Dry Dock Co.,) cited above.

In addition to reviewing the evaluation and selection records, we spoke to relevant Air Force officials and to the public-sector offeror. While the public offeror won the competition, it still had some concerns that primarily centered around its preliminary understanding of the treatment of its proposed overhead savings and some of its proposed cost adjustments.

Reviewing the Sacramento competition in this context, we found no basis to conclude that the procedures used in selecting the successful offeror deviated in any material respect from the section 2469a requirements or other applicable laws or relevant provisions of the FAR. The Air Force issued a competitive solicitation in accordance with FAR part 15, which provided for the participation of a public sector depot. We found no basis to conclude that the selection did not provide for a substantially equal opportunity for public and private offerors to compete without regard to performance location or that appropriate consideration was not given to noncost factors in the selection. Overall, the evaluation process appeared to be reasonable, fair, and consistent with the evaluation scheme in the solicitation, the Depot Competition Procedures, and the CCH. While not affecting the legal sufficiency of the selection, we nevertheless identified several issues related to the estimates supporting the cost evaluation. These issues are discussed in the body of the report.

Performance Location

Subsection (g) of 10 U.S.C. 2469a provides that our report on the competitive procedures is to include our view as to whether the procedures “provided substantially equal opportunity for public and private offerors to compete for the contract without regard to the location at which the workload is to be performed.” In addition, 10 U.S.C. 2469a(d), which lists the requirements for the selection process, provides that a public or private competitor must be permitted to perform at the location of its choosing and a competitor is not to be given preferential treatment for, or be limited to, performing the workload in place or at any other single location.

As stated in our prior review of the solicitation for the Sacramento workloads, we found no provisions in the solicitation that designated a particular location at which performance was required or preferred or that evidenced a bias toward any particular performance location. Similarly, in our review of the selection process, we found nothing to indicate that a particular performance location was required or that there was a bias toward a particular location in the evaluation of the proposals or the selection of Ogden ALC.

In the selection, the SSA expressed concern about the risks inherent in the Ogden ALC plan for transitioning the workloads from the Sacramento facility to the San Antonio and Ogden performance locations. The SSA concluded that Lockheed’s proposal to perform at the closing Sacramento facility gave that firm a slight advantage in the transition area. As we understand the 10 U.S.C 2469a provisions concerning performance location, they are to prevent the Air Force from creating an advantage for a particular location for reasons that are not reasonably related to performance or cost. We believe that the SSA’s concerns in the evaluation, which centered on Ogden ALC's ability to convince more than 300 experienced Sacramento workers to relocate to Ogden, Utah, were based upon legitimate performance considerations related to Ogden ALC’s transition plan and did not reflect bias towards performance at Sacramento.

Consideration of Noncost Factors

In accordance with 10 U.S.C. 2469a(g), our review of the selection process is to include our view as to whether “appropriate consideration was given to factors other than cost in the selection of the source for performance of the workload.” We found no basis to conclude that the Air Force did not give “appropriate consideration to noncost factors in the selection process.”

As discussed in our review of the Sacramento solicitation, the selection was to be based upon “the best value to the Government.” This selection scheme integrated a relative assessment of such noncost factors as transition, operations, and risk along with a extensive evaluation of the proposed costs. Under this evaluation method, the entity selected might, or might not, be the competitor whose proposal was determined to represent the lowest total evaluated cost.

The selection of Ogden ALC was based on the SSA’s assessment that the public offeror’s slight advantage in the operations area (a noncost consideration) combined with its lower evaluated cost resulted in the best value to the government. The evaluation and selection record shows an intensive assessment of the noncost elements of each of the proposals. For example, the SSA considered Lockheed’s plan to recruit and maintain the existing workforce in-place at the Sacramento facility to be a benefit; on the other hand, she concluded that Odgen ALC’s plans for relocating the workloads to San Antonio and Ogden were risky. The record also shows that the SSA considered and favorably noted Boeing’s (Ogden ALC’s partner) technical approach for performing the KC-135 aircraft work but expressed concern about the feasibility of Ogden ALC’s plans to implement process improvements in performing the electrical accessories work.

While the offeror selected did represent the lowest evaluated cost to the government, as the examples show, the SSA and the other evaluators considered the relative merits of the technical and management approaches of the offerors. Thus, the record provides no basis for us to conclude that factors other than cost were not given appropriate consideration as required by 10 U.S.C. 2469a.

Compliance With Other Applicable Provisions of 10 U.S.C. 2469a

In addition to addressing the section 2469a provisions, including performance location and consideration given to factors other than cost, we reviewed the Sacramento competition to determine whether it otherwise complied with the requirements of section 2469a. As noted previously, 10 U.S.C. 2469a sets forth six requirements that must be satisfied in the Sacramento solicitation and selection process.

Reviewing the evaluation and selection records in the context of the 10 U.S.C. 2469a requirements, we found that the six requirements were addressed during the evaluation and selection process in a manner that was consistent with the solicitation evaluation provisions. Thus, we found no basis to conclude that the Sacramento selection process deviated in any material respect from the requirements of 10 U.S.C. 2469a.

Compliance With Other Applicable Provisions of Law

As stated earlier, the provisions of 10 U.S.C. 2461 requiring a notice to Congress of the savings to be achieved from a conversion of a DOD function to private-sector performance, and the requirement in 10 U.S.C. 2462 that DOD is to contract with the private sector if a private firm can provide the supply or service needed at a lower cost, apply generally to conversions of DOD functions such as these workloads. Whether the Air Force must comply with either statute in a particular competition depends upon whether a public or private offeror is selected. In this case, the Air Force selected the proposal of the public-sector offeror, Ogden ALC, which represented the lowest total evaluated cost for the performance of the workloads. As we understand it, while the public offeror will use a private firm to perform the KC-135 aircraft portion of the requirement, Ogden ALC retains the overall responsibility for the performance of all of the workloads.

Section 2469a, which provides specific authority for the Sacramento competition, authorizes a public entity to use a private-sector firm as a participant in its proposal. In view of this special authority, and considering that the Ogden ALC proposal represented the overall lowest evaluated cost, we do not believe that either 10 U.S.C. 2461 or 2462 requires that portions of the proposal representing work to be performed by the public offeror or by its private partner be the subject of separate analyses. We, thus, conclude that, in this case, the selection of the low-cost public offeror was consistent with 10 U.S.C. 2462 and did not trigger the notice requirements of 10 U.S.C. 2461.

Other Matters

While we found no basis to conclude that the evaluation and selection process deviated in any material respect from the provisions of 10 U.S.C. 2469a and other applicable provisions of law, we identified several issues related to the estimates used in the cost evaluation. These issues are discussed in the body of the report.

Major Contributors to This Report

National Security and International Affairs Division, Washington, D.C.

Office of the General Counsel, Washington, D.C.

Dallas Field Office

Related GAO Products

Army Industrial Facilities: Workforce Requirements and Related Issues Affecting Depots and Arsenals (GAO/NSIAD-99-31, Nov. 30, 1998).

Navy Depot Maintenance: Weaknesses in the T406 Engine Logistics Support Decision (GAO/NSIAD-98-221, Sept. 14, 1998).

Defense Depot Maintenance: Public and Private Sector Workload Distribution Reporting Can Be Further Improved (GAO/NSIAD-98-175, July 23, 1998).

Defense Depot Maintenance: Contracting Approaches Should Address Workload Characteristics (GAO/NSIAD-98-130, June 15, 1998).

Public-Private Competitions: Review of San Antonio Depot Solicitation (GAO/OGC-98-49, May 14, 1998).

Defense Depot Maintenance: Use of Public-Private Partnering Arrangements (GAO/NSIAD-98-91, May 7, 1998).

Public-Private Competitions: Review of Sacramento Air Force Depot Solicitation (GAO/OGC-98-48, May 4, 1998).

Navy Ship Maintenance: Temporary Duty Assignments of Temporarily Excess Shipyard Personnel Are Reasonable (GAO/NSIAD-98-93, Apr. 21, 1998).

Public-Private Competitions: DOD’s Additional Support for Combining Depot Workloads Contains Weaknesses (GAO/NSIAD-98-143, Apr. 17, 1998).

Defense Depot Maintenance: DOD Shifting More Workload for New Weapon Systems to the Private Sector (GAO/NSIAD-98-8, Mar. 31, 1998).

Depot Maintenance: Lessons Learned From Transferring Alameda Naval Aviation Depot Engine Workloads (GAO/NSIAD-98-10BR, Mar. 25, 1998).

Public-Private Competitions: Access to Records Is Inhibiting Work on Congressional Mandates (GAO/T-NSIAD-98-111, Mar. 4, 1998).

Force Structure: Army’s Efforts to Improve Efficiency of Institutional Forces Have Produced Few Results (GAO/NSIAD-98-65, Feb. 26, 1998).

Public-Private Competitions: Access to Records Is Inhibiting Work on Congressional Mandates (GAO/T-NSIAD-98-101, Feb. 24, 1998).

Public-Private Competitions: DOD’s Determination to Combine Depot Workloads Is Not Adequately Supported (GAO/NSIAD-98-76, Jan. 20, 1998).

Public-Private Competitions: Processes Used for C-5 Aircraft Award Appear Reasonable (GAO/NSIAD-98-72, Jan. 20, 1998).

Defense Depot Maintenance: Information on Public and Private Sector Workload Allocations (GAO/NSIAD-98-41, Jan. 20, 1998).

Air Force Privatization-in-Place: Analysis of Aircraft and Missile System Depot Repair Costs (GAO/NSIAD-98-35, Dec. 22, 1997).

Outsourcing DOD Logistics: Savings Achievable But Defense Science Board’s Projections Are Overstated (GAO/NSIAD-98-48, Dec. 8, 1997).

Navy Regional Maintenance: Substantial Opportunities Exist to Build on Infrastructure Streamlining Progress (GAO/NSIAD-98-4, Nov. 13, 1997).

Air Force Depot Maintenance: Information on the Cost-Effectiveness of B-1 and B-52 Support Options (GAO/NSIAD-97-210BR, Sept. 12, 1997).

Navy Depot Maintenance: Privatizing Louisville Operations in Place Is Not Cost-Effective (GAO/NSIAD-97-52, July 31, 1997).

Defense Depot Maintenance: Challenges Facing DOD in Managing Working Capital Funds (GAO/T-NSIAD/AIMD-97-152, May 7, 1997).

Defense Depot Maintenance: Uncertainties and Challenges DOD Faces in Restructuring Its Depot Maintenance Program (GAO/T-NSIAD-97-112, May 1, 1997) and (GAO/T-NSIAD-97-111, Mar. 18, 1997).

Navy Ordnance: Analysis of Business Area Price Increases and Financial Losses (GAO/AIMD/NSIAD-97-74, Mar. 14,1997).

Defense Outsourcing: Challenges Facing DOD as It Attempts to Save Billions in Infrastructure Costs (GAO/T-NSIAD-97-110, Mar. 12, 1997).

High-Risk Series: Defense Infrastructure (GAO/HR-97-7, Feb. 1997).

Air Force Depot Maintenance: Privatization-in-Place Plans Are Costly While Excess Capacity Exists (GAO/NSIAD-97-13, Dec. 31, 1996).

Army Depot Maintenance: Privatization Without Further Downsizing Increases Costly Excess Capacity (GAO/NSIAD-96-201, Sept. 18, 1996).

Navy Depot Maintenance: Cost and Savings Issues Related to Privatizing-in-Place at the Louisville, Kentucky Depot (GAO/NSIAD-96-202, Sept. 18, 1996).

Defense Depot Maintenance: Commission on Roles and Mission’s Privatization Assumptions Are Questionable (GAO/NSIAD-96-161, July 15, 1996).

Defense Depot Maintenance: DOD's Policy Report Leaves Future Role of Depot System Uncertain (GAO/NSIAD-96-165, May 21, 1996).

Defense Depot Maintenance: More Comprehensive and Consistent Workload Data Needed for Decisionmakers (GAO/NSIAD-96-166, May 21, 1996).

Defense Depot Maintenance: Privatization and the Debate Over the Public-Private Mix and (GAO/T-NSIAD-96-148, Apr. 17, 1996) and (GAO/T-NSIAD-96-146, Apr. 16, 1996 ).

Military Bases: Closure and Realignment Savings Are Significant, but Not Easily Quantified (GAO/NSIAD-96-67, Apr. 8, 1996).

Depot Maintenance: Opportunities to Privatize Repair of Military Engines (GAO/NSIAD-96-33, Mar. 5, 1996).

Closing Maintenance Depots: Savings, Workload, and Redistribution Issues (GAO/NSIAD-96-29, Mar. 4, 1996).

Navy Maintenance: Assessment of the Public-Private Competition Program for Aviation Maintenance (GAO/NSIAD-96-30, Jan. 22, 1996).

Depot Maintenance: The Navy’s Decision to Stop F/A-18 Repairs at Ogden Air Logistics Center (GAO/NSIAD-96-31, Dec. 15, 1995).

Military Bases: Case Studies on Selected Bases Closed in 1988 and 1991 (GAO/NSIAD-95-139, Aug. 15, 1995).

Military Base Closures: Analysis of DOD’s Process and Recommendations for 1995 (GAO/T-NSIAD-95-132, Apr. 17, 1995).

Military Bases: Analysis of DOD’s 1995 Process and Recommendations for Closure and Realignment (GAO/NSIAD-95-133, Apr. 14, 1995).

Aerospace Guidance and Metrology Center: Cost Growth and Other Factors Affect Closure and Privatization (GAO/NSIAD-95-60, Dec. 9, 1994).

Navy Maintenance: Assessment of the Public and Private Shipyard Competition Program (GAO/NSIAD-94-184, May 25, 1994).

Depot Maintenance: Issues in Allocating Workload Between the Public and Private Sectors (GAO/T-NSIAD-94-161, Apr. 12, 1994).

Depot Maintenance (GAO/NSIAD-93-292R, Sept. 30, 1993).

Depot Maintenance: Issues in Management and Restructuring to Support a Downsized Military (GAO/T-NSIAD-93-13, May 6, 1993).

Air Logistics Center Indicators (GAO/NSIAD-93-146R, Feb. 25, 1993).

Defense Force Management: Challenges Facing DOD as It Continues to Downsize Its Civilian Work Force (GAO/NSIAD-93-123, Feb. 12, 1993).

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary, VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.