Department of Energy: Improved Oversight Could Better Ensure Opportunities for Small Business Subcontracting
Released: 2005-05-24T00:00:00

Letter

Background

DOE has major sites and facilities around the country where the department carries out its missions, including developing, maintaining, and securing the nation’s nuclear weapons capability; cleaning up the nuclear and hazardous wastes resulting from more than 50 years of weapons production; and conducting basic energy and scientific research, such as mapping the human genome. DOE relies on facility management contractors to operate its facilities and accomplish its missions. This mission work is carried out under the direction of the National Nuclear Security Administration (NNSA) and DOE’s program offices, including the offices of Environmental Management and Science.

SBA develops regulations and provides guidance for federal agencies and their contractors to follow in carrying out the federal policy to provide maximum practicable contracting opportunities for small businesses, consistent with efficient contract performance. Under the Small Business Act and federal regulations, any large business with a federal prime contract that has a value over $500,000 (or $1 million for construction contracts) and that has subcontracting possibilities, must have a small business subcontracting plan, including annual goals for directing subcontracting dollars to small businesses. SBA has also issued specific guidance for developing annual subcontracting goals and reporting achievements to SBA. Although there is no specified percentage goal for small business subcontracting in law or regulations, SBA’s government- wide goal is that 40 percent of federal subcontracted dollars be directed to small businesses.

DOE’s facility management contractors carry out the SBA requirements by developing subcontracting plans and annual goals, and reporting their achievements to DOE and SBA. Facility management contractors use the following process: Develop small business subcontracting plans. Facility management contractors generally develop small business subcontracting plans as part of the overall proposal submitted in response to a DOE request for proposals on a new contract or upon extension or renewal of an existing contract. The subcontracting plan includes information on the types of work to be subcontracted and how, and to what extent, the contractor would provide subcontracting opportunities for small businesses. The proposed subcontracting plans are reviewed as part of the overall proposal and are then incorporated into the contract when it is awarded.

Develop annual small business subcontracting goals. Generally, the facility management contractors perform “make-or-buy” analyses to determine how much of their work will be subcontracted rather than performed by the contractors’ own employees. For the portion of the work the facility management contractor intends to subcontract, the facility management contractors develop small business subcontracting goals each year, after considering the proposed funding for the contracts, upcoming subcontracting opportunities, and the potential for small businesses to perform the subcontracting work. After an agreement is reached on the proposed goals, the goals are incorporated into the contract.

Report small business subcontracting achievements. Facility management contractors are required to report their small business subcontracting achievements to SBA twice each fiscal year—as of the ends of March and September. SBA guidelines specify that small business subcontracting achievements should be calculated as a percentage of total dollars subcontracted. Facility management contractors also report this achievement information to DOE.

DOE’s Small Business Office negotiates the department’s annual small business subcontracting goals with SBA, coordinates outreach efforts with the small business community, and works with NNSA and DOE’s program offices to establish and monitor annual goals. DOE’s Office of Procurement and Assistance Management and NNSA’s Office of Acquisition and Supply Management establish policies and guidance for conducting procurements according to federal and departmental regulations, and they maintain the information systems on the department’s prime contracts, including annual dollars provided to each facility management contractor. NNSA and DOE’s program offices, such as Environmental Management and Science, are responsible for providing program oversight and direction to the contractors.

In addition to the oversight responsibilities shared among DOE’s various headquarters and field organizations, SBA has a role in the oversight of facility management contractors’ small business subcontracting efforts. SBA officials in regional offices generally review proposed subcontracting plans and proposed annual small business goals. Regional SBA officials also review the reports on small business subcontracting achievements submitted by facility management contractors, and they may review large proposed subcontracts to determine if there are potential opportunities for small businesses. SBA regional offices also perform periodic compliance reviews of individual contractor’s small business subcontracting efforts, including validating the data reported by the contractors to ensure that the contractor is following SBA guidelines.

Data Reported by Facility Management Contractors Is Not Useful for DOE’s Monitoring of Their Small Business Subcontracting Performance

The data that DOE uses to monitor its facility management contractors’ performance in small business subcontracting generally shows that the contractors have exceeded their annual subcontracting goals. However, these data are not useful for monitoring purposes for two reasons. First, the data reported by the facility management contractors frequently overstated their subcontracting achievements, by failing to account for all of their subcontracts, as required by SBA reporting guidelines. As a result, the percent of total subcontracting dollars going to small businesses appeared larger than if the contractors had been following SBA’s reporting guidelines. Second, even if the facility management contractors were correctly reporting their small business subcontracting achievements, the method for calculating those achievements does not provide a true picture of contractors’ performance. The method can make a contractor appear to be performing well—for example, by directing 80 percent of subcontracting dollars annually to small businesses—when, in fact, that percentage is based on a small amount of dollars. When small business subcontracting dollars are recalculated as a percent of the annual funding on the facility management contract, the contractor’s actual achievement can turn out to be relatively low. Because contractor-reported data does not provide a true picture of the contractors’ performance, the department cannot meaningfully use the data to monitor performance and determine whether the contractors are providing small business subcontractors with the maximum practicable opportunity to provide goods and services to the facility management contractors, as required by the policy set out by the Small Business Act.

Contractor-Reported Data on Small Business Subcontracting Achievements Frequently Overstated Performance

The contractor-reported data that DOE uses to monitor facility management contractors’ performance in small business subcontracting generally showed that contractors exceeded their annual small business subcontracting goals in fiscal years 2001 through 2004. In fiscal year 2004, for example, 29 of the 34 facility management contractors reported exceeding their annual goals when they directed 33 to 88 percent of their subcontracting dollars to small businesses. These contractors reported exceeding their goals by as much as 28 percent, whereas the contractors that reported falling short of their goals for that year did so by as much as 26 percent. Whether or not individual contractors achieved their goals, the 34 contractors collectively reported directing nearly $3.3 billion to small business subcontractors in fiscal year 2004, representing a 27 percent increase over the amount they had directed to small businesses in fiscal year 2001 (see table 1). Additional information on the facility management contractors’ subcontracting goals and achievements for fiscal years 2001 through 2004 is presented in appendix I.

Although facility management contractors generally reported exceeding their subcontracting goals, the data are not useful for monitoring the contractors’ performance, because the contractors’ achievements were frequently overstated. When calculating subcontracting achievements, SBA guidelines allow contractors to exclude from their calculations only those subcontracts performed outside of the U.S. or its territories and subcontracts with affiliate organizations, such as a subsidiary or parent company. However, in fiscal year 2004, all 34 of the contractors’ reported achievements incorrectly excluded other types of subcontracts, beyond what SBA guidelines allow (see table 2). The facility management contractors we visited generally told us it was their normal practice to make these types of exclusions. In making these types of exclusions, the contractors said they were following their past practices or that DOE contracting officers had approved the additional exclusions. For example, officials representing one of these contractors pointed out that, in excluding subcontracts from the calculation of their subcontracting achievements beyond what SBA guidance allows, their company was following the subcontracting plan approved by DOE. The contractor officials said that they would not have made these exclusions, if DOE had not permitted them, nor would they have negotiated as high a subcontracting goal under those circumstances.

Facility management contractors’ practice of incorrectly excluding certain subcontracts from achievement calculations resulted in overstating their reported subcontracting achievements by making the achievements appear larger than they would have appeared, if only the appropriate exclusions had been made in the calculation. For SBA-reporting purposes, contractors calculate their subcontracting achievements by expressing the dollars that they directed to small business subcontracts as a percentage of the total amount they subcontracted. However, because they incorrectly excluded subcontracts that generally did not go to small businesses—such as subcontracts with other facility management contractors—the facility management contractors incorrectly reduced the total amount that was subcontracted (the denominator), without affecting the total amount going to small businesses (the numerator). As a result, the percentage of total subcontracted dollars going to small businesses appeared larger than it actually was.

The effect of making incorrect exclusions on achievement calculations is significant. At the 13 facility management contractors we visited, we analyzed the fiscal year 2004 dollar amounts associated with the subcontracts that were incorrectly excluded from the calculation of their reported subcontracting achievements. These excluded subcontracts totaled about $887 million. For these 13 contractors, the practice of incorrectly excluding subcontracts from their reported achievements resulted in overstating the percentage achievements by as much as 27 percent. Moreover, although 12 of the 13 contractors had appeared, from their reported achievements, to be meeting or exceeding their subcontracting goals, when the achievements were recalculated by adding the incorrectly excluded subcontracts back into the calculation, only 4 contractors would have achieved their goals. The remaining 9 contractors would have fallen short of their subcontracting goals, by as much as $31.1 million, or about 19 percent. According to contractor officials, it is also likely that these contractors would not have agreed to as high of a subcontracting goal, if they had to include these subcontracts (see table 3).

The fiscal year 2004 small business subcontracting achievements reported by many of the remaining 21 contractors that we did not visit during our study also are likely overstated, although we did not document the extent of the overstatement. Based on information these contractors reported to us, all 21 contractors had incorrectly excluded subcontracts from the calculations of fiscal year 2004 subcontracting achievements. This practice will almost always result in overstating actual achievements, because the subcontracts excluded from the calculations are generally not awarded to small businesses.

Small Business Subcontracting Reporting Method Does Not Provide a True Picture of Facility Management Contractors’ Performance

The method that facility management contractors use to calculate their small business subcontracting achievements for SBA reporting purposes does not present a true picture of a contractor’s small business subcontracting because achievements are not calculated in relation to the total value of the facility management contract. Rather, the method for calculating small business subcontracting achievements is based on a quantity determined by the facility management contractors—namely, the portion of total work under their DOE contract that they intend to subcontract out. Thus, a contractor can appear to be performing well—i.e., directing 80 percent of its subcontracting dollars to small businesses—but that percentage is not very informative about the actual amount of subcontracting occurring with small businesses.

In our view, calculating small business subcontracting achievements in this manner invariably presents a misleading picture. For example, in fiscal year 2004, contractors at two of DOE’s weapons production facilities reported substantially different small business subcontracting achievements—one reported directing 80 percent of its subcontracting dollars to small businesses (or $92.6 million) while the other reported 57 percent (or $109.9 million). However, these percentages hide the fact that the first contractor subcontracted out proportionally less of the work of the facility management contract than the second contractor, despite the fact that both facility management contracts were similar in size. Recalculating the subcontracting achievements as a percentage of the amount of funding directed by DOE to each facility management contract for the year reveals that the actual subcontracting achievements were quite similar—21 and 23 percent respectively of the contracts’ annual funding. Thus, the contractor that appeared, from the data that DOE uses, to be significantly outperforming the other was actually devoting a slightly smaller portion of its facility management contract dollars to small business subcontracting.

Recalculating the subcontracting achievement data as a percent of the annual amount funded by DOE for each facility management contract provides a very different picture of the contractors’ performance. Some contractors that appeared to be outperforming most other contractors were ranked near the bottom of the list when their achievements were measured as a percent of the annual funding on their contract. For example, the reported subcontracting achievement in fiscal year 2004 of the contractor at the Princeton Plasma Physics Laboratory appeared to be the second highest among the 34 contractors. However, after we recalculated the achievements as a percentage of annual contract funding, the contractor’s performance dropped significantly to 28th out of 34 contractors (see table 4). Conversely, 6 of the 17 contractors that were in the bottom half of the rankings of reported subcontracting achievements moved to the top half, after we recalculated the achievements as a percent of the annual amount funded on each facility management contract.

While a contractor may be subcontracting a smaller percentage of its facility management contract to small businesses than other facility management contractors, there can be several reasons for differences in contractors’ small business subcontracting achievements, which should be examined before concluding that a particular contractor is underperforming. For example, DOE and contractor officials told us that the type of work contractors perform at DOE sites is an important factor in determining the potential for small business subcontracting. Our analysis of the 34 facility management contractors’ fiscal year 2004 small business subcontracting achievements suggests that this may be true. We found that the 16 contractors primarily involved in environmental cleanup or weapons production at DOE sites generally directed a greater percentage of their contracts’ annual funding to small business subcontracting than the 16 contractors at DOE research laboratories. Contractor and DOE officials said that opportunities for small business subcontractors tend to be more limited at research laboratories than other types of DOE facilities, because research activities are often integrated, and fragmenting those activities among several contractors could negatively impact the work. Officials said that other types of work, such as facility construction or cleanup projects, generally have more subcontracting potential and can often be structured at such a scale that small businesses could perform the work.

Contract performance goals can also affect opportunities for small business subcontracting. For example, small business subcontracting opportunities diminished somewhat at an environmental cleanup site in Oak Ridge, Tennessee, after DOE and the facility management contractor, Bechtel Jacobs, renegotiated the contract to include more aggressive time lines and cost limitations for completing the work. In response to the changes in its contract, Bechtel Jacobs began subcontracting less of the work, including work that small business subcontractors had previously performed. DOE and contractor officials told us that the company wanted more control over the work to better ensure meeting the new time lines and cost limitations. Officials from local small businesses that subcontract with Bechtel Jacobs said that while they did not disagree with Bechtel Jacobs’s strategy for dealing with the changes to its contract, the small businesses had to adapt to a new business environment in which less subcontracting work would be available under that facility management contract.

Although differences in the type of work and contract performance goals may account for some of the differences in facility management contractors’ performance in small business subcontracting, there may be other reasons for those differences. For example, contractor officials said that the timing of projects or changes in the annual funding for a facility management contract could affect opportunities for small businesses to participate in work that is generally conducive to small business subcontracting. In addition, contractors at sites with similar missions and similar types of work can vary substantially in the amount of funding that they direct to small business subcontractors. For example, contractors at the Sandia and Los Alamos National Laboratories in New Mexico ranked highly among the 34 facility management contractors—8th and 12th, respectively—in the percentage of their contracts’ annual funding that they directed to small businesses in fiscal year 2004. In contrast, the contractor at the Lawrence Livermore National Laboratory in California, which conducts work similar to that of the two New Mexico laboratories, ranked much lower—27th among the 34 contractors. Contractor and DOE officials with whom we met suggested reasons for such differences in performances, for example, differences in the availability of small businesses that are capable of providing the needed goods and services or differences in the facility management contractors’ commitment to providing small businesses with the maximum practicable opportunity to participate in the work.

Furthermore, actions by DOE to increase its direct, or prime, contracting with small businesses may reduce the subcontracting opportunities available for small businesses. For example, in an effort to increase the amount of prime contract dollars going to small businesses, in 2003, DOE redirected around $30 million of work from the facility management contract for managing and operating the Strategic Petroleum Reserve in Louisiana and awarded it as a separate prime contract to a small business. Before DOE redirected the work, a contractor official said that the facility management contractor responsible for performing the work had been subcontracting most of it to small businesses. The official further stated that, as a result, DOE and the facility management contractor negotiated a lower small business subcontracting goal for fiscal year 2004 of 27.7 percent of the total dollars subcontracted, representing a 21.3 percent drop from the 49 percent goal at the end of fiscal year 2003. Although the facility management contractor exceeded its subcontracting goal in fiscal year 2004, the total dollars available for subcontracting decreased by around 12 percent ($6.1 million), and the portion going to small businesses decreased by 9.3 percent ($1.8 million).

Because the subcontracting achievement data that DOE relies on does not provide a true picture of the facility management contractors’ performance in small business subcontracting, the department cannot meaningfully use those data to monitor performance. Without data providing a truer picture of contractors’ performance, DOE lacks a basis for knowing whether differences in performance truly exist. Moreover, the department also lacks a basis for understanding whether differences in performance are legitimate—such as stemming from the types of work performed under a facility management contract—or are the result of differences in contractors’ level of commitment to small business subcontracting.

Having recognized potential weaknesses in contractor-reported achievement data, at least one federal agency we contacted is taking a more innovative approach for using data to monitor subcontracting efforts. The National Aeronautics and Space Administration (NASA), for internal management purposes, calculates small business subcontracting goals and achievements for its prime contractors as a percentage of the annual funding for the prime contracts. NASA’s small business office said it uses this method to oversee contractor performance because it (1) presents a truer picture of actual achievements and (2) provides a better basis to evaluate a contractor’s maximum practicable opportunity to subcontract with small businesses. This office said the data the contractors report to SBA can be misleading, in that a large percentage achievement can be reported even though a relatively small amount of dollars may actually be going to small business subcontracts.

Although DOE continues to base its facility management contractors’ subcontracting goals on the total dollars subcontracted, for at least one of those contractors, the department’s Environmental Management program plans to use a small business subcontracting approach that is similar to NASA’s approach. During a recent solicitation for bids for an environmental cleanup contract at the Hanford Site’s River Corridor Project in Washington State, the program’s solicitation specified that the winning bidder must subcontract 60 percent of work under the approximately $2 billion contract, with half of that directed to small businesses, or potentially about $600 million over the 7-year life of the contract. According to proposal documents, Environmental Management took this approach in part because of input from small businesses and because it would better ensure that small businesses had an opportunity to participate in a significant portion of the work under the contract.

DOE Oversight of Small Business Subcontracting by Facility Management Contractors Was Not Adequate to Address Identified Problems with Contractor- Reported Data

DOE had not taken adequate steps to address known problems with the small business subcontracting achievement data reported by its facility management contractors. Because misleading achievement data gave the appearance of a successful small business subcontracting program, DOE officials did not closely monitor facility management contractors’ subcontracting practices or follow through to ensure that the contractors complied with SBA reporting guidelines. DOE’s reliance on misleading data also resulted in a lack of assurance that the facility management contractors were providing the maximum practicable opportunities for small business subcontracting. The inadequate oversight was due, in part, to a lack of clear guidance on how DOE’s various headquarters and field organizations should coordinate and integrate their efforts to effectively oversee the small business subcontracting program.

Reliance on Misleading Data Undermined DOE’s Oversight

DOE knew there were problems with the data reported by the facility management contractors but did not take adequate corrective action. This had several undesirable consequences: (1) it created the false impression that DOE was meeting its small business subcontracting goals, (2) it caused errors in recognizing and rewarding contractor performance, and (3) it undermined efforts to ensure that facility management contractors were providing the maximum practicable subcontracting opportunities for small businesses.

False impression that DOE was meeting its small business subcontracting goals. The misleading data that DOE was using gave the appearance of a successful small business subcontracting program. As a result, DOE officials were not inclined to closely monitor facility management contractors’ subcontracting practices or to fully address known problems with the achievement data. DOE officials in the small business and procurement offices told us that they recognized in 2002 that the facility management contractors were not consistently following SBA guidelines on what subcontracts could be excluded when calculating small business subcontracting goals and achievements. In April 2002, DOE’s Small Business Office provided clarifying information in an e-mail message to small business program managers both in the DOE program offices and the facility management contractors. The information emphasized the importance of consistently following SBA guidelines. Although DOE provided subsequent training for small business program managers, DOE officials did not take adequate steps to ensure that the contractors were correctly applying SBA guidelines. Based on our analysis of fiscal year 2004 subcontracting practices, none of the 34 facility management contractors fully complied with the SBA guidelines.

In some cases, DOE actually approved the incorrect reporting practices by its facility management contractors. For example, we found 10 instances where DOE had reviewed and approved subcontracting plans that specifically called for incorrectly excluding subcontracts from the calculations of small business subcontracting goals and achievements. In two other cases, the facility management contractors specified incorrect exclusions in annual planning documents for fiscal year 2004. DOE uses these planning documents to evaluate overall contractor performance for the year. DOE contracting officers told us that they had approved or allowed the incorrect practices because they did not believe these subcontracts could practically go to small businesses anyway. For example, the facility management contractors may have had existing long- term subcontracts with large businesses to provide information technology assistance, building maintenance, or other site support services.

Recent program reviews by DOE’s Small Business Office were also not sufficient to identify and rectify these incorrect practices. In September 2002, the Secretary directed the Small Business Office to develop a strategic plan for providing maximum practicable opportunities to small businesses, including subcontracting by facility management contractors. As part of its May 2003 strategic plan, the Small Business Office stated that it would periodically review the small business subcontracting efforts of the facility management contractors. During fiscal year 2004, the Small Business Office performed reviews of the small business subcontracting programs at five of DOE’s facility management contractors. These reviews focused primarily on determining if the small business subcontracting plans contained the information required by federal regulations and whether the contractors met their annual small business subcontracting goals. The reviews did not evaluate how the facility management contractors were developing the annual small business subcontracting goals or whether all appropriate subcontracts were included in the goal and achievement calculations. DOE Small Business Office officials told us that, based on the results of our work, they are considering revising their review methodology to include these steps.

In March 2005, near the end of our review, DOE and NNSA procurement organizations issued additional guidance on small business contracting goals. The guidance stressed that the accuracy of contractor-reported data was vital to the credibility of the department’s performance in small business subcontracting. The guidance emphasized that, in calculating and reporting small business subcontracting goals and achievements, contractors could exclude only the subcontract types allowed by SBA guidelines. The guidance further states that contracting officers shall ensure that facility management contractors are aware of these procedures and do not apply additional exclusions when developing their subcontracting goals. However, given DOE’s oversight of these practices in the past, it remains to be seen whether DOE will now follow through and ensure that facility management contractors comply with this guidance.

Errors in recognizing and rewarding contractor performance. DOE’s Small Business Office established a Secretarial Small Business Awards program to recognize and acknowledge outstanding performance on an annual basis. One of these awards—the facility management contractor small business achievement award—is presented each year to the facility management contractor with the highest percentage increase in subcontract awards to small businesses from the previous year.

We reviewed the facility management contractor small business achievement awards DOE presented in fiscal years 2003 and 2004. Although the fiscal year 2003 award appears to be appropriate, the fiscal year 2004 award went to a facility management contractor who directed fewer dollars to small businesses from one year to the next. Specifically, the data DOE used to make the determination showed that the percentage of subcontracting dollars going to small businesses increased from 61 percent in fiscal year 2002 to 66 percent in fiscal year 2003, a 5 percentage point increase. However, during the same period, the total dollars subcontracted by the facility management contractor dropped from $60.7 million to $52.1 million, and the subcontracting dollars directed to small businesses dropped from $36.8 million to $34.4 million. If DOE had measured the change in the contractor’s small business subcontracting achievements by comparing them with the annual funding on the facility management contract, DOE officials would have discovered that the contractor’s performance actually declined by about 7 percent during the period (from 19.7 percent of annual funding on the contract to 13.0 percent). DOE Small Business Office officials told us that they based this award on increases in the percentage of subcontracting dollars going to small businesses rather than changes in the total dollars directed to small businesses. Officials said using changes in total dollars directed to small businesses would not be fair to all the contractors because the facility management contractors with multibillion dollar annual funding on their contracts would have an advantage in showing increased dollars to small businesses. While we agree that using dollars to compare the performance among contractors could favor those with greater annual funding, relying solely on the percentage of subcontracted dollars going to small businesses may result in rewarding declining performance, as demonstrated above.

Undermined efforts to ensure maximum practicable opportunities for small businesses. DOE’s reliance on misleading data created an environment in which DOE officials were not inclined to closely monitor the small business subcontracting program. For example, DOE allowed facility management contractors to incorrectly exclude subcontracts when calculating their small business subcontracting achievements, which led to the incorrect perception that they were achieving their subcontracting goals. Without accurate and consistent data on the subcontracting efforts of the facility management contractors, DOE has no way of determining whether a facility management contractor is providing the maximum practicable opportunity for small businesses.

Another consequence of these program weaknesses is a perception among members of the small business community that DOE’s facility management contractors could do better at providing opportunities for meaningful subcontracting work. This perception has led to distrust and criticism of DOE’s facility management contractors by small business associations and advocates. For example, during a May 2004 congressional hearing on DOE’s direct contracting with small businesses, small business advocacy groups raised concerns about whether the level of subcontracting by DOE facility management contractors reflected maximum practicable opportunities. Additionally, small business associations have expressed concerns about what they perceive as unfair treatment by DOE facility management contractors, such as using small business subcontractors to perform work that provides little opportunity for the small business to develop technical capabilities. Small business advocacy groups that we contacted said that if DOE held the facility management contractors more accountable for achieving small business subcontracting results, such as by using data that provided a truer picture of actual performance, the small business groups would have a basis for greater confidence in DOE’s small business subcontracting program.

Oversight Responsibilities for the Small Business Subcontracting Program Were Not Clearly Defined or Integrated

DOE officials did not address problems with the small business subcontracting program, in part, because the various headquarters and field organizations that must work together to provide oversight of the small business subcontracting program did not do so effectively. Oversight of facility management contractors’ small business subcontracting efforts includes reviewing and approving facility management contractors’ small business subcontracting plans and annual goals, monitoring the contractors’ progress toward achieving small business subcontracting goals, and taking action when necessary to ensure that the contractors meet their small business subcontracting goals.

Although DOE contracting officers have final responsibility for providing direction to the facility management contractors on all aspects of DOE contracts, other DOE headquarters and field offices, such as the Small Business Office and the program offices, share responsibilities for monitoring and overseeing the small business subcontracting activities of facility management contractors. DOE contracting officers, usually located at DOE field locations, have primary responsibility for communicating with facility management contractors on all aspects of the contracts, including the contractors’ small business subcontracting efforts. DOE’s Small Business Office has the overall responsibility for managing the department’s small business programs and monitoring performance toward meeting annual goals. Since the Small Business Office has no direct authority over DOE’s facility management contractors, it coordinates the subcontracting program efforts by using small business program managers within DOE’s program offices and field locations. The small business program managers at the field locations review subcontracting plans and annual goals, and they provide input to the contracting officers.

Effective oversight requires more than just carrying out these activities, but also collectively ensuring that the activities identify and correct problems and that program goals are achieved. Effective oversight was not happening, in part, because DOE guidance does not clearly define the roles, responsibilities, and needed interaction of the various headquarters and field organizations involved. DOE guidance on the small business program clearly defines only the roles of the contracting officer, which include review of small business subcontracting plans, annual goals, and reported achievements. The guidance also states that the contracting officer is responsible for ensuring that facility management contractors comply with the SBA guidelines on which subcontracts may be excluded from their subcontracting goals and achievements. However, the guidance does not clearly define roles or responsibilities for the Small Business Office, small business program managers, or others who need to coordinate their efforts to achieve department goals. Absent this guidance, we found that the collective efforts of these officials were not sufficient to identify and correct the problems discussed in this report or to ensure that program goals were being achieved.

Conclusions

Creating opportunities for small businesses to participate in providing goods and services to DOE’s facility management contractors supports federal policy and produces benefits for both DOE and the small business community. Because about 85 percent of DOE’s funding currently goes to its facility management contractors, none of which are small businesses, the small business subcontracting efforts of those facility management contractors are of even greater importance to DOE. However, misleading data has created the false impression that DOE is meeting its small business subcontracting goals, undermined DOE’s oversight of subcontracting efforts, and generated mistrust among members of the small business community. Furthermore, DOE actions to date have not adequately addressed problems with misleading data. Until the problems with contractor-reported achievement data are resolved and the program oversight issues are addressed, DOE cannot ensure that the federal policy of providing the maximum practicable opportunity for small businesses is being achieved.

Recommendations for Executive Action

To ensure that facility management contractors comply with SBA guidelines and follow a consistent practice for calculating and reporting small business subcontracting goals and achievements, we recommend that the Secretary of Energy direct the appropriate officials responsible for DOE’s Small Business Office and procurement organizations to ensure, through regular oversight and review activities, that facility management contractors comply with DOE’s March 2005 guidance on small business procurement goals.

To ensure that DOE has useful data for managing its small business subcontracting program and for measuring and comparing contractors’ performance in pursuing the maximum practicable opportunity for small business subcontracting, we recommend the Secretary of Energy direct the appropriate officials responsible for DOE’s Small Business Office and procurement organizations to use, for internal management purposes, data on facility management contractors’ annual small business subcontracting achievements calculated as a percentage of the obligated dollars facility management contractors received that year on their contract with DOE.

To improve DOE’s oversight of the small business subcontracting program and to provide the maximum practicable opportunity for small businesses to subcontract at DOE sites, we recommend that the Secretary of Energy take steps to strengthen oversight of the program, including issuing guidance clarifying the roles, responsibilities, and necessary interactions among DOE small business office, program office, and procurement officials responsible for managing the small business subcontracting program.

Agency Comments

We provided a draft of this report to DOE for review and comment. In written comments the Secretary of Energy generally agreed with the report and two of the recommendations but disagreed with one of the recommendations. The department agreed with our recommendation to strengthen oversight of its small business subcontracting program. DOE said that it is in the process of revising existing guidance to clarify oversight roles and responsibilities and the necessary coordination and integration of oversight efforts between DOE headquarters and field organizations.

The department also agreed with our recommendation to ensure that the facility management contractors follow a consistent practice for calculating and reporting their small business subcontracting goals and achievements, as outlined in DOE’s March 2005 guidance. In its comments, DOE outlined several steps it had taken to provide training, clarifying information, and guidance on the importance of accurate and consistent small business subcontracting data. However, effective oversight requires more than providing guidance and training. We believe that regular review and oversight activities will also be necessary to verify that contractors are complying with the guidance.

Regarding our recommendation that the department use data on its facility management contractors’ small business subcontracting achievements— recalculated as a percentage of the dollars obligated on their prime contract—to internally manage its program, DOE disagreed with the recommendation and proposed to continue calculating the achievements as a percentage of the total dollars subcontracted. While we agree that DOE should continue to follow SBA guidance for SBA reporting purposes, we continue to believe that this calculation method fails to provide a true picture of the facility management contractors’ small business subcontracting performance, which is necessary for effective oversight. As our report clearly demonstrated, the method can overstate the performance of contractors that devote a relatively small portion of their total contract to subcontracting. Thus, to ensure that DOE is providing the maximum practicable subcontracting opportunities to small businesses, for internal program management purposes, the department should use subcontracting achievement data calculated as a percentage of the facility management contract obligations. As described in our report, one of the department’s programs intends to use this approach to internally manage the small business subcontracting goals and achievements for the $2 billion contract to clean up the Hanford Site’s River Corridor in Washington State.

DOE also provided technical comments on the facts presented in the report, which we incorporated as appropriate. DOE’s comments and our responses are presented in appendix III.

We conducted our review from June 2004 to April 2005, in accordance with generally accepted government auditing standards. Appendix II provides details on our scope and methodology.

As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to the Secretary of Energy. We will also make copies available to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions on this report, please call me at (202) 512-3841. Other staff contributing to this report are listed in appendix IV.

Appendix I: DOE Facility Management Contractors’ Annual Small Business Subcontracting Goals and Achievements

Subcontracting achievement (percent of total dollars subcontracting achievement (percent of subcontracted)

Subcontracting achievement (percent of total dollars subcontracted)

Subcontracting achievement (percent of total dollars subcontracting achievement (percent of subcontracted)

Subcontracting achievement (percent of total dollars subcontracted)

Annual funding received on the facility management contract subcontracting achievement (percent of annual funding)

Honeywell Federal Manufacturing & Technologies, Kansas City Plant, Mo.

Subcontracting achievement (percent of total dollars subcontracted)

Annual funding received on the facility management contract subcontracting achievement (percent of annual funding)

Southeastern Universities Research Association, Thomas Jefferson National Accelerator Facility, Va.

Subcontracting achievement (percent of total dollars subcontracting achievement (percent of subcontracted)

Subcontracting achievement (percent of total dollars subcontracting achievement (percent of subcontracted)

Annual funding received on the facility management contract annual funding)

We omitted subcontracting goal and achievement data for DOE’s Mound Site in Ohio for fiscal years 2001 and 2002, because the contractor that managed and operated the site in 2001 and 2002 is not the same contractor that did so from 2003 through the duration of our study. While we included subcontracting achievement data from the previous contractor in the aggregated data for all facility management contractors in table 1, we did not include those data in this appendix because the fiscal year 2001 and 2002 data for the Mound Site came from a different source than the rest of the subcontracting goal and achievement data in this appendix. As a result, we had not assessed the reliability of the fiscal year 2001 and 2002 data for the Mound Site to be able to publish the contractor-level subcontracting goal and achievement data.

The facility management contractor managing and operating the Strategic Petroleum Reserve had a small business subcontracting goal of 47.0 percent for the first half of fiscal year 2003, but the goal was increased to 49.0 percent for the second half of the fiscal year, after the contractor was awarded a new facility management contract with DOE for the same site, and they negotiated a new subcontracting goal. However, for fiscal year 2004, the contractor’s subcontracting goal dropped to 27.7 percent. According to a contractor official, DOE and the contractor renegotiated the lower goal after DOE removed about $30 million of work from the facility management contract in fiscal year 2003. The facility management contractor had been subcontracting a majority of that work to small business subcontractors.

Appendix II: Scope and Methodology

We determined (1) the usefulness of the data reported by the Department of Energy’s (DOE) facility management contractors for monitoring contractor performance in small business subcontracting and (2) the actions that DOE has taken to address any problems identified with its facility management contractors’ small business subcontracting efforts. To conduct our work, we contacted DOE and contractor officials, as well as representatives from the Small Business Administration (SBA) and from small-business advocacy groups. We also collected data from all 34 of DOE’s facility management contracts on their subcontracting activities and took steps to assess the reliability of the contractors’ data, which included reviews of subcontracting files at 13 of the 34 contractors.

Determining the Usefulness of the Facility Management Contractor-Reported Data

To determine the usefulness of the facility management contractor- reported data for DOE monitoring purposes, we requested data from DOE’s 34 facility management contractors on their annual small business subcontracting goals and achievements. Specifically, we asked the contractors to provide the subcontracting percentage-goals that they negotiated with DOE for fiscal years 2001 through 2004, as well as the dollar figures comprising those percentages, including the total dollars to be subcontracted during the year and the portion of subcontracting dollars they expected would go to small businesses. We also asked the contractors to provide us with their subcontracting percentage achievements for fiscal years 2001 through 2004 and the dollar figures comprising those percentages—namely, the total dollars that were subcontracted and the small business subcontracting dollars. (Selected subcontracting goal and achievement data appear in app. I.) Additionally, in our data request, we asked the contractors to identify the types of subcontracts that they included or excluded from their small business subcontracting goals and achievements for fiscal year 2004. Furthermore, we requested copies of the contractors’ small business subcontracting plans that were in effect during fiscal year 2004.

We used a data collection instrument to obtain data from the facility management contractors on their fiscal year 2001 through 2004 small business subcontracting goals and achievements. In the data collection instrument, we provided tables for the contractors to enter their subcontracting information, and we included instructions on how to complete the tables. The data collection instrument also included a series of questions on various aspects of the reliability of the data they provided, as part of our assessment of data reliability described later in this appendix.

Because we received the requested information from all 34 of DOE’s facility management contractors, we did not rely on results from any subset of those contractors to conduct our analysis and, therefore, no sampling error is associated with our work. However, data gathering methods, such as the ones we used, may introduce error into the data that is not associated with statistical sampling, commonly referred to as nonsampling errors. For example, difficulties in how a particular instruction was interpreted, in the availability of the information requested, and in how the data were entered into the tables could introduce unwanted variation into the data collected. We took steps in the development of the data collection instrument to minimize these nonsampling errors. For example, we pretested a draft of the data collection instrument with 3 facility management contractors. In addition, we provided the draft to 5 knowledgeable small business and procurement officials in DOE headquarters. Within GAO the data collection instrument was reviewed by the Office of General Counsel and by experts in federal contracting and research methodology. We modified the data collection instrument as appropriate to reflect the comments and suggestions of the contractor, DOE, and GAO reviewers, and sent the data collection instrument to all 34 of DOE’s facility management contractors. We received responses from all 34 contractors.

We then assessed the reliability of the subcontracting data we received from each of the 34 facility management contractors. To assess the data reliability, we (1) analyzed the contractors’ responses to the data reliability questions that were included in the data collection instrument and (2) conducted a more detailed review of subcontract files at 13 of the 34 contractors.

First, we reviewed the contractors’ responses to the series of data reliability questions in the data collection instrument, which addressed such areas as data entry, data access, quality control procedures, and data accuracy and completeness. Follow-up questions were added as necessary. In consultation with a GAO expert in research methodology, we analyzed the contractors’ responses for weaknesses in data reliability that would make their data unusable for analysis and reporting purposes. In their responses, 7 of the 34 contractors discussed minor limitations to their data control processes that might affect how the data should be interpreted. For example, 3 contractors stated that their subcontracting data was both accurate and complete, while noting that it was such to the extent that they could make it. Another contractor reported that the data it provided for fiscal years 2001 and 2002 may be less accurate than the data for fiscal years 2003 and 2004, because prior to 2003, the contractor did not require that its small business subcontractors self-certify or provide documents that verify their small business status.

Second, we visited a nonprobability sample of 13 of the 34 facility management contractors to conduct a more detailed assessment of the reliability of the fiscal year 2004 subcontracting achievement data that they provided in response to our data collection instrument. We used several criteria to select facility management contractors for site visits, including a contractors’ association with DOE’s three largest component organizations—Environmental Management, the National Nuclear Security Administration (NNSA), and the Office of Science—as well as geographic location, and the type of work performed under the facility management contract. The three DOE component organizations we focused on control 30 of DOE’s 34 facility management contracts, and their annual budgets account for over 70 percent of DOE’s overall budget. In selecting NNSA contractors, we chose both research laboratories and weapons production facilities, to ensure inclusion of the types of work that NNSA uses to carry out its missions. In addition, to obtain geographically diverse cases, we selected contractors from four states in the Western U.S.—California, Nevada, New Mexico, and Washington—and one Eastern state, Tennessee. Nearly one-half of the 34 DOE facility management contractors are located within these five states.

Based on these selection criteria, we made visits to the following 13 facility management contractors: Battelle Memorial, Pacific Northwest National Laboratory, Wash.; Bechtel Hanford, Hanford Site, Wash.; Bechtel Jacobs, Oak Ridge Site, Tenn.; Bechtel National, Waste Treatment Plant, Wash.; Bechtel Nevada, Nevada Test Site, Nev.; Bechtel SAIC, Yucca Mountain Project, Nev.; BWXT Y-12, Y-12 National Security Complex, Tenn.; CH2M Hill Hanford, Hanford Site, Wash.; Fluor Hanford, Hanford Site, Wash.; Lockheed Martin, Sandia National Laboratories, N. Mex.; University of California, Lawrence Livermore National Laboratory, Calif.; University of California, Los Alamos National Laboratory, N. Mex.; and UT Battelle, Oak Ridge National Laboratory, Tenn.

Collectively, these 13 contractors had subcontracts worth about $4.2 billion in fiscal year 2004 (or 64.2 percent of the total subcontracting dollars for all 34 facility management contractors). Of that amount, over $2.1 billion went to small businesses, accounting for nearly two-thirds of the almost $3.3 billion in subcontract dollars that the 34 contractors directed to small businesses that year.

During our visits to these 13 facility management contractors, we looked at the reliability of the fiscal year 2004 small business subcontracting achievement data that the contractors provided in the data collection instrument. Even though our data covered fiscal years 2001 through 2004, for our more detailed review of data reliability, we focused on fiscal year 2004, because we performed most of our analyses on data from that year.

Although individual contractors may manage thousands of small business subcontracts in a given year, for each of the 13 contractors, we chose separate nonprobability samples of 5 small business subcontracts—for a total of 65 subcontracts—that the facility management contractors had included in the fiscal year 2004 small business subcontracting achievements. We selected each nonprobability sample from a complete list of the contractors’ fiscal year 2004 small business subcontracts, which we obtained prior to visiting the contractors. The criteria we used for selecting contracts was the subcontract amount and the goods and services provided. We selected subcontracts that received among the largest dollar commitments in fiscal year 2004 and that encompassed a variety of the types of goods and services provided by small business subcontractors. In total, the 65 subcontracts in our nonprobability samples accounted for 17.2 percent ($368.1 million) of the $2.1 billion dollars committed to small business subcontracts in fiscal year 2004 by the 13 facility management contractors.

At each contractor’s facility, we reviewed documents for each of the five subcontracts, in order to verify the dollar commitments to those subcontracts in fiscal year 2004, that the contractor included in their response to the data collection instrument. We also reviewed any documents certifying the small business status of those subcontractors at the time the subcontracts were awarded. In addition, to understand the effect of excluding certain subcontracts from the contractors’ small business subcontracting achievement calculations, these 13 contractors provided us with data on the dollar amount of their excluded subcontracts in fiscal year 2004.

For all 65 of the small business subcontracts we reviewed, we were able to verify through documents in the subcontract files that the amount of dollars committed to the selected subcontracts in fiscal year 2004 equaled the dollar amounts that the facility management contractors had taken credit for when reporting their small business subcontracting achievements for that year. Also, for 63 of the selected subcontracts, we were able to find documents that certified the subcontractor’s status as a small business at the time the subcontract was awarded. However, for the 2 remaining subcontracts—which were from 2 different facility management contractors—documents showed that the subcontractors had been classified as large businesses at the time the subcontract was awarded. Therefore, these 2 subcontracts should not have been reflected in those contractors’ fiscal year 2004 subcontracting achievements as small business subcontracts. Because this only occurred with 3 percent of the 65 subcontracts (representing 5.7 percent of the total $368.1 million value of those subcontracts), we did not consider it to be a material weakness of the subcontracting achievement data. In both instances where facility management contractors had mistakenly included a large business subcontract in their small business subcontracting achievements, the contractors subsequently restated their fiscal year 2004 subcontracting achievements. The subcontracting achievement data included in this report reflect the restated amounts.

Based on all of our efforts to assess data reliability, we believe that the data are sufficiently reliable for characterizing the 34 facility management contractors’ fiscal year 2004 small business subcontracting achievements.

Determining Actions Taken by DOE to Address Problems

To determine the actions that DOE has taken to address any problems identified with its facility management contractors’ small business subcontracting efforts, we visited officials at the nonprobability sample of 13 of the 34 facility management contractors previously described. We also met with the field-based DOE small business officials and contracting officers responsible for overseeing the facility management contractors’ performance in small business subcontracting.

In addition to our site visits, in DOE’s headquarters we interviewed officials from DOE’s Small Business Office and small business officials from the major component organizations, as well as DOE and NNSA procurement officials, to learn about their role and actions for overseeing the facility management contractors’ small business subcontracting efforts. From these officials, we obtained policy guidance, results of field reviews, and other documents related to DOE and NNSA’s oversight activities. We also interviewed headquarters and field-based officials from SBA’s Office of Government Contracting to further understand SBA guidelines for contractor reporting of small business subcontracting data. Furthermore, we spoke with representatives from three small business advocacy groups—the East Tennessee Environmental Business Association, the New Mexico 8(a) and Minority Business Association, and the Small Environmental Business Action Coalition.

We conducted our work from June 2004 to April 2005 in accordance with generally accepted government auditing standards.

Appendix III: Comments from the Department of Energy

The following are GAO’s comments on the Department of Energy’s letter dated May 3, 2005.

GAO Comments

1. We acknowledge in the draft report and highlights page that DOE had taken steps to address the problems with contractor-reported data, including issuing clarifying information in 2002 and additional guidance in March 2005. However, we stated that these steps had not been adequate to ensure that the facility management contractors were complying with the guidance. The draft report also stated that the reviews of the five facility management contractors performed by DOE’s Small Business Office did not include an evaluation of whether the facility management contractors were following SBA guidelines for developing small business subcontracting goals. 2. We agree that the Small Business Act and the Federal Acquisition Regulation do not require a specific level of subcontracting and that individual contractors determine the amount of work that will be subcontracted. The draft report reflects these two statements. However, we also stated in the draft report that if a contractor determines that only a small percentage of the work will be subcontracted, calculation of small business subcontracting achievements as a percentage of that percentage can be misleading. 3. We agree and modified the final report. 4. We modified the final report to acknowledge that prior to fiscal year 2000, DOE was able to include the small business subcontracting achievements of its facility management contractors toward the department’s small business prime contracting goals. However, while it may have been an acceptable past practice for the facility management contractors to exclude more subcontracts under these conditions, the practice should have been discontinued in fiscal year 2000. 5. In the draft report, we acknowledged that DOE is in the process of revising existing guidance to clarify oversight roles and responsibilities and the necessary coordination and integration of oversight efforts between DOE headquarters and field organizations. 6. In the draft report, we stated that effective oversight of facility management contractors’ practices for calculating and reporting their small business subcontracting goals and achievements will require more than providing guidance and training. We believe that regular oversight activities will also be necessary to verify that contractors are complying with this guidance. 7. Our draft report does not state that “more small business subcontracting is always better.” The draft report does discuss federal policy, set out in the Small Business Act, that small businesses shall have the maximum practicable opportunity to provide goods and services to the federal government and its contractors, consistent with efficient contract performance. The draft report also states that there can be several reasons for differences in contractors’ small business subcontracting achievements, including type of work, contract performance goals, and changes in the annual funding for a contract. 8. We disagree that the fiscal year 2004 subcontracting achievement data are premature. Although the facility management contractors have not yet submitted their fiscal year 2004 subcontracting reports to DOE, we obtained this data directly from the 34 facility management contractors so that we could include the most recent results in our report. We obtained fiscal year 2004 data after the end of the fiscal year so that it would reflect the final subcontracting activities for the year. We assessed the reliability of the data for accuracy and completeness (see app. II). In addition, in March 2005, we provided a statement of facts to the contractors for their review to ensure the accuracy of the information. 9. DOE disagreed with our recommendation that, for internal management purposes, the department use data on its contractors’ subcontracting achievements, calculated as a percentage of the dollars obligated to their prime contract with DOE. While the contractors should continue to report their subcontracting achievements as SBA requires, we continue to believe that, for internal management purposes, the data do not provide a true picture of their performance, which is necessary for the department to perform effective oversight.

Appendix IV: GAO Contacts and Staff Acknowledgment

GAO Contacts

Acknowledgments

In addition to the individuals named above, Carole Blackwell, Ellen Chu, John Delicath, Doreen Feldman, Dominic Nadarski, Judy Pagano, Laina Poon, Jeff Rueckhaus, and Ginny Vanderlinde made key contributions to this report.