Department of Energy: Improved Oversight Could Better Ensure Opportunities for Small Business Subcontracting
Published: 2005-05-13T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Federal policy requires that small businesses receive the maximum practicable subcontracting opportunity for providing goods and services to large businesses that contract directly with federal agencies. The Department of Energy (DOE) annually directs almost $20 billion to the 34 "facility management contractors" of which $3.3 billion was redirected to small business subcontractors in fiscal year 2004. DOE negotiates annual small business subcontracting goals with individual contractors and monitors their achievements. GAO was asked to (1) determine the usefulness of the data that DOE uses to monitor subcontracting performance and (2) discuss the actions that DOE has taken to address any problems with the contractors' subcontracting efforts.
section_title: What GAO Found
paragraphs: DOE's facility management contractors' small business subcontracting achievements--reported as a percentage of their total subcontracted dollars--are not useful for monitoring purposes because the reported data overstates subcontracting achievements in two ways: (1) All of the contractor-reported data incorrectly excluded some large-business subcontracts, beyond what federal reporting guidelines allow. Excluding these subcontracts made the percentage of subcontracted dollars going to small businesses appear larger than it would have, if such subcontracts were not incorrectly excluded. If these subcontracts had been included, some contractors said it was likely they would have requested lower goals. (2) Even when all relevant subcontracts are included, the contractor-reported data can still overstate contractors' subcontracting achievements. Because a contractor could decide to subcontract only a small amount of its total federal contract, the portion of subcontracted dollars going to small businesses could, by comparison, appear misleadingly large. As a result, contractor-reported data is not useful to DOE in determining its contractors' actual small business subcontracting achievements or adequately assessing whether small businesses are receiving maximum practicable subcontracting opportunities. DOE has not taken adequate steps to address known problems with the contractor-reported data. Because the data showed that the department was meeting its subcontracting goals, DOE officials were not inclined to closely monitor contractors' practices for calculating their subcontracting goals and achievements. DOE officials were aware in 2002 that the contractors were not following federal guidelines on which subcontracts to include when developing goals. DOE's Small Business Office did provide clarifying information on the requirements, but DOE officials failed to ensure that the guidelines were being followed, and problems continued. In March 2005, DOE issued additional guidance, but it is uncertain whether DOE will ensure that the guidance is followed. These oversight problems occurred, in part, because DOE has not clearly defined the roles, responsibilities, and needed interaction of the various headquarters and field organizations that collectively oversee the contractors' small business subcontracting efforts.

section_title: Letter
paragraphs: The Small Business Act, as amended by the Small Business  Reauthorization Act of 1997, established a federal policy that small  businesses shall have the maximum practicable opportunity to participate  in providing goods and services to the federal government and its direct  contractors, consistent with efficient contract performance. Directing  federal dollars to small businesses can produce cost savings, increase  competition, encourage greater innovation, and enhance small business  capacity. The policy of the federal government is to encourage contracting  with small businesses—both prime contracts, which would be contracts  directly between small businesses and federal agencies, and subcontracts,  which would be between federal prime contractors and small businesses.
The Department of Energy (DOE), the largest civilian contracting agency  in the federal government, spent $22.4 billion on contracts in fiscal year  2004. The majority of this amount—$18.9 billion—was spent on large  contracts for managing DOE’s laboratories, production facilities, and  environmental restoration sites located in various states across the nation.  Under these “facility management contracts,” a contractor is responsible  for performing, managing, and integrating work at a DOE site, often  subcontracting specific portions of the work to other businesses. In fiscal  year 2004, DOE’s 34 facility management contractors subcontracted out  about $6.5 billion, of which about $3.3 billion went to small businesses.
Federal regulations require that DOE’s facility management contractors  develop small business subcontracting plans that set forth a strategy for  providing the maximum practicable opportunities for small businesses to  participate as subcontractors in the work that the facility management  contractors oversee at DOE sites. The Small Business Administration  (SBA) provides guidelines for how contractors should develop small  business subcontracting plans and goals. These guidelines describe the  procedures that contractors should use to officially report their small  business subcontracting goals and achievements. In reporting, contractors  must express the dollars going to their small business subcontractors as a  percentage of the total dollars going to all of their subcontractors— including both large and small businesses as well as government agencies  and nonprofit organizations. Contractors generally must report on all of  their subcontracts, although SBA guidelines allow two exclusions: (1)  subcontracts performed outside of the U.S. or its territories and (2)  subcontracts with an affiliate organization, such as a subsidiary company  or a parent company.
To meet government-wide small business subcontracting goals, SBA  negotiates annual subcontracting goals with each federal agency. The goal  that DOE’s Office of Small and Disadvantaged Business Utilization  (referred to in this report as DOE’s Small Business Office) has negotiated  with SBA is for 50 percent of its total subcontracting dollars to be directed  to small businesses each year. To meet this goal, DOE contracting officers  located at the various agency sites negotiate annual goals with individual  facility management contractors and, with the assistance of small business  program managers, monitor contractors’ compliance with subcontracting  plans as well as progress toward achieving annual small business  subcontracting goals.
Despite the efforts of SBA to meet government-wide annual  subcontracting goals, there have been continuing concerns that small  businesses were not being provided the maximum practicable opportunity  to provide goods and services to the federal government and its  contractors. In March of 2002, the President announced a Small Business  Agenda that proposed steps toward creating a more dynamic environment  where small businesses could flourish, including ways to improve the  access of small businesses to federal contracting opportunities. One of the  strategies was to strengthen federal agency oversight of its contractors’  efforts to comply with small business subcontracting plans.
In this context, you asked us to review the small business subcontracting  achievements of DOE’s facility management contractors. This report  discusses (1) the usefulness of the data reported by DOE facility  management contractors for monitoring contractor performance in small  business subcontracting and (2) the actions that DOE has taken to address  any problems identified with its facility management contractors’ small  business subcontracting efforts.
To determine the usefulness of the data reported by DOE facility  management contractors on their small business subcontracting goals and  achievements, we received and analyzed data on fiscal year 2001 through  2004 subcontracting goals and achievements from DOE’s 34 facility  management contractors, as well as information on how the contractors  developed their goals and calculated their achievements. To assess the  reliability of the data that the contractors provided about their  achievements, we obtained and analyzed information from all 34  contractors about their methods for compiling subcontracting  achievement data and steps they took to ensure the accuracy and  completeness of these data. We also visited 13 of the 34 facility  management contractors, which we selected based on the contractors’  association with the department’s three largest component  organizations—Environmental Management, Science, and National  Nuclear Security Administration (NNSA)—and on the geographic location  of DOE sites. During these site visits, we obtained and analyzed supporting  documents for small business subcontracts reported in the contractors’  fiscal year 2004 achievements, and we obtained documentation on the  dollar value of any subcontracts that the contractors excluded from their  reported achievements for that year. A more detailed description of our  scope and methodology is presented in appendix II. We performed our  work from June 2004 to April 2005 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: DOE has major sites and facilities around the country where the  department carries out its missions, including developing, maintaining,  and securing the nation’s nuclear weapons capability; cleaning up the  nuclear and hazardous wastes resulting from more than 50 years of  weapons production; and conducting basic energy and scientific research,  such as mapping the human genome. DOE relies on facility management  contractors to operate its facilities and accomplish its missions. This  mission work is carried out under the direction of the National Nuclear  Security Administration (NNSA) and DOE’s program offices, including the  offices of Environmental Management and Science.
SBA develops regulations and provides guidance for federal agencies and  their contractors to follow in carrying out the federal policy to provide  maximum practicable contracting opportunities for small businesses,  consistent with efficient contract performance. Under the Small Business  Act and federal regulations, any large business with a federal prime  contract that has a value over $500,000 (or $1 million for construction  contracts) and that has subcontracting possibilities, must have a small  business subcontracting plan, including annual goals for directing  subcontracting dollars to small businesses. SBA has also issued specific  guidance for developing annual subcontracting goals and reporting  achievements to SBA. Although there is no specified percentage goal for  small business subcontracting in law or regulations, SBA’s government- wide goal is that 40 percent of federal subcontracted dollars be directed to  small businesses.
DOE’s facility management contractors carry out the SBA requirements by  developing subcontracting plans and annual goals, and reporting their  achievements to DOE and SBA. Facility management contractors use the  following process:    Develop small business subcontracting plans. Facility management  contractors generally develop small business subcontracting plans as part  of the overall proposal submitted in response to a DOE request for  proposals on a new contract or upon extension or renewal of an existing  contract. The subcontracting plan includes information on the types of  work to be subcontracted and how, and to what extent, the contractor  would provide subcontracting opportunities for small businesses. The  proposed subcontracting plans are reviewed as part of the overall  proposal and are then incorporated into the contract when it is awarded.
Develop annual small business subcontracting goals. Generally, the  facility management contractors perform “make-or-buy” analyses to  determine how much of their work will be subcontracted rather than  performed by the contractors’ own employees. For the portion of the work  the facility management contractor intends to subcontract, the facility  management contractors develop small business subcontracting goals  each year, after considering the proposed funding for the contracts,  upcoming subcontracting opportunities, and the potential for small  businesses to perform the subcontracting work. After an agreement is  reached on the proposed goals, the goals are incorporated into the  contract.
Report small business subcontracting achievements. Facility  management contractors are required to report their small business  subcontracting achievements to SBA twice each fiscal year—as of the  ends of March and September. SBA guidelines specify that small business  subcontracting achievements should be calculated as a percentage of total  dollars subcontracted. Facility management contractors also report this  achievement information to DOE.
DOE’s Small Business Office negotiates the department’s annual small  business subcontracting goals with SBA, coordinates outreach efforts with  the small business community, and works with NNSA and DOE’s program  offices to establish and monitor annual goals. DOE’s Office of  Procurement and Assistance Management and NNSA’s Office of  Acquisition and Supply Management establish policies and guidance for  conducting procurements according to federal and departmental  regulations, and they maintain the information systems on the  department’s prime contracts, including annual dollars provided to each  facility management contractor. NNSA and DOE’s program offices, such as  Environmental Management and Science, are responsible for providing  program oversight and direction to the contractors.
In addition to the oversight responsibilities shared among DOE’s various  headquarters and field organizations, SBA has a role in the oversight of  facility management contractors’ small business subcontracting efforts.  SBA officials in regional offices generally review proposed subcontracting  plans and proposed annual small business goals. Regional SBA officials  also review the reports on small business subcontracting achievements  submitted by facility management contractors, and they may review large  proposed subcontracts to determine if there are potential opportunities for  small businesses. SBA regional offices also perform periodic compliance  reviews of individual contractor’s small business subcontracting efforts,  including validating the data reported by the contractors to ensure that the  contractor is following SBA guidelines.
section_title: Data Reported by Facility Management Contractors Is Not Useful for DOE’s Monitoring of Their Small Business Subcontracting Performance
paragraphs: The data that DOE uses to monitor its facility management contractors’  performance in small business subcontracting generally shows that the  contractors have exceeded their annual subcontracting goals. However,  these data are not useful for monitoring purposes for two reasons. First,  the data reported by the facility management contractors frequently  overstated their subcontracting achievements, by failing to account for all  of their subcontracts, as required by SBA reporting guidelines. As a result,  the percent of total subcontracting dollars going to small businesses  appeared larger than if the contractors had been following SBA’s reporting  guidelines. Second, even if the facility management contractors were  correctly reporting their small business subcontracting achievements, the  method for calculating those achievements does not provide a true picture  of contractors’ performance. The method can make a contractor appear to  be performing well—for example, by directing 80 percent of  subcontracting dollars annually to small businesses—when, in fact, that  percentage is based on a small amount of dollars. When small business  subcontracting dollars are recalculated as a percent of the annual funding  on the facility management contract, the contractor’s actual achievement  can turn out to be relatively low. Because contractor-reported data does  not provide a true picture of the contractors’ performance, the department  cannot meaningfully use the data to monitor performance and determine  whether the contractors are providing small business subcontractors with  the maximum practicable opportunity to provide goods and services to the  facility management contractors, as required by the policy set out by the  Small Business Act.
subsections: section_title: Contractor-Reported Data on Small Business Subcontracting Achievements Frequently Overstated Performance
paragraphs: The contractor-reported data that DOE uses to monitor facility  management contractors’ performance in small business subcontracting  generally showed that contractors exceeded their annual small business  subcontracting goals in fiscal years 2001 through 2004. In fiscal year 2004,  for example, 29 of the 34 facility management contractors reported  exceeding their annual goals when they directed 33 to 88 percent of their  subcontracting dollars to small businesses. These contractors reported  exceeding their goals by as much as 28 percent, whereas the contractors  that reported falling short of their goals for that year did so by as much as  26 percent. Whether or not individual contractors achieved their goals, the  34 contractors collectively reported directing nearly $3.3 billion to small  business subcontractors in fiscal year 2004, representing a 27 percent  increase over the amount they had directed to small businesses in fiscal  year 2001 (see table 1). Additional information on the facility management  contractors’ subcontracting goals and achievements for fiscal years 2001  through 2004 is presented in appendix I.
Although facility management contractors generally reported exceeding  their subcontracting goals, the data are not useful for monitoring the  contractors’ performance, because the contractors’ achievements were  frequently overstated. When calculating subcontracting achievements,  SBA guidelines allow contractors to exclude from their calculations only  those subcontracts performed outside of the U.S. or its territories and  subcontracts with affiliate organizations, such as a subsidiary or parent  company. However, in fiscal year 2004, all 34 of the contractors’ reported  achievements incorrectly excluded other types of subcontracts, beyond  what SBA guidelines allow (see table 2). The facility management  contractors we visited generally told us it was their normal practice to  make these types of exclusions. In making these types of exclusions, the  contractors said they were following their past practices or that DOE  contracting officers had approved the additional exclusions. For example,  officials representing one of these contractors pointed out that, in  excluding subcontracts from the calculation of their subcontracting  achievements beyond what SBA guidance allows, their company was  following the subcontracting plan approved by DOE. The contractor  officials said that they would not have made these exclusions, if DOE had  not permitted them, nor would they have negotiated as high a  subcontracting goal under those circumstances.
Facility management contractors’ practice of incorrectly excluding certain  subcontracts from achievement calculations resulted in overstating their  reported subcontracting achievements by making the achievements  appear larger than they would have appeared, if only the appropriate  exclusions had been made in the calculation. For SBA-reporting purposes,  contractors calculate their subcontracting achievements by expressing the  dollars that they directed to small business subcontracts as a percentage  of the total amount they subcontracted. However, because they incorrectly  excluded subcontracts that generally did not go to small businesses—such  as subcontracts with other facility management contractors—the facility  management contractors incorrectly reduced the total amount that was  subcontracted (the denominator), without affecting the total amount going  to small businesses (the numerator). As a result, the percentage of total  subcontracted dollars going to small businesses appeared larger than it  actually was.
The effect of making incorrect exclusions on achievement calculations is  significant. At the 13 facility management contractors we visited, we  analyzed the fiscal year 2004 dollar amounts associated with the  subcontracts that were incorrectly excluded from the calculation of their  reported subcontracting achievements. These excluded subcontracts  totaled about $887 million. For these 13 contractors, the practice of  incorrectly excluding subcontracts from their reported achievements  resulted in overstating the percentage achievements by as much as 27  percent. Moreover, although 12 of the 13 contractors had appeared, from  their reported achievements, to be meeting or exceeding their  subcontracting goals, when the achievements were recalculated by adding  the incorrectly excluded subcontracts back into the calculation, only 4  contractors would have achieved their goals. The remaining 9 contractors  would have fallen short of their subcontracting goals, by as much as $31.1  million, or about 19 percent. According to contractor officials, it is also  likely that these contractors would not have agreed to as high of a  subcontracting goal, if they had to include these subcontracts (see table  3).
The fiscal year 2004 small business subcontracting achievements reported  by many of the remaining 21 contractors that we did not visit during our  study also are likely overstated, although we did not document the extent  of the overstatement. Based on information these contractors reported to  us, all 21 contractors had incorrectly excluded subcontracts from the  calculations of fiscal year 2004 subcontracting achievements. This practice  will almost always result in overstating actual achievements, because the  subcontracts excluded from the calculations are generally not awarded to  small businesses.
section_title: Small Business Subcontracting Reporting Method Does Not Provide a True Picture of Facility Management Contractors’ Performance
paragraphs: The method that facility management contractors use to calculate their  small business subcontracting achievements for SBA reporting purposes  does not present a true picture of a contractor’s small business  subcontracting because achievements are not calculated in relation to the  total value of the facility management contract. Rather, the method for  calculating small business subcontracting achievements is based on a  quantity determined by the facility management contractors—namely, the  portion of total work under their DOE contract that they intend to  subcontract out. Thus, a contractor can appear to be performing well—i.e.,  directing 80 percent of its subcontracting dollars to small businesses—but  that percentage is not very informative about the actual amount of  subcontracting occurring with small businesses.
In our view, calculating small business subcontracting achievements in  this manner invariably presents a misleading picture. For example, in  fiscal year 2004, contractors at two of DOE’s weapons production facilities  reported substantially different small business subcontracting  achievements—one reported directing 80 percent of its subcontracting  dollars to small businesses (or $92.6 million) while the other reported 57  percent (or $109.9 million). However, these percentages hide the fact that  the first contractor subcontracted out proportionally less of the work of  the facility management contract than the second contractor, despite the  fact that both facility management contracts were similar in size.  Recalculating the subcontracting achievements as a percentage of the  amount of funding directed by DOE to each facility management contract  for the year reveals that the actual subcontracting achievements were  quite similar—21 and 23 percent respectively of the contracts’ annual  funding. Thus, the contractor that appeared, from the data that DOE uses,  to be significantly outperforming the other was actually devoting a slightly  smaller portion of its facility management contract dollars to small  business subcontracting.
Recalculating the subcontracting achievement data as a percent of the  annual amount funded by DOE for each facility management contract  provides a very different picture of the contractors’ performance. Some  contractors that appeared to be outperforming most other contractors  were ranked near the bottom of the list when their achievements were  measured as a percent of the annual funding on their contract. For  example, the reported subcontracting achievement in fiscal year 2004 of  the contractor at the Princeton Plasma Physics Laboratory appeared to be  the second highest among the 34 contractors. However, after we  recalculated the achievements as a percentage of annual contract funding,  the contractor’s performance dropped significantly to 28th out of 34  contractors (see table 4). Conversely, 6 of the 17 contractors that were in  the bottom half of the rankings of reported subcontracting achievements  moved to the top half, after we recalculated the achievements as a percent  of the annual amount funded on each facility management contract.
While a contractor may be subcontracting a smaller percentage of its  facility management contract to small businesses than other facility  management contractors, there can be several reasons for differences in  contractors’ small business subcontracting achievements, which should be  examined before concluding that a particular contractor is  underperforming. For example, DOE and contractor officials told us that  the type of work contractors perform at DOE sites is an important factor  in determining the potential for small business subcontracting. Our  analysis of the 34 facility management contractors’ fiscal year 2004 small  business subcontracting achievements suggests that this may be true. We  found that the 16 contractors primarily involved in environmental cleanup  or weapons production at DOE sites generally directed a greater  percentage of their contracts’ annual funding to small business  subcontracting than the 16 contractors at DOE research laboratories.  Contractor and DOE officials said that opportunities for small business  subcontractors tend to be more limited at research laboratories than other  types of DOE facilities, because research activities are often integrated,  and fragmenting those activities among several contractors could  negatively impact the work. Officials said that other types of work, such as  facility construction or cleanup projects, generally have more  subcontracting potential and can often be structured at such a scale that  small businesses could perform the work.
Contract performance goals can also affect opportunities for small  business subcontracting. For example, small business subcontracting  opportunities diminished somewhat at an environmental cleanup site in  Oak Ridge, Tennessee, after DOE and the facility management contractor,  Bechtel Jacobs, renegotiated the contract to include more aggressive time  lines and cost limitations for completing the work. In response to the  changes in its contract, Bechtel Jacobs began subcontracting less of the  work, including work that small business subcontractors had previously  performed. DOE and contractor officials told us that the company wanted  more control over the work to better ensure meeting the new time lines  and cost limitations. Officials from local small businesses that subcontract  with Bechtel Jacobs said that while they did not disagree with Bechtel  Jacobs’s strategy for dealing with the changes to its contract, the small  businesses had to adapt to a new business environment in which less  subcontracting work would be available under that facility management  contract.
Although differences in the type of work and contract performance goals  may account for some of the differences in facility management  contractors’ performance in small business subcontracting, there may be  other reasons for those differences. For example, contractor officials said  that the timing of projects or changes in the annual funding for a facility  management contract could affect opportunities for small businesses to  participate in work that is generally conducive to small business  subcontracting. In addition, contractors at sites with similar missions and  similar types of work can vary substantially in the amount of funding that  they direct to small business subcontractors. For example, contractors at  the Sandia and Los Alamos National Laboratories in New Mexico ranked  highly among the 34 facility management contractors—8th and 12th,  respectively—in the percentage of their contracts’ annual funding that  they directed to small businesses in fiscal year 2004. In contrast, the  contractor at the Lawrence Livermore National Laboratory in California,  which conducts work similar to that of the two New Mexico laboratories,  ranked much lower—27th among the 34 contractors. Contractor and DOE  officials with whom we met suggested reasons for such differences in  performances, for example, differences in the availability of small  businesses that are capable of providing the needed goods and services or  differences in the facility management contractors’ commitment to  providing small businesses with the maximum practicable opportunity to  participate in the work.
Furthermore, actions by DOE to increase its direct, or prime, contracting  with small businesses may reduce the subcontracting opportunities  available for small businesses. For example, in an effort to increase the  amount of prime contract dollars going to small businesses, in 2003, DOE  redirected around $30 million of work from the facility management  contract for managing and operating the Strategic Petroleum Reserve in  Louisiana and awarded it as a separate prime contract to a small business.  Before DOE redirected the work, a contractor official said that the facility  management contractor responsible for performing the work had been  subcontracting most of it to small businesses. The official further stated  that, as a result, DOE and the facility management contractor negotiated a  lower small business subcontracting goal for fiscal year 2004 of 27.7  percent of the total dollars subcontracted, representing a 21.3 percent  drop from the 49 percent goal at the end of fiscal year 2003. Although the  facility management contractor exceeded its subcontracting goal in fiscal  year 2004, the total dollars available for subcontracting decreased by  around 12 percent ($6.1 million), and the portion going to small businesses  decreased by 9.3 percent ($1.8 million).
Because the subcontracting achievement data that DOE relies on does not  provide a true picture of the facility management contractors’  performance in small business subcontracting, the department cannot  meaningfully use those data to monitor performance. Without data  providing a truer picture of contractors’ performance, DOE lacks a basis  for knowing whether differences in performance truly exist. Moreover, the  department also lacks a basis for understanding whether differences in  performance are legitimate—such as stemming from the types of work  performed under a facility management contract—or are the result of  differences in contractors’ level of commitment to small business  subcontracting.
Having recognized potential weaknesses in contractor-reported  achievement data, at least one federal agency we contacted is taking a  more innovative approach for using data to monitor subcontracting  efforts. The National Aeronautics and Space Administration (NASA), for  internal management purposes, calculates small business subcontracting  goals and achievements for its prime contractors as a percentage of the  annual funding for the prime contracts. NASA’s small business office said  it uses this method to oversee contractor performance because it (1)  presents a truer picture of actual achievements and (2) provides a better  basis to evaluate a contractor’s maximum practicable opportunity to  subcontract with small businesses. This office said the data the  contractors report to SBA can be misleading, in that a large percentage  achievement can be reported even though a relatively small amount of  dollars may actually be going to small business subcontracts.
Although DOE continues to base its facility management contractors’  subcontracting goals on the total dollars subcontracted, for at least one of  those contractors, the department’s Environmental Management program  plans to use a small business subcontracting approach that is similar to  NASA’s approach. During a recent solicitation for bids for an  environmental cleanup contract at the Hanford Site’s River Corridor  Project in Washington State, the program’s solicitation specified that the  winning bidder must subcontract 60 percent of work under the  approximately $2 billion contract, with half of that directed to small  businesses, or potentially about $600 million over the 7-year life of the  contract. According to proposal documents, Environmental Management  took this approach in part because of input from small businesses and  because it would better ensure that small businesses had an opportunity to  participate in a significant portion of the work under the contract.
section_title: DOE Oversight of Small Business Subcontracting by Facility Management Contractors Was Not Adequate to Address Identified Problems with Contractor- Reported Data
paragraphs: DOE had not taken adequate steps to address known problems with the  small business subcontracting achievement data reported by its facility  management contractors. Because misleading achievement data gave the  appearance of a successful small business subcontracting program, DOE  officials did not closely monitor facility management contractors’  subcontracting practices or follow through to ensure that the contractors  complied with SBA reporting guidelines. DOE’s reliance on misleading  data also resulted in a lack of assurance that the facility management  contractors were providing the maximum practicable opportunities for  small business subcontracting. The inadequate oversight was due, in part,  to a lack of clear guidance on how DOE’s various headquarters and field  organizations should coordinate and integrate their efforts to effectively  oversee the small business subcontracting program.
subsections: section_title: Reliance on Misleading Data Undermined DOE’s Oversight
paragraphs: DOE knew there were problems with the data reported by the facility  management contractors but did not take adequate corrective action. This  had several undesirable consequences: (1) it created the false impression  that DOE was meeting its small business subcontracting goals, (2) it  caused errors in recognizing and rewarding contractor performance, and  (3) it undermined efforts to ensure that facility management contractors  were providing the maximum practicable subcontracting opportunities for  small businesses.
False impression that DOE was meeting its small business  subcontracting goals. The misleading data that DOE was using gave the  appearance of a successful small business subcontracting program. As a  result, DOE officials were not inclined to closely monitor facility  management contractors’ subcontracting practices or to fully address  known problems with the achievement data. DOE officials in the small  business and procurement offices told us that they recognized in 2002 that  the facility management contractors were not consistently following SBA  guidelines on what subcontracts could be excluded when calculating small  business subcontracting goals and achievements. In April 2002, DOE’s  Small Business Office provided clarifying information in an e-mail message  to small business program managers both in the DOE program offices and  the facility management contractors. The information emphasized the  importance of consistently following SBA guidelines. Although DOE  provided subsequent training for small business program managers, DOE  officials did not take adequate steps to ensure that the contractors were  correctly applying SBA guidelines. Based on our analysis of fiscal year  2004 subcontracting practices, none of the 34 facility management  contractors fully complied with the SBA guidelines.
In some cases, DOE actually approved the incorrect reporting practices by  its facility management contractors. For example, we found 10 instances  where DOE had reviewed and approved subcontracting plans that  specifically called for incorrectly excluding subcontracts from the  calculations of small business subcontracting goals and achievements. In  two other cases, the facility management contractors specified incorrect  exclusions in annual planning documents for fiscal year 2004. DOE uses  these planning documents to evaluate overall contractor performance for  the year. DOE contracting officers told us that they had approved or  allowed the incorrect practices because they did not believe these  subcontracts could practically go to small businesses anyway. For  example, the facility management contractors may have had existing long- term subcontracts with large businesses to provide information  technology assistance, building maintenance, or other site support  services.
Recent program reviews by DOE’s Small Business Office were also not  sufficient to identify and rectify these incorrect practices. In September  2002, the Secretary directed the Small Business Office to develop a  strategic plan for providing maximum practicable opportunities to small  businesses, including subcontracting by facility management contractors.  As part of its May 2003 strategic plan, the Small Business Office stated that  it would periodically review the small business subcontracting efforts of  the facility management contractors. During fiscal year 2004, the Small  Business Office performed reviews of the small business subcontracting  programs at five of DOE’s facility management contractors. These reviews  focused primarily on determining if the small business subcontracting  plans contained the information required by federal regulations and  whether the contractors met their annual small business subcontracting  goals. The reviews did not evaluate how the facility management  contractors were developing the annual small business subcontracting  goals or whether all appropriate subcontracts were included in the goal  and achievement calculations. DOE Small Business Office officials told us  that, based on the results of our work, they are considering revising their  review methodology to include these steps.
In March 2005, near the end of our review, DOE and NNSA procurement  organizations issued additional guidance on small business contracting  goals. The guidance stressed that the accuracy of contractor-reported data  was vital to the credibility of the department’s performance in small  business subcontracting. The guidance emphasized that, in calculating and  reporting small business subcontracting goals and achievements,  contractors could exclude only the subcontract types allowed by SBA  guidelines. The guidance further states that contracting officers shall  ensure that facility management contractors are aware of these  procedures and do not apply additional exclusions when developing their  subcontracting goals. However, given DOE’s oversight of these practices  in the past, it remains to be seen whether DOE will now follow through  and ensure that facility management contractors comply with this  guidance.
Errors in recognizing and rewarding contractor performance. DOE’s  Small Business Office established a Secretarial Small Business Awards  program to recognize and acknowledge outstanding performance on an  annual basis. One of these awards—the facility management contractor  small business achievement award—is presented each year to the facility  management contractor with the highest percentage increase in  subcontract awards to small businesses from the previous year.
We reviewed the facility management contractor small business  achievement awards DOE presented in fiscal years 2003 and 2004.  Although the fiscal year 2003 award appears to be appropriate, the fiscal  year 2004 award went to a facility management contractor who directed  fewer dollars to small businesses from one year to the next. Specifically,  the data DOE used to make the determination showed that the percentage  of subcontracting dollars going to small businesses increased from 61  percent in fiscal year 2002 to 66 percent in fiscal year 2003, a 5 percentage  point increase. However, during the same period, the total dollars  subcontracted by the facility management contractor dropped from $60.7  million to $52.1 million, and the subcontracting dollars directed to small  businesses dropped from $36.8 million to $34.4 million. If DOE had  measured the change in the contractor’s small business subcontracting  achievements by comparing them with the annual funding on the facility  management contract, DOE officials would have discovered that the  contractor’s performance actually declined by about 7 percent during the  period (from 19.7 percent of annual funding on the contract to 13.0  percent). DOE Small Business Office officials told us that they based this  award on increases in the percentage of subcontracting dollars going to  small businesses rather than changes in the total dollars directed to small  businesses. Officials said using changes in total dollars directed to small  businesses would not be fair to all the contractors because the facility  management contractors with multibillion dollar annual funding on their  contracts would have an advantage in showing increased dollars to small  businesses. While we agree that using dollars to compare the performance  among contractors could favor those with greater annual funding, relying  solely on the percentage of subcontracted dollars going to small  businesses may result in rewarding declining performance, as  demonstrated above.
Undermined efforts to ensure maximum practicable opportunities for  small businesses. DOE’s reliance on misleading data created an  environment in which DOE officials were not inclined to closely monitor  the small business subcontracting program. For example, DOE allowed  facility management contractors to incorrectly exclude subcontracts when  calculating their small business subcontracting achievements, which led to  the incorrect perception that they were achieving their subcontracting  goals. Without accurate and consistent data on the subcontracting efforts  of the facility management contractors, DOE has no way of determining  whether a facility management contractor is providing the maximum  practicable opportunity for small businesses.
Another consequence of these program weaknesses is a perception among  members of the small business community that DOE’s facility  management contractors could do better at providing opportunities for  meaningful subcontracting work. This perception has led to distrust and  criticism of DOE’s facility management contractors by small business  associations and advocates. For example, during a May 2004 congressional  hearing on DOE’s direct contracting with small businesses, small business  advocacy groups raised concerns about whether the level of  subcontracting by DOE facility management contractors reflected  maximum practicable opportunities. Additionally, small business  associations have expressed concerns about what they perceive as unfair  treatment by DOE facility management contractors, such as using small  business subcontractors to perform work that provides little opportunity  for the small business to develop technical capabilities. Small business  advocacy groups that we contacted said that if DOE held the facility  management contractors more accountable for achieving small business  subcontracting results, such as by using data that provided a truer picture  of actual performance, the small business groups would have a basis for  greater confidence in DOE’s small business subcontracting program.
section_title: Oversight Responsibilities for the Small Business Subcontracting Program Were Not Clearly Defined or Integrated
paragraphs: DOE officials did not address problems with the small business  subcontracting program, in part, because the various headquarters and  field organizations that must work together to provide oversight of the  small business subcontracting program did not do so effectively. Oversight  of facility management contractors’ small business subcontracting efforts  includes reviewing and approving facility management contractors’ small  business subcontracting plans and annual goals, monitoring the  contractors’ progress toward achieving small business subcontracting  goals, and taking action when necessary to ensure that the contractors  meet their small business subcontracting goals.
Although DOE contracting officers have final responsibility for providing  direction to the facility management contractors on all aspects of DOE  contracts, other DOE headquarters and field offices, such as the Small  Business Office and the program offices, share responsibilities for  monitoring and overseeing the small business subcontracting activities of  facility management contractors. DOE contracting officers, usually located  at DOE field locations, have primary responsibility for communicating  with facility management contractors on all aspects of the contracts,  including the contractors’ small business subcontracting efforts. DOE’s  Small Business Office has the overall responsibility for managing the  department’s small business programs and monitoring performance  toward meeting annual goals. Since the Small Business Office has no  direct authority over DOE’s facility management contractors, it  coordinates the subcontracting program efforts by using small business  program managers within DOE’s program offices and field locations. The  small business program managers at the field locations review  subcontracting plans and annual goals, and they provide input to the  contracting officers.
Effective oversight requires more than just carrying out these activities,  but also collectively ensuring that the activities identify and correct  problems and that program goals are achieved. Effective oversight was not  happening, in part, because DOE guidance does not clearly define the  roles, responsibilities, and needed interaction of the various headquarters  and field organizations involved. DOE guidance on the small business  program clearly defines only the roles of the contracting officer, which  include review of small business subcontracting plans, annual goals, and  reported achievements. The guidance also states that the contracting  officer is responsible for ensuring that facility management contractors  comply with the SBA guidelines on which subcontracts may be excluded  from their subcontracting goals and achievements. However, the guidance  does not clearly define roles or responsibilities for the Small Business  Office, small business program managers, or others who need to  coordinate their efforts to achieve department goals. Absent this  guidance, we found that the collective efforts of these officials were not  sufficient to identify and correct the problems discussed in this report or  to ensure that program goals were being achieved.
section_title: Conclusions
paragraphs: Creating opportunities for small businesses to participate in providing  goods and services to DOE’s facility management contractors supports  federal policy and produces benefits for both DOE and the small business  community. Because about 85 percent of DOE’s funding currently goes to  its facility management contractors, none of which are small businesses,  the small business subcontracting efforts of those facility management  contractors are of even greater importance to DOE. However, misleading  data has created the false impression that DOE is meeting its small  business subcontracting goals, undermined DOE’s oversight of  subcontracting efforts, and generated mistrust among members of the  small business community. Furthermore, DOE actions to date have not  adequately addressed problems with misleading data. Until the problems  with contractor-reported achievement data are resolved and the program  oversight issues are addressed, DOE cannot ensure that the federal policy  of providing the maximum practicable opportunity for small businesses is  being achieved.
section_title: Recommendations for Executive Action
paragraphs: To ensure that facility management contractors comply with SBA  guidelines and follow a consistent practice for calculating and reporting  small business subcontracting goals and achievements, we recommend  that the Secretary of Energy direct the appropriate officials responsible for  DOE’s Small Business Office and procurement organizations to ensure,  through regular oversight and review activities, that facility management  contractors comply with DOE’s March 2005 guidance on small business  procurement goals.
To ensure that DOE has useful data for managing its small business  subcontracting program and for measuring and comparing contractors’  performance in pursuing the maximum practicable opportunity for small  business subcontracting, we recommend the Secretary of Energy direct  the appropriate officials responsible for DOE’s Small Business Office and  procurement organizations to use, for internal management purposes, data  on facility management contractors’ annual small business subcontracting  achievements calculated as a percentage of the obligated dollars facility  management contractors received that year on their contract with DOE.
To improve DOE’s oversight of the small business subcontracting program  and to provide the maximum practicable opportunity for small businesses  to subcontract at DOE sites, we recommend that the Secretary of Energy  take steps to strengthen oversight of the program, including issuing  guidance clarifying the roles, responsibilities, and necessary interactions  among DOE small business office, program office, and procurement  officials responsible for managing the small business subcontracting  program.
section_title: Agency Comments
paragraphs: We provided a draft of this report to DOE for review and comment. In  written comments the Secretary of Energy generally agreed with the  report and two of the recommendations but disagreed with one of the  recommendations. The department agreed with our recommendation to  strengthen oversight of its small business subcontracting program. DOE  said that it is in the process of revising existing guidance to clarify  oversight roles and responsibilities and the necessary coordination and  integration of oversight efforts between DOE headquarters and field  organizations.
The department also agreed with our recommendation to ensure that the  facility management contractors follow a consistent practice for  calculating and reporting their small business subcontracting goals and  achievements, as outlined in DOE’s March 2005 guidance. In its comments,  DOE outlined several steps it had taken to provide training, clarifying  information, and guidance on the importance of accurate and consistent  small business subcontracting data. However, effective oversight requires  more than providing guidance and training. We believe that regular review  and oversight activities will also be necessary to verify that contractors are  complying with the guidance.
Regarding our recommendation that the department use data on its facility  management contractors’ small business subcontracting achievements— recalculated as a percentage of the dollars obligated on their prime  contract—to internally manage its program, DOE disagreed with the  recommendation and proposed to continue calculating the achievements  as a percentage of the total dollars subcontracted. While we agree that  DOE should continue to follow SBA guidance for SBA reporting purposes,  we continue to believe that this calculation method fails to provide a true  picture of the facility management contractors’ small business  subcontracting performance, which is necessary for effective oversight. As  our report clearly demonstrated, the method can overstate the  performance of contractors that devote a relatively small portion of their  total contract to subcontracting. Thus, to ensure that DOE is providing the  maximum practicable subcontracting opportunities to small businesses,  for internal program management purposes, the department should use  subcontracting achievement data calculated as a percentage of the facility  management contract obligations. As described in our report, one of the  department’s programs intends to use this approach to internally manage  the small business subcontracting goals and achievements for the   $2 billion contract to clean up the Hanford Site’s River Corridor in  Washington State.
DOE also provided technical comments on the facts presented in the  report, which we incorporated as appropriate. DOE’s comments and our  responses are presented in appendix III.
We conducted our review from June 2004 to April 2005, in accordance  with generally accepted government auditing standards. Appendix II  provides details on our scope and methodology.
As arranged with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after the  date of this letter. At that time, we will send copies to the Secretary of  Energy. We will also make copies available to others on request. In  addition, the report will be available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff have any questions on this report, please call me at  (202) 512-3841. Other staff contributing to this report are listed in  appendix IV.
section_title: Appendix I: DOE Facility Management Contractors’ Annual Small Business Subcontracting Goals and Achievements
paragraphs: Subcontracting  achievement  (percent of total  dollars  subcontracting  achievement  (percent of  subcontracted)
Subcontracting  achievement  (percent of total  dollars  subcontracted)
Subcontracting  achievement  (percent of total  dollars  subcontracting  achievement  (percent of  subcontracted)
Subcontracting  achievement  (percent of total  dollars  subcontracted)
Annual funding  received on the  facility  management  contract  subcontracting  achievement  (percent of  annual funding)
Honeywell Federal Manufacturing & Technologies,  Kansas City Plant,  Mo.
Subcontracting  achievement  (percent of total  dollars  subcontracted)
Annual funding  received on the  facility  management  contract  subcontracting  achievement  (percent of  annual funding)
Southeastern Universities Research Association,   Thomas Jefferson National Accelerator Facility,  Va.
Subcontracting  achievement  (percent of total  dollars  subcontracting  achievement  (percent of  subcontracted)
Subcontracting  achievement  (percent of total  dollars  subcontracting  achievement  (percent of  subcontracted)
Annual funding  received on the  facility  management  contract  annual funding)
We omitted subcontracting goal and achievement data for DOE’s Mound Site in Ohio for fiscal years  2001 and 2002, because the contractor that managed and operated the site in 2001 and 2002 is not  the same contractor that did so from 2003 through the duration of our study. While we included  subcontracting achievement data from the previous contractor in the aggregated data for all facility  management contractors in table 1, we did not include those data in this appendix because the fiscal  year 2001 and 2002 data for the Mound Site came from a different source than the rest of the  subcontracting goal and achievement data in this appendix. As a result, we had not assessed the  reliability of the fiscal year 2001 and 2002 data for the Mound Site to be able to publish the  contractor-level subcontracting goal and achievement data.
The facility management contractor managing and operating the Strategic Petroleum Reserve had a  small business subcontracting goal of 47.0 percent for the first half of fiscal year 2003, but the goal  was increased to 49.0 percent for the second half of the fiscal year, after the contractor was awarded  a new facility management contract with DOE for the same site, and they negotiated a new  subcontracting goal. However, for fiscal year 2004, the contractor’s subcontracting goal dropped to  27.7 percent. According to a contractor official, DOE and the contractor renegotiated the lower goal  after DOE removed about $30 million of work from the facility management contract in fiscal year  2003. The facility management contractor had been subcontracting a majority of that work to small  business subcontractors.
section_title: Appendix II: Scope and Methodology
paragraphs: We determined (1) the usefulness of the data reported by the Department  of Energy’s (DOE) facility management contractors for monitoring  contractor performance in small business subcontracting and (2) the  actions that DOE has taken to address any problems identified with its  facility management contractors’ small business subcontracting efforts. To  conduct our work, we contacted DOE and contractor officials, as well as  representatives from the Small Business Administration (SBA) and from  small-business advocacy groups. We also collected data from all 34 of  DOE’s facility management contracts on their subcontracting activities  and took steps to assess the reliability of the contractors’ data, which  included reviews of subcontracting files at 13 of the 34 contractors.
subsections: section_title: Determining the Usefulness of the Facility Management Contractor-Reported Data
paragraphs: To determine the usefulness of the facility management contractor- reported data for DOE monitoring purposes, we requested data from  DOE’s 34 facility management contractors on their annual small business  subcontracting goals and achievements. Specifically, we asked the  contractors to provide the subcontracting percentage-goals that they  negotiated with DOE for fiscal years 2001 through 2004, as well as the  dollar figures comprising those percentages, including the total dollars to  be subcontracted during the year and the portion of subcontracting dollars  they expected would go to small businesses. We also asked the  contractors to provide us with their subcontracting percentage  achievements for fiscal years 2001 through 2004 and the dollar figures  comprising those percentages—namely, the total dollars that were  subcontracted and the small business subcontracting dollars. (Selected  subcontracting goal and achievement data appear in app. I.) Additionally,  in our data request, we asked the contractors to identify the types of  subcontracts that they included or excluded from their small business  subcontracting goals and achievements for fiscal year 2004. Furthermore,  we requested copies of the contractors’ small business subcontracting  plans that were in effect during fiscal year 2004.
We used a data collection instrument to obtain data from the facility  management contractors on their fiscal year 2001 through 2004 small  business subcontracting goals and achievements. In the data collection  instrument, we provided tables for the contractors to enter their  subcontracting information, and we included instructions on how to  complete the tables. The data collection instrument also included a series  of questions on various aspects of the reliability of the data they provided,  as part of our assessment of data reliability described later in this  appendix.
Because we received the requested information from all 34 of DOE’s  facility management contractors, we did not rely on results from any  subset of those contractors to conduct our analysis and, therefore, no  sampling error is associated with our work. However, data gathering  methods, such as the ones we used, may introduce error into the data that  is not associated with statistical sampling, commonly referred to as  nonsampling errors. For example, difficulties in how a particular  instruction was interpreted, in the availability of the information  requested, and in how the data were entered into the tables could  introduce unwanted variation into the data collected. We took steps in the  development of the data collection instrument to minimize these  nonsampling errors. For example, we pretested a draft of the data  collection instrument with 3 facility management contractors. In addition,  we provided the draft to 5 knowledgeable small business and procurement  officials in DOE headquarters. Within GAO the data collection instrument  was reviewed by the Office of General Counsel and by experts in federal  contracting and research methodology. We modified the data collection  instrument as appropriate to reflect the comments and suggestions of the  contractor, DOE, and GAO reviewers, and sent the data collection  instrument to all 34 of DOE’s facility management contractors. We  received responses from all 34 contractors.
We then assessed the reliability of the subcontracting data we received  from each of the 34 facility management contractors. To assess the data  reliability, we (1) analyzed the contractors’ responses to the data reliability  questions that were included in the data collection instrument and (2)  conducted a more detailed review of subcontract files at 13 of the 34  contractors.
First, we reviewed the contractors’ responses to the series of data  reliability questions in the data collection instrument, which addressed  such areas as data entry, data access, quality control procedures, and data  accuracy and completeness. Follow-up questions were added as  necessary. In consultation with a GAO expert in research methodology, we  analyzed the contractors’ responses for weaknesses in data reliability that  would make their data unusable for analysis and reporting purposes. In  their responses, 7 of the 34 contractors discussed minor limitations to  their data control processes that might affect how the data should be  interpreted. For example, 3 contractors stated that their subcontracting  data was both accurate and complete, while noting that it was such to the  extent that they could make it. Another contractor reported that the data it  provided for fiscal years 2001 and 2002 may be less accurate than the data  for fiscal years 2003 and 2004, because prior to 2003, the contractor did  not require that its small business subcontractors self-certify or provide  documents that verify their small business status.
Second, we visited a nonprobability sample of 13 of the 34 facility  management contractors to conduct a more detailed assessment of the  reliability of the fiscal year 2004 subcontracting achievement data that  they provided in response to our data collection instrument. We used  several criteria to select facility management contractors for site visits,  including a contractors’ association with DOE’s three largest component  organizations—Environmental Management, the National Nuclear Security  Administration (NNSA), and the Office of Science—as well as geographic  location, and the type of work performed under the facility management  contract. The three DOE component organizations we focused on control  30 of DOE’s 34 facility management contracts, and their annual budgets  account for over 70 percent of DOE’s overall budget. In selecting NNSA  contractors, we chose both research laboratories and weapons production  facilities, to ensure inclusion of the types of work that NNSA uses to carry  out its missions. In addition, to obtain geographically diverse cases, we  selected contractors from four states in the Western U.S.—California,  Nevada, New Mexico, and Washington—and one Eastern state, Tennessee.  Nearly one-half of the 34 DOE facility management contractors are located  within these five states.
Based on these selection criteria, we made visits to the following 13  facility management contractors:  Battelle Memorial, Pacific Northwest National Laboratory, Wash.;  Bechtel Hanford, Hanford Site, Wash.;  Bechtel Jacobs, Oak Ridge Site, Tenn.;  Bechtel National, Waste Treatment Plant, Wash.;  Bechtel Nevada, Nevada Test Site, Nev.;  Bechtel SAIC, Yucca Mountain Project, Nev.;  BWXT Y-12, Y-12 National Security Complex, Tenn.;  CH2M Hill Hanford, Hanford Site, Wash.;  Fluor Hanford, Hanford Site, Wash.;  Lockheed Martin, Sandia National Laboratories, N. Mex.;  University of California, Lawrence Livermore National Laboratory, Calif.;  University of California, Los Alamos National Laboratory, N. Mex.; and  UT Battelle, Oak Ridge National Laboratory, Tenn.
Collectively, these 13 contractors had subcontracts worth about $4.2  billion in fiscal year 2004 (or 64.2 percent of the total subcontracting  dollars for all 34 facility management contractors). Of that amount, over  $2.1 billion went to small businesses, accounting for nearly two-thirds of  the almost $3.3 billion in subcontract dollars that the 34 contractors  directed to small businesses that year.
During our visits to these 13 facility management contractors, we looked  at the reliability of the fiscal year 2004 small business subcontracting  achievement data that the contractors provided in the data collection  instrument. Even though our data covered fiscal years 2001 through 2004,  for our more detailed review of data reliability, we focused on fiscal year  2004, because we performed most of our analyses on data from that year.
Although individual contractors may manage thousands of small business  subcontracts in a given year, for each of the 13 contractors, we chose  separate nonprobability samples of 5 small business subcontracts—for a  total of 65 subcontracts—that the facility management contractors had  included in the fiscal year 2004 small business subcontracting  achievements. We selected each nonprobability sample from a complete  list of the contractors’ fiscal year 2004 small business subcontracts, which  we obtained prior to visiting the contractors. The criteria we used for  selecting contracts was the subcontract amount and the goods and  services provided. We selected subcontracts that received among the  largest dollar commitments in fiscal year 2004 and that encompassed a  variety of the types of goods and services provided by small business  subcontractors. In total, the 65 subcontracts in our nonprobability samples  accounted for 17.2 percent ($368.1 million) of the $2.1 billion dollars  committed to small business subcontracts in fiscal year 2004 by the 13  facility management contractors.
At each contractor’s facility, we reviewed documents for each of the five  subcontracts, in order to verify the dollar commitments to those  subcontracts in fiscal year 2004, that the contractor included in their  response to the data collection instrument. We also reviewed any  documents certifying the small business status of those subcontractors at  the time the subcontracts were awarded. In addition, to understand the  effect of excluding certain subcontracts from the contractors’ small  business subcontracting achievement calculations, these 13 contractors  provided us with data on the dollar amount of their excluded subcontracts  in fiscal year 2004.
For all 65 of the small business subcontracts we reviewed, we were able to  verify through documents in the subcontract files that the amount of  dollars committed to the selected subcontracts in fiscal year 2004 equaled  the dollar amounts that the facility management contractors had taken  credit for when reporting their small business subcontracting  achievements for that year. Also, for 63 of the selected subcontracts, we  were able to find documents that certified the subcontractor’s status as a  small business at the time the subcontract was awarded. However, for the  2 remaining subcontracts—which were from 2 different facility  management contractors—documents showed that the subcontractors had  been classified as large businesses at the time the subcontract was  awarded. Therefore, these 2 subcontracts should not have been reflected  in those contractors’ fiscal year 2004 subcontracting achievements as  small business subcontracts. Because this only occurred with 3 percent of  the 65 subcontracts (representing 5.7 percent of the total $368.1 million  value of those subcontracts), we did not consider it to be a material  weakness of the subcontracting achievement data. In both instances  where facility management contractors had mistakenly included a large  business subcontract in their small business subcontracting achievements,  the contractors subsequently restated their fiscal year 2004 subcontracting  achievements. The subcontracting achievement data included in this  report reflect the restated amounts.
Based on all of our efforts to assess data reliability, we believe that the  data are sufficiently reliable for characterizing the 34 facility management  contractors’ fiscal year 2004 small business subcontracting achievements.
section_title: Determining Actions Taken by DOE to Address Problems
paragraphs: To determine the actions that DOE has taken to address any problems  identified with its facility management contractors’ small business  subcontracting efforts, we visited officials at the nonprobability sample of  13 of the 34 facility management contractors previously described. We also  met with the field-based DOE small business officials and contracting  officers responsible for overseeing the facility management contractors’  performance in small business subcontracting.
In addition to our site visits, in DOE’s headquarters we interviewed  officials from DOE’s Small Business Office and small business officials  from the major component organizations, as well as DOE and NNSA  procurement officials, to learn about their role and actions for overseeing  the facility management contractors’ small business subcontracting  efforts. From these officials, we obtained policy guidance, results of field  reviews, and other documents related to DOE and NNSA’s oversight  activities. We also interviewed headquarters and field-based officials from  SBA’s Office of Government Contracting to further understand SBA  guidelines for contractor reporting of small business subcontracting data.  Furthermore, we spoke with representatives from three small business  advocacy groups—the East Tennessee Environmental Business  Association, the New Mexico 8(a) and Minority Business Association, and  the Small Environmental Business Action Coalition.
We conducted our work from June 2004 to April 2005 in accordance with  generally accepted government auditing standards.
section_title: Appendix III: Comments from the Department of Energy
paragraphs: The following are GAO’s comments on the Department of Energy’s letter  dated May 3, 2005.
subsections: section_title: GAO Comments
paragraphs: 1.  We acknowledge in the draft report and highlights page that DOE had  taken steps to address the problems with contractor-reported data,  including issuing clarifying information in 2002 and additional  guidance in March 2005. However, we stated that these steps had not  been adequate to ensure that the facility management contractors were  complying with the guidance. The draft report also stated that the  reviews of the five facility management contractors performed by  DOE’s Small Business Office did not include an evaluation of whether  the facility management contractors were following SBA guidelines for  developing small business subcontracting goals.  2.  We agree that the Small Business Act and the Federal Acquisition  Regulation do not require a specific level of subcontracting and that  individual contractors determine the amount of work that will be  subcontracted. The draft report reflects these two statements.  However, we also stated in the draft report that if a contractor  determines that only a small percentage of the work will be  subcontracted, calculation of small business subcontracting  achievements as a percentage of that percentage can be misleading.  3.  We agree and modified the final report.  4.  We modified the final report to acknowledge that prior to fiscal year  2000, DOE was able to include the small business subcontracting  achievements of its facility management contractors toward the  department’s small business prime contracting goals. However, while  it may have been an acceptable past practice for the facility  management contractors to exclude more subcontracts under these  conditions, the practice should have been discontinued in fiscal year  2000.  5.  In the draft report, we acknowledged that DOE is in the process of  revising existing guidance to clarify oversight roles and responsibilities  and the necessary coordination and integration of oversight efforts  between DOE headquarters and field organizations.  6.  In the draft report, we stated that effective oversight of facility  management contractors’ practices for calculating and reporting their  small business subcontracting goals and achievements will require  more than providing guidance and training.  We believe that regular  oversight activities will also be necessary to verify that contractors are  complying with this guidance.  7.  Our draft report does not state that “more small business  subcontracting is always better.” The draft report does discuss federal  policy, set out in the Small Business Act, that small businesses shall  have the maximum practicable opportunity to provide goods and  services to the federal government and its contractors, consistent with  efficient contract performance. The draft report also states that there  can be several reasons for differences in contractors’ small business  subcontracting achievements, including type of work, contract  performance goals, and changes in the annual funding for a contract.  8.  We disagree that the fiscal year 2004 subcontracting achievement data  are premature. Although the facility management contractors have not  yet submitted their fiscal year 2004 subcontracting reports to DOE, we  obtained this data directly from the 34 facility management contractors  so that we could include the most recent results in our report. We  obtained fiscal year 2004 data after the end of the fiscal year so that it  would reflect the final subcontracting activities for the year. We  assessed the reliability of the data for accuracy and completeness (see  app. II). In addition, in March 2005, we provided a statement of facts to  the contractors for their review to ensure the accuracy of the  information.   9.  DOE disagreed with our recommendation that, for internal  management purposes, the department use data on its contractors’  subcontracting achievements, calculated as a percentage of the dollars  obligated to their prime contract with DOE.  While the contractors  should continue to report their subcontracting achievements as SBA  requires, we continue to believe that, for internal management  purposes, the data do not provide a true picture of their performance,  which is necessary for the department to perform effective oversight.
section_title: Appendix IV: GAO Contacts and Staff Acknowledgment
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: In addition to the individuals named above, Carole Blackwell, Ellen Chu,  John Delicath, Doreen Feldman, Dominic Nadarski, Judy Pagano, Laina  Poon, Jeff Rueckhaus, and Ginny Vanderlinde made key contributions to  this report.