DHS IT Contracting: Steps Taken to Enhance EAGLE II Small Business Opportunities, but Better Assessment Data Needed
Published: 2015-06-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DHS's EAGLE II is a suite of strategic sourcing contracts, valued at $22 billion, which serves as DHS's mandatory source for information technology services. Strategic sourcing is a process that moves an organization away from numerous individual procurements toward a broader aggregate approach to achieve cost savings and other efficiencies. The small business community has raised questions about strategic sourcing reducing contracting opportunities.
GAO was asked to examine the EAGLE II program. This report assesses (1) key steps DHS has taken to enhance small business participation and (2) EAGLE II goals, performance measures, and progress made to date. GAO reviewed EAGLE II acquisition planning documents and interviewed DHS procurement and small business staff. GAO analyzed federal procurement data from fiscal year 2013 through the second quarter of fiscal year 2015 and assessed reliability by comparing data to a nonprobability sample of 10 EAGLE II contracts selected from a variety of DHS components.
section_title: What GAO Found
paragraphs: Department of Homeland Security (DHS) procurement officials reported taking three key steps to enhance small business participation in the Enterprise Acquisition Gateway for Leading-Edge Solutions II (EAGLE II) contracts:
Creating small business tracks within each of EAGLE II's three lines of business, including socioeconomic set-aside tracks, to exclusively target competitions to small businesses in the first line of business. See table.
Establishing a process to maintain a steady pool of eligible small businesses by reopening the EAGLE II solicitation after requiring businesses that outgrow their small size status to leave the program.
Requiring small business track prime contractors to team only with other small businesses.
As of March 2015, DHS had issued 74 EAGLE II task orders worth an estimated $591 million, almost all of which—94 percent—went to small businesses. However, it is too soon to evaluate the full impact of these steps because only about 3 percent of the anticipated $22 billion in task orders have been issued.
DHS established five goals for EAGLE II and developed performance measures to assess progress in meeting most of them. DHS established performance measures for the three EAGLE II goals related to cost savings and efficiencies through a methodology to assess cost savings, but has not fully set performance measures for the remaining two, relating to (1) the small business socioeconomic goal and (2) enhancing DHS mission capabilities. For its socioeconomic goal, DHS assesses progress via the percentage of the value of orders issued to small businesses. However, DHS does not assess whether use of team members (other small businesses) supports this goal, although DHS procurement officials told us teaming is key to enhancing small business participation. Further, DHS has not set a performance measure for assessing how the use of teaming coordinators contributes to the EAGLE II goal of enhancing DHS's mission capabilities. According to DHS, prime contractors are required to have teaming coordinators identify subcontractors with innovative services. Federal internal control standards highlight the importance of developing measures to compare expected outcomes to actual results. Without such measures, it will be difficult for DHS to have needed information to assess the extent to which the use of team members and teaming coordinators contribute toward their respective EAGLE II goals.
section_title: What GAO Recommends
paragraphs: GAO recommends the Secretary of Homeland Security support implementation of EAGLE II goals by assessing the use of team members and teaming coordinators. DHS did not concur with GAO recommendations, citing actions already taken in soliciting and awarding the contracts. GAO believes there is still value in assessing whether team members are used for task orders, and that DHS's plan to survey EAGLE II users has merit.

section_title: Letter
paragraphs: Since 2005, the Office of Management and Budget (OMB) has placed  increased emphasis on making federal procurement more efficient and  achieving greater cost savings for the taxpayer through strategic  sourcing—a process that moves away from numerous individual  procurements to a broader aggregate approach to achieve cost savings  and other efficiencies—while asking agencies to consider the effects of  strategic sourcing on small businesses. However, because strategic  sourcing can reduce the number of available contracting opportunities,  some members of the small business community have expressed  concern that small businesses may be negatively affected. The  Department of Homeland Security (DHS) established the Enterprise  Acquisition Gateway for Leading-Edge Solutions II (EAGLE II) in fiscal  year 2013 as the successor to EAGLE, which DHS used to procure  information technology (IT) services from 2006 through 2013. EAGLE II is  a suite of strategic sourcing contracts intended to serve as the  department’s preferred source for information technology services,  including software design and development, and has an estimated value  of $22 billion. EAGLE II is also intended to help leverage DHS’s buying  power to reduce procurement costs, while also increasing obligations for  small businesses.
You asked us to review DHS’s implementation of EAGLE II and the steps  DHS took to provide opportunities to small businesses. This report  assesses (1) the key steps DHS has taken to enhance small business  participation and (2) EAGLE II’s overall goals, performance measures,  and progress made to date.
For both objectives, we reviewed EAGLE II acquisition planning  documentation and interviewed staff from the DHS’s Office of the Chief  Procurement Officer (CPO) and Office of Small and Disadvantaged  Business Utilization. To assess DHS’s progress in awarding EAGLE II  task orders to small businesses, we analyzed Federal Procurement Data  System-Next Generation (FPDS-NG) data for fiscal years 2013, 2014,  and the first two quarters of fiscal year 2015. We assessed the reliability  of the FPDS-NG data by comparing information in selected data fields to  contract documentation from 10 EAGLE II task orders and determined it  was reliable for the purposes of this report. We used a nonprobability  sample to select the task orders, choosing contracts that provided us the  opportunity to determine whether DHS staff from a variety of components  had correctly recorded if a small business set aside had been used for the  procurement. To assess EAGLE II’s goals and performance measures,  we reviewed the EAGLE II Business Case to identify goals and  performance measures that DHS specified during acquisition planning,  DHS methodologies for calculating savings, and interviewed CPO  officials. We then reviewed federal internal control standards and  assessed whether DHS had established performance measures for  program goals consistent with these standards. We obtained information  from DHS on its progress in meeting those goals as of March 2015. GAO  is also conducting a broader review on strategic sourcing for information  technology services; that review, expected to be completed in 2015, also  involves EAGLE and EAGLE II in addition to other government agency  contract vehicles.
We conducted this performance audit from November 2014 to June 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: EAGLE II is a strategic sourcing suite of contracts designed to fulfill the  majority of DHS’s needs for information technology services. For  example, EAGLE II is intended to provide computer-based information  systems and other forms of information technology to produce,  manipulate, store, communicate, and/or disseminate information and  enhance mission-critical functions. As one of DHS’s strategic sourcing  initiatives, EAGLE II is intended to increase the efficiency of the  department’s acquisition and mission support capabilities, as well as  improve fulfillment of socioeconomic contracting goals.
To promote the use of EAGLE II, DHS’s strategic sourcing policy requires  that its components use this contract vehicle when procuring information  technology services. In fact, all DHS strategic sourcing contract vehicles,  including EAGLE II, are mandatory for use unless the procurement meets  a specific exception. DHS’s efforts are in response to OMB’s strategic  sourcing memorandum that requires agencies to identify commodities  that could be purchased through strategic sourcing, which helps optimize  performance, minimize price, increase achievement of socioeconomic  goals, and improve vendor access to business opportunities.
EAGLE II is the successor to EAGLE, which DHS used to procure  information technology services from 2006 through 2013. The period of  performance for EAGLE II includes a 5-year base period, from 2013  through 2018, and one 2-year option period, from 2018 through 2020.  EAGLE II contracts are structured as multiple-award Indefinite Delivery  Indefinite Quantity (IDIQ) contracts.used when the government needs flexibility in the timing of orders within a  specified period of time. Indefinite-quantity contracts provide for an  indefinite quantity, within stated limits of supplies or services during a  fixed period. Multiple-award refers to the fact that more than one prime  contractor is awarded a contract. After award of a base IDIQ contract,  products and services are procured through individual task orders during  the contract period. DHS’s CPO was responsible for awarding the EAGLE  II IDIQ contracts, while DHS components, such as the United States  Coast Guard or Transportation Security Administration, can issue task  orders based on their needs.
Indefinite delivery contracts can be  A broad range of DHS IT service requirements are covered through  EAGLE II’s three lines of business, which DHS refers to as functional  categories, such as software design, program support, and verification  services. Within each line of business, DHS established at least two  business tracks, or pathways that will distribute business opportunities,  including tracks reserved for small businesses. These categories include  small disadvantaged businesses such as those in the Small Business  Administration’s (SBA) 8(a) Business Development Program, Service- Disabled Veteran-Owned Small Business (SDVOSB) program and in the  Historically Underutilized Business Zone (HUBZone) program.also has tracks in which both large and small businesses can compete  (unrestricted track). See table 1.
subsections: section_title: DHS Small Business Goals
paragraphs: Congress has set an annual government-wide goal of awarding not less  than 23 percent of prime contract dollars to small disadvantaged  businesses and SBA negotiates specific goals with each agency to help  ensure that the federal government collectively meets this statutory goal,  including those for various socioeconomic categories of small businesses.  DHS’s small business goal for fiscal year 2015, as negotiated with SBA,  is that 32 percent of its prime contract dollars will be awarded to small  businesses department wide with no less than 5 percent to small  disadvantaged businesses, including Section 8(a) small businesses; 5  percent to women-owned small businesses; 3 percent to SDVOSB small  businesses; and three percent to small businesses in HUBZones. Within  DHS, the Office of Small and Disadvantaged Business Utilization  (OSDBU) assists with negotiating DHS’s small business goals with SBA,  participates in small business outreach events and hosts vendor outreach  sessions. In addition, the OSDBU director makes recommendations to  contracting officers as to whether particular contract requirements should  be set aside for small businesses. DHS has not set a specific goal for the  percent of EAGLE II contracts that should be awarded to small  businesses. As explained by DHS CPO officials, however, the value of  EAGLE II task orders issued to small businesses contributes to DHS’s  department wide small business goals.
section_title: Timeline of EAGLE II Awards
paragraphs: Federal regulations require agencies to perform acquisition planning  activities, including market research, for all acquisitions to ensure that the  government meets its needs in the most effective, economical, and timely  manner possible. Development of the EAGLE II contract award was  overseen by the DHS CPO. Within the DHS CPO, the Strategic Sourcing  Program Office is responsible for the DHS Strategic Sourcing Program,  including development and implementation of EAGLE II.
In 2009, DHS CPO staff began acquisition planning for EAGLE II which  included publishing a request for information from small businesses to  gain insight into how best to structure EAGLE to provide opportunities for  small business participation and publishing an additional request to gain  similar perspectives from larger businesses. In addition, DHS’s CPO  sponsored a lessons-learned assessment of EAGLE by obtaining  feedback from DHS contracting staff and EAGLE contractors on a range  of topics including the procurement process for task orders, small  business participation, as well as communications between prospective  contractors and DHS staff.
DHS made its initial EAGLE II source selection decisions in 2012 and  2013, but its final round of source selections was not finalized until 2014  because of a series of bid protests filed by businesses, large and small,  objecting to the selection decisions.award or proposed award of a contract for procurement of goods and  services or a challenge to the terms of a solicitation for a contract.  Contractors filed a total of 56 protests with GAO regarding the EAGLE II  procurement, which were ultimately dismissed, withdrawn, or denied.  GAO dismissed most of the protests because DHS took corrective action  by reevaluating its source selection decisions for the affected business   A bid protest is a challenge to the  tracks. As a result, DHS’s final source selections were delayed. See  figure 1 for EAGLE II’s acquisition timeline and see appendix II for more  information on the bid protests.
In our past work, we have assessed how strategic sourcing initiatives  affect small businesses, including small disadvantaged businesses. In  January 2014, we found that OMB, the General Services Administration  (GSA), and selected agencies, including DHS, had taken steps to  consider small businesses, including small disadvantaged businesses, in  their strategic sourcing efforts. However, data and performance measures  that would provide a more precise understanding of the level of inclusion  were limited. We recommended the agencies, including DHS, collect  baseline data and establish performance measures on the inclusion of  small businesses in strategic sourcing initiatives. DHS concurred with the  recommendation and has implemented it.
section_title: DHS Has Taken Steps to Enhance EAGLE II Small Business Opportunities but It Is Too Soon to Assess Their Effectiveness
paragraphs: According to DHS CPO and OSDBU officials, three key steps enhanced  EAGLE II small business opportunities. Specifically, DHS established (1)  specific competition tracks for small businesses, including those with  specific socioeconomic characteristics, (2) mechanisms to maintain a  steady pool of eligible small businesses, and (3) opportunities for small  businesses to participate as core team members. As of March 31, 2015,  DHS had issued almost all EAGLE II task orders to small businesses, but  it is too soon to determine the full extent to which DHS’s key steps have  enhanced small business participation as only about 3 percent of  anticipated EAGLE II spending has occurred.
subsections: section_title: DHS Took Three Key Steps Based on Lessons- Learned to Increase Small Businesses’ Ability to Compete for Task Orders
subsections: section_title: Establishing Specific Socioeconomic Small Business Award Tracks
paragraphs: A key outcome of the acquisition planning process and lessons-learned  was DHS’s decision to combine three of EAGLE’s five functional  categories into a single functional category (termed functional category 1)  that incorporates a larger range of information technology services, such  as software design and development. Users of EAGLE had experienced  difficulty in determining which functional area related to their requirements  because of overlapping categories. DHS CPO officials estimated that  about 80 percent of EAGLE II spending would go through this new  category.
DHS then established tracks to set aside competition for task orders for  small businesses within each functional category, including those with  specific socioeconomic characteristics. Specifically, DHS created three  individual socioeconomic tracks within functional category 1 for qualifying  8(a), HUBZone, and SDVOSB small businesses and a general small  business track. An OSDBU official told us that the creation of  socioeconomic business tracks was the single most important step that  DHS took to enhance small business participation. Small businesses can  compete for task orders within the small business tracks or compete  against larger businesses within the unrestricted business track. As part  of its lessons-learned from EAGLE, CPO officials concluded that small  businesses were able to successfully compete with large businesses.  Table 2 depicts the number of EAGLE II IDIQ awards by functional  category (line of business) and business track. Each of these contractors  has the potential to compete for task orders placed by DHS contracting  staff for specific requirements.
To further enhance small business opportunities, DHS also established  procedures in the EAGLE II ordering guide—used by component  contracting staff to issue task orders under the contracts—to facilitate the  use of small business set-aside opportunities. For example, after the  contracting officer determines into which functional category a  requirement falls, the contracting officer and the component’s small  business specialist are required to determine if the competition should be  set aside for small businesses. In the case of functional category 1, this  determination expands to whether the procurement should be set aside  for competition within one or all of the small business socioeconomic  tracks. In addition, a DHS small business specialist told us that the DHS  Office of Procurement Operations sponsored meetings between  awardees within each of EAGLE II’s small business tracks and DHS  program managers to promote use of the small business tracks.
section_title: Maintaining Pool of Eligible Small Businesses over Life of EAGLE II
paragraphs: According to DHS CPO and OSDBU officials, a second key step in  enhancing small business participation was creating an “on ramp/off  ramp” mechanism as a means of maintaining an eligible pool of small  businesses throughout EAGLE II’s performance period. DHS CPO  officials explained that, during the course of EAGLE, 9 of 28 small  business IDIQ awardees had been acquired by larger firms and  consequently lost their small business status, effectively shrinking the  pool of small businesses to compete over the contract period. EAGLE did  not have a mechanism to replace these vendors with other small  businesses.
To avoid a similar situation on EAGLE II, DHS developed a mechanism  referred to as an “off ramp,” which according to DHS requires businesses  that have outgrown or otherwise lost their small business status to leave  the EAGLE II program after completing any ongoing task orders. In  contrast, according to DHS CPO officials, under EAGLE, these  businesses were permitted to compete for orders in the unrestricted track.  In addition, DHS created a new mechanism known as an “on ramp,”  which according to DHS, permits DHS to replenish the pool of small  businesses by reopening the EAGLE II solicitation to award additional  IDIQ contracts to small businesses. DHS CPO officials reported they  have not yet needed to use the on ramp/off ramp mechanisms to maintain  an eligible pool of small businesses because there has been no reduction  in the number of small businesses as of March 31, 2015.
section_title: Enabling Teaming Opportunities for Small Businesses
paragraphs: A third key step that DHS CPO and OSDBU officials considered  significant to enhancing EAGLE II small business opportunities was  specifically permitting all prime contractors to formally team with  subcontractors, referred to as core team members, to compete for  contract awards. As part of the EAGLE II solicitation process, DHS  permitted prime contractors to designate up to four core team members in  their proposals to more fully meet the broad array of information  technology service requirements. DHS also instructed that businesses  competing for IDIQs in the small business tracks may team only with  other small businesses. CPO officials also noted that the use of teaming  on EAGLE II allowed prime contractors to designate team members for  the purpose of strengthening their offers. For example, the solicitation  allowed prospective contractors to provide information on the core team  members—such as past performance—that DHS would evaluate and  consider as part of the award decisions. According to DHS OSDBU  officials, teaming was a way to allow small businesses that did not receive  an EAGLE II award to still participate at the task order level in meeting  DHS information technology services requirements, as a team member.  Further, DHS CPO officials said that the success of teaming  arrangements in performing task orders was a lesson-learned from  EAGLE, in which teaming arrangements with subcontractors were  encouraged, but only after award of the IDIQ contracts.
Our analysis of DHS CPO teaming data indicates that 127 of the 150  IDIQ contracts included teaming arrangements. Almost all of the small  business prime contractors—60 of 64—used teaming, as did many of the  86 large businesses in the unrestricted business tracks. According to  officials, DHS requires the original core team to remain together for the  life of the contract including the base and option periods except for  extraordinary circumstances and with the DHS contracting officer’s  consent.
section_title: DHS Has Issued Most Task Orders to Small Businesses, but It Is Too Soon to Evaluate the Full Effect of the Key Steps
paragraphs: As of March 31, 2015, DHS issued 74 EAGLE II task orders with a total  estimated value of $591 million, almost all of which—about 94 percent— was issued to small businesses. Over half of the total amount went to  HUBZone businesses. However, this amount represents only about  three percent of EAGLE II’s potential value of $22 billion. Figure 2 depicts  the total value of EAGLE II task orders issued by functional category and  business track from fiscal year 2013 through March 31, 2015 and the  percentage of task orders issued by business track.
DHS CPO officials stated the high percentage of task orders issued within  functional category 1 is an early indication that the steps they took to  enhance small business utilization are working as intended. However they  also noted that the trend of awards to small businesses was unlikely to  continue at the same levels over the life of the contract. They explained  that delays in awarding some of the contracts within the unrestricted  portion of functional category 1 may have contributed in particular to the  high level of HUBZone task orders. DHS did not make its final source  selection decisions for IDIQ awards within functional category 1’s  unrestricted business track until April 2014 due to a series of bid  protests. In the meantime, according to DHS CPO officials, DHS  contracting officers made greater use of functional category 1’s small  business tracks, especially the HUBZone track which was available as  early as July 2013. DHS CPO officials reported that they anticipate an  uptick in EAGLE II task orders later in fiscal year 2015 and that with the  availability of functional category 1’s unrestricted track, they expect large  businesses to win a higher amount of task orders going forward. At the  same time, however, the officials expect the overall percentage of task  orders issued to small businesses to be higher for EAGLE II than for  EAGLE. DHS officials reported that about 10 percent of EAGLE’s  obligations were issued to small businesses over the 7 years of that  contract vehicle.
section_title: DHS Established Performance Goals, but Has Not Fully Assessed Progress for Some Goals
paragraphs: DHS set five goals and objectives for EAGLE II and set performance  measures for most of the goals, as shown in Table 3.
DHS has not fully set performance measures for two of DHS’s stated  goals for EAGLE II: fulfillment of socioeconomic goals and enhancing  DHS mission performance capabilities. According to DHS strategic  sourcing guidance, staff are to collect and review performance data to  ensure program objectives are met. Further, federal internal control  standards state that the ability to compare actual performance to planned  or expected results can help managers identify and analyze significant  differences in expected outcomes. However, as reflected in table 3, DHS  does not have performance measures or collect information that would  allow them to assess the extent to which: (1) use of core team members  may be furthering EAGLE II’s socioeconomic goals and (2) the activities  of teaming coordinators may have furthered DHS’s goal of enhancing its  mission capabilities by finding subcontractors with new and innovative  services. DHS CPO officials explained that they considered inclusion of  core team members and teaming coordinators in the EAGLE II solicitation  to be sufficient to meet goals. At the same time, however, they  recognized that tracking team member use at the task order level would  be difficult. For example, the officials noted that almost five years have  passed since the time of the original EAGLE II offers in 2011, and that the  contractors may now not have the same relationships with their core team  members. In terms of teaming coordinators, DHS CPO officials told us  they have received informal feedback that teaming coordinators are  helpful in locating subcontractors with innovative technologies. However,  as indicated by the market research during acquisition planning for the  unrestricted business track, vendors reported mixed views to DHS as to  whether companies should be required to have teaming coordinators, for  example, questioning the additional costs. Without establishing  performance measures to gauge progress, and collecting needed  information, DHS will not be able to fully assess if its goals for EAGLE II  are being met.
We also identified a potential barrier to DHS’s ability to measure the full  extent of small business participation. DHS does not consistently record  the use of set asides within FPDS-NG, which may limit DHS’s ability to  accurately track results. Within FPDS-NG, the government’s procurement  database, the “type of set-aside” field is a key data element that  contracting staff are supposed to check to indicate a small business set- aside award. In our assessment of contract documentation, we found that  contracting staff did not correctly record the use of set-asides for seven of  the eight task orders awarded to small businesses in our sample. Without  correct FPDS-NG data, DHS and others will not be able to correctly  assess the extent to which DHS has used its small business tracks to set  aside task orders for small business competition. Based on the results of  our assessment, data in the FPDS-NG understates the extent to which  DHS staff have set aside task orders to promote small business  participation. During the course of our review, DHS CPO officials  corrected the errors in the seven contracts we identified after we brought  this issue to their attention. The officials agreed to take additional steps to  correct this problem going forward by creating an automated process to  check for coding errors. CPO officials are also planning to issue additional  guidance.  In addition, one DHS component reported that it was providing  additional training to contracting officers, including step-by-step  instructions on how to report set asides, to avoid future reporting errors.
section_title: Conclusions
paragraphs: DHS took a number of actions aimed at enhancing small businesses  participation in EAGLE II and a large proportion of the dollars obligated so  far have gone to small businesses. However, only a fraction of the  anticipated $22 billion in spending has occurred to date, so it is too soon  to fully assess the effectiveness of steps taken. In this early stage, DHS  has an opportunity to improve how it measures performance against its  stated goals. Specifically, the department has established the use of core  team members as a key element in enhancing opportunities for small  businesses that did not receive prime contracts and teaming coordinators  to improve DHS mission performance, but does not collect information on  the extent of use of these entities. Without this information, it will be  difficult for DHS to assess their contributions towards its EAGLE II goals  such as promoting small business participation. Further, given the mixed  reviews from vendors as to the benefits of teaming coordinators, DHS  could better position itself to ensure that new and innovative contractors  support DHS over the life of the EAGLE II contracts. Evaluating the  usefulness of these activities could also be beneficial to future DHS  strategic sourcing vehicles seeking to employ teaming approaches.
section_title: Recommendations for Executive Action
paragraphs: To support effective implementation of EAGLE II goals for enhancing  small business participation and access to innovative technology, we  recommend that the Secretary of Homeland Security direct the Office of  the Chief Procurement Officer to take the following two actions:
Collect and review information on the use of small business core team  members to determine whether their use is helping to increase small  business participation.
Collect and review information to assess how teaming coordinators  have provided DHS with greater access to innovative technology.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this product to the Secretary of Homeland Security  for comment. In its written comments, reproduced in appendix III, DHS  acknowledged our recognition of the key steps it has taken to increase  small business participation but did not concur with our two  recommendations.
Regarding our recommendation that DHS collect and review information  on the use of small business core team members to determine whether  their use is helping to increase small business participation, DHS stated  that the actions it took during the solicitation phase as well as the ordering  procedures and safeguards included in the awarded contracts have  already achieved this goal and addressed this recommendation.  While  these actions are important, monitoring contractor use of their core team  members post-award is also significant.  As DHS notes in its letter, use of  small business core team members was intended to maximize  opportunities for companies that may not have individually been  competitive. Our report shows that this approach was largely successful  in the solicitation phase since almost all small businesses that were  awarded EAGLE II contracts took advantage of this opportunity and had  at least one small business core team member. However, being a small  business core team member does not guarantee that the company will  actually be used to conduct work. As a result, the impact of these efforts  will be unknown if DHS does not gather information to monitor whether  small business core team members are actually used to conduct work at  the task order level.  For example, as noted in the report, of ten task  orders we reviewed, only 2 prime contractors used a core team member  to conduct work. Further, such information would also prove helpful to  DHS when planning for EAGLE II’s successor, just as EAGLE II benefited  from the lessons-learned from EAGLE contractors. Given that the base  period of performance for EAGLE II ends in 2018, DHS is likely to begin  its acquisition planning for EAGLE II’s successor in just a few years.
Regarding our recommendation that DHS collect and review information  to assess how teaming coordinators have provided DHS with greater  access to innovative technology, DHS explained that it did not concur  because it has no valid means to evaluate a teaming coordinator’s  individual efforts in bringing innovative products and services to EAGLE II  task orders. Nevertheless, DHS reported its plans to survey the DHS  Chief Information Officer community, as the principal user of the EAGLE II  contracts, to indicate whether they are seeing new and innovative  solutions and services in the proposals received. While this outcome may  not link directly to contractor teaming coordinator activities, it could  provide DHS with useful insights. To the extent that the survey obtains  information on how contractor proposals have contributed to enhancing  DHS’s access to new and innovative solutions and also seeks information  on factors that may contribute to the results, it has the potential to satisfy  our recommendation. Again, this information would prove useful as DHS  plans for the EAGLE II successor contracts, especially since the  contracting community reported mixed reviews about teaming  coordinators.
DHS also provided technical comments that we incorporated into the  report as appropriate.
We are sending copies to the Secretary of Homeland Security,  appropriate congressional committees, and other interested parties. In  addition, the report will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or by e-mail at mackinm@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. Key contributors to this report are  listed in appendix IV.
section_title: Appendix I: Objectives, Scope and Methodology
paragraphs: Enterprise Acquisition Gateway for Leading-Edge Solutions II (EAGLE II)  is a suite of strategic sourcing contracts, which serves as the Department  of Homeland Security’s (DHS) preferred source for procuring information  technology services. Our objectives were to assess (1) the key steps  DHS has taken to enhance small business participation and (2) EAGLE  II’s goals and performance measures and progress made to date.
To identify and describe the key steps DHS took to enhance small  business participation on EAGLE II prior to awarding the Indefinite  Delivery Indefinite Quantity (IDIQ) contracts, we reviewed DHS  acquisition planning documentation. This documentation, including the  EAGLE II acquisition plan, the market research report, and summary of  lessons-learned, contains the bases for decisions made by DHS’s Office  of the Chief Procurement Officer (CPO) before proceeding with the  EAGLE II solicitation. These decisions included deciding whether to use  EAGLE II rather than other contract vehicles to procure information  technology services, how to structure EAGLE II to increase small  business participation, and how to facilitate the use of teaming  arrangements. We also reviewed the DHS EAGLE II solicitation and  Ordering Guide to document the final decisions DHS made for  implementing EAGLE II, including how to structure the business tracks  and how ordering procedures should be used to facilitate setting aside  task orders for competition within the small business tracks. To better  understand DHS’s decision making during acquisition planning, we  interviewed staff from the DHS CPO about the planning process and the  steps they considered as key to enhancing small business participation.  We also interviewed the Director of the DHS Office of Small and  Disadvantaged Business Utilization Office (OSDBU), and a small  business specialist from DHS’s Office of Procurement Operations, to  obtain their perspectives on which steps were key to enhancing small  business participation. To obtain an additional small business  perspective, we reviewed correspondence submitted by a small business  advocacy group and interviewed its representative. In addition, to  understand how a series of bid protests delayed finalization of EAGLE II  IDIQ awards, we analyzed DHS bid protest data to identify the dates on  which the protests were filed and resolved. We also analyzed EAGLE II  source selection decisions to identify the dates on which DHS finalized its  selection decisions for each EAGLE II business track.
To determine the number and estimated value of EAGLE II task order  awards as well as the business size of the recipients, we analyzed  Federal Procurement Data System-Next Generation (FPDS-NG) data for  fiscal years 2013, 2014, and the first two quarters of fiscal year 2015. We  assessed the reliability of the FPDS-NG data by (1) performing electronic  testing of the required data fields, (2) reviewing data documentation and  validation rules, and (3) comparing data for selected data fields, including  the award date, total dollar value, and type of set-aside, to contract  documentation for 10 EAGLE II task orders. We selected a non- probability sample of task orders, selecting contracts that provided us the  opportunity to determine whether DHS staff from a variety of components  had correctly recorded if a small business set aside had been used for the  procurement. Our task order selection was made using November 2014  FPDS-NG data, at which time DHS had awarded 54 task orders. We  concluded that while the FPDS-NG data was reliable for the purposes of  assessing the number and estimated value of task orders as well as the  business size of the recipient, it was not reliable for the purpose of  determining whether DHS had set aside the task order award for small  business competition due to the errors we identified and discussed in our  report. As a result, our analysis reflects the estimated values of awards  made to EAGLE II prime contractors based on the specific functional  category business track for which they won their IDIQ award. DHS CPO  officials corrected the errors in the 7 contracts we identified, and agreed  to take additional steps to correct this problem, as discussed in this  report.
To determine the number of prime contractors that used core team  members when making offers for the IDIQ awards, we analyzed  information DHS had compiled for each IDIQ award. We assessed the  accuracy of this information by identifying whether both DHS’s information  and the listing of core team members on the Federal Business  Opportunities website listed one or more team members for 12 of the 150  IDIQ awardees. We selected 4 contracts from Functional Category 1’s  unrestricted business track, and 1 from each of the other eight business  tracks, for a total of 12 contracts. We determined that the DHS  information on core team members was reliable for the purposes of this  report. In addition, to identify if prime contractors used one or more of  their core team members on the 10 task orders, we then reviewed DHS  information indicating whether the prime contractors had used one or  more of their core team members in performing work under a task order.
To identify EAGLE II’s goals and performance measures and progress  made to date, we assessed the January 2011 EAGLE II Solutions  Business Case to identify EAGLE II goals and performance measures  that DHS specified during acquisition planning. We interviewed DHS  CPO officials to obtain their perspectives on the goals, related activities,  and performance measures and to refine our understanding of what they  intended EAGLE II to accomplish. To assess the progress EAGLE II has  made towards meeting each goal, we undertook a range of activities.  First, to identify DHS progress in meeting its socioeconomic goals, we  analyzed FPDS-NG data to assess the value of contracts awarded to  small businesses as of March 31, 2015, the most recent quarter of FPDS- NG data. Second, to identify how DHS measures cost savings, including  fees avoided by using EAGLE II rather than other strategic sourcing  vehicles and enhancements to EAGLE II’s acquisition efficiencies, we  obtained and reviewed the strategic sourcing saving methodology  sponsored by the DHS Strategic Sourcing Program Office, and obtained  saving estimates from this office. Third, to verify DHS’s assertion that it  had increased the efficiency of the acquisition process by consolidating  ordering information for its staff on the EAGLE II intranet, DHS provided  us with access and we directly examined the contents of the EAGLE II  intranet. We then reviewed federal internal control standards and  assessed whether DHS had established performance measures for  program goals consistent with these standards. We obtained information  from DHS on its progress in meeting those goals as of March 2015.
GAO is also conducting additional work about the benefits of strategic  sourcing for information technology services, including opportunities that  may exist to leverage the government’s buying power and barriers to the  use of strategic sourcing. This review, expected to be completed in 2015,  also involves EAGLE and EAGLE II in addition to other government  agency contract vehicles.
We conducted this performance audit from November 2014 to June 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Number of Bid Protests and Outcomes by Business Track
paragraphs: The Department of Homeland Security (DHS) made its initial EAGLE II  source selection decisions in 2012 and 2013, but its final round of source  selections was not finalized until 2014 because of a series of bid protests  filed by businesses, for the unrestricted and small business tracks,  objecting to the selection decisions. A bid protest is a challenge to the  award or proposed award of a contract for procurement of goods and  services or a challenge to the terms of a solicitation for a contract. A total  of 46 large and small businesses filed a total of 56 bid protests with GAO.  These protests took place from August 2012 through July 2014 and were  all resolved by late 2014. The number of bid protests varied by functional  category and by business track, with the highest number of protests (21)  filed by large businesses in functional category 1’s unrestricted business  track. Most of the bid protests (44 of the 56 bid protests filed) were  dismissed. GAO sustained none of the cases. According to DHS officials,  the agency took corrective action for the dismissed cases by reevaluating  its source selection decisions. Table 4 depicts the number of bid protests  businesses filed within each of EAGLE II’s business tracks and the  outcomes of each.
section_title: Appendix III: Comments from the Department of Homeland Security
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, W. William Russell, Assistant  Director; Peter Anderson, Sonja Bensen, Virginia Chanley, Wendell K.  Hudson, Julia Kennon, and Roxanna Sun made key contributions to this  report.