Homeland Security: Successes and Challenges in DHS's Efforts to Create an Effective Acquisition Organization
Published: 2005-03-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Department of Homeland Security (DHS) organizations are expected to work together to protect the United States from terrorism. To support this primary mission, DHS has been acquiring billions of dollars worth of goods and services. DHS also has been working to integrate the disparate acquisition processes and systems that organizations brought with them when DHS was created 2 years ago. GAO was asked to identify (1) areas where DHS has been successful in promoting collaboration among its various organizations and (2) areas where DHS still faces challenges in integrating the acquisition function across the department. GAO was also asked to assess DHS's progress in implementing an effective review process for major, complex investments.
section_title: What GAO Found
paragraphs: DHS's disparate organizations have quickly established collaborative relationships to leverage spending for various goods and services without losing focus on small businesses. DHS is using strategic sourcing, that is, formulating purchasing strategies to meet departmentwide requirements for specific commodities, such as office supplies, boats, energy, and weapons. By fostering collaboration, DHS has leveraged its buying power and savings are expected to grow. Also off to a good start is the small business program, whose reach is felt across DHS. Representatives have been designated in each DHS procurement office to help ensure that small businesses have opportunities to compete for DHS's contract dollars. In contrast, lack of clear accountability is hampering DHS's efforts to integrate the acquisition functions of its numerous organizations into an effective whole. DHS remains a collection of disparate organizations, many of which are performing functions with insufficient oversight, giving rise to an environment rife with challenges. Some of DHS's organizations have major, complex acquisition programs that are subject to a multitiered investment review process to help reduce risk and increase chances for successful outcomes in terms of cost, schedule, and performance. Part of the review process features a knowledge-based acquisition approach pioneered by successful commercial firms. DHS's adaptation of this best practices approach, however, does not require two critical management reviews and is missing some key information before decisions are made to invest additional resources. In addition, contractor tracking and oversight is not fully incorporated into DHS policy and guidance. Finally, some aspects of the review process--which has been under revision for many months--need clarification.

section_title: Letter
paragraphs: Since it was established, in March 2003, the Department of Homeland  Security (DHS) has been faced with assembling 23 separate federal  agencies and organizations with multiple missions, values, and cultures  into one cabinet-level department. This mammoth task—one of the  biggest mergers ever to take place within the federal government— involves a variety of transformational efforts, one of which is to design and  implement the necessary management structure and processes for  acquiring goods and services. In January 2003, we designated DHS’s  implementation and transformation as high-risk because of the size and  complexity of the effort and the existing challenges faced by the  components being merged into the department. As it progresses through  the early stages of its merger and transformation, DHS has an opportunity  to put into place the necessary elements to become a 21st century federal  department with a high-quality acquisition organization that effectively  supports its critically important missions.
DHS has some of the most extensive acquisition needs within the U.S.  government. In fiscal year 2004, the department obligated $9.8 billion to  acquire a wide range of goods and services—such as information systems,  new technologies, weapons, aircraft, ships, and professional services. At  the same time, DHS is working to integrate the many acquisition processes  and systems that the disparate agencies and organizations brought with  them.
Given DHS’s complex merger, you asked us to review the department’s  progress in establishing an effective acquisition organization. In response  to your request, we (1) identified areas where DHS has been successful in  promoting collaboration among its various organizations and (2) identified  areas where DHS still faces challenges in integrating the acquisition  function across the department. We also assessed the department’s  progress in implementing an effective review process for its major,  complex investments.
To conduct this work, we assessed information from DHS headquarters  and the department’s principal organizations and compared the  information against what our previous work has shown to be best  acquisition practices. We reviewed agency directives, memorandums, and  other documentation; interviewed agency officials; and analyzed agency  systems and processes. For the purposes of this report, the term  acquisition refers to the overall effort to acquire goods and services and  involves a host of activities. Acquisition begins when an agency establishes  its requirements and moves through a process that involves soliciting  providers of goods and services, awarding contracts, monitoring  performance, and handling various contract administration duties. In some  cases, acquisition can include development of a new product, such as a  weapon system or a database system. Acquisition also can be the simple  purchase of existing products. The Federal Acquisition Regulation, which  governs acquisitions within the federal government, defines the term  procurement as being synonymous with acquisition. Appendix I presents  our scope and methodology in more detail. We conducted our review from  March 2004 through February 2005 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: The Homeland Security Act of 2002 created DHS, effective March 1, 2003,  by merging agencies and organizations that specialize in one or more  aspects of homeland security. Some of those specialties are intelligence  analysis, law enforcement, border security, transportation security,  biological research, critical infrastructure protection, and disaster  recovery. The intent behind DHS’s merger and transformation was to  improve coordination, communication, and information sharing among the  multiple federal agencies responsible for protecting the homeland. Critical  to performing the homeland security mission is the effective interaction  between and integration of these agencies and organizations. Table 1  shows DHS’s eight principal organizations and their missions.
Of the 23 entities that joined DHS from other agencies, only 7 came with  their own procurement support. Providing support to the other entities— as well as a number of newly created entities, such as the offices of the  Chief Information Officer and Chief Financial Officer—is an eighth office,  the Office of Procurement Operations (Procurement Operations). That  office was not created until January 2004, almost a year after DHS came  into being. Appendix III lists all of the DHS organizations that receive  contracting support from Procurement Operations. Figure 1 shows the  sources of contracting support for DHS’s principal organizations.
To carry out acquisition effectively across a large federal organization  requires an integrated structure with standardized policies and processes,  the appropriate placement of the acquisition function within the  department, leadership that fosters good acquisition practices, and a  general framework that delineates the key phases along the path for a  major acquisition. An effective acquisition organization has in place  knowledgeable personnel who work together to meet cost, quality, and  timeliness goals while adhering to guidelines and standards for federal  acquisition.
section_title: Initiatives to Leverage Buying Power and Small Business Program Have Helped Foster Collaboration among DHS Organizations
paragraphs: In the 2 years since its creation, DHS has realized some successes in  opening the lines of communication among the various organizations  within the department through its strategic sourcing and small business  programs. Both of these efforts have involved every principal organization  in DHS, along with strong involvement from the Chief Procurement  Officer, and both have yielded positive results. DHS already has begun to  demonstrate that its strategic sourcing program can foster collaboration  across the department and at the same time maximize the department’s  overall buying power. DHS’s small business program has a presence  departmentwide, and according to DHS officials, the department exceeded  its 23 percent small business goal for fiscal year 2004.
subsections: section_title: Strategic Sourcing Program Has Encouraged Partnerships across DHS and Begun to Realize Savings
paragraphs: Under the authority of the Chief Procurement Officer, DHS created a  strategic sourcing group in October 2003 to leverage departmentwide  spending for various commodities. The group brought together diverse  expertise from throughout DHS. To identify commodities with the most  potential for savings, strategic sourcing officials conducted a spend  analysis using available acquisition databases, such as the Federal  Procurement Data System, and input from DHS senior management. The  following 15 commodities were identified as having potential to leverage  the department’s buying power:    aviation,    boats,    business wireless communications,    copiers,    energy,    enterprise software agreements,  facilities,  facilities security,    vehicle fleets,    mail,    office supplies,    professional services,    uniforms, and    weapons.
Consistent with best practices, the strategic sourcing group then  established commodity councils composed of representatives from across  DHS. The commodity councils were assigned responsibility for further  collection and refinement of historical procurement data in order to better  assess future purchasing strategies. Typically, members from the strategic  sourcing group and the DHS organization with the most expertise in a  particular commodity serve as council cochairs. For example, a Coast  Guard official is a cochair for the boats commodity council.
Commodity council cochairs said they were willing to devote time to the  strategic sourcing initiatives because they recognized the unique  opportunity DHS had to move forward to leverage buying power across  the department. Further, the cochairs were virtually unanimous in telling  us that the councils enable stakeholders to build awareness of a particular  commodity and develop strong relationships throughout DHS. They said  the councils foster a sense of community in which the various  organizations can share information, participate in forums, find  commonalities, engage in open and productive communication, and make  smarter and more collaborative business decisions. For example, the  weapons commodity council routinely shares information on ammunition.  When one DHS organization is low on ammunition, others help meet the  need. Appendix IV contains more detail on initiatives that several  commodity councils have undertaken.
In fiscal year 2004, 4 commodity councils—office supplies, boats, energy,  and weapons—reported approximately $14.1 million in cost savings and  cost avoidances, and department officials expect the savings to continue  to grow. The savings have resulted from DHS negotiating lower rates with  suppliers and leveraging resources across the department. Figure 2 depicts  the savings trend over a 12-month period. The September 2004 surge  resulted when authorized DHS employees began purchasing pistols  through two large contracts, a strategy spearheaded by the weapons  commodity council.
Some councils are encountering a problem faced by many federal  departments and agencies, namely, a shortage of comprehensive data  upon which to draw an accurate and detailed picture of what is being  spent on certain commodities over time. Strategic sourcing officials and  commodity council members told us that they cannot take full advantage  of spend analyses, nor can they accurately chart historical spending,  because DHS’s acquisition databases do not contain enough procurement  data. The problem is compounded by the fact that when parts of existing  agencies, such as the Immigration and Naturalization Service from the  Department of Justice, joined DHS, detailed information on its spending  history was not available. Without an accurate analysis of how DHS  organizations historically purchased a commodity, council members will  likely continue to rely on a patchwork of estimates, as well as information  from suppliers, to glean information on spending history and develop  purchasing strategies for the future.
In addition, some commodity council members have found it challenging  to balance council duties with the demands and responsibilities of their  full-time positions within DHS. Officials told us that council meetings and  activities sometimes stall because council members must shift attention to  their full-time positions. Many commodity councils did not make much  progress during the last month of fiscal year 2004, we were told, because  council members’ time was diverted to year-end priorities. Our prior work  on strategic sourcing shows that leading commercial companies often  establish full-time commodity managers to more effectively manage  commodities. Commodity managers help commodity councils define  requirements with internal clients, negotiate with potential vendors, and  resolve performance or other issues arising after a contract is awarded  and can help maintain consistency, stability, and a long-term strategic  focus.
section_title: Small Business Program Off to Good Start
paragraphs: DHS’s small business program has also had initial successes, reporting  that 35 percent of fiscal year 2004 obligations were awarded to small  business prime contractors, exceeding the department’s goal of   23 percent. Although reporting directly to the Deputy Secretary of DHS,  the Director of the Office of Small and Disadvantaged Business Utilization  works closely with the Chief Procurement Officer to emphasize  throughout the department the important public policy objective of small  business inclusion in acquisition activities. The small business office, in  conjunction with the procurement staff across the department, has created  an outreach program that advises small businesses on ways to market  goods and services to DHS. Small business representatives have been  designated in each DHS procurement office, and each office is required to  submit a forecast of upcoming contract opportunities above $100,000. DHS  posts this information on a Web site so that small businesses can identify  opportunities to do business with the department. The small business  office has conducted extensive outreach to DHS’s business partners  through regular seminars and has established a mentor-protégé program  that is designed to motivate and encourage large businesses to provide  mutually beneficial developmental assistance to small businesses.
The Director of the Office of Small and Disadvantaged Business Utilization  and his staff have also been directly involved in DHS’s strategic sourcing  efforts to help ensure that, even as the department leverages its buying  power, small businesses continue to have opportunities to compete for  contracts. Several commodity councils have developed strategies to  address this issue. The office supplies council has worked out an  arrangement for DHS employees to purchase office supplies from the  Department of Defense’s (DOD) Web-based Emall, where employees can  easily identify and order from small businesses. The head of procurement  in the Immigration and Customs Enforcement organization, working  closely with the weapons commodity council, awarded a contract for half  of the largest pistol procurement in the history of U.S. law enforcement to  a small business. According to the chair of the boats council, the council  plans to consult with the Coast Guard’s small business specialist to  explore future possibilities for providing opportunities to small  businesses.
section_title: Goal of Creating an Integrated Acquisition Organization Is Hampered by Unclear Policy Decisions and Staffing Disparities
paragraphs: DHS’s goal of integrating the acquisition function more broadly across the  department has not been accomplished, and the introduction of a new  policy has been unsuccessful in breaking down barriers to effective  departmentwide management. An integrated acquisition organization is  essential to the department’s success in executing policies and processes  to effectively obligate and administer billions of dollars in acquiring what  DHS needs to accomplish its mission. An October 2004 DHS management  directive emphasizes the need for an integrated acquisition organization  and reiterates the Chief Procurement Officer’s responsibility to manage,  administer, and oversee all acquisition activity across DHS and to establish  a qualified acquisition workforce. In practice, these responsibilities are  spread throughout the department with unclear accountability. Further  hampering efforts to effectively integrate the acquisition function, the  directive provides that the Coast Guard and the Secret Service are  statutorily exempt from its application. We found no reasonable basis to  conclude that the directive could not be made applicable to them. The  various organizations within DHS continue to operate in a largely  disparate manner, with a lack of centralized oversight of compliance with  the department’s acquisition regulation and policies. Staffing disparities  across the procurement organizations have only recently begun to be  addressed. We found that staffing shortfalls led Procurement Operations  to rely extensively on outside agencies for contracting support—often for  a fee—and that this office lacked adequate internal controls to properly  manage this interagency contracting activity. Because of the risks  associated with interagency contracting, we recently designated this  approach as a high-risk issue.
subsections: section_title: Not Clear How Recent Management Directive Will Drive Integration of the Acquisition Organization
paragraphs: In October 2004, the Secretary of DHS signed a management directive  entitled “Acquisition Line of Business Integration and Management.” This  directive, the department’s principal guidance for “leading, governing,  integrating, and managing” the acquisition function, states that DHS will  standardize acquisition policies and procedures and continue to  consolidate and integrate the number of systems supporting the  acquisition function. It directs managers from each organization to commit  resources to training, development, and certification of acquisition  professionals. The directive also highlights the Chief Procurement  Officer’s broad authority, including    management, administration, and oversight of departmentwide  acquisition, financial assistance, strategic sourcing, and competitive  sourcing programs;    promotion of career development and establishment of qualifications,  training, and certification standards for the acquisition and financial  assistance workforce;    development and publication of departmentwide acquisition and  financial assistance regulations, directives, policies, and procedures;    designation of all heads of contracting activities; and    development and maintenance of contracting officer warrant and  financial assistance officer programs, including designation of qualified  persons as contracting officers and financial assistance officers.
However, the directive may not achieve its goal of creating an integrated  acquisition organization, because it creates unclear working relationships  between the Chief Procurement Officer and heads of DHS’s principal  organizations. For example, the Chief Procurement Officer and the  director of Immigration and Customs Enforcement share responsibility for  recruiting and selecting key acquisition officials, preparing performance  ratings for the top manager of the contracting office, and providing  appropriate resources to support the Chief Procurement Officer’s  initiatives. The policy leaves unclear how the responsibilities will be  implemented or what enforcement authority the Chief Procurement  Officer has to ensure that initiatives are carried out. In addition, directors  are only required to “consider” the allocation of resources to meet  procurement staffing levels in accordance with the Chief Procurement  Officer’s analysis. Agreements have not been developed on how the  resources to train, develop, and certify acquisition professionals in the  principal organizations will be identified or funded. In a relatively new  department like DHS, which is still in the process of instituting  procedures, developing a strong organizational culture, and establishing  clear roles and missions, this concept of dual accountability—absent  effective implementing guidance—may not accomplish the intended goals.
The October 2004 management directive does not apply to the Coast  Guard or the Secret Service, further hampering efforts to integrate the  acquisition organization. The Coast Guard is one of the largest  organizations within DHS, with obligations accounting for about $2.1  billion in fiscal year 2004, nearly 23 percent of the department’s total.  According to the directive, the Coast Guard is exempted by statute. We  disagree. We are not aware of any explicit statutory exemption that would  prevent the application of this directive. While several provisions in the  Homeland Security Act would limit the range of management initiatives  concerning the Coast Guard, none of them would appear to be applicable  in this case. For example, the Homeland Security Act requires the Coast  Guard to be maintained as a distinct entity within the department. Other  limitations prevent the transfer of assets, alteration of missions, and  changes in reporting relationships. The act also expressly provides that  “the authorities, functions, and capabilities of the Coast Guard to perform  its missions shall be maintained intact and without significant reduction  after the transfer of the Coast Guard to the department.” We find nothing  in the directive that contravenes these limitations. Nothing in the  document would reasonably appear to threaten the status of the Coast  Guard as a distinct entity or otherwise impair its ability to perform  statutory missions.
We raised the question of statutory exemption with the DHS General  Counsel, who shared our assessment concerning the explicit statutory  exemptions. He viewed the applicability of the management directive as a  policy matter, noting that “the determination of whether the application of  all or part of this  would impact the [Coast  Guard’s] ability to perform its mission is not a legal matter and is more  appropriately made by DHS policy officials.” We agree that DHS officials,  with sufficient reasons, could make a policy decision that a particular  management directive impacts the Coast Guard’s ability to perform its  missions. In this instance, however, we found no evidence that such a  decision had been made.
The directive also provides that the Secret Service is exempted by statute.  As with the Coast Guard, we are unaware of any specific statutory  exemption that would prevent the application of the directive. The  Homeland Security Act requires the Secret Service to be maintained as a  distinct entity within the department. The 2005 Homeland Security  Appropriations Act reiterates this requirement and imposes additional  limitations on altering reporting relationships. Given the nature of the  management directive, we also conclude that there is no apparent reason  to exempt the Secret Service from its application.
section_title: In Environment of Largely Disparate Procurement Organizations, Oversight of Acquisition Activity Lacking
paragraphs: DHS’s principal organizations are, to a large extent, still functioning much  as they did in premerger days with regard to acquisition-related functions.  Embedded within seven of the procurement organizations are, for the  most part, the same contracting staffs that joined DHS from their former  agencies. The eighth organization, Procurement Operations, created to  meet the needs of the many DHS organizations that do not have colocated  procurement support, has a direct reporting chain to the Chief  Procurement Officer.
Until recently, the Chief Procurement Officer, whom DHS’s top leadership  delegated with the key responsibility of ensuring compliance with the  department’s acquisition regulation and policies, had only two staff  members to carry out this duty. Consequently, the department’s  acquisition oversight program relies extensively on self-assessments by  personnel in each procurement organization. A component of the  oversight program is a recent initiative to review DHS acquisition plans,  according to dollar thresholds, and to perform on-site evaluations of each  procurement organization. The fiscal year 2005 budget provided the Chief  Procurement Officer with five additional staff, but it is too soon to tell  whether this number will be adequate to effectively implement the  oversight program. Further, it remains unclear what the result would be if  an organization were found not to be in compliance with DHS’s acquisition  regulation and policies.
Our prior work shows that in a highly functioning acquisition organization,  the chief procurement officer is in a position to oversee compliance with  acquisition policies and processes by implementing strong oversight  mechanisms. Adequate oversight of acquisition activities across DHS is  imperative, in light of the department’s mission and the problems that have  been reported by us and inspectors general for some of the large agencies  and organizations within the department. These reports have highlighted  the lack of important management controls for monitoring contractors and  ensuring efficiencies and effectiveness in the acquisition process. For  example, the Department of Homeland Security Inspector General  reported that during its first year of operation, the Transportation Security  Administration relied extensively on contractors to accomplish its  mission. It also found that contracting officers wrote contracts without  clearly defined deliverables, and on occasion, contractors themselves  were permitted to determine requirements and define deliverables. As a  result, the cost of those initial contracts ballooned. The Transportation  Security Administration is in the process of devising policies and  procedures that require adequate procurement planning, contract  structure, and contract oversight. We have also reported that the former  U.S. Immigration and Naturalization Service did not have the basic  infrastructure—including oversight of procurement activities—to ensure  that its contracting office was effective. DHS’s Inspector General recently  reported that the Federal Emergency Management Agency discovered it  has not been reporting or tracking procurements handled by its field  offices.
section_title: Challenges to Effective Implementation of Acquisition Workforce Plan
paragraphs: In July 2003, we recommended that DHS develop a data-driven assessment  of the department’s acquisition personnel, resulting in a workforce plan  that would identify the number, location, skills, and competencies of the  workforce. DHS concurred with the recommendation and has drafted a  plan, based on best practices, that defines the acquisition workforce,  focuses on the need for continuous training, and implements a  certification program for contracting officials, program managers, and  contracting officers’ technical representatives. However, the department  faces challenges in implementing the plan.
As part of its acquisition workforce planning efforts, the department has  not conducted an assessment of whether contracting staff within DHS are  appropriately distributed for the varying workloads in each procurement  organization. Our analysis shows that some disparities may exist. We  divided the obligated fiscal year 2004 dollars for each contracting office by  the number of contracting staff. While this approach is limited in that it  does not take into account the complexity of the acquisitions being  performed, it can provide senior leadership with an indication of whether  disparities may exist in the contracting workforce. Figure 3 shows the  amount of contracting obligations per contracting staff within DHS  contracting activities.
As of September 2004, Procurement Operations had only 19 contracting  staff to support a number of DHS organizations that, taken together,  accounted for about 21 percent, or almost $2 billion, of DHS’s fiscal year  2004 obligations. That year, Procurement Operations contracting staff on  the average handled $101 million per employee, whereas the contracting  staff for the Federal Law Enforcement Training Center on the average  handled close to $2.7 million per employee. Disparities such as this may  indicate the need to assess the numbers of contracting staff across the  department to determine whether imbalances exist and whether actions  are needed to correct the imbalances.
Another challenge to effectively implementing the acquisition workforce  plan pertains to the lack of enforcement of DHS’s certification program.  While the plan calls for program managers responsible for acquisitions to  be certified in accordance with DHS’s established training and experience  requirements, in practice, the means to enforce compliance is lacking. In  January 2005, the Director for the Acquisition Workforce Program, within  the Office of the Chief Procurement Officer, determined that only   22 percent of the identified programs in the department had program  managers that had documented that they had the training and experience  requirements for certification. While the Office of the Chief Procurement  Officer has issued a management directive requiring program mangers to  meet the department’s certification and training requirements—and the  number of certified program managers has been increasing— accountability for complying with the certification program rests with the  principal organizations to whom the program managers report. At present,  according to DHS officials, no mechanism is in place to ensure that  program managers take the required training and obtain certification from  the Chief Procurement Officer.
section_title: Struggling with Staffing Shortfalls, Procurement Operations Has Relied on Interagency Agreements but Failed to Follow DHS Guidance in Doing So
paragraphs: Established almost 1 year after DHS was formed and tasked with  providing contracting support to the department organizations that did not  have their own contracting support, Procurement Operations has  struggled to manage its almost $2 billion workload because of staffing  shortfalls. Lacking in-house capability, Procurement Operations has  turned extensively to interagency contracting, and we found that  management controls were not in place to effectively oversee this activity.  Interagency contracting occurs when a federal agency obtains supplies or  services through another federal agency, either by placing orders on  existing contracts that have already been awarded by the other agency, or  by asking the other agency to award and administer contracts or issue and  administer task orders on its behalf. Use of these contracts demands a  high degree of business acumen and flexibility on the part of the federal  acquisition workforce.
We found that Procurement Operations had transferred almost 90 percent  of its obligations to other federal agencies through interagency agreements  in fiscal year 2004. For example, DHS transferred $12 million to the  Department of the Interior’s National Business Center to obtain contractor  operations and maintenance services at the Plum Island Animal Disease  Center. Interior charged DHS $62,000 for this assistance. While some of  the interagency agreements were for contracting support, others were for  program support, such as sending funds to Department of Energy  laboratories for providing a threat and capability assessment.
DHS has issued a management directive that sets forth a number of  requirements meant to ensure that internal controls are in place when  using interagency contracting. Based on a random sample of   136 interagency agreements between Procurement Operations and outside  agencies, we found that the office had not complied with the requirements  in the directive. We can project, for example, that in fiscal year 2004:    94 percent of Procurement Operations’ files did not document that the  contracting staff had conducted the required analysis of alternatives to  justify the decision to pay an outside agency for contracting support;    47 percent of the files did not identify the contracting officer’s  technical representative, although this information is required to be in  the files;    35 percent of the files did not contain the required determination and    96 percent of the files lacked an indication that contractor oversight  had been performed.
Further, we found that Procurement Operations was not tracking how  much it is paying in fees to other agencies for contracting support. On the  basis of our sample, we found that the office had spent $12.9 million in  fees in fiscal year 2004.
While the oversight problems we identified are in large part due to the  staffing shortages in the office, we also found evidence that contracting  staff lacked basic information on how to use interagency contracting. For  example, a memo from Procurement Operations to DHS’s Director of  Acquisition Policy and Oversight requested clarification on what  documentation is required in order to use another agency for contracting  support. In September 2004, DHS’s Office of General Counsel reviewed   20 of Procurement Operations’ interagency agreements and found that   16 were not legally sufficient. For example, 13 agreements appeared to  require performance by a contractor, but appropriate documentation was  not included in the contract files. Three were insufficient because it was  unclear whether the servicing agency would perform cost/price analysis  and trade-off, or whether Procurement Operations had conducted a  technical competition and expected the servicing agency to award without  a trade-off. According to a Procurement Operations official, a fiscal year  2005 initiative for the office is to establish a policy for processing  interagency agreements that provides clear guidance to staff.
subsections: section_title: Procurement Operations to Use Working Capital Fund to Increase Staffing Levels
paragraphs: Since January 2004, Procurement Operations has increased its staffing  level from 7 to 42 employees, and it plans to build to 127 staff in fiscal year  2005. Rather than use direct appropriations to fund the additional  positions, the office plans to require the DHS organizations that rely on its  contracting services to contribute to a working capital fund, to be  replenished on a no-profit basis with payments based on the extent to  which the various organizations use the office’s support. Although the  DHS budget for fiscal year 2005 includes $8.9 million to add contracting  staff through the fund, we found that the mechanics of making the fund  viable have not been worked out. Currently, Procurement Operations is  negotiating agreements with each organization it supports to determine  the terms and conditions of support and the dollar level to be contributed  to the fund. According to DHS officials, this negotiation process has been  problematic. For example, at the time of our review, the Science and  Technology Directorate and Procurement Operations were having  difficulty reaching a decision about who would be responsible for hiring  the contracting staff that would support the directorate. As of January  2005, no agreements had been reached.
section_title: Despite Adoption of Many Best Practices, Review Process for Major Investments Lacks Key Reviews and Some Management Controls
paragraphs: Some DHS organizations have large, complex, and high-cost acquisition  programs that need to be closely managed. For example, the Bureau of  Customs and Border Protection’s Automated Commercial Environment  system, which is a new trade processing system intended to improve the  movement of goods imported into the United States, is projected to cost   $5 billion, and the Coast Guard’s Deepwater Program is expected to cost  $17 billion and take 2 to 3 decades to complete. To review major,  complex investments such as these (referred to as level 1 investments),  DHS’s Office of the Chief Financial Officer has put in place a multitiered  process. DHS has taken positive steps in creating a knowledge-based  framework, or philosophy, for managing major investments; however, it  still lacks key reviews and deliverables—both best practices—within this  framework to ensure that cost and schedule estimates for major  investments are as accurate as possible. These reviews take place at  critical junctures in the process and include demonstrating knowledge  about technologies, design, and manufacturing processes. In addition, we  found that contractor oversight, an important tool for managing programs,  is not receiving high-level attention in the review process. Finally, we  identified several areas of confusion surrounding the mechanics of  implementing the process from a program manager’s perspective. The  management directive on the review process has been under revision for  many months, and DHS officials could not tell us when the directive would  be finalized.
subsections: section_title: Key Elements of DHS’s Investment Review Process
paragraphs: DHS’s investment review process involves several different levels of  review, depending on the dollar threshold and risk level of the program.  The Investment Review Board makes decisions on level 1 investments  with prior review and input from the Joint Requirements Council, which in  turn seeks input from other DHS specialists who have expertise in such  areas as asset management and information technology. In classifying  investments as level 1, DHS considers the following criteria: contract  costs; importance to DHS strategic and performance plans; high  development, operating, or maintenance costs; high risk; high return; and  significance in resource administration. Investments classified as levels   2, 3, or 4 are considered lower-level acquisitions and follow different  investment review processes. In addition, many of the major DHS  organizations, such as the Transportation Security Administration and the  Coast Guard, have their own review processes, which occur prior to  higher-level review in the department. Figure 4 illustrates who is involved  in the decision-making process and the levels of review.
section_title: Although Policy Governing the Review Process Has Established a Solid Risk Management Framework, Additional Best Practices Would Benefit DHS
paragraphs: DHS has adopted several best practices from lessons learned from leading  commercial companies and successful federal programs that, if applied  consistently, could refine its ability to reduce risk to meet cost and  delivery targets for major investments. One of the best practices is a  knowledge-based approach, or framework, for managers to hold reviews  at key decision points in order to reduce risk before investing resources in  the next phase of a program’s development. The investment review policy  provides guidance to program managers to provide knowledge about  important aspects of a product at key points in the acquisition process and  encourages them to reduce technology risk through demonstration prior  to beginning a project. The policy also encourages program managers to  demonstrate a product’s design with critical design reviews and reduce  manufacturing risk prior to a production decision. However, we found,  based on our extensive body of work on this knowledge-based approach,  that additional program reviews and knowledge deliverables could be  incorporated into the process as internal controls to better position DHS  to make well-informed decisions on its major, complex investments.
Figure 5 generally depicts the major phases of the knowledge-based  approach and the positioning of three knowledge points, or gates, when  key reviews are scheduled. The figure applies the knowledge-based  approach to DHS’s investment review framework and displays an  exclamation mark where key reviews or information are missing.
As shown in figure 5, DHS review points do not fully align with the  knowledge-based approach. For example, DHS does not require a review  to ensure that an investment’s design performs as expected before  investing in a prototype. In addition, DHS’s mandatory review to proceed  to production does not occur until after low-rate initial production is well  under way. DHS does have a review for low-rate initial production;  however, it is at the discretion of the Investment Review Board. Our past  work has shown that successful investments reduce risk by ensuring that  high levels of knowledge are achieved at these key points of development.  We found that investments that were not reviewed at the appropriate  points faced problems—such as redesign—that resulted in cost increases  and schedule delays.
We also found that some critical information is not addressed in DHS’s  investment review policy or the guidance provided to program managers.  In other cases, it is made optional. For example, before program start  (knowledge point 1) is approved, DHS policy requires program managers  to identify an acquisition’s key performance requirements and to have  technical solutions in place. This information is then used to form cost and  schedule estimates for the product’s development to ensure that a match  exists between requirements and resources. However, DHS policy does  not establish cost and schedule estimates for the acquisition based on  knowledge from preliminary designs. At knowledge point 2, while DHS  policy requires program managers to identify and resolve critical  operational issues before proceeding to production, initial reviews—such  as the system and subsystem reviews—are not mandatory.
Not all investments require the use of every piece of information included  under a knowledge-based approach. Many of DHS’s major investments use  commercial, off-the-shelf products that do not require the same level of  review as a complex, developmental investment would. However, DHS is  investing in a number of major, complex systems, such as the Coast  Guard’s Deepwater Program and the U.S. Visitor and Immigrant Status  Indicator Technology (US-VISIT), which incorporate new technology and  therefore require greater adherence to the knowledge-based approach in  order to ensure risk is reduced before committing to the next phase of the  investment. In addition, the added reviews and information included in the  knowledge-based approach may still be required for programs that use  existing technology, such as the Counter-MANPADS program, which  involves placing military technology on commercial aircraft.
In addition to the knowledge-based approach and its associated controls,  DHS’s investment review policy and guidance adopt a number of other  important acquisition management practices, such as requiring program  managers to submit acquisition plans and project management plans.  However, a key practice, contractor tracking and oversight, is not fully  incorporated in the policy and guidance. We have cited the need for  increased contractor tracking and oversight for several large DHS  programs. For example, we previously reported that the Coast Guard’s  Deepwater program needed increased management and contractor  oversight. One activity of contract tracking and oversight requires that a  quantitative set of software and system metrics are used to define and  measure product quality and contractor performance. The Coast Guard  had not developed measurable performance goals or adhered to effective  procedures for holding the contractor accountable for its ongoing  performance. In addition, we previously recommended that the US-VISIT  program should improve attention to implementing acquisition best  practices. While DHS agreed, and the US-VISIT program office has  assigned responsibility for implementing the recommended management  controls, the department has not yet developed explicit plans or time  frames for defining and implementing acquisition best practices. A list of  selected acquisition management practices and required activities is in  appendix V.
section_title: As Review Process Matures, Some Mechanics Still to Be Worked Out
paragraphs: The investment review process has been under revision for many months.  According to DHS officials, the changes will include shifting  responsibilities of some tiers in the review process and increasing the  dollar threshold for level 1 investments. To date, the new process has not  been finalized, and officials could not provide us with a time frame for  completion. In the meantime, we found unclear guidance and confusion  about several aspects of the process. In some cases, the confusion has  resulted in key stakeholders, such as the Chief Procurement Officer, not  receiving materials in time to conduct a thorough review and provide  meaningful feedback prior to investment review meetings.
The issues we found include the following:    Program managers have been provided with only draft guidance  regarding the information they are required to submit and the time  frames for submissions. This draft guidance is, in some cases, unclear.
Some DHS officials noted that their submissions to the review board  had been rejected on an inconsistent basis with no explanation.
Program managers have not received formal training on the investment  review process. Officials told us that some program managers have  been unaware of when to submit information about their programs for  review.
In practice, major investments in services are exempt from the review  process and are only reviewed when done as part of a capital  investment. Officials from the Office of the Chief Financial Officer who  are in charge of the process told us that services investments are  reviewed only when they are part of a capital investment because these  acquisitions are not complex and therefore do not need the same level  of scrutiny reserved for the acquisition of goods.
Currently, program managers receive assistance in developing their review  board submissions from a small number of staff in the Chief Financial  Officer’s Office of Program Analysis and Evaluation at the beginning of a  program and at DHS’s key decision points. However, because of limited  resources, the office has only been able to provide limited support to  programs to assist them in completing their submissions.
section_title: Conclusions
paragraphs: In the 2 years since its merger, DHS has taken strides toward putting in  place an acquisition organization that contains many promising elements,  but the steps taken so far are not enough to ensure that the department is  effectively managing the acquisition of the multitude of goods and services  it needs to meet its mission. More needs to be done to fully integrate the  department’s acquisition function, to pave the way for the Chief  Procurement Officer to fully carry out his responsibilities in a modern-day  acquisition organization, and to put in place the strong internal controls  needed to effectively manage interagency contracting activity and large,  complex investments. Unless DHS’s top leaders address these challenges,  the department is at risk of continuing to exist with a fragmented  acquisition organization that provides stopgap, ad hoc solutions. DHS has  an opportunity, while it is still involved in transformational efforts, to  avoid the complications that plague acquisition efforts in other long- established federal departments.
section_title: Recommendations for Executive Action
paragraphs: To help ensure that DHS receives the goods and services it needs at the  best value to the government, we recommend that the Secretary of  Homeland Security take the following six actions:    establish a structure to ensure continued support for commodity  councils, such as appointing full-time dedicated commodity managers,  to ensure that the commodity councils develop long-term strategies,  maintain momentum, and continue to realize savings;    provide the Office of the Chief Procurement Officer with sufficient  resources and enforcement authority to enable effective,  departmentwide oversight of acquisition policies and procedures;    conduct a departmentwide assessment of the number of contracting  staff and, if a workload imbalance is found, take steps to correct it by  re-aligning resources;    direct higher-level management attention to the implementation of the  working capital fund (which is to be used to fund contracting staff for  the Office of Procurement Operations) by, for example, determining  the level of contracting support needed by the organizations relying on  this office, ensuring that appropriate funds are committed to hire  needed contracting staff, and ensuring that funds are available on an  ongoing basis for continuity;  revise the October 2004 management directive “Acquisition Line of  Business Integration and Management” to eliminate reference to the  Coast Guard and Secret Service being statutorily exempt from  complying; and    ensure that DHS’s management directive on interagency agreements is  followed and that fees paid to other agencies are tracked.
To help ensure that DHS leadership is aware of risks as they arise during  the acquisition of major, complex systems, we recommend that the  Secretary of Homeland Security take the following seven actions:  in making revisions to the investment review policy: require for all complex, developmental investments a formal design  review between the integration and demonstration of a program to  ensure that the design is stable and has been demonstrated through  prototype testing;  require for all complex, developmental investments a review before  initial production;  require that program managers supply additional information—such as  cost and schedule estimates based on results of a preliminary design  review and critical design review—when their major, complex  programs are reviewed; and  require program managers to specifically address contractor oversight  in their submissions to investment review boards.    ensure that stakeholders, including acquisition officials in the Office of the  Chief Procurement Officer, have adequate time to review investment  submissions and provide formal input to decision-making review boards;  implement training for program managers on the investment review  process that emphasizes the importance of a knowledge-based approach;  and  require that major acquisitions of services be subject to oversight by the  investment review board.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DHS for review and comment. In  written comments, DHS generally agreed with our facts and conclusions  and concurred with all of our recommendations. Regarding three of our  recommendations on the investment review process, DHS stated that the  actions exist in current directives and are already being done. We disagree.  Our work demonstrated that DHS’s review points do not fully align with  the knowledge-based approach. For example, DHS’s mandatory review to  proceed into production does not occur until after low-rate initial  production is well under way, and the review for starting low-rate initial  production only occurs at the discretion of the Investment Review Board.  Also, DHS's framework lacks the knowledge deliverables necessary at  each key review to ensure that cost and schedule estimates for major  investments are as predictable as possible. Our past work has shown that  investments that were not reviewed at the appropriate points faced  problems that resulted in cost increases and schedule delays.
The department’s comments are reprinted in appendix II.
As arranged with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days from its  issue date. We are sending copies of this report to other interested  congressional committees, the Secretary of the Department of Homeland  Security, and the Director of the Office of Management and Budget. In  addition, the report will be available on GAO’s Web site at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (937) 258-7915. Staff making major contributions to this  report are listed in appendix VI.
section_title: Appendix I: Scope and Methodology
paragraphs: To identify the areas where the Department of Homeland Security (DHS)  has been successful in promoting collaboration among its various  organizations, we interviewed senior acquisition officials at DHS  headquarters and analyzed pertinent documents. We obtained information  from the Office of Strategic Sourcing and Acquisition Systems to analyze  how DHS has used strategic sourcing to leverage the department’s buying  power. We interviewed senior strategic sourcing officials at DHS  headquarters to obtain information on how they identified the  commodities with potential for savings. We reviewed studies, policies,  guidance, and other documents related to ongoing or proposed strategic  sourcing initiatives that leveraged buying power, cut costs, or achieved  other performance benefits. Our review found that the department’s  strategic sourcing cost savings methodologies appear reasonable;  however, we did not verify the accuracy of any strategic procurement  costs savings reported to us. We asked senior strategic sourcing and  commodity council participants about potential barriers to employing  strategic sourcing at DHS. We interviewed the Director, Office of Small  and Disadvantaged Business Utilization, and procurement officials within  the department. We obtained policies, memorandums, and other  documents from the small business office, procurement organizations, and  the U.S. Small Business Administration.
To determine areas where DHS faces challenges in integrating the  acquisition function, we reviewed DHS organizational charts to gain  insight into where the procurement offices fall in the hierarchy and to  determine the lines of responsibility and authority between the various  stakeholders in the acquisition process. We reviewed DHS policies,  guidance, and procedures governing acquisition and analyzed internal  audit reports, when available, to determine if those policies and  procedures were being followed. We obtained statistics from DHS’s Office  of the Chief Procurement Officer on the department’s procurements and  the acquisition workforce. We interviewed procurement policy officials at  headquarters and conducted interviews with personnel in each of the  procurement organizations. To assess DHS’s effectiveness in managing its  acquisition workforce, we interviewed contracting and human resource  officials at DHS headquarters. We analyzed DHS’s processes and  procedures for certifying program managers, warranting contracting  officers, and tracking the acquisition workforce training. Lastly, we  reviewed previous GAO work regarding best acquisition practices for  organizational alignment and oversight.
Because the Office of Procurement Operations has, by far, the highest  level of interagency contracting activity in the department, we examined  the office’s database of contracts to gain an understanding of the general  scope of activity involving interagency agreements. We randomly selected  and then reviewed 136 interagency agreement contract files. We  interviewed officials responsible for the data to ensure that the system  they use to track procurement activity was adequate for identifying the  sample population. We reviewed the interagency agreement contract files  to assess key aspects of the acquisition process—such as a signed  determination and findings in accordance with the Federal Acquisition  Regulation, acquisition planning, and interagency agreement contract  administration, including contractor oversight. We held follow-on  discussions with the Office of Procurement Operations to discuss  discrepancies noted in the files. Further information on our methodology  and sampling error rates is as follows:  In fiscal year 2004, the Office of Procurement Operations predominately  obtained contracting services for its customers through the issuance of  interagency agreements with other government agencies. We drew our  sample from an original population of 984 agreements. Of the 150 items we  initially sampled, the department deleted 14 of these agreements and  withdrew them from the data. As a result, we adjusted the population of  agreements to which we are making estimates to 892 agreements. All  estimates are to these 892 agreements based upon a sample size of   136 agreements. In addition, 4 files were missing and were not able to be  reviewed, and one agreement was a duplicate. This resulted in only 131  files being reviewed. As we conducted our reviews, the agency became  better at recording the agreements that should be included in its database.  Currently, Procurement Operations reports that 1,104 agreements are  contained in its records and total $1.8 billion.
We reviewed the files and documents for these 131 agreements, using as  our criteria the department’s directive on interagency agreements. The  following projections are made from our sample of 136 agreements. The  estimates from a statistical sample are always subject to some uncertainty  because the entire collection is not reviewed. This uncertainty is called  sampling error. Tables 2 and 3 show the sampling errors for certain factors  relating to the issuance of interagency agreements.
To assess the department’s progress in implementing a review process for  major, complex systems, we compared DHS’s acquisition policies for  major acquisitions to our knowledge-based approach. We used  information from several of our prior reports that examine how  commercial best practices can improve outcomes for acquisition  programs. Specifically, we compared and contrasted DHS’s investment  review process with the best practices for commercial acquisitions  identified in our past reports. Our analysis focused on whether DHS’s  policies contained the measurable criteria and management controls  necessary for minimizing cost, schedule, and performance risks. To clarify  the content of the investment review process, we met with various DHS  officials from the Office of the Chief Procurement Officer and the Office of  Program Evaluation and Analysis within the Office of the Chief Financial  Officer. We also discussed the review process with officials from DHS  procurement organizations and collected reports and analyzed available  program data on the current status of major acquisitions being carried out  by DHS.
We conducted our review from March 2004 through February 2005 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Comments from the Department of Homeland Security
section_title: Appendix III: Office of Procurement Operations’ Customers
section_title: Appendix IV: Summary of Selected Commodity Council Strategic Sourcing Initiatives
subsections: section_title: Strategic sourcing actions taken
paragraphs: Leverages DHS’s purchasing power and reduces amount spent on office supplies:    For its main strategic sourcing initiative, this council partnered with the Department of Defense’s  Electronic Mall (DOD Emall). DOD Emall is an Internet-based catalog ordering system that provides 24- hour-a-day, 7-day-a-week access to over 200 small and large office supply vendors.
Because many military services also purchase office supplies through DOD Emall, DHS can take  advantage of large volume discounts to increase its buying power.
As of June 1, 2004, DHS mandated all of its purchase cardholders to exclusively use DOD Emall to buy  office supplies. As a result, DHS has already become the third-largest federal government user of DOD  Emall. For the 4-month period starting June 1, 2004, DHS spent over $14.5 million on DOD Emall,  representing 16.7 percent of the total dollars spent on Emall.
Before the full implementation of DOD Emall, the council also generated savings by using an office  supplies blanket purchase agreement that the Transportation Security Administration had in place to  provide office supplies to other DHS principal organizations.
Through September 30, 2004, the council estimated cost savings of $8 million.
Establishes standard boat procurements to decrease life cycle costs, facilitate interoperability, reduce  training burden, and provide immediate cost savings:    For example, Customs and Border Patrol needed six boats in fiscal year 2004. Instead of creating a  new contract, the council explored whether there were any existing contract vehicles within DHS that  could satisfy the need.
The council purchased the six boats through an existing Coast Guard contract and took advantage of  large volume discounts.
As a result of these strategic sourcing efforts, the council was able to acquire each boat for $50,000  less than anticipated, resulting in a total of $300,000 in cost savings.
Identifies and consolidates emerging firearms and ammunition requirements for DHS:  In fiscal year 2004, the council planned to acquire pistols for DHS organizations to meet critical,  mission-driven requirements.
The council surveyed all DHS organizations interested in weapons, factored in the end users’  requirements, and established a list of potential vendors that could provide best value for pistols and  satisfy requirements.
As a result, DHS awarded contracts to two vendors for 65,000 pistols each over 5 years—the largest  pistol acquisition in the history of U.S. law enforcement.
As of September 20, 2004, the council has reported cost savings of over $4.1 million and $915,000 in  cost avoidance.
Identifies strategies for aggregating and centralizing DHS’s energy procurements to take advantage of  economies of scale and negotiate more competitive prices with suppliers:    Although DHS spent approximately $58 million for electricity in fiscal year 2003, only $9 million of the  spending was for negotiable energy services in states with deregulated electricity industries.  Nevertheless, the council estimates more than $705,000 in cost savings through September 30, 2004.
section_title: Appendix V: Selected Acquisition Management Best Practices
paragraphs: Plans are prepared during acquisition planning and  maintained throughout the acquisition.
Planning addresses the entire acquisition process, as well as  life cycle support of the products being acquired.
The acquisition organization has a written policy for planning  the acquisition.
Responsibility for acquisition planning activities is designated.
The acquiring organization has sufficient insight into the  contractor’s activities to manage and control the contractor  and ensure that contract requirements are met.
The acquiring organization and contractor maintain ongoing  communication; commitments are agreed to and implemented  by both parties.
All contract changes are managed throughout the life of the  contract.
The acquisition organization has a written policy for contract  tracking and oversight.
Responsibility for contract tracking and oversight activities is  designated.
The acquiring organization involves contracting specialists in  the execution of the contract.
A quantitative set of software and system metrics is used to  define and measure product quality and contractor  performance.
In addition to incentives for meeting cost and schedule  estimates, measurable, metrics-based product quality  incentives are explicitly stated in the contract.
Projectwide participation in the identification and mitigation of  risks is encouraged.
The defined acquisition process provides for the identification,  analysis, and mitigation of risks.
Milestone reviews include the status of identified risks.
The acquisition organization has a written policy for managing  acquisition risk.
Responsibility for acquisition risk management activities is  designated.
section_title: Appendix VI: GAO Contacts and Staff Acknowledgments
subsections: section_title: Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to those named above, Daniel Chen, Lily Chin, Benjamin  Federlein, Arthur James Jr., John Krump, Jose Ramos, and Russell Reiter  made key contributions to this report.