DHS Acquisitions: Additional Practices Could Help Components Better Develop Operational Requirements
Released: 2018-08-08T00:00:00

Letter

Background

To help manage its multi-billion dollar acquisition investments across its components, DHS has established policies and organizations for requirements validation, acquisition management, and budgeting. The department uses these to monitor and guide delivery of the acquisition programs the components require to close critical capability needs, enabling DHS to execute its missions and achieve its goals.

DHS and Its Components

DHS has 14 components, which, as a part of their operational missions, are responsible for assessing capability needs, developing the requirements to fill these needs, and creating acquisition programs to meet these requirements. The number and cost of acquisition programs vary by component. DHS generally defines a capability as the means to accomplish a mission or objective that may be achieved through materiel and non-materiel solutions. Once the component has a JRC-validated capability gap, and identifies and documents the need for a materiel solution, it develops the operational requirements. Requirements can be unique to an individual component, or they can be joint requirements that apply to more than one component. Within the components, program management offices are responsible for planning and executing individual programs within cost, schedule, and performance parameters, and preparing required acquisition documents.

Tracing Mission Needs to Program Requirements

The DHS requirements process generally starts with the identification of mission needs and broad capability gaps from which components develop a program’s operational requirements, key performance parameters, and more definitive technical requirements. Figure 1 depicts this traceability from mission needs to technical requirements.

Operational requirements are what the end users need to fill capability gaps and conduct the mission. Operational requirements, in part, define the purpose for the acquisition program and set boundaries for user needs. Subject matter experts, such as system engineers, support development of operational requirements to ensure that they are clearly developed. Well-defined operational requirements trace to one or more of the identified capability gaps.

After components define operational requirements, they identify some as key performance parameters that denote the most important and non- negotiable requirements that the program has to meet to fulfill its fundamental purpose. According to DHS policy, failure to meet any key performance parameter results in a re-evaluation of a program that may lead to requirements changes or program cancellation. See figure 2 below for an overview of the requirements process.

According to DHS policy on managing acquisition programs, components further decompose operational requirements into technical requirements, such as design or material specifications. For example, an operational requirement may be the ability to detect explosives at the airport. The technical requirement may then be the ability to detect metal or explosive material within certain parameters.

DHS’s Joint Requirements Council and Other Offices

Through the JRC, DHS provides oversight of operational requirements for the acquisition programs developed by its components. The JRC consists of a chair and 14 members, called principals, who are senior executives or officers that represent key DHS headquarters offices and seven of the department’s operational components. JRC principals represent the views of both their components and DHS, and validate and prioritize capability needs and operational requirements. Among other responsibilities, the JRC is to provide requirements-related advice and validate key acquisition documentation to prioritize requirements and inform DHS investment decisions for all Level 1 and Level 2 major acquisitions, as well as for programs that are joint interest, regardless of level.

Separate from the JRC, DHS’s Office of Program Accountability and Risk Management, which reports directly to the Under Secretary for Management, oversees major acquisitions and guides acquisition policy. DHS also has a separate office for budget management and a planning, programming, budgeting, and execution process to allocate resources, such as funding, to acquisition programs. In addition, the Science and Technology Directorate conducts systems engineering reviews and technology assessments of the technical solutions for major acquisition programs. The Directorate also provides department-level guidance on requirements development in its Systems Engineering Life Cycle Guidebook.

DHS’s Joint Requirements Process

Multiple DHS directives and manuals establish the framework for the department’s Joint Requirements Integration and Management System (JRIMS)—a process by which the department reviews and validates capability gaps—and requirements to mitigate those gaps. DHS further clarified its directives in April 2016 through DHS Instruction Manual 107- 01-001-01, Department of Homeland Security Manual for the Operation of the Joint Requirements Integration and Management System. The JRC also instituted a series of training courses that provide an overview of JRIMS and its core concepts. JRC validation of requirements confirms the requirements are traceable, feasible, and cost-informed.

In addition to validation by the JRC, DHS’s Under Secretary for Management approves the operational requirements that the components developed and reviews them at a series of predetermined acquisition decision events. Figure 3 depicts the acquisition life cycle established in DHS acquisition policy. DHS initially established its acquisition process in policy in November 2008.

An important aspect of acquisition decision event 2A, which begins the “Obtain” phase and system development, is the decision authority’s review and approval of key acquisition documents that establish the cost, schedule, and requirements baselines for a program. The operational requirements document and acquisition program baseline are key acquisition documents requiring this approval and include a program’s key performance parameters. DHS also revisits these baselines at subsequent acquisition decision events in order to determine whether the requirements remain achievable.

Prior GAO Work on DHS Requirements Development

We have previously reported on the importance of stable requirements and the costs of changing them.

In March 2016, we found that changes to key performance parameters have been common and are likely to continue for several reasons. While some changes may have a valid reason, such as a response to emerging threats, we found that one of the most common reasons programs changed key performance parameters was that the originally approved key performance parameters had been poorly defined. Key performance parameter changes on several programs were associated with schedule slips and cost growth. DHS leadership acknowledged that the department has had difficulty defining key performance parameters, and said that the Office of Program Accountability and Risk Management has improved its ability to help programs define key performance parameters. We recommended, among other things, that DHS should require the components to submit program funding certification memos to aid affordability discussions. DHS concurred and implemented our recommendation.

In October 2016, we found that the JRC’s structure and management approach—informed by assessments of requirements processes, guidance, and lessons learned from DHS components—are generally consistent with key practices for mergers and organizational transformations. However, we recommended that DHS’s Office of the Chief Information Officer have a more formal and consistent role than that of a non-voting advisor to the JRC, since 24 of 36 major acquisitions were information technology programs, and we previously identified poor requirements definition as a factor in failed information technology programs. DHS concurred with our recommendation and implemented it in November 2016.

In April 2017, we found that DHS’s acquisition policy was not consistent with acquisition best practices in terms of when to enter the “Obtain” phase depicted in figure 3. Specifically, best practices call for ensuring that a program’s needs are matched with available resources—such as technical and engineering knowledge, time, and funding—prior to starting product development. We recommended, among other things, that DHS require that major acquisition programs’ technical requirements be well-defined and conduct key technical reviews prior to approving programs to initiate product development, in accordance with acquisition best practices. DHS concurred with our recommendation, but has not yet implemented it.

Over Half of the Selected Programs Changed Requirements

Our analysis found that 9 of 14 programs from the seven components that we reviewed changed key performance parameters for various reasons after program approval and entry into the “Obtain” phase. DHS had initially approved most programs’ key performance parameters before DHS reestablished the JRC in November 2014. Whether these programs changed DHS-approved key performance parameters is shown in table 1.

We found that the causes of these changes varied, but included requirements did not accurately describe end user needs, were not achievable given available technologies, or that programs pursued greater capability than originally intended. Further details on the nine programs that changed their requirements are in table 2.

To mitigate these types of requirements changes, we identified several principles that are critical as the first steps to successful implementation of programs and the remainder of this report presents examples of when the principles have been implemented and when they have not.

One of the Seven Selected Components Has a Policy for Requirements Development

Among the seven DHS components we reviewed, each of which is responsible for managing major acquisition programs, only the U.S. Coast Guard has a formalized policy in place for developing requirements. Of the other six components, some are developing such policies and others rely on JRIMS guidance. In the absence of component-level policies, some sub-organizations and programs within the components have developed their own requirements policies.

U.S. Coast Guard Has an Approved Requirements Policy, While the Other Six DHS Components in Our Review Do Not

The U.S. Coast Guard, which has a long history of managing large acquisition programs, established a requirements policy to assess needs and fill capability gaps in 2009 and updated it in 2017. The most recent version of this requirements policy, the Coast Guard Operational Requirements Generation Manual, aligns its policies with DHS’s acquisition and requirements policies. The manual contains guidance on requirements development and the analytic efforts used to develop the requirements documents. The manual also describes the personnel that are to be included in requirements development and provides guidance on drafting the necessary documentation, and includes templates to do so. As part of the process, requirements development personnel work with end users to generate requirements, which the U.S. Coast Guard reviews and approves before going to the DHS JRC for validation.

The status of developing a requirements policy across the other six components is as follows: Immigration and Customs Enforcement, National Protection and Programs Directorate, Transportation Security Administration, and U.S. Citizenship and Immigration Services officials told us that they are currently developing or considering developing policies. These components have not yet set time frames for approving these policies.

A Federal Emergency Management Agency official stated that they are planning to develop a formal requirements policy but are waiting for the JRC to clarify JRIMS policy on information technology program reviews and decision authorities before doing so. However, such clarification does not prevent them from drafting an interim policy.

Customs and Border Protection has a draft requirements development policy but did not provide a definitive timeline for completion.

Although Customs and Border Protection does not yet have a finalized policy, the following sub-component operational organizations have documented their requirements policies. For example:

Border Patrol finalized a requirements management process policy on June 12, 2018 that defined roles and responsibilities throughout the process. The requirements policy was preceded by an October, 18, 2016 policy on the process for identifying capability gaps. GAO previously reported on the Border Patrol’s policy in February 2017 and recommended clarifying the roles and responsibilities of the parties involved.

The Office of Technology Innovation and Acquisition developed a draft requirements handbook in 2011 that provided guidance for the execution of activities within each stage of development, including defining operational requirements.

The Passenger Systems Program Office also documented its requirements management policy in 2010 that outlined requirements development at a high level.

While these sub-components have taken the key step of documenting their policies, without a single component policy, Customs and Border Protection may not be efficiently and effectively meeting its mission.

Without Requirements Policies, Components Risk Failing to Meet Mission Needs

In the absence of component-level policies, we found that components are less likely to establish the base of knowledge needed for requirements development. Further, we found this contributes to an inability to properly mitigate capability gaps and meet mission and end user needs.

Outcomes for a number of our case study programs illustrate the potential benefits of having component-level requirements development policies in place.

National Flood Insurance Program PIVOT (not an acronym): Federal Emergency Management Agency officials told us that the current attempt is the third effort to modernize its information technology systems after two failed attempts. Program officials said that one of the previous program attempts failed to meet capability gaps and end user needs because of a lack of clear policies for developing requirements. The officials said that failure is less likely as the program currently uses lessons learned from the previous attempts. In addition, the JRC is encouraging the component to adopt rigorous standards for developing requirements. However, without a policy to capture these lessons learned, programs within the Federal Emergency Management Agency are at risk for losing the knowledge.

National Security Cutter: The U.S. Coast Guard began requirements development for the National Security Cutter in the late 1990s, before it had established a documented requirements development policy in 2009. We found in 2010 that the lack of overarching, formalized policy resulted in requirements that were vague, not testable, not prioritized, and not supportable or defendable. In 2014, the National Security Cutter completed initial operational testing but did not fully demonstrate 7 of its 19 key performance parameters, including those related to unmanned aircraft and cutter-boat deployment in rough seas. To meet the cutter-boat deployment parameter, U.S. Coast Guard officials said that the program had to overcome differing interpretations of the parameter between the U.S. Coast Guard and its independent test officials. One key practice for requirements development is assigning roles and responsibilities, such as when and in what capacity test officials should be involved in requirements development, to avoid just such an outcome and the resulting effect on cost and schedule. U.S. Coast Guard officials stated that end users of the National Security Cutter have since demonstrated its key performance parameters during U.S. Coast Guard operations.

Electronic Baggage Screening Program: Without a finalized requirements development policy, the Transportation Security Administration’s program developed requirements that focused on how the system functioned as opposed to the capability that it would provide. Program officials said that neither the Transportation Security Administration nor the program office had a documented policy for requirements development when the program began in 2004. In this environment, the program adopted an informal approach to develop operational requirements by collecting end user input. However, officials noted that end users listed technical requirements rather than broader operational requirements. Officials told us that the program “backed into” operational requirements using these technical requirements, resulting in a system more focused on function and less on capability. Without a focus on the capability, the program risked not meeting the capability gap and end user need.

We also found an example of where a component’s policy was beneficial to a program developing requirements:

Offshore Patrol Cutter: The U.S. Coast Guard has matured its requirements development policies since the National Security Cutter program as described above. For the Offshore Patrol Cutter, the U.S. Coast Guard has six DHS-approved key performance parameters, such as operating range and duration. The U.S. Coast Guard plans to use engineering reviews and developmental and operational tests throughout the acquisition to refine and demonstrate requirements. For example, to refine the requirements and ensure end user input, the U.S. Coast Guard had an early operational assessment of the cutter’s key performance parameters and associated lower level technical requirements. According to officials, specific policies guided the assessment to, in part, ensure that the program refined key performance parameters before progressing through the remaining acquisition phases.

DHS’s JRIMS directive and manual are not designed to provide the level of specificity for component-level requirements development. JRIMS encourages components to elicit end user needs and translate them into requirements. It also authorizes the components to develop their own policies consistent with the intent of and required capability documentation in the JRIMS manual and DHS Instruction Manual.

Federal standards for internal control and key practices for requirements development, such as those in Carnegie Mellon University’s Capability Maturity Model Integration for Development, state that organizations should establish responsibility and authority by having documentation that communicates the “who, what, when, where, and why” of achieving their missions. A policy also provides a means to retain organizational knowledge and mitigate the risk of having that knowledge limited to a few personnel. Such a policy should include a documented process for developing and managing requirements which can help reduce the risk of developing a system that does not meet end user needs, cannot be adequately tested, and does not perform or function as intended. We depict four key practices for requirements management in figure 4.

DHS officials indicated to us that one factor which contributes to a component’s lack of finalized requirements policies is the prioritization of starting an acquisition over developing requirements. This situation reflects what we have found over many years at the Department of Defense. Undesirable program outcomes share a common origin; decisions are made to move forward with programs before the knowledge needed to reduce risk and support those decisions is sufficient. There are strong incentives within the acquisition culture to overpromise performance while understating cost and schedule. A key enabler of successful programs is firm, feasible requirements that are clearly defined, affordable, and clearly informed. Once programs begin, requirements should not change without assessing their potential disruption to the program.

Of note, DHS established its formal acquisition process in 2008, and did not have a similar emphasis on requirements development until 2016, when the JRIMS process was set forth. DHS requirements officials said that the renewed emphasis on requirements development at DHS requires a significant culture change among the components, pushing the components away from previous practices that undervalued well-defined requirements. They said that the components generally completed the necessary requirements documents to comply with department guidance and formats rather than to ensure that the components identified the needed capabilities and generated the correct requirements.

DHS officials said that in the past, some program offices would contract out the capability assessment and requirements development, have them approved by DHS, but not use the resulting documentation to guide the acquisition. Two component requirements officials told us that their components’ previous acquisition and requirements processes focused on obtaining funding before developing requirements.

Most components indicated that they are planning on drafting a requirements development policy. However, without specific timeframes for completing their efforts, there is a risk that management attention will not be sustained and planned actions will not be implemented. Without component-level requirements policies that are aligned with the JRC and JRIMS standards, DHS is missing an opportunity to help ensure that components’ programs are set-up from the beginning to meet end user needs and close capability gaps.

Utilization of an Independent Requirements Organization Inconsistent Across Selected Components

Three of the seven DHS components in our review have established requirements development organizations, such as offices or directorates independent of the acquisition function. Among the reasons cited by these components’ officials was recognition of the importance of the operational requirements development function for addressing capability gaps. Those that do not have separate requirements organizations cited, among other things, the smaller size of their components. However, according to key principles, independent lines of authority should develop operational requirements and manage acquisitions separately, regardless of size.

Three Components Have Independent Requirements Development Organizations but Remaining Four Components Do Not

Three of the seven DHS components in our review have established independent requirements development organizations that are separate from acquisition offices, as shown in table 3.

The three components that established requirements organizations did so at various times.

In 2009, the U.S. Coast Guard formally placed responsibility for its requirements development policy in its capabilities directorate under the Assistant Commandant for Capability, who reports to the Deputy Commandant for Operations, one level below the Vice Commandant of the Coast Guard. The capabilities directorate, which is separate from the acquisitions directorate, provides oversight and management of its requirements development process. This directorate provides expertise as well as an independent quality review of the requirements documents generated for approval.

Customs and Border Protection officials noted that they created a requirements organization in 2010 in the Office of Technology Innovation and Acquisition. In 2016, through an organizational realignment, Customs and Border Protection separated the requirements organization and established the Planning, Analysis, and Requirements Evaluation Directorate. The officials stated that due to concerns about independence from the acquisitions office, Customs and Border Protection placed this Directorate in the Operations Support office.

The Transportation Security Administration established the Office of Requirements and Capabilities Analysis in 2017, in part, because officials told us they recognized that prior requirements development efforts were not being done the right way. This new office, which is separate from the Office of Acquisition Management, reports directly to the Executive Assistant Administrator of Operations Support.

The remaining four components that we reviewed did not have separate, independent requirements development organizations. Officials from Immigration and Customs Enforcement, National Protection and Programs Directorate, and U.S. Citizenship and Immigration Services noted that they are planning on developing such organizations but have not provided specific time frames for doing so. An official from the National Protection and Programs Directorate told us that although an independent office has not been established, he serves as an independent requirements official, separate from acquisitions. Among the reasons cited by components’ officials for not having a requirements organization at the time of our review was a primary focus on the acquisition function, associated funding issues, and reliance on the JRC to help refine their requirements.

Officials also noted the smaller size of their respective components and the fewer number of major acquisitions as reasons for not having an independent requirements organization. Regardless of size, components need to ensure that requirements development is independent of acquisitions in order to guard against possible bias by acquisition officials toward a specific materiel solution.

A Separate, Independent Requirements Organization Is Critical to Addressing Capability Gaps

According to federal standards for internal controls, independent lines of authority should develop requirements and manage acquisitions separately. These standards state that management should design control activities to achieve objectives and respond to risks. In addition, authorities should segregate incompatible duties to prevent risks such as management override. For example, if requirements developers were part of the acquisition function, management could tailor operational requirements to satisfy preferred acquisition outcomes, increasing the risk that capability gaps will not be addressed. The absence of an independent requirements organization hampers the components’ ability to remove biases and identify crosscutting opportunities and investments. See figure 5 for a notional example of organizations with separate functions.

In accordance with these standards, DHS, at the department level, has separate requirements, acquisitions, and resourcing organizations—each with its own governance structure. In addition, U.S. Coast Guard policy notes that requirements development, when separated from acquisition organizations, results in an operational requirements document that conveys the user’s true needs. The policy goes on to state that the requirements development organization informs the acquisition process by ensuring requirements are traceable to strategic objectives and recommended courses of action to address capability gaps are cost informed and assessed for feasibility. According to GAO’s best practices, while these organizations should be separate, there should be consistent collaboration and feedback throughout the process.

We found examples of programs in our review that would have benefited from an independent organization at the component level.

Immigration and Customs Enforcement, TECS Modernization (not an acronym): The acquisition program office set the requirements without an understanding of the capability gaps it was trying to close. Without a requirements development office to guide development, program officials stated that they generated approximately 25,000 requirements, which consisted of both technical and operational requirements to address the capability gaps that they were unable to prioritize. The program revised its operational requirements a few times and went through a replanning initiative that included a full review of all the requirements to ensure completeness and accuracy to determine the program’s operational requirements. Immigration and Customs Enforcement officials stated that they recognize the importance of requirements development and are in the process of establishing a requirements organization.

U.S. Citizenship and Immigration Services, Transformation: The program began requirements development in 2006 in the absence of an independent organization for requirements development and has subsequently generated three operational requirements documents over a six-year period. Our review showed that the key performance parameters from the oldest document to the most recent one changed significantly. For example, the operational requirements document from 2009 had a key performance parameter called “account hardening,” which involved gathering identity and biometric evidence. The document from 2015 did not contain this parameter. In April 2015, nine years after starting requirements development, DHS leadership finalized a revised set of operational requirements after the program struggled again to meet its previous requirements.

We also found an example of where a component’s requirements organization was beneficial to a program developing requirements:

Customs and Border Protection, Cross Border Tunnel Threat: This program is analyzing alternative capabilities as it moves toward the JRC’s validation of its requirements. To aid in developing the operational requirements, Border Patrol, a sub-component of Customs and Border Protection, has its own Operational Requirements Management Division. In addition, Customs and Border Protection officials noted that its Planning, Analysis, and Requirements Evaluation Directorate is coordinating, guiding, and providing oversight to ensure the operational requirements address the capability gaps. In doing so, these requirements organizations facilitate input from subject matter experts on tunnel threats and from end user agents who have to mitigate these threats.

Majority of Selected Components Have Not Assessed Workforce Needs or Established Training for Requirements Development

We found that two components have assessed requirements workforce needs, and one has provided requirements specific training. Components gave different reasons why they have not yet taken one or more of these steps, including a lack of resources.

Two Components Have Assessed Requirements Workforce Needs, and One Has Provided Requirements Specific Training

Two of the seven components we reviewed, Federal Emergency Management Agency and Customs and Border Protection, performed assessments of workforce needs for requirements development. The Federal Emergency Management Agency assessed its requirements workforce needs in 2016 and found, among other things, that it does not have the capacity to identify and analyze capability gaps or accurately trace operational requirements to capability needs. As a result of the assessment, the agency requested additional requirements personnel in the fiscal year 2019–2023 budget cycle.

Customs and Border Protection requirements officials stated that they last conducted an assessment in 2013. They stated that the assessment identified the appropriate number and types of personnel necessary to conduct requirements development through an analysis of historical requirements workloads. In addition, Customs and Border Protection officials said that they are currently performing an assessment as part of their Acquisition Management Performance Improvement initiative. The initiative assesses training needs and availability and is due at the end of fiscal year 2018.

Requirements officials from Immigration and Customs Enforcement, National Protection and Programs Directorate, Transportation Security Administration, and U.S. Citizenship and Immigration Services told us that they have not assessed their requirements workforce needs and have no plans to do so. U.S. Coast Guard requirements officials told us that although they have not conducted a formal assessment of their workforce needs, they informally assess those needs and would like to increase the personnel who have requirements training across the organization.

Although the U.S. Coast Guard has not conducted an assessment of its workforce needs, it is the only component that has an established requirements training process. Requirements officials told us that the U.S. Coast Guard initially established training and training-related certification standards in 2007 to emulate similar changes taking place at the Department of Defense and address previous U.S. Coast Guard acquisition challenges. Specifically, the U.S. Coast Guard requirements development organization assigns end users for a two to three year rotation and provides them training and certification on requirements development. The requirements development certification program is two levels and requires both classroom-based training and on-the-job experience. The U.S. Coast Guard assigns those who complete a higher level of certification to develop requirements for more complex and costly programs. This helps to ensure that requirements personnel can give timely, relevant end user input but also differentiate between operational and technical requirements. U.S. Coast Guard requirements officials told us that the training and certification standardizes the proficiency of the requirements workforce across the component. In addition, Customs and Border Protection officials told us that they are in the process of training their personnel on operational requirements development as part of a larger training program implemented through their Acquisition Management Performance Improvement effort.

Components provided multiple reasons why they have not assessed their requirements workforce development needs or implemented a requirements training program. Specifically:

Federal Emergency Management Agency is waiting on resources to build a requirements organization and provide component-specific training.

Immigration and Customs Enforcement officials stated that they are standing up a requirements development organization and have requested additional personnel. However, they have not done a comprehensive assessment of their workforce needs nor established additional training as a result of resource constraints.

National Protection and Programs Directorate requirements officials told us that they do not currently have plans to assess the sufficiency of requirements development personnel and do not have component- specific requirements training.

Transportation Security Administration has recently established a requirements development organization but has not yet assessed its workforce needs or established component-specific training.

U.S. Citizenship and Immigration Services requirements officials told us that they have not assessed their workforce and training needs, as they are more focused on processes supporting information technology programs rather than requirements overall.

Acquisition Programs Benefit from an Adequately Staffed and Trained Requirements Workforce

Assessment and training—according to GAO’s internal controls, workforce development key principles, and DHS’s workforce guidance— are two key steps in workforce planning to ensure that the right numbers of people with the right skills are available at the right time. Specifically, an assessment should include an understanding of the goals and objectives of the component, the workforce needed to achieve the goals, and the capacity and capabilities needed to support workforce strategies. With a better understanding of the needs and current capabilities of the workforce, management can develop specific strategies to better educate the workforce and standardize skill levels. Organizations can then develop specific training to develop the workforce and fill areas of identified need with involvement of management and employees. Organizations can use a variety of instruction approaches for training—for example, classroom based learning; distance learning; or structured on- the-job training. When warranted, organizations should consider blending learning methods (such as web-based and instructor-led) within the same training effort to leverage resources in the most efficient way possible. See figure 6 for a notional workforce planning process that matches workforce needs with the goals of the organization.

The JRC approved a DHS-wide Requirements Specialization as a part of the Technology Manager Certification on June 21, 2018. In addition, JRC officials stated that they are expanding requirements development training and determining course content for the certification. We have previously found the importance of having the appropriate workforce as a factor in meeting an agency’s mission. Until the components assess their needs and take appropriate action, acquisition programs may continue to be at risk of not meeting end user needs, as they will not have a trained workforce to develop requirements.

Selected case study acquisition programs further illustrate the effect of a trained requirements development workforce.

Customs and Border Protection and Immigration and Customs Enforcement, TECS Modernization (not an acronym) programs: These programs illustrate the effect that knowledgeable requirements officials can have. Customs and Border Protection’s TECS program had an engineer with requirements development experience. According to this official, TECS Modernization traced all program requirements from the operational to the technical level in a matrix to ensure that they were valid and understood. A trained workforce, however, is one principle among many needed to provide a program with a sound start. In this case, a trained requirements official took the critical step of tracing the requirements to the gap, but his involvement cannot address the requirements and program executing issues that may arise throughout the life of a program. In fact, TECS Modernization later experienced changes to requirements and schedule.

In contrast, Immigration and Customs Enforcement’s TECS Modernization program officials told us that the program initially utilized contractors for requirements development. Rather than develop operational requirements to close the capability gap, development started with thousands of technical requirements. The program could not trace these requirements back to the capability gap, and could not implement the proposed solution. Immigration and Customs Enforcement re-started the program by bringing in trained requirements development personnel who worked with the end users to determine the appropriate operational requirements. Current Immigration and Customs Enforcement officials acknowledged the problems of the past but indicated that with the operational requirements now in place, they have a greater likelihood of success.

Transportation Security Administration, Electronic Baggage Screening and Passenger Screening Programs: End users of the screening units at an airport told us they are not aware of anyone, such as a requirements development official, with whom they can communicate emerging threats or problems with the screening units. They also said that some of the key performance parameters, such as the number of bags processed per hour, are not based on current data. In their experience, the volume of travelers and bags has increased significantly. Without a trained requirements development official with whom end users can provide input, the program risks not meeting end user needs.

U.S. Coast Guard, Offshore Patrol Cutter: Requirements officials told us that they continue to mature their requirements workforce to ensure the appropriate requirements for programs. The U.S. Coast Guard’s requirements workforce, as stated previously, utilizes an end user with requirements training as a subject matter expert on requirements development. These end users with requirements training work together with end users currently using the assets to ensure that requirements are well-defined. For this program, the U.S. Coast Guard recently held an assessment of the draft requirements for the cutter that solicited input from users across the organization. The trained requirements personnel facilitated the assessment and gathered the input to refine the requirements. While it is too early to determine how this acquisition program will perform against baselines, this initial focus on requirements is positive.

As most components recognize the need for requirements development, it is important that they assess their needs for a workforce and align those needs with training to develop a workforce that can help ensure that requirements match end user needs. DHS is taking steps to standardize training and certification across its requirements workforce to ensure that the workforce across all levels implements requirements development in accordance with JRIMS. However, DHS remains at risk until such training and certification are fully implemented throughout DHS and its components.

Conclusions

While DHS now has the JRIMS in place, which authorizes the components to create their own internal requirements development organizations, the components lag in creating the means to develop requirements and close identified capability gaps. While DHS components generally are working toward developing their own requirements policies, they have not yet established timeframes for completing this effort. Without specific timeframes, there is the risk that management attention will be lost. Further, some components do not have in place independent requirements development organizations, separate from their acquisition functions. The overlap in these responsibilities does not comport with best practices and engenders a risk that acquisition officials may override requirements developers to procure a preferred solution as opposed to the one needed by the end user.

Further, most of the components in our review have not taken steps to assess their requirements workforces and provide training. Compounding this problem is a lack of training and certification standards for requirements personnel at the agency level. Rather, components have prioritized obtaining funding and starting acquisition programs over requirements development. Not giving requirements development adequate priority is likely to contribute to poorly defined requirements and delays in achieving—or failure to achieve—the capabilities necessary to perform components’ missions.

DHS, at a department level, has recognized the importance of having a requirements policy, an independent requirements organization, and a trained workforce by establishing JRIMS, the JRC, and associated training. While the components vary in acquisition activity, it is incumbent on them to recognize the importance of these critical elements. Past acquisitions have demonstrated the need to do so.

Recommendations for Executive Action

We are making a total of 25 recommendations to the Secretary of DHS. Specifically, that the Secretary of DHS ensures that: The Commissioner of Customs and Border Protection through the Executive Assistant Commissioner for Operations Support finalizes and promulgates the Customs and Border Protection’s draft policy for requirements development. (Recommendation 1)

The Commissioner of Customs and Border Protection through the Executive Assistant Commissioner for Operations Support updates the 2013 workforce assessment to account for the independent requirements organization’s current workforce needs. (Recommendation 2)

The Commissioner of Customs and Border Protection through the Executive Assistant Commissioner for Operations Support establishes component specific training for requirements development. (Recommendation 3)

The Administrator of the Federal Emergency Management Agency establishes a policy for requirements development. (Recommendation 4)

The Administrator of the Federal Emergency Management Agency establishes an independent requirements development organization within the Federal Emergency Management Agency. (Recommendation 5)

The Administrator of the Federal Emergency Management Agency updates the 2016 workforce assessment to account for an independent requirements organization’s workforce needs. (Recommendation 6)

The Administrator of the Federal Emergency Management Agency establishes component specific training for requirements development. (Recommendation 7)

The Director of Immigration and Customs Enforcement establishes a policy for requirements development. (Recommendation 8)

The Director of Immigration and Customs Enforcement establishes the planned independent requirements development organization within Immigration and Customs Enforcement. (Recommendation 9)

The Director of Immigration and Customs Enforcement conducts a workforce assessment to account for an independent requirements organization’s workforce needs. (Recommendation 10)

The Director of Immigration and Customs Enforcement establishes component specific training for requirements development. (Recommendation 11)

The Under Secretary of Homeland Security for the National Protection and Programs Directorate finalizes and promulgates the National Protection and Programs Directorate’s draft policy for requirements development. (Recommendation 12)

The Under Secretary of Homeland Security for the National Protection and Programs Directorate establishes the planned independent requirements development organization within the National Protection and Programs Directorate. (Recommendation 13)

The Under Secretary of Homeland Security for the National Protection and Programs Directorate conducts a workforce assessment to account for an independent requirements organization’s workforce needs. (Recommendation 14)

The Under Secretary of Homeland Security for the National Protection and Programs Directorate establishes component specific training for requirements development. (Recommendation 15)

The Administrator of the Transportation Security Administration through the Executive Assistant Administrator of Operations Support finalizes and promulgates the Transportation Security Administration’s draft policy for requirements development. (Recommendation 16)

The Administrator of the Transportation Security Administration through the Executive Assistant Administrator of Operations Support conducts a workforce assessment to account for an independent requirements organization’s workforce needs. (Recommendation 17)

The Administrator of the Transportation Security Administration through the Executive Assistant Administrator of Operations Support establishes component specific training for requirements development. (Recommendation 18)

The Commandant of the U.S. Coast Guard through the Assistant Commandant for Capability conducts a workforce assessment of the U.S. Coast Guard’s capabilities directorate. (Recommendation 19)

The Director of the U.S. Citizenship and Immigration Services finalizes and promulgates the U.S. Citizenship and Immigration Services’s draft policy for requirements development. (Recommendation 20)

The Director of the U.S. Citizenship and Immigration Services establishes the planned independent requirements development organization within U.S. Citizenship and Immigration Services. (Recommendation 21)

The Director of the U.S. Citizenship and Immigration Services conducts a workforce assessment to account for an independent requirements organization’s workforce needs. (Recommendation 22)

The Director of the U.S. Citizenship and Immigration Services establishes component specific training for requirements development. (Recommendation 23)

The JRC collaborate with components on their requirements development policies and, in partnership with the Under Secretary for Management, provide oversight to promote consistency across the components. (Recommendation 24)

In addition, the Secretary of DHS should ensure that training for requirements development is consistent by establishing training and certification standards for DHS and the components’ requirements development workforces. (Recommendation 25)

Agency Comments and Our Evaluation

We provided a draft of this report for review and comment to DHS.

DHS provided written comments, which are reproduced in appendix II. In its comments, DHS concurred with all 25 of our recommendations and identified actions it plans to take to address them. DHS also provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees and the Secretary of Homeland Security. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4841 or makm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

This report discusses (1) how often selected Department of Homeland Security (DHS) programs changed requirements; and assesses the extent to which the selected components have (2) developed policies for requirements development, (3) established independent requirements organizations, and (4) taken steps to assess and train a requirements workforce. Our focus for this report was on the DHS components, as they are responsible for developing the requirements to meet end user needs.

To conduct our work, we reviewed the DHS Master Acquisition Oversight List as of April 2017 and selected seven DHS components with Level 1 and Level 2 major acquisition programs and cover a broad range of missions. The seven components are as follows:

Customs and Border Protection

Federal Emergency Management Agency

National Protection and Programs Directorate

U.S. Citizenship and Immigration Services From these seven components, we selected 14 major acquisition programs with DHS-approved key performance parameters to serve as case studies for our review. We selected a non-generalizable sample of programs based on different factors, including their acquisition phase, component, acquisition level, and whether they were information technology (IT) or non-IT. We selected the programs on these factors to reflect the broad spectrum of DHS components’ operations. In addition, we coordinated our program selection with the DHS Office of Inspector General due to its ongoing audit on the implementation of Joint Requirements Council (JRC) policies in DHS acquisition programs. See table 4 below for a description of the programs.

We also reviewed two programs that did not have DHS-approved key performance parameters at the time of our review to understand how requirements are determined before DHS validation. The two programs were Customs and Border Protection’s Cross Border Tunnel Threat and Biometric Entry-Exit Program.

To determine the extent to which the selected programs changed operational requirements, we examined key performance parameters, which the programs document in requirements and acquisitions documents, before and after DHS approval when key performance parameters should be stable. Such program documents include the operational requirements documents and acquisition program baselines. In certain cases, programs had multiple iterations of these documents. We then compared the extent to which key performance parameters changed between documents. We selected operational requirements documents and acquisition program baselines because these are the key requirements documents validated by DHS management in order for programs to begin development.

We focused on the presence of policies for requirements development, independent requirements organizations, and requirements specific workforce and training in components as our past work on major acquisitions has shown that these are the fundamental building blocks required to develop well-informed operational requirements. This selection was also informed by our standards for internal controls.

To determine the extent to which DHS components’ requirements development policies exist, as well as determine the extent to which those components established independent organizations, we reviewed component documentation pertaining to requirements development, such as instruction manuals, mission statements, and capability analyses. We also reviewed DHS documentation such as the Joint Requirements Integration and Management System Instruction Manual and the Acquisition Management Instruction to determine the requirements development guidance provided to the components. We also reviewed program-level documents such as mission need statements and operational requirements documents to determine the capability gaps that respective programs were intended to mitigate, and the programs’ key performance parameters.

To help determine assessment, training, and certification standards for DHS’s requirements development workforce, we spoke with officials from Defense Acquisition University regarding comparable standards that apply to the Department of Defense’s requirements workforce. We also reviewed training standards materials provided by these officials. In addition, we spoke with JRC and U.S. Coast Guard officials regarding their requirements development training and certification standards and reviewed available documentation.

To inform each of our objectives, we interviewed officials at various levels throughout DHS to understand their relationship to requirements development. We interviewed JRC officials to determine their interaction with components for requirements development, policies, training, and organizational standards. We also interviewed component-level officials to understand the extent to which they have implemented requirements development policies, organizations, and training for their components. We then interviewed both program officials and program end users to understand their roles in requirements development, the extent to which their feedback is incorporated into the requirements development process, and then the extent to which they receive requirements development training. In addition, we furthered this understanding through reviewing component- and program-level documentation including guidance manuals, mission needs statements, and operational requirements documents.

We assessed the components’ requirements development practices against GAO’s standards for internal control and additional supporting criteria. The standards identify key principles to help entities achieve their objectives, such as delivering capabilities to end users. Specifically, management should establish structure, responsibility, and authority including developing an organizational structure and documentation. In addition, management should have a commitment to competence by developing individuals, such as through training.

We conducted this performance audit from May 2017 to August 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Marie A. Mak, (202) 512-4841, or makm@gao.gov.

Staff Acknowledgments

In addition to the contact named above, J. Kristopher Keener, Assistant Director; James Kim; Stephen V. Marchesani; Cody Knudsen; Claire McGillem; Pete Anderson; Roxanna Sun; and Sylvia Schatz made key contributions to this report.