Homeland Security: Oversight of Neglected Human Resources Information Technology Investment Is Needed
Released: 2016-02-25T00:00:00

Letter

Background

DHS’s mission is to lead the unified national effort to secure America by preventing and deterring terrorist attacks and protecting against and responding to threats and hazards to the nation, among other things. Created in 2002, DHS merged 22 agencies and offices that specialized in one or more aspects of homeland security. The intent behind the merger that created DHS was to improve coordination, communication, and information sharing among these multiple federal agencies. Each of these agencies is responsible for specific homeland security missions and for coordinating related efforts with its sibling components, as well as external entities. Figure 1 shows a simplified and partial DHS organizational structure.

Within the department’s Management Directorate, headed by the Under Secretary for Management (USM), are the OCHCO and OCIO. The OCHCO is responsible for department-wide human capital policy and development, planning, and implementation of human capital initiatives. The OCIO is responsible for departmental IT policies, processes, and standards, and ensuring that IT acquisitions comply with DHS IT management processes, among other things.

Oversight of DHS’s Acquisition Programs

DHS acquires IT and other capabilities that are intended to improve its ability to execute its mission. DHS classifies these acquisition programs into three levels that determine the extent and scope of required project and program management, the level of reporting requirements, and the acquisition decision authority. Specifically, DHS policy defines acquisition programs as follows:

Level 1 major acquisition programs are expected to cost $1 billion or more over their life cycles.

Level 2 major acquisition programs are expected to cost at least $300 million over their life cycles.

Special interest programs, without regard to the established dollar thresholds, are designated as Level 1 or Level 2 programs. For example, a program may be raised to a higher acquisition level if its importance to DHS’s strategic and performance plans is disproportionate to its size or it has high executive visibility.

Level 3 programs are those with life-cycle cost estimates less than $300 million and are considered non-major.

As outlined in DHS’s Acquisition Management Directive 102-01, DHS’s Chief Acquisition Officer—the USM—is responsible for the management and oversight of the department’s acquisition policies and procedures. The Deputy Secretary, USM, and Component Acquisition Executives are the acquisition decision authorities for DHS’s acquisition programs. For Level 1 programs, the acquisition decision authority may be either the Deputy Secretary or USM; for level 2 programs, the acquisition decision authority may be either the USM or a Component Acquisition Executive; and for Level 3 programs, a Component Acquisition Executive is the acquisition decision authority.

As of March 2015, the department had 72 major acquisition programs and 42 non-major acquisition programs.

DHS’s Management of Human Capital Is a High- Risk Effort

In 2003, we designated the transformation of DHS as high risk because it had to transform 22 agencies—several with major management challenges—into one department. We emphasized that failure to effectively address DHS’s management and mission risks could have serious consequences for U.S. national and economic security.

In 2007 and 2009, in reporting on DHS’s progress in addressing the high- risk area since its creation, we found that DHS had made more progress in implementing its range of missions than its management functions— such as in the areas of IT and human capital—and that continued work was needed to address an array of programmatic and management challenges. Since then, DHS had continued to make important progress in strengthening and integrating its management functions; however, significant work remained for DHS to improve in these areas. For example,

As of September 2015, DHS had taken steps to identify current and future human capital needs, including the size of the workforce, its deployment across the department and components, and the knowledge, skills, abilities, and diversity needed; however, DHS had yet to fully implement its workforce planning model that was intended to allow the department to plan for its current and future organizational and workforce needs.

In February 2015, we reported that while DHS established a human capital strategic plan in 2011 and made progress in implementing it, the department had considerable work ahead to improve employee morale, which has decreased each year since 2011. For example, the Office of Personnel Management’s 2014 Federal Employee Viewpoint Survey data showed that DHS’s scores continued to decrease in all four dimensions of the survey’s index for human capital accountability and assessment.

While the department had made progress in implementing its IT Strategic Human Capital Plan for fiscal years 2010 through 2012, in January 2015 DHS shifted its IT paradigm from acquiring assets to acquiring services, and acting as a service broker (e.g., an intermediary between the purchaser of a service and the seller of that service). According to DHS officials in May 2015, this paradigm change will require a major transition in the skill sets of DHS’s IT workforce, as well as the hiring, training, and managing of those new skill sets; as such, this effort will need to be closely managed in order to succeed.

Moreover, as of September 2014, DHS faced challenges in integrating employee training management across all the components, including centralizing training and consolidating training data into one system. According to DHS officials, the department planned to address these limitations through the development and deployment of HRIT’s PALMS program.

Overview of DHS’s Human Resources Information Technology Investment

Since DHS was created, the department’s human resources environment has included fragmented systems, duplicative and paper-based processes, and little uniformity of data management practices. According to DHS, these limitations in its human resources environment are compromising the department’s ability to effectively and efficiently carry out its mission. For example,

While it is imperative that DHS responds quickly to emergencies, catastrophic events, and threats, and deploys appropriately trained, certified, and skilled personnel during these events, according to DHS, the department’s hiring process involves numerous systems and multiple hand-offs which result in extra work and prolonged hiring. This inefficient process is one factor that could have contributed to the skill and workforce gaps that we have previously identified. For example, in April 2015, we reported that 21 of the 22 major acquisition programs we reviewed faced shortfalls in their program office workforce in fiscal year 2014.

According to DHS, the department does not have information on all of its employees, which reduces its abilities to strategically manage its workforce and best deploy people in support of Homeland Security missions.

According to DHS, reporting and analyzing enterprise human capital data are currently time-consuming, labor-intensive, and challenging because the department’s data management largely consists of disconnected, standalone systems, with multiple data sources for the same content. As one example, we reported in 2014 that DHS could not provide complete information on how much it had spent on administratively uncontrollable overtime to its personnel from fiscal years 2008 through 2014. Specifically, certain components could not provide information such as duty location or payments for certain years.

To address these issues, in 2003, DHS initiated the HRIT investment, which is intended to consolidate, integrate, and modernize the department’s and its components’ human resources IT infrastructure. These components include U.S. Customs and Border Protection (CBP), the Federal Emergency Management Agency (FEMA), the Federal Law Enforcement Training Center (FLETC), U.S. Immigration and Customs Enforcement (ICE), the Transportation Security Administration (TSA), U.S. Citizenship and Immigration Services (USCIS), the U.S. Coast Guard (USCG), and the U.S. Secret Service.

HRIT is managed by DHS’s Human Capital Business Systems unit, which is within OCHCO and has overall responsibility for HRIT. Additionally, OCIO plays a key supporting role in the implementation of HRIT by reviewing headquarters’ and components’ human resources investments, identifying redundancies and efficiencies, and delivering and maintaining enterprise IT systems.

From 2003 to 2010, DHS made limited progress on the HRIT investment, as reported by DHS’s Inspector General. This was due to, among other things, limited coordination with and commitment from DHS’s components. To address this problem, in 2010 the DHS Deputy Secretary issued a memorandum emphasizing that DHS’s wide variety of human resources processes and IT systems inhibited the ability to unify DHS and negatively impacted operating costs. The memorandum stated that, without an enterprise operating model, support for DHS’s core mission was at risk and valuable workforce management information remained difficult to acquire across the department. Accordingly, the Deputy Secretary stated that DHS could no longer sustain a component-centric approach when acquiring or enhancing human resources systems, and prohibited component spending on enhancements to existing human resources systems or acquisitions of new solutions, unless those expenditures were approved by OCHCO or OCIO. The memorandum also directed these offices to develop a department-wide human resources architecture.

In 2011, in response to the Deputy Secretary’s direction, DHS completed an effort called the Human Capital Segment Architecture, which, according to DHS, defined the department’s current (or as-is) state of human capital management processes, technology, data, and relevant personnel. Further, from this current state, the department developed a comprehensive future state (or target state) and a document referred to as the Human Capital Segment Architecture blueprint that redefined the HRIT investment’s scope and implementation time frames. As part of this effort, DHS conducted a system inventory and determined that it had 422 human resources systems and applications, many of which were single- use solutions developed to respond to a small need or links to enable disparate systems to work together. DHS reported that these numerous, antiquated, and fragmented systems inhibited its ability to perform basic workforce management functions necessary to support mission critical programs.

To address this issue, the blueprint articulated that HRIT would be comprised of 15 strategic improvement opportunity areas (e.g., enabling seamless, efficient, and transparent end-to-end hiring) and outlined 77 associated projects (e.g., deploying a department-wide hiring system, establishing an integrated data repository and reporting mechanism, and developing a centralized learning center for all personnel action processing information) to implement these 15 opportunities. Each opportunity area includes from 1 to 10 associated projects. Table 1 summarizes the scope of the 15 strategic improvement opportunities— listed in the order of DHS’s assigned priority—and identifies their original planned completion dates, as of August 2011 when the blueprint was issued.

HRIT’s only ongoing program is called PALMS and is intended to fully address the Performance Management strategic improvement opportunity area and its three associated projects. PALMS is attempting to implement a commercial off-the-shelf software product that is to be provided as a service in order to enable, among other things, comprehensive enterprise-wide tracking, reporting, and analysis of employee learning and performance for DHS headquarters and its eight components. Specifically, PALMS is expected to deliver the following capabilities: Learning management. The learning management capabilities are intended to manage the life cycle of learning activities for all DHS employees and contractors. PALMS is intended to, among other things, act as a gateway for accessing training at DHS and record training information when a user has completed a course. Additionally, it is expected to replace nine disparate learning management systems with one unified system.

Performance management. The performance management capabilities are intended to move DHS’s existing primarily paper- based performance management processes into an electronic environment and capture performance-related information throughout the performance cycle (e.g., recording performance expectations discussed at the beginning of the rating period and performance ratings at the end of it).

Each component is responsible for its own PALMS implementation project, and is expected to issue a task order using a blanket purchase agreement that was established in May 2013 with an estimated value of $95 million. Before implementing PALMS, each component is completing a fit-gap assessment to, among other things, identify any requirements and critical processes that cannot be met by the preconfigured, commercial off-the-shelf system. If such component- specific requirements are identified, the component must then decide whether to have the vendor customize the system.

The headquarters PALMS program management office (PMO) is responsible for overseeing the implementation projects across the department. Additionally, OCIO is the Component Acquisition Executive responsible for overseeing PALMS.

In addition to implementing projects intended to address the strategic improvement opportunities in the blueprint, the HRIT investment also carried out the following two projects that were not included in the blueprint:

Balanced Workforce Assessment Tool: This project provided an enterprise-wide tool to automate the formerly paper-based balanced workforce strategy process to determine the appropriate mix of federal employees and contractor employees required to fulfill a specific work function in the government. DHS deployed this tool beginning in September 2013.

Workers Compensation – Medical Case Management Services: This project provided an enterprise-wide contract to enable nurses to execute case management processes and facilitate the case management activities to be performed by DHS human resources staff. As part of this, the project provided access to a web application where DHS workers’ compensation coordinators could work on cases with nurses. As of March 2015, the tool had been implemented at six components.

Best Practices for Planning and Managing IT Acquisition Programs

Entities such as the Project Management Institute, the Software Engineering Institute at Carnegie Mellon University, and GAO have developed and identified best practices to help guide organizations to effectively plan and manage their acquisitions of major IT systems. Our prior reviews have shown that proper implementation of such practices can significantly increase the likelihood of delivering promised system capabilities on time and within budget. These practices include, but are not limited to:

Project planning: Establishes project objectives and outlines the course of action required to attain those objectives. It also provides a means to track, review, and report progress and performance of the project by defining project activities and developing cost and schedule estimates, among other things.

Project monitoring and control: Provides an understanding of the project’s progress, so that appropriate corrective actions can be taken if performance deviates from plans. Effective practices in this area include, among other things, determining progress against the program plan and conducting program management reviews.

Risk management: Establishes a process for anticipating problems and taking appropriate steps to mitigate risks and minimize their impact on program commitments. It involves identifying and documenting risks, categorizing them based on their estimated impact, prioritizing them, developing risk mitigation strategies, and tracking progress in executing the strategies.

DHS Has Made Very Little Progress in Implementing HRIT; Investment Lacked Effective Management

DHS has made very little progress in delivering planned HRIT capabilities, such as end-to-end hiring and payroll action processing. While the vast majority of HRIT capabilities (called strategic improvement opportunities) were to be delivered by June 2015, only 1 has been fully implemented, and the completion dates for the other 14 are currently unknown. These delays are largely due to unplanned resource changes and the lack of involvement from the executive oversight committee. In addition, the department did not effectively manage the investment. For example, DHS did not update or maintain the HRIT schedule, have a life- cycle cost estimate, or track all associated costs. Moreover, the strategic planning document—referred to as the Human Capital Segment Architecture Blueprint—has not been updated in approximately 4.5 years and, as a result, the department does not know whether it is reflective of current priorities and goals. As a result of DHS’s ineffective management and limited progress in implementing this investment, the department is unaware of when critical weaknesses in the department’s human capital environment will be addressed, which is, among other things, impacting DHS’s ability to reduce duplication and carry out its mission.

Limited Progress Has Been Made in Implementing HRIT

DHS has made very limited progress in addressing the 15 strategic improvement opportunities and the 77 associated projects included in HRIT. According to the Human Capital Segment Architecture Blueprint, DHS planned to implement 14 of the 15 strategic improvement opportunities and 68 of the 77 associated projects by June 2015; and the remaining improvement opportunity and 9 associated projects by December 2016. However, as of November 2015, DHS had fully implemented only 1 of the strategic improvement opportunities, which included 2 associated projects. This improvement opportunity established an enterprise-wide governance process for evaluating HRIT projects and proposals prior to funding them. This process is referred to as the investment intake process and is intended to help encourage the use of enterprise-level investments, rather than component-specific investments, by preventing components from investing in duplicative systems when an existing DHS capability can meet a particular business need. Table 2 summarizes the implementation status and planned completion dates of the strategic improvement opportunities—listed in the order of DHS’s assigned priority—as of November 2015.

DHS has partially implemented five of the other strategic improvement opportunities, but it is unknown when they will be fully addressed. For example, DHS’s PALMS program is intended to fully address the blueprint’s strategic improvement opportunity for Performance Management; however, while progress to implement PALMS has been made, many actions remain before it can be fully implemented and it is unknown when those actions will be taken (discussed in more detail later). Further, HRIT officials stated that DHS has not yet started to work on the remaining nine improvement opportunities, and the officials did not know when they would be addressed.

Additionally, DHS developed an HRIT strategic plan for fiscal years 2012 through 2016 that outlined the investment’s key goals and objectives, including reducing duplication and improving efficiencies in the department’s human resources processes and systems. The strategic plan identified, among other things, two performance metrics and associated targets for delivering human resources IT services across DHS. These performance metrics were focused on reductions in the number of component-specific human resources IT services provided and increases in the number of department-wide HRIT services provided by the end of fiscal year 2016.

However, DHS has also made limited progress in achieving these two performance targets. Figure 2 provides a summary of HRIT’s progress towards achieving its service delivery performance targets.

DHS’s goal is to reduce its component-specific HRIT services by 46 percent—from 81 percent to 35 percent—however, it had reduced these services by 8 percent as of November 2015, according to OCHCO officials.

Additionally, while DHS is aiming to increase its DHS-wide HRIT services by 38 percent—from 2 percent to 40 percent—as of November 2015, OCHCO officials stated that the department had increased these services by 8 percent.

Key causes for DHS’s lack of progress in implementing HRIT and its associated strategic improvement opportunities include unplanned resource changes and the lack of involvement of the HRIT executive steering committee. These causes are discussed in detail below:

Unplanned resource changes. DHS elected to dedicate the vast majority of HRIT’s resources to implementing PALMS and addressing its problems, rather than initiating additional HRIT strategic improvement opportunities. Specifically, PALMS—which began in July 2012—experienced programmatic and technical challenges that led to years-long schedule delays. For example, while the PALMS system for headquarters was originally planned to be delivered by a vendor in December 2013, as of November 2015, the expected delivery date was delayed until the end of February 2016—an over 2-year delay. HRIT officials explained the decision to focus primarily on PALMS was due, in part, to the investment’s declining funding stream. However, in doing so, attention was concentrated on the immediate issues affecting PALMS and diverted from the longer-term HRIT mission.

Lack of involvement of the HRIT executive steering committee. The HRIT executive steering committee—which is chaired by the department’s Under Secretary for Management and co-chaired by the Chief Information Officer and Chief Human Capital Officer—is intended to be the core oversight and advisory body for all DHS-wide matters related to human capital IT investments, expenditures, projects, and initiatives. In addition, according to the committee’s charter, the committee is to approve and provide guidance on the department’s mission, vision, and strategies for the HRIT program.

However, the executive steering committee only met once from September 2013 through June 2015—in July 2014—and was minimally involved with HRIT for that almost 2 year period. It is important to note that DHS replaced its Chief Information Officer (the executive steering committee’s co-chair) in December 2013—during this gap in oversight. Also during this time period HRIT’s only ongoing program—PALMS—was experiencing significant problems, including schedule slippages and frequent turnover in its program manager position (i.e., PALMS had five different program managers during the time that the HRIT executive steering committee was minimally involved). As a result of the executive steering committee not meeting, key governance activities were not completed on HRIT. For example, the committee did not approve HRIT’s notional operational plan for fiscal years 2014 through 2019. OCHCO and OCIO officials attributed the lack of HRIT executive steering committee meetings and committee involvement in HRIT to the investment’s focus being only on the PALMS program to address its issues, as discussed earlier. However, by not regularly meeting and providing oversight during a time when a new co-chair for the executive steering committee assumed responsibility and PALMS was experiencing such problems, the committee’s guidance to the troubled program was limited.

More recently, the HRIT executive steering committee met in June and October 2015, and OCIO and OCHCO officials stated that the committee planned to meet quarterly going forward. However, while the committee’s charter specified that it meet on at least a monthly basis for the first year, the charter does not specify the frequency of meetings following that year. Furthermore, the committee’s charter has not been updated to reflect the increased frequency of these meetings.

As a result of the limited progress in implementing HRIT, DHS is unaware of when critical weaknesses in the department’s human capital environment will be addressed, which is, among other things, impacting DHS’s ability to carry out its mission. For example, the end-to-end hiring strategic improvement opportunity (which has an unknown implementation date) was intended to streamline numerous systems and multiple hand-offs in order to more efficiently and effectively hire appropriately skilled personnel, thus enabling a quicker response to emergencies, catastrophic events, and threats. As another example, the data management and sharing strategic improvement opportunity (which also has an unknown implementation date) was intended to enable the department to have visibility of all its employees, to improve its ability to strategically manage its workforce, and best deploy people in support of DHS missions. Therefore, until HRIT’s executive steering committee effectively carries out its oversight responsibility, DHS will be limited in its ability to improve HRIT investment results and accountability.

HRIT Lacked a Current Schedule, Life-Cycle Cost Estimate, and Cost Tracking

DHS Did Not Update or Maintain the HRIT Schedule

According to the GAO Schedule Assessment Guide, a key activity in effectively managing a program and ensuring progress is establishing and maintaining a schedule estimate. Specifically, a well maintained schedule enables programs to gauge progress, identify and resolve potential problems, and forecast dates for program activities and completion of the program.

In August 2011, DHS established initiation and completion dates for each of the 15 strategic improvement opportunities within the Human Capital Segment Architecture Blueprint. Additionally, HRIT developed a slightly more detailed schedule for fiscal years 2014 through 2021 that updated planned completion dates for aspects of some strategic improvement opportunities, but not all.

However, DHS did not update and maintain either schedule after they were developed. Specifically, neither schedule was updated to reflect that DHS did not implement 13 of the 15 improvement opportunities by their planned completion dates—several of which should have been implemented over 3 years ago. HRIT officials attributed the lack of schedule updates to the investment’s focus shifting to the PALMS program when it started experiencing significant schedule delays. Without developing and maintaining a current schedule showing when DHS plans to implement the strategic improvement opportunities, DHS and Congress will be limited in their ability to oversee and ensure DHS’s progress in implementing HRIT.

HRIT Did Not Have a Life- Cycle Cost Estimate

OMB requires that agencies prepare total estimated life-cycle costs for information technology investments. Program management best practices also stress that key activities in planning and managing a program include establishing a life-cycle cost estimate and tracking costs expended. A life-cycle cost estimate supports budgetary decisions and key decision points, and should include all costs for planning, procurement, and operations and maintenance of a program.

OCHCO officials stated that a draft life-cycle cost estimate for HRIT was developed, but that it was not completed or finalized because detailed projects plans for the associated projects had not been developed or approved. According to the HRIT blueprint, OCHCO roughly estimated that implementing all of the projects could cost up to $120 million. However, the blueprint specifies that this figure did not represent the life- cycle cost estimate; rather it was intended to be a preliminary estimate to initiate projects. Without a life-cycle cost estimate, DHS has limited information about how much it will cost to implement HRIT, which hinders the department’s ability to, among other things, make budgetary decisions and informed milestone review decisions.

DHS Did Not Track All Costs Incurred on HRIT

According to CMMI-ACQ and the PMBOK® Guide, programs should track program costs in order to effectively manage the program and make resource adjustments accordingly. In particular, tracking and monitoring costs enables a program to recognize variances from the plan in order to take corrective action and minimize risk.

However, DHS has not tracked the total actual costs incurred on implementing HRIT across the enterprise to date. Specifically, while the investment received line item appropriations for fiscal years 2005 through 2015 which totaled at least $180 million, DHS was unable to provide all cost information on HRIT activities since it began in 2003, including all government-related activities and component costs that were financed through the working capital fund, which, according to DHS officials from multiple offices, were provided separately from the at least $180 million appropriated specifically to HRIT. OCHCO officials attributed the lack of cost tracking to, among other things, the investment’s early reliance on contractors to track costs, and said that the costs were not well maintained nor centrally tracked, and included incomplete component- provided cost information. The components were also unable to provide us with complete information. For example, FEMA officials stated that it would require a significant administrative effort to identify how much it has spent on HRIT since inception in 2003 because of the way their financial system obligates and expends funds for Working Capital Fund activities.

USCG officials also said that compiling its expenditure information for fiscal years 2003-2009 would require a substantial administrative effort, including reviewing a significant number of paper files.

USCIS was unable to identify its HRIT-related expenditures for fiscal years 2003-2010.

Without tracking all costs associated with HRIT, including components’ costs, stakeholders are limited in making informed resource decisions, and DHS cannot provide complete and accurate information to assist congressional oversight.

HRIT’s 2011 Blueprint May Not Be Valid and Reflective of DHS’s Current Priorities and Goals

According to the HRIT executive steering committee’s charter, the Under Secretary for Management (as the chair of the committee) is to ensure that the department’s human resources IT business needs are met, as outlined in the blueprint. Additionally, according to the GPRA (Government Performance and Results Act) Modernization Act of 2010, agency strategic plans should be updated at least every 4 years. While this is a legal requirement for agency strategic plans (the Human Capital Segment Architecture blueprint does not fall under the category of an “agency strategic plan”), it is considered a best practice for other strategic planning documents, such as the blueprint.

However, the department issued the blueprint in August 2011 (approximately 4.5 years ago) and has not updated it since. As a result, the department does not know whether the remaining 14 strategic improvement opportunities and associated projects that it has not fully implemented are still valid and reflective of DHS’s current priorities, and are appropriately prioritized based on current mission and business needs. Additionally, DHS does not know whether new or emerging opportunities or business needs need to be addressed.

Officials stated that the department is still committed to implementing the blueprint, but agreed that it should be re-evaluated. To this end, following a meeting we had with DHS’s Under Secretary for Management in October 2015, in which we expressed concern about HRIT’s lack of progress, OCHCO and OCIO officials stated that HRIT was recently asked by the Deputy Under Secretary of Management in late October 2015 to re-evaluate the blueprint’s strategic improvement opportunities and to determine the way forward for those improvement opportunities and the HRIT investment. However, officials did not know when this re- evaluation and a determination for how to move forward with HRIT would occur, or be completed.

Further, according to OCIO officials, DHS has not updated its complete systems inventory since it was originally developed as part of the blueprint effort, in response to a 2010 Office of Inspector General report that stated that DHS had not identified all human resource systems at the components. This report also emphasized that without an accurate inventory of human resource systems, DHS cannot determine whether components are using redundant systems. Moreover, OCIO officials were unable to identify whether and how its inventory of human resources systems had changed.

Until DHS establishes time frames for re-evaluating the blueprint to reflect DHS’s HRIT current priorities and updates its human resources system inventory, the department will be limited in addressing the inefficient human resources environment that has plagued the department since it was first created.

DHS Justified Its Investment in the PALMS Program

DHS took several steps to justify its investment in the PALMS program for both of the program’s two main purposes (the learning management capabilities and the performance management capabilities) through multiple mechanisms. Specifically, although existing DHS guidance did not require an analysis of alternatives for PALMS because it is a Level 3 acquisition program, the department initiated such an analysis in 2010 to identify recommended approaches for pursuing a commercial off-the- shelf learning management system to replace the components’ nine existing learning systems. According to the analysis of alternatives, the nine systems at the department were disconnected from each other and did not exchange information. The components had independently purchased these learning management systems and, in some cases, had done so before DHS was established in 2002. However, DHS determined that a unified strategy for learning management systems at the department was needed, rather than disparate, component-centric efforts. In particular, DHS determined that such a strategy was necessary to provide, among other things, improved reporting, greater automation, less duplication and redundancy of training courses, better governance, and streamlined IT infrastructure.

The analysis of alternatives, which was performed by the Homeland Security Studies and Analysis Institute, included, among other things, an assessment of six alternative approaches, including status quo, implementation of two systems from separate vendors (allowing components to choose which system to use), and implementation of a single system (either centrally managed by DHS or individually managed by each component). As part of the analysis, the Institute assessed the alternative approaches based on five evaluative categories, including cost, benefits, and risks. Based on the analysis of alternatives process, the institute recommended that DHS adopt a single enterprise-wide, centrally managed learning management system as the most cost- effective approach to providing such a capability to the department.

Regarding the second purpose of PALMS—enabling performance management capabilities—the August 2011 Human Capital Segment Architecture Blueprint called on DHS to conduct an analysis of alternatives to identify the preferred approach for such a solution. Officials stated that DHS leadership ultimately determined that such an analysis for a performance management solution was unnecessary because the requirement for DHS to automate performance management functions across the department was the same as it was during DHS’s prior attempt to pursue an automated performance management system for instituting pay-for-performance—an effort that was ultimately abandoned.

Therefore, instead of conducting an analysis of alternatives on performance management system approaches for DHS enterprise-wide adoption, in January 2012, departmental leadership made an executive decision on the approach based on the findings of a December 2011 request for information from industry. In particular, the accumulated industry information highlighted that vendors for an enterprise-wide learning management solution could in most cases also provide a system that integrated performance management capabilities. This industry information validated DHS officials’ understanding that a combined solution for learning and performance management at the department was consistent with prevailing industry offerings. According to OCHCO officials, the department’s request for information from industry to help justify its preferred approach allowed for competition within industry for supplying a solution to the department. As part of the department’s considerations, officials had determined that this competition could better help to reduce overall implementation costs for a consolidated learning and performance management system, versus adopting, without competition, one of the components’ existing learning or performance management systems for DHS enterprise-wide deployment.

Additionally, OCHCO officials stated that they contacted other federal departments to determine whether existing shared services could be used by DHS to establish an integrated system for learning and performance management, but DHS determined that other departments’ contracts with service providers could not be modified to allow DHS to use the same services.

Based on the collective results of the learning management system analysis of alternatives and the request for information from industry on performance management systems, the HRIT executive steering committee exercised its executive decision-making authority and decided that an integrated, enterprise-wide learning and performance management system should be pursued for adoption at the department. DHS’s integrated solution is now being implemented by the PALMS program. By providing the executive steering committee with enough information for determining this preferred approach for the department, DHS justified its investment in the PALMS program.

Selected PALMS Capabilities Have Been Deployed to Headquarters and Two Components; but Full Implementation at Four Components Is Not Currently Planned

As previously mentioned, PALMS is intended to provide an enterprise- wide system that offers performance management capabilities, as well as learning management capabilities to headquarters and each of its components. As such, DHS headquarters PMO and the components estimate that, if fully implemented across DHS, PALMS’s learning management capabilities would be used by approximately 309,360 users, and its performance management capabilities would be used by at least 217,758 users. Table 3 identifies the total estimated number of planned users for both PALMS’s learning management capabilities and performance management capabilities if PALMS is fully implemented department-wide.

However, there is uncertainty about whether the PALMS system will be used enterprise-wide to accomplish these goals. Specifically, as of November 2015, of the eight components and headquarters, five are planning to implement both PALMS’s learning and performance management capabilities (three of which have already implemented the learning management capabilities—discussed later), two are planning to implement only the learning management capabilities, and two components are not currently planning to implement either of these PALMS capabilities, as illustrated in figure 3.

Officials from FEMA, TSA, ICE, and the USCG cited various reasons for why they were not currently planning to fully implement PALMS, which include: FEMA and ICE officials stated that they were not currently planning to implement the performance management capabilities because the program had experienced critical deficiencies in meeting the performance management-related requirements. FEMA officials stated that they do not plan to make a decision on whether they will or will not implement these performance management capabilities until the vendor can demonstrate that the system meets FEMA’s needs; as such, FEMA officials were unable to specify a date for when they plan to make that decision. ICE officials also stated that they do not plan to implement the performance management capabilities of PALMS until the vendor can demonstrate that all requirements have been met. PALMS headquarters PMO officials expected all requirements to be met by the vendor by the end of February 2016.

TSA officials stated that they were waiting on the results of their fit- gap assessment of PALMS before determining whether, from a cost and technical perspective, TSA could commit to implementing the learning and/or performance management capabilities of PALMS. TSA officials expected the fit-gap assessment to be completed by the end of March 2016.

USCG officials stated that, based on the PALMS schedule delays experienced to date, they have little confidence that the PALMS vendor could meet the component’s unique business requirements prior to the 2018 expiration of the vendor’s blanket purchase agreement. Additionally, these officials stated that the system would not meet all of its learning management requirements for about 31,000 auxiliary volunteer members and certain other employee groups. Further, although the fit gap assessment for implementing PALMS at USCG had not been fully completed, the component’s officials stated that the system would likely not fully meet the performance management requirements for all of USCG’s military components. Due to the component’s uncertainty, the officials were unable to specify when they plan to ultimately decide on whether they will implement one or both aspects of PALMS.

As a result, it is unlikely that the department will reach its expected user estimates as presented in table 3, and meet its goal of being an enterprise-wide system. Specifically, as of November 2015, the components estimate 179,360 users will use the learning management capabilities of PALMS (not the 309,360 expected, if fully implemented). Figure 4 shows the percentage of expected users from components currently planning to implement PALMS’s learning management capabilities in comparison to the total expected users if PALMS was fully implemented, as of November 2015.

Additionally, as of November 2015, the components estimate 123,200 users will use the performance management capabilities of PALMS (not the 217,758 expected, if fully implemented). Figure 5 shows the percentage of expected users from components planning to implement PALMS’s performance management capabilities in comparison to the total expected user estimate if fully implemented as intended.

Of the seven components and headquarters that are currently planning to implement the learning and/or performance management aspects of PALMS, three have completed their implementation efforts of the learning management capabilities and deployed these capabilities to users (deployed to CBP in July 2015, headquarters in October 2015, and FLETC in December 2015); two have initiated their implementation efforts on one or both aspects, but not completed them; and two have not yet initiated any implementation efforts, as of November 2015.

As a result, PALMS’s current trajectory is putting the department at risk of not meeting its goals to perform efficient, accurate, and comprehensive tracking and reporting of training and performance management data across the enterprise; and consolidating its nine learning management systems down to one. Accordingly, until FEMA decides whether it will implement the performance management capabilities of PALMS and USCG decides whether it will implement the learning and/or performance management capabilities of PALMS, the department is at risk of implementing a solution that does not fully address its problems. Moreover, until DHS determines an alternative approach if one or both aspects of PALMS is deemed not feasible for ICE, TSA, FEMA or the USCG, the department is at risk of not meeting its goal to enable enterprise-wide tracking and reporting of employee learning and performance management.

PALMS Program Had Made Mixed Progress in Implementing Key IT Acquisition Best Practices

HRIT’s PALMS program varied in its implementation of IT acquisition best practices for project planning, project monitoring, and risk management. Specifically, the program management office had implemented selected IT acquisition best practices in each of these areas; however, the program had not developed complete life-cycle cost and schedule estimates. Additionally, the PALMS PMO did not monitor total costs spent on the program or consistently document the results from progress and milestone reviews. Further, the program management office had not fully implemented selected risk management practices. Without fully implementing effective acquisition management practices, DHS is limited in monitoring and overseeing the implementation of PALMS, ensuring that the department obtains a system that improves its performance management and learning management weaknesses, reduces duplication, and delivers within cost and schedule commitments.

DHS and the PALMS PMO Had Not Fully Implemented Selected Project Planning Best Practices

PALMS Lacked a Life-Cycle Cost Estimate

According to GAO’s Cost Estimating and Assessment Guide, having a complete life-cycle cost estimate is a critical element in the budgeting process that helps decision makers to evaluate resource requirements at milestones and other important decision points. Additionally, a comprehensive cost estimate should include both government and contractor costs of the program over its full life cycle, from inception of the program through design, development, deployment, and operation and maintenance to retirement of the program.

However, according to PALMS PMO officials, they did not develop a life- cycle cost estimate for PALMS. In 2012 DHS developed an independent government cost estimate to determine the contractor-related costs to implement the PALMS system across the department (estimated to be approximately $95 million); however, this estimate was not comprehensive because it did not include government-related costs. As a result, DHS was not able to determine the impact on cost when the PALMS program experienced problems (discussed in more detail later), since the baseline cost estimate was incomplete. PALMS PMO officials stated that PALMS did not develop a life-cycle cost estimate because the program is a Level 3 acquisition program and DHS does not require such an estimate for a Level 3 program. However, while DHS acquisition policy does not require a life-cycle cost estimate for a program of this size, we maintain that such an estimate should be prepared because of the program’s risk and troubled history. Without developing a comprehensive life-cycle cost estimate, DHS is limited in making future budget decisions related to PALMS.

PALMS’s Schedule Was Incomplete and Inaccurate

As described in GAO’s Schedule Assessment Guide, a program’s integrated master schedule is a comprehensive plan of all government and contractor work that must be performed to successfully complete the program. Additionally, such a schedule helps manage program schedule dependencies. Best practices for developing and maintaining this schedule include, among other things, capturing all activities needed to do the work and reviewing the schedule after each update to ensure the schedule is complete and accurate.

While DHS had developed an integrated master schedule with the PALMS vendor, it did not appropriately maintain this schedule. Specifically, the program’s schedule was incomplete and inaccurate.

While DHS’s original August 2012 schedule planned to fully deploy both the learning and performance management capabilities in one release at each component by March 2015, the program’s September 2015 schedule did not reflect the significant change in PALMS’s deployment strategy and time frames. Specifically, the program now plans to deploy the learning management capabilities first and the performance management capabilities separately and incrementally to headquarters and the components. However, the September 2015 schedule reflected the deployment-related milestones (per component) for only the learning management capabilities and did not include the deployment-related milestones for the performance management capabilities.

In September 2015, PALMS officials stated that the deployments related to performance management were not reflected in the program’s schedule because the components had not yet determined when they would deploy these capabilities. Since then, two components have determined their planned dates for deploying these capabilities, but seven (including headquarters) remain unknown. As a result, the program does not know when PALMS will be fully implemented at all components with all capabilities. Table 4 provides a comparison of the program’s initial delivery schedule, as of August 2012, to the program’s latest schedule, as of November 2015.

Moreover, the schedule did not include all government-specific activities, including tasks related to employee union activities (such as notifying employee unions and bargaining with them, where necessary) related to the proposed implementation of the performance management capabilities. For example, time frames for when DHS planned to notify employee unions at DHS headquarters, FLETC, and USCIS were not identified in the schedule. In September 2015, PALMS program officials stated that certain government- specific tasks were not included in the schedule because the integrated master schedule was too big and difficult to manage, so the program decided to track certain government activities, such as union negotiation activities, separately. However, without an integrated master schedule that includes all government and contractor work that must be performed, the program is at risk of failing to ensure schedule dependencies are appropriately managed and that all essential activities are completed.

Additionally, the August 2015 schedule had incorrect completion dates listed for key activities. For example, DHS reported in the schedule that the actual finish date for deploying the learning management capabilities of the PALMS system at CBP was February 17, 2015; however, according to CBP officials, they did not deploy these capabilities until July 2015. In September 2015, program officials acknowledged our concerns and attributed the inaccurate dates to a lack of oversight; subsequently, the program took actions to update the dates.

Without developing and maintaining a single comprehensive schedule that fully integrates all government and contractor activities, and includes all planned deployment milestones related to performance management, DHS is limited in monitoring and overseeing the implementation of PALMS, and managing the dependencies between program tasks and milestones to ensure that it delivers capabilities when expected.

PALMS Had Implemented Selected Project Monitoring Best Practices, but Did Not Implement Others

The PALMS PMO Did Not Monitor Total Costs

According to CMMI-ACQ and the PMBOK® Guide, a key activity for tracking a program’s performance is monitoring the project’s costs by comparing actual costs to the cost estimate. The PALMS PMO—which is responsible for overseeing the PALMS implementation projects across DHS, including all of its components—monitored task order expenditures on a monthly basis. As of December 2015, DHS officials reported that they had awarded approximately $18 million in task orders to the vendor.

However, the program management office officials stated that they were not monitoring the government-related costs associated with each of the PALMS implementations. The officials stated that they were not tracking government-related implementation costs at headquarters because many of the headquarters program officials concurrently work on other acquisition projects and these officials are not required to track the amount of time spent working specifically on PALMS. The officials also said that they were not monitoring the government-related costs for each of the component PALMS implementation projects because it would be difficult to obtain and verify the cost data provided by the components. We acknowledge the department’s difficulties associated with obtaining and verifying component cost data; however, monitoring the program’s costs is essential to keeping costs on track and alerting management of potential cost overruns.

Additionally, because DHS did not develop a comprehensive life-cycle cost estimate for PALMS that included government-related costs, the program management office was unable to determine cost increases to the program because it could not compare actual cost values against a baseline cost estimate. For example, program officials were unable to identify how much the program’s cost estimate had increased when the implementation at headquarters experienced schedule delays to address deficiencies identified during testing. Without tracking and monitoring all costs associated with PALMS, the department will be unable to compare actual costs against planned estimates and thus, will be limited in its ability to fully monitor the program, which is essential for alerting the program to possible cost overruns and prompting corrective actions.

The PALMS PMO Did Not Consistently Document the Results from Performance Reviews

According to CMMI-ACQ and the PMBOK® Guide, key activities in tracking a program’s performance include conducting and documenting the results from progress and milestone reviews to determine whether there are significant issues or performance shortfalls that need to be addressed.

Although the PALMS PMO conducted reviews to monitor the program’s performance, it did not consistently document the results of its progress and milestone reviews. For example, The PALMS PMO did not document the results of the status updates that the PMO provided to DHS executives during its bi-weekly integrated project team meetings, so it is unclear whether the program was appropriately monitoring the progress of all government-specific activities.

According to PALMS PMO officials, PALMS achieved Initial Operating Capability—which was specified in the contract to be the point when the contractor would deliver an initial set of requirements to the government—in January 2015; however, the review for this major milestone was not documented. In September 2015, program officials stated that the results were not documented because this milestone did not align with the typical Initial Operating Capability milestone that is defined in DHS acquisition guidance. Specifically, DHS’s guidance defines it as when capabilities are first deployed to end users (PALMS capabilities were not deployed to any users until July 2015). Nevertheless, PALMS’s achieving Initial Operating Capability in January 2015 was still considered a major milestone that prompted a review. However, without documenting the results of the milestone review, it is unclear whether any action items were identified during this review and, if so, whether they have all been appropriately managed to closure.

Although CBP officials stated that the results of their progress reviews with the vendor were typically documented, CBP was unable to provide the results of the milestone review conducted prior to deploying the PALMS learning management capabilities in July 2015. As such, it is unclear whether any action items were identified during this review and, if so, whether CBP had appropriately managed them to closure.

In the absence of documenting PALMS’s progress and milestone reviews, including all issues and corrective actions discussed, the program cannot demonstrate that these issues and corrective actions are appropriately managed.

The PALMS PMO Had Implemented Selected Risk Management Best Practices, but Lacked Others

PALMS Was Not Regularly Tracking the Status of Its Risks and Mitigation Efforts

According to CMMI-ACQ and the PMBOK® Guide, key risk management practices include identifying risks, developing mitigation plans, and regularly tracking the status of risks and mitigation efforts. In particular, identifying risks and periodically reviewing them is the basis for sound and successful risk management. Additionally, risk mitigation plans should be developed and implemented when appropriate to proactively reduce the potential impact if a risk were to occur.

While PALMS officials had identified program risks, developed associated mitigation plans, and documented them in the HRIT investment-level risk log (which is intended to be the centralized log containing all PALMS risks and mitigation plans, including both government- and vendor-identified risks), the program did not consistently maintain this log. Specifically, the PALMS risks in this log were out of date, the log did not accurately capture the status of all of the risks identified by the program, and it was unclear which risks and associated mitigation plans were being assessed on a monthly basis. For example, In the May 2015 risk log, 16 of the 17 active PALMS risks stated that the last time any action was taken to mitigate or close any of these risks was in 2014. However, the mitigation strategy details for 5 of these active risks included information related to decisions made in 2015. As such, it was unclear which risks and mitigation plans were regularly assessed and updated in the risk log, and when actions were last taken on each of the risks.

One of the high-impact and high-probability risks from the May 2015 risk log stated that DHS needed to determine an interim solution for consolidating human resources-related data from DHS’s components by December 2014; however, the status of this risk had not been updated since August 2014 and it was unclear whether this was still a risk or had been realized as an issue.

Additionally, while the HRIT investment-level risk management plan identified that the PALMS program was to, among other things, generate weekly status reports to document the status of decisions made during risk review meetings and identify planned completion dates for each step of the risk mitigation plans, the program was not always complying with these processes. For example, the program was not developing the required weekly risk status reports or identifying planned completion dates for its risk mitigation plan steps.

Program officials acknowledged that the PALMS risks in the HRIT risk log were out of date and inaccurate, and the program was not complying with all of the documented processes in the HRIT risk management plan. Program officials attributed this to, among other things, the PMO’s focus being on meeting upcoming deadlines; as such, implementing certain processes identified in the HRIT risk management plan were not a priority. However, by not carrying out these key risk management functions, program officials introduced additional risk to the program.

In October 2015 and in response to us identifying these issues, PALMS officials stated that they were in the process of validating and updating the risks and mitigation plans in the HRIT risk log to address these issues, as well as were updating their risk management processes to align with the documented processes in the HRIT risk management plan. The program completed this validation update process in October 2015; however, the updated log continued to have these issues. For example, the PALMS PMO had not yet identified the planned completion dates for each mitigation step (where appropriate). Further, this updated log— which is intended to be the program’s centralized log of all government- and vendor-identified PALMS risks—did not contain all of the vendor- identified risks. For example, two component-specific risks that were identified in the vendor-maintained risk log were not included in the program’s centralized risk log. As such, it is unclear whether the program is appropriately managing these risks. Until a comprehensive risk log is established that accurately captures the status of all risks (including both government- and vendor-identified risks) and mitigation plans, and includes planned completion dates for each mitigation step (where appropriate), the program is limited in effectively managing all of its risks.

The PALMS PMO Did Not Prioritize the Program’s Risks or Document Criteria for Elevating Component Risks

According to CMMI-ACQ and PMBOK® Guide risk management best practices, effective risk management includes evaluating and categorizing risks using defined risk categories and parameters, such as probability and impact, and determining each risk’s relative priority. Risk prioritization helps to determine where resources for risk mitigation can be applied to provide the greatest positive impact on the program. The parameters for evaluating, categorizing, and prioritizing risks should include defined thresholds (e.g., for cost, schedule, performance) that, when exceeded, trigger management attention and mitigation activities. These risk parameters should be documented so that they are available for reference throughout the life of the project and are used to provide common and consistent criteria for prompting management attention.

While the PALMS program had categorized its risks and assigned parameters to them, including probability and impact, the program did not prioritize its risks or document criteria for elevating them to management. Specifically, the PALMS PMO did not use the assigned parameters to determine each risk’s relative priority and overall risk level (i.e., high, medium, and low). PALMS officials acknowledged in June 2015 that the risks were not prioritized in the logs, but said, based on the experience of the PALMS PMO staff, officials are able to determine each risk’s priority by reviewing the assigned probability and impact parameters. However, this is an inadequate method for managing risks. Specifically, it introduces unnecessary subjectivity by relying heavily on officials to make prioritization decisions, rather than using the assigned parameters to determine and document each risk’s relative priority.

Additionally, the program had not documented criteria for elevating component risks to the program management office. As mentioned earlier, each component is responsible for overseeing its own PALMS implementation project, while the program management office at headquarters is responsible for overseeing the implementation projects across the department. According to program officials, as part of this effort, each component is to follow the risk management processes documented in PALMS’s vendor-developed risk management plan (which is a separate plan from the HRIT-level risk management plan used by the program management office, as discussed earlier). While the PALMS vendor-developed risk management plan directed each component to track risks in a component-specific risk register, the plan did not establish criteria for when component-level risks need to be elevated to the PALMS PMO at headquarters.

In September 2015, the PALMS program manager stated that all component-level risks that are rated red (i.e., high-probability and high- impact risks) are reported to headquarters. However, this guidance was not documented and, as such, the PALMS PMO did not have reasonable assurance that the components were knowledgeable about which risks to elevate, and whether the components were appropriately elevating such risks. Program officials were unable to explain why this criterion was not documented, but in response to our concern, the program officials directed the vendor to update the PALMS risk management plan to document this criterion; the vendor completed this update in October 2015. In particular, the plan now specifies that all component-level risks that could impact when the PALMS system is to be deployed at each of the components should be elevated to the PALMS PMO and given a priority of high.

Documenting the criteria for when risks need to be elevated to the PALMS PMO should help ensure that all appropriate risks are being elevated for review. However, until the program prioritizes its risks by determining each risk’s relative priority and overall risk level, DHS is hampered in its ability to ensure that the program’s attention and resources for risk mitigation are used in the most effective manner.

Conclusions

Although the HRIT investment was initiated about 12 years ago with the intent to consolidate, integrate, and modernize the department’s human resources IT infrastructure, DHS has made very limited progress in achieving these goals. HRIT’s minimally involved executive steering committee during a time when significant problems were occurring was a key factor in the lack of progress. This is particularly problematic given that the department’s ability to efficiently and effectively carry out its mission is significantly hampered by its fragmented human resources system environment and duplicative and paper-based processes.

Moreover, DHS’s ineffective management of HRIT, such as the lack of an updated schedule and a life-cycle cost estimate, also contributed to the neglect this investment has experienced. Until DHS, among other things, maintains a schedule, develops a life-cycle cost estimate, tracks costs, and re-evaluates and updates the Human Capital Segment Architecture blueprint, the department will continue to be plagued by duplicative systems and an inefficient and ineffective human resources environment impacting in its ability to perform its mission.

Additionally, until the PALMS program effectively addresses identified weaknesses in its project planning, project monitoring, and risk management practices and implements PALMS department-wide, DHS’s performance management processes will continue to be cumbersome, time-consuming, and primarily paper-based. Further, DHS will be limited in efficiently tracking and reporting accurate, comprehensive performance and learning management data across the organization, and could risk further implementation delays.

Recommendations for Executive Action

To ensure that the HRIT investment receives necessary oversight and attention, we are recommending that the Secretary of Homeland Security direct the Under Secretary of Management take the following two actions:

Update the HRIT executive steering committee charter to establish the frequency with which HRIT executive steering committee meetings are to be held.

Ensure that the HRIT executive steering committee is consistently involved in overseeing and advising HRIT, including approving key program management documents, such as HRIT’s operational plan, schedule, and planned cost estimate.

To address HRIT’s poor progress and ineffective management, we are recommending that the Secretary of Homeland Security direct the Under Secretary of Management to direct the Chief Human Capital Officer to direct the HRIT investment take the following six actions:

Update and maintain a schedule estimate for when DHS plans to implement each of the strategic improvement opportunities.

Develop a complete life-cycle cost estimate for the implementation of HRIT.

Document and track all costs, including components’ costs, associated with HRIT.

Establish time frames for re-evaluating the strategic improvement opportunities and associated projects in the Human Capital Segment Architecture Blueprint and determining how to move forward with HRIT.

Evaluate the strategic improvement opportunities and projects within the Human Capital Segment Architecture Blueprint to determine whether they and the goals of the blueprint are still valid and reflect DHS’s HRIT priorities going forward, and update the blueprint accordingly.

Update and maintain the department’s human resources system inventory.

To improve the PALMS program’s implementation of IT acquisition best practices, we are recommending that the Secretary of Homeland Security direct the Under Secretary of Management to direct the Chief Information Officer to direct the PALMS program office to take the following six actions:

Establish a time frame for deciding whether PALMS will be fully deployed at FEMA and USCG, and determine an alternative approach if the learning and/or performance management capabilities of PALMS are deemed not feasible for ICE, FEMA, TSA, or USCG.

Develop a comprehensive life-cycle cost estimate, including all government and contractor costs, for the PALMS program.

Develop and maintain a single comprehensive schedule that includes all government and contractor activities, and includes all planned deployment milestones related to performance management.

Track and monitor all costs associated with the PALMS program.

Document PALMS’s progress and milestone reviews, including all issues and corrective actions discussed.

Establish a comprehensive risk log that maintains an aggregation of all up-to-date risks (including both government- and vendor-identified) and associated mitigation plans. Additionally, within the comprehensive risk log, identify and document planned completion dates for each risk mitigation step (where appropriate), and prioritize the risks by determining each risk’s relative priority and overall risk level.

Agency Comments and Our Evaluation

We received written comments on a draft of this report from the Director of DHS’s Departmental GAO-OIG Liaison Office. The comments are reprinted in appendix II.

In its comments, the department concurred with our 14 recommendations and provided estimated completion dates for implementing each of them. For example, by April 30, 2016, the Under Secretary of Management plans to ensure that the HRIT executive steering committee is consistently involved in overseeing and advising HRIT and the committee is expected to be reviewed quarterly by the Acquisition Review Board. These planned actions, if implemented effectively, should help DHS address the intent of our recommendations.

We also received technical comments from DHS headquarters and component officials, which we have incorporated, as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Homeland Security and other interested parties. In addition, this report is available at no charge on the GAO website at http://www.gao.gov.

Should you or your staffs have any questions on information discussed in this report, please contact Carol Cha at (202) 512-4456, ChaC@gao.gov or Rebecca Gambler at (202) 512-6912, GamblerR@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) evaluate the progress the Department of Homeland Security (DHS) has made in implementing the Human Resources Information Technology (HRIT) investment and how effectively DHS has managed the investment since completing the Human Capital Segment Architecture in August 2011, (2) describe whether DHS has justified its investment in the Performance and Learning Management System (PALMS) program, (3) determine whether PALMS is being implemented enterprise-wide, and (4) evaluate the extent to which PALMS is implementing selected information technology (IT) acquisition best practices.

To address the first part of our first objective—to evaluate the progress DHS had made in implementing the HRIT investment—we compared HRIT’s goals, scope, and implementation time frames (as defined in the Human Capital Segment Architecture Blueprint, which was completed in August 2011) to the investment’s actual accomplishments. Specifically, we compared the completed and in-progress HRIT projects against the strategic improvement opportunities and projects that were outlined in the blueprint to determine which of the improvement opportunities and projects had been fully implemented or were in-progress. We also compared DHS’s planned schedule for implementing the improvement opportunities and projects against DHS’s current planned schedule for implementing them as of November 2015. Additionally, we interviewed DHS officials from the HRIT investment, Office of the Chief Information Officer (OCIO), Office of the Chief Human Capital Officer (OCHCO), and DHS’s components to discuss the steps taken to implement HRIT, address the strategic improvement opportunities and projects in the blueprint, and meet the goals of the investment.

In addressing the second part of our first objective—to evaluate how effectively DHS managed the investment—we analyzed documentation, such as the investment’s planned and updated completion dates, program management briefings, the blueprint, cost estimates, and budget documentation, and compared it against relevant cost and schedule best practices identified by GAO, CMMI-ACQ, and the PMBOK® Guide. These best practices included developing and maintaining a schedule estimate; developing a life-cycle cost estimate; and tracking program expenditures. To determine the amount spent to date on HRIT, we asked officials from DHS headquarters and each of the eight components to provide expenditure information on HRIT since the investment began in 2003; officials were unable to provide complete information. As such, we were unable to identify the total amount spent on the investment and discuss this limitation earlier in the report.

We also analyzed DHS’s human capital investment guidance, including the 2010 Deputy Secretary memorandum that prohibited component spending on enhancements to existing human resources systems or acquisitions of new human resources solutions, unless those expenditures have been approved by OCHCO or OCIO, and compared it to the components’ current investments in human resources systems, such as those listed in DHS’s fiscal year 2016 human capital portfolio. Additionally, we interviewed officials from the OCIO, OCHCO, and DHS’s eight components to obtain additional information on how HRIT reduced or will reduce duplicative human resources systems.

To describe whether DHS justified its investment in the PALMS program, we analyzed documentation, such as the program’s business case and the documented analysis of alternatives that was conducted to identify recommended approaches for pursuing a commercial off-the-shelf learning management system. We used this information to determine the various alternative solutions that DHS assessed for delivering enterprise- wide performance and learning management capabilities. Additionally, we reviewed program management briefings provided to the HRIT Executive Steering Committee that outlined, for example, the proposed solution and rationale for such a solution. We also interviewed appropriate DHS and PALMS officials for further information regarding the process DHS used to conduct the analysis of alternatives and other steps the department took to determine its preferred solution, including determining whether DHS could use existing shared services that were being used by other federal agencies.

To determine whether PALMS is being implemented department-wide, we analyzed the program’s acquisition plan and original schedule for implementing the system department-wide, and compared it against actual program status documentation and the program’s current implementation schedule. We also obtained and analyzed information from DHS officials, the PALMS headquarters program management office and DHS’s components on each component’s implementation of PALMS, including identifying which PALMS capabilities each component planned to implement, the number of planned PALMS users, and their reported causes for why certain components were not currently planning to implement PALMS.

To evaluate the extent to which PALMS implemented selected IT acquisition best practices, we analyzed the program’s IT acquisition documentation and compared it to relevant project planning, project monitoring, and risk management best practices—including CMMI-ACQ and PMBOK® Guide practices, and best practices identified by GAO. Specifically, we analyzed program documentation, including the acquisition plan, requirements management plan, risk management plan, cost and schedule estimates, program management review briefings, meeting minutes, risk logs, and risk mitigation plans to determine the extent to which the program’s acquisition processes were consistent with the best practices. Additionally, we interviewed officials from HRIT, PALMS, OCIO, OCHCO, and DHS’s eight components to obtain additional information on the program’s risk management, project planning, and project monitoring processes.

To assess the reliability of the data that we used to support the findings in this report, we reviewed relevant program documentation to substantiate evidence obtained through interviews with agency officials. We determined that the data used in this report were sufficiently reliable, with the exception of expenditure information provided by the HRIT investment and selected risk data provided by the PALMS program. We discuss limitations with these data in the report. We have also made appropriate attribution indicating the sources of the data.

We conducted this performance audit from March 2015 to February 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the contacts named above, the following staff also made key contributions to this report: Shannin O’Neill, Assistant Director; Christopher Businsky; Rebecca Eyler; Javier Irizarry; Emily Kuhn; and David Lysy.