Homeland Security: Oversight of Neglected Human Resources Information Technology Investment Is Needed
Published: 2016-02-11T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DHS's human resources administrative environment includes fragmented systems, duplicative and paper-based processes, and little uniformity of data management practices, which according to DHS, are compromising the department's ability to effectively carry out its mission. DHS initiated HRIT in 2003 to consolidate, integrate, and modernize DHS's human resources information technology infrastructure. In 2011, DHS redefined HRIT's scope and implementation time frames.
GAO was asked to review DHS's efforts to implement the HRIT investment. GAO's objectives included, among others, evaluating the progress DHS has made in implementing the HRIT investment. GAO compared HRIT's goals and scope to the investment's actual accomplishments, and compared DHS's planned schedule for implementing strategic improvement opportunities (key areas identified by DHS as needing improvement) against its current schedule.
section_title: What GAO Found
paragraphs: The Department of Homeland Security (DHS) has made very little progress in implementing its Human Resources Information Technology (HRIT) investment in the last several years. This investment includes 15 improvement opportunities; as of November 2015, DHS had fully implemented only 1, see table below.
Key: ●Fully implemented ◐Partially implemented ○Not yet started
Source: GAO analysis of data provided by DHS officials.
a Dates reflect the last month of the quarter in which the opportunities were planned to be complete.
HRIT's limited progress was due in part to the lack of involvement of its executive steering committee—the investment's core oversight and advisory body—which was minimally involved with HRIT, such as meeting only one time during a nearly 2-year period when major problems, including schedule delays, were occurring. As a result, key governance activities, such as approval of HRIT's operational plan, were not completed. Officials acknowledged that HRIT should be re-evaluated and took early steps to do so (i.e., meeting to discuss the need to re-evaluate); however, specific actions and time frames have not been determined. Until DHS takes key actions to re-evaluate and manage this neglected investment, it is unknown when its human capital weaknesses will be addressed.
section_title: What GAO Recommends
paragraphs: GAO is making 14 recommendations to DHS to, among other things, address HRIT's poor progress and ineffective management. For example, GAO recommends that the HRIT executive steering committee is consistently involved in overseeing and advising the investment. In addition, GAO recommends DHS evaluate the HRIT investment to determine whether its goals are still valid and reflect the department's priorities. DHS concurred with all 14 recommendations and provided estimated completion dates for implementing each of them.

section_title: Letter
paragraphs: Since the Department of Homeland Security (DHS) was created in 2002  and merged 22 agencies into one department with eight components, its  human resources environment has included fragmented systems,  duplicative and paper-based processes, and little uniformity of data  management practices. According to DHS, these issues are  compromising the department’s ability to effectively and efficiently carry  out its mission to, among other things, enhance security and respond to  disasters. For example, according to DHS, while it is imperative that it  respond quickly to emergencies, catastrophic events, and threats, and  deploy appropriately trained, certified, and skilled personnel during these  events, the department’s inefficient and disjointed hiring process has  limited the department’s hiring abilities.
To address these issues, DHS initiated the Human Resources  Information Technology (HRIT) investment in 2003 to consolidate,  integrate, and modernize the department’s IT infrastructure that supports  human resources. One of the types of human resources programs to be  addressed through the HRIT umbrella was management of department- wide employee performance and learning—referred to as the  Performance and Learning Management System (PALMS). This program  is designed to implement an enterprise-wide employee performance  management and appraisal solution that is to automate the department’s  primarily paper-based performance management processes. In addition,  PALMS is to provide a system that will consolidate nine existing learning  management systems into one system and enable comprehensive  training reporting and analysis across the department.
In light of HRIT’s expected role in transforming the department’s human  resources processes and system environment, you asked us to review  DHS’s efforts to implement the investment. Our objectives were to (1)  evaluate the progress DHS has made in implementing the HRIT  investment and how effectively DHS managed the investment since  completing the Human Capital Segment Architecture in August 2011, (2)  describe whether DHS has justified its investment in the PALMS program,  (3) determine whether PALMS is being implemented enterprise-wide, and  (4) evaluate the extent to which PALMS is implementing selected IT  acquisition best practices.
To address the first part of our first objective—to evaluate the progress  DHS had made in implementing the HRIT investment—we compared  HRIT’s goals, scope, and implementation time frames to the investment’s  actual accomplishments. We also compared DHS’s planned schedule for  implementing the improvement opportunities and projects, as of August  2011, against DHS’s current planned schedule for implementing them.  For the second part of our first objective—to evaluate how effectively  DHS has managed the investment—we analyzed documentation, such as  the investment’s schedule, program management briefings, DHS’s  Human Capital Segment Architecture blueprint, cost estimates, and  budget documentation, and compared them against relevant cost and  schedule best practices identified by GAO, the Software Engineering  Institute at Carnegie Mellon University, and the Project Management  Institute, Inc. To determine the amount spent to date on HRIT, we asked  officials from DHS headquarters and the components to provide  expenditure information on HRIT since the investment began in 2003;  officials were unable to provide complete information. As such, we were  unable to identify the total amount spent on the investment and discuss  this limitation further in the report.
In addressing our second objective, we analyzed documentation, such as  the program’s business case and the documented analysis of alternatives  that was conducted to identify recommended approaches for pursuing a  commercial off-the-shelf learning management system. We used this  information to determine the various alternative solutions that DHS  assessed for delivering enterprise-wide performance and learning  management capabilities and justifying its investment in PALMS.  Additionally, we reviewed program management briefings provided to the  HRIT Executive Steering Committee that outlined, for example, the  proposed solution and rationale for such a solution.
To address our third objective, we analyzed the program’s acquisition  plan and original schedule for implementing the system department-wide,  and compared it against program status documentation and the  program’s current implementation schedule, as of November 2015.
Lastly, for our fourth objective, we analyzed the program’s IT acquisition  documentation (e.g., acquisition plan and risk logs) and compared it to  relevant project planning, project monitoring, and risk management best  practices as identified by CMMI-ACQ, the PMBOK® Guide, and GAO.  Additionally, we interviewed officials from HRIT, PALMS, the Office of the  Chief Information Officer (OCIO), the Office of the Chief Human Capital  Officer (OCHCO), and DHS’s eight components to obtain additional  information on the program’s IT acquisition processes in these areas.
We conducted this performance audit from March 2015 to February 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. See appendix I for a more  detailed discussion of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: DHS’s mission is to lead the unified national effort to secure America by  preventing and deterring terrorist attacks and protecting against and  responding to threats and hazards to the nation, among other things.  Created in 2002, DHS merged 22 agencies and offices that specialized in  one or more aspects of homeland security. The intent behind the merger  that created DHS was to improve coordination, communication, and  information sharing among these multiple federal agencies. Each of these  agencies is responsible for specific homeland security missions and for  coordinating related efforts with its sibling components, as well as  external entities. Figure 1 shows a simplified and partial DHS  organizational structure.
Within the department’s Management Directorate, headed by the Under  Secretary for Management (USM), are the OCHCO and OCIO. The  OCHCO is responsible for department-wide human capital policy and  development, planning, and implementation of human capital initiatives.  The OCIO is responsible for departmental IT policies, processes, and  standards, and ensuring that IT acquisitions comply with DHS IT  management processes, among other things.
subsections: section_title: Oversight of DHS’s Acquisition Programs
paragraphs: DHS acquires IT and other capabilities that are intended to improve its  ability to execute its mission. DHS classifies these acquisition programs  into three levels that determine the extent and scope of required project  and program management, the level of reporting requirements, and the  acquisition decision authority. Specifically, DHS policy defines acquisition  programs as follows:
Level 1 major acquisition programs are expected to cost $1 billion or  more over their life cycles.
Level 2 major acquisition programs are expected to cost at least $300  million over their life cycles.
Special interest programs, without regard to the established dollar  thresholds, are designated as Level 1 or Level 2 programs. For  example, a program may be raised to a higher acquisition level if  its importance to DHS’s strategic and performance plans is  disproportionate to its size or it has high executive visibility.
Level 3 programs are those with life-cycle cost estimates less than  $300 million and are considered non-major.
As outlined in DHS’s Acquisition Management Directive 102-01, DHS’s  Chief Acquisition Officer—the USM—is responsible for the management  and oversight of the department’s acquisition policies and procedures.  The Deputy Secretary, USM, and Component Acquisition Executives are  the acquisition decision authorities for DHS’s acquisition programs. For  Level 1 programs, the acquisition decision authority may be either the  Deputy Secretary or USM; for level 2 programs, the acquisition decision  authority may be either the USM or a Component Acquisition Executive;  and for Level 3 programs, a Component Acquisition Executive is the  acquisition decision authority.
As of March 2015, the department had 72 major acquisition programs and  42 non-major acquisition programs.
section_title: DHS’s Management of Human Capital Is a High- Risk Effort
paragraphs: In 2003, we designated the transformation of DHS as high risk because it  had to transform 22 agencies—several with major management  challenges—into one department. We emphasized that failure to  effectively address DHS’s management and mission risks could have  serious consequences for U.S. national and economic security.
In 2007 and 2009, in reporting on DHS’s progress in addressing the high- risk area since its creation, we found that DHS had made more progress  in implementing its range of missions than its management functions— such as in the areas of IT and human capital—and that continued work  was needed to address an array of programmatic and management  challenges. Since then, DHS had continued to make important progress  in strengthening and integrating its management functions; however,  significant work remained for DHS to improve in these areas. For  example,
As of September 2015, DHS had taken steps to identify current and  future human capital needs, including the size of the workforce, its  deployment across the department and components, and the  knowledge, skills, abilities, and diversity needed; however, DHS had  yet to fully implement its workforce planning model that was intended  to allow the department to plan for its current and future organizational  and workforce needs.
In February 2015, we reported that while DHS established a human  capital strategic plan in 2011 and made progress in implementing it,  the department had considerable work ahead to improve employee  morale, which has decreased each year since 2011. For example,  the Office of Personnel Management’s 2014 Federal Employee  Viewpoint Survey data showed that DHS’s scores continued to  decrease in all four dimensions of the survey’s index for human  capital accountability and assessment.
While the department had made progress in implementing its IT  Strategic Human Capital Plan for fiscal years 2010 through 2012, in  January 2015 DHS shifted its IT paradigm from acquiring assets to  acquiring services, and acting as a service broker (e.g., an  intermediary between the purchaser of a service and the seller of that  service). According to DHS officials in May 2015, this paradigm  change will require a major transition in the skill sets of DHS’s IT  workforce, as well as the hiring, training, and managing of those new  skill sets; as such, this effort will need to be closely managed in order  to succeed.
Moreover, as of September 2014, DHS faced challenges in integrating  employee training management across all the components, including  centralizing training and consolidating training data into one system.  According to DHS officials, the department planned to address these  limitations through the development and deployment of HRIT’s  PALMS program.
section_title: Overview of DHS’s Human Resources Information Technology Investment
paragraphs: Since DHS was created, the department’s human resources environment  has included fragmented systems, duplicative and paper-based  processes, and little uniformity of data management practices. According  to DHS, these limitations in its human resources environment are  compromising the department’s ability to effectively and efficiently carry  out its mission. For example,
While it is imperative that DHS responds quickly to emergencies,  catastrophic events, and threats, and deploys appropriately trained,  certified, and skilled personnel during these events, according to  DHS, the department’s hiring process involves numerous systems  and multiple hand-offs which result in extra work and prolonged  hiring. This inefficient process is one factor that could have  contributed to the skill and workforce gaps that we have previously  identified. For example, in April 2015, we reported that 21 of the 22  major acquisition programs we reviewed faced shortfalls in their  program office workforce in fiscal year 2014.
According to DHS, the department does not have information on all of  its employees, which reduces its abilities to strategically manage its  workforce and best deploy people in support of Homeland Security  missions.
According to DHS, reporting and analyzing enterprise human capital  data are currently time-consuming, labor-intensive, and challenging  because the department’s data management largely consists of  disconnected, standalone systems, with multiple data sources for the  same content. As one example, we reported in 2014 that DHS could  not provide complete information on how much it had spent on  administratively uncontrollable overtime to its personnel from fiscal  years 2008 through 2014. Specifically, certain components could not  provide information such as duty location or payments for certain  years.
To address these issues, in 2003, DHS initiated the HRIT investment,  which is intended to consolidate, integrate, and modernize the  department’s and its components’ human resources IT infrastructure.  These components include U.S. Customs and Border Protection (CBP),  the Federal Emergency Management Agency (FEMA), the Federal Law  Enforcement Training Center (FLETC), U.S. Immigration and Customs  Enforcement (ICE), the Transportation Security Administration (TSA),  U.S. Citizenship and Immigration Services (USCIS), the U.S. Coast  Guard (USCG), and the U.S. Secret Service.
HRIT is managed by DHS’s Human Capital Business Systems unit, which  is within OCHCO and has overall responsibility for HRIT. Additionally,  OCIO plays a key supporting role in the implementation of HRIT by  reviewing headquarters’ and components’ human resources investments,  identifying redundancies and efficiencies, and delivering and maintaining  enterprise IT systems.
From 2003 to 2010, DHS made limited progress on the HRIT investment,  as reported by DHS’s Inspector General. This was due to, among other  things, limited coordination with and commitment from DHS’s  components. To address this problem, in 2010 the DHS Deputy Secretary  issued a memorandum emphasizing that DHS’s wide variety of human  resources processes and IT systems inhibited the ability to unify DHS and  negatively impacted operating costs. The memorandum stated that,  without an enterprise operating model, support for DHS’s core mission  was at risk and valuable workforce management information remained  difficult to acquire across the department. Accordingly, the Deputy  Secretary stated that DHS could no longer sustain a component-centric  approach when acquiring or enhancing human resources systems, and  prohibited component spending on enhancements to existing human  resources systems or acquisitions of new solutions, unless those  expenditures were approved by OCHCO or OCIO. The memorandum  also directed these offices to develop a department-wide human  resources architecture.
In 2011, in response to the Deputy Secretary’s direction, DHS completed  an effort called the Human Capital Segment Architecture, which,  according to DHS, defined the department’s current (or as-is) state of  human capital management processes, technology, data, and relevant  personnel. Further, from this current state, the department developed a  comprehensive future state (or target state) and a document referred to  as the Human Capital Segment Architecture blueprint that redefined the  HRIT investment’s scope and implementation time frames. As part of this  effort, DHS conducted a system inventory and determined that it had 422  human resources systems and applications, many of which were single- use solutions developed to respond to a small need or links to enable  disparate systems to work together. DHS reported that these numerous,  antiquated, and fragmented systems inhibited its ability to perform basic  workforce management functions necessary to support mission critical  programs.
To address this issue, the blueprint articulated that HRIT would be  comprised of 15 strategic improvement opportunity areas (e.g., enabling  seamless, efficient, and transparent end-to-end hiring) and outlined 77  associated projects (e.g., deploying a department-wide hiring system,  establishing an integrated data repository and reporting mechanism, and  developing a centralized learning center for all personnel action  processing information) to implement these 15 opportunities. Each  opportunity area includes from 1 to 10 associated projects. Table 1  summarizes the scope of the 15 strategic improvement opportunities— listed in the order of DHS’s assigned priority—and identifies their original  planned completion dates, as of August 2011 when the blueprint was  issued.
HRIT’s only ongoing program is called PALMS and is intended to fully  address the Performance Management strategic improvement opportunity  area and its three associated projects. PALMS is attempting to implement  a commercial off-the-shelf software product that is to be provided as a  service in order to enable, among other things, comprehensive  enterprise-wide tracking, reporting, and analysis of employee learning  and performance for DHS headquarters and its eight components.  Specifically, PALMS is expected to deliver the following capabilities:  Learning management. The learning management capabilities are intended to manage the life cycle of learning activities for all DHS employees and contractors. PALMS is intended to, among other things, act as a gateway for accessing training at DHS and record  training information when a user has completed a course. Additionally,  it is expected to replace nine disparate learning management systems  with one unified system.
Performance management. The performance management capabilities are intended to move DHS’s existing primarily paper- based performance management processes into an electronic environment and capture performance-related information throughout the performance cycle (e.g., recording performance expectations discussed at the beginning of the rating period and performance ratings at the end of it).
Each component is responsible for its own PALMS implementation  project, and is expected to issue a task order using a blanket purchase  agreement that was established in May 2013 with an estimated value of  $95 million. Before implementing PALMS, each component is  completing a fit-gap assessment to, among other things, identify any  requirements and critical processes that cannot be met by the  preconfigured, commercial off-the-shelf system. If such component- specific requirements are identified, the component must then decide  whether to have the vendor customize the system.
The headquarters PALMS program management office (PMO) is  responsible for overseeing the implementation projects across the  department. Additionally, OCIO is the Component Acquisition Executive  responsible for overseeing PALMS.
In addition to implementing projects intended to address the strategic  improvement opportunities in the blueprint, the HRIT investment also  carried out the following two projects that were not included in the  blueprint:
Balanced Workforce Assessment Tool: This project provided an enterprise-wide tool to automate the formerly paper-based balanced workforce strategy process to determine the appropriate mix of federal employees and contractor employees required to fulfill a specific work function in the government. DHS deployed this tool beginning in September 2013.
Workers Compensation – Medical Case Management Services: This project provided an enterprise-wide contract to enable nurses to execute case management processes and facilitate the case management activities to be performed by DHS human resources staff. As part of this, the project provided access to a web application where DHS workers’ compensation coordinators could work on cases with nurses. As of March 2015, the tool had been implemented at six components.
section_title: Best Practices for Planning and Managing IT Acquisition Programs
paragraphs: Entities such as the Project Management Institute, the Software  Engineering Institute at Carnegie Mellon University, and GAO have  developed and identified best practices to help guide organizations to  effectively plan and manage their acquisitions of major IT systems. Our  prior reviews have shown that proper implementation of such practices  can significantly increase the likelihood of delivering promised system  capabilities on time and within budget. These practices include, but are  not limited to:
Project planning: Establishes project objectives and outlines the course of action required to attain those objectives. It also provides a means to track, review, and report progress and performance of the project by defining project activities and developing cost and schedule estimates, among other things.
Project monitoring and control: Provides an understanding of the project’s progress, so that appropriate corrective actions can be taken if performance deviates from plans. Effective practices in this area include, among other things, determining progress against the program plan and conducting program management reviews.
Risk management: Establishes a process for anticipating problems and taking appropriate steps to mitigate risks and minimize their impact on program commitments. It involves identifying and documenting risks, categorizing them based on their estimated impact, prioritizing them, developing risk mitigation strategies, and tracking progress in executing the strategies.
section_title: DHS Has Made Very Little Progress in Implementing HRIT; Investment Lacked Effective Management
paragraphs: DHS has made very little progress in delivering planned HRIT  capabilities, such as end-to-end hiring and payroll action processing.  While the vast majority of HRIT capabilities (called strategic improvement  opportunities) were to be delivered by June 2015, only 1 has been fully  implemented, and the completion dates for the other 14 are currently  unknown. These delays are largely due to unplanned resource changes  and the lack of involvement from the executive oversight committee. In  addition, the department did not effectively manage the investment. For  example, DHS did not update or maintain the HRIT schedule, have a life- cycle cost estimate, or track all associated costs. Moreover, the strategic  planning document—referred to as the Human Capital Segment  Architecture Blueprint—has not been updated in approximately 4.5 years  and, as a result, the department does not know whether it is reflective of  current priorities and goals. As a result of DHS’s ineffective management  and limited progress in implementing this investment, the department is  unaware of when critical weaknesses in the department’s human capital  environment will be addressed, which is, among other things, impacting  DHS’s ability to reduce duplication and carry out its mission.
subsections: section_title: Limited Progress Has Been Made in Implementing HRIT
paragraphs: DHS has made very limited progress in addressing the 15 strategic  improvement opportunities and the 77 associated projects included in  HRIT. According to the Human Capital Segment Architecture Blueprint,  DHS planned to implement 14 of the 15 strategic improvement  opportunities and 68 of the 77 associated projects by June 2015; and the  remaining improvement opportunity and 9 associated projects by  December 2016. However, as of November 2015, DHS had fully  implemented only 1 of the strategic improvement opportunities, which  included 2 associated projects. This improvement opportunity established  an enterprise-wide governance process for evaluating HRIT projects and  proposals prior to funding them. This process is referred to as the  investment intake process and is intended to help encourage the use of  enterprise-level investments, rather than component-specific investments,  by preventing components from investing in duplicative systems when an  existing DHS capability can meet a particular business need. Table 2  summarizes the implementation status and planned completion dates of  the strategic improvement opportunities—listed in the order of DHS’s  assigned priority—as of November 2015.
DHS has partially implemented five of the other strategic improvement  opportunities, but it is unknown when they will be fully addressed. For  example, DHS’s PALMS program is intended to fully address the  blueprint’s strategic improvement opportunity for Performance  Management; however, while progress to implement PALMS has been  made, many actions remain before it can be fully implemented and it is  unknown when those actions will be taken (discussed in more detail  later). Further, HRIT officials stated that DHS has not yet started to work  on the remaining nine improvement opportunities, and the officials did not  know when they would be addressed.
Additionally, DHS developed an HRIT strategic plan for fiscal years 2012  through 2016 that outlined the investment’s key goals and objectives,  including reducing duplication and improving efficiencies in the  department’s human resources processes and systems. The strategic  plan identified, among other things, two performance metrics and  associated targets for delivering human resources IT services across  DHS. These performance metrics were focused on reductions in the  number of component-specific human resources IT services provided and  increases in the number of department-wide HRIT services provided by  the end of fiscal year 2016.
However, DHS has also made limited progress in achieving these two  performance targets. Figure 2 provides a summary of HRIT’s progress  towards achieving its service delivery performance targets.
DHS’s goal is to reduce its component-specific HRIT services by 46 percent—from 81 percent to 35 percent—however, it had reduced these services by 8 percent as of November 2015, according to OCHCO officials.
Additionally, while DHS is aiming to increase its DHS-wide HRIT services by 38 percent—from 2 percent to 40 percent—as of November 2015, OCHCO officials stated that the department had increased these services by 8 percent.
Key causes for DHS’s lack of progress in implementing HRIT and its  associated strategic improvement opportunities include unplanned  resource changes and the lack of involvement of the HRIT executive  steering committee. These causes are discussed in detail below:
Unplanned resource changes. DHS elected to dedicate the vast majority of HRIT’s resources to implementing PALMS and addressing its problems, rather than initiating additional HRIT strategic improvement opportunities. Specifically, PALMS—which began in July 2012—experienced programmatic and technical challenges that led to years-long schedule delays. For example, while the PALMS system for headquarters was originally planned to be delivered by a vendor in December 2013, as of November 2015, the expected delivery date was delayed until the end of February 2016—an over 2-year delay. HRIT officials explained the decision to focus primarily on PALMS was due, in part, to the investment’s declining funding stream. However, in doing so, attention was concentrated on the immediate issues affecting PALMS and diverted from the longer-term HRIT mission.
Lack of involvement of the HRIT executive steering committee. The HRIT executive steering committee—which is chaired by the department’s Under Secretary for Management and co-chaired by the Chief Information Officer and Chief Human Capital Officer—is intended to be the core oversight and advisory body for all DHS-wide matters related to human capital IT investments, expenditures, projects, and initiatives. In addition, according to the committee’s charter, the committee is to approve and provide guidance on the department’s mission, vision, and strategies for the HRIT program.
However, the executive steering committee only met once from  September 2013 through June 2015—in July 2014—and was  minimally involved with HRIT for that almost 2 year period. It is  important to note that DHS replaced its Chief Information Officer (the  executive steering committee’s co-chair) in December 2013—during  this gap in oversight. Also during this time period HRIT’s only ongoing  program—PALMS—was experiencing significant problems, including  schedule slippages and frequent turnover in its program manager  position (i.e., PALMS had five different program managers during the  time that the HRIT executive steering committee was minimally  involved). As a result of the executive steering committee not meeting,  key governance activities were not completed on HRIT. For example,  the committee did not approve HRIT’s notional operational plan for  fiscal years 2014 through 2019. OCHCO and OCIO officials  attributed the lack of HRIT executive steering committee meetings  and committee involvement in HRIT to the investment’s focus being  only on the PALMS program to address its issues, as discussed  earlier. However, by not regularly meeting and providing oversight  during a time when a new co-chair for the executive steering  committee assumed responsibility and PALMS was experiencing such  problems, the committee’s guidance to the troubled program was  limited.
More recently, the HRIT executive steering committee met in June  and October 2015, and OCIO and OCHCO officials stated that the  committee planned to meet quarterly going forward. However, while  the committee’s charter specified that it meet on at least a monthly  basis for the first year, the charter does not specify the frequency of  meetings following that year. Furthermore, the committee’s charter  has not been updated to reflect the increased frequency of these  meetings.
As a result of the limited progress in implementing HRIT, DHS is unaware  of when critical weaknesses in the department’s human capital  environment will be addressed, which is, among other things, impacting  DHS’s ability to carry out its mission. For example, the end-to-end hiring  strategic improvement opportunity (which has an unknown  implementation date) was intended to streamline numerous systems and  multiple hand-offs in order to more efficiently and effectively hire  appropriately skilled personnel, thus enabling a quicker response to  emergencies, catastrophic events, and threats. As another example, the  data management and sharing strategic improvement opportunity (which  also has an unknown implementation date) was intended to enable the  department to have visibility of all its employees, to improve its ability to  strategically manage its workforce, and best deploy people in support of  DHS missions. Therefore, until HRIT’s executive steering committee  effectively carries out its oversight responsibility, DHS will be limited in its  ability to improve HRIT investment results and accountability.
section_title: HRIT Lacked a Current Schedule, Life-Cycle Cost Estimate, and Cost Tracking
subsections: section_title: DHS Did Not Update or Maintain the HRIT Schedule
paragraphs: According to the GAO Schedule Assessment Guide, a key activity in  effectively managing a program and ensuring progress is establishing and  maintaining a schedule estimate. Specifically, a well maintained schedule  enables programs to gauge progress, identify and resolve potential  problems, and forecast dates for program activities and completion of the  program.
In August 2011, DHS established initiation and completion dates for each  of the 15 strategic improvement opportunities within the Human Capital  Segment Architecture Blueprint. Additionally, HRIT developed a slightly  more detailed schedule for fiscal years 2014 through 2021 that updated  planned completion dates for aspects of some strategic improvement  opportunities, but not all.
However, DHS did not update and maintain either schedule after they  were developed. Specifically, neither schedule was updated to reflect that  DHS did not implement 13 of the 15 improvement opportunities by their  planned completion dates—several of which should have been  implemented over 3 years ago. HRIT officials attributed the lack of  schedule updates to the investment’s focus shifting to the PALMS  program when it started experiencing significant schedule delays. Without  developing and maintaining a current schedule showing when DHS plans  to implement the strategic improvement opportunities, DHS and Congress  will be limited in their ability to oversee and ensure DHS’s progress in  implementing HRIT.
section_title: HRIT Did Not Have a Life- Cycle Cost Estimate
paragraphs: OMB requires that agencies prepare total estimated life-cycle costs for  information technology investments. Program management best  practices also stress that key activities in planning and managing a  program include establishing a life-cycle cost estimate and tracking costs  expended. A life-cycle cost estimate supports budgetary decisions and  key decision points, and should include all costs for planning,  procurement, and operations and maintenance of a program.
OCHCO officials stated that a draft life-cycle cost estimate for HRIT was  developed, but that it was not completed or finalized because detailed  projects plans for the associated projects had not been developed or  approved. According to the HRIT blueprint, OCHCO roughly estimated  that implementing all of the projects could cost up to $120 million.  However, the blueprint specifies that this figure did not represent the life- cycle cost estimate; rather it was intended to be a preliminary estimate to  initiate projects. Without a life-cycle cost estimate, DHS has limited  information about how much it will cost to implement HRIT, which hinders  the department’s ability to, among other things, make budgetary decisions  and informed milestone review decisions.
section_title: DHS Did Not Track All Costs Incurred on HRIT
paragraphs: According to CMMI-ACQ and the PMBOK® Guide, programs should track  program costs in order to effectively manage the program and make  resource adjustments accordingly. In particular, tracking and monitoring  costs enables a program to recognize variances from the plan in order to  take corrective action and minimize risk.
However, DHS has not tracked the total actual costs incurred on  implementing HRIT across the enterprise to date. Specifically, while the  investment received line item appropriations for fiscal years 2005 through  2015 which totaled at least $180 million, DHS was unable to provide all  cost information on HRIT activities since it began in 2003, including all  government-related activities and component costs that were financed  through the working capital fund, which, according to DHS officials from  multiple offices, were provided separately from the at least $180 million  appropriated specifically to HRIT. OCHCO officials attributed the lack of  cost tracking to, among other things, the investment’s early reliance on  contractors to track costs, and said that the costs were not well  maintained nor centrally tracked, and included incomplete component- provided cost information. The components were also unable to provide  us with complete information. For example,  FEMA officials stated that it would require a significant administrative effort to identify how much it has spent on HRIT since inception in 2003 because of the way their financial system obligates and expends funds for Working Capital Fund activities.
USCG officials also said that compiling its expenditure information for fiscal years 2003-2009 would require a substantial administrative effort, including reviewing a significant number of paper files.
USCIS was unable to identify its HRIT-related expenditures for fiscal years 2003-2010.
Without tracking all costs associated with HRIT, including components’  costs, stakeholders are limited in making informed resource decisions,  and DHS cannot provide complete and accurate information to assist  congressional oversight.
section_title: HRIT’s 2011 Blueprint May Not Be Valid and Reflective of DHS’s Current Priorities and Goals
paragraphs: According to the HRIT executive steering committee’s charter, the Under  Secretary for Management (as the chair of the committee) is to ensure  that the department’s human resources IT business needs are met, as  outlined in the blueprint. Additionally, according to the GPRA  (Government Performance and Results Act) Modernization Act of 2010,  agency strategic plans should be updated at least every 4 years. While  this is a legal requirement for agency strategic plans (the Human Capital  Segment Architecture blueprint does not fall under the category of an  “agency strategic plan”), it is considered a best practice for other strategic  planning documents, such as the blueprint.
However, the department issued the blueprint in August 2011  (approximately 4.5 years ago) and has not updated it since. As a result,  the department does not know whether the remaining 14 strategic  improvement opportunities and associated projects that it has not fully  implemented are still valid and reflective of DHS’s current priorities, and  are appropriately prioritized based on current mission and business  needs. Additionally, DHS does not know whether new or emerging  opportunities or business needs need to be addressed.
Officials stated that the department is still committed to implementing the  blueprint, but agreed that it should be re-evaluated. To this end, following  a meeting we had with DHS’s Under Secretary for Management in  October 2015, in which we expressed concern about HRIT’s lack of  progress, OCHCO and OCIO officials stated that HRIT was recently  asked by the Deputy Under Secretary of Management in late October  2015 to re-evaluate the blueprint’s strategic improvement opportunities  and to determine the way forward for those improvement opportunities  and the HRIT investment. However, officials did not know when this re- evaluation and a determination for how to move forward with HRIT would  occur, or be completed.
Further, according to OCIO officials, DHS has not updated its complete  systems inventory since it was originally developed as part of the  blueprint effort, in response to a 2010 Office of Inspector General report  that stated that DHS had not identified all human resource systems at the  components. This report also emphasized that without an accurate  inventory of human resource systems, DHS cannot determine whether  components are using redundant systems. Moreover, OCIO officials  were unable to identify whether and how its inventory of human resources  systems had changed.
Until DHS establishes time frames for re-evaluating the blueprint to reflect  DHS’s HRIT current priorities and updates its human resources system  inventory, the department will be limited in addressing the inefficient  human resources environment that has plagued the department since it  was first created.
section_title: DHS Justified Its Investment in the PALMS Program
paragraphs: DHS took several steps to justify its investment in the PALMS program for  both of the program’s two main purposes (the learning management  capabilities and the performance management capabilities) through  multiple mechanisms. Specifically, although existing DHS guidance did  not require an analysis of alternatives for PALMS because it is a Level 3  acquisition program, the department initiated such an analysis in 2010  to identify recommended approaches for pursuing a commercial off-the- shelf learning management system to replace the components’ nine  existing learning systems. According to the analysis of alternatives, the  nine systems at the department were disconnected from each other and  did not exchange information. The components had independently  purchased these learning management systems and, in some cases, had  done so before DHS was established in 2002. However, DHS determined  that a unified strategy for learning management systems at the  department was needed, rather than disparate, component-centric efforts.  In particular, DHS determined that such a strategy was necessary to  provide, among other things, improved reporting, greater automation, less  duplication and redundancy of training courses, better governance, and  streamlined IT infrastructure.
The analysis of alternatives, which was performed by the Homeland  Security Studies and Analysis Institute, included, among other things,  an assessment of six alternative approaches, including status quo,  implementation of two systems from separate vendors (allowing  components to choose which system to use), and implementation of a  single system (either centrally managed by DHS or individually managed  by each component). As part of the analysis, the Institute assessed the  alternative approaches based on five evaluative categories, including  cost, benefits, and risks. Based on the analysis of alternatives process,  the institute recommended that DHS adopt a single enterprise-wide,  centrally managed learning management system as the most cost- effective approach to providing such a capability to the department.
Regarding the second purpose of PALMS—enabling performance  management capabilities—the August 2011 Human Capital Segment  Architecture Blueprint called on DHS to conduct an analysis of  alternatives to identify the preferred approach for such a solution.  Officials stated that DHS leadership ultimately determined that such an  analysis for a performance management solution was unnecessary  because the requirement for DHS to automate performance management  functions across the department was the same as it was during DHS’s  prior attempt to pursue an automated performance management system  for instituting pay-for-performance—an effort that was ultimately  abandoned.
Therefore, instead of conducting an analysis of alternatives on  performance management system approaches for DHS enterprise-wide  adoption, in January 2012, departmental leadership made an executive  decision on the approach based on the findings of a December 2011  request for information from industry. In particular, the accumulated  industry information highlighted that vendors for an enterprise-wide  learning management solution could in most cases also provide a system  that integrated performance management capabilities. This industry  information validated DHS officials’ understanding that a combined  solution for learning and performance management at the department  was consistent with prevailing industry offerings. According to OCHCO  officials, the department’s request for information from industry to help  justify its preferred approach allowed for competition within industry for  supplying a solution to the department. As part of the department’s  considerations, officials had determined that this competition could better  help to reduce overall implementation costs for a consolidated learning  and performance management system, versus adopting, without  competition, one of the components’ existing learning or performance  management systems for DHS enterprise-wide deployment.
Additionally, OCHCO officials stated that they contacted other federal  departments to determine whether existing shared services could be used  by DHS to establish an integrated system for learning and performance  management, but DHS determined that other departments’ contracts with  service providers could not be modified to allow DHS to use the same  services.
Based on the collective results of the learning management system  analysis of alternatives and the request for information from industry on  performance management systems, the HRIT executive steering  committee exercised its executive decision-making authority and decided  that an integrated, enterprise-wide learning and performance  management system should be pursued for adoption at the department.  DHS’s integrated solution is now being implemented by the PALMS  program. By providing the executive steering committee with enough  information for determining this preferred approach for the department,  DHS justified its investment in the PALMS program.
section_title: Selected PALMS Capabilities Have Been Deployed to Headquarters and Two Components; but Full Implementation at Four Components Is Not Currently Planned
paragraphs: As previously mentioned, PALMS is intended to provide an enterprise- wide system that offers performance management capabilities, as well as  learning management capabilities to headquarters and each of its  components. As such, DHS headquarters PMO and the components  estimate that, if fully implemented across DHS, PALMS’s learning  management capabilities would be used by approximately 309,360 users,  and its performance management capabilities would be used by at least  217,758 users. Table 3 identifies the total estimated number of planned  users for both PALMS’s learning management capabilities and  performance management capabilities if PALMS is fully implemented  department-wide.
However, there is uncertainty about whether the PALMS system will be  used enterprise-wide to accomplish these goals. Specifically, as of  November 2015, of the eight components and headquarters, five are  planning to implement both PALMS’s learning and performance  management capabilities (three of which have already implemented the  learning management capabilities—discussed later), two are planning to  implement only the learning management capabilities, and two  components are not currently planning to implement either of these  PALMS capabilities, as illustrated in figure 3.
Officials from FEMA, TSA, ICE, and the USCG cited various reasons for  why they were not currently planning to fully implement PALMS, which  include:  FEMA and ICE officials stated that they were not currently planning to implement the performance management capabilities because the program had experienced critical deficiencies in meeting the performance management-related requirements. FEMA officials stated that they do not plan to make a decision on whether they will or will not implement these performance management capabilities until the vendor can demonstrate that the system meets FEMA’s needs; as such, FEMA officials were unable to specify a date for when they plan to make that decision. ICE officials also stated that they do not plan to implement the performance management capabilities of PALMS until the vendor can demonstrate that all requirements have been met. PALMS headquarters PMO officials expected all requirements to be met by the vendor by the end of February 2016.
TSA officials stated that they were waiting on the results of their fit- gap assessment of PALMS before determining whether, from a cost and technical perspective, TSA could commit to implementing the learning and/or performance management capabilities of PALMS. TSA officials expected the fit-gap assessment to be completed by the end of March 2016.
USCG officials stated that, based on the PALMS schedule delays experienced to date, they have little confidence that the PALMS vendor could meet the component’s unique business requirements prior to the 2018 expiration of the vendor’s blanket purchase agreement. Additionally, these officials stated that the system would not meet all of its learning management requirements for about 31,000 auxiliary volunteer members and certain other employee groups. Further, although the fit gap assessment for implementing PALMS at USCG had not been fully completed, the component’s officials stated that the system would likely not fully meet the performance management requirements for all of USCG’s military components. Due to the component’s uncertainty, the officials were  unable to specify when they plan to ultimately decide on whether they  will implement one or both aspects of PALMS.
As a result, it is unlikely that the department will reach its expected user  estimates as presented in table 3, and meet its goal of being an  enterprise-wide system. Specifically, as of November 2015, the  components estimate 179,360 users will use the learning management  capabilities of PALMS (not the 309,360 expected, if fully implemented).  Figure 4 shows the percentage of expected users from components  currently planning to implement PALMS’s learning management  capabilities in comparison to the total expected users if PALMS was fully  implemented, as of November 2015.
Additionally, as of November 2015, the components estimate 123,200  users will use the performance management capabilities of PALMS (not  the 217,758 expected, if fully implemented). Figure 5 shows the  percentage of expected users from components planning to implement  PALMS’s performance management capabilities in comparison to the  total expected user estimate if fully implemented as intended.
Of the seven components and headquarters that are currently planning to  implement the learning and/or performance management aspects of  PALMS, three have completed their implementation efforts of the learning  management capabilities and deployed these capabilities to users  (deployed to CBP in July 2015, headquarters in October 2015, and  FLETC in December 2015); two have initiated their implementation efforts  on one or both aspects, but not completed them; and two have not yet  initiated any implementation efforts, as of November 2015.
As a result, PALMS’s current trajectory is putting the department at risk of  not meeting its goals to perform efficient, accurate, and comprehensive  tracking and reporting of training and performance management data  across the enterprise; and consolidating its nine learning management  systems down to one. Accordingly, until FEMA decides whether it will  implement the performance management capabilities of PALMS and  USCG decides whether it will implement the learning and/or performance  management capabilities of PALMS, the department is at risk of  implementing a solution that does not fully address its problems.  Moreover, until DHS determines an alternative approach if one or both  aspects of PALMS is deemed not feasible for ICE, TSA, FEMA or the  USCG, the department is at risk of not meeting its goal to enable  enterprise-wide tracking and reporting of employee learning and  performance management.
section_title: PALMS Program Had Made Mixed Progress in Implementing Key IT Acquisition Best Practices
paragraphs: HRIT’s PALMS program varied in its implementation of IT acquisition best  practices for project planning, project monitoring, and risk management.  Specifically, the program management office had implemented selected  IT acquisition best practices in each of these areas; however, the  program had not developed complete life-cycle cost and schedule  estimates. Additionally, the PALMS PMO did not monitor total costs spent  on the program or consistently document the results from progress and  milestone reviews. Further, the program management office had not fully  implemented selected risk management practices. Without fully  implementing effective acquisition management practices, DHS is limited  in monitoring and overseeing the implementation of PALMS, ensuring that  the department obtains a system that improves its performance  management and learning management weaknesses, reduces  duplication, and delivers within cost and schedule commitments.
subsections: section_title: DHS and the PALMS PMO Had Not Fully Implemented Selected Project Planning Best Practices
subsections: section_title: PALMS Lacked a Life-Cycle Cost Estimate
paragraphs: According to GAO’s Cost Estimating and Assessment Guide, having a  complete life-cycle cost estimate is a critical element in the budgeting  process that helps decision makers to evaluate resource requirements at  milestones and other important decision points. Additionally, a  comprehensive cost estimate should include both government and  contractor costs of the program over its full life cycle, from inception of the  program through design, development, deployment, and operation and  maintenance to retirement of the program.
However, according to PALMS PMO officials, they did not develop a life- cycle cost estimate for PALMS. In 2012 DHS developed an independent  government cost estimate to determine the contractor-related costs to  implement the PALMS system across the department (estimated to be  approximately $95 million); however, this estimate was not  comprehensive because it did not include government-related costs. As a  result, DHS was not able to determine the impact on cost when the  PALMS program experienced problems (discussed in more detail later),  since the baseline cost estimate was incomplete. PALMS PMO officials  stated that PALMS did not develop a life-cycle cost estimate because the  program is a Level 3 acquisition program and DHS does not require such  an estimate for a Level 3 program. However, while DHS acquisition policy  does not require a life-cycle cost estimate for a program of this size, we  maintain that such an estimate should be prepared because of the  program’s risk and troubled history. Without developing a comprehensive  life-cycle cost estimate, DHS is limited in making future budget decisions  related to PALMS.
section_title: PALMS’s Schedule Was Incomplete and Inaccurate
paragraphs: As described in GAO’s Schedule Assessment Guide, a program’s  integrated master schedule is a comprehensive plan of all government  and contractor work that must be performed to successfully complete the  program. Additionally, such a schedule helps manage program schedule  dependencies. Best practices for developing and maintaining this  schedule include, among other things, capturing all activities needed to  do the work and reviewing the schedule after each update to ensure the  schedule is complete and accurate.
While DHS had developed an integrated master schedule with the  PALMS vendor, it did not appropriately maintain this schedule.  Specifically, the program’s schedule was incomplete and inaccurate.
While DHS’s original August 2012 schedule planned to fully deploy both the learning and performance management capabilities in one release at each component by March 2015, the program’s September 2015 schedule did not reflect the significant change in PALMS’s deployment strategy and time frames. Specifically, the program now plans to deploy the learning management capabilities first and the performance management capabilities separately and incrementally to headquarters and the components. However, the September 2015 schedule reflected the deployment-related milestones (per component) for only the learning management capabilities and did not include the deployment-related milestones for the performance management capabilities.
In September 2015, PALMS officials stated that the deployments  related to performance management were not reflected in the  program’s schedule because the components had not yet determined  when they would deploy these capabilities. Since then, two  components have determined their planned dates for deploying these  capabilities, but seven (including headquarters) remain unknown. As a  result, the program does not know when PALMS will be fully  implemented at all components with all capabilities. Table 4 provides  a comparison of the program’s initial delivery schedule, as of August  2012, to the program’s latest schedule, as of November 2015.
Moreover, the schedule did not include all government-specific activities, including tasks related to employee union activities (such as notifying employee unions and bargaining with them, where necessary) related to the proposed implementation of the performance management capabilities. For example, time frames for  when DHS planned to notify employee unions at DHS headquarters,  FLETC, and USCIS were not identified in the schedule. In September  2015, PALMS program officials stated that certain government- specific tasks were not included in the schedule because the  integrated master schedule was too big and difficult to manage, so the  program decided to track certain government activities, such as union  negotiation activities, separately. However, without an integrated  master schedule that includes all government and contractor work that  must be performed, the program is at risk of failing to ensure schedule  dependencies are appropriately managed and that all essential  activities are completed.
Additionally, the August 2015 schedule had incorrect completion dates listed for key activities. For example, DHS reported in the schedule that the actual finish date for deploying the learning management capabilities of the PALMS system at CBP was February 17, 2015; however, according to CBP officials, they did not deploy these capabilities until July 2015. In September 2015, program officials acknowledged our concerns and attributed the inaccurate dates to a lack of oversight; subsequently, the program took actions to update the dates.
Without developing and maintaining a single comprehensive schedule  that fully integrates all government and contractor activities, and includes  all planned deployment milestones related to performance management,  DHS is limited in monitoring and overseeing the implementation of  PALMS, and managing the dependencies between program tasks and  milestones to ensure that it delivers capabilities when expected.
section_title: PALMS Had Implemented Selected Project Monitoring Best Practices, but Did Not Implement Others
subsections: section_title: The PALMS PMO Did Not Monitor Total Costs
paragraphs: According to CMMI-ACQ and the PMBOK® Guide, a key activity for  tracking a program’s performance is monitoring the project’s costs by  comparing actual costs to the cost estimate. The PALMS PMO—which  is responsible for overseeing the PALMS implementation projects across  DHS, including all of its components—monitored task order expenditures  on a monthly basis. As of December 2015, DHS officials reported that  they had awarded approximately $18 million in task orders to the vendor.
However, the program management office officials stated that they were  not monitoring the government-related costs associated with each of the  PALMS implementations. The officials stated that they were not tracking  government-related implementation costs at headquarters because many  of the headquarters program officials concurrently work on other  acquisition projects and these officials are not required to track the  amount of time spent working specifically on PALMS. The officials also  said that they were not monitoring the government-related costs for each  of the component PALMS implementation projects because it would be  difficult to obtain and verify the cost data provided by the components.  We acknowledge the department’s difficulties associated with obtaining  and verifying component cost data; however, monitoring the program’s  costs is essential to keeping costs on track and alerting management of  potential cost overruns.
Additionally, because DHS did not develop a comprehensive life-cycle  cost estimate for PALMS that included government-related costs, the  program management office was unable to determine cost increases to  the program because it could not compare actual cost values against a  baseline cost estimate. For example, program officials were unable to  identify how much the program’s cost estimate had increased when the  implementation at headquarters experienced schedule delays to address  deficiencies identified during testing. Without tracking and monitoring all  costs associated with PALMS, the department will be unable to compare  actual costs against planned estimates and thus, will be limited in its  ability to fully monitor the program, which is essential for alerting the  program to possible cost overruns and prompting corrective actions.
section_title: The PALMS PMO Did Not Consistently Document the Results from Performance Reviews
paragraphs: According to CMMI-ACQ and the PMBOK® Guide, key activities in  tracking a program’s performance include conducting and documenting  the results from progress and milestone reviews to determine whether  there are significant issues or performance shortfalls that need to be  addressed.
Although the PALMS PMO conducted reviews to monitor the program’s  performance, it did not consistently document the results of its progress  and milestone reviews. For example,  The PALMS PMO did not document the results of the status updates that the PMO provided to DHS executives during its bi-weekly integrated project team meetings, so it is unclear whether the program was appropriately monitoring the progress of all government-specific activities.
According to PALMS PMO officials, PALMS achieved Initial Operating Capability—which was specified in the contract to be the point when the contractor would deliver an initial set of requirements to the government—in January 2015; however, the review for this major  milestone was not documented. In September 2015, program officials  stated that the results were not documented because this milestone  did not align with the typical Initial Operating Capability milestone that  is defined in DHS acquisition guidance. Specifically, DHS’s guidance  defines it as when capabilities are first deployed to end users (PALMS  capabilities were not deployed to any users until July 2015).  Nevertheless, PALMS’s achieving Initial Operating Capability in  January 2015 was still considered a major milestone that prompted a  review. However, without documenting the results of the milestone  review, it is unclear whether any action items were identified during  this review and, if so, whether they have all been appropriately  managed to closure.
Although CBP officials stated that the results of their progress reviews with the vendor were typically documented, CBP was unable to provide the results of the milestone review conducted prior to deploying the PALMS learning management capabilities in July 2015. As such, it is unclear whether any action items were identified during this review and, if so, whether CBP had appropriately managed them to closure.
In the absence of documenting PALMS’s progress and milestone reviews,  including all issues and corrective actions discussed, the program cannot  demonstrate that these issues and corrective actions are appropriately  managed.
section_title: The PALMS PMO Had Implemented Selected Risk Management Best Practices, but Lacked Others
subsections: section_title: PALMS Was Not Regularly Tracking the Status of Its Risks and Mitigation Efforts
paragraphs: According to CMMI-ACQ and the PMBOK® Guide, key risk management  practices include identifying risks, developing mitigation plans, and  regularly tracking the status of risks and mitigation efforts. In particular,  identifying risks and periodically reviewing them is the basis for sound  and successful risk management. Additionally, risk mitigation plans  should be developed and implemented when appropriate to proactively  reduce the potential impact if a risk were to occur.
While PALMS officials had identified program risks, developed associated  mitigation plans, and documented them in the HRIT investment-level risk  log (which is intended to be the centralized log containing all PALMS risks  and mitigation plans, including both government- and vendor-identified  risks), the program did not consistently maintain this log. Specifically,  the PALMS risks in this log were out of date, the log did not accurately  capture the status of all of the risks identified by the program, and it was  unclear which risks and associated mitigation plans were being assessed  on a monthly basis. For example,  In the May 2015 risk log, 16 of the 17 active PALMS risks stated that the last time any action was taken to mitigate or close any of these risks was in 2014. However, the mitigation strategy details for 5 of these active risks included information related to decisions made in 2015. As such, it was unclear which risks and mitigation plans were regularly assessed and updated in the risk log, and when actions were last taken on each of the risks.
One of the high-impact and high-probability risks from the May 2015 risk log stated that DHS needed to determine an interim solution for consolidating human resources-related data from DHS’s components by December 2014; however, the status of this risk had not been updated since August 2014 and it was unclear whether this was still a risk or had been realized as an issue.
Additionally, while the HRIT investment-level risk management plan  identified that the PALMS program was to, among other things, generate  weekly status reports to document the status of decisions made during  risk review meetings and identify planned completion dates for each step  of the risk mitigation plans, the program was not always complying with  these processes. For example, the program was not developing the  required weekly risk status reports or identifying planned completion  dates for its risk mitigation plan steps.
Program officials acknowledged that the PALMS risks in the HRIT risk log  were out of date and inaccurate, and the program was not complying with  all of the documented processes in the HRIT risk management plan.  Program officials attributed this to, among other things, the PMO’s focus  being on meeting upcoming deadlines; as such, implementing certain  processes identified in the HRIT risk management plan were not a  priority. However, by not carrying out these key risk management  functions, program officials introduced additional risk to the program.
In October 2015 and in response to us identifying these issues, PALMS  officials stated that they were in the process of validating and updating  the risks and mitigation plans in the HRIT risk log to address these  issues, as well as were updating their risk management processes to  align with the documented processes in the HRIT risk management plan.  The program completed this validation update process in October 2015;  however, the updated log continued to have these issues. For example,  the PALMS PMO had not yet identified the planned completion dates for  each mitigation step (where appropriate). Further, this updated log— which is intended to be the program’s centralized log of all government-  and vendor-identified PALMS risks—did not contain all of the vendor- identified risks. For example, two component-specific risks that were  identified in the vendor-maintained risk log were not included in the  program’s centralized risk log. As such, it is unclear whether the program  is appropriately managing these risks. Until a comprehensive risk log is  established that accurately captures the status of all risks (including both  government- and vendor-identified risks) and mitigation plans, and  includes planned completion dates for each mitigation step (where  appropriate), the program is limited in effectively managing all of its risks.
section_title: The PALMS PMO Did Not Prioritize the Program’s Risks or Document Criteria for Elevating Component Risks
paragraphs: According to CMMI-ACQ and PMBOK® Guide risk management best  practices, effective risk management includes evaluating and categorizing  risks using defined risk categories and parameters, such as probability  and impact, and determining each risk’s relative priority. Risk prioritization  helps to determine where resources for risk mitigation can be applied to  provide the greatest positive impact on the program. The parameters for  evaluating, categorizing, and prioritizing risks should include defined  thresholds (e.g., for cost, schedule, performance) that, when exceeded,  trigger management attention and mitigation activities. These risk  parameters should be documented so that they are available for  reference throughout the life of the project and are used to provide  common and consistent criteria for prompting management attention.
While the PALMS program had categorized its risks and assigned  parameters to them, including probability and impact, the program did not  prioritize its risks or document criteria for elevating them to management.  Specifically, the PALMS PMO did not use the assigned parameters to  determine each risk’s relative priority and overall risk level (i.e., high,  medium, and low). PALMS officials acknowledged in June 2015 that the  risks were not prioritized in the logs, but said, based on the experience of  the PALMS PMO staff, officials are able to determine each risk’s priority  by reviewing the assigned probability and impact parameters. However,  this is an inadequate method for managing risks. Specifically, it  introduces unnecessary subjectivity by relying heavily on officials to make  prioritization decisions, rather than using the assigned parameters to  determine and document each risk’s relative priority.
Additionally, the program had not documented criteria for elevating  component risks to the program management office. As mentioned  earlier, each component is responsible for overseeing its own PALMS  implementation project, while the program management office at  headquarters is responsible for overseeing the implementation projects  across the department. According to program officials, as part of this  effort, each component is to follow the risk management processes  documented in PALMS’s vendor-developed risk management plan (which  is a separate plan from the HRIT-level risk management plan used by the  program management office, as discussed earlier). While the PALMS  vendor-developed risk management plan directed each component to  track risks in a component-specific risk register, the plan did not establish  criteria for when component-level risks need to be elevated to the PALMS  PMO at headquarters.
In September 2015, the PALMS program manager stated that all  component-level risks that are rated red (i.e., high-probability and high- impact risks) are reported to headquarters. However, this guidance was  not documented and, as such, the PALMS PMO did not have reasonable  assurance that the components were knowledgeable about which risks to  elevate, and whether the components were appropriately elevating such  risks. Program officials were unable to explain why this criterion was not  documented, but in response to our concern, the program officials  directed the vendor to update the PALMS risk management plan to  document this criterion; the vendor completed this update in October  2015. In particular, the plan now specifies that all component-level risks  that could impact when the PALMS system is to be deployed at each of  the components should be elevated to the PALMS PMO and given a  priority of high.
Documenting the criteria for when risks need to be elevated to the  PALMS PMO should help ensure that all appropriate risks are being  elevated for review. However, until the program prioritizes its risks by  determining each risk’s relative priority and overall risk level, DHS is  hampered in its ability to ensure that the program’s attention and  resources for risk mitigation are used in the most effective manner.
section_title: Conclusions
paragraphs: Although the HRIT investment was initiated about 12 years ago with the  intent to consolidate, integrate, and modernize the department’s human  resources IT infrastructure, DHS has made very limited progress in  achieving these goals. HRIT’s minimally involved executive steering  committee during a time when significant problems were occurring was a  key factor in the lack of progress. This is particularly problematic given  that the department’s ability to efficiently and effectively carry out its  mission is significantly hampered by its fragmented human resources  system environment and duplicative and paper-based processes.
Moreover, DHS’s ineffective management of HRIT, such as the lack of an  updated schedule and a life-cycle cost estimate, also contributed to the  neglect this investment has experienced. Until DHS, among other things,  maintains a schedule, develops a life-cycle cost estimate, tracks costs,  and re-evaluates and updates the Human Capital Segment Architecture  blueprint, the department will continue to be plagued by duplicative  systems and an inefficient and ineffective human resources environment  impacting in its ability to perform its mission.
Additionally, until the PALMS program effectively addresses identified  weaknesses in its project planning, project monitoring, and risk  management practices and implements PALMS department-wide, DHS’s  performance management processes will continue to be cumbersome,  time-consuming, and primarily paper-based. Further, DHS will be limited  in efficiently tracking and reporting accurate, comprehensive performance  and learning management data across the organization, and could risk  further implementation delays.
section_title: Recommendations for Executive Action
paragraphs: To ensure that the HRIT investment receives necessary oversight and  attention, we are recommending that the Secretary of Homeland Security  direct the Under Secretary of Management take the following two actions:
Update the HRIT executive steering committee charter to establish the frequency with which HRIT executive steering committee meetings are to be held.
Ensure that the HRIT executive steering committee is consistently involved in overseeing and advising HRIT, including approving key program management documents, such as HRIT’s operational plan, schedule, and planned cost estimate.
To address HRIT’s poor progress and ineffective management, we are  recommending that the Secretary of Homeland Security direct the Under  Secretary of Management to direct the Chief Human Capital Officer to  direct the HRIT investment take the following six actions:
Update and maintain a schedule estimate for when DHS plans to implement each of the strategic improvement opportunities.
Develop a complete life-cycle cost estimate for the implementation of HRIT.
Document and track all costs, including components’ costs, associated with HRIT.
Establish time frames for re-evaluating the strategic improvement opportunities and associated projects in the Human Capital Segment Architecture Blueprint and determining how to move forward with HRIT.
Evaluate the strategic improvement opportunities and projects within the Human Capital Segment Architecture Blueprint to determine whether they and the goals of the blueprint are still valid and reflect DHS’s HRIT priorities going forward, and update the blueprint accordingly.
Update and maintain the department’s human resources system inventory.
To improve the PALMS program’s implementation of IT acquisition best  practices, we are recommending that the Secretary of Homeland Security  direct the Under Secretary of Management to direct the Chief Information  Officer to direct the PALMS program office to take the following six  actions:
Establish a time frame for deciding whether PALMS will be fully deployed at FEMA and USCG, and determine an alternative approach if the learning and/or performance management capabilities of PALMS are deemed not feasible for ICE, FEMA, TSA, or USCG.
Develop a comprehensive life-cycle cost estimate, including all government and contractor costs, for the PALMS program.
Develop and maintain a single comprehensive schedule that includes all government and contractor activities, and includes all planned deployment milestones related to performance management.
Track and monitor all costs associated with the PALMS program.
Document PALMS’s progress and milestone reviews, including all issues and corrective actions discussed.
Establish a comprehensive risk log that maintains an aggregation of all up-to-date risks (including both government- and vendor-identified) and associated mitigation plans. Additionally, within the comprehensive risk log, identify and document planned completion dates for each risk mitigation step (where appropriate), and prioritize the risks by determining each risk’s relative priority and overall risk level.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report from the Director  of DHS’s Departmental GAO-OIG Liaison Office. The comments are  reprinted in appendix II.
In its comments, the department concurred with our 14 recommendations  and provided estimated completion dates for implementing each of them.  For example, by April 30, 2016, the Under Secretary of Management  plans to ensure that the HRIT executive steering committee is  consistently involved in overseeing and advising HRIT and the committee  is expected to be reviewed quarterly by the Acquisition Review Board.  These planned actions, if implemented effectively, should help DHS  address the intent of our recommendations.
We also received technical comments from DHS headquarters and  component officials, which we have incorporated, as appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of  Homeland Security and other interested parties. In addition, this report is  available at no charge on the GAO website at http://www.gao.gov.
Should you or your staffs have any questions on information discussed in  this report, please contact Carol Cha at (202) 512-4456, ChaC@gao.gov  or Rebecca Gambler at (202) 512-6912, GamblerR@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) evaluate the progress the Department of  Homeland Security (DHS) has made in implementing the Human  Resources Information Technology (HRIT) investment and how effectively  DHS has managed the investment since completing the Human Capital  Segment Architecture in August 2011, (2) describe whether DHS has  justified its investment in the Performance and Learning Management  System (PALMS) program, (3) determine whether PALMS is being  implemented enterprise-wide, and (4) evaluate the extent to which  PALMS is implementing selected information technology (IT) acquisition  best practices.
To address the first part of our first objective—to evaluate the progress  DHS had made in implementing the HRIT investment—we compared  HRIT’s goals, scope, and implementation time frames (as defined in the  Human Capital Segment Architecture Blueprint, which was completed in  August 2011) to the investment’s actual accomplishments. Specifically,  we compared the completed and in-progress HRIT projects against the  strategic improvement opportunities and projects that were outlined in the  blueprint to determine which of the improvement opportunities and  projects had been fully implemented or were in-progress. We also  compared DHS’s planned schedule for implementing the improvement  opportunities and projects against DHS’s current planned schedule for  implementing them as of November 2015. Additionally, we interviewed  DHS officials from the HRIT investment, Office of the Chief Information  Officer (OCIO), Office of the Chief Human Capital Officer (OCHCO), and  DHS’s components to discuss the steps taken to implement HRIT,  address the strategic improvement opportunities and projects in the  blueprint, and meet the goals of the investment.
In addressing the second part of our first objective—to evaluate how  effectively DHS managed the investment—we analyzed documentation,  such as the investment’s planned and updated completion dates,  program management briefings, the blueprint, cost estimates, and budget  documentation, and compared it against relevant cost and schedule best  practices identified by GAO, CMMI-ACQ, and the PMBOK® Guide. These  best practices included developing and maintaining a schedule estimate;  developing a life-cycle cost estimate; and tracking program expenditures.  To determine the amount spent to date on HRIT, we asked officials from  DHS headquarters and each of the eight components to provide  expenditure information on HRIT since the investment began in 2003;  officials were unable to provide complete information. As such, we were  unable to identify the total amount spent on the investment and discuss  this limitation earlier in the report.
We also analyzed DHS’s human capital investment guidance, including  the 2010 Deputy Secretary memorandum that prohibited component  spending on enhancements to existing human resources systems or  acquisitions of new human resources solutions, unless those  expenditures have been approved by OCHCO or OCIO, and compared it  to the components’ current investments in human resources systems,  such as those listed in DHS’s fiscal year 2016 human capital portfolio.  Additionally, we interviewed officials from the OCIO, OCHCO, and DHS’s  eight components to obtain additional information on how HRIT reduced  or will reduce duplicative human resources systems.
To describe whether DHS justified its investment in the PALMS program,  we analyzed documentation, such as the program’s business case and  the documented analysis of alternatives that was conducted to identify  recommended approaches for pursuing a commercial off-the-shelf  learning management system. We used this information to determine the  various alternative solutions that DHS assessed for delivering enterprise- wide performance and learning management capabilities. Additionally, we  reviewed program management briefings provided to the HRIT Executive  Steering Committee that outlined, for example, the proposed solution and  rationale for such a solution. We also interviewed appropriate DHS and  PALMS officials for further information regarding the process DHS used  to conduct the analysis of alternatives and other steps the department  took to determine its preferred solution, including determining whether  DHS could use existing shared services that were being used by other  federal agencies.
To determine whether PALMS is being implemented department-wide, we  analyzed the program’s acquisition plan and original schedule for  implementing the system department-wide, and compared it against  actual program status documentation and the program’s current  implementation schedule. We also obtained and analyzed information  from DHS officials, the PALMS headquarters program management office  and DHS’s components on each component’s implementation of PALMS,  including identifying which PALMS capabilities each component planned  to implement, the number of planned PALMS users, and their reported  causes for why certain components were not currently planning to  implement PALMS.
To evaluate the extent to which PALMS implemented selected IT  acquisition best practices, we analyzed the program’s IT acquisition  documentation and compared it to relevant project planning, project  monitoring, and risk management best practices—including CMMI-ACQ  and PMBOK® Guide practices, and best practices identified by GAO.  Specifically, we analyzed program documentation, including the  acquisition plan, requirements management plan, risk management plan,  cost and schedule estimates, program management review briefings,  meeting minutes, risk logs, and risk mitigation plans to determine the  extent to which the program’s acquisition processes were consistent with  the best practices. Additionally, we interviewed officials from HRIT,  PALMS, OCIO, OCHCO, and DHS’s eight components to obtain  additional information on the program’s risk management, project  planning, and project monitoring processes.
To assess the reliability of the data that we used to support the findings in  this report, we reviewed relevant program documentation to substantiate  evidence obtained through interviews with agency officials. We  determined that the data used in this report were sufficiently reliable, with  the exception of expenditure information provided by the HRIT investment  and selected risk data provided by the PALMS program. We discuss  limitations with these data in the report. We have also made appropriate  attribution indicating the sources of the data.
We conducted this performance audit from March 2015 to February 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Homeland Security
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the contacts named above, the following staff also made  key contributions to this report: Shannin O’Neill, Assistant Director;  Christopher Businsky; Rebecca Eyler; Javier Irizarry; Emily Kuhn; and  David Lysy.