PALMS Does Not Address 

Department Needs 

June 30, 2017 

OIG-17-91 
 

 

 

 

 

 

 

DHS OIG HIGHLIGHTS
 

PALMS Does Not Address Department Needs 

June 30, 2017 

Why We Did 

This Audit 

The Department of 

Homeland Security remains 

on the Government 

Accountability Office’s (GAO) 

high-risk list for 

management of its human 

capital and integration. In 

2013, DHS entered into an 

agreement to provide the 

Department with a 

Performance and Learning 

Management System 

(PALMS) to improve the 

efficiency and oversight of 

DHS workforce training and 

performance management. 

PALMS is a talent 

management system that 

integrates learning and 

performance management 

capabilities. 

What We 

Recommend 

We made seven 

recommendations to address 

challenges associated with 

PALMS acquisition and to 

improve future acquisitions. 

For Further Information: 

Contact our Office of Public Affairs at 

(202) 254-4100, or email us at 

DHS-OIG.OfficePublicAffairs@oig.dhs.gov 

What We Found 

DHS PALMS does not address the Department’s critical 

need for an integrated, department-wide learning and 

performance management system. As of October 2016, 

PALMS has not met DHS operational requirements for 

effective administration of employee learning and 

performance management activities. This occurred 

because the PALMS program office did not effectively 

implement its acquisition methodology and did not 

monitor contractor performance. GAO also reported in its 

February 2016 report, GAO-16-253, that the Department 

experienced programmatic and technical challenges that 

led to years-long schedule delays. As a result, despite 

spending $24.2 million as of February 2017, DHS PALMS 

does not achieve the intended benefits or address the 

Department’s needs. 

In addition, between August 2013 and November 2016, 

the Department spent more than: 

x $5.7 million for unused and partially used 

subscriptions; 

x $11 million to extend contracts of existing learning 

management systems; and 

x $813,000 for increased program management costs. 

The Department also did not identify $72,902 in financial 

credits stemming from the contractor not meeting 

performance requirements between June and September 

2015. 

DHS Response 

In its response to our draft report, DHS reported that it 

remains committed to ensuring efficient and effective 

delivery of capability to meet the enterprise-wide 

performance and learning needs, including addressing the 

operational concerns of all of its components. Accordingly, 

DHS agreed with our report recommendations. 

www.oig.dhs.gov OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Washington, DC 20528 / www.oig.dhs.gov 

June 30, 2017 

MEMORANDUM FOR: 	 The Honorable Chip Fulghum 

Acting Under Secretary for Management and 

Chief Financial Officer 

Richard Staropli 

Chief Information Officer 

FROM: John Roth ~~"\(o~ Inspector General 

SUBJECT: 	 PALMS Does Not Address Department Needs 

Attached for your action is our final report, PALMS Does Not Address 

Department Needs. We incorporated the formal comments from the Department 

in the final report. 

The report contains seven recommendations aimed at enhancing the 

Department's overall effectiveness of its programs. Your office concurred with 

all seven recommendations. Based on information provided in your response to 

the draft report, we consider recommendations 1, 5, and 7 open and 

unresolved. As prescribed by the Department of Homeland Security Directive 

077-01, Follow-Up and Resolutions for the Office ofInspector General Report 

Recommendations, within 90 days of the date of this memorandum, please 

provide our office with a written response that includes your (1) agreement or 

disagreement, (2) corrective action plan, and (3) target completion date for each 

recommendation. Also, please include responsible parties and any other 

supporting documentation necessary to inform us about the current status of 

the recommendation. Until your response is received and evaluated, the 

recommendations will be considered open and unresolved. 

Based on information provided in your response to the draft report, we 

consider recommendations 2, 3, 4 and 6 open and resolved. Once your office 

has fully implemented the recommendations, please submit a formal closeout 

letter to us within 30 days so that we may close the recommendations. The 

memorandum should be accompanied by evidence of completion of agreedupon corrective actions and of the disposition of any monetary amounts. 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Consistent with our responsibility under the Inspector General Act, we are 

providing copies of our report to appropriate congressional committees with 

oversight and appropriation responsibility over the Department of Homeland 

Security. We will post the report on our website for public dissemination. 

Please call me with any questions, or your staff may contact John V. Kelly, 

Deputy Inspector General, at (202) 254-4100. You can also send your response 

to OIGAuditsFollowup@oig.dhs.gov. 

Attachment 

www.oig.dhs.gov 2 
 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Table of Contents 

Background .................................................................................................... 1 

Results of Audit .............................................................................................. 2 

DHS PALMS Does Not Meet Department Needs ...................................... 3 

Recommendations ................................................................................. 5 

PALMS Program Office Did Not Effectively Execute Its Acquisition 

Methodology .......................................................................................... 7 

Recommendations ................................................................................. 8 

Additional Costs Due to Implementation Delays................................... 11 

Recommendations ............................................................................... 13 

Appendixes 

Appendix A: Objective, Scope, and Methodology ................................. 15 

Appendix B: DHS Comments to the Draft Report ................................. 17 

Appendix C: PALMS QASP Performance Measures ............................... 21 

Appendix D: Subscriptions Purchased before PALMS Became 

Operational...................................................................... 22 

Appendix E: Office of Audits Major Contributors to This Report ........... 23 

Appendix F: Report Distribution .......................................................... 24 

Abbreviations 

BPA blanket purchase agreement 

CBP U.S. Customs and Border Protection 

ECD estimated completion date 

FEMA Federal Emergency Management Agency 

FLETC Federal Law Enforcement Training Center 

GAO Government Accountability Office 

HRIT Human Resources Information Technology 

ICE U.S. Immigration and Customs Enforcement 

IT Information Technology 

LMS learning management systems 

OCPO Office of the Chief Procurement Officer 

OIG Office of Inspector General 

PALMS Performance and Learning Management System 

PARM Office of Program Accountability and Risk Management 

QASP quality assurance surveillance plan 

SaaS software as a service 

www.oig.dhs.gov OIG-17-91 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

SELC Systems Engineering Life Cycle 

SIO Strategic Improvement Opportunities 

TSA Transportation Security Administration 

USCG United States Coast Guard 

USCIS U.S. Citizenship and Immigration Services 

USSS United States Secret Service 

www.oig.dhs.gov OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Background 

In support of Department of Homeland Security Efficiency Review Initiatives to 

streamline human resources and information technology operations, DHS 

completed a study in 2011 that examined department-wide learning 

management systems (LMS). The study found that DHS components 

independently operated and governed nine separate systems from four different 

vendors and recommended the implementation of a single department-wide 

LMS as the most cost-effective approach. 

Following multiple studies and market research, the Department concluded in 

January 2012 that a talent management system that integrates learning 

management and performance management capabilities would be the most 

cost-effective and efficient method. DHS developed a business case in July 

2012 for the acquisition of a department-wide system for performance and 

learning management. The business case analyzed the rationale and benefits of 

consolidating learning and performance management capabilities into a single 

department-wide system. The business case also concluded that a departmentwide system provides a more efficient learning and performance management 

capability at a lower cost than the sum of existing systems at DHS and the 

components. Based on the market research and analyses, the Department 

chose to acquire a “software as a service,” or SaaS, solution. The SaaS 

business model consists of an annual ‘rental’ fee, or subscription, that a 

service provider charges for customers to use software applications. The 

Department estimated the implementation timeframe for a SaaS solution to be 

30 to 90 days, including customization, data conversion, and configuration. 

In May 2013, DHS entered into a 5-year blanket purchase agreement (BPA) 

with Visionary Integration Professionals — with an estimated $95 million 

ceiling — to provide DHS with a department-wide employee performance and 

learning management system, known as the Performance and Learning 

Management System, or PALMS. DHS established a program office within the 

Human Capital Business Systems to implement PALMS. The Department 

originally scheduled PALMS to become operational at nine components on a 

staggered schedule beginning with DHS headquarters in September 2013 and 

eight components to be operational on or before March 2015. Headquarters 

issued the first implementation task order against the BPA in August 2013 

with a delivery target of December 2013. Due to unmet operational needs, the 

www.oig.dhs.gov 1 OIG-17-91 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Department delayed PALMS multiple times and accepted1 the system as 

operational in January 2015. However, the system only became partially 

operational for DHS headquarters users in October 2015. 

In a 2016 Government Accountability Office (GAO) report, Oversight of 

Neglected Human Resources Information Technology Investment Is Needed, 

GAO-16-253, GAO reported that the Department experienced programmatic 

and technical challenges that led to years-long schedule delays. The 

Department had not developed complete life-cycle cost and schedule estimates, 

monitored total cost spent on PALMS, or consistently documented the results 

from progress and milestone reviews. GAO also found that the program 

management office had not fully implemented selected risk management 

practices. 

Results of Audit 

PALMS does not address the Department’s critical need for an integrated 

department-wide system; it has not met DHS operational requirements for 

effective administration of employee learning and performance management 

activities. This occurred because the PALMS program office did not effectively 

implement the acquisition methodology selected for PALMS and did not 

monitor contractor performance. As a result, despite spending $24.2 million as 

of February 2017, DHS PALMS does not achieve the intended benefits or 

address the Department’s need. 

In addition, between August 2013 and November 2016, the Department spent 

more than: 

x $5.7 million for unused and partially used subscriptions; 

x $11 million to extend contracts of existing LMS; and 

x $813,000 for increased program management costs. 

The Department also did not identify $72,902 in financial credits stemming 

from the contractor not meeting performance requirements between June and 

September 2015. 

 1 Acceptance consists of approval of operational requirements after assessing whether PALMS 

can support day-to-day business and user scenarios, and the system is sufficient and correct 

for business use. 

www.oig.dhs.gov 2 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

DHS PALMS Does Not Meet Department Needs 

In 2013, DHS entered into an agreement to acquire PALMS to address its 

critical need for an integrated enterprise system with automated performance 

and learning management capabilities. The Department expected PALMS to 

improve the efficiency and reliability of performance and learning management 

processes. By implementing a department-wide system, DHS intended to 

integrate and automate performance and learning management activities of its 

workforce. Based on the Business Case, DHS expected PALMS to achieve cost 

savings of more than $52 million over a 5-year period — $40.8 million in 

supervisory labor costs and $11.2 million for volume discounts. However, the 

Department did not achieve those efficiencies or cost savings because it has 

not implemented the performance management capability as of October 2016 

and has not met the requirements of three major DHS components. 

Furthermore, the Department accepted PALMS as operational before verifying 

all the operational requirements. 

The Department accepted PALMS as operational in January 2015 without 

verifying that 131 of the 4812 (27 percent) operational requirements were 

functional and in compliance with system specifications. As summarized in 

table 1, PALMS program officials did not test 97 requirements, accepted 21 

requirements despite failed test results, and did not verify 13 requirements for 

compliance with system specifications. After the testing phase in December 

2014, seven different stakeholders, including an executive director within the 

Office of the Chief Human Capital Officer and representatives from five 

components and Headquarters, expressed serious concerns regarding PALMS’ 

operational capability. In November 2014 and January 2015, the executive 

director brought these serious concerns to the attention of the Chief 

Information Officer and the Chief Human Capital Officer. In December 2014, 

component representatives from U.S. Immigration and Customs Enforcement 

(ICE), Transportation Security Administration (TSA), United States Secret 

Service, United States Coast Guard, Federal Emergency Management Agency 

(FEMA), and DHS headquarters also expressed concerns about PALMS’ unmet 

requirements to the PALMS program office. Despite these concerns, DHS 

accepted PALMS in January 2015 as meeting minimum operational capability. 

 2 There were a total of 483 requirements listed in the Department’s requirements traceability 

matrix. Of the 483 requirements, OIG found two duplicate requirements that decreased the 

total number of requirements to 481. Our report focused on the 131 requirements that the 

Department did not verify before PALMS acceptance. Based on our analysis of the 

Department’s requirement traceability matrix, we did not review the remaining 350 

requirements.

www.oig.dhs.gov 3 OIG-17-91 
 

Requirements Reason for not verifying the Requirements 

97 Not Tested 

66 Designated for Component Testing 

29 Deferred for completion after Acceptance 

2 Designated as ‘Not Applicable’ 

21 Accepted with 

Failed Results 

11 Passed by management 

10 Did not retest failed requirements 

13 Not Verified 13 Did not check for system specifications 

Source: Office of Inspector General (OIG) analysis of PALMS acceptance test results. 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Table 1: Unverified Requirements 

Although the Department accepted PALMS as operational in January 2015, the 

system’s learning management capability was not operational for DHS 

headquarters users until October 2015, and the performance management 

capability is still not operational due to unmet operational requirements. In 

February 2016, the Department launched a multiphase pilot of the 

performance management capability to determine the system’s readiness for 

deployment at DHS headquarters in fiscal year 2017. The first phase of the 

pilot resulted in a 13 percent success rate on the ability to duplicate employee 

performance plans and 0 percent success rate to produce required reports with 

Federal terms and requirements. As of June 2016, the second phase of the 

pilot continued to uncover new functionality issues. Based on the results of the 

first and second phases of the pilot, in September 2016, the DHS headquarters 

performance management team recommended DHS headquarters offices not 

use PALMS’ performance management capability for the fiscal year 2017 

performance period. The team further recommended that DHS headquarters 

offices continue to use existing paper forms for employee performance plans 

and appraisals. 

Despite these issues and recommendations, a September 2016 acquisition 

decision memorandum, based on a June 2016 acquisition review board 

decision, directed the Chief Information Officer to stop further development of 

three critical performance management requirements and allow DHS 

components to implement PALMS ‘as is.’ The memo also deferred the 

implementation of PALMS for FEMA, Coast Guard, and TSA because the 

system does not meet these components’ needs. However, the Department 

continued to pay PALMS subscription fees for performance management 

capabilities and service fees for department-wide system support. 

FEMA, Coast Guard, and TSA reported disadvantages of PALMS, including 

access restrictions and limited reporting capabilities. Specifically, large 

segments of their workforce may not be able to access PALMS due to Personal 

Identity Verification card requirements and off-line training limitations. These 

www.oig.dhs.gov 4 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

components also noted that the limited reporting functions would reduce their 

capability to effectively track and report on workforce training. 

The U.S. Customs and Border Protection (CBP) also noted that PALMS does not 

generate accurate reports of employee training. CBP issued a task order in 

June 2015 to the PALMS contractor for $136,235 to develop additional PALMS 

reporting capabilities. However, CBP’s field offices could not generate accurate 

reports on employee training as of October 2016. 

Without a system that meets DHS’ operational requirements, the Department 

cannot address its capability needs or achieve its intended training and 

performance management goals. 

Recommendations 

Recommendation 1: We recommend that the DHS Chief Information Officer 

evaluate the current functional capabilities of PALMS and develop a plan to 

address the Department’s outstanding capability needs for learning and 

performance management of DHS personnel. 

Recommendation 2: We recommend that the DHS Chief Information Officer 

conduct an analysis of PALMS BPA pricing and terms and renegotiate as 

necessary to reflect the actual capabilities delivered. 

DHS Comments and OIG Analysis 

In its response to our draft report, DHS concurred with all seven of our report 

recommendations. We incorporated the Department’s comments, responses to 

our recommendations, and our analysis with the applicable recommendations 

in the report. We also included a copy of the management comments in their 

entirety in appendix B. DHS also provided technical comments to our draft 

report, which we considered in our analysis. 

DHS Comments: DHS reported that it remains committed to ensuring efficient 

and effective delivery of capability to meet the enterprise-wide performance and 

learning needs, including addressing the operational concerns of all of its 

components. DHS agreed that PALMS did not deliver an integrated capability, 

resulting in three components remaining on their current systems and not 

migrating to the enterprise solution. DHS noted that the acceptance of PALMS 

operational requirements, despite failed test results, was a tradeoff decision 

within the delivery of the ‘software as a service’ product. 

www.oig.dhs.gov 5 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

DHS also noted that 29 of the 97 unverified requirements were deferred for 

Final Operating Capability (FOC) delivery and 27 of them were identified as 

passed before DHS headquarters went live in October 2015. 

OIG Analysis: We believe the report accurately notes that the program officials 

did not test the 29 requirements before PALMS acceptance in January 2015 

because they were deferred for completion after the system acceptance. The 

system’s learning management capability was not operational for the 

headquarters users until October 2015, and the performance management 

capability is still not operational due to unmet operational requirements. 

DHS Comments to Recommendation 1: DHS concurred with our 

recommendation and stated that the outstanding capability needs for DHS 

learning management and performance management are being analyzed, 

reviewed, and developed under the DHS Human Resources Information 

Technology portfolio, Strategic Improvement Opportunities (SIO) #4 and #7, 

respectively. For SIO #4, learning management, and SIO #7, performance 

management, DHS is currently evaluating and developing the consolidated list 

of enterprise-level functional and technical requirements with all DHS 

components. Estimated completion date (ECD): May 31, 2017. 

OIG Analysis of DHS Comments: The Department’s proposed corrective 

actions do not fully address the recommendation. The evaluation and review of 

the outstanding capability needs under the DHS Human Resources 

Information Technology SIO #4 and #7 should also include an evaluation of 

DHS PALMS’ capability to address the consolidated list of enterprise level 

functional and technical requirements that will be developed. Until DHS 

includes an evaluation of PALMS’ current capabilities as part of the SIO #4 and 

#7 evaluations and reviews, we consider this recommendation unresolved and 

open. The estimated completion date has passed. We will track the status until 

action is taken for the recommendation. 

DHS Comments to Recommendation 2: DHS concurred with our 

recommendation and stated that the DHS Office of the Chief Procurement 

Officer renegotiated the BPA subscription rate for tier 4 from $35.75 to $26.81, 

as of June 2, 2016. DHS also stated that the Office of the Chief Procurement 

Officer (OCPO) is having ongoing discussions with the PALMS vendor to 

renegotiate BPA pricing and terms for future subscription renewal dates. ECD: 

May 31, 2017. 

OIG Analysis of DHS Comments: We consider DHS’ proposed corrective 

action to be responsive to the recommendation. DHS OCPO should renegotiate 

www.oig.dhs.gov 6 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

the subscription and program management prices to reflect the limited 

capabilities of PALMS for performance management. The recommendation is 

considered open and resolved. It will remain open until DHS provides support 

of the renegotiated BPA pricing and terms based on the PALMS’ current 

capability. DHS will also need to provide support for the ongoing discussions 

with the PALMS vendor to renegotiate BPA pricing and terms for future 

subscription renewal dates. The estimated completion date has passed. We will 

track the status until action is taken for the recommendation. 

PALMS Program Office Did Not Effectively Execute Its 

Acquisition Methodology 

The Department required the PALMS contractor to follow DHS’ Systems 

Engineering Life Cycle (SELC) methodology and an agile development approach 

for PALMS implementation. DHS Acquisition directives and instructions require 

all information technology acquisitions to conduct SELC activities. The SELC is 

a systems engineering methodology for managing the acquisition of programs 

from inception to disposal. The PALMS contractor created a plan to follow the 

SELC methodology with seven review stages, each with specific exit criteria to 

successfully move to the next phase. However, DHS did not implement this 

plan or perform any of the required system engineering reviews. 

As part of the agile development approach, the PALMS contractor developed a 

requirements management plan in August 2013 to manage PALMS operational 

requirements including conversion of requirements into “user stories.” In 

October 2015, the Department approved and accepted this plan as complete 

even though the contractor did not develop user stories for 99 percent of the 

requirements. An agile development approach is a repetitive and incremental 

process to develop information technology capabilities in which requirements 

are expressed as user stories. The purpose of the user stories is to ensure that 

the requirements are tied into a functional system that can be changed as 

feedback is provided. However, the contractor did not develop user stories for 

478 of the 481 PALMS requirements. 

DHS Did Not Perform the Required Contractor Performance Monitoring 

The PALMS program office did not perform quality assurance monitoring of 

contractor performance. Federal regulations require agencies to use a quality 

assurance surveillance plan (QASP) to ensure that services the contractor 

provides conform to contract requirements. The Department’s September 2014 

QASP outlined 15 performance measures that provide incentives and penalties 

www.oig.dhs.gov 7 OIG-17-91 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

based on the contractor’s performance results. See Appendix C for the full list 

of performance measures. 

One of the QASP performance measures indicates that the contractor, 

Visionary Integration Professionals, must complete data migration in 

accordance with the approved schedule. Table 2 describes the performance 

measure as provided in QASP. According to the September 2014 DHS PALMS 

task order, the contractor should have completed the migration of 

headquarters’ data by January 2015. However, the contractor did not complete 

the migration until October 2015. 

Table 2: PALMS QASP Performance Measure 

Service: Migration Schedule 

Performance 

Standard Migration is completed in accordance with approved schedule. 

Incentive / Credit Actual Service +/- % Performance Score 

On-time payment of 

deliverables as defined in the 

task order. 

Payment withheld for each 

deliverable defined in the task 

order until such time as the 

Government accepts each 

deliverable. Additionally, 25% of 1 

month of the PALMS subscription 

value is credited if the services are 

delayed by greater than 30 days. 

Source: DHS headquarters, PALMS BPA QASP (September 10, 2014). 

Based on the performance measure shown in table 2, DHS should have 

received $72,901.683 in financial credits toward subscription costs. A full 

assessment of the contractor’s performance may allow the Department to 

identify additional opportunities for financial compensation. 

Recommendations 

Recommendation 3: We recommend that the DHS Chief Information Officer 

evaluate PALMS contractor performance across DHS components using the 

PALMS BPA QASP, and ensure that the Department receives full financial 

credit for contractor non-performance. 

 3 According to Office of the Chief Human Capital Officer executives, PALMS was shut down 

from approximately January 2015 to May 2015. Furthermore, contract records show that there 

were no PALMS subscription values to use in our calculation during this period. Therefore, we 

excluded these months from our calculation. 

www.oig.dhs.gov 8 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Recommendation 4: We recommend that the Undersecretary for Management 

implement controls for the Department and component Chief Information 

Officers to oversee information technology acquisitions for compliance with 

systems engineering requirements and agile acquisition methodologies. 

Recommendation 5: We recommend that the Undersecretary for Management 

implement controls to ensure that technical monitoring of contracts occurs in 

accordance with Federal acquisition regulations. 

DHS Comments and OIG Analysis 

DHS Comments: DHS stated that the PALMS HQ integrated master schedule 

(IMS) shows 100 percent completion of PALMS Systems Engineering Life Cycle 

(SELC) reviews, including the seven review stages and all tasks within those 

stages. 

OIG Analysis: Our analysis of the IMS tasks the program office identified as 

equivalent of the SELC stage reviews revealed that the program office has 

developed only 5 of the 47 deliverables and work products listed in the SELC 

plan to support each stage of the PALMS program. The program office has not 

updated those five documents or the SELC plan since their early drafts in 

August 2013. We requested supporting documentation for all the IMS tasks 

identified as SELC reviews. The program officials responded that the only 

documentation available is the IMS itself showing the review of the technical 

deliverables. 

The Department’s SELC is the framework used to guide all DHS projects with 

specific guidance provided for each type of acquisition mechanism. It is 

composed of multiple stages with a defined set of activities and corresponding 

documents to record the result of activities. Stage reviews are conducted to 

validate that the acquisition has completed requirements for each stage and is 

ready to advance to the next stage. Based on the review of the IMS tasks the 

program office identified as SELC reviews, none of the seven PALMS SELC 

stage reviews were completed in the prescribed manner outlined by DHS policy. 

DHS Comments to Recommendation 3: The Department concurred with our 

report recommendation and stated that DHS Office of the Chief Information 

Officer will evaluate the PALMS contractor performance across all DHS 

components using the PALMS QASP to ensure all performance standards were 

met and identify areas where they were not met. DHS CIO will determine if 

financial credits are applicable and, if so, will process them for payment, as 

appropriate. ECD: October 31, 2017. 

www.oig.dhs.gov 9 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

OIG Analysis of DHS Comments: We consider DHS’ proposed corrective 

action to be responsive to the recommendation. The recommendation is 

considered open and resolved. It will remain open until we receive a: 

1. Copy of Office of the Chief Information Officer’s evaluation of the PALMS 

contractor performance across all DHS components using the PALMS 

QASP and identifying areas the vendor did not meet. 

2. List of any financial credits identified by the DHS CIO and support for 

any payments resulting from the financial credits. 

DHS Comments to Recommendation 4: The Department concurred with this 

recommendation and stated that the USM and CIO have implemented robust 

Acquisition Lifecycle Framework and SELC processes for Information 

Technology (IT) acquisitions. The USM has also recently approved an 

acquisition policy for the management of Level 3 acquisitions. PALMS pre-dated 

the latest implementations of these processes. However, the CIO will work with 

the DHS Office of Program Accountability and Risk Management (PARM) for 

Headquarters IT systems to develop and begin to enforce SELC tailoring plans 

that provide the appropriate level of systems engineering and agile acquisition 

methodologies to ensure best value from IT investments. ECD: June 30, 2017. 

OIG Analysis of DHS Comments: We consider DHS’ proposed corrective 

action to be responsive to the recommendation. The recommendation is 

considered open and resolved and will remain open until we receive and 

analyze the SELC tailoring plans that PARM develops with the appropriate level 

of systems engineering and agile acquisition mythologies to ensure best value 

from IT investments. We will also need support showing PARM’s plan to enforce 

the SELC tailoring plans. 

DHS Comments to Recommendation 5: The Department concurred with this 

recommendation and stated that DHS OCPO already has procedures and 

policies in place for Contracting Officer Representatives (COR). Contracting 

Officers ensure COR appointments are assigned to individuals only after a 

thorough review of training history. CORs must be certified at Level II or Level 

III depending on the complexity of the contract. DHS will reiterate the 

importance of CORs’ responsibilities to technically monitor performance and 

deliverables at Task Order Kick-Off Meetings and in the Ordering Guide. ECD: 

June 30, 2017. 

OIG Analysis of DHS Comments: DHS’ proposed correcting action does not 

fully address the recommendation. A review of training history or the 

reiteration of the importance of COR’s responsibilities does not serve as a 

control to ensure technical monitoring of ongoing contracts. Despite multiple 

www.oig.dhs.gov 10 OIG-17-91 
Migration Schedule Contract 

Extension 

Costs ($) 

Program 

Management 

Cost Increases 

Total Costs 

Original Actual ($)

FLETC Sep-13 Dec-15 864,206 -- 864,206 

HQ Sep-13 Oct-15 1,258,335 $469,898 1,728,233 

ICE Mar-14 Jun-16 1,493,319 -- 1,493,319 

USCIS Sep-14 Oct-16 1,055,473 $344,031 1,399,504 

CBP Mar-15 Jul-15 Unknown* -- Unknown* 

USSS Sep-14 Pending 230,000 -- 230,000 

TSA Sep-13 Deferred 5,361,250 -- 5,361,250 

FEMA Sep-14 Deferred 782,564 -- 782,564 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

delays, the contracting officer or the program manager did not ensure that the 

COR conducted technical monitoring of the contracts. An effective control to 

identify and correct such noncompliance with Federal regulations and DHS 

policies needs to be designed and implemented. This recommendation will 

remain unresolved and open until DHS implements controls to ensure 

technical monitoring of contracts in accordance with Federal acquisition 

regulations. 

Additional Costs Due to Implementation Delays 

DHS incurred more than $11.8 million in additional costs due to PALMS 

implementation delays. The implementation delays occurred because the 

system did not meet the Department’s operational requirements. This caused 

the components to spend at least $11 million for extending contracts of 

components’ existing, legacy LMS contracts and $813,929 for PALMS program 

management costs. 

In 2012, DHS created a migration schedule to overlap with the expiration of 

existing LMS contracts. For example, DHS headquarters’ initial LMS contract 

expired in December 2013. According to the initial migration schedule in the 

BPA, DHS headquarters should have implemented PALMS in September 2013. 

However, DHS headquarters did not begin using PALMS until October 2015, 

which resulted in an additional cost of at least $1.26 million due to multiple 

extensions. Components also faced implementation delays and spent at least 

an additional $9.79 million to retain their legacy contracts. See table 3 for 

additional information. 

Table 3: PALMS Implementation and Legacy LMS Contract Extensions 

www.oig.dhs.gov 11 OIG-17-91 
-- 

 

 

 

Wasted 

Subscriptions Funds Reason 

Unused 206,083 

25,500 

$4,692,187 

$537,540 

Expired before PALMS became operational 

Purchased excess PALMS subscriptions 

Partially 

Used 114,075 $535,763 Unused until PALMS became operational 

Total 345,658 $5,765,490 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

USCG Mar-14 Deferred Unknown* Unknown* 

Total $11,045,147 $813,929 $11,859,076 

Source: OIG analysis of components’ LMS contracts and PALMS migration schedule.
 

*Coast Guard (USCG) and CBP did not know the number of legacy LMS contract extensions or 

the amount; therefore, OIG could not include their extension costs.
 

DHS also spent a total of $813,929 for increased program management costs 

as part of the PALMS implementation for DHS headquarters and U.S. 

Citizenship and Immigration Services (USCIS). 

DHS Wasted More than $5.7 Million for PALMS Subscriptions 

DHS spent more than $5.7 million for PALMS subscriptions that components 

did not use or only used for parts of the subscription periods. The PALMS BPA 

pricing terms allowed DHS to pay for active system users on an annual basis 

rather than for a specified number of users up front for a specified period of 

time. However, the Department purchased 345,658 subscriptions before 

PALMS was operational for active DHS users. As a result, as shown in table 4, 

the Department did not use 231,583 of the 345,658 subscriptions and used 

114,075 of the subscriptions for parts of the subscription period. 

Table 4: Unused and Partially Used PALMS Subscriptions 

Source: OIG analysis of PALMS task orders and implementation schedule. 

DHS headquarters and components paid more than $4.6 million for 206,083 

subscriptions that expired before PALMS became operational. DHS 

headquarters also spent $537,540 for 25,500 PALMS subscriptions in excess of 

the 16,000 needed for DHS headquarters users. In addition, DHS paid 

$535,763 for 114,075 subscriptions that were used after the system became 

operational. This caused DHS to spend more than $5.7 million for 

subscriptions that DHS employees were not able to use for performance and 

learning management. 

To illustrate the waste in unused subscriptions that expired before PALMS 

became operational, CBP spent $1,570,835 in September 2013 for 67,360 of 

www.oig.dhs.gov 12 OIG-17-91 
 

Date Subscriptions Cost of Unused 

and Partially 

Used ($) 

Task 

Order Expired Operational Purchased Unused Partially Used 

1st 1/26/15 7/13/15 67,360 67,360 N/A 1,570,835

2nd 5/31/16 67,630 N/A 67,630 164,046

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

the 206,083 unused subscriptions. These subscriptions had an initial 1-year 

duration and were to expire in September 2014, even though CBP and the 

vendor extended the expiration date to January 2015; the subscriptions 

expired without being used because PALMS was not operational by that date. 

To further illustrate the waste in partially used subscriptions, CBP paid $1.4 

million in June 2015 for an additional 67,630 subscriptions that it could not 

use until PALMS became operational in July 2015. Because CBP could not use 

the subscriptions from June 1 to July 12, 2015, CBP wasted $164,046 for the 

additional set of subscriptions purchased. 

Table 5: CBP Unused and Partially Used Subscriptions 

Source: OIG analysis of CBP task orders and implementation schedule. 

According to GAO’s Standards for Internal Control,4 “waste is the act of using or 

expending resources carelessly, extravagantly, or to no purpose.” DHS 

expended funds carelessly or to no purpose because the Department 

purchased subscriptions before the system became operational and DHS users 

could not use them for learning and performance management. As a result, 

DHS wasted more than $5.7 million for PALMS subscriptions. See Appendix D 

for a summary of subscriptions the DHS purchased before PALMS became 

operational. 

Recommendations 

Recommendation 6: We recommend that the DHS Chief Information Officer 

develop guidelines to ensure information technology contracts include clear 

definition of subscription services, pricing methodologies, and payment and 

acceptance terms. 

Recommendation 7: We recommend that the DHS Chief Information Officer 

put controls in place to ensure future task orders for PALMS subscriptions 

 4 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G, September 

www.oig.dhs.gov 13 OIG-17-91 

2014 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

include consumption-based pricing for active system users as described in the 

PALMS BPA. 

DHS Comments and OIG Analysis 

DHS Comments to Recommendation 6: The Department concurred with our 

recommendation and stated that the DHS CIO will work with the DHS OCPO to 

develop guidelines to ensure information technology contracts include clear 

definition of subscription services, pricing, methodologies, and payment and 

acceptance terms. ECD: June 30, 2017. 

OIG Analysis of DHS Comments: We consider DHS’ proposed corrective 

action to be responsive to the recommendation. The recommendation is 

considered open and resolved. It will remain open until we receive and analyze 

the Department’s guidelines to ensure information technology contracts 

include clear definition of subscription services, pricing, methodologies, and 

payment and acceptance terms. 

DHS Comments to Recommendation 7: The Department concurred with 

recommendation 7 and stated that the current controls in place are for DHS 

components to review its user accounts prior to funding and issue an order for 

subscriptions against the Enterprise Talents Management System (ETMS) 

ETMS/PALMS BPA. The BPA term states that DHS is to only pay for active (or 

current) system users on annual basis. DHS Components are to apply the 

appropriate tier rate (in accordance with BPA Pricing) based upon the 

cumulative amount of users. This will be emphasized in the update to the 

ordering guide. ECD: May 31, 2017. 

OIG Analysis of DHS Comments: DHS’ proposed corrective action does not 

fully address the recommendation. The terms in the BPA or emphasis in the 

ordering guide may not serve as a control that would ensure future task orders 

for PALMS subscriptions include consumption-based pricing based on active 

system users. We consider this recommendation unresolved and open until 

DHS implements controls that would ensure future task orders for PALMS 

subscriptions include consumption-based pricing based on active system 

users. The estimated completion date has passed. We will track the status until 

action is taken for the recommendation. 

www.oig.dhs.gov 14 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix A 

Objective, Scope, and Methodology 

The DHS Office of Inspector General was established by the Homeland Security 

Act of 2002 (Public Law 107ï296) by amendment to the Inspector General Act of 

1978. 

We conducted this audit to determine whether the PALMS acquisition 

addressed the Department’s critical capability need for an integrated 

department-wide learning and performance management system. To achieve 

our audit objective, we interviewed officials in the DHS Directorate for 

Management, Coast Guard, CBP, TSA, USCIS, FEMA, ICE, Secret Service, and 

the Federal Law Enforcement Training Center. We also obtained and reviewed 

public laws, DHS directives, congressional budget requests, contract 

documents, project management documents, and financial documents. 

To determine whether PALMS met the Department’s needs, we reviewed 

PALMS’ 2012 Business Case; acquisition documents; 2013 blanket purchase 

agreement and 2013–2016 task orders; Requirements Traceability Matrix as of 

February 2016; User Acceptance Testing results and briefs; Human Resources 

Information Technology Executive Steering Committee meeting briefs and 

minutes; and components’ business cases. We obtained and analyzed data 

related to September 2014 QASP, the contractor performance, and contractor 

invoices to determine the financial credit amount. 

To determine the amount of additional costs for legacy systems extensions, we 

examined contracts in effect from September 2013 to December 2016 and 

performed analysis based on the DHS schedule for PALMS implementation. To 

determine the funds expended for subscriptions before the system was 

operational, we evaluated the subscription task orders issued from August 

2013 through September 2016 and implementation schedule. We calculated it 

based on the subscription beginning date and the date the system became 

operational. From this we identified funds that were wasted between August 

2013 and November 2016. 

We assessed the controls related to the acquisition of PALMS. We conducted a 

limited assessment of the PALMS SELC plan and the use of the agile 

methodology for compliance with the DHS policies and procedures. Our limited 

assessment would not necessarily disclose all material weaknesses in this 

control structure. However, our assessment disclosed weaknesses in the 

oversight of this contract, which are discussed in the body of the report. 

www.oig.dhs.gov 15 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

We did not rely on computer-processed data to materially support findings, 

conclusions, or recommendations in this report. 

We conducted this performance audit between October 2015 and October 2016 

pursuant to the Inspector General Act of 1978, as amended, and according to 

generally accepted government auditing standards. Those standards require 

that we plan and perform the audit to obtain sufficient, appropriate evidence to 

provide a reasonable basis for our findings and conclusions based upon our 

audit objectives. We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based upon our audit objectives. 

www.oig.dhs.gov 16 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix B 

DHS Comments to the Draft Report 

www.oig.dhs.gov 17 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 18 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 19 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 20 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix C 

PALMS QASP Performance Measures 

Service Level Measure Performance Standard 

Program Management Performance Measures 

Risk Management Risks are managed effectively. 

Status Reporting Status reports are delivered on schedule. 

Implementation and Integration Performance Measures 

Training Students are satisfied with the training. 

Data Conversion Existing ETMS data is converted correctly. 

Migration Schedule Migration is completed in accordance with 

approved schedule. 

Implementation Migration is completed on schedule. 

Production Service Delivery Performance Measures 

Production Service Availability Each environment (e.g., development, test, 

training, and production) is available to 

perform functions as designed. 

Production Service Response The production system provides timely 

response to user requests. 

Security Operations Security audit findings are resolved in a 

timely manner so the service maintains 

Authority to Operate. 

Firewall Management Compliance with firewall configuration and 

change management processes. 

Physical and Information Security Proactive physical security. 

Time to Resolve Help Desk Tickets Responsive level 3 support service is provided. 

Help Desk Response Help Desk calls answered promptly. 

Return to Operations (RTO) Return to Operations in the event of disaster. 

Production Service Availability Environment is completed with subscription 

terms. 

Source: DHS headquarters, PALMS BPA QASP (September 10, 2014) 

ETMS – enterprise talent management system 

www.oig.dhs.gov 21 OIG-17-91 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix D 

Subscriptions Purchased Before PALMS Became Operational

 Task Orders for 

Subscriptions 

Number 

Purchased 

Unused 

Subscriptions 

Partially Used 

Subscriptions

Subscription 

Start Date 

Subscription 

Expiration Date 

System 

Operational Date 

 Total Cost of 

Subscriptions ($) 

Cost of Unused and 

Partially Used ($)

DHS HQ 1st 41,500 41,500 -- 8/30/2013 1/26/2015 10/6/2015 967,780 967,780 

DHS HQ 2nd 41,500 25,500 16,000 6/1/2015 5/31/2016 874,820 654,895 

CBP 1st 67,360 67,360 -- 9/30/2013 1/26/2015 7/13/2015 1,570,835 1,570,835 

CBP 2nd 67,630 -- 67,630 6/1/2015 5/31/2016 1,425,640 164,046 

USCIS 1st 21,245 21,245 -- 6/1/2015 5/31/2016 9/30/2016 447,845 447,845 

USCIS 2nd 21,245 -- 21,245 6/1/2016 5/31/2017 569,578 188,819 

FLETC 1st 2,000 2,000 -- 1/15/2014 1/15/2015 12/10/2015 46,640 46,640 

FLETC 2nd 2,000 -- 2,000 6/1/2015 5/31/2016 42,160 22,177 

USSS 1st 6,500 6,500 -- 9/9/2015 9/8/2016 11/30/2016 137,020 137,020 

USSS 2nd 7,200 -- 7,200 9/9/2016 9/8/2017 193,032 43,366 

ICE 1st 23,000 23,000 -- 6/1/2015 5/31/2016 6/1/2016 484,840 484,840 

USCG 44,478 44,478 -- 9/28/2013 2/28/2015 N/A 1,037,227 1,037,227 

Grand Total 345,658 231,583 114,075 $7,797,417 $5,765,490 

Source: OIG analysis of PALMS task orders and implementation schedule. 

www.oig.dhs.gov 22 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix E 

Office of Audits Major Contributors to This Report 

Carolyn Hicks, Director 

Brooke Bebow, Director 

LaParacina Williams, Audit Manager 

Johnson Joseph, Lead Auditor 

LaTrina McCowin, Auditor 

Enrique Leal, Auditor 

David Porter, Auditor 

Beth Windisch, Program Analyst 

Kevin Dolloson, Communications Analyst 

Megan McNulty, Independent Referencer 

www.oig.dhs.gov 23 OIG-17-91 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix F 

Report Distribution 

Department of Homeland Security 

Secretary 

Deputy Secretary 

Chief of Staff 

General Counsel 

Executive Secretary 

Director, GAO/OIG Liaison Office 

Assistant Secretary for Office of Policy 

Assistant Secretary for Office of Public Affairs 

Assistant Secretary for Office of Legislative Affairs 

Office of Management and Budget 

Chief, Homeland Security Branch 

DHS OIG Budget Examiner 

Congress 

Congressional Oversight and Appropriations Committees 

www.oig.dhs.gov 24 OIG-17-91 
 

 

 

 

 

 

 

ADDITIONAL INFORMATION AND COPIES 

To view this and any of our other reports, please visit our website at: www.oig.dhs.gov. 

For further information or questions, please contact Office of Inspector General Public Affairs 

at: DHS-OIG.OfficePublicAffairs@oig.dhs.gov. Follow us on Twitter at: @dhsoig. 

OIG HOTLINE 

To report fraud, waste, or abuse, visit our website at www.oig.dhs.gov and click on the red 

"Hotline" tab. If you cannot access our website, call our hotline at (800) 323-8603, fax our 

hotline at (202) 254-4297, or write to us at: 

Department of Homeland Security 

Office of Inspector General, Mail Stop 0305 

Attention: Hotline 

245 Murray Drive, SW 

Washington, DC 20528-0305