CBP Continues to Improve 

its Ethics and Integrity 

Training, but Further 

Improvements are 

Needed 

May 31, 2017 

OIG-17-60 
 

 

 

 

 

 

DHS OIG Highlights

CBP Continues to Improve Its Ethics and Integrity

Training, but Further Improvements are Needed
 

May 31, 2017 

Why We Did 

This Audit 

In House Report 114–215 

(July 21, 2015), which 

accompanied the 

Consolidated Appropriations 

Act, 2016, Public Law No. 

114–113, the U.S. House of 

Representatives, Committee 

on Appropriations, directed 

the Office of Inspector 

General to assess and report 

on CBP’s ethics and integrity 

training for its agents and 

officers [H.R. Rep. No.114– 

215, at 22 (2015)]. We 

conducted this audit to 

determine whether CBP made 

training improvements; 

tracked training completion 

and effectiveness; and 

communicated in a 

comprehensive manner its 

integrity strategy. 

What We 

Recommend 

We made two 

recommendations for CBP to 

improve its oversight and 

management of its ethics and 

integrity training. 

For Further Information: 

Contact our Office of Public Affairs at 

(202) 254-4100, or email us at 

DHS-OIG.OfficePublicAffairs@oig.dhs.gov 

What We Found 

U.S. Customs and Border Protection (CBP) has made 

improvements to, and continues to develop, its ethics and 

integrity training for officers and agents. It tracks training 

completion, and has begun to measure and assess training 

effectiveness. 

However, the Performance and Learning Management 

System used to track training completion needs 

improvement. Also, locally developed training content on 

ethics and integrity varies by location and operating 

environment, and CBP does not maintain a repository or 

any formal process for the field to share locally developed 

information. As a result, CBP misses valuable opportunities 

to deliver consistent high-quality ethics and integrity 

training courses across multiple operating environments 

and components. 

Finally, CBP has not effectively communicated or followed 

up with the field on its overall Integrity and Personal 

Accountability Strategy. One purpose of the strategy is to 

ensure that ethics and integrity training is provided for all 

CBP employees. More broadly, the strategy aims to promote 

a culture of integrity and accountability by increasing 

awareness through messaging, training, and enhanced 

communication. If employees have not received or do not 

understand the importance of the integrity strategy, CBP 

cannot succeed in achieving this important initiative. 

CBP Response 

CBP concurred with the two recommendations and has 

initiated actions that should improve its ethics and integrity 

training. We consider recommendation 1 resolved and open 

and recommendation 2 resolved and closed. 

www.oig.dhs.gov OIG-17-60 
OFFICE OF INSPECTOR GENERAL 

Department of Homeland Securlty 

Washington, DC 20528 / www.oig.dhs.gov 

May 31, 2017 

MEMORANDUM FOR: 	 Kevin K. McAleenan 

Acting Commissioner 

U.S. Customs and Border Protection 

Christopher Hall 

Assistant Commissioner 

Office of Training and Development 

U.S. Customs and Border Protection 

FROM: John Roth ~~~~ Inspector General 

SUBJECT: CBP Continues to Improve its Ethics and Integrity 

Training, but Furlher Improvements are Needed 

Attached for your action is our final report, CBP Continues to Improve its Ethics 

and Integrity Training, but Furlher Improvements are Needed. We incorporated 

the formal comments provided by your office. 

The report contains two recommendations aimed at improving CBP's ability to 

more effectively oversee and manage its ethics and integrity training program 

and foster a workforce culture of integrity. Your office concurred with the 

recommendations. Based on information provided in the response to the draft 

report, we consider recommendation 1 resolved and open and recommendation 

2 is resolved and closed. Once your office has fully implemented the 

recommendation, please submit a formal closeout letter to us within 30 days so 

that we may close the recommendation. The memorandum should be 

accompanied by evidence of completion of agreed-upon corrective actions. 

Please send your response or closure request to 

OIGAuditsFollowup@oig.dhs.gov. 

Consistent with our responsibility under the Inspector General Act, we will 

provide copies of our report to congressional committees with oversight and 

appropriation responsibility over the Department of Homeland Security. We will 

post the report on our website for public dissemination. 

Please call me with any questions, or your staff may contact Maureen Duddy, 

Deputy Assistant Inspector General for Audits, at (202) 254-4100. 

Attachment 
 

 

 

 

 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Table of Contents 

Background .................................................................................................... 1 

Results of Audit ............................................................................................. 3 

CBP Could Share Local Best Practices More Effectively and 

Emphasize Its Integrity Strategy ........................................................... 4 

The Performance and Learning Management System for Tracking 

Training Completion Needs Improvement ............................................... 6 

CBP Has Begun to Measure and Assess Training Effectiveness and 

Apply Evaluation Data .......................................................................... 7 

Conclusion...................................................................................................... 9 

Recommendations......................................................................................... 10 

Appendixes 

Appendix A: Objective, Scope, and Methodology ................................. 12 

Appendix B: CBP Comments to the Draft Report .................................. 16 

Appendix C: Ethics and Integrity Training Improvements 

 Since FY 2012 ................................................................. 19 

Appendix D: The Kirkpatrick Model Levels of Measurement, Benefits, 

 and Limitations .............................................................. 21 

Appendix E: Office of Audits Major Contributors to This Report .......... 23 

Appendix F: Report Distribution .......................................................... 24 

Abbreviations 

AMO Air and Marine Operations 

ATC Advanced Training Center 

CBP U.S. Customs and Border Protection 

CFR Code of Federal Regulations 

DUI Driving Under the Influence 

GAO Government Accountability Office 

HQ Headquarters 

OCC Office of Chief Counsel 

OFO Office of Field Operations 

OIG Office of Inspector General 

www.oig.dhs.gov OIG-17-60 
 

 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

OPR Office of Professional Responsibility 

 (formerly Internal Affairs) 

OTD Office of Training and Development 

PALMS Performance and Learning Management System 

P.L. Public Law 

SIP Self-Inspection Program 

SLCP Second Level Command Preparation 

SOP Standard Operating Procedure 

TRAEN Training, Records, and Enrollment Network 

USBP U.S. Border Patrol 

www.oig.dhs.gov OIG-17-60 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Background 

U.S. Customs and Border Protection (CBP) is one of the Department of 

Homeland Security’s largest and, according to CBP, one of the most complex 

components, with a priority mission of keeping terrorists and their weapons 

out of the United States. To add to its complexity, CBP is also responsible for 

securing the border — approximately 7,000 miles of land borders and 95,000 

miles of coastline — and facilitating lawful international trade and travel. The 

agency takes a comprehensive approach to border management and control, 

combining customs, immigration, border security, and agricultural protection 

into one coordinated and supportive activity. 

CBP recognizes that its employees operate in an environment in which even a 

single corrupt act may pose a threat to national security; thus, a culture of 

integrity and a strong, aggressive program to address corruption and 

misconduct are vital to the agency and to the Nation. In 2011, the Homeland 

Security Studies and Analysis Institute completed a review of CBP’s workforce 

integrity. The Institute found that CBP did not have comprehensive guidance 

for its integrity programs and initiatives. One year later, the Government 

Accountability Office (GAO) GAO Report-13-59, Additional Actions Needed to 

Strengthen CBP Efforts to Mitigate Risk of Employee Corruption and Misconduct, 

similarly concluded that CBP did not have an integrity strategy as required in 

the Secure Borders, Safe Travel, Legal Trade: U.S. Custom’s and Border 

Protection Fiscal Year 2009–2014 Strategic Plan. 

In light of these reports, CBP began drafting an Integrity and Personal 

Accountability Strategy (integrity strategy), which was formally announced in 

September 2014. That same month the DHS Secretary delegated criminal 

misconduct investigative authority to CBP as part of what the former 

Commissioner called “a larger effort to hold the workforce accountable for 

maintaining a high standard of integrity and aligning CBP with law 

enforcement best practices throughout the country.” 

CBP officials report that the agency’s integrity strategy establishes a unified 

and multi-layered approach organized around four related mission areas: 

prevention, detection, investigation, and response to corruption and 

misconduct. The strategy also addresses cross-cutting issues of integration and 

awareness. In a cover memo, CBP’s Commissioner defines personal integrity as 

“a commitment that each of us makes to doing the right things, the right way, 

at the right time, including the reporting of inappropriate, corrupt, or criminal 

actions when observed.” The integrity mission is to “strengthen CBP’s culture 

of unwavering integrity and professionalism by increasing awareness and 

implementing comprehensive, integrated programs and capabilities designed to 

www.oig.dhs.gov 1 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

prevent, detect, investigate, and respond to instances of corruption and 

misconduct across all levels in the agency.” The implementation process 

requires measuring, analyzing, and ensuring CBP’s collective actions reinforce 

integrity as a core value. 

Within CBP, the Office of Training and Development (OTD) is responsible for 

centralized leadership and direction of all training programs, as well as the 

continuous improvement and expansion of CBP learning capabilities. This 

includes training at the: 

x Advanced Training Center — a CBP training center that houses the 

Instructional Design Center, Distance Learning Center, and The Law 

Enforcement Officer/Agent Safety and Compliance Directorate; formerly, 

the Use of Force Center of Excellence; 

x Pre-Academy — training at duty stations prior to the academy; 

x Basic Academy — the Boot Camp for officers and agents; and 

x Post-Academy — on-the-job training. 

CBP officials said that field training is managed by individual field locations 

and aligns with each unique operating environment. 

The Office of Chief Counsel (OCC) provides mandatory ethics training to select 

employees subject to financial disclosure rules, and all CBP employees are 

required to take an online Integrity Awareness course annually. Instead of 

routinely offering standalone courses in ethics and integrity, CBP typically 

includes this type of training in more comprehensive courses. 

As an illustration of ongoing congressional interest in this area, House Report 

114–215 explaining the fiscal year appropriations for the Department of 

Homeland Security, the U.S. House of Representatives, Committee on 

Appropriations, directed the Office of Inspector General to “assess and report 

on CBP’s ethics and integrity training for its agents and officers, as well as CBP 

mechanisms for operational oversight related to ethics and integrity” [H.R. Rep. 

No. 114-215, at 22 (2015)]. Because of the broad and undefined nature of this 

mandate, we corresponded with congressional staff who acknowledged the 

steps CBP had taken to address concerns regarding allegations of misconduct 

associated with CBP’s rapid hiring of border patrol agents. Those steps 

included improving training and assuming criminal investigative authority from 

U.S. Immigration and Customs Enforcement for allegations of CBP misconduct. 

Nevertheless, concerns remained, and OIG was tasked with determining how 

training has improved, and how CBP is tracking whether training is effective, 

as well as providing rigorous oversight with regard to allegations of personnel 

conduct. As a result, we evaluated ongoing OIG projects and found another 

www.oig.dhs.gov 2 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

audit team was looking at whether conduct and discipline is equitable and 

consistent department-wide. Although the DHS Conduct and Discipline audit 

will not directly answer whether CBP is providing rigorous oversight with 

regard to allegations of personnel misconduct, it will include a high-level 

overview of CBP’s process for reporting and addressing misconduct, as well as 

some limited testing of CBP’s disciplinary action cases. We then revised our 

objective to focus on whether CBP made training improvements; tracked and 

measured training completion and effectiveness; and communicated in a 

comprehensive manner its integrity strategy. 

Results of Audit 

CBP has made improvements to, and continues to develop, its ethics and 

integrity training for officers and agents. The agency tracks training completion 

using various methods and evaluation tools to measure and assess training 

effectiveness. As a result, CBP has created, expanded, and redesigned training 

courses to promote ethics and integrity at the Advanced Training Center (ATC), 

the academies, and in the field. However, CBP has not effectively 

communicated or conducted follow-up with the field on its overall integrity 

strategy. As a result, CBP cannot fully accomplish the goals of this important 

strategy. 

CBP mandates annual integrity training for all employees. CBP is developing 

ethics training for all employees, which is expected to become operational on 

Performance and Learning Management System (PALMS) during calendar year 

2017. It develops and delivers instruction on these topics in multiple academy 

and ATC modules. Locally developed training on ethics and integrity varies by 

location and operating environment, and CBP does not maintain a repository or 

any formal process for the field to share locally developed information. 

Consequently, CBP misses valuable opportunities to deliver consistent, highquality ethics and integrity training courses across multiple operating 

environments and components. 

CBP tracks and assesses training completion using PALMS for all agency-wide 

mandatory training. In addition, academies use ACADIS — an electronic 

personnel and training system — to store and track academy training-related 

data. Nonetheless, there are concerns over PALMS reporting functionality and 

accuracy. Having reliable data is essential for CBP to meet fully its reporting 

requirements and oversee its training program. 

CBP uses various evaluation tools to measure and assess training 

effectiveness. For example, OTD began formally using the Kirkpatrick Model in 

late 2015 to evaluate its advanced training and academy courses, while 

www.oig.dhs.gov 3 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

continuing to rely on End-of-Course Evaluation forms to obtain training 

participant feedback in the field. According to the DHS Learning Evaluation 

Guide, dated April 2016, although it is not the only model to evaluate training, 

the Kirkpatrick Model is well suited for DHS needs. Evaluating training 

effectiveness is an ongoing process. 

Finally, CBP has not effectively communicated or followed up with the field on 

its overall integrity strategy. One purpose of the strategy is to ensure that 

ethics and integrity training is provided for all CBP employees. More broadly, 

the strategy aims to promote a culture of integrity and accountability by 

increasing awareness through messaging, training, and effective 

communication. If employees have not received or do not understand the 

importance of the integrity strategy, CBP cannot succeed in achieving this 

important initiative. 

CBP Could Share Local Best Practices More Effectively and 

Emphasize Its Integrity Strategy 

CBP has made improvements in the development and delivery of ethics and 

integrity training. The agency has introduced new courses and programs for its 

employees, its academies, and its field locations. However, it could share local 

best practices more effectively. CBP developed an integrity strategy to promote 

a culture of integrity and accountability by increasing awareness through 

messaging, training, and effective communication. Yet, CBP has not effectively 

communicated or followed up with the field on its overall integrity strategy. 

Training Improvements and Best Practices 

CBP policy requires the component to maintain a workforce that demonstrates 

high standards of ethical and professional conduct to ensure the proper 

performance of government service. One purpose of CBP’s integrity strategy is 

to ensure that ethics and integrity training is provided for all CBP employees. 

CBP has taken notable steps to improve the quality and availability of ethics 

and integrity training. Instruction on this topic is included in multiple courses 

offered to employees, supervisors, and managers; in-person at the ATC; and 

through various training methods at CBP’s academies and field locations (see 

appendix C). Another improvement is OCC’s planned fiscal year 2017 

implementation of its annual and recurring ethics training for all of its 

employees, which is offered on-line through PALMS. 

OTD works with the Office of Field Operation (OFO), U.S. Border Patrol (USBP), 

and Air and Marine Operations (AMO) academies to develop the pre-, basic, 

www.oig.dhs.gov 4 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

and post-academy training, including ethics and integrity elements. According 

to CBP officials, the training program at the academies has been improved 

significantly through scenario-based training. 

We also found and observed many examples of high-quality training developed 

in the field. This training was not shared throughout CBP or with other field 

locations because CBP does not have a repository or any formal process to 

facilitate that result. The OFO field offices, USBP sectors, and AMO units 

(collectively, “the field”) have autonomy to develop training unique to their 

locations, missions, and operating conditions. According to field officials, they 

coordinate with their local OCC representatives to develop and deliver ethics 

training sessions. 

Specifically, when we visited the USBP at Laredo Sector, personnel reported 

working with OCC to add an ethics training module in 2016 to its week-long 

annual training. OCC officials said that the ethics class, depending on the 

number of agents’ questions leads to valuable integrity-related discussions. 

Aspects of the annual training include mandatory training and others, such as 

fraud document detection; an Intel session; relevant policies; first aid/CPR; 

financial literacy; and weapons and agent specific re-certification. 

During the Tucson field visit, we found best practices at OFO, USBP, and AMO. 

For example, the USBP at Tucson Sector developed the following programs and 

initiatives to address its unique operating environment and ethics and integrity 

issues: 

x A Tactical Awareness Training Program now part of basic academy training 

was developed after two agents were assaulted and did not have the 

necessary skills to defend themselves. This program conveys agents’ rights 

and responsibilities pertaining to use of deadly force and threat 

assessments. It also covers mindset, agent preparedness, handling 

situations appropriately, security, and situational awareness. 

x The Driving-Under-the-Influence (DUI) initiative was developed in Tucson 

Sector after Agent Michael V. Gallagher was killed by a drunk driver. The 

associated mandatory DUI training touches all agents and focuses on 

making sure employees understand the impact of their decisions, on and off 

duty. This mandatory DUI training is conducted nationwide. 

x The Report Writing and Courtroom Testimony course was created by OCC at 

Tucson Sector to improve agent abilities in both writing and testifying. This 

training helps agents convey evidence-based and corroborated facts, and 

www.oig.dhs.gov 5 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

prepares them for cross examination and credible testimony. This Report 

Writing and Courtroom Testimony training is offered nationwide. 

Although there are several options for CBP employees to obtain ethics and 

integrity training, the agency does not have a repository or any formal process 

for the field to share locally developed training for use agency-wide. As a result, 

CBP misses valuable opportunities to deliver consistent component-wide ethics 

and integrity training. A CBP training official said sharing locally developed 

training through a course catalogue for all components would be beneficial. 

CBP’s Integrity Strategy 

Throughout our site visits, senior leaders and training officers at Laredo’s 

USBP, AMO, and OFO field locations stated that they were unaware of the 

integrity strategy. Additionally, officials at the USBP, AMO, and OFO academies 

also had not heard of this document. 

CBP’s integrity strategy outlines the responsibility of every person in CBP to 

internalize and promote a culture of integrity. It calls for an implementation 

process to measure and analyze the collective reinforcement of integrity as a 

core value and the need for awareness through integrity messaging, training, 

and effective communication. According to Office of Policy and Planning 

officials, senior leaders in the field should have communicated the integrity 

strategy’s goals and objectives to their respective field locations. Without 

headquarters (HQ) officials effectively communicating the strategy and having 

meaningful follow-up with senior field leaders, CBP cannot ensure successful 

implementation of the integrity strategy. 

The Performance and Learning Management System for 

Tracking Training Completion Needs Improvement 

CBP uses multiple processes and systems to track ethics and integrity training. 

Directors and program managers at HQ and field offices self-report the 

completion of all mandatory training through CBP’s Self-Inspection Program 

(SIP). CBP’s Management Inspection Division is responsible for validating 

annual SIP reports. 

Additionally, CBP uses PALMS to track completion of its online CBP-wide 

annual ethics and integrity awareness training.1 CBP also uses training 

attendance sign-in sheets, certificates of completion, manual files, 

1 Prior to June 2015, CBP used Training Records and Enrollment Network System (TRAEN) to 

track training completion. 

www.oig.dhs.gov 6 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

spreadsheets, and post-academy files to track its locally developed field 

training. Academies also use ACADIS — an electronic personnel and training 

system — to store and track academy training-related data. According to DHS, 

CBP HQ, and field officials, in 2015 and 2016 PALMS was neither accurate nor 

was the reporting module fully operational. These deficiencies led CBP’s SIP 

division to waive training reporting for fiscal year 2015. 

We observed CBP officials at the OTD in May 2016 unsuccessfully attempt to 

generate CBP reports of completed and overdue training, noting PALMS error 

messages and inaccurate totals. During field visits in July and August of that 

same year at Laredo and Tucson, we observed, and training officials said, they 

could not run accurate reports of completed and overdue training. Five months 

later in October 2016, the PALMS vendor developed a temporary solution to 

produce accurate CBP HQ level reports. However, field office training personnel 

remain unable to generate accurate training reports through PALMS. CBP HQ 

officials said the vendor will implement a solution to this problem sometime in 

2017. 

Without consistent and accurate PALMS reporting, CBP: 

x	 duplicates efforts to run reports and ensure training completion; 

x	 cannot determine who received, completed, and needs to attend training 

— meaning staff may not receive critical training to complete their job 

functions and mission requirements; and 

x	 cannot satisfy mandated reporting requirements and plan for future 

training needs. 

Having reliable data is essential for CBP to properly manage its training 

program. A DHS OIG review of the implementation and functionality of PALMS 

is ongoing in a separate audit, with an anticipated final report in fiscal year 

2017. As a result, we do not make a recommendation to address this issue in 

this report. 

CBP Has Begun to Measure and Assess Training Effectiveness 

and Apply Evaluation Data 

Although CBP does not measure the effectiveness of every individual training 

course, it does have an overarching methodology to evaluate its training 

program. For example, OTD and academies use the Kirkpatrick Model to 

measure the overall effectiveness of their own training courses; and the field 

uses OTD’s End-of-Course Evaluation form. The latter operates in the same 

way as Kirkpatrick level 1 to obtain training participant feedback. This 

distinction is due to OTD’s decision to formally adopt the Kirkpatrick Model in 

www.oig.dhs.gov 7 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

late 2015 as its tool for evaluating OTD advanced courses. Based on evaluation 

feedback from Kirkpatrick and OTD’s End-of-Course Evaluation forms, CBP 

has created, expanded, and redesigned training courses to promote ethics and 

integrity at OTD, the academies, and in the field. 

Federal regulations require agencies to evaluate their training programs at 

least annually to determine how well they contribute to mission 

accomplishment and meet organizational performance goals.2 DHS defines 

training evaluation as a process used to measure the value and effectiveness of 

training. The Kirkpatrick Model for evaluating training effectiveness is an 

industry standard. The April 2016 DHS Learning Evaluation Guide makes note 

that, although the Kirkpatrick Model is not the only model to evaluate training, 

it is well suited for DHS needs. 

The Kirkpatrick Model provides a framework for organizations to measure how 

well training provides employees with the necessary skills, knowledge, and 

abilities to successfully accomplish their jobs and support the organization’s 

mission and goals. This framework contains four different levels of evaluation 

used to measure the effectiveness of training. Table 1 highlights the four 

evaluation levels outlined in the DHS Learning Evaluation Guide. See appendix 

D for a detailed explanation of the evaluation levels, as well as the model’s 

benefits and limitations. 

Table 1: Kirkpatrick’s Four Levels of Evaluation 

Measurement Results Feedback 

Methods 

Level 1-

Reaction 

To what degree trainees 

react favorably to the 

learning event 

Negative reactions can 

indicate potential 

shortcomings with a 

training course 

Surveys, 

evaluations, and 

questionnaires 

Level 2-

Learning 

To what degree trainees 

acquire intended 

knowledge/skills from the 

learning event 

Student performs 

required task while in 

the training environment 

Knowledge test 

and skill 

demonstration 

Level 3-

Behavior 

To what degree trainees 

apply what they learned 

during training back on 

the job 

Transfer of learning from 

training to work 

environment, identifying 

barriers 

Surveys, 

observations, 

and interviews 

Level 4-

Results 

To what degree targeted 

outcomes occur, as a 

result of the training and 

reinforcement 

Return on investment, 

residing at executive or 

training director level for 

analysis 

Questionnaires, 

evaluations, and 

surveys 

Source: OIG analysis of the DHS Learning Evaluation Guide, dated April 2016 

2 Title 5 Code of Federal Regulations (CFR) § 410.202 Planning and Evaluating Training 

www.oig.dhs.gov 8 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Since OTD issued its Standard Operating Procedure (SOP) in November 2015, 

officials said they tested the Kirkpatrick Model levels 1 through 3 to determine 

the most appropriate use of the model for each OTD-developed course. OTD 

provided OIG with an evaluation plan for the Second Level Command 

Preparation (SLCP) course that detailed the entire training evaluation. The 

SLCP plan specifies the type of information that needs to be collected; the 

process for collecting the data; roles and responsibilities of CBP officials; and 

how to use the data collected to evaluate training courses and make 

improvements. The OTD official said each OTD-delivered course has a unique 

evaluation plan in draft or final format for implementation in fiscal year 2017. 

The USBP, OFO, and AMO academies use the Kirkpatrick Model to measure 

training program effectiveness and currently use the ACADIS system to 

manage the results. Level 3 feedback is reviewed to improve curriculum using 

comprehensive reports including data provided by participants and their 

supervisors or training officers. Participant feedback is collected in the End-ofCourse Evaluation form for field developed and locally delivered training to 

measure training effectiveness and assist in future course development. 

Finally, at the Laredo and Tucson field offices, USBP officials said they meet 

regularly and review training feedback to enhance existing and future 

instruction, like the Leadership and Supervisors training in Tucson and the 1-

hour ethics training in Laredo. 

As a result of CBP’s evaluation tools and data collection, OTD has expanded its 

course development and improved the manner of delivery for its 2016 Revised 

Senior Leadership and other training. All academies have enhanced ethicsrelated training modules and integrated more scenario-based training for 

officers and agents. These trainings as listed in appendix C include Basic 

Training Program Redesign Academy; Post-Academy Training Redesign; 

Enhanced Scenario Based Training; and OCC-Nationality and Immigration Law 

and Applied Authorities. As CBP officials continue to assess evaluation results 

and the best use of the Kirkpatrick Model, they will be able to make more 

informed decisions on training improvements. 

Conclusion 

CBP has made improvements to, and continues to develop, its ethics and 

integrity training for officers and agents. It tracks training completion using 

various methods and evaluation tools to measure and assess training 

effectiveness. This has led to the creation, expansion, and redesign of courses 

to promote ethics and integrity at the Advanced Training Center, the 

academies, and in the field. 

www.oig.dhs.gov 9 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

However, further improvements are needed. CBP does not maintain a 

repository or any formal process for the field to share high-quality, locally 

developed training. Nor has the agency effectively communicated to, or followed 

up with, the field regarding implementation of its overall integrity strategy. 

Correcting these deficiencies will enhance CBP’s ability to more effectively 

oversee and manage its ethics and integrity training program and foster a 

workforce culture of integrity. 

Recommendations 

Recommendation 1: We recommend that the Assistant Commissioner for 

Training create a repository for the field offices to maintain and share unique 

field-developed training. 

Recommendation 2: We recommend that the Commissioner highlight the 

importance of CBP’s integrity strategy throughout the agency and follow up 

with field staff to ensure it is effectively communicated. 

Management Comments and OIG Analysis 

Below is a summary of CBP’s comments to the draft report and the OIG 

analysis. Verbatim comments are in appendix B. 

Recommendation 1 

CBP Comments: Concur. CBP's Office of Training and Development (OTD) 

Distance Learning Center (DLC) and the National Training Plan (NTP) will host 

a meeting with the CBP operational components of U.S. Border Patrol, Office of 

Field Operations, and Air and Marine Operations to coordinate and develop a 

repository to share locally developed training and best practices. Estimated 

Completion Date (ECD): September 30, 2017 

OIG Analysis: CBP’s planned action is responsive to the intent of the 

recommendation and recommendation #1 is considered resolved and open. OIG 

will close the recommendation when CBP provides evidence of fully 

implementing the creation of a repository to share unique field-developed 

training. 

Recommendation 2 

CBP Comments: Concur. On December 22, 2016, CBP’s Commissioner issued 

a message to all CBP employees redistributing CBP's Integrity and Personal 

Accountability Strategy, and reemphasized the importance of the strategy, 

www.oig.dhs.gov 10 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

which outlines the responsibility of every employee — including CBP's 

leadership — to foster a culture of integrity. CBP is confident that it has met 

the intent of the recommendation and respectfully requests closure. 

OIG Analysis: CBP’s action is responsive to the intent of the recommendation 

and recommendation #2 is resolved and closed. 

www.oig.dhs.gov 11 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix A 

Objective, Scope, and Methodology 

The Department of Homeland Security Office of Inspector General was 

established by the Homeland Security Act of 2002 (Public Law 107−296) by 

amendment to the Inspector General Act of 1978. 

House Report 114-215, which accompanied Consolidated Appropriations Act, 

2016, Public Law (P.L.) No. 114–113, directed OIG to “assess and report on 

CBP’s ethics and integrity training for its agents and officers, as well as CBP 

mechanisms for operational oversight related to ethics and integrity” [H. R. 

Rep. No. 114-215, at 22 (2015)]. Because of the broad and undefined nature of 

this mandate, we corresponded with congressional staff who acknowledged the 

steps CBP had taken to address concerns regarding allegations of misconduct 

associated with CBP’s rapid hiring of border patrol agents. Those improvements 

included improving training and assuming criminal investigative authority from 

U.S. Immigration and Customs Enforcement for allegations of CBP misconduct. 

However, concerns remained, and OIG was tasked with determining how 

training has improved and how CBP is tracking whether training is effective, as 

well as providing rigorous oversight with regard to allegations of personnel 

conduct. 

As a result, we evaluated ongoing OIG projects and found another audit team 

was looking at whether conduct and discipline is equitable and consistent 

department-wide. Although the DHS Conduct and Discipline audit will not 

directly answer whether CBP is providing rigorous oversight with regard to 

allegations of personnel misconduct, it will include a high-level overview of 

CBP’s process for reporting and addressing misconduct, as well as some 

limited testing of CBP’s disciplinary action cases. We then revised our objective 

to focus on whether CBP made training improvements, communicated in a 

comprehensive manner its integrity strategy, and tracked and measured 

training completion and effectiveness. To accomplish our objective we reviewed 

CBP’s oversight of ethics and integrity training from fiscal years 2012 through 

2016. 

We researched and reviewed Federal, DHS, Office of Personnel Management 

(OPM), and CBP training requirements, policies, and procedures pertaining to 

ethics and integrity training. We researched industry standards pertaining to 

training evaluation methods. We also reviewed prior DHS OIG, General 

Accountability Office (GAO), Homeland Security Studies and Analysis Institute, 

Police Executive Research Forum, and the Integrity Advisory Panel reports for 

previously identified findings related to our audit. We determined whether HQ 

www.oig.dhs.gov 12 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

communicated the 2014 CBP Integrity and Personal Accountability Strategy to 

the field and its impact on ethics and integrity related training. 

We assessed CBP’s control structure, policies, procedures, and practices 

applicable to its ethics and integrity training program. Our limited assessment 

would not necessarily disclose all material weaknesses in this control 

structure. However, our assessment disclosed weaknesses in tracking of 

training, and communication of its integrity strategy and field-developed 

training. These weaknesses are discussed in the body of this report. 

We conducted interviews with officials from CBP offices and other offices to 

determine roles and responsibilities in the development, implementation, and 

oversight of the integrity strategy and CBP’s ethics and integrity training 

program: 

CBP Advanced Training Center in Harper’s Ferry, West Virginia 

x OTD Instructional Design Center 

x OTD The Law Enforcement Officer/Agent Safety and Compliance 

Directorate, formerly the Use of Force Center For Excellence 

x OTD Distance Learning Center 

CBP Headquarters, Washington, DC 

x CBP Air and Marines Operations (AMO) Training and Safety Standards 

Group 

x CBP Office of Border Patrol HQ Mission Readiness Operations Division 

(MROD) Training & Traumatic Incident Management Division 

x CBP Office of Chief Counsel (OCC) 

x CBP Office of Field Operations Analytical Management Systems Control 

Office (AMSCO) 

x CBP Office of Field Operations (OFO) HQ Training Branch 

x CBP Office of Field Operations Integrity and Professionalism Standards 

Division (ISPD) 

x CBP Office of Human Resources Management (HRM) 

x CBP Office of Information Technology (OIT) 

x CBP Office of Policy and Planning (OPP) 

x CBP Office of Professional Responsibility (OPR – formerly Internal Affairs) 

x CBP Office of Public Affairs (OPA) 

x CBP Office of Training and Development (OTD) 

x CBP OTD Budget Office 

x CBP Self-Inspection Program (SIP) Office 

x DHS Office Chief Human Capital Officer (OCHO) PALMS Administrator 

x DHS-OIG Investigations 

x DHS-OIG Workforce and Training Development Office 

x GAO 

www.oig.dhs.gov 13 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

To understand CBP’s field training operations and related ethics and integrity 

training, we conducted site visits and interviewed officials at Laredo, Texas 

(AMO, USBP, OFO, OPR, and OCC); Tucson, Arizona (AMO, USBP, OFO, and 

OCC); USBP basic academy in Artesia, New Mexico; and OFO and AMO basic 

academies in Brunswick, Georgia. We reviewed and analyzed HQ directives to 

field locations and academies, locally developed policies, and training SOP 

manuals. We examined course syllabi and curricula; in-service and other 

training agendas; musters; and webcasts. Auditors attended academy 

trainings; and observed several scenario-based training with ethics and 

integrity content. The audit team used a data collection instrument as part of 

our review process to substantiate and record field observations and testing 

performed. 

We evaluated the integrity of records management by reviewing electronic and 

manual files used for tracking training completion, such as attendance sign-in 

sheets; certificates of completion; PALMS and Training, Records, and 

Enrollment Network (TRAEN) system-generated reports; spreadsheets, and 

post-academy training files. We also received demonstrations of primary 

systems of record including PALMS; TRAEN; and ACADIS. 

We performed limited data reliability testing of PALMS data because of known 

data integrity issues and concerns over the system’s reporting functionality. We 

observed CBP officials at the OTD in May 2016 unsuccessfully attempt to 

generate reports of completed and overdue training, noting PALMS error 

messages and inaccurate totals. In October of that same year, the PALMS 

vendor developed a temporary solution for CBP’s Advanced Training Center 

Distance Learning Center, enabling them to run accurate reports for training 

completion. In October we did a limited validation of the system’s reporting 

functionality. From site visits we judgmentally selected 42 training completion 

certificates for the annual CBP Integrity Awareness training, see table 2 for 

details. The exception was the Brunswick, GA AMO Academy, at which there 

were only two agents assigned to that location. 

Table 2 – Sample Selection of Training Completion Certificates for 

Integrity Awareness Training 

Location USBP OFO AMO Total 

Laredo, TX 5 5 5 15 

Tucson, AZ 5 5 5 15 

Academies 5 5 2 12 

Total 42 

Source: OIG analysis of CBP data 

www.oig.dhs.gov 14 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

We compared our certificates to OTD’s October PALMS-generated report to 

determine whether the system accurately reflected training completion for our 

sample. We did not identify any significant discrepancies and believe, for the 

purpose of answering the congressional request, the PALMS data was 

sufficiently reliable to meet our audit objectives. 

For assessing CBP’s evaluation of training effectiveness, we reviewed DHS, 

OTD, OPM, and industry best practices. We evaluated End-of-Course 

Evaluations; Kirkpatrick Model Levels 1–3 surveys; after action reports; 

evaluation plans; and other documentary evidence showing that CBP measured 

the effectiveness of ethics and integrity training. 

We conducted this performance audit between April and October 2016 

pursuant to the Inspector General Act of 1978, as amended, and according to 

the generally accepted government auditing standards. Those standards 

require that we plan and perform the audit to obtain sufficient and appropriate 

evidence to provide a reasonable assurance for our findings and conclusions 

based upon our audit objectives. We believe that the audit evidence obtained 

provides a reasonable basis for our findings and conclusions based on our 

audit objectives. 

www.oig.dhs.gov 15 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix B 

CBP’s Comments to the Draft Report 

www.oig.dhs.gov 16 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 17 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 18 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix C 

Ethics and Integrity Training Improvements since FY 2012 

Since 2012, CBP has implemented several new training courses and programs 

that include ethics and integrity elements. The following tables list notable 

improvements. 

Customs and Border Protection-Wide 

Training/Program Name Implementation 

Timeframe FYs 

Annual Integrity Awareness Training 2012 

New Employee Orientation 2012 

Integrity Committees 2014 

Integrity Advisory Panel 2015 

Below 100 Training 2016 

Ethics, Integrity, and Professionalism Musters 2013 to 2016 

Michael V. Gallagher (Anti-DUI) 2015 

Instructional Systems Specialists (OTD) 2012 

Revised Senior Leadership Training (OTD) 2016 

Integrity and Professionalism Campaign 2014 

Supervisory Technical Training Course (USB) 2012 

Supervisory Professionalism Engagement 

Dialogue Trainings (OFO) 2015 

Integrity Officer Program (OFO) 2014 

Source: OIG analysis of CBP data 

www.oig.dhs.gov 19 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Academy 

Training/Program Name Implementation 

Timeframe FYs 

Basic Training Program Redesign (USBP) 2016 

Post-Academy Training Redesign (USBP) 2016 

Enhanced Scenario Based Trainings (USBP and 

OFO) 

2016 

Office of Chief Counsel-Nationality and 

Immigration Law and Applied Authorities (116 

hours) 

2016 

Redesigned Anti-Corruption and Integrity 

Reinforcement (AMO and OFO) 

2013 

Law Enforcement Professionalism (OFO) 2016 

Source: OIG analysis of CBP data 

Field 

Training/Program Name Implementation 

Timeframe FYs 

One-Hour Ethics Training (Laredo) 2016 

PRIDE Initiative (Laredo) 2015 

Ask the Director (Tucson) 2016 

Flying Armed (Tucson) 2014 

Leadership for Supervisors (Tucson) 2016 

Leadership for Non-Supervisors (Tucson) 2012 

Redesigned Report Writing and Courtroom 

Testimony (Tucson) 

2013 

Source: OIG analysis of CBP data 

www.oig.dhs.gov 20 OIG-17-60 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix D 

The Kirkpatrick Model Levels of Measurement, Benefits, and 

Limitations 

OTD’s Standard Operating Procedure (SOP), dated November 2015, details the 

four levels of measurement of the Kirkpatrick Model. 

Level 1 feedback is intended to capture the trainee’s view of the training and 

student confidence and attitude about skills obtained, instructor performance, 

and training environment. The data is intended for the course managers and 

instructor personnel and should be collected for all students at the end of short 

courses and throughout longer trainings. CBP Form 800 is the standard level 1 

form used to collect trainee reaction data. 

Level 2 evaluations are arguably the most important level of evaluation for OTD 

because it ensures students have either demonstrated or failed to demonstrate 

critical work tasks. Trends from this data can be used to validate the 

instructional strategy and reliability of the training delivery. 

Level 3 feedback evaluates the transfer and application of skills learned in the 

training to on-the-job performance. This is achieved by determining at what 

level graduates are performing the critical tasks from the course in the 

workplace. The graduate’s supervisor or someone with knowledge of the 

individual’s performance completes the surveys, interviews, or field 

observations to make this determination. 

Level 4 determines training’s impact and benefit to the organization and will 

only be conducted on a case-by-case basis developed by the Director of the 

Instructional Design Center (IDC). 

According to “A Critical Analysis of Evaluation Practice: The Kirkpatrick Model 

and the Principle of Beneficence,” in the Journal of Evaluation and Program 

Planning 27, 2004, there are benefits and limitations of the model. 

Benefits 

x The Kirkpatrick Model addresses the need of training professionals to 

understand training evaluation in a systematic way. 

x The four-level model provides a straightforward system or language for 

talking about training outcomes and the kinds of information that can be 

provided to assess the extent to which training programs have achieved 

certain objectives. 

x The four-level model provides a method of determining how training 

affects the organization’s ability to accomplish its missions. 

www.oig.dhs.gov 21 OIG-17-60 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

x	 The Kirkpatrick Model simplifies the complex process of training 

evaluation. 

Limitations 

x	 The four-level model does not take into consideration individual or 

contextual influences that affect how employees learn and organizations 

design and delivery training. 

x	 The learning culture of an organization, work unit goals, climate for 

learning transfer, and adequacy of employee work tools and equipment 

all affect how successful employees will be in the learning process; the 

Kirkpatrick model relies on an assumption that without learning, 

employees will not change behaviors, but research has failed to confirm 

this link. 

x	 Additionally, the Kirkpatrick Model assumes that the importance of 

information gathered increases moving from level 1 to level 4. However, 

evaluators may have certain preconceived notions that affect their ability 

to conduct accurate and reliable evaluations. For example, if employees, 

trainers, or evaluators are potentially faced with negative actions, based 

on the evaluation data, an evaluator may skew the data to produce 

favorable results. 

www.oig.dhs.gov 22	 OIG-17-60 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix E 

Office of Audits Major Contributors to This Report 

Brooke Bebow, Director 

Cecilia Carroll, Audit Manager 

Tessa Clement, Program Analyst 

Eddie Jones, Auditor 

Falon Newman-Duckworth, Auditor 

Michael Staver, Program Analyst 

Kevin Dolloson, Communications Analyst 

Ellen Gallagher, Communications Analyst 

Corneliu Buzesan, Independent Referencer 

www.oig.dhs.gov 23 OIG-17-60 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix F 

Report Distribution 

Department of Homeland Security 

Secretary 

Deputy Secretary 

Chief of Staff 

General Counsel 

Executive Secretary 

Director, GAO/OIG Liaison Office 

Assistant Secretary for Office of Policy 

Assistant Secretary for Office of Public Affairs 

Assistant Secretary for Office of Legislative Affairs 

CBP Commissioner 

CBP Assistant Commissioner for Training 

CBP Audit Liaison Office 

Office of Management and Budget 

Chief, Homeland Security Branch 

DHS OIG Budget Examiner 

Congress 

Congressional Oversight and Appropriations Committees 

www.oig.dhs.gov 24 OIG-17-60 
 

 

 

 

 

 

 

ADDITIONAL INFORMATION AND COPIES 

To view this and any of our other reports, please visit our website at: www.oig.dhs.gov. 

For further information or questions, please contact Office of Inspector General Public Affairs 

at: DHS-OIG.OfficePublicAffairs@oig.dhs.gov. Follow us on Twitter at: @dhsoig. 

OIG HOTLINE 

To report fraud, waste, or abuse, visit our website at www.oig.dhs.gov and click on the red 

"Hotline" tab. If you cannot access our website, call our hotline at (800) 323-8603, fax our 

hotline at (202) 254-4297, or write to us at: 

Department of Homeland Security 

Office of Inspector General, Mail Stop 0305 

Attention: Hotline 

245 Murray Drive, SW 

Washington, DC 20528-0305