Department of Homeland Security: Strategic Management of Training Important for Successful Transformation
Released: 2005-09-23T00:00:00

Letter

Background

The Homeland Security Act of 2002 created DHS, bringing together 22 agencies and programs responsible for important aspects of homeland security. The intent behind the creation of a single department was to improve coordination, communication, and information sharing among these previously separate entities, thereby increasing their effectiveness in protecting the nation’s security. Each of these organizations brought with it the capacity and expertise to provide training for its particular aspect of homeland security. For example, in several cases such as the Coast Guard and FLETC, this training capacity, as well as the management systems supporting it was transferred intact with the creation of the new department. In other cases, such as CBP and U.S. Immigration and Customs Enforcement (ICE), the training functions of legacy organizations were merged. Table 1 presents information on selected training characteristics of components in our review, including the origin of each component’s training function. In addition, the Act led to the creation of the CHCO position in DHS responsible for, among other human capital topics, oversight and planning of the training of employees. The CHCO, who reports directly to the department’s Under Secretary for Management, has primary responsibility for defining and developing the department’s role regarding training. Figure 1 depicts these positions as well as the department’s major components in the context of DHS’s overall organizational structure.

Training both new and current staff to fill new roles and work in different ways will play a crucial part in the ability of federal departments and agencies, such as DHS, as they work to successfully transform their organizations. In 2004, we issued an assessment guide that introduces a framework for evaluating the management of training in the federal government. As presented in our guide, the training process can be segmented into four broad, interrelated phases: (1) planning/front-end analysis, (2) design/development, (3) implementation, and (4) evaluation. For each of these phases, we summarize key attributes of effective training programs and offer related issues and questions. Using this framework, this report identifies selected strategic training practices, with a focus on the planning and evaluation phases, that may offer an opportunity for others in DHS to build on experiences and practices discussed below.

The results of a governmentwide survey conducted by the Office of Personnel Management in 2004 on human capital practices and employee attitudes suggest that efforts to identify and build upon examples of good training practice within DHS may be particularly relevant. For each of the eight questions in the 2004 Federal Human Capital Survey that focused on training related topics, the percentage of DHS respondents providing positive responses (typically the top two options on a five-point scale) was lower than the governmentwide average. In fact, the DHS response ranged from 5 to 20 percentage points lower than the governmentwide average for the same questions. For example, 54 percent of respondents at DHS indicated that they received the training they needed in order to perform their jobs, compared to 60 percent governmentwide. Half (50 percent) of DHS respondents said that they were either satisfied or very satisfied with the training they received for their present jobs, as opposed to 55 percent that expressed these levels of satisfaction governmentwide. The largest difference involved having electronic access to learning and training programs, where 51 percent of DHS respondents responded positively, compared to 71 percent governmentwide. A DHS official told us that the department is aware of the challenges reflected in these data and is currently exploring options with the Office of Personnel Management to conduct further analysis. The aim of this work would be to identify areas where DHS might target additional attention as well as provide a baseline for future attitude measures.

DHS Has Made Progress in Addressing Departmentwide Training Issues, but Challenges May Impede Its Efforts

DHS has made progress in addressing departmentwide training issues and these efforts reflect some of the elements of a strategic approach toward training as described in our previous work. Most training-related activities at DHS—such as planning, delivery, and evaluation—primarily take place at the component level and relate to mission issues. Therefore, any successful approach regarding departmentwide training issues will require the concerted and coordinated efforts of multiple components within DHS as well as the ability of the CHCO to effectively lead a network of different training organizations. The department’s current efforts, although promising, are still in the early stages and they face significant challenges. Unless these challenges are successfully addressed they may impede DHS’s ability to achieve its departmentwide training goals.

DHS Recently Developed a Departmental Training Strategy

DHS recently developed a coordinated departmental training strategy that supports broader human capital and organizational goals and objectives. We have previously reported that effective organizations establish clear goals with an authority structure able to carry out strategies and tactics, that is, the day-to-day activities needed to support the organization’s vision and mission. By so doing, a well-designed training function can be directly linked to the organization’s strategic goals and help to ensure that the skills and competencies of its workforce enable the organization to perform its mission effectively.

DHS’s department-level training strategy is presented in its human capital and training strategic plans. Issued in October 2004, its human capital strategic plan includes selected training strategies, such as developing a leadership curriculum to ensure consistency of organizational values across the department and using training to support the implementation of the new DHS human capital management system, MAXHR. In July 2005, DHS issued its first departmental training plan, Department of Homeland Security Learning and Development Strategic Plan, which provides a strategic vision for departmentwide training. This plan is a significant and positive step toward addressing departmentwide training challenges. The plan identifies four short-term goals for fiscal year 2006 and one long-term goal for fiscal years 2006 through 2010. Among the short-term goals are such tasks as defining the scope of training activities and improving the governance process between the CHCO office and individual organizational components, supporting the rollout of MAXHR, identifying/implementing best practices, and addressing specific concerns regarding DHS’s training facilities and advanced distributed learning studies. The plan also articulates a long-term goal for DHS to “become a recognized world-class learning organization where managers and supervisors effectively lead people.”

Each of these goals is followed by supporting strategies and tactics. For example, to achieve its goal of ensuring the best use of training resources through the identification and implementation of best practices, the plan identifies specific strategies, one of which is to improve the awareness of ongoing DHS training activities among organizational components. This strategy is, in turn, supported by still more specific tactics such as developing a site on the DHS Interactive system to facilitate the sharing of information across the training community.

More significant than the fact that DHS issued a training strategic plan document is the fact that DHS followed an inclusive and collaborative process while developing it. We have previously reported that for high- performing, results-oriented organizations, a strategic plan is not simply a paper-driven exercise or onetime event, but rather the result of a dynamic and inclusive process wherein key stakeholders are consulted and involved in the identification of priorities and the formation of strategies. When creating its plan, DHS consulted training leaders at components throughout the department, in addition to others, to help develop and review its content. Several training leaders we spoke with thought highly of this process and the extent to which it provided them opportunities to contribute and comment on the draft plan.

DHS has made considerable progress in addressing departmentwide training issues through the development of its first training strategic plan. However, there are areas where future efforts can be improved.

Linkage to DHS organizational and human capital strategic plans. Our past work on strategic planning and management practices shows that effective strategic plans describe the alignment between an agency’s long- term goals and objectives and the specific strategies planned to achieve them. Clearly linking training tactics with particular organizational objectives creates a direct line of sight that can both facilitate the ability of staff to work toward mission goals and enable stakeholders to provide meaningful oversight. In the introduction to the DHS training strategic plan, the department’s CHCO highlights the value of this practice stating that “the key purpose of plan is to align our education, training and professional development efforts with the President’s Management Agenda and the Department’s vision, mission, core values and strategic plan.”

The DHS training strategic plan contains examples of goals, strategies, and tactics that align with and support goals found in the department’s human capital and organizational strategic plans; however, these linkages are never actually identified or discussed in the plan itself. For example, the DHS training strategic plan contains a goal and several tactics related to MAXHR training. These, in turn, support a MAXHR goal and strategy in the department’s human capital strategic plan as well as the “organizational excellence” goal of the DHS strategic plan. However, the training strategic plan does not show these linkages. Identifying such linkages, either in the training plan itself or in an appendix, would more clearly communicate to both internal and external stakeholders the connections and justifications for specific training goals, strategies, and tactics.

DHS’s own human capital strategic plan provides an illustration of one way to communicate linkages between goals and strategies contained in the plan and the broader organizational goals they are intended to support. For example, in an appendix, the DHS human capital strategic plan contains a matrix that directly links strategies, such as developing a new Senior Executive Service (SES) performance management system, with specific objectives contained in the DHS strategic plan as well as the President’s Management Agenda human capital standards for success.

Usefulness of performance measures. We have previously reported several key characteristics of effective strategic and management plans, including the need for performance measures. Appropriate performance measures along with accompanying targets are important tools to enable internal and external stakeholders to effectively track the progress the department is making toward achieving its training goals and objectives. To this end, organizations may use a variety of performance measures— output, efficiency, customer service, quality, and outcome—each of which focuses on a different aspect of performance.

The DHS training strategic plan contains few specific performance measures for its goals or strategies and all of these are output measures. For example, the plan makes use of output measures in its requirement that certain actions, such as the development of a new management directive or the chartering of a team, be completed by the end of fiscal year 2006, and in establishing a deadline for when reports need to be completed in order to be included in the 2007 plan. In contrast to output measures like these, which gauge the level of activity or effort by measuring whether a particular thing is produced or service performed, other types of measures, such as measures of customer satisfaction or program outcomes, focus on the impact or results of activities. By appropriately broadening the mix of measures it uses and more clearly identifying targets against which DHS can assess its performance, DHS can improve the usefulness of its plan. After we completed our audit work, DHS training officials informed us that they decided to delay the development of performance measures until the rollout of the plan, when they could be developed by individual teams, as needed. They subsequently informed us that these teams will be held accountable to establish further performance measures that are outcome based and results oriented.

DHS’s human capital strategic plan again provides an illustration of how the department’s training strategic plan might begin to work toward the inclusion of different types of performance measures. For example, accompanying the strategy that DHS assess the feasibility of establishing a 21st Century Leadership Training and Development Institute, the plan identifies two performance measures—customer satisfaction and cost of delivery—along with specific targets for each. For the customer satisfaction measure, the plan establishes a target of 4.5 on a scale from 1 to 5. The plan also includes specific tactics to achieve the strategy, such as developing and obtaining cross-organizational support, developing measures and methodologies for leadership training, and implementing a learning management system, along with key milestone dates for completing them.

The department may benefit from considering the experiences of leading organizations regarding the development of results-oriented performance measurement. In general, results-oriented organizations we have studied that were successful in measuring their performance developed measures that were tied to program goals and demonstrated the degree to which the desired results were achieved, limited to the vital few that were considered essential to producing data for decision making, responsive to multiple priorities, and responsibility linked to establish accountability for results.

Similar to the consultative process DHS followed when developing the goals and strategies contained in its training strategic plan, decisions concerning the selection of an appropriate set of performance measures should also be based on input from key stakeholders to determine what is important to them when assessing the department’s performance regarding training. Clear and appropriate performance measures, developed in this way, can also provide DHS with valuable information, especially significant in the current fiscal environment, when it seeks to justify requests for resources from Congress.

Training Councils and Groups Created by DHS Increase Communication across Components

Under the overall direction of the CHCO office, DHS has established a structure of training councils and groups that cover a wide range of issues and include representatives from each organizational component within DHS. The department is in the process of using these bodies to facilitate communication and the sharing of information within its diverse training community. In some instances, these councils and groups foster greater collaboration and coordination on training policies, programs, and the sharing of training opportunities. We have previously reported that agencies with a strategic approach to training recognize the importance of having training officials and other human capital professionals work in partnership with other agency leaders and stakeholders on training efforts.

The Training Leaders Council (TLC) plays a vital role in DHS’s efforts to foster communication and interchange among the department’s various training communities. This council consists of senior training leaders from each of the department’s components as well as representatives from several department-level headquarters staff and support organizations with an interest in training-related issues. Started in October 2004 and formally chartered by the CHCO in March 2005, the TLC’s mission is to establish and sustain a collaborative community with the aim of promoting high-quality training, education, and development throughout DHS. To this end, it functions as a convener of training leaders from throughout the department and provides an overarching framework for several preexisting training groups and councils that were reestablished as standing committees of the TLC. Membership of the TLC consists of senior training leaders from each DHS component. In addition, most of these leaders as well as other training staff serve on one or more of its subgroups. See figure 2 for descriptions of the TLC and each of its subgroups.

One key function of the TLC and these other training groups is to serve as a “community of practice” wherein officials can discuss common training challenges and share knowledge and best practices. For example, the Training Evaluation and Quality Assurance Group, composed of DHS training professionals responsible for evaluating and ensuring the quality of DHS training programs, conducted an informal survey of evaluation practices in various components with the intent of identifying effective evaluation approaches. A training official involved in the group told us that this survey was particularly important for the department’s newer organizations, such as the Directorate for Information Analysis and Infrastructure Protection, which need to establish new practices from scratch. According to this official, his directorate and other organizations within DHS plan to use the group as a way to tap into the experience of other components within the department, such as CBP and FLETC, which have considerably more experience with training evaluation.

In addition to sharing information about training practices, these groups can also provide a forum for exchanging practical information with the goal of making more efficient use of existing resources. For example, one training official told us that as a result of information obtained at TLC meetings, the official became aware of the existence of free training space available at facilities of two other components located in the Washington, D.C., metro area. Also, as a result of participating in these meetings, the official’s organization was able to send an additional person to the Federal Executive Institute after becoming aware that another component had surplus spaces and was offering them at a reduced price to other components within DHS.

Another role carried out by the TLC is to collaborate on the formulation of training policies and advise the department’s CHCO accordingly. For example, the TLC, in cooperation with staff from the CHCO office and an external contractor, conducted a survey of training sites throughout the department in 2004. This study cataloged available physical resources and site capacities with the aim of identifying potential opportunities to share these resources more efficiently, consolidate unneeded or duplicative sites, and identify other opportunities to increase training collaboration and effectiveness. This effort resulted in a series of recommendations that were subsequently incorporated into the department’s training strategic plan.

The activities of the department’s Advanced Distributed Learning Group (ADLG) provides another example of how training officials from different components have worked together to develop proposals for solutions to departmentwide challenges. This group identified several issues in the area of technology and learning, including the need for a compatible IT infrastructure across components and the fact that some components lacked established systems with which to coordinate and manage training opportunities and attendance. Working with an outside consultant, the ADLG’s efforts resulted in a proposal that DHS create a new Advanced Distributed Learning (ADL) Program Management Office to oversee the process of setting common standards. This proposal was subsequently included as part of the department’s training strategic plan. In addition, the ADLG’s work also led to DHS entering into a memorandum of understanding with the Office of Management and Budget and the Office of Personnel Management to create a DHS headquarters learning management system. Throughout this process, the ADLG represented the interests of the DHS training community as it worked with representatives from the Chief Information Officer’s office and other functions within the department, as well as outside consultants.

Despite these positive steps, DHS’s effort to foster communication and coordination through departmentwide training councils and groups is at a relatively early stage and so far has produced varied results. Some training organizations, such as the TLC and ADLG, have met regularly leading to tangible results, while others such as the Training Evaluation and Quality Assurance Group, have met a few times and have only begun to set the groundwork for substantive coordination and collaboration in these areas. In addition, a training official told us that even active organizations like the TLC have encountered difficulties related to the relative lack of staff support for these efforts. As a result, additional burdens sometimes fall to the leaders and members of these groups who, in addition to serving on one or more departmental training groups or councils, must carry out full- time training positions at their home components.

DHS Provides Training to Address Selected Departmentwide Needs

Another way DHS addresses departmentwide training issues is to directly provide training interventions or resources that address selected departmentwide needs, goals, or objectives. Three examples of the areas where DHS has worked to directly provide or support training on the departmental level are the following: (1) training related to the implementation of MAXHR, (2) DHS leadership development, and (3) training related to civil rights and civil liberties.

Training for MAXHR implementation. DHS’s new human capital management system, known as MAXHR, represents a fundamental change in many of the department’s human capital policies and procedures that will affect a large majority—approximately 110,000—of its civilian employees. MAXHR covers many key human capital areas, such as pay, performance management, classification, labor relations, adverse actions, and employee appeals, and will be implemented in phases affecting increasing numbers of employees over the next several years.

DHS correctly recognizes that a substantial investment in training is a key aspect of effectively implementing MAXHR, and in particular, the new performance management system it establishes. The need for in-depth performance management and employee development training is further supported by the department’s results on the 2004 Federal Human Capital Survey. In this survey, just over half of DHS respondents—51 percent— believe supervisors or team leaders in their work units encourage their development at work, significantly less than the governmentwide response of 64 percent. DHS officials said they plan to educate all affected DHS employees on the details of the new system, how it will affect them, and the purpose of the changes. To do this, the department decided to develop, coordinate, and manage MAXHR training centrally through the CHCO office and offered its first training in May 2005. DHS plans to continue to provide its workforce with MAXHR training over the next several years following a phased approach that takes into account both when individual provisions of the new regulations take effect as well as the different audiences that exist within the DHS community, including human capital personnel, supervisors, and general employees. See figure 3 for a depiction of planned training during 2005 and its intended audiences.

The department has worked with contractors to develop training that uses a variety of approaches, including classroom instruction, ADL, handbooks, manuals, and quick reference guides, depending on specific needs. For example, in May 2005, labor relations/employee relations specialists and attorneys in the department received 2-½ days of training on the provisions of the new regulations and the major difference between them and previous programs. Structured as a “train the trainer” type intervention intended to prepare participants to conduct supervisor briefings in their own components, this was an instructor-led course held at sites across the country. In addition to educating individuals about the regulations, procedures, and systems associated with MAXHR and the adoption of a new performance management system, the department also plans to offer training specifically targeted to developing the skills and behaviors that will be necessary for its successful implementation. For example, in July 2005 supervisors began to receive training on techniques for providing meaningful feedback to, coaching, and mentoring employees.

DHS leadership development training. Leadership development is another area top management in DHS acknowledged as appropriate for departmentwide training to supplement existing component-level offerings. In 2004, the Secretary of Homeland Security announced the “One DHS” policy that identified the need to establish a common leadership competency framework for the department, as well as a unified training curriculum for current and future leaders. The purpose of this framework was to identify the skills, abilities, and attributes necessary for success as a DHS leader and to establish measurable standards for evaluation.

To this end, the CHCO established the DHS Leadership Training and Development Group (LTDG), comprising training officials from each DHS component who combined an expertise in leadership development with personal knowledge of the missions and unique aspects of their particular organizational components. The LTDG met regularly from late 2003 to mid-2004. During this time, the group developed a set of new core leadership competencies for DHS supervisors, managers, and executives, which it issued in April 2004. According to a DHS official, since the development of these new competencies they have been used by one component as part of its own leadership development plan and they have also helped to guide and inform current MAXHR leadership development efforts.

DHS has recently taken steps regarding another facet of its leadership development initiative—its SES Candidate Development Program. In June 2005, DHS issued a management directive establishing the SES Candidate Development Program, which included a rigorous selection process and critical leadership development opportunities, such as mentoring, developmental assignments, and action learning designed to give SES candidates experience in different job roles. DHS initially announced that it planned to implement the program in fiscal year 2005, but now may delay doing so until fiscal year 2006.

Civil rights/civil liberties training. A third area in which DHS has taken steps to provide or support departmental training involves civil rights and civil liberties. FLETC’s Behavioral Science Division and Legal Division, working with the DHS Office of Civil Rights and Civil Liberties, produced several training interventions, including Web-based, CD-ROM, and in- person programs designed to increase sensitivity and understanding in protecting human and constitutional rights. As part of this effort, FLETC held diversity seminars that focused on promoting understanding and respect of religious practices, particularly involving those of the Arab and Muslim communities. In another example of this effort, the Office of Civil Rights and Civil Liberties produced Web-based training on current policies regarding racial profiling.

Our interviews with DHS training leaders suggest that further improvements can be made in communicating the availability of selected departmentwide training programs and resources. Staff at the Office of Civil Rights and Civil Liberties provided copies of its civil rights and liberties programs to training offices at each component in the department. While some senior training officials told us that their components actively disseminated this material by placing it on the component’s training Web site or incorporating it into preexisting courses, other senior training officials we spoke with were unaware of any departmental training on these topics. In addition, other officials told us that their component’s training office had independently developed its own material on Arabic sensitivity training, wholly apart from similar efforts undertaken by others in the department. More specifically, they told us that their development of certain training modules predated the development of very similar modules later prepared by DHS’s Office of Civil Rights and Civil Liberties and FLETC, leading these officials to conclude that they may have been able to assist departmental efforts by sharing their work had they been aware of them.

Challenges May Impede Achievement of Departmentwide Training Goals

As DHS moves forward, it faces challenges to achieving departmentwide training goals. These challenges include lack of common management information systems, the absence of commonly understood training terminology across components, the lack of specificity in authority and accountability relationships between the CHCO office and components, insufficient planning for effective implementation, and insufficient resources for ensuring effective implementation of training strategies.

Sharing of Training Information across Components Made More Difficult by Lack of Compatible Infrastructure and Commonly Understood Terminology

The formation of DHS from 22 legacy agencies and programs has created challenges to achieving departmentwide training goals. Of particular concern to the training officials we spoke with are the lack of common management information systems and the absence of commonly understood training terminology across components. The training functions at DHS’s components largely operate as they did before the creation of the department, with many of the same policies, practices, and infrastructures of their former organizations, and within these organizations are, for the most part, the same training leaders. It will take time for these organizations to evolve into a coordinated, integrated department. We have previously reported that successful transformations of large organizations, even those faced with less strenuous reorganizations and pressure for immediate results than DHS, can take from 5 to 7 years to fully take hold.

One issue DHS officials raised was the lack of common or compatible management information systems, such as information technology or financial management, which can inform decision makers’ efforts to make efficient use of training resources across components. For example, DHS officials stated that a key challenge they encountered involved the difficulty of knowing what others were doing outside their particular offices or components. DHS lacks any unified sourcebook that employees could consult for the names, telephone numbers, and other relevant information of key contact persons in areas such as acquisition. Obtaining accurate information about resources and products available in the marketplace as well as data on users, vendors, and kinds of work has been a challenge to that effort. Another issue cited by officials concerned the lack of compatibility between learning management systems across components. In addition, some training officials expressed concerns about the accuracy or timeliness of some training data, which can limit or at least considerably delay their ability to track and fully account for funds spent on training and training-related travel. DHS has several efforts under way to address these issues, including the development of an online training facilities inventory intended to increase awareness of existing resources across the department and its decision to begin developing common ADL policies and standards.

Officials also told us that there was little or no common understanding among DHS organizational components regarding the meaning of such basic terms as “subject matter expert,” “orientation,” and even “training.” The lack of commonly understood terminology has presented challenges when officials from different components, including those participating in departmental training councils and groups, try to share practices with each other. These officials told us that the lack of commonly understood terminology can also affect their interactions with outside entities, such as contractors and state and local agencies. Besides facilitating communications and enabling components to share practices, a DHS official told us that a common nomenclature would increase the transparency of training practices to external contractors as well as the internal DHS training community. The department’s training strategic plan calls for the creation of a common training language and glossary of terms in fiscal year 2006, and officials told us that they are currently in the early stages of creating such a glossary.

Authority and Accountability Relationships between CHCO Office and Components Are Not Sufficiently Specific

An effective management control environment appropriately assigns authority and delegates responsibility to the proper personnel to achieve organizational goals and objectives. In such an environment, staff members who are delegated responsibility are given corresponding authority. In light of this, DHS’s adoption of a “dual accountability” governance structure in 2004 presents certain challenges. Under this concept, heads of organizational components and the CHCO share responsibility for effective training in DHS. With a shared responsibility for DHS training, both the CHCO and component heads should have appropriate authority for making decisions regarding training. DHS does not specify how authority for training matters will be shared between the CHCO office and components for budgeting, staffing, and policy (e.g., determining which training functions, if any, should remain with components or be performed by DHS headquarters). The DHS management directive on training currently in place is a high-level two- page document that provides very few specifics on policies, procedures, and authorities for the CHCO office and the components.

The department recognizes the need to clarify the responsibilities and authorities of the CHCO office and the components, as indicated by its inclusion in the DHS training strategic plan. Many of the tactics included in the plan would be difficult to successfully implement without first having a clear understanding of the responsibilities and authorities of the key organizations involved. More specifically, in the absence of clear authority relationships, decisions regarding how particular component training goals and strategies are to be incorporated in the DHS training strategic plan, or which training facilities should be consolidated to achieve departmental efficiencies, will be difficult to make. Without moving ahead with this effort in a timely fashion and completing the process of specifying how the CHCO office and components will share authority over training matters, it will be difficult for DHS to make the progress necessary on its departmentwide training agenda if it is to effectively implement the many strategies and tactics planned for fiscal year 2006.

In addition, DHS’s efforts at coordinating training across components and clarifying roles and relationships between departmental functions and organizational components may be further hampered by the fact that the management directive governing the integration of the human capital function claims that the Coast Guard and the Secret Service are statutorily exempt from its application. We found no reasonable basis to conclude that the directive could not be made applicable to them and are not aware of any explicit statutory exemption that would prevent the application of this directive. Moreover, exempting the Coast Guard and the Secret Service from the provisions of this directive casts doubt on the authority and accountability relationships between these components and the CHCO, potentially complicating the department’s objective of clarifying the responsibilities, accountability, and authorities of the CHCO office and the components set forth in DHS’s training strategic plan.

Planning May Be Insufficiently Detailed to Ensure Effective Implementation in “Dual Accountability” Environment

In and of itself, DHS’s dual accountability authority structure is not an obstacle to implementation of departmentwide training efforts. However, without detailed implementation plans, it presents potentially significant challenges. Because of this shared authority, DHS will need to take great care when planning for departmentwide training initiatives involving multiple organizational components to ensure that resources are aligned with organizational units performing activities, especially related to cross- organizational sharing of training and delivery of common training. The lack of comprehensive and rigorous planning can lead to confusion over responsibilities, lack of coordination, and missed deadlines. Regular and rigorous use of detailed implementation plans is necessary to implement decisions and carry out activities in a coordinated manner.

After we completed our audit work, DHS informed us that it plans to establish 31 tactic teams to take ownership of each of the tactics presented in the DHS training strategic plan to be completed by the end of fiscal year 2006. As of mid-August 2005, DHS provided us with documentation indicating that 3 of these teams have been established to date. These teams appear to have taken promising steps toward the establishment of detailed plans for implementing their respective training tactics by developing draft objectives, deliverables, and closure criteria. But as fiscal year 2006 approaches, time is short for the CHCO office and the components to establish the remaining teams and then take the actions necessary to develop and put in place the detailed plans that will be critical for effectively implementing DHS’s many training tactics by the end of the coming fiscal year. The TLC’s ADLG has made use of this type of detailed approach in a report proposing a distance learning architecture for the department. Appended to its report is a detailed plan outlining the major activities, milestones, resources, and components needed to support the successful implementation of the proposal.

Resources Needed to Implement Departmental Training Strategy

Several training officials told us they were concerned about the lack of dedicated resources and related capacity to carry out departmental initiatives. At the time we started our review, the CHCO office had only one full-time permanent employee dedicated to carrying out these activities; consequently, both training leaders and staff from organizational components were relied on to contribute to departmentwide efforts. After we concluded our audit work, a DHS official told us that the CHCO office had recently hired two additional full-time training staff: an ADL program manager and a staffer to oversee a recently approved SES candidate development program and headquarters operational leadership development. Individual components have also provided some assistance to departmentwide efforts through the appointment of temporary personnel. In late 2004, CBP and FLETC each detailed a staff member to the CHCO office to work on training-related projects. In addition, DHS has contracted for services to address selected departmentwide issues, such as setting common standards for ADL and reviewing DHS training facilities.

DHS’s departmental training councils and groups are almost exclusively staffed by component training leaders who already have full-time training commitments. The department’s training strategic plan identifies many tactics for fiscal year 2006—including creating a common training language and glossary of training terms, establishing a repository for course catalog information, and developing a DHS training Web site—that will require considerable staff support to implement. Successful and timely completion of these and other initiatives will depend on sufficient resources being provided.

DHS’s Approach to Planning and Evaluation of Training Demonstrates Some Elements of Strategic Practice, but Is Still Evolving

It is essential for federal agencies to ensure that their training efforts are part of—and are driven by—their organizational strategic and performance planning processes. We have reported that aligning training with strategic priorities and systematically evaluating training activities play key roles in helping agencies to ensure that training is strategically focused on improving performance and meeting overall organizational goals. Strategic training practices in several DHS components or programs may provide models or insights to others in the department regarding ways to improve training practices. In areas where some components employed strategic practices, other components did not.

Aligning Training with Organizational Priorities Is a Strategic Practice

We have previously reported that agencies demonstrating a strategic approach to training align their training efforts with overall strategic priorities. To do this, agencies can employ a variety of practices, such as linking training activities to strategic planning and budgeting and performing front-end analysis to ensure that training activities are not initiated in an ad hoc, uncoordinated manner, but rather are focused on improving performance toward the agency’s goals. Some components in DHS applied the strategic practice of aligning training with organizational priorities, while others did not.

CBP links its new and existing training activities to its strategic priorities when planning for its strategic initiatives and expenditures. Importantly, the head of training at CBP is at the decision-making table with other CBP leaders to help establish training priorities consistent with the priorities of the CBP Commissioner. Relevant program managers are asked, “What training do you need to achieve the goals in your strategic plan?” Such discussions took place during planning for CBP’s custom trade pact initiative.

During each budget cycle, CBP’s central training office issues a “call for training” to its mission and mission support customers to estimate CBP’s training needs for existing training activities and prioritize these needs based upon the Commissioner’s priorities. Prior to establishing this process, training was mostly decided on a first come, first serve basis without clear and transparent linkages to organizational priorities. CBP’s current process results in an annual training plan in which training needs are identified by priority as well as major occupational type, such as border patrol agent or CBP officer. Training decisions are based on whether training requested is critical, necessary, or “nice to have.” During fiscal year training plan implementation, CBP tracks actual training activity through a central database to determine whether CBP is using its planned training resources. By tracking plan usage through a centrally managed database, CBP is able to reallocate unused training funds prior to the end of the fiscal year for either training activities that were not included in its original plan because of capacity constraints, or for emerging training priorities.

The Coast Guard has adopted a strategic and analytic approach to training through its use of the Human Performance Technology (HPT) model—a front-end training assessment process to determine the cause of performance problems. The process starts with the assumption that many factors influence individual and unit performance and it is important to determine what the factors are before concluding that training is the solution. From its HPT analysis, the Coast Guard determines whether training is needed or whether another type of solution, such as a policy change, would be more appropriate. For example, in addressing a problem in aviation maintenance, a Coast Guard working group looked at likely causes of its performance problems and concluded that focusing on making aviation maintenance training better was not the only solution. More specifically, training officials encountered problems with job dissatisfaction and subpar performance from aviation chief warrant officers. In this case, training officials used HPT to analyze the nature of work performed by those responsible for aviation maintenance and concluded that there was not a good match between job skills and responsibilities. Specifically, over the last 20 years, the scope and nature of the work performed by chief warrant officers changed significantly from maintaining components to managing aircraft systems. Performance problems were mainly caused by significant changes in the job functions of these officers over the years rather than by a lack of adequate training.

In cases where the HPT analysis concludes that training is warranted, a training analysis is performed to determine the specific training interventions. For example, in implementing activities related to the Maritime Transportation Security Act, the Coast Guard analyzed its training needs through the HPT process to determine training necessary to help maritime inspectors reduce the exposure of ports and waterways to terrorist activities. The analysis identified the skills and knowledge necessary for new maritime inspector tasks and provided training interventions, such as developing job aids and targeted classes, to prepare inspectors for the tasks most relevant to support their new role. New courses were piloted and then subjected to multilevel evaluations to assess their effectiveness and potential impact on employee performance.

Systematic Evaluation of Training Is a Strategic Practice

Agencies demonstrating a strategic approach to training employ a variety of practices, such as systematically evaluating training, actively incorporating feedback during training design, and using feedback from multiple perspectives. Several components and programs we examined at DHS used these practices, while others did not.

One commonly accepted model used for assessing and evaluating training programs consists of five levels of assessment (see fig. 4). In our review, virtually all components captured Level I data focusing on end-of-course reactions, while several also collected Level II data focusing on changes in employee skill, knowledge, or abilities. Several components evaluated, or were planning to evaluate, the impact of selected training programs on individual behavior, represented by Level III evaluations.

To measure the real impact of training, however, agencies need to move beyond data focused primarily on inputs and outputs and develop additional indicators that help determine how training efforts contribute to the accomplishment of agency goals and objectives. At a couple of components, DHS officials told us they conducted Level IV evaluations, which assess the effectiveness of training interventions. We found no examples of the department or its components measuring the return on investment of training activities (Level V). Training effectiveness should be measured against organizational performance; however, not all levels of training evaluation require or are suitable for return on investment analysis. Determining whether training programs merit the cost of using such an approach depends upon the programs’ significance and appropriateness.

CBP takes a systematic approach to evaluating its training activities through its National Training Evaluation Program (NTEP) to help program managers and trainers make more informed decisions on the effectiveness of training courses and their delivery. Despite the fact that CBP is a large and decentralized organization, NTEP has enabled it to collect course evaluation information and make this information available to a wide range of users in a timely manner. NTEP has also standardized evaluation data to allow for comparison of training throughout various field locations. Before the rollout of NTEP, CBP did not use a standard mechanism for collecting evaluation data, which, according to a CBP official, made it difficult to gather evaluation data nationally.

CBP focuses on collecting both end-of-course student reactions (Level I) and supervisor assessments of student on-the-job performance after attending the training (Level III). Electronic or paper-based evaluations are entered into the NTEP information system. The “close to real time” online data enables supervisors to perform trend analysis on training quality and provides opportunities for them to troubleshoot training deficiencies and identify high-performing courses. The NTEP online system allows CBP employees access to evaluation data on a need to know basis with four levels of access, while enabling them to locate evaluation data for any training class by date. Evaluation reports are aggregated for review by senior CBP officials.

A CBP official told us that collecting course evaluation data is labor intensive, especially since many field operations still use paper processes. In addition, CBP has experienced a relatively low submission rate for Level I evaluation data for many of its training classes. The official told us that this was especially true for end-of-course reactions from staff in the field, where only about one-third of officer-related course participants submit evaluation forms. Given cost and labor challenges, CBP has targeted areas for evaluation that it believes are important, such as training related to its “One Face at the Border” initiative.

In addition, agencies with a strategic approach to training do not wait until the conclusion of a training intervention to conduct evaluations. Rather, they approach evaluation through an iterative process capable of informing all stages of training. DHS’s CHCO office used multiple forms of feedback from employees to develop its training strategy for MAXHR. From February through April 2005, the department administered surveys and conducted focus groups to obtain information on the needs, attitudes, and reactions of different communities affected by MAXHR. Shortly after issuing its new human capital regulations, the department provided basic information to all employees on the nature and timeline of changes they could expect under MAXHR through a Web broadcast. After the broadcast an online survey was used to obtain feedback from employees regarding the broadcast itself and their general feelings and concerns about the MAXHR initiative.

DHS followed this initial survey with a larger survey to gather additional feedback on how information regarding MAXHR had been communicated, as well as specific areas where employees wanted additional information. Concerns about the need for training were prominent among the more than 9,000 responses received, with respondents ranking training as the second most serious challenge to the successful implementation of MAXHR. According to a senior DHS official, the survey results will inform subsequent training and communication efforts.

DHS also collected evaluative feedback by conducting a series of focus groups held in locations across the country. The aim of these sessions was to validate the design of the performance management program established under MAXHR and identify concerns that would inform the development of additional training. Consistent with the strategic training practice of seeking out different perspectives when redesigning and assessing training efforts, DHS staff held separate focus groups for bargaining unit employees, non-bargaining unit employees, and supervisors and managers at all of the locations visited. This enabled them to identify issues of particular concern to each of these groups as well as issues common to all three. For example, both the bargaining unit and non-bargaining unit employee focus groups raised concerns about supervisors having inadequate skills for fairly administering the new performance management system. This concern was also shared by supervisors and managers themselves who expressed the need for additional skills training in areas such as goal setting and providing performance feedback. The sessions validated the CHCO office’s plans to offer performance management training to supervisors and managers before the implementation of the new system and assisted in refining issues for future training.

FLETC’s methods for evaluating its major training programs include feedback from multiple perspectives when examining the benefits of training on actual employee job performance. FLETC’s Level III evaluations obtain feedback from both trainees and their supervisors to inform future improvements to training curricula. Evaluation results are compiled into a comprehensive report used during FLETC’s periodic curriculum reviews on its major training programs, such as the Natural Resource Police Training Program. The report contains detailed feedback from both the trainee and supervisor perspectives 6 months to 1 year after the trainee has attended the training program. For example, for the Natural Resource Police Training Program, FLETC analyzed how well the program prepared trainees in all aspects of their jobs. In this case, analysis identified those courses that had benefited program trainees the least— including determining speed from skid marks and death notification. Training designers can use report information to improve program curricula and refocus training on knowledge and skill areas most critical to performing the job. In addition to Level III evaluation results, its training designers make program and individual class changes by using other methods of evaluation, such as direct student feedback after classes and trainee examinations, which determine how well the trainees understood the course material immediately following the program.

DHS Has Used Training in an Effort to Help Its Workforce as the Department Undergoes Transformation and Cultural Change

The creation of DHS resulted in significant cultural and transformational challenges for the department. We have previously reported that training is one way organizations successfully address cultural issues while simultaneously facilitating new ways to work toward the achievement of organizational goals. Among the DHS components in our review, some merged cultures from different legacy organizations (CBP, ICE), another component came as a small organization that greatly expanded when joining DHS (Federal Air Marshal Service), while others joined DHS intact (Secret Service, Coast Guard, FLETC), and still another was previously a part of a larger legacy organization (CIS). Each component faces the need to find a way to identify itself as part of the larger DHS organization, that is, with a sense of affiliation rather than as an outsider looking in. At the same time, components must either maintain their existing cultures or develop new cultures to adapt to changing missions and needs. The key is to build upon positive aspects of the components’ cultures as the larger organization develops its own culture.

Agencies that undergo successful transformations change more than just their organizational charts, they also make fundamental changes in basic operations, such as how they approach strategic human capital management. DHS understands this, and the MAXHR initiative is part of an effort by the department to fundamentally change its approach to human capital management by establishing a personnel system that is flexible, performance oriented, and market based.

The Secretary of Homeland Security and other top officials have actively supported the role of training in implementing these changes by making it a leadership expectation that all DHS executives, managers, and supervisors be personally involved as both participants in and supporters of MAXHR training efforts. The CHCO office, working with the assistance of outside contractors, has developed several training interventions aimed at providing these groups with the tools and information needed to champion the benefits of a performance-based culture and successfully implement MAXHR in their components.

In August 2005, DHS sponsored a 2-½ day training program for 350 to 400 of the department’s senior executives and flag officers. The program covered a range of topics, including an update on current DHS priorities; techniques and best practices for how senior leaders can effectively support and implement these priorities; as well as specific management, communication, and training approaches that can be used to support the creation of a performance-based culture. The Secretary, Deputy Secretary, and Under Secretary for Management all participated in the program, which also featured presentations from human capital and organizational change experts from outside the department. In addition to its focus on MAXHR implementation, which included both large and small group sessions wherein participants could discuss performance management and share information on practices, the course also provided a forum for the department’s top leadership and senior executives to review the then recently issued recommendations resulting from the Secretary’s Second Stage Review process.

Another training intervention sponsored by the department directly targets managers and supervisors who will be responsible for carrying out many of the key behaviors associated with the new system and whose active support is viewed by DHS as critical for achieving the transformation to a performance-based culture. The 2-½ day program focuses on developing and improving interpersonal, managerial, and other so-called soft skills. DHS expects to provide the training to approximately 12,000 managers and supervisors throughout the department.

On the component level, training has also played an important role in CBP’s effort to transform from the traditional, largely siloed approach used by its legacy agencies when protecting our borders to a new integrated concept that it believes is more in line with its current needs. Officials noted that the merger into CBP led to some resistance from employees who had not yet understood or accepted the reasons for the merger. These same officials acknowledged that they must continue to work at informing employees why changes were made and provide vehicles for better integration through training. For example, in the “One Face at the Border” initiative, supervisory training has incorporated some elements of cultural integration by including a session on bridging the culture gap. Officials at CBP designed and piloted a training module to be added to the supervisory curriculum specifically targeting how they can more effectively understand the value and perspective of staff coming out of the legacy organizational cultures.

In addition, training played a key role in facilitating the transition of CBP’s workforce from its three legacy organizations. Training for the new CBP officer and CBP agriculture specialist positions aimed to improve coordination and communication across inspection functions and enhance the flexibility of CBP’s workforce. Specifically, CBP created a series of training courses to provide former Customs and former Immigration and Naturalization Service officers with the knowledge and skills necessary to carry out the responsibilities of this new position. To develop this training, CBP-wide working groups identified and validated critical tasks for the new frontline CBP officer to perform. A mix of training delivery methods were used (i.e., e-learning and instructor led), and classroom knowledge and skills were reinforced with on-the-job training. CBP provided extensive train-the-trainer courses so that trainers could return to their field sites and instruct officers there. (See app. 2.)

Conclusions

DHS must continue to make progress on three important aspects of training as it moves forward: (1) forging an effective role for training at the departmental level and implementing its departmentwide training strategy; (2) taking a strategic approach to training practices, in part by building upon examples of good practice to be found among its former organizations, as well as considering other examples of strategic practices; and (3) finding ways that training can help to foster organizational transformation and cultural change within the department. To date, DHS has taken positive steps in these areas, yet significant challenges lie ahead.

The ability to make decisions from a departmentwide perspective and then effectively implement them will help determine whether training in DHS achieves its intended results. Strong leadership will play a critical role in this process. To be successful, DHS will need to have both a clear plan of action as well as the ability to anticipate and overcome several implementation challenges. The creation of the TLC and the development of the department’s first training strategic plan both represent a good start in this process. Better performance measures, more specific milestones, and the inclusion of performance targets would make DHS’s strategic training plan a more useful tool for both internal and external stakeholders to use in tracking the department’s progress toward achieving its training objectives. Clarifying authority relationships between the CHCO and component heads, developing detailed implementation plans, and giving appropriate attention to providing resources to implement training initiatives when setting funding priorities are also likely to be critical factors in building and sustaining an effective role for department-level training at DHS.

A strategic approach toward training is also very important as DHS seeks to build on its current efforts and strives to move forward. As we have noted, some programs and components in DHS already use specific strategic training practices, and other components within the department can benefit from their example. As DHS implements new training programs, such as the large-scale, multistage training being developed to support the implementation of MAXHR, it has a valuable opportunity to reflect the lessons learned from these experiences in subsequent departmentwide training efforts. Finally, the transition to a new department has brought with it cultural challenges, and training can play a role in both defining and refining an effective DHS culture without sacrificing the cultural history of its components.

Recommendations for Executive Action

To help DHS establish and implement an effective and strategic approach to departmentwide training, we recommend that the Secretary of Homeland Security take the following actions: adopt additional good strategic planning and management practices to enhance the department’s training strategic plan by (1) creating a clearer crosswalk between specific training goals and objectives and DHS’s organizational and human capital strategic goals and (2) developing appropriate training performance measures and targets; clearly specify authority and accountability relationships between the CHCO office and organizational components regarding training as a first step to addressing issues DHS has identified for fiscal year 2006; ensure that the department and component organizations develop detailed implementation plans and related processes for training initiatives; and when setting funding priorities, give appropriate attention to providing resources to support training councils and groups to further DHS’s capacity to achieve its departmentwide training goals.

Agency Comments

We provided a draft of this report to the Secretary of Homeland Security for comment and received written comments from DHS that are reprinted in appendix III. In addition, we received technical comments and clarifications, which we incorporated where appropriate. DHS generally agreed with our recommendations.

We will provide copies of this report to the Secretary of Homeland Security and other interested parties. Copies will also be provided to others upon request. In addition, this report is available at no charge on the GAO Web site at http://www.gao.gov.

If you have any questions about this report, please contact me at (202) 512- 9490 or stalcupg@gao.gov. Major contributors to this report were Kimberly Gianopoulos, Assistant Director; Peter J. Del Toro; Robert Yetvin; and Gerard Burke.

Appendix I: Scope and Methodology

To achieve our objectives, we reviewed training at the Department of Homeland Security (DHS) at both the departmental and component levels. When examining training at the departmental level, we collected, reviewed, and analyzed the department’s training rules, procedures, policies, and organizational charts; departmental, human capital, and training strategic plans; human capital and training management directives; Internet and intranet Web pages; and other relevant documents. To further our understanding of training at DHS and the issues and challenges involved, we interviewed training and human capital officials in the Office of the Chief Human Capital Officer and the leaders and coleaders of DHS’s training councils and groups. We also observed the January 2005 meeting of the Training Leaders Council. We supplemented our review of departmental training at DHS by examining the department’s effort to use training related to MAXHR to foster transformation and cultural change in the department.

In addition, we reviewed training at major organizational components in DHS and selected the six largest components based on staff size and budget. Using these criteria, we reviewed training at Customs and Border Protection (CBP), Citizenship and Immigration Services, Immigration and Customs Enforcement (including the Federal Air Marshal Service, the Federal Protective Service, and the Leadership Development Center), the Coast Guard, the Secret Service, and the Transportation Security Administration. See figure 5 for a depiction of the DHS organizational structure in place during the time of our review. These components collectively represent about 95 percent of the total staff at DHS. We also included the Federal Law Enforcement Training Center because of the special role it plays in training employees from other DHS components.

When examining training at selected components, we reviewed component-level strategic, human capital, and training plans when available; training budget requests and expenditure documents; training procedures, policies, and organizational charts; rules and policies for identifying and prioritizing training programs; Internet and intranet Web pages; selected training course materials; and other relevant documents produced by these components. To further our understanding of training at the component level, we also interviewed training officials at each of the selected components and identified these individuals based on their knowledge, experience, and leadership roles. We conducted our interviews at component headquarters or field offices located in the Washington, D.C., area. In addition, as part of our review of DHS’s efforts to foster transformation and cultural change, we observed training related to CBP’s “One Face at the Border” initiative in northern Virginia.

To help determine whether DHS used a strategic approach in planning and evaluating its training activities at the departmental or component levels, we referenced criteria contained in our guide for assessing strategic training and development efforts in the federal government. This guide outlines a framework for assessing training efforts, consisting of a set of principles and key questions that federal agencies can use to ensure that their training investments are targeted strategically and not wasted on efforts that are irrelevant, duplicative, or ineffective. We selected our case examples based on their suitability for demonstrating specific strategic training practices. Other components within DHS may, or may not, be engaged in similar practices. To determine whether DHS followed leading management practices in planning and implementing departmentwide training, we also drew on our previous work on strategic planning and effective management practices.

We did not include within our scope training intended for audiences external to DHS, and we generally covered training and training management in effect during the period in which we did our work. We conducted our work from November 2004 through July 2005 in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from DHS, which are reprinted in appendix III. The comments are addressed in the Agency Comments section of this report.

Appendix II: “One Face at the Border” Training at U.S. Customs and Border Protection

One of the initial goals for creating DHS was to better protect the United States from terrorists entering the country, and ports of entry are the means through which terrorists can enter. The creation of CBP within DHS merged border inspection functions at U.S. ports of entry, which had previously been performed by three separate agencies. Known as “One Face at the Border,” this initiative created the positions of CBP officer and CBP agriculture specialist that combined aspects of three former inspector functions. This initiative aimed to improve coordination and communication of inspections to better protect the nation’s borders from terrorists as well as to improve entry for legitimate travel and trade.

To successfully make the transition to these new positions, significant training was needed. Specifically, CBP created a series of training courses to provide former U.S. Customs and former Immigration and Naturalization Service officers with the knowledge and skills necessary to carry out the responsibilities of this new position. In addition, CBP officers received training to meet CBP’s new mission priority of terrorism prevention. Although the emphasis was on cross-training legacy officers, the new curriculum was also geared to new hires. Because agricultural inspections are more specialized, CBP officers receive training sufficient to enable them to identify potential agricultural threats, make initial regulatory decisions, and determine when to make referrals to CBP agriculture specialists. More detailed agricultural inspections are performed by these specialists who have substantial training and background in agricultural issues.

A variety of training delivery methods were used (e.g., e-learning and classroom) and these training methods were reinforced with extensive on- the-job training. In addition to traditional content areas (e.g., cross-training for former U.S. Customs officers includes courses on immigration fundamentals and immigration law), training courses also covered CBP’s new priority mission of preventing terrorism (e.g., training in detecting possible terrorists and fraudulent documents, honing interviewing skills, and making appropriate referrals to staff for additional inspection). CBP emphasizes on-the-job training in an effort not to place inspectors on the job without direct supervisory and tutorial backup. Training for new recruits has also been modified to include a preacademy orientation program at the port location where the recruit will eventually work before he or she receives academy training. This is a 72-day course for CBP officers and a 46-day course for CBP agriculture specialists.

CBP’s main strategy to prepare for field delivery of training was to provide extensive train-the-trainer courses so that trainers could return to their field sites and instruct officers there. Training priorities were established with the idea of spacing the training out so that field offices would not be overwhelmed. For example, CBP rolled out its primary cross-training to airports, while antiterrorism training was rolled out to land borders.

Officials reported that cross-training benefited CBP officers since they have gained more knowledge by learning both immigration and customs laws and procedures. This increase in knowledge has the potential benefit of providing more variety in job tasks as well as increasing the opportunities for advancement since an officer can now apply for supervisory-level positions that had previously been open only to former U.S. Customs or Immigration and Naturalization Service officers.

Change has not come about without challenges, however, as many officers were reported to have resisted changes to their responsibilities, mainly related to the difficulties in learning a new set of procedures and laws. Officials noted that there has been an enormous amount of required training for CBP officers, and it can sometimes be overwhelming. For former officers, in addition to completing an extensive cross-training schedule and new training related to terrorism prevention, there are many other required courses related to their mission. For example, training modules are required in areas such as body scanning, hazardous materials, cargo inspection, and seized assets.

Although staffing challenges may ultimately be relieved with trained officers able to perform dual inspections, officials noted that it has been extremely difficult to take staff off-line to complete the “One Face at the Border” training. One official said that classes have been very difficult to schedule because of the constant pressure to staff operations. For example, in one case, a class was canceled right after it began because the trainees were pulled out to staff their inspection booths. This official also noted that trainers have had to be very flexible to accommodate staff schedules to ensure that training occurs.

Appendix III: Comments from the Department of Homeland Security