Department of Homeland Security: Strategic Management of Training Important for Successful Transformation
Published: 2005-09-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Training can play a key role in helping the Department of Homeland Security (DHS) successfully address the challenge of transformation and cultural change and help ensure that its workforce possesses the knowledge and skills needed to effectively respond to current and future threats. This report discusses (1) how DHS is addressing or planning to address departmentwide training and the related challenges it is encountering; (2) examples of how DHS training practices, specifically those related to planning and evaluation, reflect strategic practices; and (3) examples of how DHS uses training to foster transformation and cultural change.
section_title: What GAO Found
paragraphs: DHS has taken several positive steps toward establishing an effective departmentwide approach to training, yet significant challenges remain. Progress made in addressing departmentwide training issues, but efforts are still in the early stages and face several challenges. Actions taken by DHS include issuing its first training strategic plan in July 2005, establishing training councils and groups to increase communication across components, and directly providing training for specific departmentwide needs. However, several challenges may impede DHS from achieving its departmental training goals. First, the sharing of training information across components is made more difficult by the lack of common or compatible information management systems and a commonly understood training terminology. Second, authority and accountability relationships between the Office of the Chief Human Capital Officer and organizational components are not sufficiently clear. Third, DHS's planning may be insufficiently detailed to ensure effective and coordinated implementation of departmentwide training efforts. Finally, according to training officials, DHS lacks resources needed to implement its departmental training strategy. Examples of planning and evaluation of training demonstrate some elements of strategic practice. Specific training practices at both the component and departmental levels may provide useful models or insights to help others in DHS adopt a more strategic approach to training. We found that some components of DHS apply these practices, while others do not. For example, Customs and Border Protection (CBP) aligns training priorities with strategic goals through planning and budgeting processes. In the area of evaluation, the Federal Law Enforcement Training Center obtains feedback from both the trainee and the trainee's job supervisor to inform training program designers in order to make improvements to the program curriculum. Training has been used to help DHS's workforce as it undergoes transformation and cultural change. The creation of DHS from different legacy organizations, each with its own distinct culture, has resulted in significant cultural and transformation challenges for the department. At the departmental level, one of the ways DHS is addressing these challenges is by encouraging the transformation to a shared performance-based culture through the implementation of its new human capital management system, MAXHR. DHS considers training to be critical to effectively implementing this initiative and defining its culture. Toward that end, the department is providing a wide range of training, including programs targeted to executives, managers, and supervisors. For example, at the component level, CBP has developed cross-training to equip employees with the knowledge needed to integrate inspection functions once carried out by three different types of inspectors at three separate agencies.

section_title: Letter
paragraphs: Addressing an organization’s culture—that is, its underlying assumptions,  beliefs, values, attitudes, and expectations—is at the heart of any serious  organizational transformation or change management initiative. As the  Department of Homeland Security (DHS) strives to protect the nation from  terrorism, it faces significant challenges in transforming from 22 separate  agencies and programs to a single coordinated department, requiring the  integration of approximately 180,000 employees as well as multiple  management systems and processes. In recognition of these challenges,  we have designated the implementation and transformation of the  department as high risk.
Training and development can play a key role in helping DHS successfully  address the challenge of transformation and cultural change and help  ensure that its workforce possesses the knowledge, skills, and  competencies needed to effectively respond to current and future threats.  In addition, a strategic approach to the management of training can help to  effectively target limited resources and further the achievement of its  organizational goals. To this end, our framework for assessing training  management provides a way for DHS to recognize and develop such an  approach.
You asked us to examine how DHS, as a federal agency undergoing  transformation, uses training to help achieve its organizational goals. In  response to your request, this report discusses (1) how DHS is addressing  or planning to address departmentwide training and the related challenges  it is encountering; (2) examples of how DHS training practices, specifically  those related to planning and evaluation, reflect strategic practices; and  (3) examples of how DHS uses training to foster transformation and  cultural change.
To achieve our objectives, we reviewed internal training at both the  departmental and component levels, focusing on the systems and  processes in place to manage, plan, and evaluate training for DHS’s  workforce. To this end, we analyzed training, management, and planning  documents and interviewed numerous officials responsible for training  issues in the Office of the Chief Human Capital Officer (CHCO office) and  at six organizational components collectively responsible for training 95  percent of the DHS workforce. To determine whether DHS used a strategic  approach in managing, planning, and evaluating its training activities, we  drew on our previous work regarding strategic planning and effective  management practices, as well as criteria contained in our guide for  assessing strategic training and development efforts in the federal  government. We recognize that DHS provides a significant amount of  training to external audiences, such as state and local governments;  however, given the nature of your request and interest in examples of how  DHS is using training to foster its organizational transformation, we did  not include training intended for audiences external to DHS within the  scope of our review. For more information on our scope and methodology,  see appendix I.
We conducted our work from November 2004 through July 2005 in  accordance with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: The Homeland Security Act of 2002 created DHS, bringing together 22  agencies and programs responsible for important aspects of homeland  security. The intent behind the creation of a single department was to  improve coordination, communication, and information sharing among  these previously separate entities, thereby increasing their effectiveness in  protecting the nation’s security. Each of these organizations brought with  it the capacity and expertise to provide training for its particular aspect of  homeland security. For example, in several cases such as the Coast Guard  and FLETC, this training capacity, as well as the management systems  supporting it was transferred intact with the creation of the new  department. In other cases, such as CBP and U.S. Immigration and  Customs Enforcement (ICE), the training functions of legacy organizations  were merged. Table 1 presents information on selected training  characteristics of components in our review, including the origin of each  component’s training function. In addition, the Act led to the creation of  the CHCO position in DHS responsible for, among other human capital  topics, oversight and planning of the training of employees. The CHCO,  who reports directly to the department’s Under Secretary for Management,  has primary responsibility for defining and developing the department’s  role regarding training. Figure 1 depicts these positions as well as the  department’s major components in the context of DHS’s overall  organizational structure.
Training both new and current staff to fill new roles and work in different  ways will play a crucial part in the ability of federal departments and  agencies, such as DHS, as they work to successfully transform their  organizations. In 2004, we issued an assessment guide that introduces a  framework for evaluating the management of training in the federal  government. As presented in our guide, the training process can be  segmented into four broad, interrelated phases: (1) planning/front-end  analysis, (2) design/development, (3) implementation, and (4) evaluation.  For each of these phases, we summarize key attributes of effective training  programs and offer related issues and questions. Using this framework,  this report identifies selected strategic training practices, with a focus on  the planning and evaluation phases, that may offer an opportunity for  others in DHS to build on experiences and practices discussed below.
The results of a governmentwide survey conducted by the Office of  Personnel Management in 2004 on human capital practices and employee  attitudes suggest that efforts to identify and build upon examples of good  training practice within DHS may be particularly relevant. For each of the  eight questions in the 2004 Federal Human Capital Survey that focused on  training related topics, the percentage of DHS respondents providing  positive responses (typically the top two options on a five-point scale) was  lower than the governmentwide average. In fact, the DHS response ranged  from 5 to 20 percentage points lower than the governmentwide average for  the same questions. For example, 54 percent of respondents at DHS  indicated that they received the training they needed in order to perform  their jobs, compared to 60 percent governmentwide. Half (50 percent) of  DHS respondents said that they were either satisfied or very satisfied with  the training they received for their present jobs, as opposed to 55 percent  that expressed these levels of satisfaction governmentwide. The largest  difference involved having electronic access to learning and training  programs, where 51 percent of DHS respondents responded positively,  compared to 71 percent governmentwide. A DHS official told us that the  department is aware of the challenges reflected in these data and is  currently exploring options with the Office of Personnel Management to  conduct further analysis. The aim of this work would be to identify areas  where DHS might target additional attention as well as provide a baseline  for future attitude measures.
section_title: DHS Has Made Progress in Addressing Departmentwide Training Issues, but Challenges May Impede Its Efforts
paragraphs: DHS has made progress in addressing departmentwide training issues and  these efforts reflect some of the elements of a strategic approach toward  training as described in our previous work. Most training-related activities  at DHS—such as planning, delivery, and evaluation—primarily take place  at the component level and relate to mission issues. Therefore, any  successful approach regarding departmentwide training issues will require  the concerted and coordinated efforts of multiple components within DHS  as well as the ability of the CHCO to effectively lead a network of different  training organizations. The department’s current efforts, although  promising, are still in the early stages and they face significant challenges.  Unless these challenges are successfully addressed they may impede  DHS’s ability to achieve its departmentwide training goals.
subsections: section_title: DHS Recently Developed a Departmental Training Strategy
paragraphs: DHS recently developed a coordinated departmental training strategy that  supports broader human capital and organizational goals and objectives.  We have previously reported that effective organizations establish clear  goals with an authority structure able to carry out strategies and tactics,  that is, the day-to-day activities needed to support the organization’s vision  and mission. By so doing, a well-designed training function can be directly  linked to the organization’s strategic goals and help to ensure that the  skills and competencies of its workforce enable the organization to  perform its mission effectively.
DHS’s department-level training strategy is presented in its human capital  and training strategic plans. Issued in October 2004, its human capital  strategic plan includes selected training strategies, such as developing a  leadership curriculum to ensure consistency of organizational values  across the department and using training to support the implementation of  the new DHS human capital management system, MAXHR. In July 2005,  DHS issued its first departmental training plan, Department of Homeland  Security Learning and Development Strategic Plan, which provides a  strategic vision for departmentwide training. This plan is a significant and  positive step toward addressing departmentwide training challenges. The  plan identifies four short-term goals for fiscal year 2006 and one long-term  goal for fiscal years 2006 through 2010. Among the short-term goals are  such tasks as defining the scope of training activities and improving the  governance process between the CHCO office and individual  organizational components, supporting the rollout of MAXHR,  identifying/implementing best practices, and addressing specific concerns  regarding DHS’s training facilities and advanced distributed learning  studies. The plan also articulates a long-term goal for DHS to “become a  recognized world-class learning organization where managers and  supervisors effectively lead people.”
Each of these goals is followed by supporting strategies and tactics. For  example, to achieve its goal of ensuring the best use of training resources  through the identification and implementation of best practices, the plan  identifies specific strategies, one of which is to improve the awareness of  ongoing DHS training activities among organizational components. This  strategy is, in turn, supported by still more specific tactics such as  developing a site on the DHS Interactive system to facilitate the sharing of  information across the training community.
More significant than the fact that DHS issued a training strategic plan  document is the fact that DHS followed an inclusive and collaborative  process while developing it. We have previously reported that for high- performing, results-oriented organizations, a strategic plan is not simply a  paper-driven exercise or onetime event, but rather the result of a dynamic  and inclusive process wherein key stakeholders are consulted and  involved in the identification of priorities and the formation of strategies.  When creating its plan, DHS consulted training leaders at components  throughout the department, in addition to others, to help develop and  review its content. Several training leaders we spoke with thought highly  of this process and the extent to which it provided them opportunities to  contribute and comment on the draft plan.
DHS has made considerable progress in addressing departmentwide  training issues through the development of its first training strategic plan.  However, there are areas where future efforts can be improved.
Linkage to DHS organizational and human capital strategic plans.  Our past work on strategic planning and management practices shows that  effective strategic plans describe the alignment between an agency’s long- term goals and objectives and the specific strategies planned to achieve  them. Clearly linking training tactics with particular organizational  objectives creates a direct line of sight that can both facilitate the ability of  staff to work toward mission goals and enable stakeholders to provide  meaningful oversight. In the introduction to the DHS training strategic  plan, the department’s CHCO highlights the value of this practice stating  that “the key purpose of  plan is to align our education, training and  professional development efforts with the President’s Management  Agenda and the Department’s vision, mission, core values and strategic  plan.”
The DHS training strategic plan contains examples of goals, strategies, and  tactics that align with and support goals found in the department’s human  capital and organizational strategic plans; however, these linkages are  never actually identified or discussed in the plan itself. For example, the  DHS training strategic plan contains a goal and several tactics related to  MAXHR training. These, in turn, support a MAXHR goal and strategy in the  department’s human capital strategic plan as well as the “organizational  excellence” goal of the DHS strategic plan. However, the training strategic  plan does not show these linkages. Identifying such linkages, either in the  training plan itself or in an appendix, would more clearly communicate to  both internal and external stakeholders the connections and justifications  for specific training goals, strategies, and tactics.
DHS’s own human capital strategic plan provides an illustration of one  way to communicate linkages between goals and strategies contained in  the plan and the broader organizational goals they are intended to support.  For example, in an appendix, the DHS human capital strategic plan  contains a matrix that directly links strategies, such as developing a new  Senior Executive Service (SES) performance management system, with  specific objectives contained in the DHS strategic plan as well as the  President’s Management Agenda human capital standards for success.
Usefulness of performance measures. We have previously reported  several key characteristics of effective strategic and management plans,  including the need for performance measures. Appropriate performance  measures along with accompanying targets are important tools to enable  internal and external stakeholders to effectively track the progress the  department is making toward achieving its training goals and objectives.  To this end, organizations may use a variety of performance measures— output, efficiency, customer service, quality, and outcome—each of which  focuses on a different aspect of performance.
The DHS training strategic plan contains few specific performance  measures for its goals or strategies and all of these are output measures.  For example, the plan makes use of output measures in its requirement  that certain actions, such as the development of a new management  directive or the chartering of a team, be completed by the end of fiscal  year 2006, and in establishing a deadline for when reports need to be  completed in order to be included in the 2007 plan. In contrast to output  measures like these, which gauge the level of activity or effort by  measuring whether a particular thing is produced or service performed,  other types of measures, such as measures of customer satisfaction or  program outcomes, focus on the impact or results of activities. By  appropriately broadening the mix of measures it uses and more clearly  identifying targets against which DHS can assess its performance, DHS  can improve the usefulness of its plan. After we completed our audit work,  DHS training officials informed us that they decided to delay the  development of performance measures until the rollout of the plan, when  they could be developed by individual teams, as needed. They  subsequently informed us that these teams will be held accountable to  establish further performance measures that are outcome based and  results oriented.
DHS’s human capital strategic plan again provides an illustration of how  the department’s training strategic plan might begin to work toward the  inclusion of different types of performance measures. For example,  accompanying the strategy that DHS assess the feasibility of establishing a  21st Century Leadership Training and Development Institute, the plan  identifies two performance measures—customer satisfaction and cost of  delivery—along with specific targets for each. For the customer  satisfaction measure, the plan establishes a target of 4.5 on a scale from 1  to 5. The plan also includes specific tactics to achieve the strategy, such as  developing and obtaining cross-organizational support, developing  measures and methodologies for leadership training, and implementing a  learning management system, along with key milestone dates for  completing them.
The department may benefit from considering the experiences of leading  organizations regarding the development of results-oriented performance  measurement. In general, results-oriented organizations we have studied  that were successful in measuring their performance developed measures  that were  tied to program goals and demonstrated the degree to which the desired  results were achieved,  limited to the vital few that were considered essential to producing data  for decision making,  responsive to multiple priorities, and  responsibility linked to establish accountability for results.
Similar to the consultative process DHS followed when developing the  goals and strategies contained in its training strategic plan, decisions  concerning the selection of an appropriate set of performance measures  should also be based on input from key stakeholders to determine what is  important to them when assessing the department’s performance  regarding training. Clear and appropriate performance measures,  developed in this way, can also provide DHS with valuable information,  especially significant in the current fiscal environment, when it seeks to  justify requests for resources from Congress.
section_title: Training Councils and Groups Created by DHS Increase Communication across Components
paragraphs: Under the overall direction of the CHCO office, DHS has established a  structure of training councils and groups that cover a wide range of issues  and include representatives from each organizational component within  DHS. The department is in the process of using these bodies to facilitate  communication and the sharing of information within its diverse training  community. In some instances, these councils and groups foster greater  collaboration and coordination on training policies, programs, and the  sharing of training opportunities. We have previously reported that  agencies with a strategic approach to training recognize the importance of  having training officials and other human capital professionals work in  partnership with other agency leaders and stakeholders on training  efforts.
The Training Leaders Council (TLC) plays a vital role in DHS’s efforts to  foster communication and interchange among the department’s various  training communities. This council consists of senior training leaders from  each of the department’s components as well as representatives from  several department-level headquarters staff and support organizations  with an interest in training-related issues. Started in October 2004 and  formally chartered by the CHCO in March 2005, the TLC’s mission is to  establish and sustain a collaborative community with the aim of promoting  high-quality training, education, and development throughout DHS. To this  end, it functions as a convener of training leaders from throughout the  department and provides an overarching framework for several  preexisting training groups and councils that were reestablished as  standing committees of the TLC. Membership of the TLC consists of senior  training leaders from each DHS component. In addition, most of these  leaders as well as other training staff serve on one or more of its  subgroups. See figure 2 for descriptions of the TLC and each of its  subgroups.
One key function of the TLC and these other training groups is to serve as  a “community of practice” wherein officials can discuss common training  challenges and share knowledge and best practices. For example, the  Training Evaluation and Quality Assurance Group, composed of DHS  training professionals responsible for evaluating and ensuring the quality  of DHS training programs, conducted an informal survey of evaluation  practices in various components with the intent of identifying effective  evaluation approaches. A training official involved in the group told us that  this survey was particularly important for the department’s newer  organizations, such as the Directorate for Information Analysis and  Infrastructure Protection, which need to establish new practices from  scratch. According to this official, his directorate and other organizations  within DHS plan to use the group as a way to tap into the experience of  other components within the department, such as CBP and FLETC, which  have considerably more experience with training evaluation.
In addition to sharing information about training practices, these groups  can also provide a forum for exchanging practical information with the  goal of making more efficient use of existing resources. For example, one  training official told us that as a result of information obtained at TLC  meetings, the official became aware of the existence of free training space  available at facilities of two other components located in the Washington,  D.C., metro area. Also, as a result of participating in these meetings, the  official’s organization was able to send an additional person to the Federal  Executive Institute after becoming aware that another component had  surplus spaces and was offering them at a reduced price to other  components within DHS.
Another role carried out by the TLC is to collaborate on the formulation of  training policies and advise the department’s CHCO accordingly. For  example, the TLC, in cooperation with staff from the CHCO office and an  external contractor, conducted a survey of training sites throughout the  department in 2004. This study cataloged available physical resources and  site capacities with the aim of identifying potential opportunities to share  these resources more efficiently, consolidate unneeded or duplicative  sites, and identify other opportunities to increase training collaboration  and effectiveness. This effort resulted in a series of recommendations that  were subsequently incorporated into the department’s training strategic  plan.
The activities of the department’s Advanced Distributed Learning Group  (ADLG) provides another example of how training officials from different  components have worked together to develop proposals for solutions to  departmentwide challenges. This group identified several issues in the  area of technology and learning, including the need for a compatible IT  infrastructure across components and the fact that some components  lacked established systems with which to coordinate and manage training  opportunities and attendance. Working with an outside consultant, the  ADLG’s efforts resulted in a proposal that DHS create a new Advanced  Distributed Learning (ADL) Program Management Office to oversee the  process of setting common standards. This proposal was subsequently  included as part of the department’s training strategic plan. In addition, the  ADLG’s work also led to DHS entering into a memorandum of  understanding with the Office of Management and Budget and the Office  of Personnel Management to create a DHS headquarters learning  management system. Throughout this process, the ADLG represented the  interests of the DHS training community as it worked with representatives  from the Chief Information Officer’s office and other functions within the  department, as well as outside consultants.
Despite these positive steps, DHS’s effort to foster communication and  coordination through departmentwide training councils and groups is at a  relatively early stage and so far has produced varied results. Some training  organizations, such as the TLC and ADLG, have met regularly leading to  tangible results, while others such as the Training Evaluation and Quality  Assurance Group, have met a few times and have only begun to set the  groundwork for substantive coordination and collaboration in these areas.  In addition, a training official told us that even active organizations like the  TLC have encountered difficulties related to the relative lack of staff  support for these efforts. As a result, additional burdens sometimes fall to  the leaders and members of these groups who, in addition to serving on  one or more departmental training groups or councils, must carry out full- time training positions at their home components.
section_title: DHS Provides Training to Address Selected Departmentwide Needs
paragraphs: Another way DHS addresses departmentwide training issues is to directly  provide training interventions or resources that address selected  departmentwide needs, goals, or objectives. Three examples of the areas  where DHS has worked to directly provide or support training on the  departmental level are the following: (1) training related to the  implementation of MAXHR, (2) DHS leadership development, and   (3) training related to civil rights and civil liberties.
Training for MAXHR implementation. DHS’s new human capital  management system, known as MAXHR, represents a fundamental change  in many of the department’s human capital policies and procedures that  will affect a large majority—approximately 110,000—of its civilian  employees. MAXHR covers many key human capital areas, such as pay,  performance management, classification, labor relations, adverse actions,  and employee appeals, and will be implemented in phases affecting  increasing numbers of employees over the next several years.
DHS correctly recognizes that a substantial investment in training is a key  aspect of effectively implementing MAXHR, and in particular, the new  performance management system it establishes. The need for in-depth  performance management and employee development training is further  supported by the department’s results on the 2004 Federal Human Capital  Survey. In this survey, just over half of DHS respondents—51 percent— believe supervisors or team leaders in their work units encourage their  development at work, significantly less than the governmentwide response  of 64 percent. DHS officials said they plan to educate all affected DHS  employees on the details of the new system, how it will affect them, and  the purpose of the changes. To do this, the department decided to develop,  coordinate, and manage MAXHR training centrally through the CHCO office  and offered its first training in May 2005. DHS plans to continue to provide  its workforce with MAXHR training over the next several years following a  phased approach that takes into account both when individual provisions  of the new regulations take effect as well as the different audiences that  exist within the DHS community, including human capital personnel,  supervisors, and general employees. See figure 3 for a depiction of planned  training during 2005 and its intended audiences.
The department has worked with contractors to develop training that uses  a variety of approaches, including classroom instruction, ADL, handbooks,  manuals, and quick reference guides, depending on specific needs. For  example, in May 2005, labor relations/employee relations specialists and  attorneys in the department received 2-½ days of training on the  provisions of the new regulations and the major difference between them  and previous programs. Structured as a “train the trainer” type  intervention intended to prepare participants to conduct supervisor  briefings in their own components, this was an instructor-led course held  at sites across the country. In addition to educating individuals about the  regulations, procedures, and systems associated with MAXHR and the  adoption of a new performance management system, the department also  plans to offer training specifically targeted to developing the skills and  behaviors that will be necessary for its successful implementation. For  example, in July 2005 supervisors began to receive training on techniques  for providing meaningful feedback to, coaching, and mentoring employees.
DHS leadership development training. Leadership development is  another area top management in DHS acknowledged as appropriate for  departmentwide training to supplement existing component-level  offerings. In 2004, the Secretary of Homeland Security announced the  “One DHS” policy that identified the need to establish a common  leadership competency framework for the department, as well as a unified  training curriculum for current and future leaders. The purpose of this  framework was to identify the skills, abilities, and attributes necessary for  success as a DHS leader and to establish measurable standards for  evaluation.
To this end, the CHCO established the DHS Leadership Training and  Development Group (LTDG), comprising training officials from each DHS  component who combined an expertise in leadership development with  personal knowledge of the missions and unique aspects of their particular  organizational components. The LTDG met regularly from late 2003 to  mid-2004. During this time, the group developed a set of new core  leadership competencies for DHS supervisors, managers, and executives,  which it issued in April 2004. According to a DHS official, since the  development of these new competencies they have been used by one  component as part of its own leadership development plan and they have  also helped to guide and inform current MAXHR leadership development  efforts.
DHS has recently taken steps regarding another facet of its leadership  development initiative—its SES Candidate Development Program. In June  2005, DHS issued a management directive establishing the SES Candidate  Development Program, which included a rigorous selection process and  critical leadership development opportunities, such as mentoring,  developmental assignments, and action learning designed to give SES  candidates experience in different job roles. DHS initially announced that  it planned to implement the program in fiscal year 2005, but now may  delay doing so until fiscal year 2006.
Civil rights/civil liberties training. A third area in which DHS has taken  steps to provide or support departmental training involves civil rights and  civil liberties. FLETC’s Behavioral Science Division and Legal Division,  working with the DHS Office of Civil Rights and Civil Liberties, produced  several training interventions, including Web-based, CD-ROM, and in- person programs designed to increase sensitivity and understanding in  protecting human and constitutional rights. As part of this effort, FLETC  held diversity seminars that focused on promoting understanding and  respect of religious practices, particularly involving those of the Arab and  Muslim communities. In another example of this effort, the Office of Civil  Rights and Civil Liberties produced Web-based training on current policies  regarding racial profiling.
Our interviews with DHS training leaders suggest that further  improvements can be made in communicating the availability of selected  departmentwide training programs and resources. Staff at the Office of  Civil Rights and Civil Liberties provided copies of its civil rights and  liberties programs to training offices at each component in the  department. While some senior training officials told us that their  components actively disseminated this material by placing it on the  component’s training Web site or incorporating it into preexisting courses,  other senior training officials we spoke with were unaware of any  departmental training on these topics. In addition, other officials told us  that their component’s training office had independently developed its  own material on Arabic sensitivity training, wholly apart from similar  efforts undertaken by others in the department. More specifically, they  told us that their development of certain training modules predated the  development of very similar modules later prepared by DHS’s Office of  Civil Rights and Civil Liberties and FLETC, leading these officials to  conclude that they may have been able to assist departmental efforts by  sharing their work had they been aware of them.
section_title: Challenges May Impede Achievement of Departmentwide Training Goals
paragraphs: As DHS moves forward, it faces challenges to achieving departmentwide  training goals. These challenges include lack of common management  information systems, the absence of commonly understood training  terminology across components, the lack of specificity in authority and  accountability relationships between the CHCO office and components,  insufficient planning for effective implementation, and insufficient  resources for ensuring effective implementation of training strategies.
subsections: section_title: Sharing of Training Information across Components Made More Difficult by Lack of Compatible Infrastructure and Commonly Understood Terminology
paragraphs: The formation of DHS from 22 legacy agencies and programs has created  challenges to achieving departmentwide training goals. Of particular  concern to the training officials we spoke with are the lack of common  management information systems and the absence of commonly  understood training terminology across components. The training  functions at DHS’s components largely operate as they did before the  creation of the department, with many of the same policies, practices, and  infrastructures of their former organizations, and within these  organizations are, for the most part, the same training leaders. It will take  time for these organizations to evolve into a coordinated, integrated  department. We have previously reported that successful transformations  of large organizations, even those faced with less strenuous  reorganizations and pressure for immediate results than DHS, can take  from 5 to 7 years to fully take hold.
One issue DHS officials raised was the lack of common or compatible  management information systems, such as information technology or  financial management, which can inform decision makers’ efforts to make  efficient use of training resources across components. For example, DHS  officials stated that a key challenge they encountered involved the  difficulty of knowing what others were doing outside their particular  offices or components. DHS lacks any unified sourcebook that employees  could consult for the names, telephone numbers, and other relevant  information of key contact persons in areas such as acquisition. Obtaining  accurate information about resources and products available in the  marketplace as well as data on users, vendors, and kinds of work has been  a challenge to that effort. Another issue cited by officials concerned the  lack of compatibility between learning management systems across  components. In addition, some training officials expressed concerns about  the accuracy or timeliness of some training data, which can limit or at  least considerably delay their ability to track and fully account for funds  spent on training and training-related travel. DHS has several efforts under  way to address these issues, including the development of an online  training facilities inventory intended to increase awareness of existing  resources across the department and its decision to begin developing  common ADL policies and standards.
Officials also told us that there was little or no common understanding  among DHS organizational components regarding the meaning of such  basic terms as “subject matter expert,” “orientation,” and even “training.”  The lack of commonly understood terminology has presented challenges  when officials from different components, including those participating in  departmental training councils and groups, try to share practices with  each other. These officials told us that the lack of commonly understood  terminology can also affect their interactions with outside entities, such as  contractors and state and local agencies. Besides facilitating  communications and enabling components to share practices, a DHS  official told us that a common nomenclature would increase the  transparency of training practices to external contractors as well as the  internal DHS training community. The department’s training strategic plan  calls for the creation of a common training language and glossary of terms  in fiscal year 2006, and officials told us that they are currently in the early  stages of creating such a glossary.
section_title: Authority and Accountability Relationships between CHCO Office and Components Are Not Sufficiently Specific
paragraphs: An effective management control environment appropriately assigns  authority and delegates responsibility to the proper personnel to achieve  organizational goals and objectives. In such an environment, staff  members who are delegated responsibility are given corresponding  authority. In light of this, DHS’s adoption of a “dual accountability”  governance structure in 2004 presents certain challenges. Under this  concept, heads of organizational components and the CHCO share  responsibility for effective training in DHS. With a shared responsibility for  DHS training, both the CHCO and component heads should have  appropriate authority for making decisions regarding training. DHS does  not specify how authority for training matters will be shared between the  CHCO office and components for budgeting, staffing, and policy (e.g.,  determining which training functions, if any, should remain with  components or be performed by DHS headquarters). The DHS  management directive on training currently in place is a high-level two- page document that provides very few specifics on policies, procedures,  and authorities for the CHCO office and the components.
The department recognizes the need to clarify the responsibilities and  authorities of the CHCO office and the components, as indicated by its  inclusion in the DHS training strategic plan. Many of the tactics included in  the plan would be difficult to successfully implement without first having a  clear understanding of the responsibilities and authorities of the key  organizations involved. More specifically, in the absence of clear authority  relationships, decisions regarding how particular component training  goals and strategies are to be incorporated in the DHS training strategic  plan, or which training facilities should be consolidated to achieve  departmental efficiencies, will be difficult to make. Without moving ahead  with this effort in a timely fashion and completing the process of  specifying how the CHCO office and components will share authority over  training matters, it will be difficult for DHS to make the progress  necessary on its departmentwide training agenda if it is to effectively  implement the many strategies and tactics planned for fiscal year 2006.
In addition, DHS’s efforts at coordinating training across components and  clarifying roles and relationships between departmental functions and  organizational components may be further hampered by the fact that the  management directive governing the integration of the human capital  function claims that the Coast Guard and the Secret Service are  statutorily exempt from its application. We found no reasonable basis to  conclude that the directive could not be made applicable to them and are  not aware of any explicit statutory exemption that would prevent the  application of this directive. Moreover, exempting the Coast Guard and the  Secret Service from the provisions of this directive casts doubt on the  authority and accountability relationships between these components and  the CHCO, potentially complicating the department’s objective of  clarifying the responsibilities, accountability, and authorities of the CHCO  office and the components set forth in DHS’s training strategic plan.
section_title: Planning May Be Insufficiently Detailed to Ensure Effective Implementation in “Dual Accountability” Environment
paragraphs: In and of itself, DHS’s dual accountability authority structure is not an  obstacle to implementation of departmentwide training efforts. However,  without detailed implementation plans, it presents potentially significant  challenges. Because of this shared authority, DHS will need to take great  care when planning for departmentwide training initiatives involving  multiple organizational components to ensure that resources are aligned  with organizational units performing activities, especially related to cross- organizational sharing of training and delivery of common training. The  lack of comprehensive and rigorous planning can lead to confusion over  responsibilities, lack of coordination, and missed deadlines. Regular and  rigorous use of detailed implementation plans is necessary to implement  decisions and carry out activities in a coordinated manner.
After we completed our audit work, DHS informed us that it plans to  establish 31 tactic teams to take ownership of each of the tactics  presented in the DHS training strategic plan to be completed by the end of  fiscal year 2006. As of mid-August 2005, DHS provided us with  documentation indicating that 3 of these teams have been established to  date. These teams appear to have taken promising steps toward the  establishment of detailed plans for implementing their respective training  tactics by developing draft objectives, deliverables, and closure criteria.  But as fiscal year 2006 approaches, time is short for the CHCO office and  the components to establish the remaining teams and then take the actions  necessary to develop and put in place the detailed plans that will be  critical for effectively implementing DHS’s many training tactics by the  end of the coming fiscal year. The TLC’s ADLG has made use of this type  of detailed approach in a report proposing a distance learning architecture  for the department. Appended to its report is a detailed plan outlining the  major activities, milestones, resources, and components needed to support  the successful implementation of the proposal.
section_title: Resources Needed to Implement Departmental Training Strategy
paragraphs: Several training officials told us they were concerned about the lack of  dedicated resources and related capacity to carry out departmental  initiatives. At the time we started our review, the CHCO office had only  one full-time permanent employee dedicated to carrying out these  activities; consequently, both training leaders and staff from organizational  components were relied on to contribute to departmentwide efforts. After  we concluded our audit work, a DHS official told us that the CHCO office  had recently hired two additional full-time training staff: an ADL program  manager and a staffer to oversee a recently approved SES candidate  development program and headquarters operational leadership  development. Individual components have also provided some assistance  to departmentwide efforts through the appointment of temporary  personnel. In late 2004, CBP and FLETC each detailed a staff member to  the CHCO office to work on training-related projects. In addition, DHS has  contracted for services to address selected departmentwide issues, such  as setting common standards for ADL and reviewing DHS training  facilities.
DHS’s departmental training councils and groups are almost exclusively  staffed by component training leaders who already have full-time training  commitments. The department’s training strategic plan identifies many  tactics for fiscal year 2006—including creating a common training  language and glossary of training terms, establishing a repository for  course catalog information, and developing a DHS training Web site—that  will require considerable staff support to implement. Successful and timely  completion of these and other initiatives will depend on sufficient  resources being provided.
section_title: DHS’s Approach to Planning and Evaluation of Training Demonstrates Some Elements of Strategic Practice, but Is Still Evolving
paragraphs: It is essential for federal agencies to ensure that their training efforts are  part of—and are driven by—their organizational strategic and  performance planning processes. We have reported that aligning training  with strategic priorities and systematically evaluating training activities  play key roles in helping agencies to ensure that training is strategically  focused on improving performance and meeting overall organizational  goals. Strategic training practices in several DHS components or  programs may provide models or insights to others in the department  regarding ways to improve training practices. In areas where some  components employed strategic practices, other components did not.
subsections: section_title: Aligning Training with Organizational Priorities Is a Strategic Practice
paragraphs: We have previously reported that agencies demonstrating a strategic  approach to training align their training efforts with overall strategic  priorities. To do this, agencies can employ a variety of practices, such as  linking training activities to strategic planning and budgeting and  performing front-end analysis to ensure that training activities are not  initiated in an ad hoc, uncoordinated manner, but rather are focused on  improving performance toward the agency’s goals. Some components in  DHS applied the strategic practice of aligning training with organizational  priorities, while others did not.
CBP links its new and existing training activities to its strategic priorities  when planning for its strategic initiatives and expenditures. Importantly,  the head of training at CBP is at the decision-making table with other CBP  leaders to help establish training priorities consistent with the priorities of  the CBP Commissioner. Relevant program managers are asked, “What  training do you need to achieve the goals in your strategic plan?” Such  discussions took place during planning for CBP’s custom trade pact  initiative.
During each budget cycle, CBP’s central training office issues a “call for  training” to its mission and mission support customers to estimate CBP’s  training needs for existing training activities and prioritize these needs  based upon the Commissioner’s priorities. Prior to establishing this  process, training was mostly decided on a first come, first serve basis  without clear and transparent linkages to organizational priorities. CBP’s  current process results in an annual training plan in which training needs  are identified by priority as well as major occupational type, such as  border patrol agent or CBP officer. Training decisions are based on  whether training requested is critical, necessary, or “nice to have.” During  fiscal year training plan implementation, CBP tracks actual training  activity through a central database to determine whether CBP is using its  planned training resources. By tracking plan usage through a centrally  managed database, CBP is able to reallocate unused training funds prior to  the end of the fiscal year for either training activities that were not  included in its original plan because of capacity constraints, or for  emerging training priorities.
The Coast Guard has adopted a strategic and analytic approach to training  through its use of the Human Performance Technology (HPT) model—a  front-end training assessment process to determine the cause of  performance problems. The process starts with the assumption that many  factors influence individual and unit performance and it is important to  determine what the factors are before concluding that training is the  solution. From its HPT analysis, the Coast Guard determines whether  training is needed or whether another type of solution, such as a policy  change, would be more appropriate. For example, in addressing a problem  in aviation maintenance, a Coast Guard working group looked at likely  causes of its performance problems and concluded that focusing on  making aviation maintenance training better was not the only solution.  More specifically, training officials encountered problems with job  dissatisfaction and subpar performance from aviation chief warrant  officers. In this case, training officials used HPT to analyze the nature of  work performed by those responsible for aviation maintenance and  concluded that there was not a good match between job skills and  responsibilities. Specifically, over the last 20 years, the scope and nature of  the work performed by chief warrant officers changed significantly from  maintaining components to managing aircraft systems. Performance  problems were mainly caused by significant changes in the job functions  of these officers over the years rather than by a lack of adequate training.
In cases where the HPT analysis concludes that training is warranted, a  training analysis is performed to determine the specific training  interventions. For example, in implementing activities related to the  Maritime Transportation Security Act, the Coast Guard analyzed its  training needs through the HPT process to determine training necessary to  help maritime inspectors reduce the exposure of ports and waterways to  terrorist activities. The analysis identified the skills and knowledge  necessary for new maritime inspector tasks and provided training  interventions, such as developing job aids and targeted classes, to prepare  inspectors for the tasks most relevant to support their new role. New  courses were piloted and then subjected to multilevel evaluations to  assess their effectiveness and potential impact on employee performance.
section_title: Systematic Evaluation of Training Is a Strategic Practice
paragraphs: Agencies demonstrating a strategic approach to training employ a variety  of practices, such as systematically evaluating training, actively  incorporating feedback during training design, and using feedback from  multiple perspectives. Several components and programs we examined at  DHS used these practices, while others did not.
One commonly accepted model used for assessing and evaluating training  programs consists of five levels of assessment (see fig. 4). In our review,  virtually all components captured Level I data focusing on end-of-course  reactions, while several also collected Level II data focusing on changes in  employee skill, knowledge, or abilities. Several components evaluated, or  were planning to evaluate, the impact of selected training programs on  individual behavior, represented by Level III evaluations.
To measure the real impact of training, however, agencies need to move  beyond data focused primarily on inputs and outputs and develop  additional indicators that help determine how training efforts contribute to  the accomplishment of agency goals and objectives. At a couple of  components, DHS officials told us they conducted Level IV evaluations,  which assess the effectiveness of training interventions. We found no  examples of the department or its components measuring the return on  investment of training activities (Level V). Training effectiveness should  be measured against organizational performance; however, not all levels of  training evaluation require or are suitable for return on investment  analysis. Determining whether training programs merit the cost of using  such an approach depends upon the programs’ significance and  appropriateness.
CBP takes a systematic approach to evaluating its training activities  through its National Training Evaluation Program (NTEP) to help program  managers and trainers make more informed decisions on the effectiveness  of training courses and their delivery. Despite the fact that CBP is a large  and decentralized organization, NTEP has enabled it to collect course  evaluation information and make this information available to a wide  range of users in a timely manner. NTEP has also standardized evaluation  data to allow for comparison of training throughout various field locations.  Before the rollout of NTEP, CBP did not use a standard mechanism for  collecting evaluation data, which, according to a CBP official, made it  difficult to gather evaluation data nationally.
CBP focuses on collecting both end-of-course student reactions (Level I)  and supervisor assessments of student on-the-job performance after  attending the training (Level III). Electronic or paper-based evaluations  are entered into the NTEP information system. The “close to real time”  online data enables supervisors to perform trend analysis on training  quality and provides opportunities for them to troubleshoot training  deficiencies and identify high-performing courses. The NTEP online  system allows CBP employees access to evaluation data on a need to  know basis with four levels of access, while enabling them to locate  evaluation data for any training class by date. Evaluation reports are  aggregated for review by senior CBP officials.
A CBP official told us that collecting course evaluation data is labor  intensive, especially since many field operations still use paper processes.  In addition, CBP has experienced a relatively low submission rate for  Level I evaluation data for many of its training classes. The official told us  that this was especially true for end-of-course reactions from staff in the  field, where only about one-third of officer-related course participants  submit evaluation forms. Given cost and labor challenges, CBP has  targeted areas for evaluation that it believes are important, such as  training related to its “One Face at the Border” initiative.
In addition, agencies with a strategic approach to training do not wait until  the conclusion of a training intervention to conduct evaluations. Rather,  they approach evaluation through an iterative process capable of  informing all stages of training. DHS’s CHCO office used multiple forms  of feedback from employees to develop its training strategy for MAXHR.  From February through April 2005, the department administered surveys  and conducted focus groups to obtain information on the needs, attitudes,  and reactions of different communities affected by MAXHR. Shortly after  issuing its new human capital regulations, the department provided basic  information to all employees on the nature and timeline of changes they  could expect under MAXHR through a Web broadcast. After the broadcast  an online survey was used to obtain feedback from employees regarding  the broadcast itself and their general feelings and concerns about the  MAXHR initiative.
DHS followed this initial survey with a larger survey to gather additional  feedback on how information regarding MAXHR had been communicated,  as well as specific areas where employees wanted additional information.  Concerns about the need for training were prominent among the more  than 9,000 responses received, with respondents ranking training as the  second most serious challenge to the successful implementation of MAXHR.  According to a senior DHS official, the survey results will inform  subsequent training and communication efforts.
DHS also collected evaluative feedback by conducting a series of focus  groups held in locations across the country. The aim of these sessions was  to validate the design of the performance management program  established under MAXHR and identify concerns that would inform the  development of additional training. Consistent with the strategic training  practice of seeking out different perspectives when redesigning and  assessing training efforts, DHS staff held separate focus groups for  bargaining unit employees, non-bargaining unit employees, and  supervisors and managers at all of the locations visited. This enabled them  to identify issues of particular concern to each of these groups as well as  issues common to all three. For example, both the bargaining unit and  non-bargaining unit employee focus groups raised concerns about  supervisors having inadequate skills for fairly administering the new  performance management system. This concern was also shared by  supervisors and managers themselves who expressed the need for  additional skills training in areas such as goal setting and providing  performance feedback. The sessions validated the CHCO office’s plans to  offer performance management training to supervisors and managers  before the implementation of the new system and assisted in refining  issues for future training.
FLETC’s methods for evaluating its major training programs include  feedback from multiple perspectives when examining the benefits of  training on actual employee job performance. FLETC’s Level III  evaluations obtain feedback from both trainees and their supervisors to  inform future improvements to training curricula. Evaluation results are  compiled into a comprehensive report used during FLETC’s periodic  curriculum reviews on its major training programs, such as the Natural  Resource Police Training Program. The report contains detailed feedback  from both the trainee and supervisor perspectives 6 months to 1 year after  the trainee has attended the training program. For example, for the  Natural Resource Police Training Program, FLETC analyzed how well the  program prepared trainees in all aspects of their jobs. In this case, analysis  identified those courses that had benefited program trainees the least— including determining speed from skid marks and death notification.  Training designers can use report information to improve program  curricula and refocus training on knowledge and skill areas most critical  to performing the job. In addition to Level III evaluation results, its  training designers make program and individual class changes by using  other methods of evaluation, such as direct student feedback after classes  and trainee examinations, which determine how well the trainees  understood the course material immediately following the program.
section_title: DHS Has Used Training in an Effort to Help Its Workforce as the Department Undergoes Transformation and Cultural Change
paragraphs: The creation of DHS resulted in significant cultural and transformational  challenges for the department. We have previously reported that training is  one way organizations successfully address cultural issues while  simultaneously facilitating new ways to work toward the achievement of  organizational goals. Among the DHS components in our review, some  merged cultures from different legacy organizations (CBP, ICE), another  component came as a small organization that greatly expanded when  joining DHS (Federal Air Marshal Service), while others joined DHS intact  (Secret Service, Coast Guard, FLETC), and still another was previously a  part of a larger legacy organization (CIS). Each component faces the need  to find a way to identify itself as part of the larger DHS organization, that  is, with a sense of affiliation rather than as an outsider looking in. At the  same time, components must either maintain their existing cultures or  develop new cultures to adapt to changing missions and needs. The key is  to build upon positive aspects of the components’ cultures as the larger  organization develops its own culture.
Agencies that undergo successful transformations change more than just  their organizational charts, they also make fundamental changes in basic  operations, such as how they approach strategic human capital  management. DHS understands this, and the MAXHR initiative is part of an  effort by the department to fundamentally change its approach to human  capital management by establishing a personnel system that is flexible,  performance oriented, and market based.
The Secretary of Homeland Security and other top officials have actively  supported the role of training in implementing these changes by making it  a leadership expectation that all DHS executives, managers, and  supervisors be personally involved as both participants in and supporters  of MAXHR training efforts. The CHCO office, working with the assistance of  outside contractors, has developed several training interventions aimed at  providing these groups with the tools and information needed to champion  the benefits of a performance-based culture and successfully implement  MAXHR in their components.
In August 2005, DHS sponsored a 2-½ day training program for 350 to 400  of the department’s senior executives and flag officers. The program  covered a range of topics, including an update on current DHS priorities;  techniques and best practices for how senior leaders can effectively  support and implement these priorities; as well as specific management,  communication, and training approaches that can be used to support the  creation of a performance-based culture. The Secretary, Deputy Secretary,  and Under Secretary for Management all participated in the program,  which also featured presentations from human capital and organizational  change experts from outside the department. In addition to its focus on  MAXHR implementation, which included both large and small group  sessions wherein participants could discuss performance management and  share information on practices, the course also provided a forum for the  department’s top leadership and senior executives to review the then  recently issued recommendations resulting from the Secretary’s Second  Stage Review process.
Another training intervention sponsored by the department directly targets  managers and supervisors who will be responsible for carrying out many  of the key behaviors associated with the new system and whose active  support is viewed by DHS as critical for achieving the transformation to a  performance-based culture. The 2-½ day program focuses on developing  and improving interpersonal, managerial, and other so-called soft skills.  DHS expects to provide the training to approximately 12,000 managers and  supervisors throughout the department.
On the component level, training has also played an important role in  CBP’s effort to transform from the traditional, largely siloed approach  used by its legacy agencies when protecting our borders to a new  integrated concept that it believes is more in line with its current needs.  Officials noted that the merger into CBP led to some resistance from  employees who had not yet understood or accepted the reasons for the  merger. These same officials acknowledged that they must continue to  work at informing employees why changes were made and provide  vehicles for better integration through training. For example, in the “One  Face at the Border” initiative, supervisory training has incorporated some  elements of cultural integration by including a session on bridging the  culture gap. Officials at CBP designed and piloted a training module to be  added to the supervisory curriculum specifically targeting how they can  more effectively understand the value and perspective of staff coming out  of the legacy organizational cultures.
In addition, training played a key role in facilitating the transition of CBP’s  workforce from its three legacy organizations. Training for the new CBP  officer and CBP agriculture specialist positions aimed to improve  coordination and communication across inspection functions and enhance  the flexibility of CBP’s workforce. Specifically, CBP created a series of  training courses to provide former Customs and former Immigration and  Naturalization Service officers with the knowledge and skills necessary to  carry out the responsibilities of this new position. To develop this training,  CBP-wide working groups identified and validated critical tasks for the  new frontline CBP officer to perform. A mix of training delivery methods  were used (i.e., e-learning and instructor led), and classroom knowledge  and skills were reinforced with on-the-job training. CBP provided  extensive train-the-trainer courses so that trainers could return to their  field sites and instruct officers there. (See app. 2.)
section_title: Conclusions
paragraphs: DHS must continue to make progress on three important aspects of  training as it moves forward: (1) forging an effective role for training at the  departmental level and implementing its departmentwide training strategy;  (2) taking a strategic approach to training practices, in part by building  upon examples of good practice to be found among its former  organizations, as well as considering other examples of strategic practices;  and (3) finding ways that training can help to foster organizational  transformation and cultural change within the department. To date, DHS  has taken positive steps in these areas, yet significant challenges lie ahead.
The ability to make decisions from a departmentwide perspective and then  effectively implement them will help determine whether training in DHS  achieves its intended results. Strong leadership will play a critical role in  this process. To be successful, DHS will need to have both a clear plan of  action as well as the ability to anticipate and overcome several  implementation challenges. The creation of the TLC and the development  of the department’s first training strategic plan both represent a good start  in this process. Better performance measures, more specific milestones,  and the inclusion of performance targets would make DHS’s strategic  training plan a more useful tool for both internal and external stakeholders  to use in tracking the department’s progress toward achieving its training  objectives. Clarifying authority relationships between the CHCO and  component heads, developing detailed implementation plans, and giving  appropriate attention to providing resources to implement training  initiatives when setting funding priorities are also likely to be critical  factors in building and sustaining an effective role for department-level  training at DHS.
A strategic approach toward training is also very important as DHS seeks  to build on its current efforts and strives to move forward. As we have  noted, some programs and components in DHS already use specific  strategic training practices, and other components within the department  can benefit from their example. As DHS implements new training  programs, such as the large-scale, multistage training being developed to  support the implementation of MAXHR, it has a valuable opportunity to  reflect the lessons learned from these experiences in subsequent  departmentwide training efforts. Finally, the transition to a new  department has brought with it cultural challenges, and training can play a  role in both defining and refining an effective DHS culture without  sacrificing the cultural history of its components.
section_title: Recommendations for Executive Action
paragraphs: To help DHS establish and implement an effective and strategic approach  to departmentwide training, we recommend that the Secretary of  Homeland Security take the following actions:    adopt additional good strategic planning and management practices to  enhance the department’s training strategic plan by (1) creating a clearer  crosswalk between specific training goals and objectives and DHS’s  organizational and human capital strategic goals and (2) developing  appropriate training performance measures and targets;    clearly specify authority and accountability relationships between the  CHCO office and organizational components regarding training as a first  step to addressing issues DHS has identified for fiscal year 2006;    ensure that the department and component organizations develop detailed  implementation plans and related processes for training initiatives; and    when setting funding priorities, give appropriate attention to providing  resources to support training councils and groups to further DHS’s  capacity to achieve its departmentwide training goals.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Secretary of Homeland Security  for comment and received written comments from DHS that are reprinted  in appendix III. In addition, we received technical comments and  clarifications, which we incorporated where appropriate. DHS generally  agreed with our recommendations.
We will provide copies of this report to the Secretary of Homeland  Security and other interested parties. Copies will also be provided to  others upon request. In addition, this report is available at no charge on  the GAO Web site at http://www.gao.gov.
If you have any questions about this report, please contact me at (202) 512- 9490 or stalcupg@gao.gov. Major contributors to this report were Kimberly  Gianopoulos, Assistant Director; Peter J. Del Toro; Robert Yetvin; and  Gerard Burke.
section_title: Appendix I: Scope and Methodology
paragraphs: To achieve our objectives, we reviewed training at the Department of  Homeland Security (DHS) at both the departmental and component levels.  When examining training at the departmental level, we collected,  reviewed, and analyzed the department’s training rules, procedures,  policies, and organizational charts; departmental, human capital, and  training strategic plans; human capital and training management  directives; Internet and intranet Web pages; and other relevant documents.  To further our understanding of training at DHS and the issues and  challenges involved, we interviewed training and human capital officials in  the Office of the Chief Human Capital Officer and the leaders and  coleaders of DHS’s training councils and groups. We also observed the  January 2005 meeting of the Training Leaders Council. We supplemented  our review of departmental training at DHS by examining the department’s  effort to use training related to MAXHR to foster transformation and  cultural change in the department.
In addition, we reviewed training at major organizational components in  DHS and selected the six largest components based on staff size and  budget. Using these criteria, we reviewed training at Customs and Border  Protection (CBP), Citizenship and Immigration Services, Immigration and  Customs Enforcement (including the Federal Air Marshal Service, the  Federal Protective Service, and the Leadership Development Center), the  Coast Guard, the Secret Service, and the Transportation Security  Administration. See figure 5 for a depiction of the DHS organizational  structure in place during the time of our review. These components  collectively represent about 95 percent of the total staff at DHS. We also  included the Federal Law Enforcement Training Center because of the  special role it plays in training employees from other DHS components.
When examining training at selected components, we reviewed  component-level strategic, human capital, and training plans when  available; training budget requests and expenditure documents; training  procedures, policies, and organizational charts; rules and policies for  identifying and prioritizing training programs; Internet and intranet Web  pages; selected training course materials; and other relevant documents  produced by these components. To further our understanding of training  at the component level, we also interviewed training officials at each of  the selected components and identified these individuals based on their  knowledge, experience, and leadership roles. We conducted our  interviews at component headquarters or field offices located in the  Washington, D.C., area. In addition, as part of our review of DHS’s efforts  to foster transformation and cultural change, we observed training related  to CBP’s “One Face at the Border” initiative in northern Virginia.
To help determine whether DHS used a strategic approach in planning and  evaluating its training activities at the departmental or component levels,  we referenced criteria contained in our guide for assessing strategic  training and development efforts in the federal government. This guide  outlines a framework for assessing training efforts, consisting of a set of  principles and key questions that federal agencies can use to ensure that  their training investments are targeted strategically and not wasted on  efforts that are irrelevant, duplicative, or ineffective. We selected our case  examples based on their suitability for demonstrating specific strategic  training practices. Other components within DHS may, or may not, be  engaged in similar practices. To determine whether DHS followed leading  management practices in planning and implementing departmentwide  training, we also drew on our previous work on strategic planning and  effective management practices.
We did not include within our scope training intended for audiences  external to DHS, and we generally covered training and training  management in effect during the period in which we did our work. We  conducted our work from November 2004 through July 2005 in accordance  with generally accepted government auditing standards. We requested  comments on a draft of this report from DHS, which are reprinted in  appendix III. The comments are addressed in the Agency Comments  section of this report.
section_title: Appendix II: “One Face at the Border” Training at U.S. Customs and Border Protection
paragraphs: One of the initial goals for creating DHS was to better protect the United  States from terrorists entering the country, and ports of entry are the  means through which terrorists can enter. The creation of CBP within  DHS merged border inspection functions at U.S. ports of entry, which had  previously been performed by three separate agencies. Known as “One  Face at the Border,” this initiative created the positions of CBP officer and  CBP agriculture specialist that combined aspects of three former inspector  functions. This initiative aimed to improve coordination and  communication of inspections to better protect the nation’s borders from  terrorists as well as to improve entry for legitimate travel and trade.
To successfully make the transition to these new positions, significant  training was needed. Specifically, CBP created a series of training courses  to provide former U.S. Customs and former Immigration and  Naturalization Service officers with the knowledge and skills necessary to  carry out the responsibilities of this new position. In addition, CBP officers  received training to meet CBP’s new mission priority of terrorism  prevention. Although the emphasis was on cross-training legacy officers,  the new curriculum was also geared to new hires. Because agricultural  inspections are more specialized, CBP officers receive training sufficient  to enable them to identify potential agricultural threats, make initial  regulatory decisions, and determine when to make referrals to CBP  agriculture specialists. More detailed agricultural inspections are  performed by these specialists who have substantial training and  background in agricultural issues.
A variety of training delivery methods were used (e.g., e-learning and  classroom) and these training methods were reinforced with extensive on- the-job training. In addition to traditional content areas (e.g., cross-training  for former U.S. Customs officers includes courses on immigration  fundamentals and immigration law), training courses also covered CBP’s  new priority mission of preventing terrorism (e.g., training in detecting  possible terrorists and fraudulent documents, honing interviewing skills,  and making appropriate referrals to staff for additional inspection). CBP  emphasizes on-the-job training in an effort not to place inspectors on the  job without direct supervisory and tutorial backup. Training for new  recruits has also been modified to include a preacademy orientation  program at the port location where the recruit will eventually work before  he or she receives academy training. This is a 72-day course for CBP  officers and a 46-day course for CBP agriculture specialists.
CBP’s main strategy to prepare for field delivery of training was to provide  extensive train-the-trainer courses so that trainers could return to their  field sites and instruct officers there. Training priorities were established  with the idea of spacing the training out so that field offices would not be  overwhelmed. For example, CBP rolled out its primary cross-training to  airports, while antiterrorism training was rolled out to land borders.
Officials reported that cross-training benefited CBP officers since they  have gained more knowledge by learning both immigration and customs  laws and procedures. This increase in knowledge has the potential benefit  of providing more variety in job tasks as well as increasing the  opportunities for advancement since an officer can now apply for  supervisory-level positions that had previously been open only to former  U.S. Customs or Immigration and Naturalization Service officers.
Change has not come about without challenges, however, as many officers  were reported to have resisted changes to their responsibilities, mainly  related to the difficulties in learning a new set of procedures and laws.  Officials noted that there has been an enormous amount of required  training for CBP officers, and it can sometimes be overwhelming. For  former officers, in addition to completing an extensive cross-training  schedule and new training related to terrorism prevention, there are many  other required courses related to their mission. For example, training  modules are required in areas such as body scanning, hazardous materials,  cargo inspection, and seized assets.
Although staffing challenges may ultimately be relieved with trained  officers able to perform dual inspections, officials noted that it has been  extremely difficult to take staff off-line to complete the “One Face at the  Border” training. One official said that classes have been very difficult to  schedule because of the constant pressure to staff operations. For  example, in one case, a class was canceled right after it began because the  trainees were pulled out to staff their inspection booths. This official also  noted that trainers have had to be very flexible to accommodate staff  schedules to ensure that training occurs.
section_title: Appendix III: Comments from the Department of Homeland Security