Department of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel
Published: 2011-01-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Because the U.S. Department of State (State) is the lead U.S. foreign affairs agency, its personnel require certain knowledge, skills, and abilities to address the global challenges and security threats facing the United States. State devoted about $255 million to personnel training in fiscal year 2010; the department's Foreign Service Institute (FSI) is the primary training provider for State's more than 66,000 Foreign Service, civil service, and locally employed staff (LE staff) worldwide. GAO was asked to examine (1) State's purpose and structure for training personnel and (2) the extent to which State's training incorporates elements for effective training programs. GAO reviewed and analyzed data and documentation related to the agency's training efforts; completed a training assessment using a tool developed based on prior GAO guidance; and interviewed officials in Washington, D.C., and at 12 overseas posts.
section_title: What GAO Found
paragraphs: State's purpose for training personnel is to develop the men and women the United States requires to fulfill its leadership role in world affairs and to advance and defend U.S. interests. State guidance outlines key training roles, including FSI's primary role in developing training policies and facilitating necessary training, and the Bureau of Human Resources' role in assigning employees to training and working with FSI to help ensure it meets their needs. Other bureaus, offices, and posts also share responsibilities for training. FSI currently offers more than 700 classroom courses, and has recently increased its focus on distance learning. Overall, about 40 percent of personnel training over the last 5 fiscal years, on average, was in foreign language skills. Other training for personnel generally focused on developing leadership, management, and other professional and technical skills and knowledge. State has taken many steps to incorporate the interrelated elements of an effective training program--planning, design, implementation, and evaluation--into its extensive training for personnel; however, the department's strategic approach to workforce training has several key weaknesses. The department demonstrated a variety of ways in which it has endeavored to develop an effective training program, such as by compiling an annual training plan, and implementing a range of training evaluation mechanisms and a learning management system that can be used to track training delivery. However, GAO's analysis found several gaps in the department's efforts to strategically plan and prioritize training, ensure efficient and effective training design and delivery, and determine whether or how training and development efforts contribute to improved performance and desired results. For example: (1) State lacks a systematic, comprehensive training needs assessment process incorporating all bureaus and overseas posts. (2) State developed training continuums to provide information for employees about training opportunities, career paths, and how training can help employees attain career goals, but the continuums do not provide complete and accurate information, and other guidance does not cover all personnel. (3) State lacks formal guidance for curriculum design and for data collection and analysis, and thus cannot be assured that proper practices and procedures are systematically and comprehensively applied. (4) State could not sufficiently demonstrate consistent and appropriate support for training, because the department does not track detailed information on training cost and delivery that would allow for an analysis and comparison of employees in different groups, bureaus, regions, or posts. (5) State's performance measures for training generally do not fully address training goals, and are generally output- rather than outcome-oriented.
section_title: What GAO Recommends
paragraphs: GAO is making several recommendations for State to improve strategic planning and evaluation of the department's efforts to train personnel, including for improvements to State's efforts to assess training needs and efforts to ensure training achieves desired results. State reviewed a draft of this report and generally agreed with our recommendations.

section_title: Letter
paragraphs: The Honorable Daniel K. Akaka  United States Senate  As the lead department for U.S. foreign affairs, the Department of State  (State) plays the primary role in developing and implementing U.S. foreign  policy. In support of that role, State’s personnel require certain knowledge,  skills, and abilities to equip them to address the complex security threats  and challenges of global interdependence that accompany 21st century  diplomacy. Recent departmental initiatives—in particular, “Diplomacy  3.0,” a multiyear effort launched in March 2009 with a primary aim of  increasing the size of State’s Foreign Service by 25 percent and the civil  service by 13 percent—have underscored the importance of training to  equip personnel to fulfill State’s leadership role in world affairs and to  advance and defend U.S. interests abroad. Nongovernmental organizations  such as the American Academy of Diplomacy have confirmed that  challenges facing the United States—including the threat of Al-Qaeda and  other terrorist organizations, HIV/AIDS and other pandemics,  environmental degradation, nuclear proliferation, and failed states— require a significantly more robust foreign affairs capacity featuring skilled  professionals. In fiscal years 2006 through 2010, State’s funding for training  personnel grew by about 62 percent, and the department requested more  than $266 million in fiscal year 2011 for programs providing training in  professional skills such as foreign language proficiency, area studies,  information technology, consular duties, and others needed for the  conduct of foreign relations. State’s Foreign Service Institute (FSI) is the  primary training provider for the department’s more than 66,000 Foreign  Service, civil service, and locally employed staff (LE staff) worldwide.
In response to your request for information about State’s training of its  personnel, we examined (1) State’s purpose and structure for training  personnel, and (2) the extent to which State’s personnel training  incorporates elements of effective federal training programs.
Our analysis focuses primarily on the training that FSI provides, including  leadership, management, professional, and area studies training,  contributing to diplomatic readiness of State’s Foreign Service and civil  service personnel and LE staff overseas. In addition, in light of work that  we recently published on shortfalls in State personnel’s foreign language  skills, this report does not focus on language training. This report does  not include within its scope an assessment of “hard skills” (e.g., security  and law enforcement) training provided by State’s Bureau of Diplomatic  Security.
To address our objectives, we reviewed and analyzed data and  documentation related to State’s training efforts, such as strategic and  workforce planning documents, information and data on recent FSI course  offerings, data on personnel participation in training for fiscal years 2006  to 2010, and overall funding for training during that time period. We also  reviewed legislative, regulatory, and State policy and procedural criteria  relevant to training. In addition, we reviewed training evaluation  mechanisms used by each of the four FSI schools—the Leadership and  Management School, School of Applied Information Technology, School of  Language Studies, and School of Professional and Area Studies—as well as  within each of 10 divisions of the School of Professional and Area Studies.  We analyzed responses to training-related questions included in the  American Foreign Service Association’s 2009 survey of State Foreign  Service members, including more than 1,000 responses to an open-ended  question regarding whether and how State training could be improved.  While the results of this survey are not generalizeable, they provided  valuable insights into potential areas for improvement. We interviewed  key officials from nongovernmental organizations including the Center for  Strategic and International Studies and the Stimson Center, as well as from  26 State bureaus and offices in Washington, D.C., including FSI, the  Bureau of Human Resources, and the six geographic bureaus. We  conducted semistructured telephone interviews with State officials with  training-related responsibilities at 12 overseas missions. We selected a  nongeneralizeable sample of countries designed to ensure geographic  diversity; our criteria for selection also included factors such as the size of  the post and hardship differential. We also interviewed officials from  State’s regional training centers located in Bangkok, Thailand; Ft.  Lauderdale, Florida; and Frankfurt, Germany. With input from State, we  completed a training assessment to determine the extent to which the  department’s personnel training incorporates elements of effective  training programs—planning, design, implementation, and evaluation. We  used the results of this assessment to identify any gaps in State’s training  based on criteria identified in GAO, the Office of Personnel Management  (OPM), State, and other legislative and regulatory guidance and policy.  Appendix I contains additional details about our scope and methodology.
We conducted this performance audit from July 2009 to January 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: State defines its mission as advancing freedom by helping build and  sustain a more democratic, secure, and prosperous world composed of  well-governed states that respond to the needs of their people, reduce  widespread poverty, and act responsibly within the international system.
To achieve this mission, State relies on more than 66,000 Foreign Service,  civil service, and LE staff at its headquarters in Washington, D.C., and  serving at 271 missions in 189 countries worldwide. State’s training and  development program supports its strategic goal of strengthening consular  and management capabilities and attempts to develop and maintain a  workforce qualified to achieve its mission.
subsections: section_title: Distribution and Numbers of State Employees
paragraphs: Nearly a third of State’s workforce are Foreign Service and civil service  direct hires, and over half of State’s workforce are LE staff. The remaining  ining  workforce consists of domestic contractors and temporary personnel. See  workforce consists of domestic contractors and temporary personnel. See  figure 1.  figure 1.
According to State’s Five Year Workforce and Leadership Succession Plan,  about two-thirds of State’s Foreign Service employees are assigned to  overseas posts and the remaining one-third are employed domestically.
Almost all of State’s civil service employees are assigned at domestic  locations. See figure 2.
During fiscal years 2006 through 2010, State has increased its Foreign  Service and civil service workforce by about 17 percent, setting priority on  filling personnel shortages created in part by demands in Iraq and  Afghanistan. In March 2009, State announced plans to increase its Foreign  Service workforce by 25 percent and the civil service workforce by 13  percent by fiscal year 2014 as part of the Diplomacy 3.0 initiative. By  September 30, 2010, State had hired more than 1,900 Foreign Service and  civil service employees in new positions and planned to fill  approximately 2,000 additional new positions through 2014 to address  projected needs.
Figure 3 shows State’s actual and projected numbers of Foreign Service  and civil service personnel in fiscal years 2006 through 2014.
section_title: State’s Organization
paragraphs: Each of State’s six geographic bureaus—the Bureaus of African Affairs,  East Asian and Pacific Affairs, European and Eurasian Affairs, Near  Eastern Affairs, South and Central Asian Affairs, and Western Hemisphere  Affairs—coordinates the conduct of U.S. foreign relations concerning a  specific region of the world. Functional bureaus, such as the Bureau of  Economic, Energy, and Business Affairs and Bureau of Political-Military  Affairs, generally manage and coordinate specific issues and activities  within the department. In addition, various offices report to the Secretary’s  office, including the Office of the Inspector General and Office of  Intelligence and Research. See appendix III for State’s organizational  chart.
section_title: State’s Workforce Training Policy
paragraphs: State outlines its policies for employee training and career development in  two publications. The Foreign Affairs Manual describes the functional  statements and organization responsibilities and authorities assigned to  each of State’s major components. The Foreign Affairs Handbooks provide  detailed procedural implementation of policies and guidance outlined in  the Foreign Affairs Manual. In certain cases, the policies outlined in the  manual and handbooks reflect legislative criteria for training Foreign  Service, civil service, and LE staff.
Foreign Service and LE staff. The objective of the Foreign Service Act,  as amended, is to strengthen and improve the Foreign Service of the  United States. The Act requires the Secretary of State to maintain and  operate an institute for training to promote career development within the  Foreign Service. This institute is to provide necessary training and  instruction in the field of foreign relations to the members of the Foreign  Service, including foreign national employees—that is, LE staff who are  not U.S. citizens—who provide clerical, administrative, technical, fiscal,  and other support at foreign service posts abroad, and to employees of the  department and other U.S. departments and agencies. The Foreign Service  Act also requires the Secretary of State to establish a professional  development program to assure that members of the Foreign Service  obtain skills and knowledge required at various stages of their careers,  with primary attention to training for career candidate officers and mid- career officers. In addition to department policies, local labor laws, which  vary from country to country, also apply to all LE staff.
Civil service. State’s civil service workforce is regulated by OPM  guidelines and federal laws. In particular, according to OPM, the  Government Employees Training Act, as amended, created a framework  for agencies to plan, establish, implement, evaluate, and fund training and  development programs designed to improve the quality and performance  of the workforce. In addition, Executive Orders have provided the  Secretary of State with additional presidential direction on  implementation of the government Employees Training Act by directing  that agencies (1) develop training programs to address both short- and  long-range program needs specific to occupations or organizational groups  and (2) conduct periodic training needs assessments.
section_title: Principles of Effective Federal Training Programs
paragraphs: GAO’s previously issued guide for assessing federal strategic training and  development efforts identifies 32 attributes, as well as corresponding  indicators for each attribute, relating to four broad, interrelated elements  of an effective training program: (1) planning, (2) design, (3)  implementation, and (4) evaluation. Figure 4 depicts general  relationships between these four elements that help to produce a strategic  approach to federal training and development efforts.
The guide serves as a flexible framework for assessing how agencies plan,  design, implement, and evaluate training and development programs that  contribute to improved organizational performance and enhanced  employee skills and competencies. For each of the four elements of the  training and development process, the guide provides a set of attributes or  key questions to consider when assessing a training program, as well as a  list of indicators to look for related to each key question (see app. II for a  detailed listing of the elements and their associated attributes). Because  the guide is meant to serve as a flexible framework, an agency’s training  and development program is not necessarily expected to address every  indicator. However, the guide can be used to identify potential gaps or  strategic weaknesses in an agency’s training program.
Table 1 lists examples of attributes, as well as supporting indicators, for  assessing each of the four elements outlined in the guide.
section_title: State Has Developed an Extensive Training Program in Support of Its Mission, Primarily through the Foreign Service Institute
subsections: section_title: Purpose and Key Responsibilities for Training
paragraphs: State has articulated its training and professional development mission in  various agency plans and guidance and implements this mission mainly  through FSI. According to the organizational directive outlined in the  Foreign Affairs Manual, State “is fully committed to the career  development of all its employees, consistent with organizational needs, in  order to improve service, increase efficiency and economy, and build and  maintain a force of skilled and efficient employees.” The department’s  Annual Training Plan states that “the purpose of the department’s training  program is to develop the men and women our nation requires to fulfill our  leadership role in world affairs and to advance and defend U.S. interests.”
Located at the George P. Shultz National Foreign Affairs Training Center  in Arlington, Virginia, FSI was established in 1947 to promote career  development within the Foreign Service and to provide necessary training  and instruction in the field of foreign relations to members of the Foreign  Service and to employees of the department and of other agencies. It is  State’s primary training provider for personnel, offering entry-, mid-, and  senior-level training for employees as they progress through their careers,  maintaining personnel training records, and overseeing personnel requests  for external training.
The Foreign Affairs Manual identifies training oversight authorities for  State officials and implementation responsibilities for FSI, the Bureau of  Human Resources, principal officers at post, bureau officials, managers,  supervisors, and employees. Table 2 highlights key authorities and  responsibilities.
The Foreign Affairs Manual also states that managers, supervisors, and  employees all have responsibilities in regard to training. Managers and  supervisors are responsible for determining specific employee training  needs and ensuring that employees receive training for effective job  performance. Employees are responsible for considering training that will  enable them to improve their performance and prepare them for greater  responsibilities at the department.
section_title: FSI Offers Wide Range of Training Opportunities to State Personnel
paragraphs: FSI is organized into four schools, each with multiple divisions: the School  of Language Studies, with seven divisions; the School of Applied  Information Technology, with three divisions; the Leadership and  Management School, with four divisions; and the School of Professional  and Area Studies, with 10 divisions. Each school is headed by a dean. FSI’s  Executive Director’s office provides general oversight and management  for FSI’s operations, and the Transition Center provides training and other  resources for personnel on topics such as career transitions and preparing  for life overseas. See figure 5.
Each school specializes in providing training in specific areas of study. See  table 3.
FSI offers over 700 different classroom courses at its Arlington campus,  regional centers, and overseas posts. FSI publishes a course catalog listing  the times and dates of upcoming courses, and officials told us that FSI  advertises new courses at quarterly meetings with bureau training officers  and through department cables. The duration of classroom courses can  range from a half day to 2 years for language training. In fiscal year 2010,  FSI had over 2,100 offerings of non-language classroom courses, which  include courses that focus on job-related professional and technical skills,  as well as leadership and management skills, at its domestic and overseas  locations. In addition to providing training at the main campus, FSI offers  courses at other locations in the United States, regional centers, and  overseas locations. About 86 percent of these courses were offered at  domestic locations and 14 percent at overseas locations. Approximately  70 percent of all FSI non-language courses in 2010 were offered at the FSI  campus in Arlington, and roughly half of the remaining courses at  domestic locations were offered at State headquarters in Washington, D.C.  See figure 6.
FSI has also increased its focus on distance learning in recent years.  According to State officials, in addition to offering classroom training, FSI  began offering distance learning courses in 2002. In fiscal year 2010, FSI  offered 190 customized courses, including courses with little or no  interaction with an instructor that allow participants to complete the  course at their own pace; real-time courses in which students and  instructors participate simultaneously via various technologies; and  “blended courses,” which combine various delivery methods such as time  spent in the classroom and online. In addition, FSI offers about 3,000  commercially developed courses that are available at all times in a wide  variety of topics, including Microsoft applications and various business  topics. According to FSI officials, distance learning allows FSI to provide  training for LE staff overseas who otherwise would have been unable to  access training.
section_title: State Provides Additional Training and Career Development Opportunities
paragraphs: While FSI is State’s primary training provider, other bureaus in the  department also offer training for State and other federal government  employees. For example, the Office of Training and Performance Support  within the Bureau of Diplomatic Security offers security and law  enforcement training to personnel, including Special Agents, Regional  Security Officers, and others. The Diplomatic Security Training Center  provides about 85 instructor-led courses, including the Foreign Affairs  Counter Threat course for personnel who are deploying to critical threat  environments such as Iraq, Afghanistan, and Pakistan. Other  departmentwide bureau-sponsored training includes ethics training by the  Office of the Legal Advisor; Equal Employment Opportunity laws and  regulations training by the Office of Civil Rights; and emergency  preparedness, property record keeping, procurement, logistics  management, federal assistance, and safety awareness training by the  Bureau of Administration.
State also offers several career development programs for employees. For  example, in 2005 State introduced the Career Development Program for  Foreign Service employees, which sets requirements for advancement into  the senior ranks and focuses on developing appropriate professional,  leadership, language, and technical skills at each level. State’s Civil Service  Mid-Level Rotational Program provides opportunities for mid-level civil  service employees to rotate to other bureaus to broaden their skills,  increase their knowledge, and enhance their personnel and professional  growth. In addition, State has mentoring programs for entry-level, as well  as more experienced, employees.
According to State officials, bureaus, offices, and posts may also develop  and offer their own training and professional development opportunities  when the bureau, office, or post has specific needs that make it more  efficient for it to develop the training itself or seek training outside the  agency. Officials from several bureaus, offices, and posts told us they have  designed various training programs that are tailored to the specific needs  of their employees, which may include orientation, on-the-job training,  mentorship opportunities, and annual conferences, workshops, or  seminars. For example:    The Bureau of Economic, Energy, and Business Affairs organizes a half- day orientation session every 6 months for employees who are new to the  bureau.
The Bureaus of European and Eurasian Affairs; International  Organizations; Consular Affairs; and Democracy, Human Rights and Labor  host annual conferences and workshops for various employees in their  bureaus.
The Bureau of Intelligence and Research coordinates with training  programs at other federal agencies in the intelligence community to  arrange for its employees to attend specialized training in subject matter  that is not available through FSI.
The Bureau of International Narcotics and Law Enforcement provides  training sessions that help employees gain familiarization with various  areas of law enforcement.
Training officials at 11 of the 12 overseas posts we interviewed said their  post sponsors a language program, which may include classes in the native  language for Americans and English language courses for LE staff.
Officials at 8 of the 12 overseas posts we interviewed said they host some  type of orientation for employees new to the post.
In addition, officials told us that three of State’s regional bureaus— Western Hemisphere Affairs, European and Eurasian Affairs, and East  Asian and Pacific Affairs—operate regional centers that service various  needs of the posts in their respective regions in Fort Lauderdale, Florida;  Frankfurt, Germany; and Bangkok, Thailand. Officials at all of the  regional centers told us they have a model for providing some training for  employees in their respective region, including LE staff. These officials  also noted that some of these courses were developed by FSI, while other  courses have been designed and developed by the regional center.
State employees may also participate in various external training  opportunities. For example, officials noted that certain civil service  employees in particular may take advantage of human resource or  technical courses offered through outside vendors such as the U.S.  Department of Agriculture Graduate School, that are not available through  FSI. In addition, employees may participate in several long-term external  career development opportunities, which are generally available to  tenured Foreign Service and civil service employees who are at the mid-  and senior-grade levels. These programs normally last from 9 to 12 months  and include the military schools (War Colleges and Commands) and  nondegree fellowships, as well as bachelors and masters degree programs  at various colleges and universities. For example, State data showed that a  total of about 130 Foreign Service and civil service employees participated  in long-term academic training in the 2009-2010 academic year. Further, in  2009, almost 80 State employees participated in long-term development  programs related to interdepartmental collaboration on national security  and were assigned to long-term rotational positions at agencies such as the  Department of Defense and the Office of the Director of National  Intelligence.
section_title: Types and Amount of Training Vary for Foreign Service, Civil Service, and LE Staff
paragraphs: State personnel receive varying types and amounts of training, depending  on their position or category, career stage, specific training needs, and  available resources. State data on employee training hours in fiscal years  2006 through 2010 show the following:    Almost 90 percent of all training hours took place in the classroom, with  about half of these hours for language training and the other half for non- language training.
Time spent in distance learning comprised about 7 percent of all training  hours.
Training by external providers comprised 5 percent of training hours.
About 85 percent of employee hours in classroom training was for Foreign  Service employees, with about half of this time for language training;  about 15 percent of FSI classroom training was for civil service and LE  staff.
Civil service employees spent about 70 percent of total hours in external  training.
LE staff spent the largest amount of time in distance learning—about 47  percent of total hours in this type of training.
Figures 7 and 8 illustrate the types and amounts of training that State  employees received in fiscal years 2006 through 2010.
section_title: State Workforce Training Incorporates Many Aspects of Effective Training Programs, but Strategic Weaknesses Exist
paragraphs: State has taken many steps to incorporate the interrelated elements of an  effective training program—planning, design, implementation, and  evaluation—into its training for personnel, but the department’s strategic  approach to workforce training has several key weaknesses. State  demonstrated a variety of examples of ways in which the department has  endeavored to develop an effective training program, such as by compiling  an annual training plan and implementing a range of training evaluation  mechanisms. However, in our analysis of the extent to which State’s  training program reflects key attributes identified in prior GAO guidance,  we found several key gaps in the department’s efforts to strategically plan  and prioritize training, ensure efficient and effective training design and  delivery, and determine whether or how training and development efforts  contribute to improved performance and desired results. These issue areas  are generally connected with various attributes and indicators associated  with more than one element of the training and development process, as  planning, design, implementation, and evaluation efforts are often  interrelated.
subsections: section_title: State’s Training Reflects Aspects of Effective Training Programs
paragraphs: State’s personnel training reflects numerous aspects of effective training  programs, based on our assessment using the criteria GAO previously  identified. (See app. II for a detailed listing of the elements with their  associated attributes.) For example, we identified the following positive  practices relevant to the major, interrelated elements of the training and  development process.
Planning. State maintains a workforce training plan, as required by  federal regulations. FSI leads efforts to prepare the training plan annually  with input from other bureaus and offices; the plan is linked to State’s  overall strategic plan, and presents a business case for proposed training  investments, including training provided by FSI, the Bureau of Diplomatic  Security, and other personnel training programs, such as external and  long-term training opportunities. FSI’s director—the chief training officer  for the department—and the Director General of the Foreign Service and  Director of Human Resources are members of the department’s senior  management, reporting to State’s Under Secretary for Management.  According to State, in demonstration of its commitment to training, the  department dedicates almost 1.5 percent—approximately $214 million in  fiscal year 2010—of its program budget to personnel training. State’s  Foreign Affairs Manual and Foreign Affairs Handbooks include  information on the department’s legislative authorities and policies that  may relate to, or require training and development for, Foreign Service,  civil service, and LE staff. In addition, FSI publishes an annual schedule of  courses, which provides information for employees on the more than 700  classroom courses FSI offers, as well as approximately 190 custom- developed distance learning courses.
Design. FSI highlighted its involvement of curricula and education experts  as well as subject matter or technical experts in the development of  courses as a positive practice. FSI officials stated all courses are designed  to meet specific learning objectives. The schedule of courses generally  includes information for each course such as a brief description, any  prerequisites, course objectives, and relevant competencies and  precepts. State has also designed programs that incorporate various  cross-training opportunities and targeted experiences. For example,  Foreign Service and civil service personnel may participate in rota assignments, congressional fellowships, OPM leadership seminars, or  programs through the National Defense University, among other  opportunities; and LE staff may participate in short-term details to other posts, or opportunities to “shadow” a more experienced employee,  according to State officials. Additionally, State has made an effort to  design and develop courses that use advances in technologies to enha nce  its training efforts, another example of a positive practice. The number of  distance learning offerings, as well as employee participation in distance learning, has increased in recent years. For example, State’s latest annual  training plan reported that FSI developed 20 new custom distance learnin courses during the prior year. opportunities at some overseas posts, which may include group  instruction via videoconference alo FSI has also piloted “blended” learnin ng with online course content.
In addition, during fiscal years 2006 through 2009, the total number of hours that  personnel spent completing distance learning courses more than doubled—from about  113,000 hours in fiscal year 2006 to about 254,000 in fiscal year 2009. Data include hours  spent completing all distance learning course offerings, including custom-developed  courses, by Foreign Service, civil service, and LE staff over the time period. well as to encourage employees to accept the goals of training efforts and  apply knowledge gained through training on the job. For example, FSI has  an attendance policy requiring students to participate in 80 to 100 percent  of a classroom course—depending upon duration and subject matter—and  pass any applicable testing requirement in order to receive official course  credit. The employee’s home bureau must generally reimburse FSI in the  event that an employee does not show up or complete a course. In  addition, FSI generally reviews and approves all applications for external  training. Employees generally must sign an agreement to reimburse State  if they do not complete an external training course they registered for, and  employees must complete a post-training evaluation for any external  training. Several posts we interviewed noted they have training  agreements in place for training provided to LE staff, which may require  personnel to submit a report to supervisors on the outcome of training or  reimburse State for costs such as travel in the event they do not  successfully complete a training course.
Evaluation. State has a range of training evaluation mechanisms in place,  including mid- and post-training course evaluations, some incorporating  supervisors as well as employees. Efforts to evaluate training in order to  assess the extent to which it contributes to improved results is considered  an essential component of an effective training program. The department  also has a learning management system that can track delivery of training.  For example, as noted previously, State provided data to us on training  hours and courses completed by Foreign Service, civil service, and LE  staff over the past 5 fiscal years, including data on language and non- language classroom and distance learning, as well as external training.  Additionally, since 2006 FSI has conducted an annual training survey; for  example, FSI reported most respondents to the 2010 survey were, in  general, satisfied or very satisfied with training. Further, to comply with  an OPM mandate, according to State, in 2007 the department conducted  the biennial Quality of Work Life Survey, which addresses human capital  issues and trends. State reported that the 2007 survey found a majority of  respondents agreed with the statement, “I receive adequate training to do  my job.”
section_title: Weaknesses Exist in State’s Strategic Approach to Personnel Training
paragraphs: Although State’s training practices and procedures reflect numerous  attributes and indicators of an effective training program, we found gaps  in six key areas (see table 4). Each of these issue areas broadly relates to  multiple elements, attributes, and indicators throughout the interrelated  training and development process. While an agency’s training program is  not necessarily expected to address every indicator identified in the GAO  guidance, based on our assessment, we identified strategic weaknesses  related to these six issue areas as particularly important to ensuring  effective planning, design, implementation, and evaluation of personnel  training.
Although State has several practices in place to identify training needs, the  department lacks a systematic, comprehensive training needs assessment  process clearly incorporating all bureaus and posts, particularly at the  occupational and individual levels. Our previous work identifying elements  of effective training states that an agency can use organizational,  occupational, and individual training needs assessments to help ensure  training is connected to improving individual and agency performance in  achieving results. OPM guidance refers to this as a multilevel training  needs assessment process and states that to be successful, an assessment  process should be ongoing, involve management at all levels, and be  integrated into the agency’s program planning and budgeting process.  Further, under Executive Order, agencies are to review, not less than  annually, programs to identify training needs, establish priorities for  training, and provide resources in accordance with those priorities. In  addition, State’s Foreign Affairs Manual states that bureau executive  directors are responsible for identifying bureau training needs. State  guidance also notes that not conducting training needs assessments may  cause time and resources to be wasted in developing and teaching skills  that employees do not need, while true training needs and barriers may go  unaddressed.
We found that State has some processes in place to identify broad or  cross-cutting training needs at the organizational level. For example, FSI  officials told us the institute identifies training needs and strategy in part  through FSI officials’ participation in various departmental committees or  working groups that may address training issues or discuss training needs.  They noted that FSI’s Director, who acts as the Chief Training Officer for  the department, attends the Secretary’s weekly staff meetings and holds  monthly meetings with the Director General of the Foreign Service and  Director of Human Resources. The Deputy Director participates in  periodic meetings with the Bureau of Human Resources Office of Career  Development and Assignments. FSI officials added they were holding  weekly meetings with officials from the bureau on how to address training  needs related to the recent increase in hiring. Further, they said the Dean  of the School of Applied Information Technology participates in periodic  meetings with State’s Chief Information Officer to stay abreast of  information technology training needs. According to State’s Bureau of  Human Resources, State aims to identify training needs related in  particular to the department’s foreign policy mission and emerging issues.
However, State’s processes for identifying both organizational, cross- cutting, and more specific occupational training needs do not clearly  incorporate all bureaus and posts. For example, officials from a number of  bureaus told us that their bureaus had not conducted formal training  needs assessments. Only one bureau indicated it had recently conducted  an assessment relevant to training needs. To help identify cross-cutting as  well as more specific occupational training needs, FSI officials said the  institute holds quarterly meetings with bureau training officers, providing  opportunities to discuss any bureau training needs and to share  information, for example, about upcoming course offerings. However,  several bureau training officials we met with noted that a bureau with a  specific training need would generally reach out to FSI on a case-by-case  or ad hoc basis. They said that FSI is generally responsive to ad hoc  requests regarding specific training needs, but discussions at the quarterly  meetings typically address topics such as training-related administrative  processes, rather than addressing bureau training needs.
State’s Office of Inspector General (OIG) previously recommended that  State conduct annual training needs assessments and implement a more  comprehensive and systematic planning process for training incorporating  all bureaus and posts, such as by requiring a training section in the  department’s annual bureau and mission strategic plans, to help identify  and address long-term training needs across locations and all categories of  employees in the department. To address prior OIG findings, State  reported that the department had added a training “annex” to all bureau  and mission plans. However, State officials indicated the plans no longer  include a training annex, owing to the department’s decision to streamline  the strategic planning process in order to make it less burdensome for  bureaus and posts. As a result, officials noted some bureau and mission  plans include information on training goals, needs, or priorities, while  others include no reference to training. None of the 12 posts we  interviewed reported receiving guidance from headquarters that could  help the post prioritize, plan, and budget for training. Some bureau and  post officials stated they could benefit from additional guidance, and  indicated that a more systematic training planning process across bureaus  and posts could be helpful. However, several officials noted that such a  process could become a “paperwork exercise” unless there were also a  commitment to follow-through with implementation.
In addition, relevant to occupational training needs, the Foreign Affairs  Manual states that training officers in each bureau are responsible for  developing a bureau career guide outlining profiles of major occupations  in the bureau including descriptions of the knowledge, skills, and abilities  needed for each occupation and grade. However, most of the bureaus we  met with had not developed a bureau career guide. Officials indicated  they would generally rely instead on broader departmental guidance to  help identify relevant training. For example, State guidance states that civil  service employees must demonstrate competencies established by OPM.  In addition, State has broadly identified “core precepts” for Foreign  Service entry-, mid-, and senior-level employees, which are updated  periodically. Technical competencies have also been identified for certain  groups of employees, such as administrative or post management officers.  Officials indicated that more specific roles and responsibilities, as well as  any skills or abilities—though not necessarily training—required for  different Foreign Service, civil service, and LE staff positions, are  generally identified in job announcements. They noted they also rely on  FSI guidance on training for employees in different career paths.
Also, to help identify individual training needs, bureau and post officials  noted that Foreign Service employees generally rely on career  development officers, in addition to supervisors at post, to provide  guidance and counseling on training and development, including  identifying necessary training. However, some post officials noted this  support may vary. A number of Foreign Service employees responding to  the open-ended question on training in the 2009 American Foreign Service  Association survey also cited concerns about career development officer  support. For example, some said their career development officers  generally had too many people assigned to them and were ineffective at  disseminating relevant information; were not responsive to e-mails or  phone calls; and were more focused on filling positions than on supporting  Foreign Service Officers’ career development. According to State’s Bureau  of Human Resources, as of October 2010, State had 41 career development  officers (CDO), each supporting at least 250 Foreign Service generalists  and specialists. They noted that serving specialists and mid-level  generalists in particular, some CDOs support as many as 450 employees.  The bureau also has two dedicated staff—an assignments officer and a  training officer—who help ensure that entry-level personnel are enrolled  in required training. Human resources officials said that recent increases  in State personnel had presented challenges to the provision of career  development counseling, and that, although State had created new entry- level CDO positions as a result, the need for additional mid-level CDOs  was under review. They added that they have also been working to  streamline administrative and technical processes to allow officers to  spend more of their time providing individual guidance and counseling on  training.
Further, although State encourages all employees to complete an  individual development plan (IDP), which can be a component of an  individual training needs assessment process, bureau and post officials  indicated that few Foreign Service or civil service employees have  completed an IDP. As a result, it is not clear whether Foreign Service and  civil service employees have adequate opportunity to discuss with their  supervisors any training they may need to improve individual performance  or prepare for future assignments. We previously reported that the use of  IDPs to identify both short- and long-term developmental needs of each  employee can help an agency incorporate employees’ developmental  goals, and integrate the need for continuous and career-long learning, into  its planning processes. State requires only certain entry-level civil service  employees to complete an IDP. Some bureau officials stated they thought  the IDP should be a broader requirement. Officials from several of the  bureaus we met with noted that while the IDP is encouraged, it is not  widely used by domestic bureau personnel, although training needs may  be identified for civil service personnel as part of the annual performance  appraisal process. Bureau and post officials indicated that, although  training needs could be covered as part of the Foreign Service appraisal  process, in contrast to the civil service appraisal, the Foreign Service  appraisal form does not explicitly address training, and officials generally  have not seen it in the appraisals.
Post officials from 11 of the 12 posts we interviewed said their posts use a  work development plan (WDP), similar to an IDP, for LE staff. Officials  noted that posts are generally responsible for coordinating and overseeing  LE staff training. Some of these posts said it is a general practice or  requirement for supervisors to work with LE staff at post to complete a  WDP, including identifying any training needs, as part of the annual  appraisal process. Post officials noted that they may review the employee  development plans or solicit broader input from supervisors or other  relevant personnel at the post, to help identify and prioritize training as  part of the post’s annual budget and planning process. For example,  officials at one post noted they annually solicit training needs from all  sections at the post for the upcoming year and review the WDPs to  compile a prioritized list of training. However, the senior human resources  officer we spoke with at another post said that she primarily identifies  training needs and puts together an annual budget request for training  based on her own observations and oral discussions with relevant  personnel at the post. She noted that although she had attempted to solicit  broader, formal input from personnel at post on individual training needs,  she had received minimal response. She added that she will review WDPs  to help identify individual training needs, but because the WDP is not a  requirement at post she must rely largely on her own observations and  oral discussions with supervisors of LE staff.
While State does not have a systematic, comprehensive training needs  assessment process, training officials we interviewed from several bureaus  and posts cited various areas of potential training need for Foreign  Service, civil service, and LE staff. For example, some bureau officials  cited a greater general need for training personnel in areas such as project  management, strategic planning, human resources, and budgeting, as well  as in English and computer skills for LE staff in particular. Post officials  also cited a general need for additional training in areas such as project  and contract management and for strategic planning, public-private  partnerships, NGO engagement, and enhanced coverage of certain  countries in geographic area studies, among other areas. Further, a  number of Foreign Service employees responding to the open-ended  question on training in the 2009 American Foreign Service Association  annual survey cited a range of potential training needs. For example,  respondents cited needs for additional training for generalists relevant to  their particular areas of work; additional training for specialists, including  for Office Management Specialists; additional and enhanced leadership  training; and improved and expanded language training, including  expansion of language training overseas. Respondents also cited the need  for a greater focus on training through their careers, such as by  establishing and enforcing minimum requirements at various career stages  beyond the current mandatory requirements. State officials noted that  State has increased attention to LE staff training and development in  recent years, although some officials noted in the past, LE staff often  received little or no training despite significant need for training at some  posts. Because Foreign Service employees generally rotate to a new post  every 2 years, LE staff often provide the longevity, continued presence,  and support that are critical to the successful day-to-day operations of  overseas missions. In June 2009, FSI conducted a survey of post  management to identify LE staff training needs. Several bureau and post  officials told us they considered the results of the survey in planning LE  staff training. For example, officials representing the Bureaus of Near  Eastern Affairs and South and Central Asian Affairs said that as a result of  the survey, they identified a particular need for supervisory skills training  for LE staff working at certain posts in those regions, and offered two  relevant workshops in the region in fiscal year 2010. FSI officials noted  that as a result of the 2009 survey, they have been working to increase  training offerings at posts by expanding the number and reach of adjunct  faculty. They added that FSI plans to repeat the needs assessment  periodically, though they have not set any specific time frames as of  November 2010.
Since 2007, State human resource reports also noted that bureaus have not  formally conducted annual training needs assessments, and identified this  as an issue that should be addressed to help provide a realistic basis for  planning, budgeting, and directing training. According to the reports, the  Bureau of Human Resources intended to form an interoffice working  group to develop a comprehensive plan and implementation guidance to  support a department-wide effort for assessing training needs. State  officials said they had recently begun developing a plan to address the  reports’ findings, though they had not yet formed an interoffice working  group as of November 2010.
subsections: section_title: FSI Training Continuums Include Some Inaccurate and Incomplete Information
paragraphs: We found that documents on training—known as training continuums— that FSI developed for employees do not include accurate and complete  information on training. The continuums generally identify training and  development for entry-, mid-, and senior-level employees, and contain  other information related to career planning. We previously reported that  providing transparent information for employees about career maps and  paths, how training opportunities could help employees attain career  goals, competency models, and training or professional requirements, can  help agencies communicate the importance of training and their  expectations for training programs to achieve results. FSI has developed  and published various training continuums for State’s Foreign Service,  civil service, and LE staff, including the Training Continuum for Civil  Service Employees, the Training Continuum for Foreign Service  Generalists, the Training Resource Guide for Foreign Service National  Employees and Locally Employed Staff, and the Leadership and  Management Training Continuum, among others.
State’s OIG reported that FSI began developing these training continuums  in part to provide a framework for a departmental policy of mandatory  training, as well as to provide road maps tailored to the particular needs of  employees in various positions throughout their careers. In addition, the  continuums state they were designed to provide a broad overview of  appropriate training that should be considered as employees plan their  careers in the department. For example, the training continuums provide  information on leadership skills training requirements that State  established under former Secretary of State Powell for mid-level and  higher-ranked Foreign Service and civil service employees. The  continuums also generally include information on other mandatory  personnel training requirements, such as cyber security training, as well as  information on recommended and suggested courses for employees.  Specifically, the continuums state that required courses are defined as  mandatory; recommended courses should be taken to provide knowledge  and skills for successful job performance; and suggested courses are more  specialized and should be taken depending upon job duties. In addition,  the continuums generally include information on career development  opportunities. For example, they note that State’s civil service, mid-level  rotational program provides opportunities for civil service mid-level  employees to rotate to other bureaus to broaden their skills, increase their  knowledge, and enhance their personnel and professional growth. They  also provide information on State’s Career Development Program for  Foreign Service employees, which established certain mandatory and  elective professional, leadership, language, and technical skills  requirements that personnel must meet to be eligible for promotion to the  Senior Foreign Service.
However, we found several issues that raise questions about the  usefulness and reliability of the continuums as resources for employees in  planning their training and development. For example, although we found  that the Training Continuum for Civil Service Employees lists diversity  awareness, ethics, and orientation training as “required” for certain groups  of employees, and as “recommended” for other groups, a key official from  FSI’s executive office stated that these courses are in fact mandatory  requirements for all civil service employees. The official added that these  mandatory requirements would also apply to Foreign Service personnel  and are among the few across-the-board training requirements that State  considers mandatory, along with cyber security and “No FEAR Act”  training. In addition, LE staff have minimal mandatory training  requirements other than cyber security and ethics training. The FSI official  clarified that even though we found that some of the continuums list other,  more specialized courses as required for certain employees, the  department officially considers training mandatory, and tracks  completion, only if the training is listed as such by statute. The FSI official  said that in some cases, decisions regarding what courses would be listed  in continuums as required, recommended, or suggested were not fully  vetted throughout the agency, and that, as a result, the information may  not be entirely accurate or complete.
In addition, while officials noted that bureaus and posts may designate  specific courses as required for certain groups of employees, we found  that these requirements are not always identified in the training  continuums or other guidance for employees. Several bureau and post  officials noted that personnel would generally rely on the training  continuums to identify necessary training, including any specialized  training that may be required or helpful to them in performing their jobs.  However, officials from the Bureau of International Organizations said that  although FSI’s course on mulitilateral diplomacy is required of all  International Organizations personnel, it is not listed in the continuums.
Training officials from the Bureau of Democracy, Human Rights, and  Labor said Foreign Service labor officers must take the FSI labor officer  skills course, though it is not listed as a requirement in the continuum.  Some bureau officials said that employee supervisors are generally  responsible for ensuring that employees are aware of and take the  required training. Although some bureaus and posts had developed their  own training guidance—for example, on available or suggested training  opportunities, or on procedures required to register for training—the  guidance generally did not include information on any specific training  designated as required, recommended, or suggested by the bureau or post  for different employee groups.
The official from FSI’s executive office acknowledged that the continuums  do not include complete and accurate information for employees on  training, and noted they have not been reviewed to ensure they uniformly  reflect departmental policies or standards. The official added that some of  the continuums have not been updated in several years, there are not  continuums for every employee group, and because the continuums  include some discrepancies or inaccuracies, they should not be viewed as  formal or official guidance. The official also noted that FSI was  considering shifting to a new, standardized format for the training  continuums going forward that would hopefully improve the process and  make the continuums easier to update in an ongoing basis in the future.  However, greater involvement and collaboration from other bureaus  would be required in order for FSI to facilitate a comprehensive process to  develop official continuums for each job series, and the official noted that  some bureaus had not been very responsive to FSI’s requests for input in  the past.
FSI officials noted that information included in the continuums regarding  State’s Career Development Program is official, as these program  requirements have been formally approved by the department. The  officials said that State has also developed separate guidance, called  playbooks, for certain groups of Foreign Service employees regarding  Career Development Program requirements. State officials noted that  both the continuums and the playbooks are resources that can be used by  employees to identify training and help plan career development, although  the continuums were developed for general informational purposes and  were not approved by agency management, unlike the playbooks.  However, the Career Development Program only applies to State’s Foreign  Service employees. In addition, although the playbooks generally provide  information on mandatory or elective leadership or technical skills  training required for promotion, they do not provide broader information  that is included in the continuums, such as information on recommended  training that may not be relevant to promotion under the Career  Development Program.
Although State has not established mandatory training for all employees at  every career stage, some bureau and post officials said that specific  training requirements and additional guidance for different employee  groups could be helpful. In response to the State OIG’s prior  recommendation that the department establish specific mandatory  training for all employees at every career stage, State reported it intended  to eventually mandate specific training for all employees throughout their  careers; however, the mandatory training the department subsequently  established initially focused on the leadership skills training. State also  noted that although the department viewed training as critical to assuring  that employees have the necessary skills to support State’s mission,  implementing broader mandatory requirements would have serious  resource implications. Nevertheless, some bureau and post officials we  interviewed indicated that improved guidance on training priorities by  career track, as well as possibly establishing additional specialized  mandatory training requirements, could heighten the focus on training and  help ensure that employees get the training they need.
section_title: State Has Not Developed Formal, Comprehensive Curriculum Design Guidance
paragraphs: Although State provided some examples of ways in which it aims to  incorporate effective design practices for training and development  efforts, we found that the department has not developed comprehensive  written guidelines and criteria to be followed throughout the course  design process. Our prior work identified several key indicators of  effective training programs related to the training design process, such as  mechanisms, procedures, or explicit criteria for    determining whether to design training programs in-house or obtain  services from a contractor or external source;   comparing merits of different delivery mechanisms (e.g., classroom or  computer-based training) and for determining the appropriate mix of  mechanisms to ensure efficient and cost-effective delivery;   ensuring an environment conducive to effective training through  employment of quality instructors, facilitators, mentors, and coaches; involvement of line managers, technical experts, human capital  professionals, and others to develop an integrated way to address specific  performance gaps; and incorporating measures of effectiveness into course designs.
State has made an effort to incorporate effective design practices into its  training for Foreign Service, civil service, and LE staff. For example, in  demonstration of its efforts to identify cost-effective and robust options  for designing training and development, State shared sample letters of  obligation, such as a letter regarding its decision to provide in-house  training services for an FSI language course following a streamlined cost  competition. State also noted that its training includes a mix of delivery  mechanisms, including classroom, distance learning, and on-the-job  training, as delineated in its training continuums and schedule of courses.  According to FSI officials, FSI employs civil service training experts and  Foreign Service subject matter experts or technical experts to teach  courses, and FSI’s Curriculum and Staff Development Division in the  School of Professional and Area Studies offers courses for staff on training  design and delivery. They noted that all FSI instructors and course  managers are required to take training in theory and best practices in adult  learning. In April 2007, FSI launched an adjunct faculty program, which  allows State personnel who demonstrate appropriate qualifications to  provide training domestically or overseas for FSI course credit. In  addition, to help ensure the integrated development of training to help  address specific performance gaps, according to State, FSI utilizes  working groups comprised of curricula and education experts, as well as  technical experts and practitioners of the subject matter in the course  design process. As a specific example, State developed and implemented a  plan to provide a combination of classroom and distance learning,  auditorium sessions, user guides, and deskside assistance to domestic and  overseas personnel to ensure the successful launch of a new technology  platform for combining cables and e-mails. Also, officials provided some  examples in which results of course evaluations were incorporated into  subsequent course offerings.
Nevertheless, although FSI officials said that FSI generally follows  accepted industry standards in developing courses, they stated that FSI  does not have comprehensive, documented guidance or standards to be  followed throughout the curriculum design process for personnel training.  While FSI has developed some instructional and curriculum guidance that  is included in its Administrative Procedures Handbook, the sections  relevant to course development are specific to distance learning and not  classroom training. In addition, the handbook notes that the sections  containing course development process and style guides for distance  learning are under development and not available. Further, the handbook  does not include clear guidance for determining appropriate training  delivery mechanisms or whether to design training in-house or obtain  outside services, or for incorporating training evaluation results into  course designs. Because of the lack of formal curriculum design guidance,  FSI’s permanent staff, as well as those who may be on temporary  assignment or working under contract to develop training, may be  unaware of and not applying consistent and appropriate practices and  standards across the board throughout the training design process.
section_title: State Lacks Data Needed to Assure That Support for Training Is Consistent and Appropriate
paragraphs: Although State collects some information on the cost and delivery of  training, the department does not collect data needed for an analysis and  comparison of training provided to employees in different groups,  bureaus, regions, or posts. As a result, State cannot be assured that it is  providing consistent and appropriate support and funding for training and  development. Our prior work highlights the importance of quality data to  evaluating the quality and effectiveness of training and development  efforts, and found that tracking the cost and delivery of training is a key  attribute of an effective training program. We also found that the ability  of an agency to demonstrate consistent and appropriate support and  funding for training, and evidence that the agency provides needed tools  and resources to managers and employees for training, are indicators of  effective training programs.
While State provided data on overall funding for training personnel that  showed some trends, the department does not track data that would allow  a more detailed analysis and comparison across various employee groups  and locations. State provided data on overall funding for training State  personnel from fiscal years 2006 to 2011, which showed the training  budget increased from about $157 million in fiscal year 2006 (adjusted for  inflation, expressed in 2010 dollars) to around $255 million in fiscal year  2010 (adjusted for inflation, expressed in 2010 dollars). (See table 5.)
State officials noted that FSI’s budget makes up the majority of funding for  training State personnel and includes resources for training space,  technology, instructors, curriculum development, salaries of full-time  students at FSI, and FSI-funded external training costs. Bureaus also allot  some money for external training not covered by FSI—bureau allotments  ranged from a total of about $3 million in fiscal year 2006 to about $4  million in fiscal year 2009. However, according to State officials, the  estimated funding for 2006 through 2009 does not fully reflect State’s  efforts to train personnel, due to the way State tracks funding. Specifically,  the amounts do not include  funding spent by posts for training offered at post or for travel per diem  for Foreign Service, civil service, and LE staff to participate in training  regionally or in Washington, D.C.; funding spent by bureaus other than FSI for internal training-related  efforts; or   some additional training-related costs that may not be detailed as training- related in State’s operating budget.
In addition, State officials said that State does not collect detailed data  showing funding spent for internal and external training; funding for  training Foreign Service, civil service, and LE staff, and for employees in  certain specific employee groups; and funding by bureau, region, and post.
State officials noted that although they lack a detailed breakdown of  training funding, they track some data on employee participation in  training. However, these data also do not allow for a complete or detailed  analysis and comparison of training for employees in different groups or  locations. Although State provided data on training hours and courses  completed by Foreign Service, civil service, and LE staff for FSI  classroom, distance learning, language, and external training from fiscal  years 2006 through 2010, State officials said that a more detailed  breakdown of this information by bureau, region, post, or employee group  is not available. Further, although FSI’s training continuums provide some  guidance for personnel on required, recommended, and suggested training  for employees in different groups, officials said that State generally only  centrally tracks employee completion of certain specific training  mandated by statute, such as cyber security training, as well for the  required leadership skills courses. State does not generally track  participation in other required, recommended, or suggested training. As a  result, for example, State could not provide data regarding the percentage  of foreign affairs or political officers that had completed required,  recommended, or suggested training for their areas of work. This type of  data could provide information on potential variations or inconsistencies  in training and help ensure employees are given sufficient and appropriate  training and development opportunities. For example, data that State  provided on the percentage of mid- and senior-level Foreign Service and  civil service personnel who had completed the respective required  leadership courses as of May 2010 showed some differences among  employee groups (see table 6). The data showed that 100 percent of  Ambassadors had completed the Ambassadorial Seminar. In addition, the  data showed that a majority of Senior Foreign Service and Senior  Executive Service, as well as Foreign Service and civil service GS-13 to  GS-15 equivalents (FS-03 to FS-01), had taken the required leadership  training, however, a greater percentage of civil service at each level were  unenrolled or untrained. State reported that Foreign Service employees  who are recommended for promotion to the next level must complete the  respective requirement within a year or the promotion will not become  effective; in comparison, civil service personnel are informed of the  requirement but monitor their own compliance.
Although State tracks some data related to training funding and delivery,  the department does not have sufficient information that could be used to  ensure consistent and appropriate support for training, or to help  determine whether managers and employees have needed training tools  and resources. This is especially important given evidence of variances in  training across the department. Training officials we interviewed at a  number of bureaus and posts cited varying support for employee training  and development. While some bureaus and posts indicated they had  sufficient funding and support for training, others noted they faced  significant resource challenges that impacted the ability of employees to  get necessary training. For example:    Bureaus. Officials from several bureaus said that finding time or resources  for training generally was not a challenge. However, this was not the case  for other bureaus. For example, officials representing two of State’s  regional bureaus noted that a lack of time was a main constraint  preventing employees from taking needed training. A training official  representing two other bureaus said that taking employees away from  their day-to-day work for training poses significant challenges, given that  some employees have heavy workloads. The official added that in her  experience, although a majority of bureau civil service supervisors have  fulfilled the mandatory leadership training requirements, supervisors  sometimes enroll but cancel owing to the “crisis of the day.” In addition,  although officials from two bureaus said they provide individual guidance  or counseling to each Foreign Service and civil service employee in the  bureau, officials representing some other bureaus indicated their  competing responsibilities do not allow them to spend much time  providing guidance or counseling, and that their primary role was to  process training requests.
Posts. Training officials we interviewed at 12 posts cited varied support  and resources for training. While several posts said they had enough  funding for training, other posts said funding was insufficient or had  fluctuated in recent years. Further, officials at most of the posts we  interviewed said finding time for training can be a challenge at post. Only  one post said time was not an issue, because the post has a policy that  allows each employee to set aside up to 2 to 3 hours a week for training.  Officials from several posts, as well as a number of respondents to the  2009 American Foreign Service Association survey noted although Foreign  Service personnel are generally expected to obtain needed training in  between tours, personnel may not have time for the training due to the  departing post’s holding onto staff as long as possible and the subsequent  post’s requiring staff to arrive as soon as possible. In addition, while two  posts noted they benefited from having personnel dedicated to training full  time, training officials from the majority of posts we interviewed said their  posts had no one dedicated to training full time. For example, the  mission training officer at one large post said she is only able to spend  about 10 percent of her time on training-related responsibilities.
Regional training. State also has regional centers that provide training,  among other services, in partnership with FSI in three locations: the  Western Hemisphere Affairs Training Division at the Florida Regional  Center (Florida center) in Ft. Lauderdale, Florida; the Frankfurt Regional  Support Center (Frankfurt center) Training and Development Center in  Frankfurt, Germany; and the Regional Employee Development Center  (Bangkok center) in Bangkok, Thailand. Officials noted these centers  primarily support training for LE staff in their respective regions, and they  offer training on-site and at other posts in their regions. However, support  for the centers varies, and the centers do not formally serve employees  working in each of State’s regions worldwide. For example, the Florida  center is the only center with a formal agreement with FSI; under the  agreement, the Bureau of Western Hemisphere Affairs funds an FSI staff  member to direct and provide training through the Florida center. Officials  from the Frankfurt center said they did not need a formal agreement in  part because they already have personnel with adjunct faculty status. In  contrast, officials from the Bangkok center said that they hoped to  establish a more formal agreement that would allow for an FSI  representative to be assigned to the center to help them further develop  their program and obtain adjunct faculty status for their two full-time LE  staff trainers. Officials from the Bankgok center noted these two trainers  currently cannot teach any FSI courses and can only teach courses  developed by the center that would not be eligible for FSI course credit.  In addition, officials from the regional centers indicated that regional  training services for the Bureaus of African Affairs, Near Eastern Affairs,  and to some extent South and Central Asian Affairs are currently ad hoc,  given that they are not formally served by any of the above centers.  Officials from the Frankfurt center said that demand for their training  from regions not formally covered by other centers, including from  employees who work at posts in the African Affairs and Near Eastern  Affairs regions in particular, often exceeds what the Frankfurt center can  provide. For some general services operations courses, the officials said  they have had 120 applicants for 10 to 25 spots.
section_title: State Has Not Developed a Formal Plan for Evaluating the Effectiveness of Training
paragraphs: Although State has various training evaluation mechanisms in place, the  department lacks a training data collection and analysis plan. As a result, it  is not clear whether or how State systematically makes decisions  regarding how training programs will be evaluated using different methods  or tools, or how results will be used. Our prior work highlights the  importance of planning and conducting evaluations of the effectiveness of  training and development efforts and notes that a data collection and  analysis plan can set priorities for evaluations and systematically cover the  methods, timing, and responsibilities for an agency’s data collection.  While State has implemented mechanisms to evaluate training, including  course evaluations and an annual training survey, these mechanisms do  not fully incorporate LE staff, and primarily focus on participant  satisfaction or reaction to training, rather than desired results, such as  improved quality or efficiency of work.
State has a range of training evaluation mechanisms and procedures to  obtain feedback from supervisors and employees on training, such as FSI’s  annual training survey, and mid- and-post-training course evaluations. FSI  has developed and updated lists of evaluations used for different courses,  which generally include information such as the intended recipients (e.g.,  students or supervisors), frequency, and timing of the course evaluation,  as well as responsibilities for compiling the evaluations and for reviewing  responses received. FSI also provided several examples of adjustments to  training efforts in response to feedback received through its course  evaluation efforts. For example, the course manager for Office  Management Specialists training provided a November 2009 memorandum  summarizing a number of changes made to the entry-level training as a  result of course evaluations, among other factors. In addition, FSI  provided a summary of results of an October 2009 offering of the Iraq  Provincial Reconstruction Team course. The summary highlighted  changes that had been made prior to the October offering as well as  recommendations for modifications to be made in the subsequent offering  of the course.
However, while FSI has compiled lists of evaluation mechanisms used for  different courses, these lists do not include guidance on setting priorities  for training evaluations, or how to determine appropriate methods, timing,  or responsibilities for evaluating training and development efforts. They  also do not include guidelines to help ensure State makes an ongoing  effort to improve the quality and breadth of data gathered. We found  several potential gaps and areas for improvement in State’s efforts to  evaluate personnel training. For example, FSI’s annual training survey is  sent to a random sample of State’s Foreign Service and civil service  employees and eligible family members but not LE staff. Likewise, several  training-related questions are also included in the department’s biannual  Quality of Work Life Survey, which is sent to a random sample of Foreign  Service and civil service employees but not to LE staff. In addition,  although FSI conducted a June 2009 survey of LE staff training needs, the  survey was sent to post management, not to LE staff, and FSI had not set  specific time frames for repeating the survey as of November 2010. Several  officials noted that LE staff often identify training needs or issues through  the process of creating a Work Development Plan, but not all of the posts  we interviewed required these plans.
In addition, respondents to the 2010 Annual Training Survey indicated they  were generally satisfied overall with FSI training delivery, programs  offered, and customer service. However, relatively fewer respondents  indicated they were satisfied with the number of courses offered and their  ability to attend courses in their geographic region. Additionally, some  respondents said that they did not receive training that would have been  helpful, such as tradecraft (e.g., consular, political, or public diplomacy),  leadership and management, language, or information technology end-user  training. However, despite the diverse training needs and requirements of  State personnel, FSI officials told us they were not able to break out  responses to determine whether there were any differences, for example,  between responses of Foreign Service and civil service employees, or  those serving at headquarters or in overseas regions.
Course evaluations used by FSI’s schools vary and generally focus on  participant satisfaction or reaction to training, rather than knowledge  gained, behavior change, or impact of the training. We reviewed  examples of course evaluations used by each of FSI’s schools— the  Leadership and Management School, School of Applied Information  Technology, School of Language Studies, and School of Professional and  Area Studies—as well as within each of the 10 divisions under the School  of Professional and Area Studies. We found that courses generally  incorporated mid-course and end-of-course student surveys, while some  also included post-training surveys (e.g., typically several months after the  end of the course), and a few included supervisors. The surveys included  various numbers of multiple choice and open-ended questions and  generally addressed participant satisfaction or reaction to training but less  frequently included the types of questions that could be used to help  determine on-the-job behavior change or impact of training. For example,  for the Stability Operations division under FSI’s School of Professional  Area Studies, we reviewed several end-of-course surveys and post-training  surveys. We found one end-of-course survey (Iraq Familiarization)  included only questions regarding participant satisfaction with course  content; one end-of-course survey (Foundations of Reconstruction &  Stabilization Operations) included questions regarding participant  satisfaction with training as well as self-assessments of knowledge gained;  and four others (Afghanistan Familiarization, Afghanistan and Iraq  Provincial Reconstruction Team end-of-course, and Afghanistan Provincial  Reconstruction Team post-training) included the above types of questions  as well as a question regarding on-the-job behavior change resulting from  training.
While some of FSI’s training evaluation mechanisms include questions that  relate to on-the-job behavior change or the impact of training, FSI’s  Director of Curriculum and Staff Development said that the institute’s  course evaluations tend to focus primarily on participant reaction to  training. The official noted that the evaluations focus to some extent on  knowledge or skills gained, and less on behavior change or impact of  training. FSI officials noted that determining the direct impact of training  can be difficult, particularly in terms of any resulting cost savings.
section_title: Performance Measures Do Not Fully Address Training Goals
paragraphs: State’s performance measures for training generally do not fully address  training goals. We previously reported that an agency’s use of measures to  assess training and development efforts is an important aspect of effective  training programs. Our prior work also noted that training and  development performance measures, aligned with the agency’s mission  and goals, can be used to help ensure accountability and assess progress  toward achieving results. Indicators of effective training programs include  the use of both quantitative and qualitative measures that provide  meaningful data on training policies and practices, and show how specific  efforts promote mission accomplishment.
State and the U.S. Agency for International Development’s Strategic Plan  for Fiscal Years 2007-2012 includes one overarching strategic goal related  to training: “strengthening consular and management capabilities.”  However, the goal is not focused exclusively on training; the plan states  that the “overarching goal…in this area is to provide the best visa and  American Citizen services possible…and to ensure a high quality  workforce, including locally employed staff.” Among other priorities listed  for the goal, one relates to training and notes that State and FSI will  “integrate systems and coordinate strategies to improve the skill base,  diversity, and performance of our workforce.” The plan does not include  any targets or measures specific to training.
While State’s broader strategic plan does not include any training  measures, the bureau strategic resource plans for FSI and for the Bureau  of Human Resources include several training-related goals and measures,  linked to the broader departmental strategic goal on training. Specifically,  the plans include a total of five training-related goals, with two or three  indicators or measures for each goal. However, the measures generally do  not fully address the goals and are output- rather than outcome-related.  As a result, they do not provide a clear means of determining whether  State’s training efforts achieve desired results. Table 7 provides further  details about each of the training goals and measures.
section_title: Conclusions
paragraphs: State has developed an extensive program to equip its personnel to fulfill  State’s leadership role in world affairs and to advance and defend U.S.  interests. In recent years in particular, State officials noted the department  has enhanced its focus on training and development of Foreign Service,  civil service, and LE staff, such as by establishing leadership skills training  requirements and increasing distance learning. State has many training  practices and processes in place reflecting the elements of effective  training programs, including the development of an annual workforce  training plan and hundreds of classroom and distance learning offerings,  as well as cross-training and other career development opportunities for  personnel. State has incorporated the use of technology to enhance  training efforts, and implemented oversight mechanisms such as training  agreements to help ensure employees take advantage of legitimate  training. In addition, State has various mechanisms in place to evaluate  training, such as an annual training survey and course evaluations.
However, we found strategic weaknesses and room for improvement in  several key areas. First, without a systematic, comprehensive assessment  of training needs, State cannot be assured training is connected to true  needs and priorities. Second, without transparent, complete, and accurate  information for all employees on training opportunities, including on any  required or recommended training for specific positions, State employees  may be hampered in their efforts to plan for training and development  throughout their careers. In addition, without taking several steps to  address weaknesses in State’s efforts to evaluate personnel training, it is  not clear that State is sufficiently able to assess whether employees have  the information, competencies, and skills they need to work effectively.  For example, it is not clear that State’s training performance measures  provide meaningful information to help the department determine the  quality and effectiveness of training efforts. State also lacks a plan for  training data collection and analysis, as well as adequate mechanisms for  determining whether the department is providing consistent and  appropriate support and funding for training all employees—including LE  staff.
State’s budget and focus on training have increased in recent years, but the  department has also faced, and will likely continue to face, fluctuating and  constrained resources and competing priorities when determining what  training is critical to its mission. Without concerted efforts to further  incorporate effective practices, State cannot ensure training resources are  targeted strategically, are not wasted, and achieve cost-effective and  timely results desired, and thus cannot be assured that its employees are  trained and equipped to meet the challenges of their mission.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of State take the following five actions:  To help ensure that State’s personnel training is connected to improving  individual and agency performance and that department resources are  directed to actual training needs and priorities, direct FSI and the Bureau  of Human Resources, in collaboration with other bureaus and offices, as  appropriate, to develop and implement a plan for a systematic,  comprehensive training needs assessment process, incorporating all  bureaus and posts.
To enhance State’s efforts to provide transparent, complete, and accurate  information to help employees plan training and development throughout  their careers, direct FSI and other bureaus and offices, as appropriate, to  collaborate in developing and updating information for employees on  training to ensure that employees have complete and accurate guidance,  including information on any mandatory, required, and recommended  training for specific employee groups.
To help ensure that State’s performance measures for training provide  meaningful data and more fully address the department’s training goals,  direct FSI and the Bureau of Human Resources to review the performance  measures and revise or enhance the measures, as appropriate.
To enhance State’s capacity to evaluate workforce training, direct FSI and  the Bureau of Human Resources to develop a data collection and analysis  plan for training, including guidance for determining the methods, timing,  and responsibilities for training data collection, as well as how results will  be used.
To improve State’s ability to determine whether it is providing consistent  and appropriate support and funding for employee training and  development across employee groups and locations, direct FSI and the  Bureau of Human Resources to identify ways to improve the collection  and analysis of training data and results, such as by enhancing the level of  detailed information gathered to determine whether employees across  groups and locations are getting needed training, and enhancing efforts to  determine the impact of training. These efforts should also include steps to  further incorporate LE staff into State’s training evaluation mechanisms.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of State. State  provided written comments, which are reproduced in appendix IV.
State generally agreed with our recommendations, and expressed  appreciation for the report’s recognition of the wide variety of training  State offers for the department’s Foreign Service, civil service, and locally  employed staff worldwide. State highlighted the importance of assessing  training needs, and noted that regular review of training performance  measures is appropriate. The department also stated that it strives to  improve the accuracy, completeness, and usefulness of training data, and  would look for ways to enhance its ability to assess the effectiveness of  training and development efforts across employee groups and locations.
In our draft report, we recommended that the Secretary of State direct FSI  to develop formal curriculum design guidance identifying guidelines and  criteria to be followed throughout the course development process, to  help ensure that clear and consistent criteria and procedures are followed  throughout the curriculum design process. After we provided the draft  report to State, the department provided documentation developed by FSI  to address this recommendation. Specifically, State provided an official  document developed in December 2010 that describes and lays out FSI’s  course design procedures and guidance, including for determining course  objectives, identifying quality instructors, and evaluating course design. As  a result of these actions, we have removed the relevant recommendation  from the final report.
State also indicated that senior leadership from the department’s Bureau  of Human Resources and FSI were not interviewed as part of this review.  However, we disagree with this characterization of our review. During the  course of our review we conducted numerous interviews with relevant  officials representing 26 State bureaus and offices in Washington, D.C.,  including senior officials from FSI and the Bureau of Human Resources— in some cases on multiple occasions. We also conducted semi-structured  telephone interviews with State officials with training-related  responsibilities at 12 overseas missions, and from State’s regional training  centers located in Bangkok, Thailand; Ft. Lauderdale, Florida; and  Frankfurt, Germany. Additionally, we had multiple discussions and e-mail  communications regarding training-related issues and questions with  officials from FSI and the Bureau of Human Resources who were  designated by State as our primary contacts throughout the audit. These  officials provided up-to-date and accurate information to us on State’s  training efforts in response to our inquiries and represented the views of  senior management, where applicable (see app. I for additional  information on our objectives, scope, and methodology).
State also provided technical comments, which we have incorporated  throughout the report, as appropriate.
As agreed with your office, unless you publicly announce the contents of  the report earlier, we plan no further distribution until 30 days after the  report date. At that time, we will send copies of the report to interested  congressional committees and to the Secretary of State. We will also make  copies available to others upon request. In addition, the report will be  available at no charge on the GAO web site at http://www.gao.gov.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: To address our objectives, our analysis focused primarily on the Foreign  Service Institute’s (FSI) training, including leadership, management,  professional, and area studies training contributing to diplomatic  readiness of the Department of State’s (State) Foreign and civil service  personnel, as well as locally employed staff (LE staff) overseas. In light of  recent GAO work addressing shortfalls in State personnel’s foreign  language skills, this report does not focus on language training. This  report does not include an assessment of “hard skills” (e.g. security and  law enforcement) training provided by State’s Bureau of Diplomatic  Security within its scope. Specifically, this report addresses the following  questions: (1) What is State’s purpose and structure for training personnel?  (2) To what extent does State’s training for personnel incorporate the  elements of effective federal training programs?
To identify State’s purpose and structure for training personnel, we  reviewed and analyzed legislative, regulatory, and State policy and  procedural criteria relevant to training, including information contained in  State’s Foreign Affairs Manual on key training roles and responsibilities.  We also collected, reviewed, and analyzed documents and data including  State’s annual training plan; departmental and human capital strategic and  workforce planning documents; organizational charts; information  available on State’s Internet and intranet Web pages; information and data  on recent FSI course offerings; and data on personnel participation in  training for fiscal years 2006 to 2010. To assess the reliability of data on  FSI course offerings and on personnel participation in training, we  reviewed relevant documentation and interviewed knowledgeable agency  officials. We determined data on FSI course offerings and on personnel  participation in training were sufficiently reliable to permit an analysis of  where courses are being offered and of the relative amount of time  Foreign Service, civil service, and LE staff spent in training, including for  FSI language and non-language classroom and distance learning, and  external training. To further our understanding of State personnel training,  we interviewed key officials at FSI, the Bureau of Human Resources, and  other bureaus and offices in Washington, D.C.
To determine the extent to which State’s training for personnel  incorporates the elements of effective federal training programs, in  addition to the above, we completed a training assessment with input from  State, using the prior GAO guidance. Specifically, we examined State  documents and data relevant to each of the 32 attributes identified in the  GAO guidance, and the indicators associated with each attribute. We used  the results of this assessment to identify strategic weaknesses in State’s  training based on criteria contained in GAO and the Office of Personal  Management (OPM) guidance, as well State and other legislative and  regulatory guidance and policy. We reviewed training evaluation  mechanisms used by each of the four FSI schools—the Leadership and  Management School, School of Applied Information Technology, School of  Language Studies, and School of Professional and Area Studies—as well as  within each of the 10 divisions under the School of Professional and Area  Studies. We examined mid-, end-of-course, and post-training course  evaluation results received for a nonrandom sample of about 37 offerings  of various courses by the different schools and divisions. We also  reviewed results of broader evaluations of FSI’s consular and language  training. Although the results of these evaluations cannot be generalized to  all courses and locations, they represent results for a broad range of  courses across all schools and divisions. We examined State data on  overall funding for training from fiscal years 2006 to 2010, as well as on the  percentage of personnel that had completed State’s mandatory leadership  skills training requirements as of May 2010. We reviewed results of FSI’s  annual training survey of Foreign and civil service employees, as well as  responses to training-related questions included in State’s biannual Quality  of Work Life Survey and State’s responses to OPM’s biannual Federal  Human Capital Survey. To assess the reliability of these data, we reviewed  relevant documentation, including information regarding survey  methodology and response rates. We determined that these data were  sufficiently reliable to provide a general indication of employee  satisfaction with training. We also analyzed responses to training-related  questions included in the American Foreign Service Association’s 2009  survey of State Foreign Service members, including more than 1,000  responses to an open-ended question regarding whether and how State  training could be improved. While the results of this survey are not  generalizeable, they provided us with valuable insights into potential areas  for improvement.
In addition, to enhance our understanding of State’s training practices and  potential issues or challenges, we interviewed relevant officials from 26  State bureaus and offices in Washington, D.C., including FSI, the Bureau of  Human Resources, and the six geographic bureaus.  For example, we  interviewed senior officials from FSI including FSI’s Director, Deputy  Director, and the Executive Director for Management, as well as senior  leadership representing each of the four FSI schools—the Leadership and  Management School, School of Applied Information Technology, School of  Language Studies, and School of Professional and Area Studies. Within the  Bureau of Human Resources, we met with senior officials including the  Chief of Career Development, Division Director of Career Development  and Assignments, Director and Deputy Director of Overseas Employment,  and Director of Policy Coordination. We also conducted semi-structured  telephone interviews with State officials with training-related  responsibilities at 12 overseas missions. We selected a nongeneralizeable  sample of countries designed to ensure geographic diversity; our criteria  for selection also included factors such as a range of post sizes and  hardship differentials. We selected two missions located in each region— Africa, the Americas (Western Hemisphere), East Asia and the Pacific,  Europe and Eurasia, Middle East and North Africa, and South and Central  Asia. Since we did not conduct interviews with a generalizeable sample of  overseas missions, our observations from these interviews are illustrative  but may not be representative of training at all overseas locations. We also  interviewed officials from State’s regional training centers located in  Bangkok, Thailand; Ft. Lauderdale, Florida; and Frankfurt, Germany.  Finally, we interviewed key officials from nongovernmental organizations  including the Center for Strategic and International Studies and the  Stimson Center.
We conducted this performance audit from July 2009 to January 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: Attributes for Review of an Agency Training and Development Program
paragraphs: Following is a compilation of the 32 questions or attributes explored in  detail throughout GAO guidance on assessing an agency’s strategic  training and development efforts. These attributes are designed for  federal agencies to consider in ensuring that training and development  investments are targeted strategically and not wasted on efforts that are  irrelevant, duplicative, or ineffective. The attributes are distributed across  the four interrelated elements of the training and development process— planning, design, implementation, and evaluation. The guidance also  includes numerous corresponding indicators for each attribute.
subsections: section_title: Planning
paragraphs: a. Does the agency have training goals and related performance measures  that are consistent with its overall mission, goals, and culture?  b. To what extent do the agency’s strategic and annual performance  planning processes incorporate human capital professionals in partnership  with agency leadership and other stakeholders in addressing agency  priorities, including training and development efforts?  c. How does the agency determine the skills and competencies its  workforce needs to achieve current, emerging, and future agency goals  and missions and identify gaps, including those that training and  development strategies can help address?  d. How does the agency identify the appropriate level of investment to  provide for training and development efforts and prioritize funding so that  the most important training needs are addressed first?  e. What measures does the agency use in assessing the contributions that  training and development efforts make toward individual mastery of  learning and achieving agency goals?  f. How does the agency incorporate employees’ developmental goals in its  planning processes?  g. How does the agency integrate the need for continuous and lifelong  learning into its planning processes?  h. Does the agency consider governmentwide reforms and other targeted  initiatives to improve management and performance when planning its  training and development programs?  i. Does the agency have a formal process to ensure that strategic and  tactical changes are promptly incorporated in training and development  efforts as well as other human capital strategies as needed?
section_title: Design
paragraphs: a. What steps does the agency take to ensure that training is connected to  improving individual and agency performance in achieving specific  results?  b. How is the design of the training or development program integrated  with other strategies to improve performance and meet emerging  demands, such as changing work processes, measuring performance, and  providing performance incentives?  c. Does the agency use the most appropriate mix of centralized and  decentralized approaches for its training and development programs?  d. What criteria does the agency use in determining whether to design  training and development programs in-house or obtain these services from  a contractor or other external source?  e. How does the agency compare the merits of different delivery  mechanisms (such as classroom or computer-based training) and  determine what mix of mechanisms to use to ensure efficient and cost- effective delivery?  f. Does the agency determine a targeted level of improved performance in  order to ensure that the cost of a training or development program is  appropriate to achieve the anticipated benefit?  g. How well does the agency incorporate measures of effectiveness into  courses it designs?
section_title: Implementation
paragraphs: a. What steps do agency leaders take to communicate the importance of  training and developing employees, and their expectations for training and  development programs to achieve results?  b. Is there a training and performance organization that is held  accountable, along with the line executives, for the maximum  performance of the workforce?  c. Are agency managers responsible for reinforcing new behaviors,  providing useful tools, and identifying and removing barriers to help  employees implement learned behaviors on the job?  d. How does the agency select employees (or provide the opportunity for  employees to self-select) to participate in training and development  efforts?  e. What options has the agency considered in paying for employee training  and development and adjusting employee work schedules so that  employees can participate in these developmental activities?  f. Does the agency take actions to foster an environment conducive to  effective training and development?  g. What steps does the agency take to encourage employees to buy in to  the goals of training and development efforts, so that they participate fully  and apply new knowledge and skills when doing their work?  h. Does the agency collect data during implementation to ensure feedback  on its training and development programs?
section_title: Evaluation
paragraphs: a. To what extent does the agency systematically plan for and evaluate the  effectiveness of its training and development efforts?  b. Does the agency use the appropriate analytical approaches to assess its  training and development programs?  c. What performance data (including qualitative and quantitative  measures) does the agency use to assess the results achieved through  training and development efforts?  d. How does the agency incorporate evaluation feedback into the planning,  design, and implementation of its training and development efforts?  e. Does the agency incorporate different perspectives (including those of  line managers and staff, customers, and experts in areas such as financial,  information, and human capital management) in assessing the impact of  training on performance?  f. How does the agency track the cost and delivery of its training and  development programs?  g. How does the agency assess the benefits achieved through training and  development programs?  h. Does the agency compare its training investments, methods, or  outcomes with those of other organizations to identify innovative  approaches or lessons learned?
section_title: Appendix III: State Organization
paragraphs: (S)
Coelor nd Chief of Sff (S/COS)
United Ntion (USUN)
D(L)
D(S) (S/ES) for Public Diplomcy nd Public Affir (R) (M)
Affir (T) (G) (A)
Mgement (IRM) (DRL)
Business Affir(EEB) Assnt (ISN)
Oce nd Int'l (CA) (M/MED)
Affir (PM) Assnt (OES) (DS) (OBO)
Implementtion (VCI) (PRM) (FSI) (RM) (HR) (OIG) (L)
Office of U.S.
Office of Global Women Issu(F) (S/P)
Issu (S/WCI)
section_title: Appendix IV: Comments from the Department of State
subsections: section_title: GAO Comment
paragraphs: 1.  We still believe that gaps exist in guidance available for the Foreign  Service, as well as for civil service and locally employed staff, because  key tools used to provide employees with information on training are  not comprehensive. Specifically, we found that documents for  employees on training—known as training continuums—that FSI  developed do not include complete and accurate information. While  the continuums state that they were designed to provide a broad  overview of appropriate training that should be considered as  employees plan their careers in the department, including information  on mandatory, recommended, and suggested courses, we found issues  that raised questions about the usefulness and reliability of the  continuums as employee resources. For example, we found that  specific training requirements designated by bureaus and posts for  certain groups of employees are not always identified in the training  continuums. A key official from FSI’s executive office stated that in  some cases, decisions regarding what information would be listed in  the continuums were not fully vetted throughout the agency. The  official acknowledged that the continuums do not include complete  and accurate information for employees on training, and noted that the  documents have not been reviewed to ensure they uniformly reflect  departmental policies or standards.
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Anthony Moran, Assistant  Director; Lisa Helmer; Shirley Min; Joe Carney; Virginia Chanley; Kieran  Cusack; David Dayton; Patrick Lockett; Reid Lowe; and Mary Moutsos  provided significant contributions to the work. Etana Finkler, Farhanaz  Kermalli, and Mona Sehgal provided technical assistance.
section_title: Related GAO Products
paragraphs: Foreign Language Capabilities: Departments of Homeland Security,  Defense, and State Could Better Address Their Foreign Language Needs  and Capabilities and Address Shortfalls. GAO-10-715T. Washington D.C.:  July 29, 2010.
Aviation Security: Efforts to Validate TSA’s Passenger Screening  Behavior Detection Program Underway, but Opportunities Exist to  Strengthen Validation and Address Operational Challenges. GAO-10-763.  Washington D.C.: May 20, 2010.
Department of State: Persistent Staffing and Foreign Language Gaps  Compromise Diplomatic Readiness. GAO-09-1046T. Washington, D.C.:  September 24, 2009.
Department of State: Additional Steps Needed to Address Continuing  Staffing and Experience Gaps at Hardship Posts. GAO-09-874.  Washington, D.C.: September 17, 2009.
Department of State: Comprehensive Plan Needed to Address Persistent  Foreign Language Shortfalls. GAO-09-955. Washington, DC.: September  17, 2009.
Veterans’ Benefits: Increased Focus on Evaluation and Accountability  Would Enhance Training and Performance Management for Claims  Processors. GAO-08-561. Washington, D.C.: May 27, 2008.
State Department: Staffing and Foreign Language Shortfalls Persist  Despite Initiatives to Address Gaps. GAO-07-1154T. Washington D.C.:  August 1, 2007.
Border Patrol: Costs and Challenges Related to Training New Agents.  GAO-07-997T. Washington, D.C.: June 19, 2007.
Department of State: Staffing and Foreign Language Shortfalls Persist  Despite Initiatives to Address Gaps. GAO-06-894. Washington D.C.:  August 4, 2006.
Department of Homeland Security: Strategic Management of Training  Important for Successful Transformation. GAO-05-888. Washington D.C.:  September 23, 2005.
Aviation Security: Flight and Cabin Crew Member Security Training  Strengthened, but Better Planning and Internal Controls Needed.  GAO-05-781. Washington D.C.: September 6, 2005.
Human Capital: A guide for Assessing Strategic Training and  Development Efforts in the Federal Government. GAO-04-546G.  Washington D.C.: March 1, 2004.
Human Capital: Selected Agencies’ Experiences and Lessons Learned in  Designing Training and Development Programs. GAO-04-291.  Washington D.C.: January 30, 2004.
U.S. Public Diplomacy: State Department Expands Efforts but Faces  Significant Challenges. GAO-03-951. Washington, D.C.: September 4, 2003.