Department of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel
Released: 2011-03-08T00:00:00

Letter

Background

State defines its mission as advancing freedom by helping build and sustain a more democratic, secure, and prosperous world composed of well-governed states that respond to the needs of their people, reduce widespread poverty, and act responsibly within the international system.

To achieve this mission, State relies on more than 66,000 Foreign Service, civil service, and LE staff at its headquarters in Washington, D.C., and serving at 271 missions in 189 countries worldwide. State’s training and development program supports its strategic goal of strengthening consular and management capabilities and attempts to develop and maintain a workforce qualified to achieve its mission.

Distribution and Numbers of State Employees

Nearly a third of State’s workforce are Foreign Service and civil service direct hires, and over half of State’s workforce are LE staff. The remaining ining workforce consists of domestic contractors and temporary personnel. See workforce consists of domestic contractors and temporary personnel. See figure 1. figure 1.

According to State’s Five Year Workforce and Leadership Succession Plan, about two-thirds of State’s Foreign Service employees are assigned to overseas posts and the remaining one-third are employed domestically.

Almost all of State’s civil service employees are assigned at domestic locations. See figure 2.

During fiscal years 2006 through 2010, State has increased its Foreign Service and civil service workforce by about 17 percent, setting priority on filling personnel shortages created in part by demands in Iraq and Afghanistan. In March 2009, State announced plans to increase its Foreign Service workforce by 25 percent and the civil service workforce by 13 percent by fiscal year 2014 as part of the Diplomacy 3.0 initiative. By September 30, 2010, State had hired more than 1,900 Foreign Service and civil service employees in new positions and planned to fill approximately 2,000 additional new positions through 2014 to address projected needs.

Figure 3 shows State’s actual and projected numbers of Foreign Service and civil service personnel in fiscal years 2006 through 2014.

State’s Organization

Each of State’s six geographic bureaus—the Bureaus of African Affairs, East Asian and Pacific Affairs, European and Eurasian Affairs, Near Eastern Affairs, South and Central Asian Affairs, and Western Hemisphere Affairs—coordinates the conduct of U.S. foreign relations concerning a specific region of the world. Functional bureaus, such as the Bureau of Economic, Energy, and Business Affairs and Bureau of Political-Military Affairs, generally manage and coordinate specific issues and activities within the department. In addition, various offices report to the Secretary’s office, including the Office of the Inspector General and Office of Intelligence and Research. See appendix III for State’s organizational chart.

State’s Workforce Training Policy

State outlines its policies for employee training and career development in two publications. The Foreign Affairs Manual describes the functional statements and organization responsibilities and authorities assigned to each of State’s major components. The Foreign Affairs Handbooks provide detailed procedural implementation of policies and guidance outlined in the Foreign Affairs Manual. In certain cases, the policies outlined in the manual and handbooks reflect legislative criteria for training Foreign Service, civil service, and LE staff.

Foreign Service and LE staff. The objective of the Foreign Service Act, as amended, is to strengthen and improve the Foreign Service of the United States. The Act requires the Secretary of State to maintain and operate an institute for training to promote career development within the Foreign Service. This institute is to provide necessary training and instruction in the field of foreign relations to the members of the Foreign Service, including foreign national employees—that is, LE staff who are not U.S. citizens—who provide clerical, administrative, technical, fiscal, and other support at foreign service posts abroad, and to employees of the department and other U.S. departments and agencies. The Foreign Service Act also requires the Secretary of State to establish a professional development program to assure that members of the Foreign Service obtain skills and knowledge required at various stages of their careers, with primary attention to training for career candidate officers and mid- career officers. In addition to department policies, local labor laws, which vary from country to country, also apply to all LE staff.

Civil service. State’s civil service workforce is regulated by OPM guidelines and federal laws. In particular, according to OPM, the Government Employees Training Act, as amended, created a framework for agencies to plan, establish, implement, evaluate, and fund training and development programs designed to improve the quality and performance of the workforce. In addition, Executive Orders have provided the Secretary of State with additional presidential direction on implementation of the government Employees Training Act by directing that agencies (1) develop training programs to address both short- and long-range program needs specific to occupations or organizational groups and (2) conduct periodic training needs assessments.

Principles of Effective Federal Training Programs

GAO’s previously issued guide for assessing federal strategic training and development efforts identifies 32 attributes, as well as corresponding indicators for each attribute, relating to four broad, interrelated elements of an effective training program: (1) planning, (2) design, (3) implementation, and (4) evaluation. Figure 4 depicts general relationships between these four elements that help to produce a strategic approach to federal training and development efforts.

The guide serves as a flexible framework for assessing how agencies plan, design, implement, and evaluate training and development programs that contribute to improved organizational performance and enhanced employee skills and competencies. For each of the four elements of the training and development process, the guide provides a set of attributes or key questions to consider when assessing a training program, as well as a list of indicators to look for related to each key question (see app. II for a detailed listing of the elements and their associated attributes). Because the guide is meant to serve as a flexible framework, an agency’s training and development program is not necessarily expected to address every indicator. However, the guide can be used to identify potential gaps or strategic weaknesses in an agency’s training program.

Table 1 lists examples of attributes, as well as supporting indicators, for assessing each of the four elements outlined in the guide.

State Has Developed an Extensive Training Program in Support of Its Mission, Primarily through the Foreign Service Institute

Purpose and Key Responsibilities for Training

State has articulated its training and professional development mission in various agency plans and guidance and implements this mission mainly through FSI. According to the organizational directive outlined in the Foreign Affairs Manual, State “is fully committed to the career development of all its employees, consistent with organizational needs, in order to improve service, increase efficiency and economy, and build and maintain a force of skilled and efficient employees.” The department’s Annual Training Plan states that “the purpose of the department’s training program is to develop the men and women our nation requires to fulfill our leadership role in world affairs and to advance and defend U.S. interests.”

Located at the George P. Shultz National Foreign Affairs Training Center in Arlington, Virginia, FSI was established in 1947 to promote career development within the Foreign Service and to provide necessary training and instruction in the field of foreign relations to members of the Foreign Service and to employees of the department and of other agencies. It is State’s primary training provider for personnel, offering entry-, mid-, and senior-level training for employees as they progress through their careers, maintaining personnel training records, and overseeing personnel requests for external training.

The Foreign Affairs Manual identifies training oversight authorities for State officials and implementation responsibilities for FSI, the Bureau of Human Resources, principal officers at post, bureau officials, managers, supervisors, and employees. Table 2 highlights key authorities and responsibilities.

The Foreign Affairs Manual also states that managers, supervisors, and employees all have responsibilities in regard to training. Managers and supervisors are responsible for determining specific employee training needs and ensuring that employees receive training for effective job performance. Employees are responsible for considering training that will enable them to improve their performance and prepare them for greater responsibilities at the department.

FSI Offers Wide Range of Training Opportunities to State Personnel

FSI is organized into four schools, each with multiple divisions: the School of Language Studies, with seven divisions; the School of Applied Information Technology, with three divisions; the Leadership and Management School, with four divisions; and the School of Professional and Area Studies, with 10 divisions. Each school is headed by a dean. FSI’s Executive Director’s office provides general oversight and management for FSI’s operations, and the Transition Center provides training and other resources for personnel on topics such as career transitions and preparing for life overseas. See figure 5.

Each school specializes in providing training in specific areas of study. See table 3.

FSI offers over 700 different classroom courses at its Arlington campus, regional centers, and overseas posts. FSI publishes a course catalog listing the times and dates of upcoming courses, and officials told us that FSI advertises new courses at quarterly meetings with bureau training officers and through department cables. The duration of classroom courses can range from a half day to 2 years for language training. In fiscal year 2010, FSI had over 2,100 offerings of non-language classroom courses, which include courses that focus on job-related professional and technical skills, as well as leadership and management skills, at its domestic and overseas locations. In addition to providing training at the main campus, FSI offers courses at other locations in the United States, regional centers, and overseas locations. About 86 percent of these courses were offered at domestic locations and 14 percent at overseas locations. Approximately 70 percent of all FSI non-language courses in 2010 were offered at the FSI campus in Arlington, and roughly half of the remaining courses at domestic locations were offered at State headquarters in Washington, D.C. See figure 6.

FSI has also increased its focus on distance learning in recent years. According to State officials, in addition to offering classroom training, FSI began offering distance learning courses in 2002. In fiscal year 2010, FSI offered 190 customized courses, including courses with little or no interaction with an instructor that allow participants to complete the course at their own pace; real-time courses in which students and instructors participate simultaneously via various technologies; and “blended courses,” which combine various delivery methods such as time spent in the classroom and online. In addition, FSI offers about 3,000 commercially developed courses that are available at all times in a wide variety of topics, including Microsoft applications and various business topics. According to FSI officials, distance learning allows FSI to provide training for LE staff overseas who otherwise would have been unable to access training.

State Provides Additional Training and Career Development Opportunities

While FSI is State’s primary training provider, other bureaus in the department also offer training for State and other federal government employees. For example, the Office of Training and Performance Support within the Bureau of Diplomatic Security offers security and law enforcement training to personnel, including Special Agents, Regional Security Officers, and others. The Diplomatic Security Training Center provides about 85 instructor-led courses, including the Foreign Affairs Counter Threat course for personnel who are deploying to critical threat environments such as Iraq, Afghanistan, and Pakistan. Other departmentwide bureau-sponsored training includes ethics training by the Office of the Legal Advisor; Equal Employment Opportunity laws and regulations training by the Office of Civil Rights; and emergency preparedness, property record keeping, procurement, logistics management, federal assistance, and safety awareness training by the Bureau of Administration.

State also offers several career development programs for employees. For example, in 2005 State introduced the Career Development Program for Foreign Service employees, which sets requirements for advancement into the senior ranks and focuses on developing appropriate professional, leadership, language, and technical skills at each level. State’s Civil Service Mid-Level Rotational Program provides opportunities for mid-level civil service employees to rotate to other bureaus to broaden their skills, increase their knowledge, and enhance their personnel and professional growth. In addition, State has mentoring programs for entry-level, as well as more experienced, employees.

According to State officials, bureaus, offices, and posts may also develop and offer their own training and professional development opportunities when the bureau, office, or post has specific needs that make it more efficient for it to develop the training itself or seek training outside the agency. Officials from several bureaus, offices, and posts told us they have designed various training programs that are tailored to the specific needs of their employees, which may include orientation, on-the-job training, mentorship opportunities, and annual conferences, workshops, or seminars. For example: The Bureau of Economic, Energy, and Business Affairs organizes a half- day orientation session every 6 months for employees who are new to the bureau.

The Bureaus of European and Eurasian Affairs; International Organizations; Consular Affairs; and Democracy, Human Rights and Labor host annual conferences and workshops for various employees in their bureaus.

The Bureau of Intelligence and Research coordinates with training programs at other federal agencies in the intelligence community to arrange for its employees to attend specialized training in subject matter that is not available through FSI.

The Bureau of International Narcotics and Law Enforcement provides training sessions that help employees gain familiarization with various areas of law enforcement.

Training officials at 11 of the 12 overseas posts we interviewed said their post sponsors a language program, which may include classes in the native language for Americans and English language courses for LE staff.

Officials at 8 of the 12 overseas posts we interviewed said they host some type of orientation for employees new to the post.

In addition, officials told us that three of State’s regional bureaus— Western Hemisphere Affairs, European and Eurasian Affairs, and East Asian and Pacific Affairs—operate regional centers that service various needs of the posts in their respective regions in Fort Lauderdale, Florida; Frankfurt, Germany; and Bangkok, Thailand. Officials at all of the regional centers told us they have a model for providing some training for employees in their respective region, including LE staff. These officials also noted that some of these courses were developed by FSI, while other courses have been designed and developed by the regional center.

State employees may also participate in various external training opportunities. For example, officials noted that certain civil service employees in particular may take advantage of human resource or technical courses offered through outside vendors such as the U.S. Department of Agriculture Graduate School, that are not available through FSI. In addition, employees may participate in several long-term external career development opportunities, which are generally available to tenured Foreign Service and civil service employees who are at the mid- and senior-grade levels. These programs normally last from 9 to 12 months and include the military schools (War Colleges and Commands) and nondegree fellowships, as well as bachelors and masters degree programs at various colleges and universities. For example, State data showed that a total of about 130 Foreign Service and civil service employees participated in long-term academic training in the 2009-2010 academic year. Further, in 2009, almost 80 State employees participated in long-term development programs related to interdepartmental collaboration on national security and were assigned to long-term rotational positions at agencies such as the Department of Defense and the Office of the Director of National Intelligence.

Types and Amount of Training Vary for Foreign Service, Civil Service, and LE Staff

State personnel receive varying types and amounts of training, depending on their position or category, career stage, specific training needs, and available resources. State data on employee training hours in fiscal years 2006 through 2010 show the following: Almost 90 percent of all training hours took place in the classroom, with about half of these hours for language training and the other half for non- language training.

Time spent in distance learning comprised about 7 percent of all training hours.

Training by external providers comprised 5 percent of training hours.

About 85 percent of employee hours in classroom training was for Foreign Service employees, with about half of this time for language training; about 15 percent of FSI classroom training was for civil service and LE staff.

Civil service employees spent about 70 percent of total hours in external training.

LE staff spent the largest amount of time in distance learning—about 47 percent of total hours in this type of training.

Figures 7 and 8 illustrate the types and amounts of training that State employees received in fiscal years 2006 through 2010.

State Workforce Training Incorporates Many Aspects of Effective Training Programs, but Strategic Weaknesses Exist

State has taken many steps to incorporate the interrelated elements of an effective training program—planning, design, implementation, and evaluation—into its training for personnel, but the department’s strategic approach to workforce training has several key weaknesses. State demonstrated a variety of examples of ways in which the department has endeavored to develop an effective training program, such as by compiling an annual training plan and implementing a range of training evaluation mechanisms. However, in our analysis of the extent to which State’s training program reflects key attributes identified in prior GAO guidance, we found several key gaps in the department’s efforts to strategically plan and prioritize training, ensure efficient and effective training design and delivery, and determine whether or how training and development efforts contribute to improved performance and desired results. These issue areas are generally connected with various attributes and indicators associated with more than one element of the training and development process, as planning, design, implementation, and evaluation efforts are often interrelated.

State’s Training Reflects Aspects of Effective Training Programs

State’s personnel training reflects numerous aspects of effective training programs, based on our assessment using the criteria GAO previously identified. (See app. II for a detailed listing of the elements with their associated attributes.) For example, we identified the following positive practices relevant to the major, interrelated elements of the training and development process.

Planning. State maintains a workforce training plan, as required by federal regulations. FSI leads efforts to prepare the training plan annually with input from other bureaus and offices; the plan is linked to State’s overall strategic plan, and presents a business case for proposed training investments, including training provided by FSI, the Bureau of Diplomatic Security, and other personnel training programs, such as external and long-term training opportunities. FSI’s director—the chief training officer for the department—and the Director General of the Foreign Service and Director of Human Resources are members of the department’s senior management, reporting to State’s Under Secretary for Management. According to State, in demonstration of its commitment to training, the department dedicates almost 1.5 percent—approximately $214 million in fiscal year 2010—of its program budget to personnel training. State’s Foreign Affairs Manual and Foreign Affairs Handbooks include information on the department’s legislative authorities and policies that may relate to, or require training and development for, Foreign Service, civil service, and LE staff. In addition, FSI publishes an annual schedule of courses, which provides information for employees on the more than 700 classroom courses FSI offers, as well as approximately 190 custom- developed distance learning courses.

Design. FSI highlighted its involvement of curricula and education experts as well as subject matter or technical experts in the development of courses as a positive practice. FSI officials stated all courses are designed to meet specific learning objectives. The schedule of courses generally includes information for each course such as a brief description, any prerequisites, course objectives, and relevant competencies and precepts. State has also designed programs that incorporate various cross-training opportunities and targeted experiences. For example, Foreign Service and civil service personnel may participate in rota assignments, congressional fellowships, OPM leadership seminars, or programs through the National Defense University, among other opportunities; and LE staff may participate in short-term details to other posts, or opportunities to “shadow” a more experienced employee, according to State officials. Additionally, State has made an effort to design and develop courses that use advances in technologies to enha nce its training efforts, another example of a positive practice. The number of distance learning offerings, as well as employee participation in distance learning, has increased in recent years. For example, State’s latest annual training plan reported that FSI developed 20 new custom distance learnin courses during the prior year. opportunities at some overseas posts, which may include group instruction via videoconference alo FSI has also piloted “blended” learnin ng with online course content.

In addition, during fiscal years 2006 through 2009, the total number of hours that personnel spent completing distance learning courses more than doubled—from about 113,000 hours in fiscal year 2006 to about 254,000 in fiscal year 2009. Data include hours spent completing all distance learning course offerings, including custom-developed courses, by Foreign Service, civil service, and LE staff over the time period. well as to encourage employees to accept the goals of training efforts and apply knowledge gained through training on the job. For example, FSI has an attendance policy requiring students to participate in 80 to 100 percent of a classroom course—depending upon duration and subject matter—and pass any applicable testing requirement in order to receive official course credit. The employee’s home bureau must generally reimburse FSI in the event that an employee does not show up or complete a course. In addition, FSI generally reviews and approves all applications for external training. Employees generally must sign an agreement to reimburse State if they do not complete an external training course they registered for, and employees must complete a post-training evaluation for any external training. Several posts we interviewed noted they have training agreements in place for training provided to LE staff, which may require personnel to submit a report to supervisors on the outcome of training or reimburse State for costs such as travel in the event they do not successfully complete a training course.

Evaluation. State has a range of training evaluation mechanisms in place, including mid- and post-training course evaluations, some incorporating supervisors as well as employees. Efforts to evaluate training in order to assess the extent to which it contributes to improved results is considered an essential component of an effective training program. The department also has a learning management system that can track delivery of training. For example, as noted previously, State provided data to us on training hours and courses completed by Foreign Service, civil service, and LE staff over the past 5 fiscal years, including data on language and non- language classroom and distance learning, as well as external training. Additionally, since 2006 FSI has conducted an annual training survey; for example, FSI reported most respondents to the 2010 survey were, in general, satisfied or very satisfied with training. Further, to comply with an OPM mandate, according to State, in 2007 the department conducted the biennial Quality of Work Life Survey, which addresses human capital issues and trends. State reported that the 2007 survey found a majority of respondents agreed with the statement, “I receive adequate training to do my job.”

Weaknesses Exist in State’s Strategic Approach to Personnel Training

Although State’s training practices and procedures reflect numerous attributes and indicators of an effective training program, we found gaps in six key areas (see table 4). Each of these issue areas broadly relates to multiple elements, attributes, and indicators throughout the interrelated training and development process. While an agency’s training program is not necessarily expected to address every indicator identified in the GAO guidance, based on our assessment, we identified strategic weaknesses related to these six issue areas as particularly important to ensuring effective planning, design, implementation, and evaluation of personnel training.

Although State has several practices in place to identify training needs, the department lacks a systematic, comprehensive training needs assessment process clearly incorporating all bureaus and posts, particularly at the occupational and individual levels. Our previous work identifying elements of effective training states that an agency can use organizational, occupational, and individual training needs assessments to help ensure training is connected to improving individual and agency performance in achieving results. OPM guidance refers to this as a multilevel training needs assessment process and states that to be successful, an assessment process should be ongoing, involve management at all levels, and be integrated into the agency’s program planning and budgeting process. Further, under Executive Order, agencies are to review, not less than annually, programs to identify training needs, establish priorities for training, and provide resources in accordance with those priorities. In addition, State’s Foreign Affairs Manual states that bureau executive directors are responsible for identifying bureau training needs. State guidance also notes that not conducting training needs assessments may cause time and resources to be wasted in developing and teaching skills that employees do not need, while true training needs and barriers may go unaddressed.

We found that State has some processes in place to identify broad or cross-cutting training needs at the organizational level. For example, FSI officials told us the institute identifies training needs and strategy in part through FSI officials’ participation in various departmental committees or working groups that may address training issues or discuss training needs. They noted that FSI’s Director, who acts as the Chief Training Officer for the department, attends the Secretary’s weekly staff meetings and holds monthly meetings with the Director General of the Foreign Service and Director of Human Resources. The Deputy Director participates in periodic meetings with the Bureau of Human Resources Office of Career Development and Assignments. FSI officials added they were holding weekly meetings with officials from the bureau on how to address training needs related to the recent increase in hiring. Further, they said the Dean of the School of Applied Information Technology participates in periodic meetings with State’s Chief Information Officer to stay abreast of information technology training needs. According to State’s Bureau of Human Resources, State aims to identify training needs related in particular to the department’s foreign policy mission and emerging issues.

However, State’s processes for identifying both organizational, cross- cutting, and more specific occupational training needs do not clearly incorporate all bureaus and posts. For example, officials from a number of bureaus told us that their bureaus had not conducted formal training needs assessments. Only one bureau indicated it had recently conducted an assessment relevant to training needs. To help identify cross-cutting as well as more specific occupational training needs, FSI officials said the institute holds quarterly meetings with bureau training officers, providing opportunities to discuss any bureau training needs and to share information, for example, about upcoming course offerings. However, several bureau training officials we met with noted that a bureau with a specific training need would generally reach out to FSI on a case-by-case or ad hoc basis. They said that FSI is generally responsive to ad hoc requests regarding specific training needs, but discussions at the quarterly meetings typically address topics such as training-related administrative processes, rather than addressing bureau training needs.

State’s Office of Inspector General (OIG) previously recommended that State conduct annual training needs assessments and implement a more comprehensive and systematic planning process for training incorporating all bureaus and posts, such as by requiring a training section in the department’s annual bureau and mission strategic plans, to help identify and address long-term training needs across locations and all categories of employees in the department. To address prior OIG findings, State reported that the department had added a training “annex” to all bureau and mission plans. However, State officials indicated the plans no longer include a training annex, owing to the department’s decision to streamline the strategic planning process in order to make it less burdensome for bureaus and posts. As a result, officials noted some bureau and mission plans include information on training goals, needs, or priorities, while others include no reference to training. None of the 12 posts we interviewed reported receiving guidance from headquarters that could help the post prioritize, plan, and budget for training. Some bureau and post officials stated they could benefit from additional guidance, and indicated that a more systematic training planning process across bureaus and posts could be helpful. However, several officials noted that such a process could become a “paperwork exercise” unless there were also a commitment to follow-through with implementation.

In addition, relevant to occupational training needs, the Foreign Affairs Manual states that training officers in each bureau are responsible for developing a bureau career guide outlining profiles of major occupations in the bureau including descriptions of the knowledge, skills, and abilities needed for each occupation and grade. However, most of the bureaus we met with had not developed a bureau career guide. Officials indicated they would generally rely instead on broader departmental guidance to help identify relevant training. For example, State guidance states that civil service employees must demonstrate competencies established by OPM. In addition, State has broadly identified “core precepts” for Foreign Service entry-, mid-, and senior-level employees, which are updated periodically. Technical competencies have also been identified for certain groups of employees, such as administrative or post management officers. Officials indicated that more specific roles and responsibilities, as well as any skills or abilities—though not necessarily training—required for different Foreign Service, civil service, and LE staff positions, are generally identified in job announcements. They noted they also rely on FSI guidance on training for employees in different career paths.

Also, to help identify individual training needs, bureau and post officials noted that Foreign Service employees generally rely on career development officers, in addition to supervisors at post, to provide guidance and counseling on training and development, including identifying necessary training. However, some post officials noted this support may vary. A number of Foreign Service employees responding to the open-ended question on training in the 2009 American Foreign Service Association survey also cited concerns about career development officer support. For example, some said their career development officers generally had too many people assigned to them and were ineffective at disseminating relevant information; were not responsive to e-mails or phone calls; and were more focused on filling positions than on supporting Foreign Service Officers’ career development. According to State’s Bureau of Human Resources, as of October 2010, State had 41 career development officers (CDO), each supporting at least 250 Foreign Service generalists and specialists. They noted that serving specialists and mid-level generalists in particular, some CDOs support as many as 450 employees. The bureau also has two dedicated staff—an assignments officer and a training officer—who help ensure that entry-level personnel are enrolled in required training. Human resources officials said that recent increases in State personnel had presented challenges to the provision of career development counseling, and that, although State had created new entry- level CDO positions as a result, the need for additional mid-level CDOs was under review. They added that they have also been working to streamline administrative and technical processes to allow officers to spend more of their time providing individual guidance and counseling on training.

Further, although State encourages all employees to complete an individual development plan (IDP), which can be a component of an individual training needs assessment process, bureau and post officials indicated that few Foreign Service or civil service employees have completed an IDP. As a result, it is not clear whether Foreign Service and civil service employees have adequate opportunity to discuss with their supervisors any training they may need to improve individual performance or prepare for future assignments. We previously reported that the use of IDPs to identify both short- and long-term developmental needs of each employee can help an agency incorporate employees’ developmental goals, and integrate the need for continuous and career-long learning, into its planning processes. State requires only certain entry-level civil service employees to complete an IDP. Some bureau officials stated they thought the IDP should be a broader requirement. Officials from several of the bureaus we met with noted that while the IDP is encouraged, it is not widely used by domestic bureau personnel, although training needs may be identified for civil service personnel as part of the annual performance appraisal process. Bureau and post officials indicated that, although training needs could be covered as part of the Foreign Service appraisal process, in contrast to the civil service appraisal, the Foreign Service appraisal form does not explicitly address training, and officials generally have not seen it in the appraisals.

Post officials from 11 of the 12 posts we interviewed said their posts use a work development plan (WDP), similar to an IDP, for LE staff. Officials noted that posts are generally responsible for coordinating and overseeing LE staff training. Some of these posts said it is a general practice or requirement for supervisors to work with LE staff at post to complete a WDP, including identifying any training needs, as part of the annual appraisal process. Post officials noted that they may review the employee development plans or solicit broader input from supervisors or other relevant personnel at the post, to help identify and prioritize training as part of the post’s annual budget and planning process. For example, officials at one post noted they annually solicit training needs from all sections at the post for the upcoming year and review the WDPs to compile a prioritized list of training. However, the senior human resources officer we spoke with at another post said that she primarily identifies training needs and puts together an annual budget request for training based on her own observations and oral discussions with relevant personnel at the post. She noted that although she had attempted to solicit broader, formal input from personnel at post on individual training needs, she had received minimal response. She added that she will review WDPs to help identify individual training needs, but because the WDP is not a requirement at post she must rely largely on her own observations and oral discussions with supervisors of LE staff.

While State does not have a systematic, comprehensive training needs assessment process, training officials we interviewed from several bureaus and posts cited various areas of potential training need for Foreign Service, civil service, and LE staff. For example, some bureau officials cited a greater general need for training personnel in areas such as project management, strategic planning, human resources, and budgeting, as well as in English and computer skills for LE staff in particular. Post officials also cited a general need for additional training in areas such as project and contract management and for strategic planning, public-private partnerships, NGO engagement, and enhanced coverage of certain countries in geographic area studies, among other areas. Further, a number of Foreign Service employees responding to the open-ended question on training in the 2009 American Foreign Service Association annual survey cited a range of potential training needs. For example, respondents cited needs for additional training for generalists relevant to their particular areas of work; additional training for specialists, including for Office Management Specialists; additional and enhanced leadership training; and improved and expanded language training, including expansion of language training overseas. Respondents also cited the need for a greater focus on training through their careers, such as by establishing and enforcing minimum requirements at various career stages beyond the current mandatory requirements. State officials noted that State has increased attention to LE staff training and development in recent years, although some officials noted in the past, LE staff often received little or no training despite significant need for training at some posts. Because Foreign Service employees generally rotate to a new post every 2 years, LE staff often provide the longevity, continued presence, and support that are critical to the successful day-to-day operations of overseas missions. In June 2009, FSI conducted a survey of post management to identify LE staff training needs. Several bureau and post officials told us they considered the results of the survey in planning LE staff training. For example, officials representing the Bureaus of Near Eastern Affairs and South and Central Asian Affairs said that as a result of the survey, they identified a particular need for supervisory skills training for LE staff working at certain posts in those regions, and offered two relevant workshops in the region in fiscal year 2010. FSI officials noted that as a result of the 2009 survey, they have been working to increase training offerings at posts by expanding the number and reach of adjunct faculty. They added that FSI plans to repeat the needs assessment periodically, though they have not set any specific time frames as of November 2010.

Since 2007, State human resource reports also noted that bureaus have not formally conducted annual training needs assessments, and identified this as an issue that should be addressed to help provide a realistic basis for planning, budgeting, and directing training. According to the reports, the Bureau of Human Resources intended to form an interoffice working group to develop a comprehensive plan and implementation guidance to support a department-wide effort for assessing training needs. State officials said they had recently begun developing a plan to address the reports’ findings, though they had not yet formed an interoffice working group as of November 2010.

FSI Training Continuums Include Some Inaccurate and Incomplete Information

We found that documents on training—known as training continuums— that FSI developed for employees do not include accurate and complete information on training. The continuums generally identify training and development for entry-, mid-, and senior-level employees, and contain other information related to career planning. We previously reported that providing transparent information for employees about career maps and paths, how training opportunities could help employees attain career goals, competency models, and training or professional requirements, can help agencies communicate the importance of training and their expectations for training programs to achieve results. FSI has developed and published various training continuums for State’s Foreign Service, civil service, and LE staff, including the Training Continuum for Civil Service Employees, the Training Continuum for Foreign Service Generalists, the Training Resource Guide for Foreign Service National Employees and Locally Employed Staff, and the Leadership and Management Training Continuum, among others.

State’s OIG reported that FSI began developing these training continuums in part to provide a framework for a departmental policy of mandatory training, as well as to provide road maps tailored to the particular needs of employees in various positions throughout their careers. In addition, the continuums state they were designed to provide a broad overview of appropriate training that should be considered as employees plan their careers in the department. For example, the training continuums provide information on leadership skills training requirements that State established under former Secretary of State Powell for mid-level and higher-ranked Foreign Service and civil service employees. The continuums also generally include information on other mandatory personnel training requirements, such as cyber security training, as well as information on recommended and suggested courses for employees. Specifically, the continuums state that required courses are defined as mandatory; recommended courses should be taken to provide knowledge and skills for successful job performance; and suggested courses are more specialized and should be taken depending upon job duties. In addition, the continuums generally include information on career development opportunities. For example, they note that State’s civil service, mid-level rotational program provides opportunities for civil service mid-level employees to rotate to other bureaus to broaden their skills, increase their knowledge, and enhance their personnel and professional growth. They also provide information on State’s Career Development Program for Foreign Service employees, which established certain mandatory and elective professional, leadership, language, and technical skills requirements that personnel must meet to be eligible for promotion to the Senior Foreign Service.

However, we found several issues that raise questions about the usefulness and reliability of the continuums as resources for employees in planning their training and development. For example, although we found that the Training Continuum for Civil Service Employees lists diversity awareness, ethics, and orientation training as “required” for certain groups of employees, and as “recommended” for other groups, a key official from FSI’s executive office stated that these courses are in fact mandatory requirements for all civil service employees. The official added that these mandatory requirements would also apply to Foreign Service personnel and are among the few across-the-board training requirements that State considers mandatory, along with cyber security and “No FEAR Act” training. In addition, LE staff have minimal mandatory training requirements other than cyber security and ethics training. The FSI official clarified that even though we found that some of the continuums list other, more specialized courses as required for certain employees, the department officially considers training mandatory, and tracks completion, only if the training is listed as such by statute. The FSI official said that in some cases, decisions regarding what courses would be listed in continuums as required, recommended, or suggested were not fully vetted throughout the agency, and that, as a result, the information may not be entirely accurate or complete.

In addition, while officials noted that bureaus and posts may designate specific courses as required for certain groups of employees, we found that these requirements are not always identified in the training continuums or other guidance for employees. Several bureau and post officials noted that personnel would generally rely on the training continuums to identify necessary training, including any specialized training that may be required or helpful to them in performing their jobs. However, officials from the Bureau of International Organizations said that although FSI’s course on mulitilateral diplomacy is required of all International Organizations personnel, it is not listed in the continuums.

Training officials from the Bureau of Democracy, Human Rights, and Labor said Foreign Service labor officers must take the FSI labor officer skills course, though it is not listed as a requirement in the continuum. Some bureau officials said that employee supervisors are generally responsible for ensuring that employees are aware of and take the required training. Although some bureaus and posts had developed their own training guidance—for example, on available or suggested training opportunities, or on procedures required to register for training—the guidance generally did not include information on any specific training designated as required, recommended, or suggested by the bureau or post for different employee groups.

The official from FSI’s executive office acknowledged that the continuums do not include complete and accurate information for employees on training, and noted they have not been reviewed to ensure they uniformly reflect departmental policies or standards. The official added that some of the continuums have not been updated in several years, there are not continuums for every employee group, and because the continuums include some discrepancies or inaccuracies, they should not be viewed as formal or official guidance. The official also noted that FSI was considering shifting to a new, standardized format for the training continuums going forward that would hopefully improve the process and make the continuums easier to update in an ongoing basis in the future. However, greater involvement and collaboration from other bureaus would be required in order for FSI to facilitate a comprehensive process to develop official continuums for each job series, and the official noted that some bureaus had not been very responsive to FSI’s requests for input in the past.

FSI officials noted that information included in the continuums regarding State’s Career Development Program is official, as these program requirements have been formally approved by the department. The officials said that State has also developed separate guidance, called playbooks, for certain groups of Foreign Service employees regarding Career Development Program requirements. State officials noted that both the continuums and the playbooks are resources that can be used by employees to identify training and help plan career development, although the continuums were developed for general informational purposes and were not approved by agency management, unlike the playbooks. However, the Career Development Program only applies to State’s Foreign Service employees. In addition, although the playbooks generally provide information on mandatory or elective leadership or technical skills training required for promotion, they do not provide broader information that is included in the continuums, such as information on recommended training that may not be relevant to promotion under the Career Development Program.

Although State has not established mandatory training for all employees at every career stage, some bureau and post officials said that specific training requirements and additional guidance for different employee groups could be helpful. In response to the State OIG’s prior recommendation that the department establish specific mandatory training for all employees at every career stage, State reported it intended to eventually mandate specific training for all employees throughout their careers; however, the mandatory training the department subsequently established initially focused on the leadership skills training. State also noted that although the department viewed training as critical to assuring that employees have the necessary skills to support State’s mission, implementing broader mandatory requirements would have serious resource implications. Nevertheless, some bureau and post officials we interviewed indicated that improved guidance on training priorities by career track, as well as possibly establishing additional specialized mandatory training requirements, could heighten the focus on training and help ensure that employees get the training they need.

State Has Not Developed Formal, Comprehensive Curriculum Design Guidance

Although State provided some examples of ways in which it aims to incorporate effective design practices for training and development efforts, we found that the department has not developed comprehensive written guidelines and criteria to be followed throughout the course design process. Our prior work identified several key indicators of effective training programs related to the training design process, such as mechanisms, procedures, or explicit criteria for determining whether to design training programs in-house or obtain services from a contractor or external source; comparing merits of different delivery mechanisms (e.g., classroom or computer-based training) and for determining the appropriate mix of mechanisms to ensure efficient and cost-effective delivery; ensuring an environment conducive to effective training through employment of quality instructors, facilitators, mentors, and coaches; involvement of line managers, technical experts, human capital professionals, and others to develop an integrated way to address specific performance gaps; and incorporating measures of effectiveness into course designs.

State has made an effort to incorporate effective design practices into its training for Foreign Service, civil service, and LE staff. For example, in demonstration of its efforts to identify cost-effective and robust options for designing training and development, State shared sample letters of obligation, such as a letter regarding its decision to provide in-house training services for an FSI language course following a streamlined cost competition. State also noted that its training includes a mix of delivery mechanisms, including classroom, distance learning, and on-the-job training, as delineated in its training continuums and schedule of courses. According to FSI officials, FSI employs civil service training experts and Foreign Service subject matter experts or technical experts to teach courses, and FSI’s Curriculum and Staff Development Division in the School of Professional and Area Studies offers courses for staff on training design and delivery. They noted that all FSI instructors and course managers are required to take training in theory and best practices in adult learning. In April 2007, FSI launched an adjunct faculty program, which allows State personnel who demonstrate appropriate qualifications to provide training domestically or overseas for FSI course credit. In addition, to help ensure the integrated development of training to help address specific performance gaps, according to State, FSI utilizes working groups comprised of curricula and education experts, as well as technical experts and practitioners of the subject matter in the course design process. As a specific example, State developed and implemented a plan to provide a combination of classroom and distance learning, auditorium sessions, user guides, and deskside assistance to domestic and overseas personnel to ensure the successful launch of a new technology platform for combining cables and e-mails. Also, officials provided some examples in which results of course evaluations were incorporated into subsequent course offerings.

Nevertheless, although FSI officials said that FSI generally follows accepted industry standards in developing courses, they stated that FSI does not have comprehensive, documented guidance or standards to be followed throughout the curriculum design process for personnel training. While FSI has developed some instructional and curriculum guidance that is included in its Administrative Procedures Handbook, the sections relevant to course development are specific to distance learning and not classroom training. In addition, the handbook notes that the sections containing course development process and style guides for distance learning are under development and not available. Further, the handbook does not include clear guidance for determining appropriate training delivery mechanisms or whether to design training in-house or obtain outside services, or for incorporating training evaluation results into course designs. Because of the lack of formal curriculum design guidance, FSI’s permanent staff, as well as those who may be on temporary assignment or working under contract to develop training, may be unaware of and not applying consistent and appropriate practices and standards across the board throughout the training design process.

State Lacks Data Needed to Assure That Support for Training Is Consistent and Appropriate

Although State collects some information on the cost and delivery of training, the department does not collect data needed for an analysis and comparison of training provided to employees in different groups, bureaus, regions, or posts. As a result, State cannot be assured that it is providing consistent and appropriate support and funding for training and development. Our prior work highlights the importance of quality data to evaluating the quality and effectiveness of training and development efforts, and found that tracking the cost and delivery of training is a key attribute of an effective training program. We also found that the ability of an agency to demonstrate consistent and appropriate support and funding for training, and evidence that the agency provides needed tools and resources to managers and employees for training, are indicators of effective training programs.

While State provided data on overall funding for training personnel that showed some trends, the department does not track data that would allow a more detailed analysis and comparison across various employee groups and locations. State provided data on overall funding for training State personnel from fiscal years 2006 to 2011, which showed the training budget increased from about $157 million in fiscal year 2006 (adjusted for inflation, expressed in 2010 dollars) to around $255 million in fiscal year 2010 (adjusted for inflation, expressed in 2010 dollars). (See table 5.)

State officials noted that FSI’s budget makes up the majority of funding for training State personnel and includes resources for training space, technology, instructors, curriculum development, salaries of full-time students at FSI, and FSI-funded external training costs. Bureaus also allot some money for external training not covered by FSI—bureau allotments ranged from a total of about $3 million in fiscal year 2006 to about $4 million in fiscal year 2009. However, according to State officials, the estimated funding for 2006 through 2009 does not fully reflect State’s efforts to train personnel, due to the way State tracks funding. Specifically, the amounts do not include funding spent by posts for training offered at post or for travel per diem for Foreign Service, civil service, and LE staff to participate in training regionally or in Washington, D.C.; funding spent by bureaus other than FSI for internal training-related efforts; or some additional training-related costs that may not be detailed as training- related in State’s operating budget.

In addition, State officials said that State does not collect detailed data showing funding spent for internal and external training; funding for training Foreign Service, civil service, and LE staff, and for employees in certain specific employee groups; and funding by bureau, region, and post.

State officials noted that although they lack a detailed breakdown of training funding, they track some data on employee participation in training. However, these data also do not allow for a complete or detailed analysis and comparison of training for employees in different groups or locations. Although State provided data on training hours and courses completed by Foreign Service, civil service, and LE staff for FSI classroom, distance learning, language, and external training from fiscal years 2006 through 2010, State officials said that a more detailed breakdown of this information by bureau, region, post, or employee group is not available. Further, although FSI’s training continuums provide some guidance for personnel on required, recommended, and suggested training for employees in different groups, officials said that State generally only centrally tracks employee completion of certain specific training mandated by statute, such as cyber security training, as well for the required leadership skills courses. State does not generally track participation in other required, recommended, or suggested training. As a result, for example, State could not provide data regarding the percentage of foreign affairs or political officers that had completed required, recommended, or suggested training for their areas of work. This type of data could provide information on potential variations or inconsistencies in training and help ensure employees are given sufficient and appropriate training and development opportunities. For example, data that State provided on the percentage of mid- and senior-level Foreign Service and civil service personnel who had completed the respective required leadership courses as of May 2010 showed some differences among employee groups (see table 6). The data showed that 100 percent of Ambassadors had completed the Ambassadorial Seminar. In addition, the data showed that a majority of Senior Foreign Service and Senior Executive Service, as well as Foreign Service and civil service GS-13 to GS-15 equivalents (FS-03 to FS-01), had taken the required leadership training, however, a greater percentage of civil service at each level were unenrolled or untrained. State reported that Foreign Service employees who are recommended for promotion to the next level must complete the respective requirement within a year or the promotion will not become effective; in comparison, civil service personnel are informed of the requirement but monitor their own compliance.

Although State tracks some data related to training funding and delivery, the department does not have sufficient information that could be used to ensure consistent and appropriate support for training, or to help determine whether managers and employees have needed training tools and resources. This is especially important given evidence of variances in training across the department. Training officials we interviewed at a number of bureaus and posts cited varying support for employee training and development. While some bureaus and posts indicated they had sufficient funding and support for training, others noted they faced significant resource challenges that impacted the ability of employees to get necessary training. For example: Bureaus. Officials from several bureaus said that finding time or resources for training generally was not a challenge. However, this was not the case for other bureaus. For example, officials representing two of State’s regional bureaus noted that a lack of time was a main constraint preventing employees from taking needed training. A training official representing two other bureaus said that taking employees away from their day-to-day work for training poses significant challenges, given that some employees have heavy workloads. The official added that in her experience, although a majority of bureau civil service supervisors have fulfilled the mandatory leadership training requirements, supervisors sometimes enroll but cancel owing to the “crisis of the day.” In addition, although officials from two bureaus said they provide individual guidance or counseling to each Foreign Service and civil service employee in the bureau, officials representing some other bureaus indicated their competing responsibilities do not allow them to spend much time providing guidance or counseling, and that their primary role was to process training requests.

Posts. Training officials we interviewed at 12 posts cited varied support and resources for training. While several posts said they had enough funding for training, other posts said funding was insufficient or had fluctuated in recent years. Further, officials at most of the posts we interviewed said finding time for training can be a challenge at post. Only one post said time was not an issue, because the post has a policy that allows each employee to set aside up to 2 to 3 hours a week for training. Officials from several posts, as well as a number of respondents to the 2009 American Foreign Service Association survey noted although Foreign Service personnel are generally expected to obtain needed training in between tours, personnel may not have time for the training due to the departing post’s holding onto staff as long as possible and the subsequent post’s requiring staff to arrive as soon as possible. In addition, while two posts noted they benefited from having personnel dedicated to training full time, training officials from the majority of posts we interviewed said their posts had no one dedicated to training full time. For example, the mission training officer at one large post said she is only able to spend about 10 percent of her time on training-related responsibilities.

Regional training. State also has regional centers that provide training, among other services, in partnership with FSI in three locations: the Western Hemisphere Affairs Training Division at the Florida Regional Center (Florida center) in Ft. Lauderdale, Florida; the Frankfurt Regional Support Center (Frankfurt center) Training and Development Center in Frankfurt, Germany; and the Regional Employee Development Center (Bangkok center) in Bangkok, Thailand. Officials noted these centers primarily support training for LE staff in their respective regions, and they offer training on-site and at other posts in their regions. However, support for the centers varies, and the centers do not formally serve employees working in each of State’s regions worldwide. For example, the Florida center is the only center with a formal agreement with FSI; under the agreement, the Bureau of Western Hemisphere Affairs funds an FSI staff member to direct and provide training through the Florida center. Officials from the Frankfurt center said they did not need a formal agreement in part because they already have personnel with adjunct faculty status. In contrast, officials from the Bangkok center said that they hoped to establish a more formal agreement that would allow for an FSI representative to be assigned to the center to help them further develop their program and obtain adjunct faculty status for their two full-time LE staff trainers. Officials from the Bankgok center noted these two trainers currently cannot teach any FSI courses and can only teach courses developed by the center that would not be eligible for FSI course credit. In addition, officials from the regional centers indicated that regional training services for the Bureaus of African Affairs, Near Eastern Affairs, and to some extent South and Central Asian Affairs are currently ad hoc, given that they are not formally served by any of the above centers. Officials from the Frankfurt center said that demand for their training from regions not formally covered by other centers, including from employees who work at posts in the African Affairs and Near Eastern Affairs regions in particular, often exceeds what the Frankfurt center can provide. For some general services operations courses, the officials said they have had 120 applicants for 10 to 25 spots.

State Has Not Developed a Formal Plan for Evaluating the Effectiveness of Training

Although State has various training evaluation mechanisms in place, the department lacks a training data collection and analysis plan. As a result, it is not clear whether or how State systematically makes decisions regarding how training programs will be evaluated using different methods or tools, or how results will be used. Our prior work highlights the importance of planning and conducting evaluations of the effectiveness of training and development efforts and notes that a data collection and analysis plan can set priorities for evaluations and systematically cover the methods, timing, and responsibilities for an agency’s data collection. While State has implemented mechanisms to evaluate training, including course evaluations and an annual training survey, these mechanisms do not fully incorporate LE staff, and primarily focus on participant satisfaction or reaction to training, rather than desired results, such as improved quality or efficiency of work.

State has a range of training evaluation mechanisms and procedures to obtain feedback from supervisors and employees on training, such as FSI’s annual training survey, and mid- and-post-training course evaluations. FSI has developed and updated lists of evaluations used for different courses, which generally include information such as the intended recipients (e.g., students or supervisors), frequency, and timing of the course evaluation, as well as responsibilities for compiling the evaluations and for reviewing responses received. FSI also provided several examples of adjustments to training efforts in response to feedback received through its course evaluation efforts. For example, the course manager for Office Management Specialists training provided a November 2009 memorandum summarizing a number of changes made to the entry-level training as a result of course evaluations, among other factors. In addition, FSI provided a summary of results of an October 2009 offering of the Iraq Provincial Reconstruction Team course. The summary highlighted changes that had been made prior to the October offering as well as recommendations for modifications to be made in the subsequent offering of the course.

However, while FSI has compiled lists of evaluation mechanisms used for different courses, these lists do not include guidance on setting priorities for training evaluations, or how to determine appropriate methods, timing, or responsibilities for evaluating training and development efforts. They also do not include guidelines to help ensure State makes an ongoing effort to improve the quality and breadth of data gathered. We found several potential gaps and areas for improvement in State’s efforts to evaluate personnel training. For example, FSI’s annual training survey is sent to a random sample of State’s Foreign Service and civil service employees and eligible family members but not LE staff. Likewise, several training-related questions are also included in the department’s biannual Quality of Work Life Survey, which is sent to a random sample of Foreign Service and civil service employees but not to LE staff. In addition, although FSI conducted a June 2009 survey of LE staff training needs, the survey was sent to post management, not to LE staff, and FSI had not set specific time frames for repeating the survey as of November 2010. Several officials noted that LE staff often identify training needs or issues through the process of creating a Work Development Plan, but not all of the posts we interviewed required these plans.

In addition, respondents to the 2010 Annual Training Survey indicated they were generally satisfied overall with FSI training delivery, programs offered, and customer service. However, relatively fewer respondents indicated they were satisfied with the number of courses offered and their ability to attend courses in their geographic region. Additionally, some respondents said that they did not receive training that would have been helpful, such as tradecraft (e.g., consular, political, or public diplomacy), leadership and management, language, or information technology end-user training. However, despite the diverse training needs and requirements of State personnel, FSI officials told us they were not able to break out responses to determine whether there were any differences, for example, between responses of Foreign Service and civil service employees, or those serving at headquarters or in overseas regions.

Course evaluations used by FSI’s schools vary and generally focus on participant satisfaction or reaction to training, rather than knowledge gained, behavior change, or impact of the training. We reviewed examples of course evaluations used by each of FSI’s schools— the Leadership and Management School, School of Applied Information Technology, School of Language Studies, and School of Professional and Area Studies—as well as within each of the 10 divisions under the School of Professional and Area Studies. We found that courses generally incorporated mid-course and end-of-course student surveys, while some also included post-training surveys (e.g., typically several months after the end of the course), and a few included supervisors. The surveys included various numbers of multiple choice and open-ended questions and generally addressed participant satisfaction or reaction to training but less frequently included the types of questions that could be used to help determine on-the-job behavior change or impact of training. For example, for the Stability Operations division under FSI’s School of Professional Area Studies, we reviewed several end-of-course surveys and post-training surveys. We found one end-of-course survey (Iraq Familiarization) included only questions regarding participant satisfaction with course content; one end-of-course survey (Foundations of Reconstruction & Stabilization Operations) included questions regarding participant satisfaction with training as well as self-assessments of knowledge gained; and four others (Afghanistan Familiarization, Afghanistan and Iraq Provincial Reconstruction Team end-of-course, and Afghanistan Provincial Reconstruction Team post-training) included the above types of questions as well as a question regarding on-the-job behavior change resulting from training.

While some of FSI’s training evaluation mechanisms include questions that relate to on-the-job behavior change or the impact of training, FSI’s Director of Curriculum and Staff Development said that the institute’s course evaluations tend to focus primarily on participant reaction to training. The official noted that the evaluations focus to some extent on knowledge or skills gained, and less on behavior change or impact of training. FSI officials noted that determining the direct impact of training can be difficult, particularly in terms of any resulting cost savings.

Performance Measures Do Not Fully Address Training Goals

State’s performance measures for training generally do not fully address training goals. We previously reported that an agency’s use of measures to assess training and development efforts is an important aspect of effective training programs. Our prior work also noted that training and development performance measures, aligned with the agency’s mission and goals, can be used to help ensure accountability and assess progress toward achieving results. Indicators of effective training programs include the use of both quantitative and qualitative measures that provide meaningful data on training policies and practices, and show how specific efforts promote mission accomplishment.

State and the U.S. Agency for International Development’s Strategic Plan for Fiscal Years 2007-2012 includes one overarching strategic goal related to training: “strengthening consular and management capabilities.” However, the goal is not focused exclusively on training; the plan states that the “overarching goal…in this area is to provide the best visa and American Citizen services possible…and to ensure a high quality workforce, including locally employed staff.” Among other priorities listed for the goal, one relates to training and notes that State and FSI will “integrate systems and coordinate strategies to improve the skill base, diversity, and performance of our workforce.” The plan does not include any targets or measures specific to training.

While State’s broader strategic plan does not include any training measures, the bureau strategic resource plans for FSI and for the Bureau of Human Resources include several training-related goals and measures, linked to the broader departmental strategic goal on training. Specifically, the plans include a total of five training-related goals, with two or three indicators or measures for each goal. However, the measures generally do not fully address the goals and are output- rather than outcome-related. As a result, they do not provide a clear means of determining whether State’s training efforts achieve desired results. Table 7 provides further details about each of the training goals and measures.

Conclusions

State has developed an extensive program to equip its personnel to fulfill State’s leadership role in world affairs and to advance and defend U.S. interests. In recent years in particular, State officials noted the department has enhanced its focus on training and development of Foreign Service, civil service, and LE staff, such as by establishing leadership skills training requirements and increasing distance learning. State has many training practices and processes in place reflecting the elements of effective training programs, including the development of an annual workforce training plan and hundreds of classroom and distance learning offerings, as well as cross-training and other career development opportunities for personnel. State has incorporated the use of technology to enhance training efforts, and implemented oversight mechanisms such as training agreements to help ensure employees take advantage of legitimate training. In addition, State has various mechanisms in place to evaluate training, such as an annual training survey and course evaluations.

However, we found strategic weaknesses and room for improvement in several key areas. First, without a systematic, comprehensive assessment of training needs, State cannot be assured training is connected to true needs and priorities. Second, without transparent, complete, and accurate information for all employees on training opportunities, including on any required or recommended training for specific positions, State employees may be hampered in their efforts to plan for training and development throughout their careers. In addition, without taking several steps to address weaknesses in State’s efforts to evaluate personnel training, it is not clear that State is sufficiently able to assess whether employees have the information, competencies, and skills they need to work effectively. For example, it is not clear that State’s training performance measures provide meaningful information to help the department determine the quality and effectiveness of training efforts. State also lacks a plan for training data collection and analysis, as well as adequate mechanisms for determining whether the department is providing consistent and appropriate support and funding for training all employees—including LE staff.

State’s budget and focus on training have increased in recent years, but the department has also faced, and will likely continue to face, fluctuating and constrained resources and competing priorities when determining what training is critical to its mission. Without concerted efforts to further incorporate effective practices, State cannot ensure training resources are targeted strategically, are not wasted, and achieve cost-effective and timely results desired, and thus cannot be assured that its employees are trained and equipped to meet the challenges of their mission.

Recommendations for Executive Action

We recommend that the Secretary of State take the following five actions: To help ensure that State’s personnel training is connected to improving individual and agency performance and that department resources are directed to actual training needs and priorities, direct FSI and the Bureau of Human Resources, in collaboration with other bureaus and offices, as appropriate, to develop and implement a plan for a systematic, comprehensive training needs assessment process, incorporating all bureaus and posts.

To enhance State’s efforts to provide transparent, complete, and accurate information to help employees plan training and development throughout their careers, direct FSI and other bureaus and offices, as appropriate, to collaborate in developing and updating information for employees on training to ensure that employees have complete and accurate guidance, including information on any mandatory, required, and recommended training for specific employee groups.

To help ensure that State’s performance measures for training provide meaningful data and more fully address the department’s training goals, direct FSI and the Bureau of Human Resources to review the performance measures and revise or enhance the measures, as appropriate.

To enhance State’s capacity to evaluate workforce training, direct FSI and the Bureau of Human Resources to develop a data collection and analysis plan for training, including guidance for determining the methods, timing, and responsibilities for training data collection, as well as how results will be used.

To improve State’s ability to determine whether it is providing consistent and appropriate support and funding for employee training and development across employee groups and locations, direct FSI and the Bureau of Human Resources to identify ways to improve the collection and analysis of training data and results, such as by enhancing the level of detailed information gathered to determine whether employees across groups and locations are getting needed training, and enhancing efforts to determine the impact of training. These efforts should also include steps to further incorporate LE staff into State’s training evaluation mechanisms.

Agency Comments and Our Evaluation

We provided a draft of this report to the Department of State. State provided written comments, which are reproduced in appendix IV.

State generally agreed with our recommendations, and expressed appreciation for the report’s recognition of the wide variety of training State offers for the department’s Foreign Service, civil service, and locally employed staff worldwide. State highlighted the importance of assessing training needs, and noted that regular review of training performance measures is appropriate. The department also stated that it strives to improve the accuracy, completeness, and usefulness of training data, and would look for ways to enhance its ability to assess the effectiveness of training and development efforts across employee groups and locations.

In our draft report, we recommended that the Secretary of State direct FSI to develop formal curriculum design guidance identifying guidelines and criteria to be followed throughout the course development process, to help ensure that clear and consistent criteria and procedures are followed throughout the curriculum design process. After we provided the draft report to State, the department provided documentation developed by FSI to address this recommendation. Specifically, State provided an official document developed in December 2010 that describes and lays out FSI’s course design procedures and guidance, including for determining course objectives, identifying quality instructors, and evaluating course design. As a result of these actions, we have removed the relevant recommendation from the final report.

State also indicated that senior leadership from the department’s Bureau of Human Resources and FSI were not interviewed as part of this review. However, we disagree with this characterization of our review. During the course of our review we conducted numerous interviews with relevant officials representing 26 State bureaus and offices in Washington, D.C., including senior officials from FSI and the Bureau of Human Resources— in some cases on multiple occasions. We also conducted semi-structured telephone interviews with State officials with training-related responsibilities at 12 overseas missions, and from State’s regional training centers located in Bangkok, Thailand; Ft. Lauderdale, Florida; and Frankfurt, Germany. Additionally, we had multiple discussions and e-mail communications regarding training-related issues and questions with officials from FSI and the Bureau of Human Resources who were designated by State as our primary contacts throughout the audit. These officials provided up-to-date and accurate information to us on State’s training efforts in response to our inquiries and represented the views of senior management, where applicable (see app. I for additional information on our objectives, scope, and methodology).

State also provided technical comments, which we have incorporated throughout the report, as appropriate.

As agreed with your office, unless you publicly announce the contents of the report earlier, we plan no further distribution until 30 days after the report date. At that time, we will send copies of the report to interested congressional committees and to the Secretary of State. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO web site at http://www.gao.gov.

Appendix I: Objectives, Scope, and Methodology

To address our objectives, our analysis focused primarily on the Foreign Service Institute’s (FSI) training, including leadership, management, professional, and area studies training contributing to diplomatic readiness of the Department of State’s (State) Foreign and civil service personnel, as well as locally employed staff (LE staff) overseas. In light of recent GAO work addressing shortfalls in State personnel’s foreign language skills, this report does not focus on language training. This report does not include an assessment of “hard skills” (e.g. security and law enforcement) training provided by State’s Bureau of Diplomatic Security within its scope. Specifically, this report addresses the following questions: (1) What is State’s purpose and structure for training personnel? (2) To what extent does State’s training for personnel incorporate the elements of effective federal training programs?

To identify State’s purpose and structure for training personnel, we reviewed and analyzed legislative, regulatory, and State policy and procedural criteria relevant to training, including information contained in State’s Foreign Affairs Manual on key training roles and responsibilities. We also collected, reviewed, and analyzed documents and data including State’s annual training plan; departmental and human capital strategic and workforce planning documents; organizational charts; information available on State’s Internet and intranet Web pages; information and data on recent FSI course offerings; and data on personnel participation in training for fiscal years 2006 to 2010. To assess the reliability of data on FSI course offerings and on personnel participation in training, we reviewed relevant documentation and interviewed knowledgeable agency officials. We determined data on FSI course offerings and on personnel participation in training were sufficiently reliable to permit an analysis of where courses are being offered and of the relative amount of time Foreign Service, civil service, and LE staff spent in training, including for FSI language and non-language classroom and distance learning, and external training. To further our understanding of State personnel training, we interviewed key officials at FSI, the Bureau of Human Resources, and other bureaus and offices in Washington, D.C.

To determine the extent to which State’s training for personnel incorporates the elements of effective federal training programs, in addition to the above, we completed a training assessment with input from State, using the prior GAO guidance. Specifically, we examined State documents and data relevant to each of the 32 attributes identified in the GAO guidance, and the indicators associated with each attribute. We used the results of this assessment to identify strategic weaknesses in State’s training based on criteria contained in GAO and the Office of Personal Management (OPM) guidance, as well State and other legislative and regulatory guidance and policy. We reviewed training evaluation mechanisms used by each of the four FSI schools—the Leadership and Management School, School of Applied Information Technology, School of Language Studies, and School of Professional and Area Studies—as well as within each of the 10 divisions under the School of Professional and Area Studies. We examined mid-, end-of-course, and post-training course evaluation results received for a nonrandom sample of about 37 offerings of various courses by the different schools and divisions. We also reviewed results of broader evaluations of FSI’s consular and language training. Although the results of these evaluations cannot be generalized to all courses and locations, they represent results for a broad range of courses across all schools and divisions. We examined State data on overall funding for training from fiscal years 2006 to 2010, as well as on the percentage of personnel that had completed State’s mandatory leadership skills training requirements as of May 2010. We reviewed results of FSI’s annual training survey of Foreign and civil service employees, as well as responses to training-related questions included in State’s biannual Quality of Work Life Survey and State’s responses to OPM’s biannual Federal Human Capital Survey. To assess the reliability of these data, we reviewed relevant documentation, including information regarding survey methodology and response rates. We determined that these data were sufficiently reliable to provide a general indication of employee satisfaction with training. We also analyzed responses to training-related questions included in the American Foreign Service Association’s 2009 survey of State Foreign Service members, including more than 1,000 responses to an open-ended question regarding whether and how State training could be improved. While the results of this survey are not generalizeable, they provided us with valuable insights into potential areas for improvement.

In addition, to enhance our understanding of State’s training practices and potential issues or challenges, we interviewed relevant officials from 26 State bureaus and offices in Washington, D.C., including FSI, the Bureau of Human Resources, and the six geographic bureaus. For example, we interviewed senior officials from FSI including FSI’s Director, Deputy Director, and the Executive Director for Management, as well as senior leadership representing each of the four FSI schools—the Leadership and Management School, School of Applied Information Technology, School of Language Studies, and School of Professional and Area Studies. Within the Bureau of Human Resources, we met with senior officials including the Chief of Career Development, Division Director of Career Development and Assignments, Director and Deputy Director of Overseas Employment, and Director of Policy Coordination. We also conducted semi-structured telephone interviews with State officials with training-related responsibilities at 12 overseas missions. We selected a nongeneralizeable sample of countries designed to ensure geographic diversity; our criteria for selection also included factors such as a range of post sizes and hardship differentials. We selected two missions located in each region— Africa, the Americas (Western Hemisphere), East Asia and the Pacific, Europe and Eurasia, Middle East and North Africa, and South and Central Asia. Since we did not conduct interviews with a generalizeable sample of overseas missions, our observations from these interviews are illustrative but may not be representative of training at all overseas locations. We also interviewed officials from State’s regional training centers located in Bangkok, Thailand; Ft. Lauderdale, Florida; and Frankfurt, Germany. Finally, we interviewed key officials from nongovernmental organizations including the Center for Strategic and International Studies and the Stimson Center.

We conducted this performance audit from July 2009 to January 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Attributes for Review of an Agency Training and Development Program

Following is a compilation of the 32 questions or attributes explored in detail throughout GAO guidance on assessing an agency’s strategic training and development efforts. These attributes are designed for federal agencies to consider in ensuring that training and development investments are targeted strategically and not wasted on efforts that are irrelevant, duplicative, or ineffective. The attributes are distributed across the four interrelated elements of the training and development process— planning, design, implementation, and evaluation. The guidance also includes numerous corresponding indicators for each attribute.

Planning

a. Does the agency have training goals and related performance measures that are consistent with its overall mission, goals, and culture? b. To what extent do the agency’s strategic and annual performance planning processes incorporate human capital professionals in partnership with agency leadership and other stakeholders in addressing agency priorities, including training and development efforts? c. How does the agency determine the skills and competencies its workforce needs to achieve current, emerging, and future agency goals and missions and identify gaps, including those that training and development strategies can help address? d. How does the agency identify the appropriate level of investment to provide for training and development efforts and prioritize funding so that the most important training needs are addressed first? e. What measures does the agency use in assessing the contributions that training and development efforts make toward individual mastery of learning and achieving agency goals? f. How does the agency incorporate employees’ developmental goals in its planning processes? g. How does the agency integrate the need for continuous and lifelong learning into its planning processes? h. Does the agency consider governmentwide reforms and other targeted initiatives to improve management and performance when planning its training and development programs? i. Does the agency have a formal process to ensure that strategic and tactical changes are promptly incorporated in training and development efforts as well as other human capital strategies as needed?

Design

a. What steps does the agency take to ensure that training is connected to improving individual and agency performance in achieving specific results? b. How is the design of the training or development program integrated with other strategies to improve performance and meet emerging demands, such as changing work processes, measuring performance, and providing performance incentives? c. Does the agency use the most appropriate mix of centralized and decentralized approaches for its training and development programs? d. What criteria does the agency use in determining whether to design training and development programs in-house or obtain these services from a contractor or other external source? e. How does the agency compare the merits of different delivery mechanisms (such as classroom or computer-based training) and determine what mix of mechanisms to use to ensure efficient and cost- effective delivery? f. Does the agency determine a targeted level of improved performance in order to ensure that the cost of a training or development program is appropriate to achieve the anticipated benefit? g. How well does the agency incorporate measures of effectiveness into courses it designs?

Implementation

a. What steps do agency leaders take to communicate the importance of training and developing employees, and their expectations for training and development programs to achieve results? b. Is there a training and performance organization that is held accountable, along with the line executives, for the maximum performance of the workforce? c. Are agency managers responsible for reinforcing new behaviors, providing useful tools, and identifying and removing barriers to help employees implement learned behaviors on the job? d. How does the agency select employees (or provide the opportunity for employees to self-select) to participate in training and development efforts? e. What options has the agency considered in paying for employee training and development and adjusting employee work schedules so that employees can participate in these developmental activities? f. Does the agency take actions to foster an environment conducive to effective training and development? g. What steps does the agency take to encourage employees to buy in to the goals of training and development efforts, so that they participate fully and apply new knowledge and skills when doing their work? h. Does the agency collect data during implementation to ensure feedback on its training and development programs?

Evaluation

a. To what extent does the agency systematically plan for and evaluate the effectiveness of its training and development efforts? b. Does the agency use the appropriate analytical approaches to assess its training and development programs? c. What performance data (including qualitative and quantitative measures) does the agency use to assess the results achieved through training and development efforts? d. How does the agency incorporate evaluation feedback into the planning, design, and implementation of its training and development efforts? e. Does the agency incorporate different perspectives (including those of line managers and staff, customers, and experts in areas such as financial, information, and human capital management) in assessing the impact of training on performance? f. How does the agency track the cost and delivery of its training and development programs? g. How does the agency assess the benefits achieved through training and development programs? h. Does the agency compare its training investments, methods, or outcomes with those of other organizations to identify innovative approaches or lessons learned?

Appendix III: State Organization

(S)

Coelor nd Chief of Sff (S/COS)

United Ntion (USUN)

D(L)

D(S) (S/ES) for Public Diplomcy nd Public Affir (R) (M)

Affir (T) (G) (A)

Mgement (IRM) (DRL)

Business Affir(EEB) Assnt (ISN)

Oce nd Int'l (CA) (M/MED)

Affir (PM) Assnt (OES) (DS) (OBO)

Implementtion (VCI) (PRM) (FSI) (RM) (HR) (OIG) (L)

Office of U.S.

Office of Global Women Issu(F) (S/P)

Issu (S/WCI)

Appendix IV: Comments from the Department of State

GAO Comment

1. We still believe that gaps exist in guidance available for the Foreign Service, as well as for civil service and locally employed staff, because key tools used to provide employees with information on training are not comprehensive. Specifically, we found that documents for employees on training—known as training continuums—that FSI developed do not include complete and accurate information. While the continuums state that they were designed to provide a broad overview of appropriate training that should be considered as employees plan their careers in the department, including information on mandatory, recommended, and suggested courses, we found issues that raised questions about the usefulness and reliability of the continuums as employee resources. For example, we found that specific training requirements designated by bureaus and posts for certain groups of employees are not always identified in the training continuums. A key official from FSI’s executive office stated that in some cases, decisions regarding what information would be listed in the continuums were not fully vetted throughout the agency. The official acknowledged that the continuums do not include complete and accurate information for employees on training, and noted that the documents have not been reviewed to ensure they uniformly reflect departmental policies or standards.

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Anthony Moran, Assistant Director; Lisa Helmer; Shirley Min; Joe Carney; Virginia Chanley; Kieran Cusack; David Dayton; Patrick Lockett; Reid Lowe; and Mary Moutsos provided significant contributions to the work. Etana Finkler, Farhanaz Kermalli, and Mona Sehgal provided technical assistance.

Related GAO Products

Foreign Language Capabilities: Departments of Homeland Security, Defense, and State Could Better Address Their Foreign Language Needs and Capabilities and Address Shortfalls. GAO-10-715T. Washington D.C.: July 29, 2010.

Aviation Security: Efforts to Validate TSA’s Passenger Screening Behavior Detection Program Underway, but Opportunities Exist to Strengthen Validation and Address Operational Challenges. GAO-10-763. Washington D.C.: May 20, 2010.

Department of State: Persistent Staffing and Foreign Language Gaps Compromise Diplomatic Readiness. GAO-09-1046T. Washington, D.C.: September 24, 2009.

Department of State: Additional Steps Needed to Address Continuing Staffing and Experience Gaps at Hardship Posts. GAO-09-874. Washington, D.C.: September 17, 2009.

Department of State: Comprehensive Plan Needed to Address Persistent Foreign Language Shortfalls. GAO-09-955. Washington, DC.: September 17, 2009.

Veterans’ Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors. GAO-08-561. Washington, D.C.: May 27, 2008.

State Department: Staffing and Foreign Language Shortfalls Persist Despite Initiatives to Address Gaps. GAO-07-1154T. Washington D.C.: August 1, 2007.

Border Patrol: Costs and Challenges Related to Training New Agents. GAO-07-997T. Washington, D.C.: June 19, 2007.

Department of State: Staffing and Foreign Language Shortfalls Persist Despite Initiatives to Address Gaps. GAO-06-894. Washington D.C.: August 4, 2006.

Department of Homeland Security: Strategic Management of Training Important for Successful Transformation. GAO-05-888. Washington D.C.: September 23, 2005.

Aviation Security: Flight and Cabin Crew Member Security Training Strengthened, but Better Planning and Internal Controls Needed. GAO-05-781. Washington D.C.: September 6, 2005.

Human Capital: A guide for Assessing Strategic Training and Development Efforts in the Federal Government. GAO-04-546G. Washington D.C.: March 1, 2004.

Human Capital: Selected Agencies’ Experiences and Lessons Learned in Designing Training and Development Programs. GAO-04-291. Washington D.C.: January 30, 2004.

U.S. Public Diplomacy: State Department Expands Efforts but Faces Significant Challenges. GAO-03-951. Washington, D.C.: September 4, 2003.