Department of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel
Published: 2011-03-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This testimony discusses the U.S. Department of State's (State) efforts to train its personnel. It is based on our report, which is being released today. Because State is the lead U.S. foreign affairs agency, its personnel require certain knowledge, skills, and abilities to equip them to address the global security threats and challenges facing the United States--including the threat of Al-Qaeda and other terrorist organizations, HIV/AIDS and other pandemics, environmental degradation, nuclear proliferation, and failed states. In fiscal years 2006 through 2010, State's funding for training personnel grew by about 62 percent, and the department requested more than $266 million in fiscal year 2011 for programs providing training in professional skills such as foreign language proficiency, area studies, information technology, consular duties, and others needed for the conduct of foreign relations. State's Foreign Service Institute (FSI) is the primary training provider for the department's more than 66,000 Foreign Service, civil service, and locally employed staff worldwide. Our prior work has identified staffing and foreign language shortfalls at State, including challenges the department has faced in filling positions at the mid-level in particular, and in attracting qualified personnel for some hardship posts. The department is currently in the midst of what it has called the most challenging military-to-civilian transition in U.S. history in Iraq, one of the posts of greatest hardship where State personnel serve. Recent departmental initiatives--in particular, "Diplomacy 3.0," a multiyear effort launched in March 2009 with a primary aim of increasing the size of State's Foreign Service by 25 percent and the civil service by 13 percent--have underscored the importance of training to equip personnel to fulfill State's leadership role in world affairs and to advance and defend U.S. interests abroad. Today's testimony will discuss State's purpose and structure for training personnel, including leadership, management, professional, and area studies training, contributing to diplomatic readiness of State's Foreign Service and civil service personnel and locally employed staff overseas. It will also discuss the extent to which State's personnel training incorporates elements of effective federal training programs.
section_title: What GAO Found
paragraphs: We found that State has taken many steps to incorporate the interrelated elements of an effective training program--planning, design, implementation, and evaluation--into its training for personnel, but the department's strategic approach to workforce training could be improved in several key areas. Specifically, we identified five areas where State can improve its training. First, State lacks a comprehensive training needs assessment process incorporating all bureaus and posts. Second, State developed guidance for employees about training opportunities, career paths, and how training can help employees attain career goals, but the guidance does not provide complete and accurate information. Third, State lacks a data collection and analysis plan for evaluating training, and thus cannot be assured that proper practices and procedures are systematically and comprehensively applied. Fourth, State could not sufficiently demonstrate consistent and appropriate support for training, because the department does not track detailed information on training cost and delivery that would allow for an analysis and comparison of employees in different groups, bureaus, regions, or posts. Lastly, State performance measures for training generally do not fully address training goals, and are generally output-rather than outcome-oriented.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the U.S. Department of State’s  (State) efforts to train its personnel. My testimony is based on our report,  which is being released today. Because State is the lead U.S. foreign  affairs agency, its personnel require certain knowledge, skills, and abilities  to equip them to address the global security threats and challenges facing  the United States—including the threat of Al-Qaeda and other terrorist  organizations, HIV/AIDS and other pandemics, environmental degradation,  nuclear proliferation, and failed states. In fiscal years 2006 through 2010,  State’s funding for training personnel grew by about 62 percent, and the  department requested more than $266 million in fiscal year 2011 for  programs providing training in professional skills such as foreign language  proficiency, area studies, information technology, consular duties, and  others needed for the conduct of foreign relations. State’s Foreign Service  Institute (FSI) is the primary training provider for the department’s more  than 66,000 Foreign Service, civil service, and locally employed staff  worldwide.
Our prior work has identified staffing and foreign language shortfalls at  State, including challenges the department has faced in filling positions at  the mid-level in particular, and in attracting qualified personnel for some  hardship posts. The department is currently in the midst of what it has  called the most challenging military-to-civilian transition in U.S. history in  Iraq, one of the posts of greatest hardship where State personnel serve.  Recent departmental initiatives—in particular, “Diplomacy 3.0,” a  multiyear effort launched in March 2009 with a primary aim of increasing  the size of State’s Foreign Service by 25 percent and the civil service by 13  percent—have underscored the importance of training to equip personnel  to fulfill State’s leadership role in world affairs and to advance and defend  U.S. interests abroad.
Today I will discuss State’s purpose and structure for training personnel,  including leadership, management, professional, and area studies training,  contributing to diplomatic readiness of State’s Foreign Service and civil  service personnel and locally employed staff overseas. I will also discuss  the extent to which State’s personnel training incorporates elements of  effective federal training programs.
Over the course of our work on this issue, we reviewed and analyzed data  and documentation related to State’s training efforts, such as strategic and  workforce planning documents, funding data, and data on personnel  participation in training, as well as legislative, regulatory, and State policy  and procedural criteria relevant to training. In addition, we reviewed  training evaluation mechanisms used by FSI. We interviewed key officials  from 26 State bureaus and offices in Washington, D.C., including FSI, the  Bureau of Human Resources, and the six geographic bureaus. We also  conducted semistructured telephone interviews with State officials with  training-related responsibilities at 12 overseas missions, and interviewed  officials from State’s regional training centers located in Bangkok,  Thailand; Ft. Lauderdale, Florida; and Frankfurt, Germany. With input  from State, we completed a training assessment to determine the extent to  which the department’s personnel training incorporates elements of  effective training programs. We used the results of this assessment to  identify any gaps in State’s training based on criteria identified in GAO, the  Office of Personnel Management (OPM), State, and other legislative and  regulatory guidance and policy. In addition, in light of work that we  recently published on shortfalls in State personnel’s foreign language  skills, we did not focus on language training. We also did not include  within our scope an assessment of “hard skills” (e.g., security and law  enforcement) training provided by State’s Bureau of Diplomatic Security.  We conducted this performance audit from July 2009 to January 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives. More information on our scope and  methodology and detailed findings is available in the full report.
In brief, Mr. Chairman, we found that State has taken many steps to  incorporate the interrelated elements of an effective training program— planning, design, implementation, and evaluation—into its training for  personnel, but the department’s strategic approach to workforce training  could be improved in several key areas. Specifically, we identified five  areas where State can improve its training. First, State lacks a  comprehensive training needs assessment process incorporating all  bureaus and posts. Second, State developed guidance for employees about  training opportunities, career paths, and how training can help employees  attain career goals, but the guidance does not provide complete and  accurate information. Third, State lacks a data collection and analysis plan  for evaluating training, and thus cannot be assured that proper practices  and procedures are systematically and comprehensively applied. Fourth,  State could not sufficiently demonstrate consistent and appropriate  support for training, because the department does not track detailed  information on training cost and delivery that would allow for an analysis  and comparison of employees in different groups, bureaus, regions, or  posts. Lastly, State performance measures for training generally do not  fully address training goals, and are generally output-rather than outcome- oriented.
State’s Annual Training Plan states that “the purpose of the department’s  training program is to develop the men and women our nation requires to  fulfill our leadership role in world affairs and to advance and defend U.S.  interests.” FSI is State’s primary training provider, offering entry-, mid-,  and senior-level training for employees as they progress through their  careers. State guidance outlines key training roles, including FSI’s primary  role in developing training policies and facilitating necessary training, and  the Bureau of Human Resource’s role in assigning employees to training  and working with FSI to help ensure it meets their needs. Other bureaus,  offices, and posts also share responsibilities for training. FSI offers over  700 classroom courses, and has recently increased its focus on distance  learning. We found that overall, about 40 percent of personnel training  over the last 5 fiscal years, on average, was in foreign language skills.  Other training for personnel generally focused on developing leadership,  management, and other professional and technical skills and knowledge.
State’s personnel training reflects numerous aspects of effective training  programs, based on our assessment using the criteria GAO previously  identified. For example, State maintains a workforce training plan, as  required by federal regulations. FSI leads efforts to prepare the plan  annually; the plan is linked to State’s overall strategic plan, and presents a  business case for proposed training investments. FSI also publishes an  annual schedule of courses, which provides information for employees on  FSI classroom and distance learning course offerings. The schedule of  courses generally includes information for each course such as a brief  description, any prerequisites, course objectives, and relevant  competencies. As another example of a positive practice, State has made  an effort to use advances in technologies to enhance training efforts. The  number of distance learning offerings, as well as employee participation in  distance learning, has increased in recent years. For example, State’s latest  annual training plan reported that FSI developed 20 new custom distance  learning courses during the prior year, and data showed time spent by  personnel completing distance learning courses more than doubled from  fiscal years 2006 through 2009—from about 113,000 hours in fiscal year  2006 to about 254,000 in fiscal year 2009. In addition, we found that State  has a range of training evaluation mechanisms in place, including mid- and  post-training course evaluations. Since 2006, FSI has conducted an annual  training survey. FSI reported most respondents to the 2010 survey were, in  general, satisfied or very satisfied with training.
However, although State has developed an extensive program for training  personnel, our analysis found several gaps in the department’s efforts to  strategically plan and prioritize training, ensure efficient and effective  training design and delivery, and determine whether or how training and  development efforts contribute to improved performance and desired  results. Each of the issue areas we identified broadly relates to multiple  elements, attributes, and indicators throughout the interrelated training  and development process. While an agency’s training program is not  necessarily expected to address every indicator identified in the GAO  guidance, based on our assessment, we identified strategic weaknesses  related to these areas as particularly important to ensuring effective  planning, design, implementation, and evaluation of personnel training.
For example, we found that    State lacks a systematic, comprehensive training needs assessment  process incorporating all bureaus and overseas posts. Since 2007, State  human resource reports noted that bureaus have not formally conducted  annual training needs assessments, and identified this as an issue that  should be addressed to help provide a realistic basis for planning,  budgeting, and directing training. According to the reports, the Bureau of  Human Resources intended to form an interoffice working group to  develop a comprehensive plan and implementation guidance to support a  department-wide effort for assessing training needs. However, State had  not yet formed an interoffice working group as of November 2010.
State developed guidance—known as training continuums— to provide  information for employees about training opportunities, career ladders  and paths, and how training can help employees attain career goals, but  the guidance documents do not provide complete and accurate  information for employees. While the documents state that they were  designed to provide a broad overview of appropriate training that should  be considered as employees plan their careers in the department,  including information on mandatory, recommended, and suggested  courses, we found issues that raised questions about their usefulness and  reliability as employee resources. For example, we found that specific  training requirements designated by bureaus and posts for certain groups  of employees are not always identified in the training guidance. A key  official from FSI’s executive office acknowledged that the guidance  documents do not include complete and accurate information for  employees on training, and noted that the documents have not been  reviewed to ensure they uniformly reflect departmental policies or  standards.
State has not developed a data collection and analysis plan for evaluating  training, which could help ensure that appropriate procedures and criteria  for evaluating training are systematically applied across the board. As a  result, it is not clear whether or how State systematically makes decisions  regarding how training programs will be evaluated using different methods  or tools, or how results will be used. Our prior work highlights the  importance of planning and conducting evaluations of the effectiveness of  training and development efforts and notes that a data collection and  analysis plan can set priorities for evaluations and systematically cover the  methods, timing, and responsibilities for an agency’s data collection.  While State has implemented mechanisms to evaluate training, including  course evaluations and an annual training survey, these mechanisms do  not fully incorporate locally employed staff, and primarily focus on  participant satisfaction or reaction to training, rather than desired results,  such as improved quality or efficiency of work.
State could not sufficiently demonstrate consistent and appropriate  support for training, because it does not track detailed data and  information on training cost and delivery that would allow for an analysis  and comparison of employees in different employee groups, bureaus,  regions, and posts. For example, State could not provide data on the  percentage of foreign affairs or political officers that had completed  required, recommended, or suggested training for their areas of work.  Although State tracks some data related to training funding and delivery,  the department does not have sufficient information that could be used to  ensure consistent and appropriate support for training, or to help  determine whether managers and employees have needed training tools  and resources. This is especially important given evidence of variances in  training across the department. For example, while training officials we  interviewed at some bureaus and posts indicated they had sufficient  funding and support for training, others noted they faced significant  resource challenges that impacted the ability of employees to get  necessary training.
State has developed several training-related goals and measures, but the  measures do not fully address the goals, and are generally output rather  than outcome oriented. As a result, they do not provide a clear means of  determining whether State’s training efforts achieve desired results. For  example, one training goal listed in FSI’s fiscal year 2012 strategic  resource plan, “workforce meets priority diplomatic and operational  requirements as a result of FSI training,” includes priorities and objectives  to expand and enhance language training, support training in stability  operations, support new hire training, and enhance public diplomacy  training. However, the goal’s two measures, “language training success  rate at FSI,” and “development of training continuum to support State’s  Office of Reconstruction and Stabilization,” are both output measures and  do not fully address the priority areas for the goal, such as support for new  hire training or public diplomacy training.
State’s budget and focus on training have increased in recent years, but the  department has also faced, and will likely continue to face, fluctuating and  constrained resources and competing priorities when determining what  training is critical to its mission. Without concerted efforts to further  incorporate effective practices, State cannot ensure training resources are  targeted strategically, are not wasted, and achieve cost-effective and  timely results desired, and thus cannot be assured that its employees are  trained and equipped to meet the challenges of their mission.
Our report being issued today includes several recommendations for the  Secretary of State to improve strategic planning and evaluation of the  department’s efforts to train personnel, including for improvements to  State’s efforts to assess training needs and efforts to ensure training  achieves desired results. State generally agreed with our findings and  recommendations.
Mr. Chairman, this concludes my prepared statement. I would be pleased  to respond to any questions you or other Members of the Subcommittee  may have at this time.
subsections: section_title: GAO Contact and Staff Acknowledgments
paragraphs: For questions regarding this testimony, please contact Jess T. Ford,  (202)512-4268 or fordj@gao.gov. In addition to the contact named above,  Anthony Moran, Assistant Director; Lisa Helmer; Shirley Min; Joe Carney;  Virginia Chanley; Kieran Cusack; David Dayton; Patrick Lockett; Reid  Lowe; and Mary Moutsos provided significant contributions to the work.  Contact points for our offices of Congressional Relations and Public  Affairs may be found on the last page of this testimony.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.