Acquisition Workforce: Federal Agencies Obtain Training to Meet Requirements, but Have Limited Insight into Costs and Benefits of Training Investment
Released: 2013-04-16T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: The acquisition workforce manages and oversees billions of dollars in acquisition programs and contracts to help federal agencies get what they need, at the right time, and at a reasonable price; therefore, it is important that agencies provide adequate training to this workforce. In this review, GAO identified (1) the role of OFPP and the FAI in assisting agencies in meeting certification requirements; (2) agencies' approaches to providing training; and (3) the extent to which agencies collect information on the costs and benefits of their acquisition training. To determine OFPP and the FAI roles, GAO analyzed relevant legislation. GAO obtained information from 23 federal agencies on their training approaches through a questionnaire, and selected 4 agencies--the Departments of Education and the Treasury, DHS, and VA--to provide illustrative examples. GAO used its questionnaire and a subsequent data call to obtain information on how agencies collect information on the costs and benefits of their training.

section_title: What GAO Found
paragraphs: The Office of Federal Procurement Policy (OFPP) sets standards and policies for the federal acquisition workforce, and has established certification requirements, including minimal training, for the three main acquisition roles--contracting staff, Contracting Officer's Representatives, and Program/Project Managers--to promote the development of government-wide core acquisition competencies and facilitate mobility across agencies. DOD follows separate certification standards. The Federal Acquisition Institute (FAI), which is responsible for fostering and promoting the training and development of the acquisition workforce, works closely with OFPP and has initiatives underway to improve the collection and management of training information, including cost data and course evaluations; streamline communication of acquisition training guidance; and coordinate efforts to leverage acquisition workforce training resources throughout the government.

To support efforts for the acquisition workforce to attain and maintain federal certification requirements, most agencies (17 of 23) provide the majority of their acquisition training using external sources--vendors, FAI, the Defense Acquisition University, or other agencies. The Departments of Homeland Security (DHS), the Treasury, and Veterans Affairs (VA) operate their own permanent centers with dedicated resources that train the agency's acquisition workforce. Education reported using a different approach to providing acquisition training--it gets most of its training from other government entities. Federal agencies face similar challenges in providing training to their acquisition workforce. The top challenges reported by agencies in obtaining training for their acquisition workforce involved having sufficient resources. Twenty of 23 agencies identified obtaining adequate funding and 19 of 23 identified obtaining sufficient staff to manage training as challenging. In addition, almost half of the agencies reported that the fundamental step of identifying the acquisition workforce is a challenge especially when members of the workforce are involved in acquisitions as a secondary and not primary duty.

The training cost data that agencies collect is not comparable and agencies have limited information on the benefits of their acquisition workforce training investments. Although almost all agencies provided some cost data in response to GAO's questionnaire and subsequent data call, the agencies' cost data did not allow for government-wide assessment of their training investment. To date, FAI's efforts to collect training cost data has also met with limited success. Cost data collected in 2012 by GAO and FAI included different cost components-- such as facilities, travel, and instructors--which do not allow for government-wide analysis. Having comparable training cost data are important to inform FAI efforts to establish government-wide contracts for training. As for determining benefits of training, 7 of 23 federal agencies reported having no metrics, not even basic endof- course evaluations. Without basic data, agencies do not have insight into the benefits of their acquisition workforce training efforts.

section_title: What GAO Recommends
paragraphs: GAO recommends that OFPP help ensure that agencies collect and report comparable cost data and assess the benefits of acquisition training by (1) providing further guidance on the cost data agencies are to report annually, and (2) requiring agencies to analyze course evaluations, at a minimum, to help assess the benefits of training investments. OFPP concurred with the recommendations and indicated it has started to take actions to provide additional guidance.

section_title: Letter
paragraphs: Constrained budgets and the need to address gaps in critical federal  workforce skills point to the need for federal agencies to ensure their  training and development programs are effective, in order to provide  personnel with the skills essential to fulfilling their agencies’ missions. The  shortage of trained acquisition personnel hinders agencies from  managing and overseeing acquisition programs and contracts that have  become more expensive and increasingly complex. As a result, the  federal government is at risk for significant overcharges and wasteful  spending of the billions of dollars it spends for goods and services each  year. For fiscal year 2011, 23 Chief Financial Officers Act agencies had  about 83,000 federal employees who managed and oversaw over $160  billion in acquisitions. The Office of Management and Budget’s Office of  Federal Procurement Policy (OFPP), the Federal Acquisition Institute  (FAI), and the executive agencies play important roles in helping to  ensure that acquisition workforce professionals meet federal training  standards and that training dollars are used effectively.

The acquisition workforce plays a key role in managing programs and  overseeing contracts to help agencies get what they need, at the right  time, and at a reasonable price. Accordingly, you asked us to evaluate  approaches of federal agencies, other than the Department of Defense  (DOD), to train the acquisition workforce. We reviewed the efforts of 23  federal agencies to provide training to the acquisition workforce, as well  as the efforts of OFPP and FAI to support this training throughout the  government. Specifically, this report addresses the (1) role of OFPP and  FAI in promoting Federal Acquisition Certification standards and assisting  agencies in meeting acquisition workforce training requirements, (2)  approaches agencies use to provide training to their acquisition  workforces, and (3) the extent to which agencies collect information on  the costs and benefits of their acquisition training. You were also  interested in the grants workforce, which we will report on separately.

To identify the roles of OFPP and FAI in promoting Federal Acquisition  Certification standards, we analyzed legislation on their authority and  relevant guidance they provided to agencies, and interviewed their  officials. To provide government-wide information on federal agencies’  approaches to providing acquisition workforce training, we administered a  questionnaire to 23 agencies from July to August 2012 to obtain data  about their current acquisition-specific training approaches, including the  sources used for courses (in particular whether courses were provided by  the agency itself or by external sources, such as a commercial sector  vendor, FAI, the Defense Acquisition University (DAU) or another  agency), facilities, budgetary and staffing resources, methods of training,  and challenges faced in providing acquisition training.2 We received  questionnaire responses from all 23 agencies, although not every agency  responded to every question. One agency provided responses from its  headquarters procurement office and three bureaus. We consolidated the  information from the four surveys to ensure the tabulations eliminated  duplicative counting of the agency and to provide averaged data, as  appropriate. A summary of the agencies’ responses to each question is  provided in appendix I. To determine the information agencies gather on  the costs and benefits of their training approaches, we supplemented the  questionnaire responses by soliciting cost and metrics data from the  agencies using a data collection instrument from October to November  2012. We also reviewed the agencies’ most recent Acquisition Human  Capital Plans (AHCP), dated March 2012.

The Department of Defense is excluded from this review because we have previously  reported on its acquisition workforce training and its follows separate acquisition workforce  certification standards specified in the Defense Acquisition Workforce Improvement Act,  Pub. L. No. 101-510, § 1202(a) (1990) (codified as amended at 10 U.S.C. §§ 1701-1764  (2013)). See GAO, Defense Acquisition Workforce: Better Identification, Development,  and Oversight Needed for Personnel Involved in Acquiring Services, GAO-11-892  (Washington, D.C.: Sept. 28, 2011) and Defense Acquisition Workforce: DOD’s Training  Program Demonstrates Many Attributes of Effectiveness, but Improvement Is Needed,  GAO-11-22 (Washington, D.C.: Oct. 28, 2010).

Veterans Affairs (VA) to provide illustrative examples. DHS, Treasury,  and VA are agencies that operate dedicated acquisition workforce training  facilities—permanent centers with dedicated resources that provide  training specifically for the agency’s acquisition workforce. Education is a  smaller agency that has no dedicated training facility for its acquisition  workforce. We also discussed the extent to which these agencies were  aware of and had applied leading training investment practices reported  in our prior work, which the Office of Personnel Management and experts  had agreed should be implemented by agencies to support effective  training investment decisions.3 Of the eight leading practices GAO  identified, we focused on four dealing primarily with determining the costs  and effectiveness of training: (1) identifying the appropriate level of  investment to provide for training and development efforts and prioritizing  funding so that the most important training needs are addressed first; (2)  having criteria for determining whether to design training and  development programs in-house or obtain these services from a  contractor or other external source; (3) tracking the cost and delivery of  training and development programs agency-wide; and (4) evaluating the  benefits achieved through training and development programs, including  improvements in individual and agency performance. Further details of  our objectives, scope, and methodology are provided in appendix II.

We conducted this performance audit from February 2012 to March 2013,  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The Office of Personnel Management, as the federal government’s  human capital manager, provides leadership and guidance on  establishing and operating efficient federal training and development  programs throughout the government. It provides advice and assistance  to agencies on training and development programs so that those  programs support strategic human capital investment.

GAO, Federal Training Investments: Office of Personnel Management and Agencies Can  Do More to Ensure Cost-Effective Decisions, GAO-12-878 (Washington, D.C.: Sept. 17,  2012).

OFPP has specific responsibilities related to acquisition workforce training  to help ensure that the skills needed to handle the complexities of  acquisition programs and contracts are maintained. The Office of Federal  Procurement Policy Act requires executive agencies and the OFPP  Administrator to establish management policies and procedures for the  effective management of the acquisition workforce—including education,  training, and career development.4 In 2005, OFPP defined the civilian  acquisition workforce to include, at a minimum, professionals serving in  the contracting series (GS-1102), Contracting Officers, Purchasing series  (GS-1105), Program/Project Managers (P/PM), and Contracting Officer’s  Representatives (COR) or the equivalent, and additional positions  identified by the agency. Participation in the acquisition workforce may  be on a full-time, part-time, or occasional basis.

41 U.S.C. § 1703.  41 U.S.C. § 1703(c)(2).

The Federal Supply Schedules program consists of contracts awarded by the General  Services Administration or the Department of Veterans Affairs for similar or comparable  goods or services, established with more than one supplier, at varying prices. Federal  Acquisition Regulation (FAR) § 8.401 and § 8.402. The Schedules offer a large group of  commercial products and services ranging from office supplies to information technology  services. Governmentwide Acquisition Contracts (GWAC) are considered multi-agency  contracts but, unlike other multi-agency contracts, are not subject to the same  requirements and limitations, such as documentation that the contract is in the best  interest of the government as set forth under the Economy Act. GWACs are contracts for  information technology established by one agency for government-wide use that are  operated—(1) by an executive agent designated by the Office of Management and Budget  pursuant to 40 U.S.C. § 11302(e); or (2) under a delegation of procurement authority  issued by the General Services Administration (GSA). FAR § 2.101. The Clinger-Cohen  Act of 1996 authorized GWACs to be used to buy information technology goods and  services. Pub. L. No. 101-510, § 1202 (1996), (codified at 40 U.S.C. § 11314(a)(2)). All  agencies can order goods and services directly through the Schedules contracts and  GWACs.

Board of Directors, works closely with OFPP as well as the DAU to  collaborate on many issues involved with training and developing the  federal acquisition workforce. The Defense Acquisition Workforce  Improvement Act required the Secretary of Defense to establish and  maintain DAU in order to provide training for the DOD acquisition  workforce.8 FAI differs notably from DAU, however, in that it shares much  of the responsibility for training the civilian acquisition workforce with the  agencies and has fewer budgetary and staffing resources. For example,  FAI currently has 12 staff, while DAU has over 700 faculty and staff.

The Administrator of OFPP is also required to ensure that agencies  collect and maintain standardized information on the acquisition  workforce. Agencies, in consultation with OFPP, can structure their  management and oversight of training for their acquisition workforce in  such a manner as best supports the agency. Key agency positions  involved in managing the acquisition workforce within the agencies are  described in table 1.

Acquisition training-related responsibilities  Responsible for developing and maintaining an agency acquisition career  management program to ensure an adequate professional acquisition workforce.  Participates in the strategic planning and performance evaluation process to develop  strategies and specific plans for training to rectify deficiencies in the skills and  abilities of the acquisition workforce. Responsible, in consultation with the Acquisition  Career Manager, for reporting acquisition workforce data to the Human Capital Office  for the agency’s human capital plan.  Responsible for providing management direction of an agency’s procurement  system. In some agencies, CAOs oversee or assume this role to help ensure  implementation of agency acquisition workforce policy. Responsible for identifying  training and development needs of the acquisition workforce.   Leads an agency’s acquisition career management program and ensures that its  acquisition workforce meets the skills and capabilities required in OMB, OFPP, and  agency policies.

10 U.S.C. § 1746. provided by the agency, which may be taught using in-house staff or  contractor employees; (2) government entity training provided by another  federal agency, FAI, or the DAU; and (3) vendor training provided by a  commercial sector company, which is generally available to multiple  agencies at the same time. For the purposes of this report we refer to  government entity and vendor training as external sources.

section_title: OFPP and FAI Provide Policy, Guidance, and Assistance to Help Improve Government-wide Training Efforts
paragraphs: OFPP sets standards and policies for the training and development of the  federal acquisition workforce. Its efforts include strengthening workforce  planning requirements and setting standards for core acquisition training  by establishing certification requirements. FAI conducts activities that  support and assist civilian agencies in training and development of their  acquisition workforces. FAI efforts include: improving the collection and  management of training information, including cost data and course  evaluations; streamlining the communication of acquisition training  guidance; and coordinating efforts to maximize acquisition workforce  training investments government-wide.
subsections: section_title: OFPP Promotes Government-wide Practices for Acquisition Workforce Training and Development
paragraphs: The Federal Acquisition Certification requirements were issued in 2005 for contracting  professionals and 2007 for Contracting Officer’s Representatives and Program/Project  Managers. provided in the agencies’ plans to develop a government-wide snapshot  of the acquisition workforce and to identify practices for sharing  throughout the government. Table 2 provides an overview of OFPP’s  legislative responsibilities related to acquisition workforce training.

41 U.S.C. § 1201(a)(1). workforce. FAI reported that while its spending plan for fiscal year  2012 could provide $3 million for courses, training requests submitted  by the agencies totaled in excess of $18 million.

FAI is working with a number of vendors who provide acquisition  workforce training to establish government contracts that would  provide uniform costs for standardized training courses needed for the  FAC programs. FAI plans to have contracts in place in 2013 that  would be available for agencies to use to provide fiscal year 2014  training.

Assisting federal agencies with their acquisition human capital  FAI offers guidance and direction on human capital planning through  its instructions for agencies’ annual AHCP submissions. Beginning in  2010, OFPP has required agencies to submit an annual AHCP each  March that provides the agency’s strategies and goals for increasing  both the capacity and capability of the acquisition workforce. To assist  the agencies in preparing each submission, FAI provides them with a  report template that includes specific topics to be included. We  observed that the fiscal year 2012 AHCP submissions provided  information on the agencies’ goals for strengthening the acquisition  workforce, including certification goals, and planned training initiatives  for the current and future fiscal years.

41 U.S.C. § 1201(a)(11).  41 U.S.C. § 1201(a)(3).

FAI recently upgraded the government-wide acquisition career  information management system it maintains for all agencies.  In  2011, FAI replaced the Acquisition Career Management Information  System (ACMIS) with the FAI Training Application System (FAITAS),  which allows registration for courses, tracking of individuals’ training  records, and other information management tools. For example,  agencies can now manage the certification approval process, track an  individual’s continuous learning progress, and search for courses  offered by FAI and other agencies that use FAITAS to enroll  participants in their courses. FAI reports that five agencies currently  use the FAITAS registration function to alert other agencies to unfilled  seats within a specific agency’s training course. Additional features of  the system include a course scheduling module; a business  intelligence tool for agencies to identify training locations and course  availability; and a communication tool for broader outreach to the  acquisition workforce. For example, an OFPP official explained that  FAITAS can be used to send e-mail updates on OFPP policy,  guidance, or other OFPP or FAI initiatives to all individuals registered  in the system.

FAI conducts periodic surveys of agencies to collect data about the  acquisition workforce for OFPP.  For example, FAI conducts the bi- annual Acquisition Workforce Competency Survey to collect  information on agency acquisition workforces’ skills and abilities and  to identify competency gaps within and across agencies. FAI officials  explained that the survey results provide a government-wide view of  which courses are in demand that informs their efforts to achieve  economies of scale when providing training.

FAI reports an annual count of the acquisition workforce as it has  done for more than the past decade; however, its count has been  limited to professionals serving in four job series—General Business  and Industry (GS-1101), Contracting (GS-1102), Purchasing (GS- 1105), and Procurement Clerical and Assistance (GS-1106). FAI  began gathering data on CORs and P/PMs as part of the acquisition  workforce count in fiscal year 2007. In its recent report on the  acquisition workforce count, FAI reported that 33,271 personnel were  employed in the acquisition workforce’s four main job series in fiscal  year 2010, and an additional 47,959 and 4,186 personnel were  identified as CORs and P/PMs, respectively, by the agencies in their  AHCPs—which represents about 85,000 individuals in the civilian  acquisition workforce for fiscal year 2010. FAI does not directly collect  data for the workforce count, and has acknowledged that agencies  have some problems identifying CORs and P/PMs but the data have  been improving.

Facilitating interagency intern and training programs, as  According to officials, FAI also conducts outreach and leverages  training courses government-wide. In 2012, FAI met with officials from  each agency to discuss training efforts and locate training spaces that  could be used by FAI and others when supplying courses through  vendors. FAI officials reported that these agency visits provided  insights regarding how the agencies’ training programs varied in terms  of their organization and resources and identified duplicative training  efforts.

In addition, FAI provides opportunities to share information across the  acquisition workforce community. FAI began providing webinars on  current acquisition issues in fiscal year 2012, and launched a  newsletter for the acquisition workforce community in December  2012. According to officials, the newsletter will provide more in-depth  information on policy changes, human capital initiatives, and tools and  technology enhancements.

FAI hosts, coordinates, and participates in roundtables at the  Interagency Acquisition Career Management Committee meetings to  foster discussion among agencies and share information on their  training or educational challenges and needs. According to FAI  officials, these meetings provide opportunities to share leading  practices, identify challenges, and discuss potential initiatives. Recent  discussion topics included plans to hold a competency-based  certification workshop, roll out of additional FAITAS workforce  management tools, and results of FAI’s bi-annual Acquisition  Workforce Competency Survey.

In July 2012, FAI established the Federal Acquisition Council on  Training to help share information on agencies’ training efforts and  standardize acquisition training throughout the government. According  to FAI officials, the council’s goal is to help ensure no acquisition  workforce training seat, whether offered by FAI or another agency,  goes unfilled. Agencies are asked to open up unfilled training seats to  other agencies whenever available, and to use FAITAS as the official  course registration system to communicate available training seats  and enroll participants for courses to reduce overhead and  administrative costs for agencies.

Periodically analyzing acquisition career fields and evaluating  the effectiveness of training and career development programs  for acquisition personnel14  According to officials, FAI has efforts underway to standardize the  end-of-course participant evaluations administered by the vendors  who provide FAI-sponsored courses. According to FAI officials,  course participants will not receive their course completion  certifications until they complete the course evaluation. Currently FAI  does not analyze course evaluations administered by the vendors;  however, officials commented that the use of standardized  evaluations could enable them to work with vendors to improve the  consistency of the information provided in training courses. A pilot is  underway using standard evaluations for FAI courses taught by  vendors being provided to one agency, and FAI plans to extend the  pilot to other agencies early in 2013.

section_title: Agencies Generally Use External Sources for Acquisition Training and Face Similar Workforce Training Challenges
paragraphs: Most agencies approach acquisition workforce training through classroom  courses taught by external sources—vendors, FAI, DAU, or other  agencies. While all agencies have meeting spaces for training, three  operate permanent centers with dedicated resources that train the  agency’s acquisition workforce. The agencies’ current training focus is to  provide courses through which their acquisition workforces may attain or  maintain their FAC certifications. Agencies reported facing several  challenges in providing acquisition-related training. The areas reported as  being the most challenging are related to staffing and budgetary  resources. Some agencies also reported challenges with the identification  of their acquisition workforce, which is a fundamental step needed for  managing the workforce and its training. Agencies also reported that  additional assistance from OFPP and FAI would help their acquisition  workforce training efforts. In addition, the agencies reported that their  acquisition workforces are challenged in finding time in their workload to  attend training. For more details on the approaches, budgetary and  staffing resources, and other challenges faced by the agencies, see  appendix I for a summary of the 23 agencies’ responses to the  questionnaire we administered.

41 U.S.C. § 1201(a)(4) and 41 U.S.C. § 1201(a)(7).
subsections: section_title: Agencies Generally Rely on External Training Sources
paragraphs: Agencies provide acquisition workforce training predominantly through  external sources—government entities or vendors. In fact, 17 agencies  reported that the majority of their acquisition workforce training comes  from having their workforce attend training provided by external sources  rather than from their agency. The other government entities may provide  training to an agency without seeking additional reimbursement, as is the  case with FAI, DAU, and some federal agencies; however, others, such  as the VA, charge a fee to attend their training to recoup their costs.15 Of  the remaining 6 agencies, 5 reported that they hold agency-sponsored  courses to provide the majority, if not all, of their acquisition workforce  training, and one agency did not report. Some agencies use contractor  personnel to instruct a portion of their agency-sponsored training. Figure  1 illustrates the sources of training that agencies reported using to  provide training to their acquisition workforce in fiscal year 2011.

Although FAI does not seek reimbursement for individual classes, civilian agencies “pay”  for the courses since they provide the funding for FAI’s operations through the mandatory  fee paid equal to five percent of the dollar amount of acquisitions the agency makes  through the Federal Supply Schedule and the Governmentwide Acquisition Contracts.

The Department of Transportation provided data that does not total 100 percent due to its averaging  of the responses from its bureaus.

The Department of Justice data is an average of the responses submitted by 4 procurement offices— headquarters and 3 bureaus.

A leading training investment practice involves agencies taking steps to  identify the appropriate level of investment to provide for training and  development efforts and to prioritize funding so that the most important  training needs are addressed first.16 The four agencies we selected  illustrate that agencies use different approaches to manage and provide  acquisition workforce training, due to such factors as the size of their  workforce, the need for certification training, and the resources dedicated  to training. Three agencies—DHS, Treasury, and VA—chose to have  permanent training facilities dedicated to providing courses to the  acquisition workforce. These three agencies differed, however, in the  percentage of courses they provided to meet their acquisition workforce  needs. DHS manages its entire acquisition workforce training at the  department level and recently expanded the number of courses provided  by the agency. However, part of DHS’s strategy is to make use of DAU  and FAI courses that address its needs, which can be obtained at no  additional charge. VA also manages all of its training at the department  level and in 2008 established a training academy, the VA Acquisition  Academy, to provide all acquisition-related courses to agency personnel  who need it.  Alternately, Treasury built its training program on an existing  training institute within the Internal Revenue Service, which employs the  majority of Treasury’s acquisition workforce, to provide agency courses  available to all of its bureaus and to other agencies. Of the three  dedicated training facilities, VA is the only one to provide agency- sponsored training for 100 percent of its acquisition training courses. VA,  DHS, and Treasury all reported using contractors to instruct the agency- provided training at their respective facilities. Education does not have a  dedicated facility and reported that it provides only about 10 percent of its  acquisition workforce training via agency-sponsored courses, obtaining  about 90 percent of needed training through external sources.

GAO-12-878. training needs. In their responses to our questionnaire, the agencies did  not explain how effectiveness is determined. As we discuss later in this  report, based on a subsequent data call we found that the agencies have  limited insight into the effectiveness of their acquisition workforce training  courses. Figure 2 illustrates the factors and number of agencies that  reported them as influencing their training source selections.

Agencies’ current focus is to provide courses that allow their acquisition  workforces to attain or maintain their FAC certification. In response to our  questionnaire, most agencies—17 of 23—reported that they are able to  find sufficient courses to do this. In their AHCP submission for fiscal year  2011, most agencies (15) reported certification rates of more than 75  percent for their contracting staff, but 3 agencies reported over 90 percent  of their contracting staff were certified. The majority of the 23 agencies  reported that more than 90 percent of their CORs and P/PM staff had  obtained their basic FAC certifications—with many reaching 100 percent.  Agency officials explained that the agency’s certification rates fluctuate  due to changes in the workforce—including experienced staff leaving the  agency and less experienced staff being in the process of obtaining  certifications. Table 3 shows the certification rates reported by agencies  for fiscal year 2011.

Officials from three of the four selected agencies—DHS, Treasury, and  VA—explained that they prioritize their training funding based on the need  for acquisition staff to be certified. Education, where the majority of the  acquisition workforce has attained their certifications already, reported it  is focusing on courses that meet staff needs to maintain certifications or  develop expertise.

section_title: Agencies Face Challenges with Having Adequate Training Resources, Identifying the Acquisition Workforce, and Staff Having Time to Attend Training
paragraphs: The top challenges reported by agencies in obtaining training for their  acquisition workforces were having sufficient resources—both staffing to  manage the training program and budgetary resources—to provide  training. Specifically, 20 agencies reported that obtaining adequate  funding is a challenge, and 19 reported that obtaining sufficient staff to  manage the acquisition workforce training is a challenge. As for  determining the appropriate level of investment, we have previously  reported that when assessing opportunities for training, the agency  should consider the competing demands confronting the agency, the  limited resources available, and how these demands can best be met with  available resources. Figure 3 provides a summary of the number of  agencies that reported challenges involved in obtaining sufficient  acquisition workforce training.

Although almost all agencies reported that obtaining an adequate level of  funding is challenging, less than half—10 of 23—reported that their  current acquisition training budgets are insufficient to meet their training  needs. The acquisition workforce training budgets for fiscal year 2012  reported by 22 of the agencies ranged from $0 to $39,598,000 for an  agency with its own acquisition workforce training facility. These budgets  correspond to acquisition workforces that ranged in size from 186 to  11,867 employees for fiscal year 2010 that handled anywhere from about  900 to 1.4 million contracting actions with obligations ranging from $132  million to $25 billion in fiscal year 2011.

Almost all the agencies—20 of 23—reported that they would like  additional FAI assistance to help their acquisition workforce members  attain federal certification by providing more course offerings or additional  topics. In addition, to accommodate travel restrictions and budget  constraints, some agencies would like FAI to provide more virtual courses  for personnel that are geographically dispersed. FAI has some efforts  underway that may somewhat address these issues, but their results are  still uncertain as they are just beginning. FAI officials noted that while  agencies want more offerings, FAI still has under-attended classes that  have to be canceled. FAI officials also commented that developing on-line  courses can also be very expensive and the cost to maintain currency of  information is an often times overlooked cost.

GAO-11-892. personnel are dispersed throughout many organizations, come from a  variety of career fields, and are often involved in acquisitions as a  secondary and not a primary duty. The agencies’ Acquisition Career  Managers told us that they face similar challenges in identifying CORs  and other personnel with acquisition-related responsibilities. Three of the  four agencies selected for further insight—DHS, Education, and VA— acknowledged that they continue to be challenged by identifying some  members of their acquisition workforces.

A majority of the agencies—15 of 23—reported that they would like  additional assistance from OFPP to improve their acquisition workforce  training by either of two ways. Thirteen agencies supported the idea of  OFPP creating separate job series for additional acquisition workforce  categories, such as P/PMs. Twelve agencies supported the idea of OFPP  maintaining a government-wide database to identify and track members.  According to FAI and OFPP officials, they are sympathetic to agency  concerns about identifying the total acquisition workforce. As for creating  separate job series to more easily identify members of the acquisition  workforce, OFPP notes that any changes would need to be made by the  Office of Personnel Management and that some acquisition positions,  such as CORs, may not lend themselves to becoming a unique job series  because the work is performed as a collateral duty. As for maintaining a  government-wide database to identify and track acquisition workforce  members, FAI officials noted that agencies are encouraged to use the  FAITAS registration system to track their workforce’s individual training  records and certifications. However, registration into the system is  currently voluntary.

Agencies also reported that their acquisition workforces are challenged in  finding time in their workload to attend training. Most agencies—14 of  23—considered the ability to find time as extremely or very challenging.  Figure 4 provides a summary of the number of agencies that reported  challenges to their staff accessing and attending acquisition-related  training.

section_title: Lack of Comparable Cost Data and Limited Insights On Benefits of Training Hinder Efforts to Maximize Resources Government-wide
paragraphs: Agencies collect some training cost data and limited information about the  benefits of their acquisition workforce training. Based on responses to our  questionnaire, a supplemental data request, and discussions with  agencies’ and FAI officials, we found that many agencies do not collect  data on the costs of training provided to their acquisition workforce that  can be used to inform agency and government-wide training resource  investment decisions. In addition, some agencies do not have metrics to  assess the effectiveness of their training.
subsections: section_title: Government-wide Training Cost Data Reported Is Not Comparable
paragraphs: GAO-12-878 and GAO-04-546G. data eventually provided by the agencies included different cost  components, which did not lend themselves to comparative analysis. For  example, some agencies provided data on the cost per seat of specific  courses, while others provided the total costs for delivery of each course.  Some agencies provided the costs associated with the development of an  in-house course, while others included development costs with delivery  costs as a total cost of obtaining the course from a vendor. Although the  data do not allow direct comparisons among agencies, the data show a  range of costs for similar courses. For example, five agencies provided a  per seat cost for a COR Refresher course, which ranged from $97 to  $363 per seat. In addition, five agencies provided a per seat cost for a  Cost Analysis and Negotiation Techniques course, which ranged from  $282 to $925 per seat. However, a number of factors can produce the  variation in the agency-reported costs and these factors, along with the  agencies’ data collection methods, can limit the ability to make  government-wide decisions on acquisition workforce training investments.   According to agency officials, the costs of individual acquisition workforce  training courses can vary greatly among agencies due to a variety of  factors, including: the number of times per year the course is provided,  actual attendance numbers, location of training, and whether courses are  tailored with agency-specific information.

FAI officials reported that they also received limited responses to their  request for training cost data in the agencies’ latest AHCPs, which made  it impossible for them to complete their planned comparative analyses.  The instructions for the FAI data request did not include definitions of the  cost components to be reported, which are important to help ensure  consistent reporting. Due to the limited response on cost data, FAI  initiated a subsequent data call. As of November 2012, FAI had received  responses from 20 of the 23 agencies, but again was unable to perform a  comparative analysis of the cost data due to various limitations in the data  it received. FAI officials noted that collecting cost data from agencies is  an evolving process and that having comparable training cost data is  important to help FAI in its efforts to maximize the use of acquisition  workforce training dollars government-wide. FAI plans to request cost  data annually with the AHCPs; however, the guidance for completing the  next submission does not include definitions of the cost data to be  reported, which may again yield data that cannot be compared across  agencies. Improved reporting of cost data could assist FAI as it moves  forward with its plans to award government-wide contracts for acquisition  workforce training in 2014 by allowing FAI to obtain insight into the  agencies’ training costs when obtaining training from a variety of external  sources.

section_title: Agencies Gather Limited Information on Benefits of Acquisition Workforce Training Investment
paragraphs: About half of the agencies—12 of 23—did not have insight into whether  their acquisition workforce training investment is improving individual  skills or agency performance.  In particular, in response to our  questionnaire and subsequent data request 7 of 23 agencies reported  having no metrics to monitor or assess the effectiveness of their  acquisition workforce training efforts, a measure of whether the training  investment is improving individual skills or agency performance.  Five  agencies did not provide information to support use of metrics to measure  the training benefits, either improving individual skills or agency  performance, of acquisition training in response to our supplemental data  request.

We have previously reported that training programs can be assessed by  measuring (1) participant reaction to the training program, (2) changes in  employee behavior or performance, and (3) the impact of the training on  program or organizational results, which may include a return on  investment assessment that compares training costs to derived benefits.  Of the 11 agencies that provided information to support their use of  metrics, 3 reported using end-of-course evaluations to measure  participants’ reaction, one reported using end-of-course tests to measure  changes in employees’ knowledge, and one reported using post-course  surveys to supervisors or participants to measure if what was learned  affected the participants’ behavior. The other 6 agencies reported  measures aimed at determining the impact of training on the agency’s  mission. Furthermore, DHS officials said that they plan to begin using  post-course surveys of participants and supervisors in fiscal year 2013,  and they are pursuing the development of additional measures to  evaluate the impact of training on the agency. VA is also pursuing the  development of additional metrics, such as proxy measures for return on  investment, to evaluate the impact of training on the agency. Participant  evaluations offer limited insight into improvements in individual and  agency performance; however, we recognize that higher levels of  evaluation (such as evaluating the impact on organizational performance  or comparing training costs to derived benefits) can be challenging to  conduct because of the difficulty and costs associated with data collection  and the complexity in directly linking training and development programs  to improved individual and organizational performance.

Officials at some of the selected agencies told us that when the course is  provided by an external source, such as FAI or DAU, they rely on the  external source to provide end-of-course evaluations to the participants.  As we noted earlier, 17 of 23 agencies obtain the majority of their  acquisition training from external sources. In particular, some agencies  noted that they do not collect or assess the end-of-course evaluations,  relying solely on the external source to use the evaluations as they  believe appropriate. FAI officials noted that if the agencies do not review  the evaluations themselves, they are missing the opportunity to ensure  that the courses are effective in training their acquisition personnel. We  have previously reported that it is increasingly important for agencies to  be able to evaluate their training programs and demonstrate how these  efforts help develop employees and improve the agencies’ performance  because it can aid decision makers in managing scarce resources and  provide credible information on how training has affected individual and  organizational performance.22 Although FAI has an initiative underway to  standardize the evaluations provided for its courses, its impact may be  limited if agencies do not obtain the evaluation results and use them to  evaluate the effectiveness of the training.

section_title: Conclusions
paragraphs: GAO-12-878 and GAO-04-546G. to ensure that every training seat is filled—to assist agencies in meeting  OFPP requirements and leveraging federal government training  resources, these efforts face significant obstacles. Agencies, FAI, and  OFPP lack data on the acquisition workforce itself and the benefits of  training, as well as cost data that can be used to make comparisons.  These limitations hinder government-wide efforts to share information  during these times of constrained budgets. OFPP, FAI, and the agencies  need basic information on how much agencies are spending to train the  acquisition workforce. Providing definitions and guidance about the  elements of costs agencies should include in their annual acquisition  human capital plans will help OFPP and FAI in their efforts to collect  comparable data across agencies. FAI can then analyze and share  information to help agencies make choices regarding how best to  dedicate resources to effectively train the acquisition workforce.  Additionally, comparable costs data will be helpful to inform FAI efforts to  award government-wide contracts for standard acquisition workforce  certification training.

Currently, the main focus of monitoring and tracking acquisition workforce  training efforts in agencies is on completion of training to attain and  maintain the FAC certifications. Although the higher levels of evaluation  (such as evaluating the impact on organizational performance or  comparing training costs to derived benefits) can be challenging to  conduct due to costs and complexity, agencies should, at a minimum,  evaluate participant reaction to the training program through end of  course evaluations. However, this is a basic measure that some agencies  are lacking because the evaluations currently being administered go to  the provider of the class and not the agency paying for the training.  Therefore, these agencies have little insight into how the workforce  perceives the training they received. We note that FAI has initiatives  underway to standardize the course evaluations for the courses it  provides and to make completion of the course evaluations mandatory to  receive course credit. At present, this effort only applies to FAI-sponsored  courses and will not include agencies that use other vendors, and does  not guarantee that agencies will analyze the results; it is important for all  agencies to collect and analyze this basic information.

section_title: Recommendations for Executive Action
paragraphs: To help ensure that agencies collect and report comparable cost data and  perform a minimal assessment of the benefits of their acquisition training  investments to aid in the coordination and evaluation of the use of  resources government-wide, we recommend the Director of the Office of  Management and Budget direct the Administrator of the Office of Federal  Procurement Policy, in consultation with the Director of the Federal  Acquisition Institute, to take the following two actions: provide further guidance, including definitions, on the types of costs  that agencies should include in their Acquisition Human Capital Plan  submission to help determine total training investment; and  require all agencies, at a minimum, to collect and analyze participant  evaluations of all acquisition workforce training as a first step to help  assess the effectiveness of their training investment.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to OFPP, FAI, and the selected  agencies—DHS, Education, Treasury, and VA. OFPP and FAI concurred  with our recommendations in oral comments and e-mail responses.  Education and Treasury responded via e-mail with no comments. DHS  and VA provided technical comments, which we incorporated as  appropriate.

In oral comments, OFPP and FAI agreed with our recommendations and  noted that they have begun drafting guidance to federal agencies.  Consistent with our report, OFPP and FAI emphasized the importance of  acquisition workforce management tools that improve the government’s  ability to leverage acquisition resources especially during this time of  budgetary constraints. These officials highlighted FAITAS, which can  serve as a workforce management tool and training application system.  Currently, use of this system is voluntary and some agencies use the  system to enroll participants in FAI courses, communicate available  training seats within a specific agency’s training course, and track their  workforce’s individual training records and certifications. Recognizing the  potential benefits of this system in helping to coordinate and evaluate the  use of training resources government-wide, OFPP stated that it is  considering making FAITAS reporting mandatory for civilian agencies.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  Administrator of OFPP, the Director of FAI, and the Secretaries of DHS,  Education, Treasury, and VA. In addition, the report will be available at no  charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or MartinB@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix III.

section_title: Appendix I: Summary of Agencies’ Response to GAO Questionnaire on Civilian Acquisition Workforce Training
subsections: section_title: In July 2012, GAO administered a government-wide survey of the 23 civilian CFO Act agencies designed to gather information regarding the training approach and methods, staffing and budgetary resources, and costs associated with providing acquisition training—including the operation of facilities dedicated to training the acquisition workforce. We received responses from 100 percent of the agencies, although not every agency responded to each question. A summary of the number of agencies’ who responded to each question is provided below. We removed the open-ended narrative responses, but included the ranges of open-ended data. Details of our survey methodology are provided in appendix II.
paragraphs: The objectives of this review were to evaluate the training approaches of  federal agencies, other than the Department of Defense, for the  acquisition workforce. Specifically, this report addresses the (1) role of the  Office of Federal Procurement Policy (OFPP) and the Federal Acquisition  Institute (FAI) in promoting Federal Acquisition Certification (FAC)  standards and assisting agencies in meeting acquisition workforce  training requirements; (2) approaches agencies use to provide training to  their acquisition workforces; and (3) the extent to which agencies track  information on the costs and benefits of their acquisition training.

To identify the roles of OFPP and FAI in promoting FAC standards, we  analyzed legislation on their authority, the OFPP acquisition workforce  strategic plan, and relevant guidance provided by them to agencies. We  interviewed officials at OFPP and FAI about efforts to (1) provide  oversight and assist agencies in meeting training requirements; (2)  leverage government-wide resources; and (3) share leading practices.  Since FAI’s initiatives are in various stages of implementation, we did not  evaluate their effectiveness; however, in several instances, we note our  initial observations of the efforts.

31 U.S.C. § 901. used a standard set of questions to obtain data about their training  approaches, including the sources for course offerings (in particular  whether courses were provided by the agency, a commercial sector  vendor, FAI, the Defense Acquisition University, or another agency),  facilities, budgetary and staffing resources, methods of training, and  challenges faced in providing acquisition training. Because this was not a  sample survey, it has no sampling errors. However, the practical  difficulties of conducting any survey may introduce errors, commonly  referred to as nonsampling errors. For example, difficulties in interpreting  a particular question, sources of information available to respondents, or  entering data into a database or analyzing them can introduce unwanted  variability into the survey results. We took steps in developing the  questionnaire, collecting the data, and analyzing them to minimize such  nonsampling error.  After we drafted the questionnaire, we asked for  comments from federal officials knowledgeable about acquisition  workforce and training issues and from independent GAO survey  professionals. We conducted pretests to check that (1) the questions  were clear and unambiguous, (2) terminology was used correctly, (3) the  questionnaire did not place an undue burden on agency officials, (4) the  information could feasibly be obtained, and (5) the survey was  comprehensive and unbiased. We chose the two pretest sites to include  perspectives from agencies with robust acquisition workforce training  programs and training facilities. We made changes to the content and  format of the questionnaire after both reviews and after each of the  pretests, based on the feedback we received.  We sent the questionnaire  by e-mail in an attached Microsoft Word form that respondents could  return electronically after marking checkboxes or entering responses into  open answer boxes on July 5, 2012. We sent the questionnaire jointly to  the Chief Acquisition Officer, Acquisition Career Manager, and Chief  Learning Officer of each agency, asking for a consolidated agency  response. After two weeks, we sent a reminder to everyone who had not  responded. All questionnaires were returned by August 10, 2012. We  received questionnaire responses from 100 percent of the agencies,  although not all agencies answered every question. The Department of  Justice provided four responses—one from its headquarters procurement  office and three from bureau procurement offices. We consolidated the  information from the four surveys to ensure the tabulations eliminated  duplicative counting of the agency and to provide averaged data, as  appropriate. In questions regarding the agency’s opinions regarding  challenges faced, we deferred to the response of the headquarters  procurement office.

To confirm our understanding of the variety of issues addressed and  obtain additional insights on the agencies’ responses to our  questionnaire, we discussed our preliminary results with representatives  from the 23 agencies at meetings of the Chief Acquisition Officers  Council, Interagency Acquisition Career Management Committee, and the  Chief Learning Officers Council. Table 4 provides a summary of the  functions of these councils as related to government-wide efforts for  training the acquisition workforce.

We also reviewed the agencies’ annual Acquisition Human Capital Plans  (AHCP) due March 31, 2012, to collect information on certification rates,  and corroborate information agencies provided in response to the  questionnaire on acquisition workforce size and training approaches.

To determine the information agencies track on the costs and benefits of  the training provided, we supplemented the questionnaire responses by  soliciting additional cost and metrics data from the agencies using a data  collection instrument. We developed our data collection instrument based  on cost information provided from agencies in response to our  questionnaire. We sent agency-tailored data collection instruments by e- mail in an attached Microsoft Excel spreadsheet that respondents could  return electronically after updating or providing new data on October 15,  2012. We pre-populated each agency’s spreadsheet with any data  previously provided by the agency, and sent the data collection  instrument to the Acquisition Career Manager and the official who  provided the response to the questionnaire. After two weeks, we sent a  reminder to everyone who had not responded. The data collection  instruments were returned by November 28, 2012. We received data  collection instruments from 91 percent of the agencies.

To gather illustrative examples and more detailed explanations regarding  training approaches and data tracked related to costs and benefits, we  selected four agencies—the Departments of Education (Education),  Homeland Security (DHS), the Treasury (Treasury), and Veterans Affairs  (VA)—for further review. DHS, Treasury, and VA are agencies that  operate dedicated acquisition workforce training facilities—permanent  centers with dedicated resources that provide training specifically for the  agency’s acquisition workforce. Education, which is one of the smallest  agencies, has no dedicated facility and has reported issues with access  to acquisition workforce training courses. We interviewed agency officials  to determine the extent to which they were aware of and using the leading  training investment practices that the Office of Personnel Management  and experts agreed should be implemented by agencies to support  effective training investment decisions.3 Table 5 provides a summary of  these leading practices.

Of the eight leading practices GAO identified, we focused on the four  practices dealing primarily with determining costs and effectiveness of  training: (practice 1) identifying the appropriate level of investment to  provide for training and development efforts and prioritize funding so that  the most important training needs are addressed first; (practice 4) having  criteria for determining whether to design training and development  programs in-house or obtain these services from a contractor or other  external source; (practice 6) tracking the cost and delivery of its training  and development programs agency-wide; and (practice 7) evaluating the  benefits achieved through training and development programs, including  improvements in individual and agency performance.

We conducted this performance audit from February 2012 to March 2013,  in accordance with generally accepted government auditing standards.   Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives.  We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

section_title: Appendix III: GAO Contact and Staff Acknowledgements
subsections: section_title: GAO Contact

section_title: Staff Acknowledgements
paragraphs: In addition to the contact name above, Penny Berrier (Assistant Director);  R. Eli DeVan; Mya Dinh; Jean McSween; Aku Pappoe; Kenneth Patton;  Carol D. Petersen; Sylvia Schatz; and Suzanne Sterling made key  contributions to this report.


Question-summary annotations: questions: question: What are the responsibilities of the OFPP in the acquisition workforce?
answer: The Office of Federal Procurement Policy (OFPP) sets standards and policies for the federal acquisition workforce, and has established certification requirements, including minimal training, for the three main acquisition roles--contracting staff, Contracting Officer's Representatives, and Program/Project Managers--to promote the development of government-wide core acquisition competencies and facilitate mobility across agencies.
child_questions: question: How do DOD certification standards compare to those of OFPP?
answer: DOD follows separate certification standards.

question: What are the responsibilities of the FAI?
answer: The Federal Acquisition Institute (FAI), which is responsible for fostering and promoting the training and development of the acquisition workforce, works closely with OFPP and has initiatives underway to improve the collection and management of training information, including cost data and course evaluations; streamline communication of acquisition training guidance; and coordinate efforts to leverage acquisition workforce training resources throughout the government.

questions: question: Why do most agencies provide the majority of their acquisition training using external sources?
answer: To support efforts for the acquisition workforce to attain and maintain federal certification requirements, most agencies (17 of 23) provide the majority of their acquisition training using external sources--vendors, FAI, the Defense Acquisition University, or other agencies.

question: What does the Department of Homeland Security operate a permanent center for?
answer: The Departments of Homeland Security (DHS), the Treasury, and Veterans Affairs (VA) operate their own permanent centers with dedicated resources that train the agency's acquisition workforce.

question: How is Education different from most federal agencies?
answer: Education reported using a different approach to providing acquisition training--it gets most of its training from other government entities.

question: What are the top challenges reported by agencies in obtaining training for their acquisition workforce?
answer: Federal agencies face similar challenges in providing training to their acquisition workforce. The top challenges reported by agencies in obtaining training for their acquisition workforce involved having sufficient resources. Twenty of 23 agencies identified obtaining adequate funding and 19 of 23 identified obtaining sufficient staff to manage training as challenging.
child_questions: question: What did almost half the agencies report regarding identifying the acquisition workforce?
answer: In addition, almost half of the agencies reported that the fundamental step of identifying the acquisition workforce is a challenge especially when members of the workforce are involved in acquisitions as a secondary and not primary duty.

questions: question: Why do agencies have limited information on the benefits of their acquisition workforce training investments?
answer: The training cost data that agencies collect is not comparable and agencies have limited information on the benefits of their acquisition workforce training investments. Although almost all agencies provided some cost data in response to GAO's questionnaire and subsequent data call, the agencies' cost data did not allow for government-wide assessment of their training investment.

question: What has FAI’s efforts to collect training cost data met with?
answer: To date, FAI's efforts to collect training cost data has also met with limited success.
child_questions: question: What were some of the cost components studied while researching training cost data in 2012 by GAO and FAI?
answer: Cost data collected in 2012 by GAO and FAI included different cost components-- such as facilities, travel, and instructors--which do not allow for government-wide analysis.

question: Why are comparable training cost data important?
answer: Having comparable training cost data are important to inform FAI efforts to establish government-wide contracts for training.

question: Why does a lack of metrics make studying the benefits of acquisition workforce training difficult?
answer: As for determining benefits of training, 7 of 23 federal agencies reported having no metrics, not even basic endof- course evaluations. Without basic data, agencies do not have insight into the benefits of their acquisition workforce training efforts.

questions: question: Why is the acquisition workforce important?
answer: The acquisition workforce manages and oversees billions of dollars in acquisition programs and contracts to help federal agencies get what they need, at the right time, and at a reasonable price; therefore, it is important that agencies provide adequate training to this workforce.

question: What was identified in the GAO review?
answer: In this review, GAO identified (1) the role of OFPP and the FAI in assisting agencies in meeting certification requirements; (2) agencies' approaches to providing training; and (3) the extent to which agencies collect information on the costs and benefits of their acquisition training.

question: How did the GAO determine OFPP and FAI roles?
answer: To determine OFPP and the FAI roles, GAO analyzed relevant legislation. GAO obtained information from 23 federal agencies on their training approaches through a questionnaire, and selected 4 agencies--the Departments of Education and the Treasury, DHS, and VA--to provide illustrative examples.
child_questions: question: How did GAO utilize its questionnaire and data call?
answer: GAO used its questionnaire and a subsequent data call to obtain information on how agencies collect information on the costs and benefits of their training.