Acquisition Workforce: Federal Agencies Obtain Training to Meet Requirements, but Have Limited Insight into Costs and Benefits of Training Investment
Released: 2013-04-16T00:00:00

Letter

Background

The Office of Personnel Management, as the federal government’s human capital manager, provides leadership and guidance on establishing and operating efficient federal training and development programs throughout the government. It provides advice and assistance to agencies on training and development programs so that those programs support strategic human capital investment.

GAO, Federal Training Investments: Office of Personnel Management and Agencies Can Do More to Ensure Cost-Effective Decisions, GAO-12-878 (Washington, D.C.: Sept. 17, 2012).

OFPP has specific responsibilities related to acquisition workforce training to help ensure that the skills needed to handle the complexities of acquisition programs and contracts are maintained. The Office of Federal Procurement Policy Act requires executive agencies and the OFPP Administrator to establish management policies and procedures for the effective management of the acquisition workforce—including education, training, and career development.4 In 2005, OFPP defined the civilian acquisition workforce to include, at a minimum, professionals serving in the contracting series (GS-1102), Contracting Officers, Purchasing series (GS-1105), Program/Project Managers (P/PM), and Contracting Officer’s Representatives (COR) or the equivalent, and additional positions identified by the agency. Participation in the acquisition workforce may be on a full-time, part-time, or occasional basis.

41 U.S.C. § 1703. 41 U.S.C. § 1703(c)(2).

The Federal Supply Schedules program consists of contracts awarded by the General Services Administration or the Department of Veterans Affairs for similar or comparable goods or services, established with more than one supplier, at varying prices. Federal Acquisition Regulation (FAR) § 8.401 and § 8.402. The Schedules offer a large group of commercial products and services ranging from office supplies to information technology services. Governmentwide Acquisition Contracts (GWAC) are considered multi-agency contracts but, unlike other multi-agency contracts, are not subject to the same requirements and limitations, such as documentation that the contract is in the best interest of the government as set forth under the Economy Act. GWACs are contracts for information technology established by one agency for government-wide use that are operated—(1) by an executive agent designated by the Office of Management and Budget pursuant to 40 U.S.C. § 11302(e); or (2) under a delegation of procurement authority issued by the General Services Administration (GSA). FAR § 2.101. The Clinger-Cohen Act of 1996 authorized GWACs to be used to buy information technology goods and services. Pub. L. No. 101-510, § 1202 (1996), (codified at 40 U.S.C. § 11314(a)(2)). All agencies can order goods and services directly through the Schedules contracts and GWACs.

Board of Directors, works closely with OFPP as well as the DAU to collaborate on many issues involved with training and developing the federal acquisition workforce. The Defense Acquisition Workforce Improvement Act required the Secretary of Defense to establish and maintain DAU in order to provide training for the DOD acquisition workforce.8 FAI differs notably from DAU, however, in that it shares much of the responsibility for training the civilian acquisition workforce with the agencies and has fewer budgetary and staffing resources. For example, FAI currently has 12 staff, while DAU has over 700 faculty and staff.

The Administrator of OFPP is also required to ensure that agencies collect and maintain standardized information on the acquisition workforce. Agencies, in consultation with OFPP, can structure their management and oversight of training for their acquisition workforce in such a manner as best supports the agency. Key agency positions involved in managing the acquisition workforce within the agencies are described in table 1.

Acquisition training-related responsibilities Responsible for developing and maintaining an agency acquisition career management program to ensure an adequate professional acquisition workforce. Participates in the strategic planning and performance evaluation process to develop strategies and specific plans for training to rectify deficiencies in the skills and abilities of the acquisition workforce. Responsible, in consultation with the Acquisition Career Manager, for reporting acquisition workforce data to the Human Capital Office for the agency’s human capital plan. Responsible for providing management direction of an agency’s procurement system. In some agencies, CAOs oversee or assume this role to help ensure implementation of agency acquisition workforce policy. Responsible for identifying training and development needs of the acquisition workforce. Leads an agency’s acquisition career management program and ensures that its acquisition workforce meets the skills and capabilities required in OMB, OFPP, and agency policies.

10 U.S.C. § 1746. provided by the agency, which may be taught using in-house staff or contractor employees; (2) government entity training provided by another federal agency, FAI, or the DAU; and (3) vendor training provided by a commercial sector company, which is generally available to multiple agencies at the same time. For the purposes of this report we refer to government entity and vendor training as external sources.

OFPP and FAI Provide Policy, Guidance, and Assistance to Help Improve Government-wide Training Efforts

OFPP sets standards and policies for the training and development of the federal acquisition workforce. Its efforts include strengthening workforce planning requirements and setting standards for core acquisition training by establishing certification requirements. FAI conducts activities that support and assist civilian agencies in training and development of their acquisition workforces. FAI efforts include: improving the collection and management of training information, including cost data and course evaluations; streamlining the communication of acquisition training guidance; and coordinating efforts to maximize acquisition workforce training investments government-wide.

OFPP Promotes Government-wide Practices for Acquisition Workforce Training and Development

The Federal Acquisition Certification requirements were issued in 2005 for contracting professionals and 2007 for Contracting Officer’s Representatives and Program/Project Managers. provided in the agencies’ plans to develop a government-wide snapshot of the acquisition workforce and to identify practices for sharing throughout the government. Table 2 provides an overview of OFPP’s legislative responsibilities related to acquisition workforce training.

41 U.S.C. § 1201(a)(1). workforce. FAI reported that while its spending plan for fiscal year 2012 could provide $3 million for courses, training requests submitted by the agencies totaled in excess of $18 million.

FAI is working with a number of vendors who provide acquisition workforce training to establish government contracts that would provide uniform costs for standardized training courses needed for the FAC programs. FAI plans to have contracts in place in 2013 that would be available for agencies to use to provide fiscal year 2014 training.

Assisting federal agencies with their acquisition human capital FAI offers guidance and direction on human capital planning through its instructions for agencies’ annual AHCP submissions. Beginning in 2010, OFPP has required agencies to submit an annual AHCP each March that provides the agency’s strategies and goals for increasing both the capacity and capability of the acquisition workforce. To assist the agencies in preparing each submission, FAI provides them with a report template that includes specific topics to be included. We observed that the fiscal year 2012 AHCP submissions provided information on the agencies’ goals for strengthening the acquisition workforce, including certification goals, and planned training initiatives for the current and future fiscal years.

41 U.S.C. § 1201(a)(11). 41 U.S.C. § 1201(a)(3).

FAI recently upgraded the government-wide acquisition career information management system it maintains for all agencies. In 2011, FAI replaced the Acquisition Career Management Information System (ACMIS) with the FAI Training Application System (FAITAS), which allows registration for courses, tracking of individuals’ training records, and other information management tools. For example, agencies can now manage the certification approval process, track an individual’s continuous learning progress, and search for courses offered by FAI and other agencies that use FAITAS to enroll participants in their courses. FAI reports that five agencies currently use the FAITAS registration function to alert other agencies to unfilled seats within a specific agency’s training course. Additional features of the system include a course scheduling module; a business intelligence tool for agencies to identify training locations and course availability; and a communication tool for broader outreach to the acquisition workforce. For example, an OFPP official explained that FAITAS can be used to send e-mail updates on OFPP policy, guidance, or other OFPP or FAI initiatives to all individuals registered in the system.

FAI conducts periodic surveys of agencies to collect data about the acquisition workforce for OFPP. For example, FAI conducts the bi- annual Acquisition Workforce Competency Survey to collect information on agency acquisition workforces’ skills and abilities and to identify competency gaps within and across agencies. FAI officials explained that the survey results provide a government-wide view of which courses are in demand that informs their efforts to achieve economies of scale when providing training.

FAI reports an annual count of the acquisition workforce as it has done for more than the past decade; however, its count has been limited to professionals serving in four job series—General Business and Industry (GS-1101), Contracting (GS-1102), Purchasing (GS- 1105), and Procurement Clerical and Assistance (GS-1106). FAI began gathering data on CORs and P/PMs as part of the acquisition workforce count in fiscal year 2007. In its recent report on the acquisition workforce count, FAI reported that 33,271 personnel were employed in the acquisition workforce’s four main job series in fiscal year 2010, and an additional 47,959 and 4,186 personnel were identified as CORs and P/PMs, respectively, by the agencies in their AHCPs—which represents about 85,000 individuals in the civilian acquisition workforce for fiscal year 2010. FAI does not directly collect data for the workforce count, and has acknowledged that agencies have some problems identifying CORs and P/PMs but the data have been improving.

Facilitating interagency intern and training programs, as According to officials, FAI also conducts outreach and leverages training courses government-wide. In 2012, FAI met with officials from each agency to discuss training efforts and locate training spaces that could be used by FAI and others when supplying courses through vendors. FAI officials reported that these agency visits provided insights regarding how the agencies’ training programs varied in terms of their organization and resources and identified duplicative training efforts.

In addition, FAI provides opportunities to share information across the acquisition workforce community. FAI began providing webinars on current acquisition issues in fiscal year 2012, and launched a newsletter for the acquisition workforce community in December 2012. According to officials, the newsletter will provide more in-depth information on policy changes, human capital initiatives, and tools and technology enhancements.

FAI hosts, coordinates, and participates in roundtables at the Interagency Acquisition Career Management Committee meetings to foster discussion among agencies and share information on their training or educational challenges and needs. According to FAI officials, these meetings provide opportunities to share leading practices, identify challenges, and discuss potential initiatives. Recent discussion topics included plans to hold a competency-based certification workshop, roll out of additional FAITAS workforce management tools, and results of FAI’s bi-annual Acquisition Workforce Competency Survey.

In July 2012, FAI established the Federal Acquisition Council on Training to help share information on agencies’ training efforts and standardize acquisition training throughout the government. According to FAI officials, the council’s goal is to help ensure no acquisition workforce training seat, whether offered by FAI or another agency, goes unfilled. Agencies are asked to open up unfilled training seats to other agencies whenever available, and to use FAITAS as the official course registration system to communicate available training seats and enroll participants for courses to reduce overhead and administrative costs for agencies.

Periodically analyzing acquisition career fields and evaluating the effectiveness of training and career development programs for acquisition personnel14 According to officials, FAI has efforts underway to standardize the end-of-course participant evaluations administered by the vendors who provide FAI-sponsored courses. According to FAI officials, course participants will not receive their course completion certifications until they complete the course evaluation. Currently FAI does not analyze course evaluations administered by the vendors; however, officials commented that the use of standardized evaluations could enable them to work with vendors to improve the consistency of the information provided in training courses. A pilot is underway using standard evaluations for FAI courses taught by vendors being provided to one agency, and FAI plans to extend the pilot to other agencies early in 2013.

Agencies Generally Use External Sources for Acquisition Training and Face Similar Workforce Training Challenges

Most agencies approach acquisition workforce training through classroom courses taught by external sources—vendors, FAI, DAU, or other agencies. While all agencies have meeting spaces for training, three operate permanent centers with dedicated resources that train the agency’s acquisition workforce. The agencies’ current training focus is to provide courses through which their acquisition workforces may attain or maintain their FAC certifications. Agencies reported facing several challenges in providing acquisition-related training. The areas reported as being the most challenging are related to staffing and budgetary resources. Some agencies also reported challenges with the identification of their acquisition workforce, which is a fundamental step needed for managing the workforce and its training. Agencies also reported that additional assistance from OFPP and FAI would help their acquisition workforce training efforts. In addition, the agencies reported that their acquisition workforces are challenged in finding time in their workload to attend training. For more details on the approaches, budgetary and staffing resources, and other challenges faced by the agencies, see appendix I for a summary of the 23 agencies’ responses to the questionnaire we administered.

41 U.S.C. § 1201(a)(4) and 41 U.S.C. § 1201(a)(7).

Agencies Generally Rely on External Training Sources

Agencies provide acquisition workforce training predominantly through external sources—government entities or vendors. In fact, 17 agencies reported that the majority of their acquisition workforce training comes from having their workforce attend training provided by external sources rather than from their agency. The other government entities may provide training to an agency without seeking additional reimbursement, as is the case with FAI, DAU, and some federal agencies; however, others, such as the VA, charge a fee to attend their training to recoup their costs.15 Of the remaining 6 agencies, 5 reported that they hold agency-sponsored courses to provide the majority, if not all, of their acquisition workforce training, and one agency did not report. Some agencies use contractor personnel to instruct a portion of their agency-sponsored training. Figure 1 illustrates the sources of training that agencies reported using to provide training to their acquisition workforce in fiscal year 2011.

Although FAI does not seek reimbursement for individual classes, civilian agencies “pay” for the courses since they provide the funding for FAI’s operations through the mandatory fee paid equal to five percent of the dollar amount of acquisitions the agency makes through the Federal Supply Schedule and the Governmentwide Acquisition Contracts.

The Department of Transportation provided data that does not total 100 percent due to its averaging of the responses from its bureaus.

The Department of Justice data is an average of the responses submitted by 4 procurement offices— headquarters and 3 bureaus.

A leading training investment practice involves agencies taking steps to identify the appropriate level of investment to provide for training and development efforts and to prioritize funding so that the most important training needs are addressed first.16 The four agencies we selected illustrate that agencies use different approaches to manage and provide acquisition workforce training, due to such factors as the size of their workforce, the need for certification training, and the resources dedicated to training. Three agencies—DHS, Treasury, and VA—chose to have permanent training facilities dedicated to providing courses to the acquisition workforce. These three agencies differed, however, in the percentage of courses they provided to meet their acquisition workforce needs. DHS manages its entire acquisition workforce training at the department level and recently expanded the number of courses provided by the agency. However, part of DHS’s strategy is to make use of DAU and FAI courses that address its needs, which can be obtained at no additional charge. VA also manages all of its training at the department level and in 2008 established a training academy, the VA Acquisition Academy, to provide all acquisition-related courses to agency personnel who need it. Alternately, Treasury built its training program on an existing training institute within the Internal Revenue Service, which employs the majority of Treasury’s acquisition workforce, to provide agency courses available to all of its bureaus and to other agencies. Of the three dedicated training facilities, VA is the only one to provide agency- sponsored training for 100 percent of its acquisition training courses. VA, DHS, and Treasury all reported using contractors to instruct the agency- provided training at their respective facilities. Education does not have a dedicated facility and reported that it provides only about 10 percent of its acquisition workforce training via agency-sponsored courses, obtaining about 90 percent of needed training through external sources.

GAO-12-878. training needs. In their responses to our questionnaire, the agencies did not explain how effectiveness is determined. As we discuss later in this report, based on a subsequent data call we found that the agencies have limited insight into the effectiveness of their acquisition workforce training courses. Figure 2 illustrates the factors and number of agencies that reported them as influencing their training source selections.

Agencies’ current focus is to provide courses that allow their acquisition workforces to attain or maintain their FAC certification. In response to our questionnaire, most agencies—17 of 23—reported that they are able to find sufficient courses to do this. In their AHCP submission for fiscal year 2011, most agencies (15) reported certification rates of more than 75 percent for their contracting staff, but 3 agencies reported over 90 percent of their contracting staff were certified. The majority of the 23 agencies reported that more than 90 percent of their CORs and P/PM staff had obtained their basic FAC certifications—with many reaching 100 percent. Agency officials explained that the agency’s certification rates fluctuate due to changes in the workforce—including experienced staff leaving the agency and less experienced staff being in the process of obtaining certifications. Table 3 shows the certification rates reported by agencies for fiscal year 2011.

Officials from three of the four selected agencies—DHS, Treasury, and VA—explained that they prioritize their training funding based on the need for acquisition staff to be certified. Education, where the majority of the acquisition workforce has attained their certifications already, reported it is focusing on courses that meet staff needs to maintain certifications or develop expertise.

Agencies Face Challenges with Having Adequate Training Resources, Identifying the Acquisition Workforce, and Staff Having Time to Attend Training

The top challenges reported by agencies in obtaining training for their acquisition workforces were having sufficient resources—both staffing to manage the training program and budgetary resources—to provide training. Specifically, 20 agencies reported that obtaining adequate funding is a challenge, and 19 reported that obtaining sufficient staff to manage the acquisition workforce training is a challenge. As for determining the appropriate level of investment, we have previously reported that when assessing opportunities for training, the agency should consider the competing demands confronting the agency, the limited resources available, and how these demands can best be met with available resources. Figure 3 provides a summary of the number of agencies that reported challenges involved in obtaining sufficient acquisition workforce training.

Although almost all agencies reported that obtaining an adequate level of funding is challenging, less than half—10 of 23—reported that their current acquisition training budgets are insufficient to meet their training needs. The acquisition workforce training budgets for fiscal year 2012 reported by 22 of the agencies ranged from $0 to $39,598,000 for an agency with its own acquisition workforce training facility. These budgets correspond to acquisition workforces that ranged in size from 186 to 11,867 employees for fiscal year 2010 that handled anywhere from about 900 to 1.4 million contracting actions with obligations ranging from $132 million to $25 billion in fiscal year 2011.

Almost all the agencies—20 of 23—reported that they would like additional FAI assistance to help their acquisition workforce members attain federal certification by providing more course offerings or additional topics. In addition, to accommodate travel restrictions and budget constraints, some agencies would like FAI to provide more virtual courses for personnel that are geographically dispersed. FAI has some efforts underway that may somewhat address these issues, but their results are still uncertain as they are just beginning. FAI officials noted that while agencies want more offerings, FAI still has under-attended classes that have to be canceled. FAI officials also commented that developing on-line courses can also be very expensive and the cost to maintain currency of information is an often times overlooked cost.

GAO-11-892. personnel are dispersed throughout many organizations, come from a variety of career fields, and are often involved in acquisitions as a secondary and not a primary duty. The agencies’ Acquisition Career Managers told us that they face similar challenges in identifying CORs and other personnel with acquisition-related responsibilities. Three of the four agencies selected for further insight—DHS, Education, and VA— acknowledged that they continue to be challenged by identifying some members of their acquisition workforces.

A majority of the agencies—15 of 23—reported that they would like additional assistance from OFPP to improve their acquisition workforce training by either of two ways. Thirteen agencies supported the idea of OFPP creating separate job series for additional acquisition workforce categories, such as P/PMs. Twelve agencies supported the idea of OFPP maintaining a government-wide database to identify and track members. According to FAI and OFPP officials, they are sympathetic to agency concerns about identifying the total acquisition workforce. As for creating separate job series to more easily identify members of the acquisition workforce, OFPP notes that any changes would need to be made by the Office of Personnel Management and that some acquisition positions, such as CORs, may not lend themselves to becoming a unique job series because the work is performed as a collateral duty. As for maintaining a government-wide database to identify and track acquisition workforce members, FAI officials noted that agencies are encouraged to use the FAITAS registration system to track their workforce’s individual training records and certifications. However, registration into the system is currently voluntary.

Agencies also reported that their acquisition workforces are challenged in finding time in their workload to attend training. Most agencies—14 of 23—considered the ability to find time as extremely or very challenging. Figure 4 provides a summary of the number of agencies that reported challenges to their staff accessing and attending acquisition-related training.

Lack of Comparable Cost Data and Limited Insights On Benefits of Training Hinder Efforts to Maximize Resources Government-wide

Agencies collect some training cost data and limited information about the benefits of their acquisition workforce training. Based on responses to our questionnaire, a supplemental data request, and discussions with agencies’ and FAI officials, we found that many agencies do not collect data on the costs of training provided to their acquisition workforce that can be used to inform agency and government-wide training resource investment decisions. In addition, some agencies do not have metrics to assess the effectiveness of their training.

Government-wide Training Cost Data Reported Is Not Comparable

GAO-12-878 and GAO-04-546G. data eventually provided by the agencies included different cost components, which did not lend themselves to comparative analysis. For example, some agencies provided data on the cost per seat of specific courses, while others provided the total costs for delivery of each course. Some agencies provided the costs associated with the development of an in-house course, while others included development costs with delivery costs as a total cost of obtaining the course from a vendor. Although the data do not allow direct comparisons among agencies, the data show a range of costs for similar courses. For example, five agencies provided a per seat cost for a COR Refresher course, which ranged from $97 to $363 per seat. In addition, five agencies provided a per seat cost for a Cost Analysis and Negotiation Techniques course, which ranged from $282 to $925 per seat. However, a number of factors can produce the variation in the agency-reported costs and these factors, along with the agencies’ data collection methods, can limit the ability to make government-wide decisions on acquisition workforce training investments. According to agency officials, the costs of individual acquisition workforce training courses can vary greatly among agencies due to a variety of factors, including: the number of times per year the course is provided, actual attendance numbers, location of training, and whether courses are tailored with agency-specific information.

FAI officials reported that they also received limited responses to their request for training cost data in the agencies’ latest AHCPs, which made it impossible for them to complete their planned comparative analyses. The instructions for the FAI data request did not include definitions of the cost components to be reported, which are important to help ensure consistent reporting. Due to the limited response on cost data, FAI initiated a subsequent data call. As of November 2012, FAI had received responses from 20 of the 23 agencies, but again was unable to perform a comparative analysis of the cost data due to various limitations in the data it received. FAI officials noted that collecting cost data from agencies is an evolving process and that having comparable training cost data is important to help FAI in its efforts to maximize the use of acquisition workforce training dollars government-wide. FAI plans to request cost data annually with the AHCPs; however, the guidance for completing the next submission does not include definitions of the cost data to be reported, which may again yield data that cannot be compared across agencies. Improved reporting of cost data could assist FAI as it moves forward with its plans to award government-wide contracts for acquisition workforce training in 2014 by allowing FAI to obtain insight into the agencies’ training costs when obtaining training from a variety of external sources.

Agencies Gather Limited Information on Benefits of Acquisition Workforce Training Investment

About half of the agencies—12 of 23—did not have insight into whether their acquisition workforce training investment is improving individual skills or agency performance. In particular, in response to our questionnaire and subsequent data request 7 of 23 agencies reported having no metrics to monitor or assess the effectiveness of their acquisition workforce training efforts, a measure of whether the training investment is improving individual skills or agency performance. Five agencies did not provide information to support use of metrics to measure the training benefits, either improving individual skills or agency performance, of acquisition training in response to our supplemental data request.

We have previously reported that training programs can be assessed by measuring (1) participant reaction to the training program, (2) changes in employee behavior or performance, and (3) the impact of the training on program or organizational results, which may include a return on investment assessment that compares training costs to derived benefits. Of the 11 agencies that provided information to support their use of metrics, 3 reported using end-of-course evaluations to measure participants’ reaction, one reported using end-of-course tests to measure changes in employees’ knowledge, and one reported using post-course surveys to supervisors or participants to measure if what was learned affected the participants’ behavior. The other 6 agencies reported measures aimed at determining the impact of training on the agency’s mission. Furthermore, DHS officials said that they plan to begin using post-course surveys of participants and supervisors in fiscal year 2013, and they are pursuing the development of additional measures to evaluate the impact of training on the agency. VA is also pursuing the development of additional metrics, such as proxy measures for return on investment, to evaluate the impact of training on the agency. Participant evaluations offer limited insight into improvements in individual and agency performance; however, we recognize that higher levels of evaluation (such as evaluating the impact on organizational performance or comparing training costs to derived benefits) can be challenging to conduct because of the difficulty and costs associated with data collection and the complexity in directly linking training and development programs to improved individual and organizational performance.

Officials at some of the selected agencies told us that when the course is provided by an external source, such as FAI or DAU, they rely on the external source to provide end-of-course evaluations to the participants. As we noted earlier, 17 of 23 agencies obtain the majority of their acquisition training from external sources. In particular, some agencies noted that they do not collect or assess the end-of-course evaluations, relying solely on the external source to use the evaluations as they believe appropriate. FAI officials noted that if the agencies do not review the evaluations themselves, they are missing the opportunity to ensure that the courses are effective in training their acquisition personnel. We have previously reported that it is increasingly important for agencies to be able to evaluate their training programs and demonstrate how these efforts help develop employees and improve the agencies’ performance because it can aid decision makers in managing scarce resources and provide credible information on how training has affected individual and organizational performance.22 Although FAI has an initiative underway to standardize the evaluations provided for its courses, its impact may be limited if agencies do not obtain the evaluation results and use them to evaluate the effectiveness of the training.

Conclusions

GAO-12-878 and GAO-04-546G. to ensure that every training seat is filled—to assist agencies in meeting OFPP requirements and leveraging federal government training resources, these efforts face significant obstacles. Agencies, FAI, and OFPP lack data on the acquisition workforce itself and the benefits of training, as well as cost data that can be used to make comparisons. These limitations hinder government-wide efforts to share information during these times of constrained budgets. OFPP, FAI, and the agencies need basic information on how much agencies are spending to train the acquisition workforce. Providing definitions and guidance about the elements of costs agencies should include in their annual acquisition human capital plans will help OFPP and FAI in their efforts to collect comparable data across agencies. FAI can then analyze and share information to help agencies make choices regarding how best to dedicate resources to effectively train the acquisition workforce. Additionally, comparable costs data will be helpful to inform FAI efforts to award government-wide contracts for standard acquisition workforce certification training.

Currently, the main focus of monitoring and tracking acquisition workforce training efforts in agencies is on completion of training to attain and maintain the FAC certifications. Although the higher levels of evaluation (such as evaluating the impact on organizational performance or comparing training costs to derived benefits) can be challenging to conduct due to costs and complexity, agencies should, at a minimum, evaluate participant reaction to the training program through end of course evaluations. However, this is a basic measure that some agencies are lacking because the evaluations currently being administered go to the provider of the class and not the agency paying for the training. Therefore, these agencies have little insight into how the workforce perceives the training they received. We note that FAI has initiatives underway to standardize the course evaluations for the courses it provides and to make completion of the course evaluations mandatory to receive course credit. At present, this effort only applies to FAI-sponsored courses and will not include agencies that use other vendors, and does not guarantee that agencies will analyze the results; it is important for all agencies to collect and analyze this basic information.

Recommendations for Executive Action

To help ensure that agencies collect and report comparable cost data and perform a minimal assessment of the benefits of their acquisition training investments to aid in the coordination and evaluation of the use of resources government-wide, we recommend the Director of the Office of Management and Budget direct the Administrator of the Office of Federal Procurement Policy, in consultation with the Director of the Federal Acquisition Institute, to take the following two actions: provide further guidance, including definitions, on the types of costs that agencies should include in their Acquisition Human Capital Plan submission to help determine total training investment; and require all agencies, at a minimum, to collect and analyze participant evaluations of all acquisition workforce training as a first step to help assess the effectiveness of their training investment.

Agency Comments and Our Evaluation

We provided a draft of this report to OFPP, FAI, and the selected agencies—DHS, Education, Treasury, and VA. OFPP and FAI concurred with our recommendations in oral comments and e-mail responses. Education and Treasury responded via e-mail with no comments. DHS and VA provided technical comments, which we incorporated as appropriate.

In oral comments, OFPP and FAI agreed with our recommendations and noted that they have begun drafting guidance to federal agencies. Consistent with our report, OFPP and FAI emphasized the importance of acquisition workforce management tools that improve the government’s ability to leverage acquisition resources especially during this time of budgetary constraints. These officials highlighted FAITAS, which can serve as a workforce management tool and training application system. Currently, use of this system is voluntary and some agencies use the system to enroll participants in FAI courses, communicate available training seats within a specific agency’s training course, and track their workforce’s individual training records and certifications. Recognizing the potential benefits of this system in helping to coordinate and evaluate the use of training resources government-wide, OFPP stated that it is considering making FAITAS reporting mandatory for civilian agencies.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Administrator of OFPP, the Director of FAI, and the Secretaries of DHS, Education, Treasury, and VA. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4841 or MartinB@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Summary of Agencies’ Response to GAO Questionnaire on Civilian Acquisition Workforce Training

In July 2012, GAO administered a government-wide survey of the 23 civilian CFO Act agencies designed to gather information regarding the training approach and methods, staffing and budgetary resources, and costs associated with providing acquisition training—including the operation of facilities dedicated to training the acquisition workforce. We received responses from 100 percent of the agencies, although not every agency responded to each question. A summary of the number of agencies’ who responded to each question is provided below. We removed the open-ended narrative responses, but included the ranges of open-ended data. Details of our survey methodology are provided in appendix II.

The objectives of this review were to evaluate the training approaches of federal agencies, other than the Department of Defense, for the acquisition workforce. Specifically, this report addresses the (1) role of the Office of Federal Procurement Policy (OFPP) and the Federal Acquisition Institute (FAI) in promoting Federal Acquisition Certification (FAC) standards and assisting agencies in meeting acquisition workforce training requirements; (2) approaches agencies use to provide training to their acquisition workforces; and (3) the extent to which agencies track information on the costs and benefits of their acquisition training.

To identify the roles of OFPP and FAI in promoting FAC standards, we analyzed legislation on their authority, the OFPP acquisition workforce strategic plan, and relevant guidance provided by them to agencies. We interviewed officials at OFPP and FAI about efforts to (1) provide oversight and assist agencies in meeting training requirements; (2) leverage government-wide resources; and (3) share leading practices. Since FAI’s initiatives are in various stages of implementation, we did not evaluate their effectiveness; however, in several instances, we note our initial observations of the efforts.

31 U.S.C. § 901. used a standard set of questions to obtain data about their training approaches, including the sources for course offerings (in particular whether courses were provided by the agency, a commercial sector vendor, FAI, the Defense Acquisition University, or another agency), facilities, budgetary and staffing resources, methods of training, and challenges faced in providing acquisition training. Because this was not a sample survey, it has no sampling errors. However, the practical difficulties of conducting any survey may introduce errors, commonly referred to as nonsampling errors. For example, difficulties in interpreting a particular question, sources of information available to respondents, or entering data into a database or analyzing them can introduce unwanted variability into the survey results. We took steps in developing the questionnaire, collecting the data, and analyzing them to minimize such nonsampling error. After we drafted the questionnaire, we asked for comments from federal officials knowledgeable about acquisition workforce and training issues and from independent GAO survey professionals. We conducted pretests to check that (1) the questions were clear and unambiguous, (2) terminology was used correctly, (3) the questionnaire did not place an undue burden on agency officials, (4) the information could feasibly be obtained, and (5) the survey was comprehensive and unbiased. We chose the two pretest sites to include perspectives from agencies with robust acquisition workforce training programs and training facilities. We made changes to the content and format of the questionnaire after both reviews and after each of the pretests, based on the feedback we received. We sent the questionnaire by e-mail in an attached Microsoft Word form that respondents could return electronically after marking checkboxes or entering responses into open answer boxes on July 5, 2012. We sent the questionnaire jointly to the Chief Acquisition Officer, Acquisition Career Manager, and Chief Learning Officer of each agency, asking for a consolidated agency response. After two weeks, we sent a reminder to everyone who had not responded. All questionnaires were returned by August 10, 2012. We received questionnaire responses from 100 percent of the agencies, although not all agencies answered every question. The Department of Justice provided four responses—one from its headquarters procurement office and three from bureau procurement offices. We consolidated the information from the four surveys to ensure the tabulations eliminated duplicative counting of the agency and to provide averaged data, as appropriate. In questions regarding the agency’s opinions regarding challenges faced, we deferred to the response of the headquarters procurement office.

To confirm our understanding of the variety of issues addressed and obtain additional insights on the agencies’ responses to our questionnaire, we discussed our preliminary results with representatives from the 23 agencies at meetings of the Chief Acquisition Officers Council, Interagency Acquisition Career Management Committee, and the Chief Learning Officers Council. Table 4 provides a summary of the functions of these councils as related to government-wide efforts for training the acquisition workforce.

We also reviewed the agencies’ annual Acquisition Human Capital Plans (AHCP) due March 31, 2012, to collect information on certification rates, and corroborate information agencies provided in response to the questionnaire on acquisition workforce size and training approaches.

To determine the information agencies track on the costs and benefits of the training provided, we supplemented the questionnaire responses by soliciting additional cost and metrics data from the agencies using a data collection instrument. We developed our data collection instrument based on cost information provided from agencies in response to our questionnaire. We sent agency-tailored data collection instruments by e- mail in an attached Microsoft Excel spreadsheet that respondents could return electronically after updating or providing new data on October 15, 2012. We pre-populated each agency’s spreadsheet with any data previously provided by the agency, and sent the data collection instrument to the Acquisition Career Manager and the official who provided the response to the questionnaire. After two weeks, we sent a reminder to everyone who had not responded. The data collection instruments were returned by November 28, 2012. We received data collection instruments from 91 percent of the agencies.

To gather illustrative examples and more detailed explanations regarding training approaches and data tracked related to costs and benefits, we selected four agencies—the Departments of Education (Education), Homeland Security (DHS), the Treasury (Treasury), and Veterans Affairs (VA)—for further review. DHS, Treasury, and VA are agencies that operate dedicated acquisition workforce training facilities—permanent centers with dedicated resources that provide training specifically for the agency’s acquisition workforce. Education, which is one of the smallest agencies, has no dedicated facility and has reported issues with access to acquisition workforce training courses. We interviewed agency officials to determine the extent to which they were aware of and using the leading training investment practices that the Office of Personnel Management and experts agreed should be implemented by agencies to support effective training investment decisions.3 Table 5 provides a summary of these leading practices.

Of the eight leading practices GAO identified, we focused on the four practices dealing primarily with determining costs and effectiveness of training: (practice 1) identifying the appropriate level of investment to provide for training and development efforts and prioritize funding so that the most important training needs are addressed first; (practice 4) having criteria for determining whether to design training and development programs in-house or obtain these services from a contractor or other external source; (practice 6) tracking the cost and delivery of its training and development programs agency-wide; and (practice 7) evaluating the benefits achieved through training and development programs, including improvements in individual and agency performance.

We conducted this performance audit from February 2012 to March 2013, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: GAO Contact and Staff Acknowledgements

GAO Contact

Staff Acknowledgements

In addition to the contact name above, Penny Berrier (Assistant Director); R. Eli DeVan; Mya Dinh; Jean McSween; Aku Pappoe; Kenneth Patton; Carol D. Petersen; Sylvia Schatz; and Suzanne Sterling made key contributions to this report.