text: Background Training Is Important for Effective Grants Management In fiscal year 2017, the federal government awarded approximately $675 billion in grants to state and local governments. As shown in figure 1, approximately 80 percent of the grant dollars awarded by the federal government in fiscal year 2017 came from the three agencies we reviewed for this report—HHS, USDA, and Education. A range of skills are needed to manage the various tasks associated with the grants lifecycle. For example, during the award phase, grant staff at federal grant-making agencies are to send all grantees a grant award notification that provides details about the grant, including the amount of the award; and the general terms and conditions of the grant, including statutory and regulatory requirements. Figure 2 below illustrates the four distinct phases of the grants lifecycle. Given the billions of dollars in federal grants funding that are awarded every year, effective training could help provide grants managers with the skills and competencies they need to better manage and oversee those dollars. As one example of the importance of rigorous grants management and training, in April 2017 we found that Education grants staff inconsistently documented key required monitoring activities and, as a result, about $21 million in discretionary grants lacked the correct documentation of grantee performance. We recommended that Education establish and implement detailed written supervisory review procedures for official grant files to provide reasonable assurance that grant staff perform and document key monitoring activities. Education officials agreed with the recommendation and said they would develop a department-wide standard operating procedure (SOP) that will, among other things, provide standards for timeliness of documenting key monitoring and administrative activities and require the periodic review of grant files. Officials expect to complete the SOP by September 30, 2018. In 2011, OMB established the Council on Financial Assistance Reform (COFAR), an interagency group of executive branch officials with the stated aim of creating a more streamlined and accountable structure to coordinate financial assistance, including grants. In 2012 and again in fiscal years 2016 and 2017, COFAR identified the need to develop a qualified and professional workforce as one of six priorities to guide its work on grants management reform. According to OMB staff, they disbanded COFAR on June 15, 2017 as part of OMB’s efforts to reduce grants-related requirements once COFAR had recommended policies and actions to effectively deliver financial assistance. COFAR’s recommendations resulted in the Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards, which is intended to improve performance, transparency, and oversight for federal awards. Moving forward, the responsibility of coordinating financial assistance priorities was given to the Chief Financial Officers Council (CFOC), a group of 24 agency chief and deputy chief financial officers that work together to improve financial management in the U.S. government. According to OMB staff, the controller of OMB’s Office of Federal Financial Management is the chair of the CFOC. In addition, OPM is responsible for providing leadership and guidance over federal agency training to ensure the effective promotion and coordination of federal agency training programs and operations. Further, the President’s Management Agenda established “results-oriented accountability for grants” as a cross-agency priority goal to “maximize the value of grant funding by applying a risk-based, data-driven framework that balances compliance requirements with demonstrating successful results for the American taxpayer.” Certification Standards for the Grants Workforce In 2013, we examined grant workforce and training issues and found there were no specific government-wide training requirements for the federal grants workforce. As of June 2018, this continued to be the case. By contrast, there are government-wide training requirements for the acquisitions workforce intended to help ensure its quality and effectiveness. For example, OMB’s Office of Federal Procurement Policy (OFPP) provides government-wide guidance on managing the acquisitions workforce. The Federal Acquisition Institute, which coordinates with the OFPP, promotes the development of the civilian acquisitions workforce. Further, OFPP has developed Federal Acquisition Certification requirements for acquisition professionals serving as contracting staff, contracting officer’s representatives, and program/project managers. Notably, in fiscal year 2017, the federal government spent approximately $166 billion more on grants to state and local governments than it did on federal acquisitions. OMB staff explained that the acquisitions workforce faces more requirements because contracts have more uniform requirements and are specified in law. They stated that grants, on the other hand, are diverse and are established by individual statutes with varying conditions. Our work in the acquisitions area identifies the importance of providing reasonable assurance of an appropriately trained staff through certification. Certification programs are designed to ensure that individuals attain the knowledge and skills required to perform in a particular occupation or role by establishing consistent standards. For example, for the acquisition workforce, OFPP requires a minimum set of career-specific courses, along with education and experience requirements, to obtain certification. To ensure acquisition professionals remain current on acquisition policies and practices, OFPP also requires the acquisition workforce to meet continuing learning requirements. See appendix I for a comparison of training for the federal acquisition workforce versus the federal grants workforce. Education, HHS, and USDA delegate the decision to their various sub- agencies of whether grants employees should obtain professional grants certifications. Of the 11 sub-agencies we reviewed, 3 at HHS—the Centers for Medicare and Medicaid Services Discretionary Grants Office, the Health Resources and Services Administration, and the National Institutes of Health—and 2 at Education—the Office of Special Education and Rehabilitative Services and the Office of Post-Secondary Education—required certification of some of their grants employees. Officials at the remaining 6 sub-agencies offered certification to their grants employees on an optional basis. USDA sub-agency officials said they often recommend the certificate program to their grants employees, and Education’s sub-agency officials at the Office of Elementary and Secondary Education said they nominate staff to take the grants certificate program whom they believe would benefit the office most by receiving the training. While COFAR officials explored the possibility of establishing certification standards for the grants workforce by September 2015, OMB staff said they determined that certification was not the most appropriate course of action for the grants workforce for several reasons including risk management and internal control concerns and the need for a variety of skills for the grants workforce. As previously mentioned, OMB disbanded COFAR in June 2017, and CFOC took over COFAR’s responsibilities. When we spoke with OMB staff in the fall of 2017, they said their focus had shifted from establishing certification standards for the grants workforce to providing guidance on needed competencies and enabling the grants workforce to obtain them. OPM, OMB, and CFOC Have Taken Some Steps to Help Provide Grants Training but Have Opportunities for Further Improvements OPM, OMB, and CFOC Developed a Grants Competency Model Among Other Steps OPM, in consultation with OMB and the CFOC, took several steps to ensure the federal grants management workforce has access to grants management competencies and training. For example, OPM identified grants management competencies that could be used in agency efforts for workforce planning, training and development, performance management, recruitment, and selection. After establishing grants management competencies, OPM officials told us they established the 1109 job series partly because OMB and CFOC staff requested a new grants management job series in response to the increased grant awards and staffing needs created because of the 2009 American Recovery Act. Figure 3 illustrates the timeline of the main steps taken by OPM, OMB, and CFOC over the last decade. In 2008, OPM initiated a government-wide study to identify critical competencies for grants management work. After the government-wide study was completed, OPM issued a memorandum to all federal agencies announcing a grants management competency model that included general competencies such as accountability, writing, and computer skills. OPM also included technical competencies such as grants management, financial analysis, and compliance. In our prior work, we found that grants management competency models can be used to establish an overall framework to guide agencies’ training efforts. Before OPM established the 1109 job series in 2010, no other agency- specific job classification series existed for the many federal employees responsible for carrying out managerial and administrative tasks related to grants, including ensuring compliance with OMB and agency policies and procedures. In the absence of a specific job classification, we reported in 2013 that officials at selected agencies told us they had classified these employees under a variety of other job series that did not focus on grants, such as general, administrative, and subject-matter job titles. According to OPM officials, the agency’s development of the “Position Classification Flysheet for the Grants Management Series (1109)” leveraged the competencies and tasks from the Competency Model for Grants Management and input from federal agencies’ subject matter experts on grants management work.The Flysheet includes a job series definition, a basic job title, general occupational information, and a link to the position classification standard. The 1109 job series manage, supervise, lead, or perform administrative business, policy, and analytical work involving the: (1) management, award, or obligation of funds for grants; (2) competitive or non-competitive evaluation of grants proposals; and/or (3) administration or termination, and/or closeout of grants and/or grants assistance and agreement awards. The work requires knowledge of laws, regulations, rules, policies, procedures, and financial methods to help ensure accountability of the grant funds. As of fiscal year 2016, grant-making agencies reported 2,035 federal employees in the 1109 job series, and HHS reported 38 percent of those employees (see figure 4). We used fiscal year 2016 data to determine the agency-wide numbers of 1109 job series employees because this was the most recent set of full year data available at the time of our analysis. The federal grants workforce also includes a wide range of employees in other non-1109 job series positions. OPM does not collect data on grants workforce employees in these other job series positions as they span a large number of different job series that can vary by agency. Non-1109 employees working on grants typically possess expert knowledge in the specific area necessary to meet a grant’s goals (e.g., announcing the terms and conditions of a grant, recommending potential grantees, and monitoring grantees’ progress in achieving the grants goals). Reflecting the wide variety of federal programs that grants support, these individuals typically possess expertise in a specialized program or subject. A number of factors affect usage of the 1109 job series within agencies. According to OMB staff, various agency employees have told them that many agency employees would rather be classified as a subject matter specialist, such as a scientist, rather than a grants management specialist whose primary tasks are grants management under the 1109 job series. In addition, OMB staff said that some agencies preferred recruiting staff using a more general non-1109 job series classification. OMB staff also said that some agencies indicated their grants workforce employees do not want to be classified as grants specialists because the other job series are more general and are a better fit in terms of the needed subject matter expert skills and duties. We found that one of our selected agencies, Education, does not use the 1109 job series at all because, according to Education sub-agency officials, they require grants employees to have specialized grant program content knowledge in the field of their grant program focus, such as rehabilitation, special education, behavior science, and other areas (e.g., standards and assessments, state accountability systems). The sub- agency officials said that 1109 grants management specialists would not have the specific content knowledge and experience associated with the specific educational grant programs that Education requires. We also found that over 61 percent of HHS grants workforce employees and over 90 percent of the USDA grants workforce was not part of the 1109 job series. OPM officials told us that, in April 2017, they started a government-wide Grants Management Post Classification Implementation Study that may change the Grants Management Classification Flysheet and revalidate the Competency Model for Grants Management Work. OPM officials developed the study after meeting with grant-making agency HHS and will include a survey of the grants management workforce government- wide. OPM officials also stated they are in the final stages of developing and clearing the government-wide survey and anticipate issuing it in the fall of 2018. They said the study will take several additional months to complete because the team must review the results of the government- wide survey and update competencies, job classifications, and compliance policy/requirements. OMB and CFOC Have Provided Some Grants Training and Guidance, but Use Has Been Limited Among Selected Agencies OMB’s role with the grants management workforce includes issuing government-wide guidance and providing a framework that enables agencies to take actions to align their grants training with OMB’s internal control standards. In this role, OMB has taken some actions to provide grants guidance for federal agencies that include the Career Roadmap Report, Career Roadmap Builder, and Grants Training 101. However, we found that almost all of the officials we interviewed at the 11 selected sub- agencies were not familiar with the Career Roadmap Report and Career Roadmap Builder. Additionally, almost all of them did not mention using Grants Training 101 as part of their grants workforce training. Financial Assistance Career Roadmap OMB, in collaboration with the CFOC, COFAR, and federal awarding agencies, developed the Financial Assistance Career Roadmap Report in June 2017. OMB staff said that the Career Roadmap Report is one vehicle used to address grants training for the federal agency grants workforce. It is a tool for federal agencies to identify and document the competencies needed for successful job performance of federal financial assistance management professionals. According to the CFOC, the competencies and related elements outlined in the Career Roadmap Report are to be used to identify and prioritize training needs for the federal financial assistance management workforce. This is an optional tool for the federal grants workforce and may be customized to reflect an organization’s unique requirements and specifications. That workforce includes the grants management 1109 job series employees, as well as employees performing grants responsibilities as program, finance, and audit experts who are classified under other job series. During the initial development of the Career Roadmap Report, a team consisting of OMB staff and industrial and organizational psychologists collected financial assistance research and documentation from OMB, federal awarding agencies, and OPM. The team analyzed this information to identify foundational competencies and create a draft competency model which OMB reviewed. The team also facilitated two workshops with specialists on financial assistance management to gather feedback on the Career Roadmap Report. Figure 5 below shows the 14 different competencies from the Career Roadmap Report that are divided into two types of competencies: functional and leadership. After the report’s release, CFOC developed and released an interactive version called the Career Roadmap Builder available to the public online. This version allows users to build their own customized financial assistance management Career Roadmap based on their specific mission and needs. To obtain a custom Career Roadmap Report, users complete several steps in the Career Roadmap Builder involving selection of one or more of nine functional competencies; one or more of three job levels (foundational, practitioner, or one of three proficiency levels for each functional competency (basic, intermediate, or advanced); an option to include a leadership competency; and one of three different leadership levels (entry, mid, or senior) and a leadership proficiency rating (basic, intermediate, or advanced). The user then receives a customized report with relevant competencies, career levels, a sample of the associated developmental experiences and recommended training courses. Department-level officials we spoke with at HHS, USDA, and Education were familiar with the Career Roadmap Report. However, almost all of the officials we interviewed at the 11 selected sub-agencies were not aware the Career Roadmap Report was available to them. All but one of the officials we spoke with at four HHS sub-agencies said they were unaware of the Career Roadmap Report and grants management competencies. While USDA’s agency-wide Federal Financial Assistance Committee received a copy of the Career Roadmap Report in August 2017 and discussed it at their monthly meetings, almost all of the officials at the four USDA sub-agencies we reviewed said they had not received it. However, three sub-agency officials were familiar with the report because they had been involved with agency-wide efforts to provide grants management competency support and information. All other USDA sub-agency officials with whom we spoke were unfamiliar with the Career Roadmap Report or the grants workforce competencies. Almost all of the officials we interviewed at three Education sub- agencies were unaware of the Career Roadmap Report. However, one official from one sub-agency was familiar with the Career Roadmap Report as he had been part of the Career Roadmap Report development process. OMB staff stated they publicized the report by sending a “Controller Alert” on July 3, 2017 to agency chief financial officers and to members of the Financial Assistance Committee for E-Government notifying them of its availability and OMB’s future plans to map it to existing training resources, place it on OPM’s website, and develop an online interactive tool including position competencies. However, we found it difficult to locate the “Controller Alert” on the COFC website as it is not located on the same tab where the Career Roadmap Report is published but instead in a news section that users may not know to search. Further, OMB’s “Controller Alert” states that it “does not constitute official guidance or prescribe specific tasks for agencies beyond consideration of appropriate steps to address the issue.” OMB did not issue any official government- wide memorandums to explain that it supported the Career Roadmap Report, or that the report included updated competencies for both the 1109 and non-1109 job series workforce. Our internal control standards state that management should internally communicate the necessary quality information to achieve the entity’s objectives. However, if all levels of an agency are not aware of government grants workforce competencies and guidance, the agency may not be able to ensure that grants workforce employees have the training resources needed to develop and maintain skills to achieve the objectives of grant awards. OMB Grants Training 101 OMB also worked with federal grant-making agencies, COFAR, and the CFOC to establish Grants Training 101, a set of five online training modules designed to provide federal officials a basic knowledge of grants and cooperative agreements. According to OMB staff, the Grants Training 101 webpage states that the training is not designed to provide detailed administrative, accounting, and audit requirements specific to statutory provisions, agency regulation, and guidance because agencies need to have flexibility in designing grants training programs to meet those grant- specific statutory requirements. OMB staff said they designed the training modules in response to a request from the federal grants community for a government-wide grants management training resource to ensure some level of consistent training among grant-awarding agencies. In addition, OMB staff said it was optional for agencies to incorporate Grants Training 101 into established grants training and that each agency is responsible for the means by which they conduct grants management training. Only one of the agencies we reviewed had plans to include OMB’s Grants Training 101 as part of its grant-training program. HHS officials said they are developing an internal online grants 101 course and plan to incorporate parts of OMB’s Grants Training 101. However, most agency and sub-agency officials we spoke with did not use OMB’s Grants Management 101 as part of their grants workforce training. OMB staff said that Grants Management 101 modules cover the grant lifecycle and the requirements of the Uniform Guidance, and are intended to complement other trainings that agencies provide to their grants managers. OMB staff said that agencies make the decision whether to use the Grants Training 101 modules and can integrate parts of the training modules into their agency- specific training requirements. For example, officials at one of the agencies—Education—stated they cover many of OMB’s Grants Training 101 learning objectives through their cross-cutting grant training program courses as well as sub-agency specific training. Furthermore, OMB staff said that each agency would have to internally monitor grants employees’ completion of the grants training modules. OMB and CFOC Do Not Collect Detailed User Data or Feedback to Determine Usefulness of Grants Training and Guidance OMB staff told us that OPM initially had the responsibility of hosting the first two modules of Grants Training 101 on the OPM website while the remaining three modules were under development. After these remaining modules were completed, all five of the modules were moved to the CFOC webpage. In addition, OPM was responsible for collecting the Grants Training 101 user and completion data. OMB provided us the Grants Training 101 data which totaled 1,277 users registered between December 2015 and November 2017; however, we found that the data were incomplete due to missing data fields. OMB staff stated that the Grants 101 training website was moved to the CFOC webpage so the general public can access it. The CFOC will not collect data on the access dates, the agency names, or the number of Grants Training 101 users; however, the CFOC will collect data on the number of visitors that go to the Grants Training 101 website. OMB staff also said that agencies can decide to track Grants Training 101 users internally because OMB and the CFOC will not collect specific data on users. In addition, OMB staff said OMB and CFOC have not collected any formal Grants Training 101 feedback from users and have no plans to do so. OMB reported that a total of 175 visitors went on the Career Roadmap Report website between September 2017 and January 2018. Our Standards for Internal Control in the Federal Government advise management to process data into quality information that is appropriate, current, complete, accurate, accessible, and provided on a timely basis. It further states that management should also evaluate the processed information and revise when necessary so that it can be used to make informed decisions. In addition, our 2004 Human Capital Guide states that it is increasingly important for agencies to be able to evaluate their training and development programs to demonstrate how these efforts help develop employees and improve the agencies’ performance. As part of this approach, the Human Capital Guide also states that assessing training and development efforts should consider feedback from employees. OMB, CFOC, and COFAR devoted time and multiple resources to developing the Career Roadmap Report to identify and document the competencies needed for successful job performance of federal financial assistance management professionals. Obtaining more detailed user information and regular feedback from federal agencies on the usefulness of the Career Roadmap Report and the online Career Roadmap Builder could help OMB and CFOC to evaluate the effectiveness of these grant training tools. In addition, obtaining user information and feedback from federal agencies on the usefulness of Grants Training 101 can also help OMB and CFOC evaluate its effectiveness. HHS, USDA, and Education Vary in Following Selected Leading Training Practices In 2004, we issued a framework of principles and key questions that federal agencies can use to ensure that their training and development investments are targeted strategically and are not wasted on efforts that are irrelevant, duplicative, or ineffective. Our framework identifies four components of the training and development process: (1) Planning, (2) Design and Development, (3) Implementation, and (4) Evaluation. Within each component, the guide identifies leading practices and questions for agencies to consider when assessing each of these four components. We compared current grants training practices at the selected agencies and sub-agencies with selected leading training practices from the guide. We found variation among sub-agencies in following those selected training practices. Planning: skills and competencies assessment. In our guide, we stated that effective workforce planning and training begins with a skills and competency assessment. A leading practice under this component is that agencies use an organization-wide knowledge and skills inventory and industry benchmarks to help identify performance problems in their workforces. We stated that workforce planning should entail the collection of valid and reliable data on such indicators as distribution of employees’ skills and competencies. Officials we interviewed at all the selected sub-agencies explained that grants training needs are primarily identified by grants management supervisors or self-identified by grants workforce employees. The training needs are identified on an ad hoc basis during (1) manager evaluations or observations of employee performance, (2) annual and semiannual performance assessments, and (3) employee career individual development plans. When it came to implementing a more rigorous process involving a knowledge and skills inventory or the collection of valid and reliable data, we found varied use among the 3 agencies and 11 sub-agencies with only some employing such a method. The four HHS sub-agencies we reviewed assess new grants workforce employees’ knowledge, skills, and abilities by identifying skills gaps when onboarding new grants workforce employees, through supervisor observation of employee performance, or employee feedback. In fiscal year 2015, USDA’s Food and Nutrition Service (FNS) sub-agency started holding monthly meetings with its Regional Grants Management Division Directors to identify national training needs for its grants management staff. In fiscal year 2017, FNS also conducted a nationwide qualitative survey of its grants employees to identify training gaps and needs. The remaining three sub-agencies we reviewed informally identify skills gaps and training needs through ongoing discussions between supervisors and grants employees and during annual performance evaluations. Officials from Education’s central Learning and Development office stated they issue a department-wide competency assessment and training needs assessment to the various department sub-agencies annually or bi-annually. Officials from Education’s Office of Elementary and Secondary Education sub- agency told us they also conduct their own grants workforce learning needs assessment examining grants tasks, content knowledge, and general skills. Officials at the other two Education sub-agencies told us they assess skills gaps and training needs through ongoing discussions between supervisors and grants employees, supervisor observation of employee performance, and also during annual performance evaluations. Without a formal knowledge and skills inventory or collection of valid and reliable data on the grants workforce’s skills and competencies, some sub-agencies may be limited in identifying performance problems, competency gaps, and training needs in their grants workforce. Design and development: using a mix of approaches, sources, and delivery. Design and Development involves identifying specific training and development initiatives that the agency will use, along with other strategies, to improve individual and agency performance. One of the leading practices under this component is choosing the most appropriate mix of centralized and decentralized management of training programs; internal and external training sources; and training delivery mechanisms (e.g., classroom, computer-based, on the job, etc.). All three agencies provide the majority of their grants training at the sub-agency level. In most cases, the sub-agencies use a mix of training sources and delivery methods in developing and implementing their grants training programs, including identifying training needs and training content, as detailed in appendix III. HHS and USDA primarily use decentralized approaches to grants training while Education uses a hybrid approach of centralized and decentralized grants training. Although there is no overarching grants training program across HHS, the department’s central offices provide topic-specific training to Chief Grants Management Officers (CGMO) within each sub-agency on an ad hoc basis as new grant policies or requirements are developed. CGMOs then decide how to disseminate this information within their respective sub-agencies (e.g., through webinars, teleconferences, or ad hoc trainings). An HHS council comprised of CGMOs also meets on a quarterly or biannual basis to discuss new grants policy and requirements. Further, HHS’s central grants offices are developing a foundational “Grants 101” course to help standardize a baseline of grants knowledge across all of HHS’s sub-agencies, which they expect to complete by November 2018. Currently, the sub- agencies provide the majority of grants-specific training, which focuses on grants topics and mission requirements relevant to their specific areas. USDA’s Office of Chief Financial Officer (OCFO) provides some required training courses across the agency such as suspension and debarment and federal appropriations law training; however, these trainings are not specific to just the grants workforce employees. The sub-agencies provide all grants-specific training. Of the three selected agencies, Education provides the most central office training. For example, Education’s OCFO provides agency-wide training on discretionary and formula grants financial and budgetary courses; Learning and Development provides introductory grant courses; and Risk Management Services provides risk-based grants training covering topics including cost analysis, budgetary review, monitoring grants, and uniform guidance. Additionally, Education’s sub-agencies provide mission- and program-specific grants training to augment the centrally provided trainings. Centralized and decentralized training approaches may present different advantages for agencies and sub-agencies. On the one hand, efficiencies may be achieved by centralizing the design and delivery of some grants training that has widespread applicability throughout the agency. Additionally, if each sub-agency is responsible for implementing its own grants training program, the potential exists for inconsistent grants workforce training across the agency. On the other hand, each sub- agency is able to tailor the training to its own needs when it manages and provides the training itself. In making this decision, it is important for agencies to carefully analyze and consider trade-offs. Implementation: establishing agency-level accountability. Implementation involves ensuring effective and efficient delivery of training and development opportunities in an environment that supports learning. One of the leading training practices under this component is an agency organization that is held accountable, along with the line executives, for the maximum performance of the workforce. According to our Human Capital Guide, there are different ways of ensuring accountability, including establishing clear lines of authority in agency policies, issuing agency-wide guidance to ensure consistency, and establishing a central oversight office, among others. We found variation among the three selected agencies in following this leading training practice with HHS and Education having some agency level of accountability but USDA having less. HHS’ central Office of Grants Policy, Oversight, and Evaluation assigns desk officers to work with sub-agency CGMOs in helping them understand available training resources and needs. HHS also has an Executive Committee for Grants Administration Policy Council that meets quarterly to discuss regulations, policies, and grants administrative requirements. This committee is made up of CGMOs from each HHS sub-agency. HHS describes the roles of officials involved in overseeing grants management in an agency- wide grants policy manual. USDA has not defined roles for central offices to hold them accountable for grants training. While its central OCFO provides some guidance on federal financial assistance policies and grants terms and conditions, and ensures department-wide training requirements are met, USDA has no agency-wide grants training guidance, no agency-wide grants manual, or a central office that oversees grants training at the component level. Education officials stated that the agency has two agency-wide grants policy manuals and some Education offices have roles in overseeing grants training. For example, the central Learning and Development office provides some oversight of employee development, training programs, and providers. Further, Education officials stated that Risk Management Services oversees Education’s licensure training program across the sub- agencies, and OCFO provides agency-wide training on financial management of grants. Holding a central office accountable for grants training can provide agencies with reasonable assurance that training is being delivered efficiently and effectively and that grant staff have sufficient developmental opportunities. In this way, agencies can better ensure the maximum performance of the grants workforce. Evaluation: using data to assess training results. Evaluation involves assessing the extent to which training and development efforts contribute to improved performance and results. A selected leading training practice under this component is the use of performance data (both qualitative and quantitative measures) to assess the results achieved through training and development efforts. The three agencies we reviewed primarily conduct evaluation at the sub- agency level. The sub-agencies vary as to how they carry out their evaluations and few use any quantitative performance measures to determine if training was successful. HHS officials stated the central offices do not measure the effectiveness of training, nor is there centralized information sharing on how well training works. Officials at the HHS sub- agencies we reviewed told us they primarily use informal feedback such as ongoing conversations between employees and supervisors after training completion and supervisor observations of employee performance to determine if grants training is successful. Officials at HHS’ Health Resources and Services Administration also said they receive data regarding employee scores on required grants training courses. Some HHS sub- agencies use an external vendor for some grants training and employees complete a survey at the end of each of these courses, but HHS officials do not see those results. HHS officials rely on employee feedback after training completion to determine if external vendor training is effective. Officials at the USDA sub-agencies we reviewed told us they primarily use informal feedback through supervisory review of employee performance and employee individual development training plans; internal local level reviews and audits of grant processes; and some course completion surveys. Officials at Education’s central Learning and Development office told us they conduct electronic course evaluation surveys. Officials at the Education sub-agencies we reviewed told us they primarily use informal feedback from employees, supervisor observation of an employee’s progress after training, and some course evaluations. While informal, qualitative feedback from employees taking grants training is useful, it is not quantifiable or measurable. Using a balanced approach that reflects feedback from employees as well as organizational results is more effective in terms of evaluating the usefulness of grants training efforts. Many of the issues discussed above regarding following leading training practices stem from limited oversight of the sub-agencies, which we describe in the next section. Selected Agencies Provide Limited Monitoring and Oversight of Sub- agencies’ Grants Training Efforts Selected Agencies Cannot Readily Identify All Employees Working on Grants and Provide Limited Oversight of Sub- Agencies’ Grants Training Efforts As previously mentioned, the federal grants workforce consists of employees in the OPM Grants Management Specialist 1109 job series as well as employees in various other OPM job series (referred to as non- 1109s in this report). HHS and USDA both employ 1109s as well as non- 1109s in their respective grants workforces while Education only employs non-1109s. According to HHS, USDA, and Education officials, each sub- agency is responsible for identifying its grants workforce employees and ensuring they receive needed grants training. However, the central offices do not have a reporting mechanism tracking sub-agencies’ grants workforce. After querying each sub-agency, at our request, officials from the three agencies provided us with data on 1109 and non-1109 grants personnel. As figure 6 shows, the majority of grants personnel at the three agencies we reviewed are non-1109 employees. Standards for Internal Control in the Federal Government state that, “Management should demonstrate commitment to recruit, develop, and retain competent individuals.” Furthermore, internal controls state that “management evaluates competence of personnel across the entity in relation to established policies.” Since the agencies we reviewed cannot readily identify their total grants workforce, they have limited ability to evaluate the competence of grants personnel across the entity to ensure they are receiving needed training. Since the three agencies we reviewed do not centrally monitor their sub- agencies’ identification of grants employees, they cannot readily identify the agency’s total grants workforce. Consequently, the selected agencies do not have reasonable assurance that all employees working on grants across their agency are receiving needed grants training and have the necessary knowledge, skills, and abilities to properly manage, administer, and monitor grants. Central offices at HHS, USDA, and Education provide limited oversight of the types of training sub-agencies provide to their grants workforce. Our Human Capital Guide identifies having an agency organization that is held accountable, along with the line executives, for the maximum performance of the workforce as a leading practice. Further, the guide states that the agency’s training organization and line executives should work together to establish control mechanisms to ensure that agency employees successfully complete required and assigned training and development. Additionally, the guide states that agencies must assign authority and delegate responsibility to the proper personnel and establish clear accountability for maximizing workforce performance. However, as mentioned earlier, there is no overarching office responsible at the selected agencies for overseeing the types of grants training sub- agencies provide. Additionally, the central offices at the selected agencies do not evaluate sub-agency grants training efforts. We found variation among the 11 sub-agencies’ grants training programs (as shown in appendix III), which highlights the importance of central office oversight for making sure the training variation is appropriate. As a result of these issues, the selected agencies do not have assurance that grants training provided across the various sub-agencies is sufficient in meeting the needs of the various employees working on grants. Since there is no overarching central office at any of the three agencies we reviewed actively being held accountable for sub-agency grants training programs, HHS, USDA, and Education cannot ensure that all of the sub-agencies working on grants are sufficiently training their grants employees. Without central agency oversight and accountability across sub-agency grants training programs, not all grants employees may be sufficiently trained on grants processes and procedures, which could affect grant oversight in terms of grants employees monitoring grants properly. Conclusions Given the importance of grants as a tool to achieve federal objectives and the large outlays the federal government makes to fund them each year, it is critical that the people who manage these grants—the federal grants workforce—be well-trained to handle their responsibilities. To help provide training to this workforce, OPM, OMB, and CFOC created grants management competencies, a grants job series, some grants training, and a career roadmap. However, they have not widely publicized the roadmap and some sub-agencies we reviewed were unaware of it. Moreover, OMB and the CFOC are not collecting detailed data on users or feedback, which limits their ability to determine how useful these resources are to the federal grants workforce. The selected agencies varied in following selected leading training practices and they provided limited monitoring and oversight of their sub- agencies’ grants training efforts. Without sufficient monitoring and oversight, the agencies cannot have reasonable assurance that their sub- agencies are sufficiently training their grants workforce so they have the necessary knowledge, skills, and abilities to properly manage, administer, and monitor the billions of dollars that the federal government spends on grants annually. Recommendations for Executive Action We are making a total of five recommendations, including two to OMB and one to each of the selected agencies in our review. Specifically: OMB’s Office of Federal Financial Management’s Controller (the CFOC chair) should ensure CFOC formally publicizes the Career Roadmap guidance among the 24 CFO agencies through memorandums, briefings, trainings, regular CFOC meetings, or technical assistance and clearly posts its “Controller Alert” on the CFOC website with the Career Roadmap Report. (Recommendation 1) The Director of OMB, working with CFOC, should (1) collect data metrics regularly on the Career Roadmap Builder online tool and Grants Training 101 to determine how widely the resources are being used, and (2) obtain periodic feedback from federal agencies on the usefulness of these tools and any needed improvements. (Recommendation 2) The Secretary of HHS should establish a process to monitor and evaluate HHS’s grants training at the central office level. This process should include (1) a method for identifying all employees working on grants across the agency, and (2) oversight procedures to evaluate the sufficiency of sub-agencies’ grants training efforts including the incorporation of leading practices related to assessing competencies, training approaches, accountability, and training results. (Recommendation 3) The Secretary of USDA should establish a process to monitor and evaluate USDA’s grants training at the central office level. This process should include (1) a method for identifying all employees working on grants across the agency, and (2) oversight procedures to evaluate the sufficiency of sub-agencies’ grant-training efforts including the incorporation of leading practices related to assessing competencies, training approaches, accountability, and training results. (Recommendation 4) The Secretary of Education should establish a process to monitor and evaluate Education’s grants training at the central office level. This process should include (1) a method for identifying all employees working on grants across the agency, and (2) oversight procedures to evaluate the sufficiency of sub-agencies’ grants training efforts including the incorporation of leading practices related to assessing competencies, training approaches, accountability, and training results. (Recommendation 5) Agency Comments and Our Evaluation We provided a draft of this product to Education, HHS, OMB, OPM, and USDA for review and comment. In written comments reproduced in appendixes IV and V respectively, HHS concurred and Education generally concurred with our findings and recommendations directed at them. Both agencies described the steps they were taking to implement our recommendations. In an email, the Chief Learning Officer said that USDA concurred with our findings and recommendation. In an email, a Management Analyst said that OPM had no comments on the draft report. OMB staff provided us with oral comments stating that the agency partially concurred with our first two recommendations. Specifically, for our first recommendation, OMB generally agreed with our finding that the Career Roadmap guidance should be better publicized. However, OMB believes this is not its responsibility but rather the responsibility of federal agencies. OMB stated that federal agencies could incorporate a method into their improvement plans to ensure that sub-agencies are made aware of the Career Roadmap Guidance. We believe that, as the federal government’s central management agency and developer of the Career Roadmap, OMB has a responsibility for ensuring that federal agencies are aware of the Career Roadmap guidance by formally publicizing it through memorandums, briefings, trainings, regular CFOC meetings, or technical assistance. For the portion of our first recommendation that discusses clearly posting the “Controller Alert,” OMB stated it will look at the alert’s placement on the CFOC website to see if the agency can make it more prominent. We continue to believe that the “Controller Alert” should be easily accessible to anyone visiting the website and should be located on the same page as the Career Roadmap, where it would have greater visibility. For our second recommendation, OMB agreed that user feedback data regarding the Career Roadmap Builder and Grants Training 101 is useful. However, OMB stated that while it will continue to collect data on the number of users, it believes that federal agencies should be responsible for collecting specific, detailed user data if they are using those resources. We continue to believe that OMB and CFOC would benefit from collecting specific, detailed user data on these tools, which they devoted time and multiple resources to developing. Collecting detailed data metrics that go beyond the number of users can help OMB and CFOC to better evaluate the effectiveness of these grants training tools. Additionally, OMB stated the agency is committed to working with CFOC to review the Grants Training 101 module to determine how useful it is and if any improvements or adjustments are needed. All five agencies provided technical comments on the report draft, which we incorporated where appropriate. We are sending copies of this report to the Secretaries of Education, HHS, and USDA and to the Directors of OMB and OPM. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI. Appendix I: Comparison of Federal Acquisition Training and Grants Workforce Training Appendix II: Grants Workforce by Job Series for Health and Human Services, Agriculture, and Education as of March 2018 Appendix III: Grants Training Programs at the Departments of Health and Human Services, Agriculture, and Education The Department of Health and Human Services (HHS). HHS is a large agency with 11 sub-agencies administering a wide variety of health and human services that takes a decentralized approach to training its grants workforce. While HHS’ central Assistant Secretary for Financial Resources (ASFR) office provides grant policy and regulatory guidance updates to HHS sub-agencies, ASFR officials said they leave the decision on how to implement grants training to each of those sub-agencies. The selected sub-agencies we reviewed—the Administration for Children and Families, Centers for Medicare and Medicaid Services, Health Resources and Services Administration, and National Institutes of Health—all implement their own grants training programs and procedures. The four sub-agencies at HHS that we reviewed take different approaches in how they implement their respective grants training programs. For example, some sub-agencies require that grant personnel take required courses while others make them optional; some provide internal grants training while others also use the services of an external training vendor; and some require certification while others make it optional. Table 3 highlights some of the grants training programs’ characteristics at the four HHS sub-agencies we reviewed. The Department of Agriculture (USDA). USDA is made up of 29 agencies and offices at more than 4,500 locations across the country and abroad. While its central Office of the Chief Financial Officer (OCFO) provides some guidance on federal financial assistance policies and grants terms and conditions, and ensures department-wide training requirements are met, it, like HHS, leaves the decision on how to implement grants training to each of its sub-agencies. The selected sub- agencies we reviewed—the Food and Nutrition Service, Forest Service, National Institute of Food and Agriculture, and Rural Development—all implemented their own respective grants training programs and procedures. Table 4 highlights some of the grants training programs’ characteristics at the four USDA sub-agencies we reviewed. The Department of Education (Education). Education approaches grants training by combining both centralized and decentralized approaches for its eight principal offices that conduct grant work. Education’s central OCFO offers broad financial grants training such as Oversight of Financial Management of Ed Formula/Discretionary Grants and Discretionary Grant Budget Reviews. Education’s central Learning and Development office offers broad introductory grants training such as Introduction to Grants and Cooperative Agreements, Uniform Administrative Guidance, and Cost Principals. According to Education officials, Education’s Risk Management Services (RMS) offers risk management-based grants training including Discretionary Grants Overview, Conducting a Cost Analysis and Budget Review, Monitoring Grants, Suspension and Debarment, and Risk Assessment and Risk Mitigation. RMS also manages Education’s licensing program and oversees training for new license holders geared towards grants administration. In addition to these central office trainings, each Education sub-agency also provides specific training tailored for its mission as verified by the three Education sub-agencies we reviewed—the Office of Special Education and Rehabilitative Services (OSERS), the Office of Elementary and Secondary Education (OESE), and the Office of Post-Secondary Education. For example, according to Education officials, OSERS trains grant staff on the Individuals with Disabilities Education Act grant application review process, and OESE recently identified a need for and developed and taught a course on improving the grantee communication process. Table 5 highlights some of the grants training programs’ characteristics at the three Education sub-agencies we reviewed. Appendix IV: Comments from the Department of Health and Human Services Appendix V: Comments from the Department of Education Appendix VI: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Tom James (Assistant Director), Jyoti Gupta (Analyst-in-Charge), Benjamin Adrian, Dawn Bidne, Jeff DeMarco, Karin Fangman, Joseph Fread, Robert Gebhart, Shirley Hwang, Serena Lo, Sharon Miller, Meredith Moles, Steven Putansu, Kayla Robinson, Robert Robinson, Cynthia Saunders, Stewart Small, and Dan Webb made key contributions to this report.
target: In fiscal year 2017, the federal government awarded approximately $675 billion in grants to state and local governments. GAO was asked to review the federal grants workforce training. GAO reviewed (1) OPM's, OMB's, and the CFOC's actions to address the grants workforce's training needs; (2) the extent to which grants workforce training at selected agencies is consistent with leading practices; and (3) how selected agencies monitor and oversee training of their grants workforce. GAO selected HHS, USDA, and Education and several of their sub-agencies based on their grants spending and numbers of grants management specialists. GAO reviewed OPM and OMB memorandums and guidance, compared selected agency training practices against leading training practices, and interviewed officials. The Offices of Personnel Management (OPM) and Management and Budget (OMB) and the Chief Financial Officers Council (CFOC) have taken some steps to help ensure the federal grants workforce receives training. For example, OMB worked with the CFOC to issue five basic grants training modules and a “Career Roadmap” for grants managers; however, they did not widely publicize the resources. Many of the officials with whom GAO spoke at selected sub-agencies at the Departments of Health and Human Services (HHS), Agriculture (USDA), and Education (Education) were unfamiliar with the Career Roadmap and made limited use of the training resources. Further, OMB and CFOC do not collect detailed user data or feedback, limiting their abilities to determine the usefulness of these resources. GAO found that sub-agencies at HHS, USDA, and Education vary in following leading training practices for planning, designing, implementing, and evaluating their grants training programs. Additionally, HHS, USDA, and Education could not readily identify grants management specialists—the 1109 job series—or employees in other job series working on grants without querying each sub-agency. These agencies cannot do so because their central offices do not have a reporting mechanism tracking their sub-agencies' grants workforce. Further, agency central offices do not evaluate sub-agency grants training efforts. Without sufficient monitoring and oversight, the agencies cannot have reasonable assurance that their sub-agencies are sufficiently training their grants workforce so they have the necessary knowledge, skills, and abilities to properly manage, administer, and monitor the billions of dollars that the federal government spends on grants annually.