Federal Workforce: OPM and Agencies Need to Strengthen Efforts to Identify and Close Mission-Critical Skills Gaps
Released: 2015-01-30T00:00:00

Letter

Background

Mission-critical skills gaps within specific federal agencies as well as across the federal workforce pose a high risk to the nation because they impede the government from cost effectively serving the public and achieving results. We first designated strategic human capital management across the government as a high-risk issue in 2001 because of the federal government’s long-standing lack of a consistent approach to human capital management. In February 2011, we narrowed the focus of this high-risk issue to the need for agencies to close mission- critical skills gaps. At that time, we noted that agencies faced challenges effectively and efficiently meeting their missions across a number of areas, including acquisition management and foreign language capabilities.

Trends in federal workforce retirement threaten to aggravate the problem of skills gaps. As Figure 1 shows, 30 percent of all career permanent employees who were on board as of September 30, 2013, will be eligible to retire by 2018. Moreover, some agencies such as the Department of Housing and Urban Development and the Small Business Administration will have particularly high retirement eligibility rates by 2018.

Various factors affect when eligible individuals actually retire, which can be seen in the number of new retirement claims received by OPM each month. In October 2014, OPM was hit with a surge in new retirement claims—10,155, the highest number since February 2014. However, in the intervening months, there was considerable fluctuation in new claims. Some amount of retirement and other forms of attrition can be beneficial because it creates opportunities to bring fresh skills on board and allows organizations to restructure themselves to better meet program goals and fiscal realities. Nevertheless, if turnover is not strategically managed and monitored and succession plans are not in place, gaps can develop in an agency’s institutional knowledge and leadership as experienced employees retire.

In addition to a potential wave of employee retirements, other human- capital related risks are threatening the performance of federal agencies, including current budget and long-term fiscal pressures, declining levels of employee satisfaction, and the changing nature of federal work with an increasing number of positions requiring advanced degrees and other skills.

Effectively addressing mission-critical skills gaps requires a multi-faceted response from OPM and agencies. In February 2013, we noted that OPM and agencies would need to use a strategic approach that (1) involves top management, employees, and other stakeholders; (2) identifies the critical skills and competencies that will be needed to achieve current and future programmatic results; (3) develops strategies that are tailored to address skills gaps; (4) builds the internal capability needed to address administrative, training, and other requirements important to support workforce planning strategies; and (5) includes plans to monitor and evaluate progress toward closing skills gaps and meeting other human capital goals using a variety of appropriate metrics. We also noted that further progress in closing skills gaps will depend on, among other things, the extent to which OPM develops a predictive capacity to identify newly emerging skills gaps beyond those areas already identified.

Lessons Learned From Initial Efforts to Close Mission-Critical Skills Gaps Could Strengthen Future Approaches

Working Group’s Effort Yields Three Key Lessons for Identifying Skills Gaps

Based on its deliberations from September 2011 through March 2012, the CHCO Council Working Group (Working Group) identified skills gaps in six government-wide, mission-critical occupations (MCO): cybersecurity, auditor, human resources specialist, contract specialist, economist, and the STEM family. It also identified mission-critical competencies, including data analysis, strategic thinking, influencing and negotiating, problem solving, and grants management competencies. As part of these deliberations, individual agencies identified agency-specific MCOs and were responsible for designing strategies to close skills gaps in those occupations.

Although this effort was an important step forward, because of various methodological shortcomings, the Working Group did not address a more comprehensive list of mission-critical skills gaps. As we discuss later in this report, in 2015, OPM plans to identify a new set of MCOs. Going forward, it will be important for OPM and the CHCO Council to use lessons learned from its initial efforts to inform this next round of work. Specifically, the Working Group’s experience underscores the importance of (1) using a robust, data-driven approach to identify potential MCOs early in the process; (2) prioritizing occupations using criteria that consider programmatic impact; and (3) consulting with subject matter experts and other stakeholders prior to identifying MCOs.

Since 2011, our work has identified skills gaps in nearly two dozen occupations across the government. In some cases, such as cybersecurity, the gaps we identified were affecting multiple agencies and were consistent with the Working Group’s findings. But our work also identified additional skills gaps, both government-wide and agency specific, that were having a significant programmatic impact, such as:

Oil and Gas Management. In January 2014, we found that hiring and retention challenges at the Department of Interior (Interior) have resulted in fewer inspections of oil and gas facilities, which according to officials results in an increased risk to human health and safety due to a spill or accident. In 2012, Interior’s Bureau of Land Management had an attrition rate among petroleum engineers that, according to OPM data, is more than double the average federal attrition rate. Although Congress has provided Interior with the authority to establish higher minimum rates of basic pay for key inspection occupations, we noted in January 2014 that it was uncertain how Interior would address staffing shortfalls over time. Interior generally agreed with our recommendation that it should systematically collect data on hiring times for key oil and gas positions, ensure the accuracy of the data, analyze the data to identify the causes of delays, and expedite the hiring process. In response to our recommendation, Interior stated that its bureaus have begun a more systematic collection and analysis of hiring data to identify the causes of delays and to help expedite the hiring process.

Telecommunications. In December 2013, we found that a decline in telecommunication expertise across multiple agencies compounded the General Services Administration’s (GSA) challenges in transitioning those agencies to a new network of telecommunications services, contributing to delays and cost overruns of 44 percent. Moreover, according to GSA, customer agencies are concerned that the shortage of telecommunications specialists will get worse because there are not enough to replace experienced workers nearing retirement. GSA has yet to fully study the issue of addressing mission- critical skills gaps and agreed that understanding expertise shortfalls would be useful for future transition planning purposes. Officials from GSA and OPM agreed with our recommendation on the need to better examine potential government-wide telecommunications expertise shortfalls and have agreed to coordinate on efforts to do so. While this recommendation was still open at the time of our review, GSA’s Office of Human Resources Management plans to take several actions such as identifying and validating technical competencies, developing competency models, and performing a workforce assessment against the models.

A Data-Driven Approach Could Help OPM and Agencies Identify a More Complete Set of Mission-Critical Occupations

Working Group officials observed that a lesson learned from their MCO selection process was that they did not use workforce information and data analytics sufficiently early in the process. Instead, to identify an initial list of MCOs, the Working Group started with an environmental scan that consisted of our reports and academic studies. The Working Group used data analytics, such as the levels of attrition within an occupation, only after it had identified the initial set of MCOs. This approach only supported the selection of MCOs the Working Group already made and did not subject the full range of federal occupations to the same analytical criteria. As part of our development of our 2013 High-Risk report, we discussed this approach with the Working Group’s leadership, specifically questioning why the Working Group did not use workforce data as the starting point for its selection process.2013 discussion of lessons learned, Working Group officials concluded that the MCO selection process would have benefitted from analyzing staffing gap data and associated trends prior to identifying an initial set of MCOs. These officials determined that using these workforce data would have given the Working Group a better sense of which occupations had the biggest skills gaps.

Prioritization Criteria Should Consider Potential Programmatic Impact

In February 2012, to help prioritize their efforts, Working Group officials limited the scope of their work by creating criteria that at least half of the 24 CHCO Council agencies needed to report having a skills gap in that occupation, and that those agencies collectively needed to employ at least 95 percent of the occupation’s workforce. Working Group officials explained that limiting the number of MCOs using these criteria enabled them to focus on occupations found in most agencies. The officials also noted that it was important to establish a threshold for the number of agencies with a given skills gap so that the Working Group could focus resources on addressing skills gaps that had the greatest reach across the government. Officials added that skills gaps that existed at only a few agencies were the agencies’ responsibility to address.

While we recognize the importance of prioritizing MCOs to a manageable number, the Working Group’s approach overlooked skills gaps that may not have met the criteria but still had the potential for significant programmatic impact. For example, in August 2014, we found that, in part because of staffing shortages, the Department of Justice’s Bureau of Prisons was not activating new prison facilities in a timely manner, thereby aggravating the problem of prison overcrowding. Additionally, we found in December 2013 that the Department of Transportation’s Federal Railroad Administration lacked a plan to have sufficient safety inspectors to carry out oversight of such initiatives as positive train control—a communications system designed to prevent events like train- Indeed, Working Group officials later recognized that to-train collisions.establishing such a strict standard for a government-wide MCO resulted in eliminating from consideration other important occupations, such as nurses and financial analysts. Because the Working Group’s selection criteria created such a strict and narrow standard for government-wide skills gaps, its efforts failed to capture the broader range of skills gaps that were affecting agencies’ abilities to meet their missions. As we note later in this report, OPM’s proposed methodology for selecting a new set of MCOs does not include such a strict definition for government-wide skills gaps.

Early Stakeholder Input Can Help Improve the Identification of Skills Gaps

Our principles for effective strategic workforce planning note that agencies should involve stakeholders in developing and implementing future workforce strategies. However, the Working Group did not thoroughly consult occupational experts from other interagency councils or organizations prior to identifying MCOs. Instead, Working Group officials only considered consulting relevant stakeholders to discuss strategies for addressing skills gaps at a March 2012 meeting—after the Working Group had already identified preliminary MCOs for selection. As a result, stakeholders could only provide advice on decisions already made rather than help with the initial screening of MCOs. As one example, members of the Council of Inspectors General on Integrity and Efficiency—an Executive Branch interagency council devoted, in part, to providing training for federal auditors—told us that they had not been consulted about the existence of skills gaps in the auditor occupation and were initially unsure why the auditor occupation was designated by the Working Group as an MCO for skills gap closure. A Working Group official indicated at a January 2012 meeting that the Working Group had sufficient subject matter experts internally, and verification of MCO selection with outside experts could lead to competing lists of MCOs and could expend significant resources. As a result of this limited outreach, however, the Working Group missed an opportunity to leverage the expertise of key outside stakeholders to get a more complete assessment of the need to close skills gaps. As we describe later in this report, OPM has recognized this deficiency and is proposing a methodology for a new interagency working group that incorporates more input from subject matter experts when identifying emerging skills gaps.

Efforts to Address Skills Gaps Are Underway with Mixed Results

OPM Provided Resources and Visibility to Efforts to Close Skills Gaps

OPM and the Working Group have made important progress in creating an infrastructure to address the six mission-critical skill gaps they identified. The GPRA Modernization Act of 2010 (GPRAMA) requires OMB to coordinate with agencies to develop cross-agency priority (CAP) goals to improve the performance and management of the federal government in several areas, including human capital management.The fiscal year 2013 budget—released in February 2012—designated OPM as leader of a two-year interim CAP goal to close skills gaps by 50 percent in three-to-five MCOs by September 30, 2013. As goal leader, the Director of OPM appointed key federal officials from each of the six government-wide mission-critical occupations to serve as sub-goal leaders for each of the six MCOs identified by the Working Group. At the time of our review, the sub-goal co-leaders for the cybersecurity workforce, for example, were the Assistant Director for Cybersecurity at the White House Office of Science and Technology Policy and the Lead for the National Initiative for Cybersecurity Education within the Department of Commerce’s National Institute of Standards and Technology. Likewise, the sub-goal leader for the economist workforce is the Assistant Secretary for Economic Policy and Chief Economist at the U.S. Department of the Treasury.

In working within their occupational communities, the sub-goal leaders selected specific strategies to decrease skills gaps in the occupational communities they represent. Multiple sub-goal leaders indicated that the quarterly meetings and availability of OPM staff and their human resources expertise provided visibility and resources to sub-goal leaders’ efforts to close skills gaps in their MCOs. For example, Auditor sub-goal leaders told us that the OPM Director assisted in conducting a study of government-wide recruitment and hiring of auditors. In partnership with OPM and stakeholders from the federal audit community, Auditor sub- goal leaders found that the current qualifications for federal auditors do not align with the nature of federal audit work. To address this challenge, OPM is studying how to change the qualifications for federal auditor positions to improve agencies’ experiences with recruiting and hiring qualified candidates for that occupation.

The meetings and performance reviews by OPM and sub-goal groups demonstrate high-level leadership commitment to address mission-critical skills gaps. We have maintained that removing skills gaps as a high-risk issue across the government will depend in part on the extent to which OPM and agencies involve top management and include plans to monitor and evaluate progress toward closing skills gaps.year 2015 budget, released in March 2014, includes a four-year human capital management CAP goal to (1) create a culture of excellence and engagement to enable higher performance, (2) build a world-class federal management team starting with the Senior Executive Service, and (3) enable agencies to hire the best talent from all segments of society. While these CAP goal elements contain workforce planning strategies and metrics relevant to closing skills gaps, there are no overall performance targets for closing skills gaps, and closing skills gaps is no longer an explicit goal.

The President’s fiscal Throughout the time period when closing skills gaps was an explicit CAP goal, top officials at OPM held the sub-goal groups accountable for making progress by holding quarterly performance review meetings. OPM also used these meetings to discuss challenges the sub-goal leaders were facing and to field requests from sub-goal leaders for how they could better facilitate sub-goal groups’ efforts. Now that the fiscal year 2013 CAP goal expired and was replaced with a fiscal year 2015 human capital management CAP goal that did not have explicit targets for closing skills gaps, OPM officials have told us that they intend to continue meeting with current sub-goal leaders through a community of practice. OPM has also worked with agencies to develop occupation-specific communities of practice within the science, technology, engineering, and mathematics (STEM) and Economist sub-goal groups. In addition, OPM has indicated that it intends to hold quarterly performance review sessions with officials in charge of efforts to close skills gaps in emerging government-wide occupations in a similar format to what was done during the prior CAP goal.

These are all important steps in the right direction and highlight OPM’s commitment to addressing skills gaps going forward. Still, the CAP goal to address skills gaps gave the entire effort government-wide focus and visibility, and provided OPM a mechanism to hold agencies accountable for results. It will be important for OPM to continue its leadership on this high-risk issue by holding occupational leaders accountable for implementing their strategies to close skills gaps and sustaining the visibility of the issue among agency officials across the government. Key focus areas include using (1) better defined, more measurable goals, (2) outcome-oriented performance metrics that align with activities to close skills gaps, and (3) key practices for project planning.

Use of Better Defined, Measurable Goals Enhances Monitoring on Closure of Skills Gaps

The CAP goal target to close skills gaps was vague and difficult to measure. The fiscal year 2013 human capital CAP goal’s target was to close skills gaps by 50 percent by the end of fiscal year 2013 in three-to- five of the six MCOs for which OPM appointed sub-goal leaders and groups of officials from across the government. Because the CAP goal target was difficult to measure, however, there is no clear basis for determining whether the CAP goal was met.

Our prior work on performance goals has indicated that setting ambitious but realistic goals is one of the factors that can influence whether goal- setting and performance measurement efforts will be successful. However, sub-goal leaders and OPM officials leading the effort stated that the 50 percent skills gap closure target was difficult to measure. Specifically:

An OPM official told us that the 50 percent target was not set appropriately and that, rather than have a single, overarching performance target, sub-goal groups needed individualized targets to allow for greater measurability.

Similarly, the STEM sub-goal co-leader told us that the CAP goal target was not designed in a way that could facilitate a sophisticated measurement of progress by the various sub-goal groups.

The Cybersecurity sub-goal leader indicated that the group’s efforts were at such an early stage of maturity that it could not gauge a 50 percent skills gap closure because it had not fully determined the nature of its skills gap. The Cybersecurity sub-goal group, therefore, did not have an effective baseline from which to measure 50 percent progress.

Align Performance Metrics with Targets and Activities for Closing Skills Gaps

Of the six sub-goal groups (Acquisition, Cybersecurity, Economist, Human Resources, STEM, and Auditor), three—the Cybersecurity, Economist, and Human Resources sub-goal groups—used performance metrics that either did not align with outcomes for closing skills gaps or did not represent their activities for closing skills gaps (the STEM and Auditor sub-goal groups did not establish metrics at all). Our prior work has outlined the importance of developing outcome-oriented performance metrics that clearly and sufficiently relate to the performance they are meant to assess. Our prior work has also discussed the importance of using performance metrics to link agencies’ goals and priorities with the actions that they are taking so that leaders can hold their organizations accountable for progress.

For example, the Acquisition sub-goal group aligned its metrics with activities and overall targets. Sub-goal leaders indicated that they focused their efforts to close skills gaps on maintaining federal acquisition competencies. To do so, the sub-goal group set a performance metric of increasing the certification rate among civilian contract specialists and produced guidance on revising agencies’ certification curricula. Sub-goal leaders indicated that increasing the certification rate among civilian contract specialists by 10 percentage points—from 75 percent to 85 percent—was adequate for the occupation’s long-term needs. To make 50 percent progress toward this target by the end of fiscal year 2013, they worked toward a metric of an 80 percent certification rate. The Acquisition sub-goal group’s efforts resulted in a 6 percentage point increase in the civilian contract specialist certification rate—to 81 percent—which surpassed its CAP goal metric and was aligned with such actions as producing guidance on reforming agencies’ certification curricula.

In contrast, the Cybersecurity sub-goal group is an example of a sub-goal group that did not fully align its performance metric with the sub-goal group’s activities. For instance, officials began an initiative in 2013 to categorize the cybersecurity specialty areas for the various occupations involved in federal cybersecurity work. This initiative, which operated through the end of fiscal year 2014, will allow managers to better assess what cybersecurity work needs to be done at agencies, and where there are gaps preventing agencies from accomplishing their cybersecurity missions. The Cybersecurity sub-goal leader noted that using the updated classifications to identify skills gaps is necessary before those skills gaps can be addressed. Officials did not, however, include a publicly-reported metric in fiscal year 2013 to track how many of the cybersecurity-related classifications had been updated.

Moreover, the Human Resources sub-goal group is an example of a sub- goal group that did not align its metric with the outcome of closing skills gaps. The Human Resources sub-goal group tracked the percentage of federal human resources personnel who registered for and completed a single course on HR University—a centralized online suite of courses and curricula managed by OPM that agencies can use for training purposes. OPM officials said that they tracked these indicators because they perceived the need to standardize human resources training curricula across agencies as a first step to closing the skills gap. OPM officials told us, however, that they are still striving to develop more outcome-oriented measures from the use of HR University. We agree that while ensuring that human resources professionals receive proper training is vital, relying on a metric of how many people register for and complete a single online course is not the most effective way to assess the outcome of closing skill gaps within the human resource occupation. OPM officials also noted that a 2014 data clean-up of HR University detected erroneous information that made the fiscal year 2013 registration and course completion rates unreliable as reported. OPM has continued to use the course registration performance metric as part of its fiscal year 2015 Agency Priority Goal to close skills gaps in the federal HR workforce. OPM officials told us that data flaws on HR University have been corrected. However, because this metric does not measure an outcome, it does not provide quality information toward goals to close skills gaps.

Key Project Planning Practices Could Improve Efforts to Address Skills Gaps

Sub-goal groups’ planning documents for efforts to address skills gaps in fiscal year 2014 generally met three of six key practices for project planning. As we have noted, a well-developed and documented project plan can help ensure that agencies are able to gauge progress, identify and resolve potential problems, and promote accountability at all levels of the project. Project plans can also encourage agency managers and stakeholders to systematically consider what is to be done, when and how it will be done, and what skills will be needed. We identified the following six key practices from our prior work: (1) identifying root causes of issues, (2) establishing objectives, (3) developing specific actions, (4) assigning roles and responsibilities, (5) establishing the duration of actions, and (6) using outcome-oriented performance metrics.

Table 1 lists each of the sub-goal groups and our evaluation of their planning documents against each of the key practices for project planning. For plans that included some information relevant to a key practice but did not provide sufficient detail nor fully addressed the key practice, we gave partial credit.

Some key practices were used consistently by sub-goal groups. As shown above, sub-goal groups’ plans generally met some of our key practices. For instance, all of the sub-goal groups structured their plans by listing specific actions for addressing skills gaps. Our prior work states that it is important to identify the specific actions necessary to achieve a plan’s objectives. By doing so, managers can properly assess the risk of not achieving the plan’s objectives.Acquisition sub-goal group listed a clear division of actions—such as publishing a revised certification curriculum and increasing the government-wide use of an acquisition workforce management system— that addressed objectives that included strengthening civilian contracting certification standards.

For example, the plan for the Some key practices were generally absent from sub-goal group project planning. Sub-goal groups’ plans were less effective in meeting other key practices for project planning. For instance, only the STEM and Auditor sub-goal groups’ plans discussed the root causes of their skills gaps and the purpose for their actions.

Additionally, sub-goal groups did not consistently identify roles and responsibilities for actions to address skills gaps. While all sub-goal groups listed agencies that would be responsible for accomplishing actions, only the Auditor sub-goal group identified individual officials charged with specific actions.

Three of six sub-goal groups did not always track outcome-oriented performance metrics in their plans. For instance, the STEM sub-goal group’s plan tracked such items as the number of STEM hiring reforms that had been approved by OPM. While gaining OPM approval of hiring- policy changes is important in attracting more qualified workforce candidates, the STEM sub-goal group’s plan did not track outcomes that might result from approving such policy changes, such as the number and quality of applicants and hires for STEM positions.

OPM’s Efforts to Predict Emerging Mission-Critical Skills Gaps Are in the Early Planning Stages

OPM Plans to Strengthen the Methodology Used to Identify Emerging Skills Gaps

The interagency working group that identified the list of skills gaps in six government-wide mission-critical occupations (MCO) was re-named the Federal Agency Skills Team (FAST). OPM has tasked the group— composed of agency officials with workforce planning and data analysis skills—with implementing a standard and repeatable methodology that identifies and addresses government-wide skills gaps as well as mission- critical competencies over a 4-year cycle. In the first year, OPM officials stated that FAST intends to meet regularly until it identifies a new set of government-wide skills gaps, which OPM officials expect will occur by June 2015. Our analysis of FAST’s proposed methodology identified three features that incorporate lessons learned from the Working Group’s initial efforts, which were described earlier. First, FAST is to use a data- driven approach as the initial step for identifying a broad list of skills gaps. Specifically, officials from the 27 Chief Human Capital Officers (CHCO) Council agencies will be expected to compare their agency’s mission- critical or priority occupations against several factors. According to OPM officials, those factors include:

2-year retention rate, quit rate, retirement rate, and applicant quality.

Second, in addition to being more data-driven, FAST will use a multi- faceted approach to identify a select number of government-wide MCOs. This is to be achieved by using teams of at least three to four individuals to monitor internal and external environments using various sources— including news articles, reports, and interviews with public and private sector subject matter experts—to detect trends that may influence an agency’s current and future workforce needs.

Third, our analysis determined that the proposed methodology does not have a strict numerical standard for what constitutes a government-wide skills gap. This could allow for more discretion to address occupations with the potential for significant programmatic impacts even though an absolute majority of agencies may not have skills gaps in those areas.

After this identification process, FAST is to develop strategies to address the new set of skills gaps during the remaining years of the 4-year cycle. In years two and three, OPM—in conjunction with FAST—is to designate leaders from within the selected government-wide occupations who will develop and implement plans to address those skills gaps. Finally, in year four, FAST plans to evaluate and monitor outcomes to determine the effectiveness of those strategies. In addition, during the fourth year FAST plans to incorporate lessons learned into a revised process for identifying skills gaps at the federal level.

OPM’s Plan to Capture Staffing Data Using Its Database Lacks a Schedule

OPM’s plan to capture staffing data for select occupations using a database is still under development. Each year, OPM collects staffing data through a reporting process where the CHCO Council agencies are required to provide OPM with information—such as the target number of employees and projected attrition—for five government-wide MCOs (cybersecurity, human resources specialist, economist, contract specialist, auditor), and three-to-five agency-specific or STEM MCOs. Among other things, OPM uses these data to determine if an agency’s workforce has a staffing gap in any of those occupations.

In response to an administration initiative directed, in part, to reduce the human resources reporting burden on agencies, OPM is exploring how it can replace this annual reporting process by using its database containing federal workforce information—known as the Enterprise Human Resources Integration (EHRI)—to capture the same agency staffing data. However, an OPM official stated that no timeframe currently exists for using EHRI to capture agencies’ MCO staffing data. In the interim, an OPM official stated that they will continue collecting MCO staffing data until OPM can make the necessary investments in EHRI. Our schedule assessment guide has noted that a well-planned schedule is a fundamental management tool that can (among other things) help government agencies specify when work will be performed in the future.Moreover, our schedule assessment guide states that a consistent methodology for developing, managing, and evaluating cost estimates for certain types of programs includes the concept of scheduling the necessary work to a timeline. By establishing a schedule specifying when EHRI will be modified to capture government-wide staffing data, OPM officials will have a road map for gauging progress, identifying and resolving potential problems, promoting accountability at all levels of the agency, and determining the amount and timing of the funding needed.

OPM Lacks a Process for Collecting Government- wide Competency Data

According to an OPM official, there is no process, using EHRI or another system, for collecting consistent data on the competencies of the federal workforce, which is needed to effectively predict future mission-critical skills gaps. According to an OPM official, federal agencies’ ability to assess workforce competencies varies, which makes collection of government-wide data on competency gaps difficult. As one example, we have found that the Census Bureau has started to assess the competencies needed to carry out its future work. In contrast, Department of Commerce (Commerce) human capital officials stated that they do not conduct a Department-wide competency assessment of Commerce’s workforce. Moreover, the Department of Energy (DOE) has conducted a competency assessment for a number of engineering occupations such as nuclear engineers. DOE officials have also developed a model for conducting a Department-wide competency assessment. Key principles for effective workforce planning that we developed note that agencies must determine the competencies that are critical to successfully achieving their mission and goals. Doing so can help agencies to effectively meet demographic, technological, and other forces that are challenging government agencies to change the activities they perform and the goals that they must achieve, how they do their business, and even who does the government’s business. Therefore, as OPM develops its process for using EHRI to collect agencies’ skills gaps data, it will be important for OPM to also work with agency CHCOs to bolster the ability of agencies to assess workforce competencies by sharing competency surveys, lessons learned, and other tools and resources— and to ensure that such information can be stored in the EHRI database for government-wide workforce analysis.

OPM and Selected Agencies Could Improve Efforts to Address Skills Gaps by Strengthening Data-Driven Reviews

OPM Developed HRstat Data-Driven Reviews to Help Agencies Regularly Track Their Progress in Achieving Their Human Resources Goals

Data-driven reviews—commonly referred to as “stat” meetings—are regularly scheduled, structured meetings used by organizations to review performance metrics with department or program personnel to drive progress on agency priorities and goals. Conducting frequent stat meetings is a leadership strategy proven to help agency officials achieve results by focusing on an identified set of priorities, diagnosing problems, and deciding on the next steps to increase performance. The GPRA Modernization Act of 2010 required agencies to conduct data-driven quarterly progress reviews with key personnel responsible for the accomplishment of agency priority goals.

Building on this statutory model, OPM created HRstat, which is a CHCO- led review of the key metrics that contribute to agencies’ human resources goals, such as closing mission-critical skills gaps. launched HRstat as a 3-year pilot program in May 2012, with an initial group of eight agencies that included Commerce, DOE, and the U.S. Agency for International Development (USAID)—the three agencies we selected for our illustrative case studies. In 2013 and 2014, OPM chose eight agencies per year for successive pilots. As a result, all 24 agencies subject to the Chief Financial Officers Act of 1990, as amended, had at least one year of HRstat implementation by the end of 2014.

According to OPM guidance, agency Performance Improvement Officers and Chief Operating Officers should support HRstat reviews. assess them. The three agencies we selected all have closing critical skills gaps as one of their agency-wide human resource goals.

Selected Agencies Are Using Different Metrics to Track Skills Gaps under Their HRstat Reviews

Based on our assessment of the HRstat reviews of our selected agencies (using agency material from the third quarter of fiscal year 2014), we identified those metrics most relevant to tracking progress on closing skills gaps. As illustrated in table 2 below, we found considerable variation in the number and types of metrics agencies were using to track skills gaps.

As shown above, only DOE was tracking retirement eligibility and projected retirements, which are key indicators of where agencies might be at risk for future skills gaps. Likewise, only Commerce was tracking candidate quality, which provides an indication of whether applicants had the skills needed to perform the work. While some amount of variation is both desirable and expected given the different missions of the agencies and the flexibility OPM gave agencies in selecting metrics, the variation shown above has at least two significant downsides. First, the variation in number and types of metrics agencies are tracking suggest that some agencies’ HRstat reviews are more robust and well rounded than others in that they are measuring factors affecting skills gaps from a more complete perspective. Indeed, each metric may tell a different story about the extent and nature of current and emerging skills gaps. This could lead to different remedial actions on the part of agencies.

A second downside in the variation in numbers and types of metrics is that it limits OPM and the CHCO Council’s ability to track agencies’ progress in closing skills gaps government-wide. Our February 2013 report noted that a leading practice for successful data-driven reviews, such as HRstat, is to ensure alignment between goals, program activities, However, because our selected agencies are using and resources.different metrics, it is difficult for OPM to assess where agencies are making progress, where additional efforts are needed, and how OPM might be able to help them, if at all. It also limits the ability of agencies and OPM to discuss and share lessons learned in identifying and addressing skills gaps.

OPM officials stated that they knew of the various metrics used by agencies during their HRstat reviews. They noted that the HRstat pilot program was intended to be “agency-centric” and not a data-collection tool for OPM. While we agree that agencies should continue to have flexibility in choosing and “owning” metrics for their HRstat reviews that best meet their particular needs, a core set of metrics—while still allowing agencies discretion to include metrics that meet their specific requirements—could help strengthen the quality and consistency of the HRstat reviews from a government-wide perspective.

Moreover, a core set of metrics could foster collaboration between agencies’ use of HRstat reviews and the efforts envisioned under FAST. As we noted earlier, FAST is to use a data-driven approach as an initial step for identifying a range of skills gaps, such as a survey of hiring managers’ satisfaction with job applicants’ skills. As shown above, Commerce already tracks this same survey of hiring managers’ satisfaction during its HRstat reviews. A core set of metrics could integrate the work done by FAST with HRstat reviews.

Going forward, OPM, in conjunction with the CHCO Council, plans to identify key strategic and operational HR metrics all agencies will collect through HRstat and share with OPM. As part of that effort, it will be important for OPM and the CHCO Council to develop a core set of valid metrics that are directly aligned with the goal of identifying and addressing agency skills gaps.

Conclusions

Closing workforce skills gaps is critical for agencies to better achieve a wide range of missions, from purchasing mission-critical goods and services to carrying out the Decennial Census. While efforts to close mission-critical skills gaps are couched in discussions about staffing numbers, competencies, metrics, and similar technical terms, the ultimate goal is higher-performing, cost-effective government. However, the challenges that agencies face are diverse and were not fully captured by the CHCO Council Working Group’s first efforts to identify skills gaps in government-wide, mission-critical occupations. Although these initial efforts created an infrastructure for addressing skills gaps, to date, overall progress remains mixed. At times, goals have suffered from having targets that are difficult to measure. Likewise, agency officials have chosen to track metrics that often do not allow for an accurate assessment of progress made toward these goals for closing skills gaps.

Building the predictive capacity to identify emerging mission-critical skills gaps is also critical to making further progress in addressing this issue. Realizing this, OPM has established an interagency working group known as FAST, which is responsible for identifying and addressing current and emerging skills gaps. OPM also intends to replace its annual reporting process for collecting agency staffing data by modifying its workforce database to capture the same data. These are important steps forward. However, we are concerned about these efforts for two reasons. First, OPM has not established a time frame for modifying its workforce database to capture the same agency staffing data that it currently collects through an annual reporting process—which will reduce the human capital reporting burden on federal agencies. Second, OPM officials stated that there is no process for collecting data on the competencies of the federal workforce because agencies’ ability to assess workforce competencies varies. Helping agencies determine the competencies that are critical to successfully achieving their mission and goals will help them respond to external factors, such as changes in national security, technology, or budget constraints.

At the agency level, the use of HRstat meetings is a proven leadership strategy that could help agency officials monitor their progress toward closing skills gaps. However, OPM should take a greater leadership role in helping agencies include a core set of metrics in their HRstat reviews so that OPM and agency leaders can have a clear view of progress made closing skills gaps. While it is important for agencies to have ownership over their HRstat reviews, OPM should also maximize its opportunity to use HRstat to gain greater visibility over the federal workforce.

Recommendations for Executive Action

To assist the interagency working group, known as FAST, to better identify government-wide skills gaps having programmatic impacts and measure its progress towards closing them, we recommend that the Director of OPM—in conjunction with the CHCO Council—strengthen its approach and methodology through the following actions:

Assist FAST in developing goals for closing skills gaps with targets that are both clear and measurable.

Work with FAST to design outcome-oriented performance metrics that align with overall targets for closing skills gaps and link to the activities for addressing skills gaps. Incorporate greater input from subject matter experts, as planned.

Ensure FAST consistently follows key practices for project planning.

To ensure that OPM builds the predictive capacity to identify emerging skills gaps across the government—including the ability to collect and use reliable information on the competencies of the federal workforce for government-wide workforce analysis—we recommend that the Director of OPM take the following two actions:

Establish a schedule specifying when OPM will modify its EHRI database to capture staffing data that it currently collects from agencies through its annual workforce data reporting process.

Work with agency CHCOs to bolster the ability of agencies to assess workforce competencies by sharing competency surveys, lessons learned, and other tools and resources.

To help agencies and OPM better monitor progress toward closing skills gaps within agencies and government-wide, we recommend that the Director of OPM:

Work with the CHCO Council to develop a core set of metrics that all agencies should use as part of their HRstat data-driven reviews.

Coordinate with FAST personnel and explore the feasibility of collecting information needed by FAST as part of agencies’ HRstat reviews.

Agency Comments and Our Evaluation

We provided a draft of this report for review and comment to the Directors of OPM and OMB; the Secretaries of the Departments of Defense, Energy, and Treasury; the Administrators of the National Aeronautics and Space Administration (NASA) and USAID; the Chief Financial Officer and Assistant Secretary for Administration at the Department of Commerce; and the Special Assistant to the President and Associate Counsel to the President in the Executive Office of the President.

The following agencies had no comments on the draft report: the Departments of Defense, Energy, and Treasury, NASA, OMB, USAID, and the Office of Science and Technology Policy in the Executive Office of the President; as well as Commerce’s Bureau of the Census and National Institute of Standards and Technology.

OPM and Commerce’s Office of Human Resources Management provided technical comments that were incorporated, as appropriate.

In its written comments, reproduced in appendix II, OPM partially concurred with one recommendation, did not concur with one recommendation, and concurred with one recommendation. Specifically, OPM stated that it partially concurred with our recommendation to strengthen the approach and methodology used by the interagency working group, known as FAST, to better identify skills gaps. OPM noted it agreed with, and planned to implement, the principles of each recommended action. However, OPM said it needed to clarify how its terminology and planned process differs from the description in our recommendation. In particular, OPM stated its process will identify government-wide rather than agency-specific skills gaps as it believes our draft recommendation suggests. We recognize that FAST was established to address government-wide skills gaps and have clarified the language in our recommendation accordingly.

OPM stated that it does not concur with our recommendation to: (1) establish a schedule specifying when it will modify its EHRI database to automatically capture staffing data that it currently collects from agencies through its annual workforce data reporting process, and (2) work with agency CHCOs to bolster agencies’ ability to assess workforce competencies by sharing competency surveys, lessons learned, and other tools and resources. Regarding EHRI, OPM maintained that it is impossible for the EHRI database to automatically capture staffing data currently included in MCO Resource Charts because some of these data includes specific agency projections and targets, which are provided via a manual data feed. OPM stated that it is assessing whether EHRI can be modified to allow agencies to supply these manual feed data into the database system. We have modified our report to recognize that EHRI cannot automatically capture the same agency staffing data that are captured through the MCO Resource Charts.

In addition, OPM noted that there are funding implications associated with its ability to anticipate whether and when a modification schedule to the EHRI online database could be established. While we appreciate OPM’s funding concerns, as we mention in the report, a well-planned schedule is a fundamental management tool that can (among other things) help government agencies specify when work will be performed in the future. Moreover, scheduling the necessary work to a timeline is important for developing, managing, and evaluating cost estimates for certain types of programs. As such, a schedule, as we recommend, would help OPM determine the amount and timing of the funding needed, and help OPM identify the competing priorities that need to be balanced due to resource constraints. We therefore continue to believe OPM would benefit by implementing our recommendation.

Regarding workforce competencies, OPM noted that funding and resource constraints negatively affect its ability to support agencies’ efforts to address their workforce competencies. While funding limitations could affect OPM’s ability to take these actions, our recommendation would help OPM and agencies stretch resources by leveraging their knowledge and experience. We therefore continue to believe OPM would benefit by implementing our recommendation.

OPM concurred with our recommendation to work with the CHCO Council to develop a core set of metrics that all agencies should use as part of their HRstat data-driven reviews, and explore the feasibility of collecting information needed by FAST as part of agencies’ HRstat reviews.

We are sending copies of this report to the appropriate congressional committees, the Director of OPM, the Secretaries of Commerce and Energy, the Administrator of USAID, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

This report assesses (1) lessons learned from initial efforts to close critical skills gaps and how they can inform future initiatives, (2) what progress the Office of Personnel Management (OPM) has made in building a predictive capacity to identify future mission-critical skills gaps, and (3) how OPM and agencies are using HRstat to identify and close skills gaps.

To assess lessons learned from initial efforts to close critical skills gaps and how they can inform future initiatives, we took the following steps.

We observed the Chief Human Capital Officer (CHCO) Council skills gaps working group’s (Working Group) meetings from July 2011 to August 2012 to understand the process it used to identify government-wide skills gaps in mission-critical occupations (MCO).

We reviewed documentation, such as status reports, that described the Working Group’s activities. Furthermore, we reviewed our reports from October 2010 to May 2014 to identify skills gaps that we determined were causing programmatic impacts at agencies across the federal government. To identify our reports that identified skills gaps at agencies across the federal government, we searched through our Engagement Reporting System, our external database, and our Engagement Results Phase internal database. Reports were selected if they included references to skill gaps or related workforce conditions that might produce skill gaps (workforce shortages, competency gaps, training deficits) that were primary objective findings issued since the start of fiscal year 2011, updates on skill gaps identified since the beginning of fiscal year 2011, or updates on skill gaps identified in our 2011 and 2013 High Risk Series reports. Reports were not selected if they found skills gaps at agencies prior to fiscal year 2011, or if they referenced skills gaps that were not part of the engagement’s findings. Private sector skills gaps were also excluded because the scope of this engagement was limited to OPM’s and agencies’ progress closing skills gaps across the federal government.

We reviewed planning documents for addressing skills gaps in the six MCOs identified by the Working Group for such things as their consistency with key practices for project planning. We identified project planning key practices from our prior work assessing project planning, our guide for key practices in project schedules, and our criteria for removing issues from our high-risk list. In particular, we assessed whether each planning document included the following elements relevant to closing skills gaps: (1) identifying root causes, (2) establishing plan objectives, (3) developing specific actions needed to accomplish objectives, (4) assigning roles and responsibilities to all actions, (5) establishing the durations of actions, and (6) using outcome-oriented performance metrics. Two analysts independently assessed each planning document to determine the extent to which information that was included met all six key practices and rated each plan using a three-level scale of yes, partially, or no, and reached a level of inter-rater agreement of 100 percent. For plans that included some information relevant to a key practice but did not provide sufficient detail or did not fully address the key practice, we gave partial credit.

We reviewed documentation such as quarterly status updates to determine what progress had been made toward the interim cross- agency priority (CAP) goal for closing skills gaps. We also interviewed OPM and agency officials who are responsible for designing strategies to close skills gaps that OPM, in coordination with, the Office of Management and Budget, designated as the focus of the CAP goal.

To assess what progress OPM has made in building a predictive capacity to identify future government-wide mission-critical skills gaps, we reviewed documentation, such as OPM strategic plans and meeting minutes from the CHCO Council Executive Steering Committee, containing information about ongoing OPM initiatives that could build the predictive capacity within OPM to identify future mission-critical skills gaps. We also interviewed OPM officials who were implementing these initiatives to learn more about their status.

To assess how OPM and agencies are using HRstat to identify and close skills gaps, we selected a nongeneralizable sample of three agencies— the Departments of Commerce and Energy, and the U.S. Agency for International Development—from among the 24 agencies that had at least begun implementing HRstat at the time of our review. We selected these agencies because they had one or more of the following: (1) multiple skills gaps that we and the Working Group identified; (2) skills gaps in any of the four occupations that we or the Working Group most frequently identified (STEM (science, technology, engineering, and mathematics), acquisition, human resources, and cybersecurity); and (3) large proportions of their workforces in each of those occupations. Also, the selected agencies were among the first to implement HRstat and therefore had the most experience with the HRstat process at the time of our review. To assess how our selected agencies were using HRstat to address skills gaps, we also reviewed memorandums, internal briefings, and other material that agencies used to prepare for the reviews, and interviewed officials involved in each agency’s HRstat review process.

We conducted this performance audit from March 2014 to January 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Office of Personnel Management

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Steven Lozano, Assistant Director; Don Kiggins, Analyst-in-Charge; Devin Braun; Deirdre Duffy; Karin Fangman; Donna Miller; and Rebecca Shea made major contributions to this report.