Federal Workforce: OPM and Agencies Need to Strengthen Efforts to Identify and Close Mission-Critical Skills Gaps
Published: 2015-01-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Mission-critical skills gaps both within federal agencies and across the federal workforce pose a high risk to the nation because they impede the government from cost-effectively serving the public and achieving results.
GAO was asked to review progress OPM has made in closing government-wide skills gaps, achieving its cross-agency priority goal, and additional steps needed to better identify and address skills gaps. This report assesses (1) lessons learned from initial efforts to close critical skills gaps and how they can inform future initiatives, (2) what progress OPM has made in building a predictive capacity to identify future mission-critical skills gaps, and (3) how OPM and agencies are using HRstat to identify and close skills gaps. To address these objectives, GAO reviewed documentation; interviewed OPM officials; and reviewed the implementation of HRstat meetings at Commerce, DOE, and USAID.
section_title: What GAO Found
paragraphs: Lessons learned from initial efforts to try to close skills gaps could strengthen future approaches. For example, the Chief Human Capital Officer (CHCO) Council Working Group (Working Group) identified skills gaps in six government-wide occupations, such as cybersecurity and auditors. Although this effort was an important step forward, GAO's work has identified skills gaps in nearly two dozen occupations with significant programmatic impact. In some cases, such as cybersecurity, the skills gaps GAO identified were consistent with the Working Group's findings. But GAO's work has also identified additional skills gaps. For example, a decline in telecommunication expertise at multiple agencies contributed to delays and cost overruns of 44 percent when those agencies were transitioning to a new network of telecommunications services. The Working Group did not address a more comprehensive list of skills gaps because of various methodological shortcomings that included insufficient analysis of workforce data early in the process. In 2015, the Office of Personnel Management (OPM) and the CHCO Council plan to identify and address a new set of government-wide skills gaps. It will be important that key lessons learned from the initial efforts to identify skills gaps inform this next round of work, including the need to (1) use a data-driven approach early in the process, (2) prioritize occupations using criteria that consider programmatic impact, and (3) consult with subject matter experts and other stakeholders prior to the identification of skills gaps in occupations.
Key features of OPM's efforts to predict emerging skills gaps are in the early planning stages. GAO has previously reported that further progress in closing skills gaps will depend on, among other things, the extent to which OPM develops a capacity to predict emerging skills gaps beyond those areas already identified. A re-named interagency group, known as the Federal Agency Skills Team, plans to strengthen the methodology used to identify emerging skills gaps. Additionally, OPM officials are discussing plans to modify OPM's workforce database to capture government-wide staffing data. However, OPM will need to establish a schedule for modifying this database to ensure its implementation. OPM officials also stated that because agencies' capacity to assess workforce competencies varies, OPM does not have government-wide data on competency gaps, which is needed to identify emerging cross-agency skills gaps. In conjunction with agencies' CHCOs, OPM will need to strengthen agencies' ability to assess their competency needs that are critical to successfully achieving their mission and goals.
OPM and selected agencies that GAO reviewed—the Departments of Commerce (Commerce) and Energy (DOE), and the U.S. Agency for International Development (USAID)—could improve efforts to address skills gaps by strengthening their use of quarterly data-driven reviews, known as HRstat meetings. Specifically, the metrics used by the selected agencies during their HRstat meetings vary from agency to agency, making it difficult for OPM to assess agencies' progress in closing skills gaps government-wide. Although it is important for agencies to have their own HRstat metrics, OPM should work with the CHCO Council to develop a core set of HRstat metrics that all agencies use so that OPM may have the ability to analyze skills gap data across the government.
section_title: What GAO Recommends
paragraphs: GAO recommends that OPM (1) strengthen its methodology for identifying and addressing skills gaps, (2) establish a schedule and process for collecting government-wide staffing and competency data, and (3) develop a core set of metrics for use in agencies' HRstat reviews. OPM generally concurred with the first and third recommendations but did not concur with the second recommendation because of funding implications. GAO acknowledges there may be funding constraints; however, GAO's recommendation may help OPM address these constraints.

section_title: Letter
paragraphs: Skills gaps within the federal workforce can significantly impact agencies’  ability to serve the public and achieve their missions. For example, a lack  of seasoned acquisition staff—whose responsibilities include procuring  services, evaluating price proposals, and administering contracts—may  contribute to significant delays in purchasing mission-critical goods and  services, such as medical supplies, engineering support, and program  management. Likewise, as we noted in our earlier work, an information  technology (IT) workforce without the appropriate competencies could  impede the development of the multimillion-dollar information systems  needed to conduct federal IT projects.
An agency’s skills gap can manifest itself in a variety of ways. For the  purposes of this report, a skills gap may consist of one or more of the  following: a (1) “staffing gap,” in which an agency has an insufficient  number of individuals to complete its work, such as a lack of contracting  officers within its workforce; and/or a (2) “competency gap,” in which an  agency has individuals without the appropriate skills, abilities, or  behaviors to successfully perform the work, such as an information  technology workforce without up-to-date cybersecurity skills.
Because skills gaps within individual federal agencies—as well as across  the federal workforce—can lead to costly, less-efficient government, we  designated addressing agencies’ mission-critical skills gaps as a high-risk  area in February 2011. capital management—a GAO high-risk issue since 2001—seemed to be  improving in some respects because of actions taken by Congress, the  Office of Personnel Management (OPM), and individual agencies.  However, we also noted that considerable work was still needed going  forward to identify and address skills gaps.
At that time, we noted that strategic human  In response, OPM and agencies launched several initiatives. For  example, in September 2011, OPM and the Chief Human Capital Officer  (CHCO) Council established an interagency working group (Working  Group) to identify and mitigate selected skills gaps in mission-critical  occupations (MCO) and has thus far identified skills gaps in six areas:  information technology/cybersecurity, contract specialist/acquisition,  economist, human resource specialist, auditor, and the science,  technology, engineering, and mathematics (STEM) functional community.
GAO, High-Risk Series: An Update, GAO-13-283 (Washington, D.C.: February 2013). management, in May 2012, OPM launched HRstat—a quarterly data- driven review led by each agency’s CHCO—to review performance  metrics related to human resource goals, such as goals for closing skills  gaps. OPM expects to implement HRstat government-wide in 2015.
You asked us to review the progress made by OPM in closing  government-wide skills gaps and achieving its CAP goal, as well as what  additional steps, if any, are needed to better identify and address skills  gaps. This report assesses (1) lessons learned from initial efforts to close  critical skills gaps and how they can inform future initiatives; (2) what  progress OPM has made in building a predictive capacity to identify future  mission-critical skills gaps; and (3) how OPM and selected agencies are  using HRstat to identify and close skills gaps.
To accomplish the first objective, we observed the Working Group’s  meetings from July 2011 to August 2012 to understand the process it  used to identify government-wide skills gaps in MCOs. We also reviewed  documentation, such as status reports, that described the Working  Group’s activities, and our reports from October 2010 to May 2014 to  identify skills gaps that we determined were causing programmatic  impacts at agencies across the federal government. Moreover, we  reviewed planning documents for addressing skills gaps in the six MCOs  identified by the Working Group for such things as their alignment with  key practices for project planning that we identified through a literature  review. The criteria were drawn from our prior work assessing project  planning, our guide for key practices in project schedules, and our criteria  for removing issues from our high-risk list. In particular, we assessed  whether the planning documents included the following elements relevant  to closing skills gaps: (1) identifying root causes, (2) establishing  objectives, (3) developing specific actions, (4) assigning roles and  responsibilities, (5) planning duration of actions, and (6) using outcome- oriented performance metrics. To determine progress made toward the  interim CAP goal for closing skills gaps, we reviewed documentation,  such as quarterly status updates. We also interviewed OPM and agency  officials who design strategies to close skills gaps that OPM and the  Office of Management and Budget (OMB) designated as the focus of the  CAP goal.
To accomplish the second objective, we reviewed documentation, such  as strategic plans, containing information about ongoing OPM initiatives  that could build the predictive capacity within OPM to identify future  mission-critical skills gaps. We also interviewed OPM officials who were  implementing these initiatives to learn more about the status of OPM’s  efforts.
To accomplish the third objective, we selected a nongeneralizable sample  of three agencies— the Departments of Commerce (Commerce) and  Energy (DOE), and the U.S. Agency for International Development  (USAID)—from among the 24 agencies that had implemented HRstat at  the time of our review. We selected these agencies because they had  one or more of the following: (1) multiple skills gaps that we and the  Working Group identified; (2) skills gaps in any of the four occupations  that we or the Working Group most frequently identified (STEM,  acquisition, human resources, and cybersecurity); and (3) large  proportions of their workforces in each of those occupations. Also, the  selected agencies were among the first to implement HRstat and  therefore had the most experience implementing the HRstat process at  the time of our review.
We conducted this performance audit from March 2014 to January 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Mission-critical skills gaps within specific federal agencies as well as  across the federal workforce pose a high risk to the nation because they  impede the government from cost effectively serving the public and  achieving results. We first designated strategic human capital  management across the government as a high-risk issue in 2001  because of the federal government’s long-standing lack of a consistent  approach to human capital management. In February 2011, we narrowed  the focus of this high-risk issue to the need for agencies to close mission- critical skills gaps. At that time, we noted that agencies faced challenges  effectively and efficiently meeting their missions across a number of  areas, including acquisition management and foreign language  capabilities.
Trends in federal workforce retirement threaten to aggravate the problem  of skills gaps. As Figure 1 shows, 30 percent of all career permanent  employees who were on board as of September 30, 2013, will be eligible  to retire by 2018. Moreover, some agencies such as the Department of  Housing and Urban Development and the Small Business Administration  will have particularly high retirement eligibility rates by 2018.
Various factors affect when eligible individuals actually retire, which can  be seen in the number of new retirement claims received by OPM each  month. In October 2014, OPM was hit with a surge in new retirement  claims—10,155, the highest number since February 2014. However, in  the intervening months, there was considerable fluctuation in new claims.  Some amount of retirement and other forms of attrition can be beneficial  because it creates opportunities to bring fresh skills on board and allows  organizations to restructure themselves to better meet program goals and  fiscal realities. Nevertheless, if turnover is not strategically managed  and monitored and succession plans are not in place, gaps can develop  in an agency’s institutional knowledge and leadership as experienced  employees retire.
In addition to a potential wave of employee retirements, other human- capital related risks are threatening the performance of federal agencies,  including current budget and long-term fiscal pressures, declining levels  of employee satisfaction, and the changing nature of federal work with an  increasing number of positions requiring advanced degrees and other  skills.
Effectively addressing mission-critical skills gaps requires a multi-faceted  response from OPM and agencies. In February 2013, we noted that OPM  and agencies would need to use a strategic approach that (1) involves top  management, employees, and other stakeholders; (2) identifies the  critical skills and competencies that will be needed to achieve current and  future programmatic results; (3) develops strategies that are tailored to  address skills gaps; (4) builds the internal capability needed to address  administrative, training, and other requirements important to support  workforce planning strategies; and (5) includes plans to monitor and  evaluate progress toward closing skills gaps and meeting other human  capital goals using a variety of appropriate metrics. We also noted that  further progress in closing skills gaps will depend on, among other things,  the extent to which OPM develops a predictive capacity to identify newly  emerging skills gaps beyond those areas already identified.
section_title: Lessons Learned From Initial Efforts to Close Mission-Critical Skills Gaps Could Strengthen Future Approaches
subsections: section_title: Working Group’s Effort Yields Three Key Lessons for Identifying Skills Gaps
paragraphs: Based on its deliberations from September 2011 through March 2012, the  CHCO Council Working Group (Working Group) identified skills gaps in  six government-wide, mission-critical occupations (MCO): cybersecurity,  auditor, human resources specialist, contract specialist, economist, and  the STEM family. It also identified mission-critical competencies, including  data analysis, strategic thinking, influencing and negotiating, problem  solving, and grants management competencies. As part of these  deliberations, individual agencies identified agency-specific MCOs and  were responsible for designing strategies to close skills gaps in those  occupations.
Although this effort was an important step forward, because of various  methodological shortcomings, the Working Group did not address a more  comprehensive list of mission-critical skills gaps. As we discuss later in  this report, in 2015, OPM plans to identify a new set of MCOs. Going  forward, it will be important for OPM and the CHCO Council to use  lessons learned from its initial efforts to inform this next round of work.  Specifically, the Working Group’s experience underscores the importance  of (1) using a robust, data-driven approach to identify potential MCOs  early in the process; (2) prioritizing occupations using criteria that  consider programmatic impact; and (3) consulting with subject matter  experts and other stakeholders prior to identifying MCOs.
Since 2011, our work has identified skills gaps in nearly two dozen  occupations across the government. In some cases, such as  cybersecurity, the gaps we identified were affecting multiple agencies and  were consistent with the Working Group’s findings. But our work also  identified additional skills gaps, both government-wide and agency  specific, that were having a significant programmatic impact, such as:
Oil and Gas Management. In January 2014, we found that hiring and  retention challenges at the Department of Interior (Interior) have  resulted in fewer inspections of oil and gas facilities, which according  to officials results in an increased risk to human health and safety due  to a spill or accident. In 2012, Interior’s Bureau of Land  Management had an attrition rate among petroleum engineers that,  according to OPM data, is more than double the average federal  attrition rate. Although Congress has provided Interior with the  authority to establish higher minimum rates of basic pay for key  inspection occupations, we noted in January 2014 that it was  uncertain how Interior would address staffing shortfalls over time.  Interior generally agreed with our recommendation that it should  systematically collect data on hiring times for key oil and gas  positions, ensure the accuracy of the data, analyze the data to identify  the causes of delays, and expedite the hiring process. In response to  our recommendation, Interior stated that its bureaus have begun a  more systematic collection and analysis of hiring data to identify the  causes of delays and to help expedite the hiring process.
Telecommunications. In December 2013, we found that a decline in  telecommunication expertise across multiple agencies compounded  the General Services Administration’s (GSA) challenges in  transitioning those agencies to a new network of telecommunications  services, contributing to delays and cost overruns of 44 percent.  Moreover, according to GSA, customer agencies are concerned that  the shortage of telecommunications specialists will get worse because  there are not enough to replace experienced workers nearing  retirement. GSA has yet to fully study the issue of addressing mission- critical skills gaps and agreed that understanding expertise shortfalls  would be useful for future transition planning purposes. Officials from  GSA and OPM agreed with our recommendation on the need to better  examine potential government-wide telecommunications expertise  shortfalls and have agreed to coordinate on efforts to do so. While this  recommendation was still open at the time of our review, GSA’s Office  of Human Resources Management plans to take several actions such  as identifying and validating technical competencies, developing  competency models, and performing a workforce assessment against  the models.
subsections: section_title: A Data-Driven Approach Could Help OPM and Agencies Identify a More Complete Set of Mission-Critical Occupations
paragraphs: Working Group officials observed that a lesson learned from their MCO  selection process was that they did not use workforce information and  data analytics sufficiently early in the process. Instead, to identify an initial  list of MCOs, the Working Group started with an environmental scan that  consisted of our reports and academic studies. The Working Group used  data analytics, such as the levels of attrition within an occupation, only  after it had identified the initial set of MCOs. This approach only  supported the selection of MCOs the Working Group already made and  did not subject the full range of federal occupations to the same analytical  criteria. As part of our development of our 2013 High-Risk report, we  discussed this approach with the Working Group’s leadership, specifically  questioning why the Working Group did not use workforce data as the  starting point for its selection process.2013 discussion of lessons learned, Working Group officials concluded  that the MCO selection process would have benefitted from analyzing  staffing gap data and associated trends prior to identifying an initial set of  MCOs. These officials determined that using these workforce data would  have given the Working Group a better sense of which occupations had  the biggest skills gaps.
section_title: Prioritization Criteria Should Consider Potential Programmatic Impact
paragraphs: In February 2012, to help prioritize their efforts, Working Group officials  limited the scope of their work by creating criteria that at least half of the  24 CHCO Council agencies needed to report having a skills gap in that  occupation, and that those agencies collectively needed to employ at  least 95 percent of the occupation’s workforce. Working Group officials  explained that limiting the number of MCOs using these criteria enabled  them to focus on occupations found in most agencies. The officials also  noted that it was important to establish a threshold for the number of  agencies with a given skills gap so that the Working Group could focus  resources on addressing skills gaps that had the greatest reach across  the government. Officials added that skills gaps that existed at only a few  agencies were the agencies’ responsibility to address.
While we recognize the importance of prioritizing MCOs to a manageable  number, the Working Group’s approach overlooked skills gaps that may  not have met the criteria but still had the potential for significant  programmatic impact. For example, in August 2014, we found that, in part  because of staffing shortages, the Department of Justice’s Bureau of  Prisons was not activating new prison facilities in a timely manner,  thereby aggravating the problem of prison overcrowding. Additionally,  we found in December 2013 that the Department of Transportation’s  Federal Railroad Administration lacked a plan to have sufficient safety  inspectors to carry out oversight of such initiatives as positive train  control—a communications system designed to prevent events like train-  Indeed, Working Group officials later recognized that  to-train collisions.establishing such a strict standard for a government-wide MCO resulted  in eliminating from consideration other important occupations, such as  nurses and financial analysts. Because the Working Group’s selection  criteria created such a strict and narrow standard for government-wide  skills gaps, its efforts failed to capture the broader range of skills gaps  that were affecting agencies’ abilities to meet their missions. As we note  later in this report, OPM’s proposed methodology for selecting a new set  of MCOs does not include such a strict definition for government-wide  skills gaps.
section_title: Early Stakeholder Input Can Help Improve the Identification of Skills Gaps
paragraphs: Our principles for effective strategic workforce planning note that  agencies should involve stakeholders in developing and implementing  future workforce strategies. However, the Working Group did not  thoroughly consult occupational experts from other interagency councils  or organizations prior to identifying MCOs. Instead, Working Group  officials only considered consulting relevant stakeholders to discuss  strategies for addressing skills gaps at a March 2012 meeting—after the  Working Group had already identified preliminary MCOs for selection. As  a result, stakeholders could only provide advice on decisions already  made rather than help with the initial screening of MCOs. As one  example, members of the Council of Inspectors General on Integrity and  Efficiency—an Executive Branch interagency council devoted, in part, to  providing training for federal auditors—told us that they had not been  consulted about the existence of skills gaps in the auditor occupation and  were initially unsure why the auditor occupation was designated by the  Working Group as an MCO for skills gap closure. A Working Group  official indicated at a January 2012 meeting that the Working Group had  sufficient subject matter experts internally, and verification of MCO  selection with outside experts could lead to competing lists of MCOs and  could expend significant resources. As a result of this limited outreach,  however, the Working Group missed an opportunity to leverage the  expertise of key outside stakeholders to get a more complete assessment  of the need to close skills gaps. As we describe later in this report, OPM  has recognized this deficiency and is proposing a methodology for a new  interagency working group that incorporates more input from subject  matter experts when identifying emerging skills gaps.
section_title: Efforts to Address Skills Gaps Are Underway with Mixed Results
subsections: section_title: OPM Provided Resources and Visibility to Efforts to Close Skills Gaps
paragraphs: OPM and the Working Group have made important progress in creating  an infrastructure to address the six mission-critical skill gaps they  identified. The GPRA Modernization Act of 2010 (GPRAMA) requires  OMB to coordinate with agencies to develop cross-agency priority (CAP)  goals to improve the performance and management of the federal  government in several areas, including human capital management.The fiscal year 2013 budget—released in February 2012—designated  OPM as leader of a two-year interim CAP goal to close skills gaps by 50  percent in three-to-five MCOs by September 30, 2013. As goal leader,  the Director of OPM appointed key federal officials from each of the six  government-wide mission-critical occupations to serve as sub-goal  leaders for each of the six MCOs identified by the Working Group. At the  time of our review, the sub-goal co-leaders for the cybersecurity  workforce, for example, were the Assistant Director for Cybersecurity at  the White House Office of Science and Technology Policy and the Lead  for the National Initiative for Cybersecurity Education within the  Department of Commerce’s National Institute of Standards and  Technology. Likewise, the sub-goal leader for the economist workforce is  the Assistant Secretary for Economic Policy and Chief Economist at the  U.S. Department of the Treasury.
In working within their occupational communities, the sub-goal leaders  selected specific strategies to decrease skills gaps in the occupational  communities they represent. Multiple sub-goal leaders indicated that the  quarterly meetings and availability of OPM staff and their human  resources expertise provided visibility and resources to sub-goal leaders’  efforts to close skills gaps in their MCOs. For example, Auditor sub-goal  leaders told us that the OPM Director assisted in conducting a study of  government-wide recruitment and hiring of auditors. In partnership with  OPM and stakeholders from the federal audit community, Auditor sub- goal leaders found that the current qualifications for federal auditors do  not align with the nature of federal audit work. To address this challenge,  OPM is studying how to change the qualifications for federal auditor  positions to improve agencies’ experiences with recruiting and hiring  qualified candidates for that occupation.
The meetings and performance reviews by OPM and sub-goal groups  demonstrate high-level leadership commitment to address mission-critical  skills gaps. We have maintained that removing skills gaps as a high-risk  issue across the government will depend in part on the extent to which  OPM and agencies involve top management and include plans to monitor  and evaluate progress toward closing skills gaps.year 2015 budget, released in March 2014, includes a four-year human  capital management CAP goal to (1) create a culture of excellence and  engagement to enable higher performance, (2) build a world-class federal  management team starting with the Senior Executive Service, and (3)  enable agencies to hire the best talent from all segments of society. While  these CAP goal elements contain workforce planning strategies and  metrics relevant to closing skills gaps, there are no overall performance  targets for closing skills gaps, and closing skills gaps is no longer an  explicit goal.
The President’s fiscal  Throughout the time period when closing skills gaps was an explicit CAP  goal, top officials at OPM held the sub-goal groups accountable for  making progress by holding quarterly performance review meetings. OPM  also used these meetings to discuss challenges the sub-goal leaders  were facing and to field requests from sub-goal leaders for how they  could better facilitate sub-goal groups’ efforts. Now that the fiscal year  2013 CAP goal expired and was replaced with a fiscal year 2015 human  capital management CAP goal that did not have explicit targets for closing  skills gaps, OPM officials have told us that they intend to continue  meeting with current sub-goal leaders through a community of practice.  OPM has also worked with agencies to develop occupation-specific  communities of practice within the science, technology, engineering, and  mathematics (STEM) and Economist sub-goal groups. In addition, OPM  has indicated that it intends to hold quarterly performance review  sessions with officials in charge of efforts to close skills gaps in emerging  government-wide occupations in a similar format to what was done during  the prior CAP goal.
These are all important steps in the right direction and highlight OPM’s  commitment to addressing skills gaps going forward. Still, the CAP goal to  address skills gaps gave the entire effort government-wide focus and  visibility, and provided OPM a mechanism to hold agencies accountable  for results. It will be important for OPM to continue its leadership on this  high-risk issue by holding occupational leaders accountable for  implementing their strategies to close skills gaps and sustaining the  visibility of the issue among agency officials across the government. Key  focus areas include using (1) better defined, more measurable goals, (2)  outcome-oriented performance metrics that align with activities to close  skills gaps, and (3) key practices for project planning.
section_title: Use of Better Defined, Measurable Goals Enhances Monitoring on Closure of Skills Gaps
paragraphs: The CAP goal target to close skills gaps was vague and difficult to  measure. The fiscal year 2013 human capital CAP goal’s target was to  close skills gaps by 50 percent by the end of fiscal year 2013 in three-to- five of the six MCOs for which OPM appointed sub-goal leaders and  groups of officials from across the government. Because the CAP goal  target was difficult to measure, however, there is no clear basis for  determining whether the CAP goal was met.
Our prior work on performance goals has indicated that setting ambitious  but realistic goals is one of the factors that can influence whether goal- setting and performance measurement efforts will be successful.  However, sub-goal leaders and OPM officials leading the effort stated that  the 50 percent skills gap closure target was difficult to measure.  Specifically:
An OPM official told us that the 50 percent target was not set  appropriately and that, rather than have a single, overarching  performance target, sub-goal groups needed individualized targets to  allow for greater measurability.
Similarly, the STEM sub-goal co-leader told us that the CAP goal  target was not designed in a way that could facilitate a sophisticated  measurement of progress by the various sub-goal groups.
The Cybersecurity sub-goal leader indicated that the group’s efforts  were at such an early stage of maturity that it could not gauge a 50  percent skills gap closure because it had not fully determined the  nature of its skills gap. The Cybersecurity sub-goal group, therefore,  did not have an effective baseline from which to measure 50 percent  progress.
section_title: Align Performance Metrics with Targets and Activities for Closing Skills Gaps
paragraphs: Of the six sub-goal groups (Acquisition, Cybersecurity, Economist,  Human Resources, STEM, and Auditor), three—the Cybersecurity,  Economist, and Human Resources sub-goal groups—used performance  metrics that either did not align with outcomes for closing skills gaps or  did not represent their activities for closing skills gaps (the STEM and  Auditor sub-goal groups did not establish metrics at all). Our prior work  has outlined the importance of developing outcome-oriented performance  metrics that clearly and sufficiently relate to the performance they are  meant to assess. Our prior work has also discussed the importance of  using performance metrics to link agencies’ goals and priorities with the  actions that they are taking so that leaders can hold their organizations  accountable for progress.
For example, the Acquisition sub-goal group aligned its metrics with  activities and overall targets. Sub-goal leaders indicated that they focused  their efforts to close skills gaps on maintaining federal acquisition  competencies. To do so, the sub-goal group set a performance metric of  increasing the certification rate among civilian contract specialists and  produced guidance on revising agencies’ certification curricula. Sub-goal  leaders indicated that increasing the certification rate among civilian  contract specialists by 10 percentage points—from 75 percent to 85  percent—was adequate for the occupation’s long-term needs. To make  50 percent progress toward this target by the end of fiscal year 2013, they  worked toward a metric of an 80 percent certification rate. The Acquisition  sub-goal group’s efforts resulted in a 6 percentage point increase in the  civilian contract specialist certification rate—to 81 percent—which  surpassed its CAP goal metric and was aligned with such actions as  producing guidance on reforming agencies’ certification curricula.
In contrast, the Cybersecurity sub-goal group is an example of a sub-goal  group that did not fully align its performance metric with the sub-goal  group’s activities. For instance, officials began an initiative in 2013 to  categorize the cybersecurity specialty areas for the various occupations  involved in federal cybersecurity work. This initiative, which operated  through the end of fiscal year 2014, will allow managers to better assess  what cybersecurity work needs to be done at agencies, and where there  are gaps preventing agencies from accomplishing their cybersecurity  missions. The Cybersecurity sub-goal leader noted that using the updated  classifications to identify skills gaps is necessary before those skills gaps  can be addressed. Officials did not, however, include a publicly-reported  metric in fiscal year 2013 to track how many of the cybersecurity-related  classifications had been updated.
Moreover, the Human Resources sub-goal group is an example of a sub- goal group that did not align its metric with the outcome of closing skills  gaps. The Human Resources sub-goal group tracked the percentage of  federal human resources personnel who registered for and completed a  single course on HR University—a centralized online suite of courses and  curricula managed by OPM that agencies can use for training purposes.  OPM officials said that they tracked these indicators because they  perceived the need to standardize human resources training curricula  across agencies as a first step to closing the skills gap. OPM officials told  us, however, that they are still striving to develop more outcome-oriented  measures from the use of HR University. We agree that while ensuring  that human resources professionals receive proper training is vital, relying  on a metric of how many people register for and complete a single online  course is not the most effective way to assess the outcome of closing skill  gaps within the human resource occupation. OPM officials also noted that  a 2014 data clean-up of HR University detected erroneous information  that made the fiscal year 2013 registration and course completion rates  unreliable as reported. OPM has continued to use the course  registration performance metric as part of its fiscal year 2015 Agency  Priority Goal to close skills gaps in the federal HR workforce. OPM  officials told us that data flaws on HR University have been corrected.  However, because this metric does not measure an outcome, it does not  provide quality information toward goals to close skills gaps.
section_title: Key Project Planning Practices Could Improve Efforts to Address Skills Gaps
paragraphs: Sub-goal groups’ planning documents for efforts to address skills gaps in  fiscal year 2014 generally met three of six key practices for project  planning. As we have noted, a well-developed and documented project  plan can help ensure that agencies are able to gauge progress, identify  and resolve potential problems, and promote accountability at all levels of  the project. Project plans can also encourage agency managers and  stakeholders to systematically consider what is to be done, when and  how it will be done, and what skills will be needed. We identified the  following six key practices from our prior work: (1) identifying root causes  of issues, (2) establishing objectives, (3) developing specific actions, (4)  assigning roles and responsibilities, (5) establishing the duration of  actions, and (6) using outcome-oriented performance metrics.
Table 1 lists each of the sub-goal groups and our evaluation of their  planning documents against each of the key practices for project  planning. For plans that included some information relevant to a key  practice but did not provide sufficient detail nor fully addressed the key  practice, we gave partial credit.
Some key practices were used consistently by sub-goal groups. As  shown above, sub-goal groups’ plans generally met some of our key  practices. For instance, all of the sub-goal groups structured their plans  by listing specific actions for addressing skills gaps. Our prior work states  that it is important to identify the specific actions necessary to achieve a  plan’s objectives. By doing so, managers can properly assess the risk of  not achieving the plan’s objectives.Acquisition sub-goal group listed a clear division of actions—such as  publishing a revised certification curriculum and increasing the  government-wide use of an acquisition workforce management system— that addressed objectives that included strengthening civilian contracting  certification standards.
For example, the plan for the  Some key practices were generally absent from sub-goal group  project planning. Sub-goal groups’ plans were less effective in meeting  other key practices for project planning. For instance, only the STEM and  Auditor sub-goal groups’ plans discussed the root causes of their skills  gaps and the purpose for their actions.
Additionally, sub-goal groups did not consistently identify roles and  responsibilities for actions to address skills gaps. While all sub-goal  groups listed agencies that would be responsible for accomplishing  actions, only the Auditor sub-goal group identified individual officials  charged with specific actions.
Three of six sub-goal groups did not always track outcome-oriented  performance metrics in their plans. For instance, the STEM sub-goal  group’s plan tracked such items as the number of STEM hiring reforms  that had been approved by OPM. While gaining OPM approval of hiring-  policy changes is important in attracting more qualified workforce  candidates, the STEM sub-goal group’s plan did not track outcomes that  might result from approving such policy changes, such as the number and  quality of applicants and hires for STEM positions.
section_title: OPM’s Efforts to Predict Emerging Mission-Critical Skills Gaps Are in the Early Planning Stages
subsections: section_title: OPM Plans to Strengthen the Methodology Used to Identify Emerging Skills Gaps
paragraphs: The interagency working group that identified the list of skills gaps in six  government-wide mission-critical occupations (MCO) was re-named the  Federal Agency Skills Team (FAST). OPM has tasked the group— composed of agency officials with workforce planning and data analysis  skills—with implementing a standard and repeatable methodology that  identifies and addresses government-wide skills gaps as well as mission- critical competencies over a 4-year cycle. In the first year, OPM officials  stated that FAST intends to meet regularly until it identifies a new set of  government-wide skills gaps, which OPM officials expect will occur by  June 2015. Our analysis of FAST’s proposed methodology identified  three features that incorporate lessons learned from the Working Group’s  initial efforts, which were described earlier. First, FAST is to use a data- driven approach as the initial step for identifying a broad list of skills gaps.  Specifically, officials from the 27 Chief Human Capital Officers (CHCO)  Council agencies will be expected to compare their agency’s mission- critical or priority occupations against several factors. According to OPM  officials, those factors include:
2-year retention rate, quit rate, retirement rate, and applicant quality.
Second, in addition to being more data-driven, FAST will use a multi- faceted approach to identify a select number of government-wide MCOs.  This is to be achieved by using teams of at least three to four individuals  to monitor internal and external environments using various sources— including news articles, reports, and interviews with public and private  sector subject matter experts—to detect trends that may influence an  agency’s current and future workforce needs.
Third, our analysis determined that the proposed methodology does not  have a strict numerical standard for what constitutes a government-wide  skills gap. This could allow for more discretion to address occupations  with the potential for significant programmatic impacts even though an  absolute majority of agencies may not have skills gaps in those areas.
After this identification process, FAST is to develop strategies to address  the new set of skills gaps during the remaining years of the 4-year cycle.  In years two and three, OPM—in conjunction with FAST—is to designate  leaders from within the selected government-wide occupations who will  develop and implement plans to address those skills gaps. Finally, in year  four, FAST plans to evaluate and monitor outcomes to determine the  effectiveness of those strategies. In addition, during the fourth year FAST  plans to incorporate lessons learned into a revised process for identifying  skills gaps at the federal level.
section_title: OPM’s Plan to Capture Staffing Data Using Its Database Lacks a Schedule
paragraphs: OPM’s plan to capture staffing data for select occupations using a  database is still under development. Each year, OPM collects staffing  data through a reporting process where the CHCO Council agencies are  required to provide OPM with information—such as the target number of  employees and projected attrition—for five government-wide MCOs  (cybersecurity, human resources specialist, economist, contract  specialist, auditor), and three-to-five agency-specific or STEM MCOs.  Among other things, OPM uses these data to determine if an agency’s  workforce has a staffing gap in any of those occupations.
In response to an administration initiative directed, in part, to reduce the  human resources reporting burden on agencies, OPM is exploring how it  can replace this annual reporting process by using its database  containing federal workforce information—known as the Enterprise  Human Resources Integration (EHRI)—to capture the same agency  staffing data. However, an OPM official stated that no timeframe  currently exists for using EHRI to capture agencies’ MCO staffing data. In  the interim, an OPM official stated that they will continue collecting MCO  staffing data until OPM can make the necessary investments in EHRI.  Our schedule assessment guide has noted that a well-planned schedule  is a fundamental management tool that can (among other things) help  government agencies specify when work will be performed in the future.Moreover, our schedule assessment guide states that a consistent  methodology for developing, managing, and evaluating cost estimates for  certain types of programs includes the concept of scheduling the  necessary work to a timeline. By establishing a schedule specifying when  EHRI will be modified to capture government-wide staffing data, OPM  officials will have a road map for gauging progress, identifying and  resolving potential problems, promoting accountability at all levels of the  agency, and determining the amount and timing of the funding needed.
section_title: OPM Lacks a Process for Collecting Government- wide Competency Data
paragraphs: According to an OPM official, there is no process, using EHRI or another  system, for collecting consistent data on the competencies of the federal  workforce, which is needed to effectively predict future mission-critical  skills gaps. According to an OPM official, federal agencies’ ability to  assess workforce competencies varies, which makes collection of  government-wide data on competency gaps difficult. As one example, we  have found that the Census Bureau has started to assess the  competencies needed to carry out its future work. In contrast, Department  of Commerce (Commerce) human capital officials stated that they do not  conduct a Department-wide competency assessment of Commerce’s  workforce. Moreover, the Department of Energy (DOE) has conducted a  competency assessment for a number of engineering occupations such  as nuclear engineers. DOE officials have also developed a model for  conducting a Department-wide competency assessment. Key principles  for effective workforce planning that we developed note that agencies  must determine the competencies that are critical to successfully  achieving their mission and goals. Doing so can help agencies to  effectively meet demographic, technological, and other forces that are  challenging government agencies to change the activities they perform  and the goals that they must achieve, how they do their business, and  even who does the government’s business. Therefore, as OPM  develops its process for using EHRI to collect agencies’ skills gaps data,  it will be important for OPM to also work with agency CHCOs to bolster  the ability of agencies to assess workforce competencies by sharing  competency surveys, lessons learned, and other tools and resources— and to ensure that such information can be stored in the EHRI database  for government-wide workforce analysis.
section_title: OPM and Selected Agencies Could Improve Efforts to Address Skills Gaps by Strengthening Data-Driven Reviews
subsections: section_title: OPM Developed HRstat Data-Driven Reviews to Help Agencies Regularly Track Their Progress in Achieving Their Human Resources Goals
paragraphs: Data-driven reviews—commonly referred to as “stat” meetings—are  regularly scheduled, structured meetings used by organizations to review  performance metrics with department or program personnel to drive  progress on agency priorities and goals. Conducting frequent stat  meetings is a leadership strategy proven to help agency officials achieve  results by focusing on an identified set of priorities, diagnosing problems,  and deciding on the next steps to increase performance. The GPRA  Modernization Act of 2010 required agencies to conduct data-driven  quarterly progress reviews with key personnel responsible for the  accomplishment of agency priority goals.
Building on this statutory model, OPM created HRstat, which is a CHCO- led review of the key metrics that contribute to agencies’ human  resources goals, such as closing mission-critical skills gaps. launched HRstat as a 3-year pilot program in May 2012, with an initial  group of eight agencies that included Commerce, DOE, and the U.S.  Agency for International Development (USAID)—the three agencies we  selected for our illustrative case studies. In 2013 and 2014, OPM chose  eight agencies per year for successive pilots. As a result, all 24 agencies  subject to the Chief Financial Officers Act of 1990, as amended, had at  least one year of HRstat implementation by the end of 2014.
According to OPM guidance, agency Performance Improvement Officers and Chief  Operating Officers should support HRstat reviews. assess them. The three agencies we selected all have closing critical  skills gaps as one of their agency-wide human resource goals.
subsections: section_title: Selected Agencies Are Using Different Metrics to Track Skills Gaps under Their HRstat Reviews
paragraphs: Based on our assessment of the HRstat reviews of our selected agencies  (using agency material from the third quarter of fiscal year 2014), we  identified those metrics most relevant to tracking progress on closing  skills gaps. As illustrated in table 2 below, we found considerable  variation in the number and types of metrics agencies were using to track  skills gaps.
As shown above, only DOE was tracking retirement eligibility and  projected retirements, which are key indicators of where agencies might  be at risk for future skills gaps. Likewise, only Commerce was tracking  candidate quality, which provides an indication of whether applicants had  the skills needed to perform the work. While some amount of variation is  both desirable and expected given the different missions of the agencies  and the flexibility OPM gave agencies in selecting metrics, the variation  shown above has at least two significant downsides. First, the variation in  number and types of metrics agencies are tracking suggest that some  agencies’ HRstat reviews are more robust and well rounded than others  in that they are measuring factors affecting skills gaps from a more  complete perspective. Indeed, each metric may tell a different story about  the extent and nature of current and emerging skills gaps. This could lead  to different remedial actions on the part of agencies.
A second downside in the variation in numbers and types of metrics is  that it limits OPM and the CHCO Council’s ability to track agencies’  progress in closing skills gaps government-wide. Our February 2013  report noted that a leading practice for successful data-driven reviews,  such as HRstat, is to ensure alignment between goals, program activities,   However, because our selected agencies are using  and resources.different metrics, it is difficult for OPM to assess where agencies are  making progress, where additional efforts are needed, and how OPM  might be able to help them, if at all. It also limits the ability of agencies  and OPM to discuss and share lessons learned in identifying and  addressing skills gaps.
OPM officials stated that they knew of the various metrics used by  agencies during their HRstat reviews. They noted that the HRstat pilot  program was intended to be “agency-centric” and not a data-collection  tool for OPM. While we agree that agencies should continue to have  flexibility in choosing and “owning” metrics for their HRstat reviews that  best meet their particular needs, a core set of metrics—while still allowing  agencies discretion to include metrics that meet their specific  requirements—could help strengthen the quality and consistency of the  HRstat reviews from a government-wide perspective.
Moreover, a core set of metrics could foster collaboration between  agencies’ use of HRstat reviews and the efforts envisioned under FAST.  As we noted earlier, FAST is to use a data-driven approach as an initial  step for identifying a range of skills gaps, such as a survey of hiring  managers’ satisfaction with job applicants’ skills. As shown above,  Commerce already tracks this same survey of hiring managers’  satisfaction during its HRstat reviews. A core set of metrics could  integrate the work done by FAST with HRstat reviews.
Going forward, OPM, in conjunction with the CHCO Council, plans to  identify key strategic and operational HR metrics all agencies will collect  through HRstat and share with OPM. As part of that effort, it will be  important for OPM and the CHCO Council to develop a core set of valid  metrics that are directly aligned with the goal of identifying and  addressing agency skills gaps.
section_title: Conclusions
paragraphs: Closing workforce skills gaps is critical for agencies to better achieve a  wide range of missions, from purchasing mission-critical goods and  services to carrying out the Decennial Census. While efforts to close  mission-critical skills gaps are couched in discussions about staffing  numbers, competencies, metrics, and similar technical terms, the ultimate  goal is higher-performing, cost-effective government. However, the  challenges that agencies face are diverse and were not fully captured by  the CHCO Council Working Group’s first efforts to identify skills gaps in  government-wide, mission-critical occupations. Although these initial  efforts created an infrastructure for addressing skills gaps, to date, overall  progress remains mixed. At times, goals have suffered from having  targets that are difficult to measure. Likewise, agency officials have  chosen to track metrics that often do not allow for an accurate  assessment of progress made toward these goals for closing skills gaps.
Building the predictive capacity to identify emerging mission-critical skills  gaps is also critical to making further progress in addressing this issue.  Realizing this, OPM has established an interagency working group known  as FAST, which is responsible for identifying and addressing current and  emerging skills gaps. OPM also intends to replace its annual reporting  process for collecting agency staffing data by modifying its workforce  database to capture the same data. These are important steps forward.  However, we are concerned about these efforts for two reasons. First,  OPM has not established a time frame for modifying its workforce  database to capture the same agency staffing data that it currently  collects through an annual reporting process—which will reduce the  human capital reporting burden on federal agencies. Second, OPM  officials stated that there is no process for collecting data on the  competencies of the federal workforce because agencies’ ability to  assess workforce competencies varies. Helping agencies determine the  competencies that are critical to successfully achieving their mission and  goals will help them respond to external factors, such as changes in  national security, technology, or budget constraints.
At the agency level, the use of HRstat meetings is a proven leadership  strategy that could help agency officials monitor their progress toward  closing skills gaps. However, OPM should take a greater leadership role  in helping agencies include a core set of metrics in their HRstat reviews  so that OPM and agency leaders can have a clear view of progress made  closing skills gaps. While it is important for agencies to have ownership  over their HRstat reviews, OPM should also maximize its opportunity to  use HRstat to gain greater visibility over the federal workforce.
section_title: Recommendations for Executive Action
paragraphs: To assist the interagency working group, known as FAST, to better  identify government-wide skills gaps having programmatic impacts and  measure its progress towards closing them, we recommend that the  Director of OPM—in conjunction with the CHCO Council—strengthen its  approach and methodology through the following actions:
Assist FAST in developing goals for closing skills gaps with targets  that are both clear and measurable.
Work with FAST to design outcome-oriented performance metrics that  align with overall targets for closing skills gaps and link to the activities  for addressing skills gaps.   Incorporate greater input from subject matter experts, as planned.
Ensure FAST consistently follows key practices for project planning.
To ensure that OPM builds the predictive capacity to identify emerging  skills gaps across the government—including the ability to collect and use  reliable information on the competencies of the federal workforce for  government-wide workforce analysis—we recommend that the Director of  OPM take the following two actions:
Establish a schedule specifying when OPM will modify its EHRI  database to capture staffing data that it currently collects from  agencies through its annual workforce data reporting process.
Work with agency CHCOs to bolster the ability of agencies to assess  workforce competencies by sharing competency surveys, lessons  learned, and other tools and resources.
To help agencies and OPM better monitor progress toward closing skills  gaps within agencies and government-wide, we recommend that the  Director of OPM:
Work with the CHCO Council to develop a core set of metrics that all  agencies should use as part of their HRstat data-driven reviews.
Coordinate with FAST personnel and explore the feasibility of  collecting information needed by FAST as part of agencies’ HRstat  reviews.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report for review and comment to the Directors  of OPM and OMB; the Secretaries of the Departments of Defense,  Energy, and Treasury; the Administrators of the National Aeronautics and  Space Administration (NASA) and USAID; the Chief Financial Officer and  Assistant Secretary for Administration at the Department of Commerce;  and the Special Assistant to the President and Associate Counsel to the  President in the Executive Office of the President.
The following agencies had no comments on the draft report: the  Departments of Defense, Energy, and Treasury, NASA, OMB, USAID,  and the Office of Science and Technology Policy in the Executive Office  of the President; as well as Commerce’s Bureau of the Census and  National Institute of Standards and Technology.
OPM and Commerce’s Office of Human Resources Management  provided technical comments that were incorporated, as appropriate.
In its written comments, reproduced in appendix II, OPM partially  concurred with one recommendation, did not concur with one  recommendation, and concurred with one recommendation. Specifically,  OPM stated that it partially concurred with our recommendation to  strengthen the approach and methodology used by the interagency  working group, known as FAST, to better identify skills gaps. OPM noted  it agreed with, and planned to implement, the principles of each  recommended action. However, OPM said it needed to clarify how its  terminology and planned process differs from the description in our  recommendation. In particular, OPM stated its process will identify  government-wide rather than agency-specific skills gaps as it believes our  draft recommendation suggests. We recognize that FAST was  established to address government-wide skills gaps and have clarified the  language in our recommendation accordingly.
OPM stated that it does not concur with our recommendation to: (1)  establish a schedule specifying when it will modify its EHRI database to  automatically capture staffing data that it currently collects from agencies  through its annual workforce data reporting process, and (2) work with  agency CHCOs to bolster agencies’ ability to assess workforce  competencies by sharing competency surveys, lessons learned, and  other tools and resources. Regarding EHRI, OPM maintained that it is  impossible for the EHRI database to automatically capture staffing data  currently included in MCO Resource Charts because some of these data  includes specific agency projections and targets, which are provided via a  manual data feed. OPM stated that it is assessing whether EHRI can be  modified to allow agencies to supply these manual feed data into the  database system. We have modified our report to recognize that EHRI  cannot automatically capture the same agency staffing data that are  captured through the MCO Resource Charts.
In addition, OPM noted that there are funding implications associated with  its ability to anticipate whether and when a modification schedule to the  EHRI online database could be established. While we appreciate OPM’s  funding concerns, as we mention in the report, a well-planned schedule is  a fundamental management tool that can (among other things) help  government agencies specify when work will be performed in the future.  Moreover, scheduling the necessary work to a timeline is important for  developing, managing, and evaluating cost estimates for certain types of  programs. As such, a schedule, as we recommend, would help OPM  determine the amount and timing of the funding needed, and help OPM  identify the competing priorities that need to be balanced due to resource  constraints. We therefore continue to believe OPM would benefit by  implementing our recommendation.
Regarding workforce competencies, OPM noted that funding and  resource constraints negatively affect its ability to support agencies’  efforts to address their workforce competencies. While funding limitations  could affect OPM’s ability to take these actions, our recommendation  would help OPM and agencies stretch resources by leveraging their  knowledge and experience. We therefore continue to believe OPM would  benefit by implementing our recommendation.
OPM concurred with our recommendation to work with the CHCO Council  to develop a core set of metrics that all agencies should use as part of  their HRstat data-driven reviews, and explore the feasibility of collecting  information needed by FAST as part of agencies’ HRstat reviews.
We are sending copies of this report to the appropriate congressional  committees, the Director of OPM, the Secretaries of Commerce and  Energy, the Administrator of USAID, and other interested parties. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report assesses (1) lessons learned from initial efforts to close  critical skills gaps and how they can inform future initiatives, (2) what  progress the Office of Personnel Management (OPM) has made in  building a predictive capacity to identify future mission-critical skills gaps,  and (3) how OPM and agencies are using HRstat to identify and close  skills gaps.
To assess lessons learned from initial efforts to close critical skills gaps  and how they can inform future initiatives, we took the following steps.
We observed the Chief Human Capital Officer (CHCO) Council skills  gaps working group’s (Working Group) meetings from July 2011 to  August 2012 to understand the process it used to identify  government-wide skills gaps in mission-critical occupations (MCO).
We reviewed documentation, such as status reports, that described  the Working Group’s activities. Furthermore, we reviewed our reports  from October 2010 to May 2014 to identify skills gaps that we  determined were causing programmatic impacts at agencies across  the federal government. To identify our reports that identified skills  gaps at agencies across the federal government, we searched  through our Engagement Reporting System, our external database,  and our Engagement Results Phase internal database. Reports were  selected if they included references to skill gaps or related workforce  conditions that might produce skill gaps (workforce shortages,  competency gaps, training deficits) that were primary objective  findings issued since the start of fiscal year 2011, updates on skill  gaps identified since the beginning of fiscal year 2011, or updates on  skill gaps identified in our 2011 and 2013 High Risk Series reports.  Reports were not selected if they found skills gaps at agencies prior to  fiscal year 2011, or if they referenced skills gaps that were not part of  the engagement’s findings. Private sector skills gaps were also  excluded because the scope of this engagement was limited to OPM’s  and agencies’ progress closing skills gaps across the federal  government.
We reviewed planning documents for addressing skills gaps in the six  MCOs identified by the Working Group for such things as their  consistency with key practices for project planning. We identified  project planning key practices from our prior work assessing project  planning, our guide for key practices in project schedules, and our  criteria for removing issues from our high-risk list. In particular, we  assessed whether each planning document included the following  elements relevant to closing skills gaps: (1) identifying root causes, (2)  establishing plan objectives, (3) developing specific actions needed to  accomplish objectives, (4) assigning roles and responsibilities to all  actions, (5) establishing the durations of actions, and (6) using  outcome-oriented performance metrics. Two analysts independently  assessed each planning document to determine the extent to which  information that was included met all six key practices and rated each  plan using a three-level scale of yes, partially, or no, and reached a  level of inter-rater agreement of 100 percent. For plans that included  some information relevant to a key practice but did not provide  sufficient detail or did not fully address the key practice, we gave  partial credit.
We reviewed documentation such as quarterly status updates to  determine what progress had been made toward the interim cross- agency priority (CAP) goal for closing skills gaps. We also interviewed  OPM and agency officials who are responsible for designing  strategies to close skills gaps that OPM, in coordination with, the  Office of Management and Budget, designated as the focus of the  CAP goal.
To assess what progress OPM has made in building a predictive capacity  to identify future government-wide mission-critical skills gaps, we  reviewed documentation, such as OPM strategic plans and meeting  minutes from the CHCO Council Executive Steering Committee,  containing information about ongoing OPM initiatives that could build the  predictive capacity within OPM to identify future mission-critical skills  gaps. We also interviewed OPM officials who were implementing these  initiatives to learn more about their status.
To assess how OPM and agencies are using HRstat to identify and close  skills gaps, we selected a nongeneralizable sample of three agencies— the Departments of Commerce and Energy, and the U.S. Agency for  International Development—from among the 24 agencies that had at  least begun implementing HRstat at the time of our review. We selected  these agencies because they had one or more of the following: (1)  multiple skills gaps that we and the Working Group identified; (2) skills  gaps in any of the four occupations that we or the Working Group most  frequently identified (STEM (science, technology, engineering, and  mathematics), acquisition, human resources, and cybersecurity); and (3)  large proportions of their workforces in each of those occupations. Also,  the selected agencies were among the first to implement HRstat and  therefore had the most experience with the HRstat process at the time of  our review. To assess how our selected agencies were using HRstat to  address skills gaps, we also reviewed memorandums, internal briefings,  and other material that agencies used to prepare for the reviews, and  interviewed officials involved in each agency’s HRstat review process.
We conducted this performance audit from March 2014 to January 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Office of Personnel Management
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Steven Lozano, Assistant  Director; Don Kiggins, Analyst-in-Charge; Devin Braun; Deirdre Duffy;  Karin Fangman; Donna Miller; and Rebecca Shea made major  contributions to this report.