Workforce Investment Act: Local Areas Face Challenges Helping Employers Fill Some Types of Skilled Jobs
Published: 2013-12-02T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The economy is recovering from the recession, but employers still have difficulty filling certain jobs. DOL's Workforce Investment Act (WIA) Adult and Dislocated Worker programs are designed in part to help employers find the skilled workers they need. The programs provide participants with services including job training, which must be for occupations that are in demand. However, questions have been raised about the extent to which these programs are positioned to help supply workers for jobs that employers have difficulty filling. The conference report accompanying the Consolidated Appropriations Act of 2012 mandated that GAO assess the Adult and Dislocated Worker programs. This report addresses (1) how local workforce areas have identified occupations that are in demand and how they have guided participants toward training for them; and (2) what challenges local workforce areas have faced in helping employers fill certain jobs. GAO conducted a web-based survey of a nationally representative sample of 200 local workforce investment boards (WIB), which oversee local workforce areas, and used the results to create estimates about the population of all WIBs nationwide. GAO also interviewed DOL officials and workforce organizations.
section_title: What GAO Found
paragraphs: Based on survey results for calendar year 2012, GAO estimates that most local areas used various sources of information to identify occupations that are in demand (i.e., available jobs and occupations that are projected to grow). Local areas found all of the sources of information they used to be at least moderately useful. To identify occupations that are in demand, GAO estimates that nearly 90 percent of local areas used state job banks and occupational projections, both of which are funded by the Department of Labor (DOL). All percentages presented on this highlights page are estimates from the survey and have a sampling error of no larger than plus or minus 7 percentage points. In addition, 93 percent of areas used at least one local initiative, such as a partnership with the local economic development agency, to identify such jobs. As compared to other sources, areas reported using local initiatives because they provided more detailed information and better reflected local employers' needs. To guide participants toward training, most local areas required them to complete certain activities, such as meeting with a case manager to discuss training options (80 percent) or completing a skills assessment (78 percent). Most local areas faced challenges in guiding participants toward training. Specifically, local areas faced challenges related to participants' lack of financial or work supports, such as child care or transportation (67 percent); participants' lack of the basic skills necessary even to participate in training (66 percent); difficulty finding training providers who could quickly adapt curricula to employers' changing needs (62 percent); and high training costs (54 percent). However, in 57 percent of local areas, these challenges did not affect their ability to guide participants toward training.
Local areas had difficulty helping employers fill certain jobs for a variety of reasons, including the low skills of some participants, according to GAO estimates. Eighty percent of local areas reported that employers had difficulty filling certain jobs, some of which can be considered "middle-skilled" jobs, such as welders and machinists, because they require more than a high-school diploma but less than a 4-year college degree. These local areas had difficulty supplying such workers because participants lacked relevant qualifications or the basic skills needed to participate in related training, among other reasons. To help participants improve their skills, DOL and other agencies have encouraged the use of career pathways approaches that combine job training with basic skills education and support services. According to these agencies, such approaches aim to enable participants to secure industry relevant certification and obtain employment within an occupational area and advance to higher levels of future education and employment in the area. However, little is known about the extent to which local areas are using career pathways approaches--or how they are using these approaches--specifically to prepare participants for middle-skilled jobs that employers have had difficulty filling. Without information on whether and how local areas are using these approaches to prepare participants for such jobs, DOL may not be well-positioned to help local areas use these approaches to better meet employers' needs.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOL collect and disseminate information on how local areas have used career pathways approaches to prepare workers for middle-skilled jobs that employers have difficulty filling. DOL agreed with our recommendation..

section_title: Letter
paragraphs: The economy is recovering from the recession, but employers are still  having difficulty filling certain jobs. The Department of Labor’s (DOL)  Workforce Investment Act (WIA) Adult and Dislocated Worker programs  are designed to help jobseekers find employment and to help employers  find the skilled workers they need. In program year 2011, these programs  provided job training to over 250,000 participants, or about 13 percent of  all WIA participants, according to DOL’s published data on participants  who exited the programs. WIA requires this training to be for occupations  that are in demand, also known as demand occupations. State and local  workforce investment boards (WIB) are responsible for administering WIA  activities in approximately 600 local areas across the country. To help  local workforce areas identify demand occupations, DOL funds a variety  of sources of labor market information that are provided by states. Local  areas may also undertake additional initiatives to identify such jobs.
Previous research has shown that local workforce areas identify demand  occupations and use that information to guide participants in the WIA  Adult and Dislocated Worker programs toward training. However,  questions have been raised about the labor market information that  informs these programs and the extent to which the programs are  positioned to help supply workers for jobs that employers have had  difficulty filling. The conference report accompanying the Consolidated   mandated that GAO assess the Adult and  Appropriations Act of 2012Dislocated Worker programs. This report addresses (1) how local  workforce areas have identified occupations that are in demand and how  they have guided participants toward training for them; and (2) what  challenges local workforce areas have faced in helping employers fill  certain jobs.
In designing the sample, we did not consider indicators of the size of the local areas  overseen by WIBs, such as population or amount of WIA funding received, because such  data were not readily available. and the challenges they faced in doing so; and 3) whether employers in  their areas have had difficulty filling certain jobs, and if so, the challenges  they have faced in helping employers fill those jobs. We did not conduct a  skills gap analysis because reliable data were not available. This report  does not contain all the results from the survey. The survey and a more  complete tabulation of the results can be viewed at GAO-14-20SP. We  also interviewed DOL officials as well as representatives from national  workforce associations and organizations with expertise in workforce  issues. See appendix I for more information on our scope and  methodology.
We conducted this performance audit from July 2012 to December 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The WIA Adult and Dislocated Worker programs provide employment  services to a wide range of participants. The Adult Program serves all  individuals age 18 and older, and the Dislocated Worker Program serves  individuals who have been or will be terminated or laid off from  employment, among others. The Adult Program prioritizes certain  services for recipients of public assistance and other low-income  individuals when program funds are limited. To allow individuals to  participate, both programs offer supportive services such as  transportation, childcare, housing, and needs-related payments under  certain circumstances. WIA requires that the Adult and Dislocated  Worker programs and other federally funded employment and training  programs provide services through one-stop centers—now called  American Job Centers—so that jobseekers and employers can find  assistance at a single location.
DOL’s Employment and Training Administration (ETA) administers the  Adult and Dislocated Worker programs and oversees their  implementation, which is carried out by states and localities. WIA requires  that each state have one or more local workforce investment areas, each  governed by a local workforce investment board (WIB). WIBs’  responsibilities include setting policy and overseeing service delivery for  the local workforce area, coordinating workforce investment activities with  economic development strategies, and developing relationships with  employers. Our previous work has identified innovative collaborations  used by some WIBs to help employers meet their needs for workers with  certain skills, among other needs. WIBs also select the entities to  operate American Job Centers and conduct oversight of the American  Job Center network.
The Adult and Dislocated Worker programs provide participants with a  variety of employment services. The programs provide three levels of  service: core, intensive, and training.
Core services include basic services such as job searches and labor  market information, and may be accessed with or without staff  assistance.  Intensive services include such activities as comprehensive  assessment and case management, which require greater staff  involvement. Intensive services are available to participants who are  unable to obtain or retain employment after receiving at least one core  service.
Training services include such activities as occupational skills or on- the-job training. In order to be eligible for training services,  participants must: 1) be unable to obtain or retain employment after  receiving at least one intensive service; 2) be in need of training; and  3) have the skills and qualifications to successfully complete the  training program, among other requirements.
To determine whether participants need training and to assess their skills,  WIBs may require them to complete certain activities. We previously  found that most WIBs required participants to complete skills  assessments or tests and gather information about the occupation for   After  which they wanted training before entering a training program. participants are determined to be eligible for training, DOL requires that  they select approved training providers in consultation with case  managers, but participants ultimately choose the training programs in  which they participate.demand occupations, which DOL interprets to include both currently   WIA requires that training be directly linked to  available jobs as well as occupations that are projected to grow in the  future.
In fiscal year 2013, the Adult and Dislocated Worker programs received a  total of about $1.9 billion in appropriations, down from about $2.5 billion in  fiscal year 2000 (a 24 percent reduction). funding allocations, states must meet agreed upon levels of performance.  WIA requires performance measures that gauge program results for  jobseekers in the areas of entered employment, employment retention,  earnings, and credential attainment. In addition, WIA requires measures  of customer satisfaction for jobseekers and for employers. States may  request waivers to report performance measures known as common  measures, which do not include credential attainment. WIA requires  states to negotiate performance levels with DOL and holds states  accountable for achieving these levels by tying them to financial sanctions  and incentive funding.
subsections: section_title: Sources of Information for Identifying Demand Occupations
paragraphs: Fiscal year 2000 appropriations are presented in nominal dollars and are not adjusted  for inflation. addition, local areas may undertake other initiatives to identify demand  occupations.
section_title: Local Areas Used Various Sources to Identify Demand Occupations and Required Participants to Complete Certain Activities before Entering Training
subsections: section_title: Most Local Areas Used a Variety of Sources to Identify Demand Occupations
paragraphs: We estimate that a majority of local areas used various sources of  information to identify demand occupations (see fig. 1). All but one of  these sources—online job postings, which are funded by private  companies and organizations—are funded primarily by DOL and provided  by state labor market information offices or state workforce agencies. The  sources local areas used the most were state job banks and industry and  occupational projections. Specifically, we estimate that nearly 90 percent  of local areas used state job banks to identify available jobs, and nearly  90 percent of local areas used industry and occupational projections to  identify occupations that are projected to grow.
Local areas found all of the sources of information they used to identify  demand occupations at least moderately useful. Local areas found  regional labor market analysts or economists the most useful of the  sources for identifying occupations that are projected to grow, although  they used them slightly less than industry and occupational projections,  as shown in figure 1 above. However, 28 percent of local areas did not  have access to such analysts or economists because their states did not  employ them or did not know whether their states employed them.
In addition to using these sources of information, most of which are  provided by state agencies, we estimate that 93 percent of all areas  undertook at least one local initiative to identify demand occupations. DOL encourages areas to use local initiatives to identify such  occupations. Specifically, DOL has issued guidance encouraging local  areas to use information from employers and economic development  partnerships, among other sources, to identify available jobs and  occupations that are projected to grow. As compared to other sources of  information, about 90 percent of areas used local initiatives for at least  one of the following reasons:
They more accurately reflected the current and expected needs of  local employers.
They provided more detailed information.
They provided more useful information.
The confidence interval for the estimate of 93 percent is 88 to 97 percent. the state labor market information office, state workforce agency, or state  or local economic development agency, while about one-third purchased  it from private companies. Of the local initiatives, areas found discussions  or focus groups with employers about their hiring plans or skill needs  most useful for identifying demand occupations.
Most of the areas that undertook local initiatives used WIA funding to  support them (see fig. 3). Some local areas also used other funding  sources—such as the Employment Service (Wagner-Peyser) program,  the state workforce agency, economic development agencies, and private  funding—to support these initiatives.
We estimate that, in 2012, a majority of local areas required participants  to complete certain activities before entering a training program (see fig.  4).case managers to discuss labor market information and training options;  2) complete a skills or interest assessment; or 3) obtain information about  the occupation for which they wanted training. When local areas did not  require participants to complete these activities, they typically encouraged  participants to complete them.
For example, many local areas required participants to: 1) meet with  In half of local areas, employment prospects were greater outside the  local area than inside the area. When employment prospects were  greater in neighboring areas, we estimate that 62 percent of local areas  guided participants toward training for those jobs to a large or moderate  extent.
section_title: Most Local Areas Overcame Challenges in Guiding Participants Toward Training
paragraphs: Most local areas faced challenges in guiding participants toward training,  but were still able to guide them toward training. We estimate that two- thirds of local areas faced major or moderate challenges related to (1)  participants’ lack of financial or work supports, such as child care or  transportation, and (2) participants’ lack of basic skills necessary to  participate in training, such as reading and math skills (see fig. 5). In 57  percent of local areas, however, the challenges they experienced did not  negatively affect their ability to guide participants toward training. In the  38 percent of local areas where these challenges had a negative effect,  more than half of areas guided fewer participants toward training, and  one-third guided fewer participants toward their first choice of training.
section_title: Local Areas Had Difficulty Helping Employers Fill Certain Jobs for a Variety of Reasons, Including the Low Skills of Some Participants
subsections: section_title: Challenges in Helping Fill Jobs Included Participants’ Lack of Qualifications and Basic Skills
paragraphs: Local areas cited a variety of reasons why they had difficulty helping  employers fill certain jobs in 2012, including participants’ lack of relevant  qualifications and basic skills, according to GAO estimates. Eighty  percent of local areas we surveyed said that employers had difficulty  filling jobs in certain occupations. Local areas most frequently cited the  following jobs as those that employers had difficulty filling: welders,  machinists, heath care occupations, computer occupations, and truck  drivers (see app. II for a complete list of occupations identified by WIBs  as those that employers had difficulty filling). In almost all cases, the  WIBs we surveyed also identified these as high growth occupations in  their local areas. According to workforce experts, some of these jobs,  such as welders and machinists, can be considered “middle-skilled” jobs  because they require more than a high-school diploma but less than a 4- year college degree. Other jobs that require a 4-year degree or more— such as some registered nurses—are considered “high-skilled” jobs. Of  the local areas that reported that employers had difficulty filling certain  jobs and were able to identify these jobs, 81 percent indicated that  employers used American Job Centers to try to fill these positions.  Among the most common reasons why these local areas had difficulty  supplying such workers were participants’ lack of relevant qualifications  and lack of the basic skills needed to participate in training (see fig. 6).  While DOL officials said that local areas should provide participants who  lack skills and work supports with the services they need to enter training  for middle-skilled jobs, they said it is not realistic to expect the Adult and  Dislocated Worker programs to train low-skilled participants for high- skilled positions that require advanced degrees, due to the skill level and  length of training required. However, DOL officials noted that these  programs do serve some middle- and high-skilled participants who obtain  employment after receiving only core services.
In addition, workforce experts cited decreased funding and the lack of a  performance measure on skill improvement as additional challenges. WIA  funding has decreased by 24 percent since 2000, and experts we  interviewed told us that local areas face difficult tradeoffs in making  decisions about spreading limited training funds over a broad population  of participants. Specifically, they said that because it takes more time and  resources to prepare low-skilled participants to enter training, local areas  must balance helping low-skilled workers improve their skills with serving  as many individuals as possible. In addition, experts noted that WIA does  not currently have a performance measure to track improvements in  participants’ basic skills. Workforce experts we interviewed said that  current measures reward states and local areas for placing participants  into employment quickly and do not reward basic skills improvements or  progress toward a degree or certificate. As a result of these challenges,  experts said that local areas may not guide participants toward training  programs for middle-skilled jobs, which tend to be longer and more  expensive than other training programs and typically require a certain  basic skill level. However, DOL officials disagreed that WIA performance  measures encourage local areas to place participants into employment  quickly. Also, DOL officials cited customer choice as a reason why some  low-skilled participants do not receive training for middle-skilled jobs.  Officials said that, when participants select training, they consider not only  the guidance they receive from case managers but also their own  interests and how long they can afford to be out of the workforce.
section_title: Little is Known About the Use of Career Pathways to Prepare Workers for Certain Hard-to-Fill Jobs
paragraphs: To help participants in WIA and other programs improve their skills, gain  employment, and progress in a career, DOL has coordinated with the  Departments of Education (Education) and Health and Human Services  (HHS) to fund and provide guidance for career pathways approaches,  which link basic skills education, occupational training, and support  services. According to these agencies, these approaches aim to enable  individuals to secure industry relevant certification and obtain employment  within an occupational area and to advance to higher levels of future  education and employment in that area. In 2010, DOL and Education  launched the Career Pathways Initiative by providing 1-year grants to  nine states and two tribal entities to develop these approaches. Based on  the lessons learned from these grants, DOL has taken steps to help other  states and local areas implement these approaches. Specifically, DOL  has issued guidance on developing career pathways systems; provided  technical assistance tools, such as webinars; shared promising practices;  and maintained a Career Pathways Community of Practice website. In  addition, DOL, Education, and HHS—as members of the Career  Pathways Working Group—have collaborated on joint technical  assistance efforts. For example, in 2012, DOL issued a joint letter with  Education and HHS to encourage local areas’ use of career pathways  approaches to help prepare low-skilled participants for employment. The  interagency working group plans to issue a national catalog of career  pathways toolkits in the future. DOL has also funded grants that are not  specifically targeted at career pathways approaches but can be used for  this purpose. In 2012, DOL awarded 26 Workforce Innovation Fund  grants to support innovative approaches for design and delivery of  employment and training services; 17 of the 26 grantees are using these  grants to develop career pathways approaches, according to DOL.
DOL and workforce experts find career pathways approaches promising,  and efforts are underway to gather more information about their  implementation and effectiveness. According to DOL officials and experts  we interviewed, career pathways approaches are a promising strategy  that state and federal agencies should adopt to support lower-skilled  individuals. Efforts are underway to collect information about how many  states are using career pathways approaches. DOL conducted a  preliminary review of the extent to which states are using these  approaches, and found that all but seven states are participating in some  type of career pathways initiative at either the state or local level. To  supplement the information collected in this initial review, the Career  Pathways Working Group plans to issue a national Request for  Information (RFI) during program year 2013 to obtain more detailed  feedback from states and local areas on career pathways successes and  any obstacles to implementation. In addition, HHS has contracted with  Abt Associates Inc. to conduct a 10-year evaluation of nine innovative  career pathways programs across the country, known as the Innovative  Strategies for Increasing Self-Sufficiency project. This evaluation will  measure the effects of these programs on participants’ employment,  earnings, and other outcomes, and will also include a study of program  implementation and operations as well as a cost-benefit analysis.
However, little is known about the extent to which local areas are using  career pathways approaches—or how they are using these approaches— specifically to prepare participants for middle-skilled jobs that employers  have difficulty filling. According to the guidance issued by the Career  Pathways Working Group, one essential component of career pathways  approaches is a specific focus on local workforce needs, and DOL  officials said that these approaches are one way to prepare participants  for middle-skilled jobs that employers have difficulty filling. Nevertheless,  DOL and other agencies do not plan to collect and disseminate  information on whether and how local areas are using these approaches  to prepare participants for such jobs as part of the RFI or the evaluation of  career pathways programs. According to federal government internal  control standards, agencies should have information on whether their  programs are meeting their intended objectives. whether and how local areas are using career pathways approaches to  prepare participants for middle-skilled jobs that employers have difficulty  filling, DOL may not be well-positioned to help local areas use these  approaches to better meet employers’ needs.
section_title: Conclusions
paragraphs: GAO, Internal Control Standards: Internal Control Management and Evaluation Tool,  GAO-01-1008G (Washington, D.C.: August 2001). the extent to which these programs are positioned to help supply workers  for hard-to-fill jobs, our work suggests that the programs have difficulty  supplying workers for certain types of middle-skilled jobs due to the low  skill level of some program participants, among other reasons. While it  appears that these participants are receiving training, their lack of basic  skills may be an obstacle to obtaining training that would prepare them for  the middle-skilled jobs that employers are having difficulty filling. Further,  both DOL and workforce experts emphasized that this problem exists in a  larger context, and that helping participants improve their basic skills can  involve a considerable investment of time and resources.
Given that a variety of factors affect whether skilled jobs are filled, it may  not be realistic to expect the Adult and Dislocated Worker programs to  fully meet all employers’ needs, but these programs can play an important  role. DOL has made efforts to help improve participants’ skills through  career pathways approaches and plans to gather more information about  their implementation. Despite these efforts, our findings suggest that  there are some local areas in which employers’ needs for certain types of  middle-skilled workers are not being met. Without additional information— beyond what it already plans to collect—on whether and how local areas  are using career pathways approaches to prepare participants for the  types of middle-skilled jobs that employers are having difficulty filling,  DOL may not be well-positioned to help local areas use these approaches  to better meet employers’ needs.
section_title: Recommendation
paragraphs: To help local areas better meet employers’ needs for skilled workers, we  recommend that the Secretary of Labor:  collect information on the use of career pathways approaches by local  areas specifically to prepare workers for the types of middle-skilled  jobs that employers have difficulty filling; and  to the extent such approaches are being used, disseminate  information about them, such as implementation challenges,  strategies, and results.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report and the e-supplement containing our  detailed survey results to the Secretary of Labor for review and comment.  DOL provided a written response (see app. III). DOL agreed with our  recommendation and did not provide any comments on the e-supplement.
Specifically, DOL’s response noted that the agency is committed to  promoting effective labor market information resources and using career  pathways approaches to prepare workers for career advancement and to  help employers gain a skilled workforce. DOL agreed that there is value in  learning more about how local areas are using career pathways  approaches. To identify promising career pathways approaches used by  local areas, DOL plans to (1) leverage existing investments in the  Workforce Innovation Fund and other DOL initiatives; (2) continue to  engage with federal and non-federal partners through the work of the  interagency Career Pathways Working Group; and (3) widely disseminate  information learned from studying these approaches through DOL’s  career pathways Community of Practice, other communication vehicles,  and technical assistance activities, as appropriate.
We are sending copies of this report to the appropriate congressional  committees and the Secretary of Labor. In addition, the report is available  at no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7215 or sherrilla@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our review focused on: 1) how local workforce areas have identified  occupations that are in demand and how they have guided participants  toward training for them; and 2) what challenges local workforce areas  have faced in helping employers fill certain jobs.
To address both objectives, we reviewed relevant federal laws and  regulations, as well as the Department of Labor’s (DOL) guidance to  states and local workforce areas. We also conducted a web-based survey  of a nationally representative sample of 200 workforce investment boards  (WIB) and interviewed nine WIBs that were not part of our survey sample  (see WIB Survey and WIB Interviews below). In addition, we interviewed  DOL officials as well as representatives from the National Association of  Workforce Boards, the National Association of State Workforce Agencies,  and the W.E. Upjohn Institute for Employment Research. To address the  challenges local areas face in helping employers fill certain jobs, we also  interviewed representatives from the National Skills Coalition and the  Center for Law and Social Policy (CLASP). We did not conduct a skills  gap analysis as part of this study primarily because reliable data on the  conditions associated with a skills gap were not available at the local or  state levels (see Skills Gap Analysis below).
subsections: section_title: WIB Survey
subsections: section_title: Design and Implementation
paragraphs: To address both objectives, we conducted a web-based survey of a  nationally representative sample of 200 WIBs. The questionnaire included  questions about which sources of labor market information local areas  used to identify demand occupations, the sources they found most useful,  how they guided participants toward training, and the challenges they  faced in guiding participants toward training in calendar year 2012. In  addition, we asked respondents whether employers in their areas have  had difficulty filling certain jobs, and if so, the challenges they have  experienced in helping employers fill those jobs.
We selected a stratified random probability sample from DOL’s list of all  578 WIBs nationwide as of November 2012, which included both local  and statewide WIBs. With this probability sample, each member of the  study population had a nonzero probability of being included, and that  probability could be computed for any member. We stratified the  population into 10 groups based on U.S. Census Bureau regions to  ensure geographic distribution. Each sample element was subsequently  weighted in the analysis to account statistically for all the members of the  population, including those who were not selected. In designing the  sample, we did not consider indicators of the size of the local areas  overseen by WIBs, such as population or amount of WIA funding received  because such data were not readily available. Our sample included 195  local WIBs and 5 statewide WIBs.
To minimize errors arising from differences in how questions might be  interpreted and to reduce variability in responses that should be  qualitatively the same, we conducted pretests with four local WIBs and  one statewide WIB over the telephone. To ensure that we obtained a  variety of perspectives on our survey, we selected WIBs with diversity on  the following criteria: 1) Employment and Training Administration (ETA)  region, 2) whether the WIB was located in an urban or rural area, and 3)  state and local unemployment rate. Based on feedback from these  pretests, we revised the questionnaire in order to improve the clarity of  the questions. An independent survey specialist within GAO also  reviewed a draft of the questionnaire prior to its administration. In  addition, we asked knowledgeable DOL officials to comment on the  questionnaire and incorporated their comments as appropriate.
After completing the pretests, we administered the survey. On March 7,  2013, we sent an e-mail announcement of the questionnaire to executive  directors and other knowledgeable representatives of the WIBs in our  sample, notifying them that our online questionnaire would be activated  within a week. On March 13, 2013, we sent a second e-mail message to  these WIB representatives in which we informed them that the  questionnaire was available online and provided them with unique  passwords and usernames. We also followed up by telephone and e-mail  with these WIB representatives as needed to clarify their survey  responses. We collected responses through April 15, 2013. We received  completed questionnaires from 148 WIBs, for a 74 percent unweighted  GAO-14-20SP response rate. Detailed survey results are available at: .
section_title: Analysis of Responses and Data Quality
paragraphs: We used standard descriptive statistics to analyze responses to the  questionnaire. All estimates based on the survey results are subject to  sampling error. Because we followed a probability procedure based on  random selections, our sample is only one of a large number of samples  that we might have drawn. Since each sample could have provided  different estimates, we express our confidence in the precision of our  particular sample’s results as a 95 percent confidence interval. This is the  interval that would contain the actual population value for 95 percent of  the samples we could have drawn. We used the results of the survey to  create estimates about the population of all WIBs nationwide. All  percentage estimates from the survey have their sampling margins of  error calculated at the 95 percent level of confidence.
To minimize nonsampling errors, and to enhance data quality, we  employed recognized survey design practices in the development of the  questionnaire and in the collection, processing, and analysis of the survey  data. For instance, as previously mentioned, we pretested the  questionnaire with WIBs to minimize errors arising from differences in  how questions might be interpreted and to reduce variability in responses  that should be qualitatively the same. We further reviewed the survey to  ensure the ordering of survey sections was appropriate and that the  questions within each section were clearly stated and easy to  comprehend. To reduce nonresponse, another source of nonsampling  error, we followed up by telephone and e-mail with WIB representatives  who had not responded to the survey to encourage them to complete it. In  reviewing the survey data, we performed automated checks to identify  inappropriate answers. We further reviewed the data for missing or  ambiguous responses and followed up with WIB representatives when  necessary to clarify their responses. On the basis of our application of  recognized survey design practices and follow-up procedures, we  determined that the data were of sufficient quality for our purposes.
section_title: WIB Interviews
paragraphs: Concurrently with our survey, we interviewed nine WIBs that were not  part of our survey sample to gather in-depth information about selected  survey topics. To select WIBs for interviews, we selected five states with  diversity regarding the following criteria: 1) state unemployment rate, 2)  geography, and 3) whether DOL officials identified the state as having  promising practices in using labor market information to guide participants  toward training. In applying these criteria, we ensured that we selected  one state with a statewide WIB (see table 2).
We then randomly selected two WIBs in each of the four states with local  WIBs (California, Michigan, New Jersey, and Virginia). In each of these  states, we asked representatives of the state WIB to classify each local  WIB in their state as either primarily urban or primarily rural. We then  randomly selected one primarily urban and one primarily rural WIB in  each state to interview, excluding those that were part of our survey  sample. We interviewed a total of eight local WIBs and one statewide  WIB. Of the eight local WIBs, five were primarily urban and three were  primarily rural.
section_title: Skills Gap Analysis
paragraphs: We did not conduct a skills gap analysis as part of this study primarily  because reliable data on the conditions associated with a skills gap were  not available at the local or state levels. According to officials from DOL’s  Bureau of Labor Statistics (BLS), the following conditions would be  associated with a skills gap in a particular occupation: 1) jobs remain  unfilled for a longer-than-normal time period, 2) wages are increasing,  and 3) unemployment is low. We explored the possibility of using existing  datasets to identify occupations that have rising wages and low  unemployment rates, and then asking the WIBs we surveyed whether  employers in their local areas had difficulty filling these jobs. However, we  concluded that reliable data on occupational unemployment rates are only  available on the national level. Because national data may not reflect  local labor market conditions, we did not incorporate such data into our  survey. Instead, we asked the WIBs we surveyed to list occupations that  employers in their area had difficulty filling and then coded their  responses to Standard Occupational Classification (SOC) codes. For  more information on the occupational coding, see appendix II.
Specifically, BLS’ Occupational Employment Statistics (OES) program  provides data on employment and wages by occupation, but officials told  us that local-level OES data on occupational wage growth may not be  reliable, and noted that data on occupational unemployment rates are not  available at the local level. In addition, we determined that state-level  data on occupational unemployment rates are not sufficiently reliable for  our purposes. Specifically, we were unable to use state-level data from  the Current Population Survey (CPS) to identify occupations with lower  than average unemployment rates because the standard error associated  with these estimates was over 20 percent in states with small  populations. While data from online job openings—such as that  published by The Conference Board—are used by some researchers to  estimate state-level unemployment by broad occupational categories,  these data are not available at the local level, and the reliability and  validity of the data is not known.
We conducted this performance audit from July 2012 to December 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Occupations That Employers Had Difficulty Filling in Calendar Year 2012
paragraphs: Our web-based survey of 200 workforce investment boards (WIB)  included a question asking if employers in their local area had difficulty  filling jobs in certain occupations in calendar year 2012 (see appendix I  for more information on the survey’s methodology). If WIBs answered  yes, they were asked to list up to five occupations that employers had  difficulty filling in open-ended boxes and also to indicate if the WIB  considered each occupation to be high growth in its local area. A total of  110 WIBs listed one occupation that was difficult to fill, 95 WIBs listed 2  occupations, 70 WIBs listed 3 occupations, 47 WIBs listed 4 occupations,  and 32 WIBs listed 5 occupations. We coded these open-ended  responses to the 2010 Standard Occupational Classification (SOC)  codes. The 2010 SOC system is used by federal statistical agencies to  classify workers into occupational categories for the purpose of collecting,  calculating, or disseminating data.
In order to code the survey responses to the most appropriate SOC code,  we searched the SOC titles or definitions or the related Direct Match Title  File (DMTF) on keywords or phrases provided in the open-ended survey  responses. Survey responses were coded to the most detailed SOC code  allowable based on the search results. In many cases, there was a one- to-one match at the detailed SOC level for many detailed occupations,  including registered nurses, welders, and machinists. Survey responses  that had a one-to-one match at a broad, minor group, or major group  SOC level were coded at that higher level, including engineers and  driver/sales workers and truck drivers.
Some survey responses resulted in multiple matches at the detailed SOC  level. If all of the matches were among related occupations, the response  was coded at the SOC level that incorporated all of the title matches. For  example, a survey response of web designer and programmer matched  to several detailed occupations that could be rolled up into the minor  group of computer occupations.
Some survey responses matched to multiple detailed occupations that  could not be rolled up to a higher SOC level. For example, a survey  response of accounting matched to accountants and auditors which is a  business and financial occupation, and also to bookkeeping, accounting,  and auditing clerks which is an office and administrative support  occupation. Responses were coded to more than one SOC code if May  2012 Occupational Employment Survey data indicated that workers were  employed in all of the matched occupations in the metropolitan statistical  area that contained the WIB. The most codes that were assigned to one  survey response were two codes. Of the 354 survey responses that were  coded, only 17 responses were assigned two SOC codes resulting in a  total of 371 SOC codes assigned.
Survey respondents frequently cited manufacturing (15) or advanced  manufacturing (14), which are considered industries, not occupations,  and matched to multiple SOC codes in different occupational groups. We  created separate categories for these two responses in order to preserve  the specificity of these responses given the relatively large number of  survey responses where WIBs indicated these jobs were difficult to fill.  Similarly, we created a separate code for the 20 responses that just  stated health care without specifying an occupation, and also for the 7  responses that just stated nurse without specifying registered nurse or  licensed practical nurse. Other survey responses that were vague or  could not otherwise be matched to SOC occupations were coded as  unknown.
A second analyst reviewed the SOC code matches and codes assigned  by the first analyst to see if the assigned codes had face validity. In cases  where there was disagreement or additional information was needed, the  two analysts met to reconcile differences. A third analyst independently  verified the counts of the codes listed in table 3 below. The number of  coded occupations represents those most frequently cited by WIBs that  responded to the survey question and cannot be generalized beyond that  population.
section_title: Appendix III: Comments from the Department of Labor
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Patrick DiBattista (Assistant  Director), Caitlin Croake, Hedieh Fusfield, Linda Siegel, and Paul Wright  made key contributions to this report. Also contributing to this report were  Carl Barden, Jessica Botsford, Ted Burik, David Chrisinger, David  Dornisch, Jill Lacey, Kathy Leslie, Mimi Nguyen, and Rhiannon Patterson.
section_title: Related GAO Products
paragraphs: Workforce Investment Act: DOL Should Do More to Improve the Quality  of Participant Data. GAO-14-4. (Washington, D.C.: December 2013).
Workforce Investment Act: Innovative Collaborations between Workforce  Boards and Employers Helped Meet Local Needs. GAO-12-97.  (Washington, D.C.: January 19, 2012).
Workforce Investment Act: Employers Found One-Stop Centers Useful in  Hiring Low-Skilled Workers; Performance Information Could Help Gauge  Employer Involvement. GAO-07-167. (Washington, D.C.: December 22,  2006).
Workforce Investment Act: Substantial Funds Are Used for Training, but  Little Is Known Nationally about Training Outcomes. GAO-05-650.  (Washington, D.C.: June 29, 2005).
Labor Market Information: Trends and Issues in Funding of State  Programs. GAO-03-336. (Washington, D.C.: December 20, 2002).