Workforce Innovation and Opportunity Act: Information on Planned Changes to State Performance Reporting and Related Challenges
Released: 2016-03-07T00:00:00

Letter

Background

Among other things, WIOA requires that DOL and Education collaborate to implement a common performance accountability system for the six core programs identified in the law (see table 1).

For the core programs, WIOA establishes six performance indicators on which states must report, with some exceptions (see table 2).

As shown in figure 1, programs are required to begin using the WIOA common performance indicators as of July 2016, two years after the law was enacted and just under six months after final regulations are due. DOL and Education officials are responsible for providing technical assistance and imposing sanctions on states that do not meet performance expectations. DOL and Education issued joint proposed regulations in April 2015 that, among other things, covered WIOA performance reporting, and in July 2015, issued a joint proposed performance reporting template. DOL issued additional proposed reporting details in September 2015. Final regulations, reflecting any changes based on public comments and agency review, will, among other things, form the basis for states’ implementation of WIOA.

Data Integration, Access, and Security

WIOA emphasizes the importance of a comprehensive system that provides integrated, seamless services to all job seekers and workers and effective strategies that meet employers’ workforce needs. DOL and Education note that the development of integrated data systems will allow for unified and streamlined participant intake (i.e., application and registration), case management, and service delivery; minimize the duplication of data; ensure consistently defined and applied data elements; facilitate compliance with reporting requirements; and provide meaningful information about core program participation to inform operations.

WIOA also requires states to use quarterly wage records, consistent with state law, to measure program performance (e.g., Unemployment Insurance (UI) wage records). Prior to WIOA, coordination among DOL- administered programs to share wage records for performance reporting was common, but access to and use of these data were less common for Education-administered programs. To help states obtain the data needed for performance reporting, particularly when participants are employed in another state or with the federal government, DOL currently funds three data exchange systems that allow agencies reporting on performance to access other states’ UI wage records or federal employment data: Wage Record Interchange System (WRIS); Wage Record Interchange System 2 (WRIS2); and Federal Employment Data Exchange System (FEDES). States compile and submit program performance data to federal systems either in aggregate or by individual participant, depending on the program.

Personally identifiable information (PII) for program participants, such as names, addresses and Social Security numbers (SSNs), among other things, is often shared between entities to link individual records across data systems and to collect program outcome information from state education and employment and training agencies. Because security breaches involving PII can be hazardous to individuals and organizations, protecting the privacy of sensitive information on program participants is a concern for DOL, Education, and its state and local partners. The Fair Information Practice Principles outline a set of eight key principles for protecting PII, including limiting the collection of personal information, disclosing how collected information will be used and obtaining consent from the individual, and protecting collected information with reasonable security safeguards against risks such as loss or unauthorized access. The eight principles are transparency, individual participation, purposes specification, data minimization, use limitation, data quality and integrity, security, and accountability and auditing.

Performance Reporting Process Prior to WIOA

While the performance reporting process is broadly similar across states, data system administration, the methods used to collect participant outcome data, and other specific practices vary by state and program. As we have reported previously, to fulfill program performance reporting requirements, state agencies generally collect participant identification and outcome data from local agencies and other sources, and then send performance reports to federal agencies (see fig. 2).

Officials in Selected States Anticipate Changes in How They Collect and Report Performance Data, Such As Integrating Data Systems, Though the Full Extent is Not Yet Known

Programs Vary in How they Currently Collect and Report Performance Data

To store participant data used for federal performance reporting, as well as for other program management purposes, each WIOA core program in the states we visited has its own case management data system (see table 3). Some of these systems were developed by the respective state or program and others were purchased from vendors. These data systems are used for case and program management and to compile program data used for federal performance reporting. The systems store participant information, such as personal identification; track program services, such as training received; and, in some cases, compile data on outcomes, such as employment or degree attainment. With the exception of the Texas Title I and Wagner-Peyser programs, the data systems are independent of each other, though in some instances, officials said data from the systems can be shared under established sharing agreements between agencies. For example, the VR program in Illinois has data sharing agreements in place with the Illinois agency that holds UI wage data and with the Social Security Administration to provide SSNs for data matching.

The performance reporting process varies based on program-specific circumstances, such as the source of reported participant outcome information and how automated the data collection and reporting processes are. Most programs in the three states we visited currently collect the employment outcome information reported to DOL and Education by matching participant data to state UI wage records. However, the VR program in all three of the states we visited currently collects employment information through individual outreach to program participants because program staff communicate regularly with participants and can readily obtain this information. In addition, the Adult Education program in New Hampshire collects employment outcome information by surveying participants. According to OCTAE, in the 2013- 14 performance year, the Adult Education programs in four additional states—Arizona, California, Hawaii, and New York—and the District of Columbia also collected participant employment information solely by surveying participants. As shown in case study 1 (see textbox), within the same state, programs may collect information differently. In New Hampshire, for example, Title I programs rely on data matching and the VR program relies on participant self-reporting and some data matching. Even when programs rely on similar sources of information, they sometimes use different mechanisms for obtaining and reporting participant information. For example, in New Hampshire Title I programs, a single official personally compiles participant outcome data and submits performance reports to DOL after the data are reviewed and approved by Title I management staff. According to the official, he obtains the outcome data from UI wage records through a standardized exchange with another state agency, New Hampshire Employment Security (see textbox). A data sharing agreement governs this process. Title I programs in Illinois similarly obtain participants’ employment information from UI wage records by exchanging data with the Illinois Department of Employment Security in accordance with a formal sharing agreement. In Texas, the reporting process is more automated for Title I programs, as participant data is automatically merged with UI wage data on a regular basis and then extracted into a federal data reporting system.

Case Study 1: Performance Reporting Processes Vary by Program in New Hampshire Title I Programs (DOL-administered): Local office staff enter information about participants, including services received, into the program’s case management data system. To collect employment and earnings information for those individuals who participated in the reporting period, an official with the Title I programs sends participant Social Security numbers (SSNs) to the New Hampshire Employment Security agency to match with the state’s Unemployment Insurance (UI) wage records. According to program officials, a data sharing agreement and standardized exchange process facilitate the sharing. The official also sends participant SSNs to the federal Wage Record Interchange System to obtain information about those who may be working outside New Hampshire. The state ultimately submits the compiled outcome data to DOL, excluding participants’ personally identifiable information.

Vocational Rehabilitation Program (Education-administered): Local office staff enter information about participants, including eligibility determinations and services received, into the program’s case management data system. Program staff contact participants about every six weeks to determine whether they need any additional services or have obtained stable employment and thus can have their cases closed. Through this individual outreach, staff also collect outcome data that the state reports to Education at an individual participant level, including personally identifiable information. The program also has a data sharing agreement with the New Hampshire Employment Security agency to match participants to employment and earnings data; some of the matched data are also used for federal performance reporting.

States We Visited Are Considering or Have Begun Integrating Program Data Systems

While officials in all three states we visited recognized they will need to collaborate more closely with core partners under WIOA, the extent to which they have already integrated their data systems, or plan to do so as a way to further this collaboration, varied. According to DOL and Education, WIOA encourages the development of integrated data systems across core programs to support integrated service delivery, case management, tracking participation across programs, and reporting performance, among other things. Based on our analysis of literature on information technology (IT) structures and discussions with state officials, data system integration can take a variety of forms, ranging from approaches that focus on sharing data between existing systems to approaches that consolidate existing systems (see fig. 3).

Among the three states we visited, Texas is in the process of integrating core program data systems, and at the time of our visits, Illinois and New Hampshire were considering what approach to take related to integration. An overall vision for data system integration is laid out in the preamble to the proposed regulations and DOL officials said they have been engaging with states about issues related to integration. However, DOL officials said that they do not specify what model of data system integration states should adopt because states’ unique circumstances, such as their data system structures and the extent of existing integration, may make certain approaches more viable than others. For example, among the three states we visited, Illinois’ core programs are housed in four separate agencies, whereas in Texas, Title I, Wagner-Peyser, and Adult Education programs are all housed within a single agency and the Vocational Rehabilitation program will be moving into that agency as of September 2016. These different structures and other factors, such as costs associated with integration, affect the states’ approaches to data system integration. DOL and OCTAE regional officials similarly observed that states they work with are taking a variety of approaches to data system integration, with some, such as Washington, Idaho, and Tennessee moving forward in different ways. For example, a DOL regional official said that Washington is exploring completely integrating its data systems and Idaho is examining linking or interfacing data from programs’ individual systems for performance reporting. An OCTAE area coordinator said that Tennessee has hired a contractor to develop its data system integration. Officials from two DOL regions also said that some states are waiting to move forward with data system integration until they have a more complete understanding of federal expectations from final regulations, and are instead focusing on other aspects of WIOA implementation in the meantime.

Texas

Texas is in the process of consolidating core programs into a single agency, the Texas Workforce Commission (TWC), and integrating program data systems at the performance reporting phase. The Adult Education program moved from the Texas Education Agency into TWC in September 2013, joining Title I and Wagner-Peyser programs. The Vocational Rehabilitation program will be joining the agency in September 2016. While this consolidation has occurred independent of WIOA, Texas officials said that it has put the state ahead of the curve in terms of data system integration. Texas is integrating its core program data systems similar to the “back-end integration” model shown in figure 3. As shown in figure 4, according to Texas officials, the case management data systems used for the Title I and Wagner-Peyser programs send participant data to a repository where UI wage data from Texas and other states—via federal exchanges, such as WRIS—are also sent. The data in this repository are merged by participants’ SSNs and uploaded to a federal performance reporting system. This automated delivery and compiling of data from multiple programs and sources in the repository is the key feature that represents “back-end integration” in Texas. The Adult Education program has its own case management data system and conducts performance reporting independently. However, Texas officials said that the automated delivery both of Adult Education data into the common repository, and of performance reports from the repository to Education, are under development. Texas officials said they also plan to integrate VR’s data system and reporting into this common repository after the program transitions into TWC.

Core programs in Illinois have not begun integrating data systems and state officials are currently considering what approach to take related to integration. Currently, core programs in Illinois each have their own independent data systems and state officials from all of the programs told us they were not yet clear what integration would mean for their data systems. At the time of our visit, officials from some of the programs said that the Illinois governor’s office was involved in providing direction to the core programs about what data system integration will look like in Illinois. In the meantime, some officials said they were considering ways to increase data sharing between core programs. Depending on the ultimate direction Illinois takes, integration may mean substantial changes for core programs or more minor adjustments to how programs share data.

New Hampshire

Officials from core programs in New Hampshire said they do not plan to develop major changes to integrate data systems because their existing, independent systems are functioning well and such changes would be too costly. Some officials discussed interfacing—automated data sharing between program data systems—as a possibility the state could explore, though they said that approach would also have associated costs.

Though the Full Extent of Changes to Data Collection and Reporting is Not Yet Known, Expected Changes Vary by Core Program

State and DOL regional officials we spoke with said states do not yet know the full extent of changes to performance data collection and reporting that will be needed under WIOA because regulations and reporting templates have not been finalized. However, based on information in the law and the proposed regulations (jointly issued by DOL and Education in April 2015), as well as conversations between states, programs, and the federal agencies, program officials recognize that some changes will be needed to collect data on and calculate the new WIOA performance indicators. In addition, DOL and Education jointly released a proposed reporting template in July 2015 that provided information about proposed definitions for core indicators of performance, some of the specific data elements that need to be collected, and how to report performance on certain core indicators. DOL also released additional proposed details about data element definitions and specifications in September 2015. Programs in all three states we visited are engaged in conversations within and across core programs. For example, when we visited the states, Illinois officials were communicating about integration and were working with a contractor to facilitate discussions across core programs, and New Hampshire officials were beginning to coordinate the development of the state plan for implementation by convening stakeholder discussions.

The DOL-administered programs have experience with collecting and reporting data for employment and earnings indicators, and Title I and Wagner-Peyser officials in all three states we visited said they expected few or relatively minor changes to their performance reporting. The employment and earnings indicators under WIOA are similar to those reported under WIA for the DOL-administered programs, though median earnings (instead of mean earnings) are now calculated in the 2nd quarter after program exit, and employment is measured in the 2nd and 4th quarters after program exit (instead of measuring employment in the 1st quarter and then employment retention in the 2nd and 3rd quarters after exit). However, the measurable skill gains indicator is new to Title I Adult and Dislocated Worker programs and these programs will have to develop ways of obtaining these data.

Because some WIOA reporting requirements are new to the Education- administered programs, Adult Education and VR programs may face more substantial changes to collecting and reporting performance data than the DOL-administered programs. For example, the median earnings performance indicator is new for the Adult Education program. In addition, the VR program has generally reported employment and earnings outcomes at a single point in time for program participants—at the point a participant exits the program (i.e., once a participant has stable employment for 90 days)—and not thereafter (as the new indicators will require). To report on the new measures, these programs may have to develop new ways to collect information about employment and earnings and may need increased access to UI data. According to DOL, WIOA is intended to expand the use of state UI wage records, consistent with state law, for reporting employment and earnings outcomes. Of the programs in the states we visited, only the Adult Education program in New Hampshire does not currently have access to earnings information from state UI data, due to a state law prohibiting it from collecting SSNs, according to program officials. As a result, the Adult Education program in New Hampshire plans to obtain earnings information for federal reporting by adding a question about earnings to its participant survey. The Adult Education programs in Illinois and Texas currently use UI data matching to report on employment outcomes, and thus already have access to a source of earnings data.

State VR officials in the states we visited said they will have to track participants longer under WIOA due to the requirement to report on outcomes after program exit, and may need new ways of collecting information after a participant leaves the program. Contractors who administer VR data systems in New Hampshire and other states said that under WIOA, VR agencies will have to change how they think about storing data and long-term maintenance because the new common performance indicators will require a reporting method that relies on case management over a significant period of time. As a result, states will have to shift from reporting on the status of a participant when he or she leaves the program to reporting the participant’s progress in the second and fourth quarters after exiting the program. For example, VR officials in Illinois and Texas said that while they currently have access to UI wage data, they generally do not use them for federal performance reporting because program staff maintain contact with participants to collect employment status updates and other information until participants exit the program. However, they said they plan to begin using UI wage data matching under WIOA to report on employment and median earnings. VR officials in New Hampshire said the program already uses UI wage data matching to collect some of its reported employment outcome data, though it does not currently have access to UI wage records for participants who find employment outside New Hampshire.

The effectiveness in serving employers performance indicator will be new to all of the core programs, and how it will be measured has yet to be finalized. The statutory deadline for developing the indicator is July 2016. DOL and Education proposed several potential approaches to measuring this indicator in the proposed regulations. According to DOL, the agencies are analyzing comments received on these proposals. Some state and DOL regional officials said WIOA’s requirement to report on effectiveness in serving employers, and the emphasis on serving the business needs of employers, has generally increased awareness of employers as customers, in addition to program participants. For example, some state officials we spoke with said they expect they will need to develop ways to coordinate outreach to and interaction with employers across programs. New Hampshire Wagner-Peyser officials said they work with an interagency business team that has representatives from each of the core program agencies in order to eliminate duplication of effort and prevent employers from being burdened with repetitive visits. Illinois Title I officials similarly said programs will have to coordinate outreach to employers and added that under WIOA, programs will have to increase employer engagement to ensure that education and training is focused on employer demand (i.e., where jobs are available).

After DOL and Education issue final regulations and the full extent of changes is known, programs will still have to incorporate the performance reporting changes into their data systems. As shown in case study 2 (see textbox), those programs that have vendor-purchased systems may collaborate with or rely on their contractors to incorporate changes, while those programs that have systems developed in-house will be responsible for incorporating changes. Regardless of whether program officials or a vendor are responsible, the ability of programs to process changes before WIOA implementation deadlines may vary. For example, New Hampshire VR officials said their data system vendor expects to continue implementing changes through summer 2016. Texas VR officials said that the last time they implemented major changes to performance reporting, implementing the changes took about two years to complete from when the changes were first discussed.

Case Study 2: Incorporating Data System Changes Using Vendors Versus In-House Staff New Hampshire Wagner-Peyser: Wagner-Peyser officials in New Hampshire expect their data system vendor to incorporate any needed changes resulting from WIOA in their case management system. According to officials, their vendor works with 30 state programs and will be implementing changes in an updated version of its case management data system product to reflect WIOA changes.

Texas Vocational Rehabilitation: According to officials, the Vocational Rehabilitation (VR) program in Texas is individually responsible for implementing data system changes. Information technology officials for the VR program said that to incorporate changes under WIOA they will receive a request from VR program officials, then develop, test, and incorporate changes into the program’s data system. The Texas officials said they had not yet received a change request from the VR program because officials are waiting for final WIOA requirements.

Federal officials are also considering ways to facilitate states’ WIOA implementation efforts. The preamble to the proposed regulations issued by DOL and Education in April 2015 states that the agencies intend to engage in a renegotiation of data sharing agreements with states (currently WRIS and WRIS2) to allow interstate wage matches for WIOA programs. DOL indicated it is considering the structure of the agreements moving forward, and will be working with Education and engaging the states in that development. In addition, the President’s fiscal year 2017 congressional budget justification for the Department of Health and Human Services (HHS) requests that select federal statistical and evaluation units be granted access to data from the National Directory of New Hires (NDNH) for, among other things, performance measurement purposes. The President’s Budget also proposes to provide state agencies with responsibilities for WIOA implementation the authority to match data with the NDNH for program administration purposes, including oversight and evaluation of these programs. Among other things, NDNH contains quarterly wage information on individual employees from state UI records and federal agencies. When we spoke with DOL and HHS officials about expanding access to NDNH, they told us that NDNH data could potentially be used for state WIOA performance reporting, though that would require that states be given legal access to NDNH data. However, when we spoke with HHS officials, they cautioned that the request is still in the early stages of consideration. Many additional details would have to be settled either in statute or in agreements between HHS and DOL, including how HHS would be reimbursed for use of the data and how the security of the data would be protected. For instance, according to the fiscal year 2017 proposal, HHS “would conduct robust privacy and security reviews before granting any state agency access to data.” When we spoke with them, HHS officials said they had not engaged in detailed discussions with DOL officials about these and other related issues.

Limited Information About Reporting Requirements and the Difficulty of Integrating Data Systems Challenge Implementation in Selected States

Limited Information on Details of Reporting Requirements Slowed Implementation Planning

Program officials across all three states we visited reported challenges in the early stages of implementing the requirements of WIOA, in part because they are awaiting final regulations and seeking more guidance from DOL and Education. Many details about the reporting requirements will not be known until final regulations are issued. Under WIOA, they were due in January 2016, though DOL officials informed us they anticipate issuing them in June 2016. Generally, early implementation efforts in the states we visited are based on the proposed regulations, and some state officials are concerned that without knowing what the final regulations will entail, they may not have enough time to implement all changes before the July 2016 deadline to begin using the new common indicators of performance. To avoid investing resources in implementing aspects of performance reporting that might later change with the publication of final regulations, some states are focusing on other WIOA efforts, such as working with new program partners and discussing ways to share participant data efficiently, according to officials we spoke with in two states and one DOL regional office.

DOL and Education have provided some interim guidance in order to assist states as they await final regulations. As we noted previously, DOL and Education issued joint proposed regulations in April 2015, as well as proposed performance reporting templates in the following months. In August 2015, DOL and Education also issued joint guidance on the vision for WIOA’s coordinated service delivery system and technical assistance resources available to states. DOL’s ETA has issued other Training and Employment Guidance Letters (TEGLs) that cover various issues, such as planning information related to Title I Youth program funds and WIOA implementation activities. OCTAE has also issued other guidance related to the Adult Education program, such as a program memorandum about the vision for the program within the workforce system, and guidance on the competition and award of program funds. In addition, DOL and Education officials told us they have worked together to provide webinars and have participated in joint conference calls with states. Some officials in the states we visited said that they have found this various guidance informative, particularly DOL’s TEGLs. However, receiving such guidance earlier would have been helpful, according to some program officials in two of the three states we visited.

DOL has also responded to states’ questions informally, according to state officials, while Education officials have said that until regulations are finalized they can provide only limited information outside the formal regulatory comment process. Officials for DOL-administered programs in all three states we visited said they contact DOL regional officials for assistance, and several officials in New Hampshire reported that their DOL regional office has specifically helped them figure out answers to questions about WIOA. For example, New Hampshire’s Wagner-Peyser officials stated that although their DOL regional officials understandably lack information on final regulations, they have been helpful in providing suggestions on how to plan for WIOA and encouraging officials not to wait until final regulations to move forward with implementation. Similarly, a DOL regional official said he meets with states every two weeks to discuss issues on the ground, such as training and WIOA definitions related to performance reporting. In contrast, Education officials from RSA and OCTAE have told states that they cannot answer questions that relate to ongoing rulemaking. For example, according to Education officials, they have responded to questions about performance indicators by referring states to the relevant provisions in WIOA and the proposed regulations.

States GAO Visited Reported that Resource Constraints and Data Security Issues Present Challenges for Integrating and Sharing Data

While WIOA encourages the integration of data systems, state capacity to pursue integration varies. In the states we visited, Texas is currently integrating its data systems, but officials in New Hampshire and Illinois said that the cost of data system integration will be a major challenge. For example, officials in every core program in New Hampshire reported that developing a new, integrated data system is cost-prohibitive. Similarly, a DOL regional official said that the majority of states in his region face challenges with data integration that include the expense of developing an integrated system and determining which systems will need to be redesigned. While states have some flexibility in how they can use their WIOA funding, DOL and Education officials told us that there are federal and state restrictions on the amount available for data integration and IT system upgrades.

To help fund data system integration, DOL and Education have requested funding to provide assistance to states. In the President’s fiscal year 2016 congressional budget justification for ETA, the agency asked for $37 million under the Workforce Data Quality Initiative, of which $30 million would be to “help states build integrated or bridged data systems to facilitate WIOA implementation… support building state-based wage data matching infrastructure to enable and/or streamline WIOA performance reporting.” Similarly, as part of its fiscal year 2016 budget request for the Adult Education program, Education asked for $1 million to provide technical assistance to states in the collection of new data elements and integration of data systems. Officials at both agencies told us that the funding requests were not intended to cover all data integration costs, but instead to provide some assistance to states. The Consolidated Appropriations Act, 2016, provided $6 million for the Workforce Data Quality Initiative and did not provide the full amount of funding Education requested to provide technical assistance to states. In the President’s Budget for fiscal year 2017, both agencies again requested funding to support states’ data integration efforts. Under the Workforce Data Quality Initiative, ETA asked for $40 million, of which $33 million would be to help states build integrated data systems to facilitate WIOA implementation. Under the Adult Education program, Education asked for $6 million to support the collection of new data elements and the integration of data systems.

Aside from cost, state and federal officials also identified other capacity concerns in this area, including limited staff expertise and antiquated IT systems. According to DOL officials, state efforts to integrate data systems may be challenged by, among other things, constraints in their ability to retain qualified IT staff due to state salaries that are low compared to those in the private sector. One DOL regional official said that due to budget cuts and layoffs in one of her states, the remaining staff lacks knowledgeable data programmers. Some program officials in New Hampshire expressed similar concerns about limited expertise to develop data system integration. For example, a New Hampshire Vocational Rehabilitation official reported that the state’s IT department has reduced its labor force and has limited staff capacity to support data system integration. According to DOL officials, many states have significantly antiquated and inflexible IT systems, making it challenging to support data system integration. Officials in two DOL regional offices similarly reported that rather than upgrading data systems, states have added patches over the years to respond to changes to reporting requirements, resulting in antiquated systems that make integration difficult.

As part of data system integration, WIOA encourages states to share and match data across programs, but officials reported that efforts to do so may be challenged by security concerns and undeveloped relationships. Officials in two of the three states we visited and a DOL regional official said that concerns over data security may impede states’ efforts to obtain participant information from core program partners. For example, a New Hampshire official said she would have concerns about sharing data with other agencies because she does not manage or oversee the data security standards they follow. These security concerns may be the result, in part, of undeveloped relationships between partners. According to a study by the Center for Regional Economic Competitiveness (CREC), one of the challenges to data sharing is a lack of trust between partners, particularly related to the intent and ability of the entity requesting data to use the data appropriately and protect it from any breach of confidentiality. The study identifies establishing mechanisms to build trust between entities interested in obtaining data from each other as one successful strategy for responding to this challenge and improving data sharing conditions.

Limitations on Available Data May Challenge the Quality of Performance Information

DOL and Education intend that WIOA’s common indicators of performance will, among other things, enable consistent outcome comparisons across states. WIOA requires the use of quarterly wage records, consistent with state laws, and the preamble to the proposed regulations emphasizes data-matching by participant SSN as a way of encouraging timely and accurate data for employment and earnings measures across the different programs and states. However, under federal law, state and local government agencies generally cannot require individuals to provide their SSNs as a condition of receiving program services. In addition, not all programs collect SSNs from some, or all, participants, thus making it difficult to match participant data. For example, according to OCTAE officials, Adult Education programs in some states have reported to them that they have difficulty collecting SSNs due to state privacy laws. OCTAE officials said that in the absence of SSNs, states can use other methods to collect data for performance reporting purposes, such as participant surveys. As shown in case study 3 (see textbox), other strategies for mitigating data gaps include the development of alternative methods of data matching, instead of using SSNs.

Case Study 3: Matching Participant Data Without SSNs In November 2014, we reported on challenges certain states experienced matching education and workforce program data as part of their efforts to develop statewide longitudinal data systems, which, among other things, allow states to follow individuals through their education and into the workforce. Officials in three of the five states visited as part of that review said state law or agency policy prohibit collecting a SSN in certain education programs. As a result, to match education and workforce data without a SSN, state officials reported they are developing algorithms to match individual records using other identifiers, such as an individual’s first name, last name, and date of birth. Wagner-Peyser officials in New Hampshire told us that they are able to match participants’ information based on other personally identifiable information (PII), such as name and date of birth, with UI data if a participant does not provide a SSN. Similarly, according to officials in Texas, the Adult Education program is able to match three of four elements (first name, last name, date of birth, or SSN) to obtain select education outcomes from the Texas Education Agency. However, Texas officials reported to us that relying on names for data matching may not be ideal, as names may change throughout an individual’s lifetime. DOL officials told us that states are making advances in obtaining participant information through secure portals for matching in which users are not able to see participants’ PII. In addition, DOL officials said they are aware of work groups that are developing ways to use advanced technology to do data matching without SSNs.

Even when programs have participant SSNs, they may face other constraints in their ability to match with UI data. According to the CREC study, the decision to share or withhold employment and wage data is largely based on how a state interprets UI data confidentiality statutes and regulations. The study found that the states that have been most successful in promoting data sharing are those with legislation that provides greater detail about both to whom and for what purposes confidential data may be disclosed.

Data gaps may continue to affect the quality of performance data reported in the Adult Education program. For example, state programs that do not collect SSNs, such as New Hampshire, may continue to utilize surveys or individual participant follow-up as a means to collect information on earnings. However, an Adult Education official in New Hampshire said that adding questions to participant outcome surveys to collect information for the WIOA median earnings performance indicator may discourage respondents from filling out the surveys and drop the response rate below OCTAE’s minimum acceptable level. Even when a state Adult Education program uses UI wage data matching, some participants do not provide SSNs—sometimes in large numbers. For example, Adult Education officials in Texas said their rate of matching participants to wage data using SSNs was about 55 percent. According to the statistical formulas OCTAE uses to calculate weighted performance outcomes for state Adult Education programs, participants whose employment outcome information is missing (i.e., whose data cannot be matched with UI wage records or who do not respond to surveys) are assumed to obtain employment at the same rates as those participants with data available. OCTAE officials said they have not assessed whether differences exist between populations that respond to surveys or who submit SSNs and those who do not respond or do not submit SSNs. Officials reported that such an analysis of nonresponse bias would not be technically feasible within each state before submitting performance data, and similarly would not be feasible for OCTAE staff since state data are reported in the aggregate.

DOL regional officials also raised concerns about the accuracy of performance data reported by states. DOL regional officials annually review a sample of case files from selected states to assess the validity of data states submit for performance reporting, and provide feedback to states based on their findings. According to DOL regional officials, these regularly recurring data validation reviews are conducted by the regional officials after states have submitted their performance data. Regional officials review the data element validation results, including the error rates and provide technical assistance to states as appropriate, according to DOL officials. Officials in two of the three DOL regional offices we spoke with said they have recently identified higher-than-desired error rates in performance data reported to the federal government in their routine reviews. For example, one regional official found that a state failed to update its data system and as a result, reported inaccurate information about the level of education completed by program participants. DOL officials told us that due to resource limitations, they have not performed any analysis to determine how such error rates may or may not affect the overall accuracy of outcome data. According to DOL officials, although the data validation reviews conducted by regional officials are not tied to sanctions, states are expected to use the findings of the reviews to improve the quality of data they submit for performance reporting in the future. OCTAE also conducts risk-based monitoring of state Adult Education programs, which covers data quality issues. One recent monitoring report found that a state program’s data system did not include required automated checks for errors and invalid data.

State Officials Identified Reporting Changes that May Be Especially Challenging, Such As Collecting and Measuring Outcome Data for Certain Populations

New WIOA reporting requirements will increase the responsibilities for Eligible Training Providers (ETPs) to track employment outcomes for training participants, including participants who are not enrolled in WIOA programs. State officials in all three states we visited said that these additional performance reporting requirements may discourage some ETPs from participating as WIOA training providers because the ETPs believe reporting on non-DOL-funded participants will be burdensome. Texas officials said that they have been working with ETPs in their state to find ways to reduce the burden while still complying with WIOA. In prior work, we identified similar concerns during the transition to WIA. However, according to DOL, to balance these concerns while still gathering enough information for participants and others to make informed choices about which training providers to use, states were ultimately able to obtain a waiver from certain reporting requirements for ETPs under WIA. Forty-one states obtained the waiver, according to DOL. DOL officials told us that they are aware of the challenges that states have expressed related to understanding where the burden lies for ETP performance reporting in both data collection and reporting. For example, DOL officials told us that not all ETPs have access to UI wage data. DOL officials said they solicited feedback on the proposed regulations and performance reporting templates in order to work through these issues, and that these details will be addressed in the final regulations and reporting templates, as well as in future joint guidance from DOL and Education on data access.

State Vocational Rehabilitation officials in two of the three states we visited noted that identifying and tracking participants receiving pre- employment transition services, as well as reporting on new WIOA performance indicators for these participants, may be difficult. WIOA emphasizes pre-employment transition services, which assist students with disabilities transitioning from secondary school into postsecondary education or employment. WIOA states that these services are to be provided to students with disabilities who are eligible or “potentially eligible” for services, but New Hampshire and Texas officials said they did not know how to identify, serve, and track “potentially eligible” participants. According to Education, each VR program grantee received terms attached to their fiscal year 2016 grant award describing the requirements for the provision of pre-employment transition services to all students with disabilities who are eligible or potentially eligible for VR services. In addition, the agency noted that RSA plans to provide more extensive guidance related to this issue in the future.

In Helping to Address Challenges States Are Facing, Federal Agencies are Balancing Benefits of Providing Timely Guidance and Considering Stakeholder Input

DOL and Education officials are largely aware of and, in some cases, actively working to help address the challenges the states we visited raised related to their ability to timely implement WIOA’s vision for performance reporting. In this process, the federal agencies are balancing the benefits of issuing guidance as early as possible with the importance of considering a broad array of stakeholder input via the ongoing rulemaking process. In some cases, the departments have tried to assuage states’ concerns through the guidance and ongoing dialogue discussed earlier. However, DOL and Education officials also acknowledge they likely cannot more fully mitigate states’ concerns until they issue final WIOA regulations and subsequently begin to develop more detailed implementation guidance. Among other things, the departments noted that in several areas, the proposed regulations sought suggestions and other input on key aspects of WIOA rather than spelling out explicit proposed rules (e.g., different ways to define program exit or ideas on how to measure the effectiveness in serving employers indicator), an approach that likely contributes to some states’ hesitancy to plan specific implementation steps or commit resources to them. According to DOL officials, the departments expect to issue final regulations in June 2016, which is after the date called for in WIOA. They attribute this delay to the scope and complexity of the new law, the volume of comments received on the proposed regulations, and the importance they are placing on a thoughtful and deliberative treatment of the many stakeholders’ perspectives and input.

Three States GAO Visited Reported No Intrusions into their Data Systems and that they Take Steps to Limit and Safeguard Personal Information Collected

Three States GAO Visited Reported No Intrusions into their Data Systems, Though Some Officials Cited Other Limited Security Breaches

Officials in all three states we visited reported that they were not aware of any outside intrusions into the electronic data systems for core WIOA programs during the years they worked in the programs. However, officials in Illinois and Texas reported other types of occasional data or security breaches that resulted in inappropriate exposure of PII for small numbers of people in limited circumstances. For example, Texas officials described a few instances of paper records with PII stolen from vehicles, a few times when unencrypted data were transferred electronically from local offices to the state, and an instance in which an employee included PII in internal emails. Officials said an electronic flag in the Texas email system currently identifies SSN-like numbers that appear in emails from program employees before they are sent so employees can remove any SSNs. An Illinois official reported that a staff member once placed paper PII in a non-secure trash container. Officials in another Illinois program said they found that some of the entities with which they shared data did not have certain security practices in place; the entities reported this information in response to a questionnaire regarding data security practices administered by the program as part of the data sharing application process. For example, some of the entities were faxing paperwork with PII. Officials said they have used the answers to this questionnaire to help tighten data security with certain sharing partners, including mandating the use of Secure File Transfer Protocol for data sharing.

Officials in all three states said they had protocols for reporting a breach of PII, often reporting the incident to a security officer or a program supervisor. For example, in the Illinois Vocational Rehabilitation program, officials told us that staff are to report data breaches to the program director and to the agency’s Chief Information Officer or to the secretary of the agency, and that such breaches must be reported to the governor and the state legislature within 24 hours. Illinois Adult Education officials also said they have procedures to investigate incidents and inform affected program participants. According to officials in New Hampshire’s Wagner-Peyser program, the program is required by the state to have a continuity of operations plan in place, which includes responding to data breaches. The response plan is coordinated by an official in the agency commissioner’s office who is responsible for overseeing data security.

Program Officials in the Three States Reported They Generally Limit the Collection and Use of Personally Identifiable Information

One of the ways that some programs reduce risks associated with the unauthorized disclosure of PII is to limit what is collected from participants, including involving participants in consenting to the collection of information, which is consistent with the Fair Information Practice Principles. According to the Privacy Act of 1974, government agencies generally cannot deny services to participants because of a refusal to provide SSNs. In 11 of the 12 programs across the three states, officials told us they allow applicants to opt out of providing their SSNs and still receive services if found otherwise eligible. One New Hampshire program only uses the last 4 digits of a participant’s SSN in its data systems and prohibits the transmission of the entire SSN. In addition, officials in at least one program in each of the three states also specified that they allow applicants to opt out of providing other types of PII, such as date of birth. While, as previously mentioned, many Adult Education program participants in the states we visited do not provide SSNs, Title I and Wagner-Peyser officials in New Hampshire and Texas told us that relatively few participants in their programs opt out of supplying their SSNs or other PII.

One New Hampshire program reduces the possibility of unauthorized PII disclosures by asking for personal identification to establish identity, but not storing the PII in any files. This practice is consistent with the Fair Information Practice Principle of minimizing data collection to only directly relevant information. Officials in the New Hampshire program said that staff ask applicants to provide identity documents (e.g., driver’s license, military separation paperwork, or Social Security card), and then staff conduct a visual verification of the documents without scanning them into their electronic data system or making copies for a paper record file. An official from one DOL region suggested that states could consider more frequently using visual verification of documents with PII without making copies. As a result, state programs would not store that PII.

Officials in all three states we visited also explained that they inform applicants for services about how the programs will use their PII, typically by asking applicants to read and sign a form indicating why their PII is being collected and how it will be used. This is consistent with the Fair Information Practice Principles. The forms explain the types of allowable uses of PII, among other things. For example, an information disclosure form for the VR program in New Hampshire informs participants that personal information is confidential and will not be released without their written permission, except when information must be released as required by federal authorities or in other circumstances specified on the form. The disclosure form states that information can be shared, with consent, with other related state programs with which the participant is involved and with entities involved in audit, evaluation, or research directly connected with administration of the program.

Officials in the Three States Reported Using Various Practices to Safeguard Information Collected

In prior work, we identified actions in the areas of security management and access controls, among others, as important for keeping data secure; these are also consistent with the Fair Information Practice Principle of protecting data through safeguards against risks. Officials in the three states we visited reported using various practices in these areas. For example, in terms of security management, state officials reported various activities related to the design and operation of their data systems, including: state data security protocols (together with federal guidance) that establish practices for handling both electronic and paper PII; data security training that staff must attend periodically, as well as staff being required to sign a confidentiality (also called nondisclosure) agreement; and data sharing agreements with another state agency that must have proper PII protection procedures included, with programs owning the PII reviewing current or proposed data sharing agreements, and entities requesting data sharing required to provide evidence of acceptable data security procedures.

Related to physical and data system access controls, state officials said that among other practices, they: use secured entry for buildings containing PII; keep data on a secure network and access to the network has levels of set privileges based on a staff member’s role; with staff computers password protected and locked after a certain time period of inactivity; and require staff to use encryption software to safeguard data whenever sending sensitive information via email or when transferring data.

In addition, the core programs in Texas have a periodic “penetration test” to try to identify possible vulnerabilities that outsiders could use to access the data. The central state entity for information resources conducts these tests individually for each state agency; staff from that entity attempt to hack into the data system as if they were outsiders. The Texas core programs also perform self-assessments of data security. New Hampshire’s central entity for information resources also conducts scans on data servers for vulnerabilities.

Agency Comments

We provided copies of this draft report to the Department of Education and the Department of Labor for review and comment. Both departments provided technical comments, which we have incorporated as appropriate. The Department of Labor provided a transmittal letter with its comments, which is reproduced in appendix I. The letter highlights DOL’s work with Education to create a unified vision of WIOA performance accountability through guidance, technical assistance, and rulemaking. The letter also cites DOL’s goal in the coming year of providing guidance on data validation, as well as other performance accountability topics such as wage record sharing and negotiations of performance targets. We also provided copies of this draft report to officials from the state programs we visited for their review, and incorporated their technical comments as appropriate.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Labor, the Secretary of Education, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7215 or sherrilla@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix II.

Appendix I: Comments from the U.S. Department of Labor

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Bill MacBlane (Assistant Director), Michael Kniss (Analyst-in-Charge), Kelsey Kreider, Ada Nwadugbo, and Paul Wright, along with James Bennett, Marisol Cruz, John de Ferrari, Alex Galuten, Danielle Giese, Farrah Graham, Amy Moran Lowe, Sheila McCoy, Walter Vance, and Charlie Willson made significant contributions to this report.

Related GAO Products

Workforce Innovation and Opportunity Act: Performance Reporting and Related Challenges. GAO-15-764R. Washington, D.C.: September 23, 2015.

Education and Workforce Data: Challenges in Matching Student and Worker Information Raise Concerns about Longitudinal Data Systems. GAO-15-27. Washington, D.C.: November 19, 2014.

Workforce Investment Act: Strategies Needed to Improve Certain Training Outcome Data. GAO-14-137. Washington, D.C.: January 31, 2014.

Information Security: Agency Responses to Breaches of Personally Identifiable Information Need to Be More Consistent. GAO-14-34. Washington, D.C.: December 9, 2013.

Workforce Investment Act: DOL Should Do More to Improve the Quality of Participant Data. GAO-14-4. Washington, D.C.: December 2, 2013.

Unemployment Insurance Information Technology: States Face Challenges in Modernization Efforts. GAO-13-859T. Washington, D.C.: September 11, 2013.

Information Technology: Department of Labor Could Further Facilitate Modernization of States’ Unemployment Insurance Systems. GAO-12-957. Washington, D.C.: September 26, 2012.

Federal Information System Controls Audit Manual (FISCAM). GAO-09-232G. Washington, D.C.: February 2, 2009.

Workforce Investment Act: Additional Actions Would Further Improve the Workforce System. GAO-07-105IT. Washington, D.C.: June 28, 2007.

Workforce Investment Act: Employers Found One-Stop Centers Useful in Hiring Low-Skilled Workers; Performance Information Could Help Gauge Employer Involvement. GAO-07-167. Washington, D.C.: December 22, 2006.

Workforce Investment Act: Labor and States Have Taken Actions to Improve Data Quality, but Additional Steps Are Needed. GAO-06-82. Washington, D.C.: November 14, 2005.

Workforce Investment Act: Substantial Funds Are Used for Training, but Little Is Known Nationally about Training Outcomes. GAO-05-650. Washington, D.C.: June 29, 2005.

Workforce Investment Act: Labor Should Consider Alternative Approaches to Implement New Performance and Reporting Requirements. GAO-05-539. Washington, D.C.: May 27, 2005.

Workforce Investment Act: States and Local Areas Have Developed Strategies to Assess Performance, but Labor Could Do More to Help. GAO-04-657. Washington, D.C.: June 1, 2004.

Workforce Investment Act: Labor Actions Can Help States Improve Quality of Performance Outcome Data and Delivery of Youth Services. GAO-04-308. Washington, D.C.: February 23, 2004.

Workforce Investment Act: Improvements Needed in Performance Measures to Provide a More Accurate Picture of WIA’s Effectiveness. GAO-02-275. Washington, D.C.: February 1, 2002.

Workforce Investment Act: Better Guidance Needed to Address Concerns Over New Requirements. GAO-02-72. Washington, D.C.: October 4, 2001.