Education and Workforce Data: Challenges in Matching Student and Worker Information Raise Concerns about Longitudinal Data Systems
Released: 2014-12-19T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: From fiscal years 2006 through 2013, the Departments of Education and Labor provided over $640 million in grants to states through the SLDS and WDQI grant programs. These grants support states' efforts to create longitudinal data systems that follow individuals through their education and into the workforce. Analyzing data in these systems may help states improve outcomes for students and workers.

GAO was asked to review the status of grantees' longitudinal data systems. This report examines (1) the extent to which SLDS and WDQI grantees match individual student and worker records and share data between the education and workforce sectors and (2) how grantees are using longitudinal data to help improve education and workforce outcomes. To answer these questions, GAO analyzed data from a 2013 survey conducted by the DQC. This survey collected information from states on data linkages among education and workforce programs and on how states use longitudinal data. In addition, GAO interviewed a nongeneralizable sample of five grantees, which were selected based on the progress they have made in matching data and on the funding they have received from the SLDS and WDQI programs. GAO also reviewed relevant federal laws and regulations.

GAO is not making recommendations in this report. GAO received technical comments on a draft of this report from the Department of Education and the Department of Labor, and incorporated them as appropriate.

section_title: What GAO Found
paragraphs: Over half of 48 grantee states that received a Statewide Longitudinal Data Systems (SLDS) or Workforce Data Quality Initiative (WDQI) grant have the ability to match data on individuals from early education into the workforce, based on GAO's analysis of 2013 Data Quality Campaign (DQC) survey data. The DQC is a nonprofit organization that supports the effective use of data to improve student achievement. In its survey, DQC collected self-reported information from states on their ability to match, or connect the same individual record, between the (1) K-12 and early education, postsecondary, and workforce sectors and between the (2) postsecondary and workforce sectors. However, as the match rate—that is, the percent of unique individual records reliably connected between databases—increases, the number of grantees able to match data decreases. GAO found that more grantees reported being able to match data among the education sectors than between the education and workforce sectors. Further, most grantees reported that they are not able to match data comprehensively. For example, only 6 of 31 grantees reported that they match K-12 data to all seven possible workforce programs covered by the DQC survey, which include adult basic and secondary education as well as unemployment insurance wage records. State officials cited several challenges to matching data, including state restrictions on the use of a Social Security number. Specifically, officials in three of five grantee states GAO spoke with said state law or agency policy prohibit collecting a Social Security number in K-12 data, which can make it more difficult to directly match individuals' K-12 and workforce records.

According to GAO analysis of the DQC survey data, grantees use some longitudinal data to inform policy decisions and to shape research agendas. All 48 grantees reported analyzing aggregate-level data to help guide school-, district-, and state-level improvement efforts. For example, 27 grantees said they analyze data on college and career readiness to help schools determine whether students are on track for success in college or in the workforce. Grantees also reported using longitudinal data to analyze outcomes for individual students. For example, 29 grantees reported that they produce early warning reports that identify students who are most likely to be at risk of academic failure or dropping out of school. Data from the DQC survey also show that 39 grantees reported developing a research agenda in conjunction with their longitudinal data systems.

section_title: Letter
paragraphs: Over the past decade, states have taken steps to create longitudinal data  systems that will allow them to track individuals’ participation in education  and workforce programs over time. Having individual-level longitudinal  data, such as information on individual characteristics and educational  attainment, may help answer long-standing questions about the  effectiveness of education and workforce programs in improving  outcomes for students as they move through school and into the  workforce, as well as for workers. These data could, for example, be used  to evaluate teacher performance, identify school graduation rates, or  determine the supply of skilled workers in relation to the demand for  workers.

From fiscal years 2006 through 2013, the federal government provided  over $640 million through two grant programs to support states’ efforts to  develop or enhance these longitudinal data systems. To date, the  Department of Education’s (Education) Statewide Longitudinal Data  Systems (SLDS) program has awarded approximately $613 million to 47  states, the District of Columbia, Puerto Rico, and the U.S. Virgin Islands.  The Department of Labor’s (DOL) Workforce Data Quality Initiative  (WDQI) grant program has awarded about $36 million to 33 states.

You asked us to review the status of grantees’ longitudinal data systems.  This report, in turn, examines: (1) the extent to which SLDS and WDQI  grantees match individual student and worker records and share data  between the education and workforce sectors and (2) how grantees are using longitudinal data to help improve education and workforce  outcomes.

To answer these questions, we analyzed state-level data from a 2013  survey conducted by the Data Quality Campaign (DQC). This survey  collected self-reported information from 49 states and the District of  Columbia on data linkages between the education and workforce  sectors—early education, K-12, postsecondary education, and workforce  programs—and on specific programs within these sectors. In addition, the  survey asked questions about how states analyze and use longitudinal  data. We reviewed the survey instrument, interviewed officials  responsible for administering the survey, and tested the data for obvious  inaccuracies and determined that these data are sufficiently reliable for  the purposes of this report. Our data analysis was limited to the 48 states  that (1) received a SLDS grant, a WDQI grant, or both and (2) responded  to DQC’s survey. These 48 states were awarded grants from fiscal year  2006 through fiscal year 2013, the most recent year for which we had  available data on grants awarded. In addition to our analysis of DQC  survey data, we interviewed Education and DOL officials. We also  conducted interviews with five grantees—Ohio, Pennsylvania, South  Dakota, Virginia, and Washington—to obtain more information about their  longitudinal data systems and, in particular, any challenges they faced in  developing and implementing them. During these interviews, we also  discussed how grantees have used longitudinal data to inform decision- making in education and workforce programs. We selected these  grantees because they differed in terms of the progress they have made  in establishing data linkages and on the federal funding they have  received from the SLDS and WDQI programs. Our findings from these  interviews are not generalizable to all grantees, but they provide  information on a range of challenges faced by grantees in developing and  implementing longitudinal data systems, as well as some examples of  how grantees are using longitudinal data. We also reviewed relevant  federal laws and regulations. While we spoke to some grantees and  reviewed grant requirements generally, we did not examine whether  states have met the conditions set forth in their grant agreements. See  appendix I for more information about our objectives, scope, and  methodology.

We conducted this performance audit from October 2013 through  November 2014 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Since the early 2000s, states have been building longitudinal data  systems to better address data collection and reporting requirements in  federal laws—such as the No Child Left Behind Act of 2001 and the  America Creating Opportunities to Meaningfully Promote Excellence in  Technology, Education, and Science Act (America COMPETES Act)— and to inform stakeholders about student achievement and school  performance. Federal, state, and private entities have provided funding  for these systems. For example, in addition to the SLDS and WDQI  programs, other recent federal grant programs including Race to the Top  and the Race to the Top-Early Learning Challenge may support states’  efforts.

The purpose of the SLDS grant program—administered by Education’s  Institute for Education Sciences, National Center for Education  Statistics—is generally to enable state educational agencies to design,  develop, implement, and expand statewide longitudinal data systems to  manage, analyze, disaggregate, and use individual student data. From  fiscal years 2006 to 2013, Education has awarded approximately $613  million in SLDS grants (see table 1). For each grant competition,  Education establishes the award period and range of grant amounts to be  awarded; SLDS grants have ranged anywhere from 3 to 5 years, with a  maximum award amount of $20 million per grantee. See appendix III for a  list of states that received SLDS grants and the amount of their awards.

Though the SLDS grant requirements have varied over time, states  generally could use SLDS funds to build K-12 longitudinal data systems  or to expand these systems to include data from other sectors, such as  early education, postsecondary education, or workforce (see table 2). The  long-term goal of the program is for states to create comprehensive “P20- W”—early learning through workforce—longitudinal data systems that,  among other things, will allow for states, districts, schools, educators, and  other stakeholders to make informed decisions and conduct research to  improve student academic achievement and close achievement gaps.

Under the WDQI grant program—administered by DOL’s Employment  and Training Administration—states are expected to fully develop their  workforce longitudinal data systems and then be able to match these data  with available education data to analyze education and workforce  outcomes.has chosen to award WDQI grants to states that have received an SLDS  grant or have a longitudinal data system in place. Among other  requirements, all grantees are required to develop or improve workforce  longitudinal data systems and enable workforce data to be matched with  education data to ultimately follow individuals through school and into the  workforce. DOL has provided funding for approximately $36 million in  WDQI grants to 33 states since fiscal year 2010 (see table 3). The award  period for each grant is 3 years. See appendix III for a list of states that  received WDQI grants and the amount of their awards.

section_title: Over Half of Grantees Could Track Some Individuals from Early Education into the Workforce, but Data Are Generally Limited
subsections: section_title: Over Half of Grantees Have the Ability to Match Some Individual Records from Early Education to Workforce
paragraphs: After analyzing data from DQC’s 2013 survey, we determined that over  half of grantees have the ability to match data—reliably connect the same  record in two or more databases—for some individuals from early   As shown in figure 1, individuals can  education and into the workforce.take different paths to move from early education into the workforce: (1)  via K-12 or (2) via K-12 and postsecondary. Regardless, as the match  rate—that is, the percent of unique student records reliably connected  between databases—increases, the number of grantees able to match  data between sectors decreases. For example, 31 of 48 grantees have  the ability to track individuals between all sectors from early education to  workforce to at least some degree, but only 6 grantees could do so at the  highest match rate.

Our analysis of the DQC survey data also shows that more grantees  match data among the education sectors than between the education and  workforce sectors, though—as was the case with matching data from  early education to workforce—the number of grantees that match data   For example, 43  decreases as the match rate increases (see table 4).grantees reported matching data between the K-12 and early education  sectors, and 31 grantees reported matching data between the K-12 and  workforce sectors at least to some degree; however, the number of  grantees that reported matching data between these same sectors drops  to 37 and 9, respectively, at a match rate of 95 percent or more.

Not all grantees are matching data between all sectors, which may  partially be the result of receiving grants with different grant requirements.  For example, all 20 grantees that received a fiscal year 2009 SLDS  ARRA grant were required to have longitudinal data systems that include  individual student-level data from preschool through postsecondary  education and into the workforce (see table 2). However, fiscal year 2012  grantees could choose from among three different grant priorities, so  some grantees may be focused on building a K-12 longitudinal data  system while others may be using their grant funds to link existing K-12  data to other sectors. In addition, grantees may have been in different  stages of developing their longitudinal data systems prior to receiving a  grant, which may help explain why some grantees are able to match data  between more sectors than others.

section_title: Most Grantees Are Not Able to Match Data Comprehensively
paragraphs: Programs Included in the Data Quality  Campaign Survey, 2013  Early education: early intervention, Head  Start/Early Head Start, special education,  state prekindergarten, subsidized child care  K-12: elementary and secondary education  Postsecondary institutions: less than 2-year  public, less than 2-year private not-for-profit,  less than 2-year private for-profit, 2-year  public, 2-year private not-for-profit, 2-year  private for-profit, 4-year and above public, 4- year and above private not-for-profit, 4-year  and above private for-profit  Workforce: unemployment insurance wage  records, unemployment benefits claim data,  Workforce Investment Act of 1998 (WIA) adult  or dislocated worker program, WIA youth  program, adult basic and secondary  education, Wagner-Peyser Act employment  services, Temporary Assistance for Needy  Families (TANF)

Of those grantees that match data, we found that few generally do so for  all of the possible programs between particular sectors (see sidebar),  based on our analysis of DQC survey data (see table 5). For example,  only 6 of 31 grantees reported that they were able to match data on all  seven programs between the K-12 and workforce sectors, which include  unemployment insurance wage records, unemployment benefit claims  data, Workforce Investment Act of 1998 (WIA) adult or dislocated worker  program, WIA youth program, adult basic and secondary education,  Wagner-Peyser Act employment services, and Temporary Assistance for  Needy Families (TANF).

We also analyzed DQC’s data to determine which programs are most  commonly matched by grantees between particular sectors (see fig. 2).  See appendix IV for a list of the specific programs matched by each  grantee.

section_title: Most Grantees That Match Data Also Share Them, but Few Share All Types of Data
paragraphs: Most grantees that match data also share data between sectors; that is,  they exchange at least one type of data (e.g., demographic, enrollment,  program participation, etc.) between two databases in at least one  direction, based on our analysis of DQC data. However, in general, few  grantees share all possible types of data (see sidebar). For example, only  3 of 36 states that match data between the postsecondary and workforce  sectors reported sharing all 10 types of data asked about by DQC, which  include information on postsecondary degree completion, earnings and  wages, and industry of employment, among others (see table 6).

Officials in all five grantee states we spoke with said matching K-12  education and workforce data is challenging without using a Social  Security number (SSN) that uniquely identifies an individual and, as a  result, some states may have greater difficulty tracking particular groups  of students over time. SLDS officials in three states—Ohio, Pennsylvania,  and Virginia—said collecting a SSN in K-12 education data is prohibited  either by state law or agency policy; in the other two states—South  Dakota and Washington—officials said collecting a SSN is optional and  whether to do so is determined at the district level.unique statewide student identifier is a technical requirement of the SLDS  grant program, states can choose the format of the identifier used.  Education suggested, in a November 2010 SLDS Technical Brief, that  states use a unique identifier distinct from a student’s SSN for privacy  reasons; however, Education also stated that states should maintain a   While establishing a  student’s SSN as a data element in order to link data between systems.  According to a 2010 report from the Social Security Administration’s  Office of the Inspector General, 28 states collect a SSN in K-12 education  data.

Unlike the SLDS program, in its evaluation criteria for WDQI grants, DOL  specifies that states use SSNs as a personal identifier, as they are  already in use throughout the workforce system. To match education and  workforce data absent a SSN, state officials said they are developing  algorithms to match individual records using other identifiers, which could  include an individual’s first name, last name, and date of birth. However, a  person’s last name can change, which Pennsylvania SLDS officials said  can make it difficult to reliably track individuals over time. Further, Ohio  WDQI officials explained that the absence of a SSN makes it particularly  difficult to track students who drop out of high school or to track high  school graduates who do not move on to the workforce. Similarly, Ohio  SLDS officials said tracking students that do not go on to postsecondary  education is a challenge because there is no readily available identifier to  determine any workforce participation by those individuals.

In four of five grantee states we spoke with, officials also cited data  governance as a challenge. Data governance is the exercise of decision- making and authority for data-related matters using agreed-upon rules  that describe who can take what actions with what information and when,  under what circumstances, and using what methods. SLDS grantees  are generally required to develop a governance structure involving both  state and local stakeholders that includes a common understanding of  data ownership, data management, as well as data confidentiality and  access. All WDQI grantees are expected to establish partnerships with  relevant workforce agencies and with state education agencies for the  purposes of data sharing. Pennsylvania and Ohio officials said it has not  been easy to get the various workforce agencies that maintain data on  individual workforce programs to share their data as the agencies often  operate independently from one another. As a result, Pennsylvania  officials said agencies are territorial about their data, making it difficult to  build consensus around developing a longitudinal data system. In Ohio,  officials said that each agency has to be approached separately to obtain  commitment to share data in a longitudinal system. Similarly, officials in  Virginia said collecting data on early education programs has been a  challenge as the data are scattered across different agencies. An official  from the Early Childhood Data Collaborative explained that it can be  easier to facilitate data matching between early education programs  under the purview of one agency, such as state prekindergarten and  special education, which are generally overseen by state educational  agencies in addition to K-12 data.

Based on our interviews with grantee states, state officials we spoke with  said they are in different stages of developing a data governance  structure. For example, Pennsylvania WDQI officials said they have not  yet established a formal data governance structure. In contrast, Virginia  officials have established a data governance structure; officials said they  spent 18 months working through the different priorities, cultures, and  agendas of the various agencies providing data to the longitudinal data  system.

State officials in all five grantee states we spoke with also said they have  had to manage public concerns about the purpose of data collection or  about data privacy. For example, in Ohio, SLDS officials told us there is a  lack of understanding about the value of building a longitudinal data  system; officials have had to counter misperceptions about what data are  being collected in the state’s longitudinal data system, what the data will  be used for, and why data need to be connected between the education  and workforce sectors. South Dakota officials said they have had to  respond to concerns from parents and other education stakeholders  about the privacy of longitudinal data.

Grantees have tried to provide information to the public about the  purposes of the data system and steps taken to safeguard information.  Forty-six grantees reported using outreach tools to communicate the  availability of the data to non-educator stakeholders, according to our  analysis of the DQC survey data. These grantees reported using  traditional outreach measures, which could include public service  announcements, press conferences and news releases, and posting  information about the data on the state education agency’s website. For  example, four of five grantee states we interviewed have web pages  dedicated to their longitudinal data systems. These web pages can  include overviews of the systems, answers to frequently asked questions,  trainings on how to use or access the data, and examples of research  studies that use the data. Further, 44 grantees reported on the DQC  survey that they take advantage of in-person opportunities, which could  include meetings, conferences, and presentations. Lastly, 35 grantees  reported using electronic or social media to promote the data, which could  include Facebook, Twitter, blogs, and webinars. In the context of  discussing the challenge of managing public concerns about data  collection or privacy, officials in three of the five grantee states we spoke  with specifically said they have provided information about how they  protect individual data. Pennsylvania SLDS officials said they took  considerable time to convey to parents and taxpayers the steps they are  taking to ensure data privacy. Similarly, Virginia officials from both grant  programs said explaining all of the precautions the state is taking with  respect to data privacy seems to help in reducing concerns. Ohio officials  said the state’s Department of Education has convened a new workgroup  to see if there are better ways to address misperceptions about data  collection and use.

Lastly, state officials cited the importance of federal funding to their efforts  to build their longitudinal data systems and expressed concerns about  sustaining their systems after their grants end. Officials we interviewed in  all five grantee states said they would not be as far along in developing  their longitudinal data systems without the federal funding provided  through the SLDS and WDQI programs. For example, officials in  Washington said they used their initial SLDS and WDQI grants to focus  on building their K-12 data system and workforce systems, respectively.

They said the second SLDS grant they received was instrumental in  building a P-20W system to connect data between all sectors. Ohio  officials said the SLDS funds have provided, among other uses, critical  funding for further development of the longitudinal data system,  technological updates, and access to technical assistance. However,  officials in all five grantee states also expressed concerns about  sustaining the systems moving forward. For example, officials in Virginia  said they have created a legislative committee to focus on sustainability  efforts and will need to request additional funding to keep the system  sustainable. Officials in Pennsylvania said they are trying to leverage the  existing technical infrastructure and use other available resources, but it  is difficult to find funding for their workforce data efforts.

section_title: Longitudinal Data Analysis Has Informed State and Local Policy Making and Helped States Shape Their Research Agendas
subsections: section_title: Grantees Reported Using Some Longitudinal Data to Inform Decision and Policy Making
paragraphs: According to our analysis of the DQC survey data and our interviews with  selected states, SLDS and WDQI grantees use longitudinal data to  examine education outcomes and to inform policy decisions. All 48  grantees responded that their state educational agency uses the data to  analyze aggregate education outcomes (see fig. 3). For example, the  three most common types of analyses are related to high school  feedback, cohort graduation or completion, and growth (i.e., changes in  the achievement of the same students over time). These aggregate data  are used to analyze a particular cohort of students and develop  information on students’ outcomes over time. They also help guide  school-, district-, and state-level improvement efforts. For example,  officials from three of the five grantee states we interviewed told us they  have used the data to assess kindergarten readiness for children who  attended state early education programs. Also, 27 grantees responded to  the DQC survey that they use the data to analyze college and career  readiness. More specifically, to better understand the courses and  achievement levels that high school graduates need to be successful in  college, Virginia followed students who graduated from high school from  2006 to 2008 and analyzed enrollment and academic achievement  patterns for different groups of students. According to agency officials in  Virginia, this analysis resulted in changes to the course requirements for  graduation.

In addition to examining education outcomes, states also use longitudinal  data to assess how cohorts of students fare once they are in the  workforce. Washington’s Education Research and Data Center, a state  center dedicated to analyzing education and workforce issues across the  P-20W spectrum, has published several studies examining workforce  outcomes for high school and college graduates. For example, one study  compared earnings for workers with bachelor’s degrees from Washington  state colleges and universities to earnings of workers with only diplomas  from public high schools.

In addition to analyzing aggregate student outcomes, grantees also  indicated that they analyze individual-level student outcomes. Our  analysis of DQC survey data shows that 45 of 48 grantees examine  outcomes for individual students (see fig. 4). Student-level data provide  teachers and parents with information they can use to improve student  achievement. For example, 32 grantees reported that the data are used in  diagnostic analysis, which help teachers identify individual students’  strengths and academic needs. Also, 29 grantees responded to the DQC  survey that they produce early warning reports, which identify students  who are most likely to be at risk of academic failure or dropping out of  school. For example, Virginia’s early warning report shows demographic  and enrollment information about an individual student; flags for warning  indicators such as attendance, GPA, and suspensions; and a record of  interventions the school has taken to help the student (see fig. 5). Further,  officials in three of the grantee states we interviewed told us that  educators have access to student-level analyses. In Pennsylvania,  teachers can use an educator dashboard, which includes longitudinal  data, to determine the educational needs of their students and adjust their  teaching plans.

Forty-one of 48 grantees reported to the DQC that they use longitudinal  data to inform policy and continuous improvement efforts. Specifically,  grantees reported that they use the data to inform school turnaround  efforts (34 grantees), evaluate intervention strategies or programs (14  grantees), or identify and reward schools that demonstrate high growth  (27 grantees), among other things. Officials in three of five grantee states  we spoke with provided more specific examples of how they use or plan  to use longitudinal data to inform their efforts. Ohio officials told us they  used longitudinal data to study students in remediation to help develop a  remediation policy. They also said they have been working on a  workforce success measures dashboard to compare outcomes across  state programs. For example, the dashboard will allow policy makers to  assess how successful the state’s adult basic education program is  compared to the state’s vocational education program. Pennsylvania  officials told us they will develop a similar dashboard. Washington state  officials told us that longitudinal data helped address a concern in the  state legislature about whether math and science teachers were leaving  to work in the private sector. Researchers identified common teacher and  school district characteristics associated with teachers who left for  employment in other fields and found that math and science teachers did  not leave the field at a higher rate than other teachers. Officials told us  that this analysis prompted the state legislature to focus its attention on  improving the recruitment of math and science teachers rather than  improving retention.

While many grantees reported on the DQC survey that they use  longitudinal data to analyze outcomes for students and workers and to  make policy decisions, officials from all five grantee states we interviewed  told us that these analyses are limited because they are still developing  their longitudinal data systems. In addition, only three of these states— Ohio, Virginia, and Washington—are conducting education to workforce  analyses. Officials in Pennsylvania and South Dakota said they plan to do  this type of analysis, but only after they finish putting all the education and  workforce data into their systems and matching these data.

section_title: Most Grantees Have Developed a Research Agenda in Conjunction with Their Longitudinal Data Systems
paragraphs: Data from the 2013 DQC survey show that 39 SLDS or WDQI grantees  have developed research agendas articulating and prioritizing research or  policy questions that can be answered with longitudinal data. These  research agendas were developed in partnership with higher education  institutions, independent researchers, or others. Of the five grantee states  we interviewed, only Virginia and Ohio have fully developed their  research agendas. Pennsylvania, South Dakota and Washington officials  told us they are in the process of doing so. State officials shared two  approaches for creating these agendas. Under the first approach,  stakeholders from various state agencies comprise a committee that  identifies research questions. Virginia took this approach and drafted a list  of “burning questions” to answer using longitudinal data. Officials in  Virginia explained that they purposefully kept the agenda broad so that  the questions will remain relevant over the long term. Washington’s  Education Research and Data Center has similarly developed a list of  critical questions it would like to answer using longitudinal data. Under the  second approach, state agencies use information requests and  stakeholder feedback on sample reports to shape the research agenda.  For example, officials from the South Dakota Department of Education  told us they have solicited feedback after training districts on the data and  reviewed requests from the governor’s office and state legislators. They  also told us that they are following the number of hits for individual reports  on the state’s Department of Education’s electronic portal.

Forty-three of 48 grantees reported that they have a process by which  researchers who are not employees of the state can propose their own  studies for approval, according to the 2013 DQC survey data. Four of the  grantee states we interviewed have established a formal request process  for researchers who would like to access longitudinal data and the fifth  state is reviewing its protocols and expects to develop a formal  application process. Officials in two grantee states told us that the request  process is intended to streamline access to the data and make it easier  for researchers to seek approval for data requests. In addition, officials in  Ohio told us that when researchers apply for access to Ohio’s data, they  must include information in their application about how the study will meet  the state’s research priorities.

section_title: Concluding Observations
paragraphs: Since fiscal year 2006, the federal government has made a significant  investment—over $640 million in SLDS and WDQI grant funds—to help  states build P20-W longitudinal data systems that track individuals from  early education and into the workforce. The different grant requirements  for linking data between sectors may have contributed to states being in  different stages of developing their longitudinal data systems. That is,  some grantees are just building their K-12 longitudinal data systems while  others are matching data between education and workforce sectors. It  remains to be seen whether all grantees will ultimately achieve the long- term goal of developing complete P20-W longitudinal data systems or  how long that will take, particularly in light of unresolved concerns about  limitations to matching data using a Social Security number and  sustainability. Further, even among those grantees that can match data  between sectors, most can only do so for a limited number of programs or  data types. As grantees continue to refine their systems, maximizing the  potential of these systems will rest, in part, with the ability to more fully  match information on specific programs and characteristics of individuals  that could help in further analyzing education and workforce outcomes.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to Education and DOL for their review.  Each provided technical comments, which we incorporated as  appropriate.

As agreed with your office, unless you publicly announce its contents  earlier, we plan no further distribution until 30 days after the date of this  letter. At that time, we will send copies of this report to the appropriate  congressional committees and the Secretaries of Education and Labor. In  addition, the report is available at no charge on GAO’s website at  http://www.gao.gov. If you or your staff have any questions about this  report, please contact me at (617) 788-0580 or nowickij@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix VII.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this report were to examine: (1) the extent to which  Statewide Longitudinal Data Systems (SLDS) and Workforce Data Quality  Initiative (WDQI) grantees match individual student and worker records  and share data between the education and workforce sectors; and (2)  how grantees are using longitudinal data to help improve education and  workforce outcomes.

To answer our objectives, we analyzed state-level data from a 2013  survey conducted by the Data Quality Campaign (DQC), a nonprofit  organization that works with state officials and others to support the  effective use of data to improve student achievement. DQC’s survey  focused on 10 “State Actions” the DQC has developed to ensure effective  data use (see table 7). DQC has conducted this annual survey since  2005. The survey data include self-reported information on how data are  matched and shared between the early education, K-12, postsecondary  education, and workforce sectors, as well as information on specific  programs within these sectors, how states analyze and use the data, and  who has access to the data.

To conduct the survey, DQC used an online tool to collect information and  invited the governor’s office in all 50 states and the District of Columbia to  participate. According to DQC, the governor’s office is in the best position  to bring stakeholders together to respond to the survey. As part of their  survey response, states were asked to provide documents or website  links as evidence of having specific policies or reports. After survey  responses were received, DQC worked with each state to ensure the  information reported was as accurate as possible.

We analyzed data from eight survey questions (see table 8 in appendix II)  to determine the extent to which SLDS and WDQI grantees match  individual records and share data among the education sectors and  between the education and workforce sectors. For the purposes of our  report, a grantee is one of the 48 states that received a SLDS grant, a  WDQI grant, or both and responded to the 2013 DQC survey. We  considered the District of Columbia to be a state. We excluded Alabama,  New Mexico and California from our review because neither Alabama nor  New Mexico received a SLDS or a WDQI grant and because California  chose not to participate in DQC’s 2013 survey. We excluded the U.S.  Virgin Islands and Puerto Rico because, while these territories received  SLDS grants, DQC did not include them in its survey. We analyzed data  on SLDS and WDQI grantee states because the SLDS and WDQI grant  programs provide federal funds for developing longitudinal data systems  and are complementary.

We considered a grantee as matching data between sectors if a grantee  matched data from at least one program between sectors (for a list of  programs included in the DQC survey, see questions 1, 4, 7, and 10 in  table 8 in appendix II). We considered a grantee as sharing data if a  grantee matched data according to our definition and also reported  exchanging at least one data element between sectors, in either direction  (for a list of data elements, see questions 2, 5, 8, and 11 in table 8 in  appendix II). We also analyzed data from another twelve survey  questions to identify how grantees are using longitudinal data to help  improve education and workforce outcomes (see table 9 in appendix II).

We conducted a data reliability assessment by reviewing the survey  instrument and related documentation, interviewing officials responsible  for administering the survey, and testing the data for obvious  inaccuracies. We determined that these data are sufficiently reliable for  the purposes of this report.

In addition to our analysis of DQC survey data we conducted interviews  with a nongeneralizable sample of five grantees as well as relevant  federal agencies and nonprofit organizations. During our interviews with  the five grantee states—Ohio, Pennsylvania, South Dakota, Virginia, and  Washington—we asked grantees to identify challenges they faced in  building and implementing longitudinal data systems and discussed how  grantees have used longitudinal data to inform decision-making in  education and workforce programs. We selected these grantees based  on factors including the differing levels of progress they have made in  establishing data linkages and the federal funding they have received  from the SLDS and WDQI programs.

Within each state, we spoke with relevant K-12, workforce, postsecondary  education, and early education officials. We also interviewed officials at  Education, DOL, and the Department of Health and Human Services to  obtain information about their roles in helping states build longitudinal  data systems. In addition, we spoke with officials from nonprofit  organizations to obtain their views on states’ implementation of  longitudinal data systems. These stakeholder organizations included the  Early Childhood Education Collaborative, the State Higher Education  Executive Officers Association, and the Workforce Data Quality  Campaign. Finally, we reviewed relevant federal laws, regulations,  requests for applications, and solicitations for grant applications to  understand the requirements of these grants.

section_title: Appendix II: Data Quality Campaign (DQC) Survey Questions
paragraphs: As explained in appendix I, we analyzed data from DQC’s 2013 survey to  answer our research objectives. Table 8 and table 9 show the specific  questions we analyzed from DQC’s survey instrument. For some  questions, DQC allowed states to select “other” as a response; we  excluded these “other” responses from our analysis.

section_title: Appendix III: Statewide Longitudinal Data Systems (SLDS) and Workforce Data Quality Initiative (WDQI) Grantees

section_title: Appendix IV: Specific Programs Matched by Statewide Longitudinal Data Systems (SLDS) and Workforce Data Quality Initiative (WDQI) Grantees between Sectors
subsections: section_title: Grantee Alaska

section_title: Subsidized Child

section_title: Less Than

section_title: Less Than

section_title: Unemployment

section_title: Unemployment

section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Janet Mascia, Assistant Director,  Jennifer Gregory, and Nisha R. Hazra made key contributions to this  report. Also contributing to this report were Deborah Bland, David  Chrisinger, Alex Galuten, Amanda Miller, Jeffrey G. Miller, Mimi Nguyen,  Yunsian Tai, and Walter Vance.

section_title: Glossary
paragraphs: This glossary is provided for reader convenience. It is not intended as a  definitive, comprehensive glossary of related terms.
subsections: section_title: Aggregate level data
paragraphs: Group statistics (numbers, percentages, averages, etc.) based on  individual student data.

section_title: College and career readiness reports
paragraphs: Reports designed to identify students who are on track for readiness or  success in college or careers.

section_title: Data governance
paragraphs: The exercise of decision-making and authority for data-related matters  using agreed-upon rules that describe who can take what actions with  what information and when, under what circumstances, and using what  methods.

section_title: Diagnostic report
paragraphs: Information on individuals designed to identify each student’s strengths  and academic needs.

section_title: Early education sector
paragraphs: Programs that serve children prior to kindergarten. Programs include:  early intervention, Head Start/Early Head Start, state prekindergarten,  special education, and subsidized child care.

section_title: Early warning report
paragraphs: A report designed to identify students who are most likely to be at risk of  academic failure or dropping out of school.

section_title: Feedback report
paragraphs: Information on outcomes for students after they graduate from a school or  district.

section_title: Growth report
paragraphs: A report that shows changes in the achievement of the same students  over time.

section_title: K-12 sector
paragraphs: Elementary and secondary education.

section_title: Postsecondary education sector
paragraphs: Institutions of higher education. Types of institutions include: less than 2- year public, 2-year public, 4-year and above public, less than 2-year  private not-for-profit, 2-year private not-for-profit, 4-year and above  private not-for-profit, less than 2-year private for-profit, 2-year private for- profit, and 4-year and above private for-profit.

section_title: Predictive report
paragraphs: A report that shows how students’ success later in the  education/workforce pipeline is related to the status of the same students  earlier in the pipeline.

section_title: Matching
paragraphs: Reliably connecting the same individual record in two or more databases.

section_title: Match rate
paragraphs: The percent of unique individual records reliably connected across  databases.

section_title: Sharing
paragraphs: Exchanging data between two databases, in either direction. Data  elements that could be shared between early education and K-12 include:  demographic, family characteristics, program participation, child-level  development data; between K-12 and postsecondary: demographic,  college readiness assessment scores, college placement assessment  scores, high school transcript data, postsecondary enrollment,  postsecondary remediation status, postsecondary progress,  postsecondary credits earned, postsecondary enrollment intensity,  postsecondary outcomes; between K-12 and workforce: demographic,  enrollment, transcript data, earnings and wages, employment status,  occupation, industry of employment; between post-secondary and  workforce: demographic, enrollment, transcript data, financial aid,  postsecondary degree completion, earnings and wages, employment  status, occupation, industry of employment.

section_title: Workforce sector
paragraphs: Programs that serve individuals in the workforce. Programs include: adult  basic and secondary education, TANF, unemployment benefits claims  data, unemployment insurance wage records, Wagner-Peyser Act  employment, WIA adult or dislocated workers program, and WIA youth  program.


Question-summary annotations: questions: question: What did GAO determine about grantee states?
answer: Over half of 48 grantee states that received a Statewide Longitudinal Data Systems (SLDS) or Workforce Data Quality Initiative (WDQI) grant have the ability to match data on individuals from early education into the workforce, based on GAO's analysis of 2013 Data Quality Campaign (DQC) survey data.

question: What is the DQC?
answer: The DQC is a nonprofit organization that supports the effective use of data to improve student achievement.
child_questions: question: What did the DQC do in its survey?
answer: In its survey, DQC collected self-reported information from states on their ability to match, or connect the same individual record, between the (1) K-12 and early education, postsecondary, and workforce sectors and between the (2) postsecondary and workforce sectors.
child_questions: question: What problems arise with match rates?
answer: However, as the match rate—that is, the percent of unique individual records reliably connected between databases—increases, the number of grantees able to match data decreases.

question: What did GAO discover about match rates?
answer: GAO found that more grantees reported being able to match data among the education sectors than between the education and workforce sectors.   Further, most grantees reported that they are not able to match data comprehensively. For example, only 6 of 31 grantees reported that they match K-12 data to all seven possible workforce programs covered by the DQC survey, which include adult basic and secondary education as well as unemployment insurance wage records.

question: What challenges did state officials site regarding matching data?
answer: State officials cited several challenges to matching data, including state restrictions on the use of a Social Security number. Specifically, officials in three of five grantee states GAO spoke with said state law or agency policy prohibit collecting a Social Security number in K-12 data, which can make it more difficult to directly match individuals' K-12 and workforce records.

questions: question: In what way do grantees use longitudinal data?
answer: According to GAO analysis of the DQC survey data, grantees use some longitudinal data to inform policy decisions and to shape research agendas.

question: What did all grantees report?
answer: All 48 grantees reported analyzing aggregate-level data to help guide school-, district-, and state-level improvement efforts.
child_questions: question: How do grantees use data in improvement efforts?
answer: All 48 grantees reported analyzing aggregate-level data to help guide school-, district-, and state-level improvement efforts. For example, 27 grantees said they analyze data on college and career readiness to help schools determine whether students are on track for success in college or in the workforce.

question: How do grantees use data to support individuals?
answer: Grantees also reported using longitudinal data to analyze outcomes for individual students. For example, 29 grantees reported that they produce early warning reports that identify students who are most likely to be at risk of academic failure or dropping out of school.

question: In what ways do data support research agendas?
answer: Data from the DQC survey also show that 39 grantees reported developing a research agenda in conjunction with their longitudinal data systems.

questions: question: What was GAO asked to review?
answer: GAO was asked to review the status of grantees' longitudinal data systems.

question: What does the report examine?
answer: This report examines (1) the extent to which SLDS and WDQI grantees match individual student and worker records and share data between the education and workforce sectors and (2) how grantees are using longitudinal data to help improve education and workforce outcomes.
child_questions: question: What data did GAO analyze?
answer: To answer these questions, GAO analyzed data from a 2013 survey conducted by the DQC.
child_questions: question: What data was collected via this survey?
answer: This survey collected information from states on data linkages among education and workforce programs and on how states use longitudinal data.

question: How did GAO collect interview data?
answer: In addition, GAO interviewed a nongeneralizable sample of five grantees, which were selected based on the progress they have made in matching data and on the funding they have received from the SLDS and WDQI programs.

question: What other sources did GAO utilize?
answer: GAO also reviewed relevant federal laws and regulations.