# Presentation: 103851

## Research Using StateLongitudinal Data Systems: Accomplishments and Challenges – The Case of Florida

**Tim R. Sass**

## Overview

- The Status of Statewide Longitudinal data Systems
- Projects Underway Using Statewide Data from Florida
- Challenges to Using Statewide Longitudinal Data for Research
- What Lies Ahead

## The Status of State Longitudinal Data Systems

- Data Quality Campaign’s “10 Essential Elements”
- A unique statewide student identifier that connects student data across key databases across years
- Student-level enrollment, demographic and program participation information
- The ability to match individual students’ test records from year to year to measure academic growth
- Information on untested students and the reasons they were not tested
- A teacher identifier system with the ability to match teachers to students
- Student-level transcript information, including information on courses completed and grades earned
- Student-level college readiness test scores
- Student-level graduation and dropout data
- The ability to match student records between the P–12 and higher education systems
- A state data audit system assessing data quality, validity and reliability

## The Status of State Longitudinal Data Systems

- States That Can Track Student Gains and Match Teachers to Students
- Arkansas, Delaware, Florida, Georgia, Hawaii, Kentucky, Louisiana, Mississippi, New Mexico, Ohio, South Carolina, Tennessee, Utah, West Virginia, Wyoming
- North Carolina researchers do this indirectly through the identity of the teacher administering exams in elementary school
- Some states have only recently acquired the ability to match students and teachers
- Kentucky (2006-07), Wyoming (2006-07)

## The Status of State Longitudinal Data Systems

## Research Projects Using Statewide Longitudinal Data From Florida

- School Choice
- “Charter Schools and Student Achievement in Florida” (Sass (2006))
- Mature charters on par with traditional public schools in math and yield higher reading scores than the average traditional public school
- Charter school competition associated with small improvement in math achievement in traditional public schools
- “Charter Schools and Educational Attainment” (Booker and Sass)
- Will investigate the impact of charter high school attendance on high school graduation and college attendance in Florida and in Chicago
- “Vouchers and Special Education” (Sass)
- Will analyze the decision to accept a voucher and the effects of vouchers on the supply of private schools and the achievement of special education students who remain in public schools

## Research Projects Using Statewide Longitudinal Data From Florida

- Peer Effects
- “Classroom Peer Effects” (Burke and Sass)
- Preliminary evidence suggests that peer influences are significant and the effects are non-linear
- Low-achieving (lowest quintile) students worse or no better off in classes with more high-achieving (top quintile) peers
- Mid-level students generally worse off with more low-achieving peers and better off with high-achieving peers
- High-achieving students better off with more high-achieving peers and worse off with a greater fraction of low-achieving peers

## Research Projects Using Statewide Longitudinal Data From Florida

- Teacher Training
- “Teacher Training, Teacher Quality and Student Achievement” (Harris and Sass (2007))
- Experience matters, especially over the first five years
- Pedagogical content knowledge matters, both pre-service and in-service
- Other forms of preparation do not seem to impact teacher effectiveness
- No significant correlation between a teacher’s college entrance exam scores and later effectiveness as a teacher
- Teacher Preparation Programs
- In conjunction with researchers from New York, Louisiana and Ohio, will be analyzing the relative effectiveness of teachers who obtain certification through various routes

## Research Projects Using Statewide Longitudinal Data From Florida

- National Board Certification
- “The Effects of NBPTS-Certified Teachers on Student Achievement” (Harris and Sass (2007))
- NBPTS certification provides a positive signal of teacher effectiveness in some cases, though the ability of NBPTS certification to identify high-quality teachers varies considerably across grades and subjects
- Results are sensitive to the exam used to measure student achievement
- No evidence that the NBPTS process itself enhances teacher quality
- Little of no evidence of positive spillovers from NBPTS certified teachers to non-NBPTS-certified teachers

## Research Projects Using Statewide Longitudinal Data From Florida

- Teacher Labor Markets
- “A Longitudinal Analysis of Teacher Mobility:  Intra-District Moves, Inter-District Moves and Departure Decisions” (Feng (2007))
- Exit decisions influenced by teacher salaries relative to other occupations and the proportion of minority students in their school
- Inter-district moves determined by relative salaries across districts and minority enrollment
- Intra-district movement toward schools with higher student achievement and fewer minority students
- Minority teachers’ job decisions less effected by minority student enrollment than those of their white colleagues

## Research Projects Using Statewide Longitudinal Data From Florida

- Accountability
- “School Accountability and Teacher Job Choice” (Feng, Figlio and Sass)
- Will investigate the impact of school accountability pressures on teachers decisions to move between schools or exit teaching entirely
- Will exploit the fact that Florida changed the mechanism for grading schools which lead some schools to face increased accountability pressure and lessened the pressure faced by other schools

## Challenges to the Use of Statewide Longitudinal Data

- Achievement Tests and the Measurement of Student Achievement Gains
- Relatively little known about the effects of alternative assessment measures on measured impacts
- Test score scaling
- Validity of using achievement tests to measure intertemporal gains in achievement
- Criterion-reference exams versus developmental scale scores
- Norming scores by grade and year
- Choice of test instrument
- Some evidence that results differ across tests (Harris and Sass (2007)

## Challenges to the Use of Statewide Longitudinal Data

- Methodology
- How well do “quasi-experimental” methods control for selection biases?
- Recent evidence on value-added models versus random assignment by Cantrell, Fullerton, Kane and Staiger (2007)
- Inconsistent Estimation Methods and Specification in “Value-Added” Achievement Models
- Student and teacher effects versus student and teacher covariates
- Random versus fixed effects models
- Gain scores versus partial persistence models
- Inclusion/exclusion of Peer Effects

## Challenges to the Use of Statewide Longitudinal Data

- Use of Teacher Effects to Measure Teacher Quality
- Not much known about inter-temporal consistency of individual teacher effects
- Koedel and Betts (2006)
- Little evidence on sensitivity of teacher effect estimates to changes in model specification
- Use in policy research versus use for accountability and merit pay

## The Future of Research Using Statewide Longitudinal Data

- Cross-State Analyses
- Due to lack of data, most statewide research so far has been limited to Florida, North Carolina and Texas
- Many studies using large urban school districts, such as New York, Chicago and Los Angeles
- Data availability in other states will help determine if previous findings are robust across geographic areas
- Variability in effectiveness of NBPTS-certified teacher in Florida and in North Carolina found by Ladd, Sass and Harris (2007)
- May be able to exploit cross-state variation in policy environments
- RAND charter school project

## The Future of Research Using Statewide Longitudinal Data

- Linking Value-Added Measures to Qualitative Evidence
- Comparison of Teacher Observations to Value-Added Measures of Teacher Quality
- Rockoff, et al.
- Comparison of Principal Evaluations of Teachers to Value-Added Measures
- Harris and Sass (2007)
- Matching Survey Evidence to Administrative Data