# Presentation: 678446

## EVALUATION & RESEARCH IN WAKE COUNTY (NC) PUBLIC SCHOOLS

- Presented by
- David Holdzkom,
- Assistant Superintendent,
- Evaulation & Research
- Wake County Public Schools

## CONTEXT

- WCPSS serves about 134,000 students, making it the 19th largest district in the U.S. and the largest district in North Carolina
- The district has 93 elementary schools, 28 middle schools, 22 high schools and 4 special schools.
- About 30% of students qualify for FR; 14% are SWD; and 7.4% are LEP.

## CONTEXT (cont’d)

- Wake County includes 12 incorporated towns plus suburbs and rural communities.
- Since 1990, the county population has increased by 92%.
- Much of this in-migration has been from areas of the country that have different traditions of school district organization.
- Major employers in the county include IBM, GlaxoSmithKline, SAS, and Cisco
- 3 research universities within 25 miles

## ORGANIZATION OF EVAL&RES DEPARTMENT

- Admin:  Assistant Superintendent and 3.5 support staff;
- Testing:  Sr. Director and 2 professionals plus 4 support staff;
- School Accountability:  Sr. Director and 5 professionals plus 2 support staff;
- Program Accountability:  Sr. Director and 4 professionals plus 3 support staff;
- Grants Admin:  Sr. Director plus 2 support staff.

## STUDENT INFORMATION RESOURCES

- NC WISE:  A comprehensive data base designed by state to provide information (demographic, achievement data) for every student.
- Student Information Files:  Maintained by WCPSS on a mainframe to provide supplementary data for every student.
- Testing files:  Designed by the state to provide both achievement and growth data for all students in Grades 3-8 plus selected high school course takers.
- Local test files:  Designed by WCPSS adds achievement data for students in K-2 as well as some additional information for students in Grades 3-5.

## THE EFFECTIVENESS INDEX

- Designed in WCPSS E&R, this analysis tool results from regression analyses for individual students that control for FR, concentration of FR in school, special education status, special education service method, etc.
- Provided for each student, with reports aggregated at the class/course and school levels.

## KINDS OF RESEARCH

- Policy Evaluation:  Healthy Schools, Magnet Schools, Student Assignment
- Program Evaluation:  Implementation of PBS, AVID, Standards-based grading, IRIS, etc.
- School Status Evaluation:  Accountability measures of state and NCLB.
- Knowledge Creation:  Effectiveness Studies; IRIS

## Program Evaluation Standards

- Utility Standards:  Study serves the info needs of the audience.
- Feasibility Standards:  Study is realistic, diplomatic, and economical.
- Propriety Standards:  Study is legal and ethical.
- Accuracy Standards:  Study reveals and conveys technically adequate info.

## POLICY EVALUATION

- Healthy Schools:  Values for 100 variables (both input and outcome) are calculated annually.  This information is used by the Leadership Team to ensure that schools remain “healthy”.
- Magnet Schools:  Measure the degree to which Board goals for these schools are met.
- Student Assignment:  How well do different schools serve sub-groups of students?

## Magnet Program Review (2007)

- Review examined degree that board’s goals for magnets were realized:
-     1. Utilization of schools;
-     2. Use of choice to promote diverse student populations, reduce concentration of poverty;
-     3. Expand educational opportunity;
-     4. Promote program innovations.

## Magnet Review (cont’d)

- Examined student assignment data and student demographic for indications of race, FRL-status, assignment to schools.
- Compared all schools with respect to concentration of FRL
- Conducted focus group interview/survey of principals to identify innovative programs.
- Examined differences of achievement between magnet/base populations, compared to district.

## PROGRAM EVALUATION

- Schools seek to implement nationally-developed programs (PBS, AVID, Reading Recovery) to meet specific needs. Evaluation focuses on fidelity of implementation as well as change in student outcomes.
- Locally developed programs (Standards-based grading, PES, etc.) try to focus more on logic modeling and outcomes.

## Evaluation of SES

- A Title I school in year 2 of school improvement was required to offer Supplemental Education Services.
- 252 students were provided tutoring service at a cost of $303,000.
- Interviews were conducted with teachers and parents; student achievement data were examined.

## SCHOOL STATUS EVALUATION

- Accountability measures of state and NCLB.
- Calculation of absolute measures of achievement (i.e., percentage of students at/above grade level) and calculation of growth.
- Calculation of student residuals/effectiveness index.

## Knowledge Creation

- Effectiveness Studies:  Using Effectiveness index to identify higher/lower performing teachers of 5 high school subjects, studied classroom practice; this method was adapted for elementary & middle schools
- IRIS:  An experimental program designed to provide reading intervention early.
- Professional Learning Communities:  Surveys intended to measure the quality/degree of implementation of PLCs in all schools.

## Project IRIS

- As part of an OCR consent decree, WCPSS agreed to analyze process of referrals to special education services and to design data-driven interventions to improve reading instruction.
- Intensive Reading Intervention Study (IRIS) was developed to respond
- The logic:  too many poor readers are referred to student support teams and ultimately to special education.  If we can improve reading early, referrals will decrease.

## Project IRIS (cont’d)

- IRIS involved random assignment of 169 students to Control or Treatment groups.
- Treatment lasted for 45 minutes 4 times a week for ten weeks.
- Students were tested on a variety of measures before treatment, after treatment and end of year.  Controls had “mirror” assessments.

## Project IRIS (cont’d)

- At the end of the study, treatment group had more positive outcomes, but these did not reach statistical significance.
- Three years later, students from both groups were identified and achievement data analyzed.  Again, no significant differences were found.

## RANDOMIZED TRIALS

- Only 1 study (IRIS) has used randomization.
- Teachers, principals, parents object to randomization:  “students aren’t guinea pigs”.
- How does intervention support/detract from the core mission of the school?
- Currently pursuing a randomized trial project with Duke University related to PLCs.

## FUTURE DIRECTIONS/ISSUES

- Designing training in data use for principals & teachers and experimenting with delivery methods/formats
- Increased emphasis on logic models and fidelity of implementation
- Assisting schools to design accountability schemes that include but are not limited to test scores

## FUTURE DIRECTIONS (CONT’D)

- Enhancing teachers’ ability to develop formative assessment plans/items
- Implementing the evaluation recommendations of the Curriculum Management Audit conducted in 2007