# Archived Information

# 

# Title: Using State Data to Achieve Performance Goals

M.P.R. Associates, Incorporated

\[ Slide 1 \]

Using State Data to Achieve Performance Goals

Regional High School Summits

M.P.R. Associates, Incorporated

[www.mprinc.com](http://www.mprinc.com/)

March to May, 2004

# \[ Slide 2 \]

# Albert Einstein said...

## "Not everything that counts can be counted and not everything that can be counted counts."

\[ Slide 3 \]

# Using Data to Improve Schools:

## Making better use of data on key aspects of high school performance and instructional practices 

\[ Slide 4 \]

# Effective Performance Measurement Systems

-   Define valid and reliable measures of student performance

-   Use appropriate analytic methods

-   Promote school improvement

\[ Slide 5 \]

# Essential Elements of No Child Left Behind (N.C.L.B.):

## Adequate Yearly Progress

-   Assess state, district, and school achievement over time

-   Identify schools in need of improvement

> 2\. High School Performance Measures

-   Student achievement

-   Graduation rates

\[ Slide 6 \]

Essential Elements of No Child Left Behind (N.C.L.B.):

### 

# State Accountability System

-   Single system

-   Statistically valid and reliable

1.  Populations

-   All students

-   Sub-groups of students

### 

### \[ Slide 7 \] Defining Terminology

\[ Slide 8 \]

# Making Sense of Data. See also Slides 13, 14, and 15.

## 

## In the year two thousand (2000), the high school graduation rate was eighty six point five percent (86.5%).

## 

-   How defined?

-   Based on what data?

-   How measured?

-   What does this mean for school improvement?

##  \[ Slide 9 \]

# Measurement Criteria

## 

1.  Statistically valid systems

    -   Single measurement approach

    -   Accurate over time

    -   Reliable data

2.  Measure Construction

-   Fair

-   Useful

###  

# \[ Slide 10 \]

# 

# Some Critical Performance Measures:

## Student achievement

## High school graduation rates

## Postsecondary transition rate

## Completion of a rigorous curriculum or program of study

## Attendance and truancy

# 

# \[ Slide 11 \] Analytic Considerations

# 

# \[ Slide 12 \]

# 

# Unpacking the Data

## Aggregate statistics can conceal useful information.

##  Making the most of your data requires:

## 

### Identifying issues of concern 

### Disaggregating variables 

### Controlling for factors affecting outcomes 

#  

# \[ Slide 13 \]

# 

# Example: No Child Left Behind (N.C.L.B.) Graduation Rate. Source is not specified. See also Slides 8, 14, and 15.

Table 1. This is a table contrasting one school district's high school
graduation percentage rates, presumably among the school district's high
school seniors, separately computed by the state and the school district
(columns, left to right), when (rows, top to bottom) the school
district's high school seniors of all ethnic groups were considered
together and when the school district's high school seniors from the
following ethnic groups: "white", "black", and "Hispanic", were
considered separately.

The state reported an eighty-six percent (86%) total high school
graduation rate among all the school district's high school seniors and
the school district reported an eighty-four percent (84%) total high
school graduation rate for all of its high school seniors. But white
high school graduation rates in the school district had the opposite
pattern: the state recorded that eighty-nine percent (89%) of the school
district's white high school seniors graduated, but the school district
recorded that more, ninety percent (90%), of its white high school
seniors graduated. Granted, those discrepancies were relatively small.

Considerably worse, state data minimized the high school non-completion
rates, that is, inflated the high school graduation rate, for the school
district's African American and Hispanic high-school seniors, compared
to the school district's self-reported high school graduation rates for
its African American and Hispanic high school seniors.

The state recorded that eighty-four percent (84%) of the school
district's African American high school seniors graduated, but the
school district recorded that only seventy-eight percent (78%) of its
African American high school seniors graduated. This is a state
discrepancy of six percentage points compared with the school district!

The discrepancy was even larger for Hispanic high school seniors in the
school district. The state reported that seventy-eight percent (78%) of
the school district's Hispanic students graduated, whereas the school
district reported that only sixty-eight percent (68%) of its Hispanic
high school seniors graduated. This is a state discrepancy of ten
percentage points compared with the school district!

Remember slide 8. How were the data and definitions specified? How was
the data measured and where did it come from? Most importantly, what
does this mean for school improvement: how bad is the scope of the
problem?

End of table 1 and slide 13.

# \[ Slide 14 \]

# 

# Example: No Child Left Behind (N.C.L.B.) Graduation Rate (continued). Source not specified. See also Slides 8, 13, and 15.

Table 2. This is a table contrasting one school district's high school
graduation percentage rates, presumably among the school district's high
school seniors, separately computed by the state and the school
district, and the high school graduation percentage rates of urban
school districts (columns, left to right), presumably of their high
school seniors, when (rows, top to bottom) the school districts' high
school seniors of all ethnic groups were considered together and when
the school districts' high school seniors from the "white", "black", and
"Hispanic" ethnic groups were considered separately.

Again, the state reported an eighty-six percent (86%) total high school
graduation rate among all the school district's high school seniors.
Again, the school district reported an eighty-four percent (84%) total
high school graduation rate for all of its high school seniors. Urban
school districts reported an eighty-two percent (82%) total high school
graduation rate of all of their high school seniors. But white high
school graduation rates in the school district had an inconsistent
pattern. Again, the state recorded that eighty-nine percent (89%) of the
school district's white high school seniors graduated, and again the
school district recorded that ninety percent (90%) of its white high
school seniors graduated, but the urban school districts recorded that
eighty-six percent (86%) of their white high school seniors graduated.
Again, the discrepancies were relatively small.

Considerably worse, state data minimized the high school non-completion
rates, that is, inflated the high school graduation rate, for the school
district's African American and Hispanic high-school seniors, compared
to the school district's self-reported high school graduation rates for
its African American and Hispanic high school seniors and especially
compared to the urban school districts' self-reported high school
graduation rates for their African American and Hispanic high school
seniors.

Again, the state recorded that eighty-four percent (84%) of the school
district's African American high school seniors graduated, but both the
school district and the urban school districts recorded that only
seventy-eight percent (78%) of their African American high school
seniors graduated. This is a six percentage point discrepancy for the
state in reference to the data from either the school district or the
urban school districts.

The discrepancy was even larger for Hispanic high school seniors in the
school district and especially for the Hispanic high school seniors in
the urban school districts. The state reported that seventy-eight
percent (78%) of the school district's Hispanic students graduated, and
the school district reported that only sixty-eight percent (68%) of its
Hispanic high school seniors graduated, but the urban school districts
reported that only sixty-two percent (62%) of their Hispanic high school
seniors graduated. This is a state discrepancy of ten percentage points
compared with the school district and a state discrepancy of sixteen
percentage points compared with the urban school districts!

End of table 2 and slide 14.

# \[ Slide 15 \]

# 

# Example: No Child Left Behind (N.C.L.B.) Graduation Rate (continued). Source is not specified. See also Slides 8, 13, and 14.

Table 3. This is a table contrasting one school district's high school
graduation percentage rates and the high school graduation percentage
rates from High School A and High School B (columns, left to right),
when (rows, top to bottom) the school district's high school seniors and
the high schools' seniors of all ethnic groups were considered together
and when the school district's high school seniors and the high schools'
seniors from the following ethnic groups: "white", "black", and
"Hispanic", were considered separately.

Again, the school district reported an eighty-four percent (84%) total
high school graduation rate for all of its high school seniors. High
School A reported a ninety-two percent (92%) total high school
graduation rate among its seniors and High School B reported a
seventy-eight percent (78%) total high school graduation rate among its
seniors

But white, African American, and Hispanic high school graduation rates
in the school district and white, African American, and Hispanic high
school graduation rates in the two high schools had inconsistent
patterns with the all-ethnicities sample and with each other.

Again, the school district recorded that ninety percent (90%) of its
white high school seniors graduated, but High School A recorded that
ninety-three percent (93%) of its white seniors graduated and High
School B recorded that only eighty percent (80%) of its white seniors
graduated.

Again, the school district recorded that seventy-eight percent (78%) of
its African American high school seniors graduated, but High School A
recorded that ninety-four percent (94%) of its African American seniors
graduated and High School B recorded that only seventy-four percent
(74%) of its African American seniors graduated.

Again, the school district recorded that sixty-eight percent (68%) of
its Hispanic high school seniors graduated, and High School B also
recorded that sixty-eight percent (68%) of its Hispanic high school
seniors graduated, but High School A had no applicable data on
graduation rates for its Hispanic seniors.

End of table 3 and slide 15.

\[ Slide 16 \]

A General Model for Examining Student Outcomes

Figure: Positive school inputs improve both school practices and student
outcomes, and good school practices improve student outcomes. End of
Figure.

Good evaluation focuses on more than student outcomes: data on school
practices and school inputs are also essential.

\[ Slide 17 \] Essential Control Variables

Examples

1.  School Practices

    -   Curricular offerings and rigor

    -   Class scheduling

    -   Professional development

2.  School Inputs

    -   Student demographics

    -   Size and urbanity

    -   Teacher experience

# \[ Slide 18 \]

# 

# Additional Data Sources can provide a broader picture of student performance.

Examples:

## Student Transcripts

###  Attendance

###  Subject area grades

###  Curricular rigor

###  Mobility & retention

## 

# \[ Slide 19 \] Using Results to Promote Improvement 

# \[ Slide 20 \]

# 

# Improving Usability

## User-tested reporting

## Dedicated time for reflection and strategic planning

## Building analytic capacity among teachers and administrators

## Periodic feedback for overall design of state and local data systems

\[ Slide 21 \] Conclusion

# 

# \[ Slide 22 \]

# Strengthening Data-Driven School Improvement

## Priorities for clear and more consistent definitions of key variables

## Unique student identifiers that protect privacy but enable tracking over time

## Interagency cooperation to promote merging of data files

\[ Slide 23 \]

# Effective Performance Measurement Systems

## Track: Produce valid and reliable data on school and student outcomes.

## Analyze: Provide policy relevant information across a range of variables.

## Improve: Assist policymakers and educators in making better decisions.