Measuring School Performance To Improve Student Achievement and To Reward Effective Programs.

This paper presents the Minneapolis Public School's (MPS) method of measuring school and student performance. MPS employs a multi-faceted system that both captures and accounts for the complexity inherent in a large urban school district. This system incorporates: (a) a hybrid model of critical indicators that report on level of performance, cross-cohort changes, growth or gain scores, and value-added measures; (b) specific standards, set in a collaborative manner by key stakeholders, and standards were then aligned with MPS criteria and real-world consequences; and (c) a cycle of continuous School Improvement Planning. The MPS hybrid model includes a set of pre-agreed upon, pre-selected indicators that were established in concert with stakeholders who are internal and external to the Minneapolis Public Schools. Moreover, this paper provides a practical application of how value-added indicators can be and should be included in any fair accountability model designed to evaluate the effectiveness of schools. The expanded growth model, referred to as value-added, addresses the differential balance of student characteristics across MPS schools such as: poverty, race, family composition, special education status, Limited English status, and neighborhood poverty concentration. The value-added approach described here is the backbone of setting up a flexible, yet equitable measurement system that can account for important student characteristics (associated with student achievement) and assist in determining the extent to which teachers and schools "add value" to student performance. Evaluating the overall performance of schools based upon a set of indicators allows the Minneapolis Public School's system to evaluate the district's effectiveness in meeting the needs of all its students. Moreover, the overall framework allows the district to place schools along a continuum that relies on both rewards and sanctions. In fact, by using the described framework schools can "earn financial rewards" called Quality Performance Awards to maintain and expand upon their excellent performance. Additionally, in extreme cases schools can be identified that "need prescriptive, corrective action," which may include the option of school reconstitution through the district's "fresh start" provisions of the teacher and principal contracts. Finally, a complete School Improvement Planning cycle is discussed. Continuous school improvement is based on detailed information reports provided to all school sites titled, "School Information Reports. " These documents set the stage for important site-based analysis and planning to occur. Samples of these yearly documents are available via the world-wide-web at: http://www.mpls. k12.mn. us/REA (click on School Information Reports). Hopefully, this paper will be useful to practitioners, administrators or researchers interested in identifying and combining school performance indicators to make high stakes accountability decisions for school districts. AERA Performance continuum-2000 1 04/18/00 Measuring school performance to improve student achievement In order for a school incentive program to be effective, that is, to have motivational impact on schools which results in improved student learning, it must be perceived as fair and worthwhile, and it's criteria must be understandable and readily communicated. (John May, in Manedeville, 1994). Evaluating School Performance. No matter where you turn, you cannot escape the pervasive discussions, debates, and hyperbole that surround the topic of evaluating student outcomes. Further, attempting to understand the role schools and districts play in accelerating or decelerating student performance can be daunting. Although evaluating school and district performance is a given in today's educational environment, there are still varying perspectives on how to best meet the challenge (Millman & Schalock, 1997). In fact, an elaborate array of state-wide and district-wide models have been proposed. These evaluation models are often put in place under the guise of addressing such things as: (a) Accountability; (b) Communicating performance to key stakeholder including the community at large; (c) Effecting change for poor performing sites or creating the impetus for self-study and self-improvement among all schools; and increasingly to (d) Identify innovative and powerful programs so that they may be replicated and shared with sites that are performing well below expectations, as well as to reward high-performing schools. It should come as no surprise that a multitude of systems have been developed to evaluate school effectiveness. Many state and local models rely on reporting level indicators of student test performance (e.g., SAT or ACT scores, or some other mean score). However, as it will be discussed later, these are imperfect methods for determining the inherent value or effectiveness of instructional programming within schools. Recent developments have pushed the frontier toward more sophisticated statistical analyses to help get the answer "right" when trying to attribute school effects v. non-school effects on student performance (Meyer, 1996). The need for more sophisticated analyses has arisen out of the need to account for the complex set of interactions that occur between students who enroll in programs and school and district ability to meet the needs of those students. To this end, more systems are relying on educational outcome indicators to assess the efficacy of educational programming (Meyer, 1996). Thus, student outcomes or student performance on clearly identified indicators have become the primary method for evaluating and holding schools and school systems accountable. Criteria for performance indicators and accountability systems. Meyer (1996) identified three critical criteria for evaluating the usefulness of performance indicators. First, school performance indicators that are included in any accountability system must assess the types of skills demanded by society. In the state of Minnesota there are some clear benchmarks all students must meet. In 1996 the state of Minnesota implemented a series of graduation requirements that 4 have been implemented incrementally within the state. For example, beginning with the graduating class of 2000, students must pass Basic Skills Tests (MBST) in the areas of reading and math. For the class of 2001 and beyond, students must pass MBSTs in the areas of reading, math and writing. Students must reach a passing score in order to be eligible to receive a high school diploma. In addition, the state has put in place a series of high standard tests referred to as the Minnesota Comprehensive Assessment (MCA) at grades 3, 5, 10, and I I I. These tests cover high-end content and knowledge in the areas of reading, math, and writing. This two-tiered system provides clearly defined outcome measures that have allowed the MPS to align curricula and district assessments to plot student performance and trajectories toward basic and high standard criteria. The second characteristic of a quality school performance indicator is its ability to accurately measure performance with respect to the outcome it purports to measure (Meyer, 1996). That is, the indicators that are selected must meet a standard in which they cannot be "corrupted. " Thus, the instruments must be valid and reliable for the purpose for which they are selected. The administration of such instruments must occur following rigorous standardized procedures. Meyer (1996) describes additional concerns about the ability to corrupt an accountability system. For example, when level performance indicators are used alone, there may be a disincentive for districts to include students at the lower margins (e.g., special education and ESL students) to avoid "lowering" the overall mean test scores. Obviously this behavior is unethical and creates a system in which some students become "invisible" within the district, resulting in no accountability for their performance. In effect, districts could become disingenuous in setting policies and begin to test only "higher performing" students so as to corrupt student performance reports. Additional policies, such as retention of students at specific grade levels can further distort level performance, as students remain at the same grade-level for a second year and are compared to norm groups based on grade. Collateral damage also can occur as teachers then begin to bid out of "low levei" and into "high level" schools. Finally, very few if any of these level models account for the high correlation of post-test performance based on pre-test performance. Indeed, few systems collect student achievement on a yearly basis to compare actual improvement across time. Finally, over and above the "corruptible" concern, performance indicators selected must measure the unique contribution schools "add" for each measured outcome. Although many systems rely on average test score performance to rank order or evaluate school performance, mean scores are highly flawed indices of school effectiveness (Meyer, 1994). Despite the flawed nature of equating school performance with point-in-time mean performance, many districts, newspapers, real estate brokers, and families resort to evaluating or making judgements about school effectiveness based on level data. The state of Minnesota currently has high standard MCA tests at 3, 5, and 10 and will be adding two additional MCA tests in reading (grade 10) and math (grade 11) during the 2000-2001 school year. 3 For example, take two schools -School A and School B that currently report overall student performance at the 80th percentile (a high performing school). Despite its seductive, simplistic approach, level indicators do not provide adequate information on how a student, school, or district reached such a level of performance. In this example there is no way to account for how these students or schools